IPWorld: MPLS Layer-3 VPN Route Reflector

Layer-3 VPN Scaling with Route Reflector

To share the client routes, without any route reflector PE router need to directly establish the MP-BGP peer session with other PE router. Now suppose if a customer has six sites that are connected with 6 PE routers the total number of require MP-BGP peer sessions can be calculated with the formula “n * (n – 1) / 2” and that are total 15 peering sessions.

Above diagram shows the 6 PE routers with full mesh connectivity, here every individual PE router have 5 MP-BGP peer sessions and total number of MP-BGP peer sessions are 15. This is a serious scalability issue as the number of PE routers increase the number of MP-BGP peer sessions in every PE router will increase whereas total number of peer sessions will dramatically be increased as every PE router need to establish a new peer session with every new PE router.

However, with use of Route Reflectors, we can reduce the number of MP-BGP sessions between the PE routers. All PE routers establish the MP-BGP peer session only with the Route Reflector, there is no need to establish the direct PE-to-PE MP-BGP peer session. This will dramatically reduce the number of peer sessions on individual PE as well as total number of peer sessions between all PE routers. Infact every PE router just have only “ONE” peer session that is with the route reflector. As shown in the following diagram, again we have six PE routers and one Route Reflector but this time we need only one peer session from every PE router to route reflector thus total number of peer sessions are only six.

Later if another PE is added in topology the only one new MP-BGP session needs to establish that is between that new PE router and route reflector. The existing PE routers remain unchanged with their existing single MP-BGP peer session with the route reflector.

You can configure the PE router as route reflector but according to standard best practices it is recommended to use P router as the route reflector. The PE routers are responsible to handle all the Layer-3 VPN related complexities as well as manage customer related VRF tables whereas P routers provide the MPLS based forwarding between PE routers. P routers do not have customer routes in fact they only have the provider internal routes

The Route Reflector functions at control plane, means it receive the route information from one PE router and distribute that information to other PE routers. However, it did not control the forwarding. Forwarding is still happening directly between PE to PE via MPLS label switching.

All active VPN routes are stored in route reflector route table bgp.l3vpn.0. It didn’t need to configure/install any VRF table as it has nothing to do with forwarding. The VRF tables are used only on PE routers where customer-based route filtering also performed.

In this diagram we can see that all PE routers will forward the customer routes to the route reflector. As we already know, route reflector stores all these routes un its bgp.l3vpn.0 routing table. Route reflector will forward all these routes to other PE routers. The receiving PE router will use its VRF import policies to match and keep routes relating to its locally attached sites.

PE3 will receive all the VPN routes from route reflector. PE3 will then filter the routes and install only the routes related to its attached customer. Here PE3 have the VRF tables, customer Green and Customer Blue so it will only install the customer Green and customer Blue routes and discards the customer RED routes. This all done by customer route targets configured on PE3 router.

We can alter this default behavior by using the “keep all” on receiving PE router. In that case PE router will install all the VPN routes in its bgp.l3vpn.0 table no matter route target is matched or not. Still, in customer specific VRF only customer related route will install depending on customer vrf import policy.

One more important point that we must know, when a PE router send the VPN route information to route reflector the route reflector must have bgp protocol next hop reachable via its inet.3 routing table. Note that VPN routes always use inet.3 table to resolve their protocol next-hop information. Suppose PE1 send a customer route to route reflector, with protocol next hop is PE1 loop-back address. The route reflector must have reachability information of PE1 loop-back address in its inet.3 route table. If route reflector can’t find reachability information in its inet.3 table the protocol next-hop of the route can’t reachable thus the route will be hidden and can’t be advertised to other PE routers.

The solution is to configured a MPLS LSP from Route Reflector loop-back address to PE router loop-back address. With this BGP next-hop of VPN routes can be resolved to an LSP. As route reflector is not part of forwarding there is no need of LSP from PE router to route reflector.

The yellow arrow lines show the LSP establish between route reflector to PE routers.

Topology Diagram

CONFIGURATIONS

P-RR

Protocol Configuration

root@P-RR1# show protocols

rsvp {

interface all;

}

mpls {

no-cspf;

label-switched-path RR-to-PE1 {

from 192.168.50.50;

to 192.168.100.1;

}

label-switched-path RR-to-PE2 {

from 192.168.50.50;

to 192.168.100.2;

}

label-switched-path RR-to-PE3 {

from 192.168.50.50;

to 192.168.100.3;

}

interface all;

}

bgp {

group RR-Group {

type internal;

local-address 192.168.50.50;

family inet {

unicast;

}

family inet-vpn {

unicast;

}

cluster 192.168.50.50;

neighbor 192.168.100.1;

neighbor 192.168.100.2;

neighbor 192.168.100.3;

}

ospf {

area 0.0.0.0 {

interface lo0.0 {

passive;

}

interface ge-0/0/0.0;

interface ge-0/0/1.0;

interface ge-0/0/2.0;

interface ge-0/0/3.0;

}

PE-1

Protocol Configuration

rsvp {

interface ge-0/0/8.0;

interface ge-0/0/9.0;

}

mpls {

no-cspf;

label-switched-path pe1-to-pe2 {

from 192.168.100.1;

to 192.168.100.2;

}

label-switched-path pe1-to-pe3 {

from 192.168.100.1;

to 192.168.100.3;

}

interface ge-0/0/8.0;

interface ge-0/0/9.0;

}

bgp {

group IBGP-PE1 {

type internal;

local-address 192.168.100.1;

family inet {

unicast;

}

family inet-vpn {

unicast;

}

neighbor 192.168.50.50;

}

ospf {

area 0.0.0.0 {

interface ge-0/0/9.0;

interface ge-0/0/8.0;

interface lo0.0 {

passive;

}

VRF Policy Configuration

set policy-options policy-statement export-blue term 10 from protocol bgp

set policy-options policy-statement export-blue term 10 from protocol direct

set policy-options policy-statement export-blue term 10 then community add vpn-cust-blue

set policy-options policy-statement export-blue term 10 then accept

set policy-options policy-statement export-blue term 20 then reject

set policy-options policy-statement import-blue term 10 from protocol bgp

set policy-options policy-statement import-blue term 10 from community vpn-cust-blue

set policy-options policy-statement import-blue term 10 then accept

set policy-options policy-statement import-blue term 20 then reject

set policy-options policy-statement export-red term 10 from protocol bgp

set policy-options policy-statement export-red term 10 from protocol direct

set policy-options policy-statement export-red term 10 then community add vpn-cust-red

set policy-options policy-statement export-red term 10 then accept

set policy-options policy-statement export-red term 20 then reject

set policy-options policy-statement import-red term 10 from protocol bgp

set policy-options policy-statement import-red term 10 from community vpn-cust-red

set policy-options policy-statement import-red term 10 then accept

set policy-options policy-statement import-red term 20 then reject

set policy-options community vpn-cust-red members target:65512:11

set policy-options community vpn-cust-blue members target:65512:21

set policy-options community vpn-cust-green members target:65512:31

VRF BLUE

set routing-instances VPN-BLUE instance-type vrf

set routing-instances VPN-BLUE interface ge-0/0/6.0

set routing-instances VPN-BLUE route-distinguisher 192.168.100.1:2

set routing-instances VPN-BLUE vrf-import import-blue

set routing-instances VPN-BLUE vrf-export export-blue

set routing-instances VPN-BLUE vrf-table-label

set routing-instances VPN-BLUE protocols bgp group EBGP-BLUE-CE1 type external

set routing-instances VPN-BLUE protocols bgp group EBGP-BLUE-CE1 peer-as 65002

set routing-instances VPN-BLUE protocols bgp group EBGP-BLUE-CE1 as-override

set routing-instances VPN-BLUE protocols bgp group EBGP-BLUE-CE1 neighbor 30.30.30.2

VRF RED

set routing-instances VPN-RED instance-type vrf

set routing-instances VPN-RED interface ge-0/0/7.0

set routing-instances VPN-RED route-distinguisher 192.168.100.1:1

set routing-instances VPN-RED vrf-import import-red

set routing-instances VPN-RED vrf-export export-red

set routing-instances VPN-RED vrf-table-label

set routing-instances VPN-RED protocols bgp group EBGP-RED-CE1 type external

set routing-instances VPN-RED protocols bgp group EBGP-RED-CE1 peer-as 65001

set routing-instances VPN-RED protocols bgp group EBGP-RED-CE1 as-override

set routing-instances VPN-RED protocols bgp group EBGP-RED-CE1 neighbor 10.10.10.2

PE-2

root@PE-2# show protocols

rsvp {

interface ge-0/0/8.0;

interface ge-0/0/9.0;

}

mpls {

no-cspf;

label-switched-path pe2-to-pe1 {

from 192.168.100.2;

to 192.168.100.1;

}

label-switched-path pe2-to-pe3 {

from 192.168.100.2;

to 192.168.100.3;

}

interface ge-0/0/8.0;

interface ge-0/0/9.0;

}

bgp {

group IBGP-PE2 {

type internal;

local-address 192.168.100.2;

family inet {

unicast;

}

family inet-vpn {

unicast;

}

neighbor 192.168.50.50;

}

ospf {

area 0.0.0.0 {

interface ge-0/0/9.0;

interface ge-0/0/8.0;

interface lo0.0 {

passive;

}

VRF Policy Configuration

set policy-options policy-statement export-green term 10 from protocol bgp

set policy-options policy-statement export-green term 10 from protocol direct

set policy-options policy-statement export-green term 10 then community add vpn-cust-green

set policy-options policy-statement export-green term 10 then accept

set policy-options policy-statement export-green term 20 then reject

set policy-options policy-statement import-green term 10 from protocol bgp

set policy-options policy-statement import-green term 10 from community vpn-cust-green

set policy-options policy-statement import-green term 10 then accept

set policy-options policy-statement import-green term 20 then reject