MPLS Series - Vol. 2 - MPLS VPN

March 5th, 2008 neteng Posted in MPLS |

Yes, I finally completed this article before the end of our current millennium. I’ve been busy transitioning to a new client at work and it involves some travel, so I just have not been motivated to get this thing finished. But I decided I just needed to sit myself down and complete it. I hope you enjoy the fruits of my labor! As always, keep the feedback coming and let me know if you find any mistakes. It’s rough being your own proofreader.

Topic

MPLS VPN

Definition

Alright folks, it’s time to tackle probably the most popular MPLS application currently in use - VPN. For service providers, MPLS has greatly simplified implementing VPN for their customers. It also makes it easier to set up connectivity between different customers. No more having to worry about building multiple point-to-point GRE tunnels and tricky/endless ACLs to make sure one customer doesn’t see another customer’s networks. Let’s take a brief look at how packets within our MPLS VPN will travel:

I ‘borrowed’ the above diagram from the Cisco Press MPLS Fundamentals book, because it’s perfect for this article. It illustrates how labels are applied to a packet traveling through an MPLS VPN. At the ingress PE router, two labels are pushed onto a packet. First, a VPN label is applied that will determine which egress PE receives the packet. Then, another label is pushed on top of that label that determines which P router will be the next-hop in the normal MPLS labeled switch path (LSP). This top label is changed through each P router it passes through in the LSP and is finally popped by the router that is penultimate to the egress PE. With only the VPN label left, the packet is passed on to the egress PE router, the label is popped off and the packet is then routed via IP to the appropriate CE router. Any P routers that exist in the LSP (labeled switch path) should have no knowledge of the customer routes/VPN labels that are ‘tunneling’ through them between PE devices. This is important to understand because should some misconfiguration occur and a P router receives a labeled packet destined for a customer VPN, it won’t have any idea what do with it, and therefore it will be dropped.

As we look into MPLS-based VPNs, you’ll see we have some new terms to understand and we’ll also need to think about good ol’ BGP a little bit differently.

BGP comes into play as the protocol used to exchange routes within a VPN. When BGP operates in this fashion, it’s most often referred to as MP-BGP (Multiprotocol BGP). It’s the preferred protocol because of it’s extensibility. The routes carried within MP-BGP are known as VPNv4 routes. As you’ll see, IPv4, VPNv4, IPv6, etc. are all referred to by BGP as address families. This is how BGP can distinguish what type of routes it’s seeing. This will make more sense in our example below.

These VPNv4 routes are essentially IPv4 routes, but with a value known as a Route Distinguisher (RD) tacked on to the front. The typical format of an RD is ASN:nn (sometimes IPv4 Address:nn - the IPv4 address being from an non-private assigned range). The RD value is used to designate which Virtual Routing and Forwarding Instance (VRF) that an IPv4 route belongs to (usually, 1 VRF = 1 RD = 1 Customer). The VRF is basically a ‘virtual’ router within the physical router. It maintains a routing table that is completely separate from the main routing table. Once a VPNv4 route is received by a participating PE, the RD is stripped and the original IPv4 route is placed into the VRF routing table.

To clarify this whole RD/VPNv4 concept, let’s assume that you have two customers, each with a network of 192.168.1.0/24. Let’s say that a packet arrives at a router connected to these customers and the packet has a destination of 192.168.1.5. Without the RD, the router would have no way of knowing the correct customer to route the packet to. Knowing we have the RD to work with, let’s say that one customer exists in a VRF with RD 5:1 (Customer A) and the other exists in a VRF with RD 260:1 (Customer B). Once the RD is appended to the actual IPv4 route, you will now have two different routes. Now with the VPNv4 syntax, the route to the 192.168.1.0/24 network at Customer A would be distinguished as 5:1:192.168.1.0/24 and the route to the 192.168.1.0/24 network at Customer B would be distinguished as 260:1:192.168.1.0/24. Pretty simple, huh? The RD is what allows MPLS VPN customer to maintain overlapping networks.

With this in mind, we now come to another important concept in MPLS VPNs: the Route Target (RT). Why do we need another value, you ask, if we can already use the RD to distinguish between VPN customers? Well, the RT comes in handy when you want to create an extranet between VPN customers. The RT is a ‘tag’ value that designates which VPNv4 routes to import and export. It’s carried within BGP as an extended attribute. It’s syntax is very similar to the RD, and often is the same value. In most cases, when there is no communication between different VPN customers, a single RT is both imported and exported into a VRF. But if two customers want to access each other’s networks, they will both need to import the RT exported by the other customer.

Referring back to our previous example, let’s assume that Customer A (which is importing and exporting an RT value of 5:1) and Customer B (importing and exporting an RT value of 260:1) want certain office locations to reach each other’s networks. On the PE’s connected to those locations, we would need to import the 5:1 RT into Customer B’s VRF and import the 260:1 RT into Customer A’s VRF. They would then be able exchange VPNv4 routes with each other.

Finally, we should point out that MP-BGP behaves much like normal BGP, and therefore, if you don’t maintain a full mesh between PE peers, you’ll need to implement route reflection. We won’t get into a discussion about that in this lesson, but may cover it at another time.

Hopefully you now have a decent grasp on the fundamentals. It’s probably a good time to dive into an example and solidify what we’ve learned.

Example

As you can see in our diagram, we’ve connected two different customers to our MPLS backbone, each with two locations. There are a multitude of ways to get the customer routes into the VRF tables. We could use OSPF, static routes, etc. to exchange routes between our PE and CE routers. We’re going to use BGP as it seems to be most common routing protocol between real-life PE-to-CE connections. This will also allow us to avoid redistributing routes into MP-BGP, since each PE-to-CE connection is essentially an eBGP connection.

First things first. Let’s get our VRFs configured on each PE router. Since each customer location maintains it’s own AS number, I’ve just chosen the lowest AS number associated with each customer for the RD:

R3:

ip vrf CustB
description Customer B
rd 1:1
route-target export 1:1
route-target import 1:1

R5:

ip vrf CustA
description Customer A
rd 2:1
route-target export 2:1
route-target import 2:1
!
ip vrf CustB
description Customer B
rd 1:1
route-target export 1:1
route-target import 1:1

R6:

ip vrf CustA
description Customer A
rd 2:1
route-target export 2:1
route-target import 2:1

Now that we have our VRFs configured, we have to apply them to the interfaces connected to our customers. Something to note is that if you already configured the IP addressing on these interfaces, applying the VRF will erase them and you’ll need to reconfigure them:

R3:

interface Serial1/0
ip vrf forwarding CustB
ip address 192.168.13.1 255.255.255.252

R5:

interface Serial1/0
ip vrf forwarding CustA
ip address 192.168.25.1 255.255.255.252

interface Serial1/2
ip vrf forwarding CustB
ip address 192.168.58.1 255.255.255.252

R6:

interface Serial1/2
ip vrf forwarding CustA
ip address 192.168.67.1 255.255.255.252

We can verify that everything has been applied correctly as follows (R3 as an example):

R3#sh ip vrf CustB
Name Default RD Interfaces
CustB 1:1 Se1/0

With our VRFs configured and now active, it’s time to work on our BGP configuration. We’ll being with the basic BGP configuration on the CE equipment. This is just plain, vanilla BGP. To simplify getting our customer networks into BGP, we’re just going to redistribute all connected routes (essentially the local networks) on these CE routers into BGP:

R1:

router bgp 1
bgp router-id 150.1.1.1
redistribute connected
neighbor 192.168.13.1 remote-as 101
no auto-summary

R2:

router bgp 2
bgp router-id 150.1.2.2
redistribute connected
neighbor 192.168.25.1 remote-as 101
no auto-summary

R7:

router bgp 7
bgp router-id 150.1.7.7
redistribute connected
neighbor 192.168.67.1 remote-as 101
no auto-summary

R8:

router bgp 8
bgp router-id 150.1.8.8
redistribute connected
neighbor 192.168.58.1 remote-as 101
no auto-summary

Pretty basic. Now, let’s configure the PE routers to form BGP adjacencies with the CE routers. We’ll be putting the PE-to-PE and PE-to-CE configuration under a single BGP process, so there’s one thing we’ll need to address. When configuring MP-BGP, the default address-family of IPv4 needs to be changed. Since the BGP processes on our PE routers will now be dealing with IPv4 and VPNv4 address families, we need to apply the no bgp default ipv4-unicast command under the BGP routing configuration:

R3:

router bgp 101
bgp router-id 150.1.3.3
no bgp default ipv4-unicast
neighbor 192.168.13.2 remote-as 1
!
address-family ipv4 vrf CustB
neighbor 192.168.13.2 remote-as 1
neighbor 192.168.13.2 activate
exit-address-family

R5:

router bgp 101
bgp router-id 150.1.5.5
no bgp default ipv4-unicast
neighbor 192.168.25.2 remote-as 2
neighbor 192.168.58.2 remote-as 8
!
address-family ipv4 vrf CustA
neighbor 192.168.25.2 remote-as 2
neighbor 192.168.25.2 activate
exit-address-family
!
address-family ipv4 vrf CustB
neighbor 192.168.58.2 remote-as 8
neighbor 192.168.58.2 activate
exit-address-family

R6:

router bgp 101
bgp router-id 150.1.6.6
no bgp default ipv4-unicast
neighbor 192.168.67.2 remote-as 7
!
address-family ipv4 vrf CustA
neighbor 192.168.67.2 remote-as 7
neighbor 192.168.67.2 activate
exit-address-family

As you can see, this looks a little different than your standard BGP configuration. First, we needed to configure a neighbor under the general BGP configuration to establish the adjacency. We then created an IPv4 address family per VRF. Under each IPv4 address-family, we needed to once again configure the neighbor statement to our CE peer. A final step is to activate the exchange of IPv4 routing to each neighbor.

Let’s verify that we’re receiving routing information via BGP from our CE neighbors. We’ll look at R5 as an example:

R5#sh ip bgp vpnv4 all
BGP table version is 9, local router ID is 150.1.5.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf CustB)
*> 150.1.8.8/32 192.168.58.2 0 0 8 ?
*> 172.31.8.0/24 192.168.58.2 0 0 8 ?
r> 192.168.58.0/30 192.168.58.2 0 0 8 ?
Route Distinguisher: 2:1 (default for vrf CustA)
*> 150.1.2.2/32 192.168.25.2 0 0 2 ?
*> 172.31.2.0/24 192.168.25.2 0 0 2 ?
r> 192.168.25.0/30 192.168.25.2 0 0 2 ?

You’ll see that we’re receiving customer routes via eBGP and they’re being associated with their configured VRFs. We can also take a look at our VRF routing tables, using Customer A as an example:

R5#sh ip route vrf CustA

Routing Table: CustA
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

192.168.25.0/30 is subnetted, 1 subnets
C 192.168.25.0 is directly connected, Serial1/0
172.31.0.0/24 is subnetted, 1 subnets
B 172.31.2.0 [20/0] via 192.168.25.2, 00:10:22
150.1.0.0/32 is subnetted, 1 subnets
B 150.1.2.2 [20/0] via 192.168.25.2, 00:10:22

As you can see, each of the customer networks are logically separated by the router.

So far, so good, but not of much use until we get the customer sites communicating with each other. We’ll now configure the VPNv4 peerings between the PE routers to exchange VPNv4 routes. To simplify things, I’ve created a full mesh between the PEs and that way we don’t have to worry about route reflectors (RR). For our example, this isn’t a big deal; but a real ISP would definitely be using RRs. You’ll also notice that we’re peering to each PE’s loopback0 address, simply for redundancy reasons:

R3:

router bgp 101
neighbor 150.1.5.5 remote-as 101
neighbor 150.1.5.5 update-source lo0
neighbor 150.1.6.6 remote-as 101
neighbor 150.1.6.6 update-source lo0
!
address-family vpnv4
neighbor 150.1.5.5 activate
neighbor 150.1.5.5 send-community extended
neighbor 150.1.6.6 activate
neighbor 150.1.6.6 send-community extended
exit-address-family

R5:

router bgp 101
neighbor 150.1.3.3 remote-as 101
neighbor 150.1.3.3 update-source lo0
neighbor 150.1.6.6 remote-as 101
neighbor 150.1.6.6 update-source lo0
!
address-family vpnv4
neighbor 150.1.3.3 activate
neighbor 150.1.3.3 send-community extended
neighbor 150.1.6.6 activate
neighbor 150.1.6.6 send-community extended
exit-address-family

R6:

router bgp 101
neighbor 150.1.3.3 remote-as 101
neighbor 150.1.3.3 update-source lo0
neighbor 150.1.5.5 remote-as 101
neighbor 150.1.5.5 update-source lo0
!
address-family vpnv4
neighbor 150.1.3.3 activate
neighbor 150.1.3.3 send-community extended
neighbor 150.1.5.5 activate
neighbor 150.1.5.5 send-community extended
exit-address-family

So now we’ve formed adjacencies amongst the PE routers and added the VPNv4 address family to the mix. In addition to activating the address-family, we had to make sure that the send-community extended configuration was in place. This is because, as we noted ealier, route targets (RT) are sent within BGP as an extended community. Here’s an example showing this:

R5#sh ip bgp vpnv4 vrf CustB 150.1.1.1
BGP routing table entry for 1:1:150.1.1.1/32, version 13
Paths: (1 available, best #1, table CustB)
Advertised to update-groups:
1
1
150.1.3.3 (metric 3) from 150.1.3.3 (150.1.3.3)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:1:1
mpls labels in/out nolabel/21

At this point, we should now have full end-to-end connectivity between our customers. In the following example, we should be able to see networks pertaining to Customer A’s site in AS1 on Customer A’s CE routing table in AS 8:

R8#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

192.168.13.0/30 is subnetted, 1 subnets
B 192.168.13.0 [20/0] via 192.168.58.1, 00:10:21
192.168.58.0/30 is subnetted, 1 subnets
C 192.168.58.0 is directly connected, Serial1/0
172.31.0.0/24 is subnetted, 2 subnets
B 172.31.1.0 [20/0] via 192.168.58.1, 00:10:21
C 172.31.8.0 is directly connected, FastEthernet0/0
150.1.0.0/32 is subnetted, 2 subnets
C 150.1.8.8 is directly connected, Loopback0
B 150.1.1.1 [20/0] via 192.168.58.1, 00:10:21

Woohoo! Houston, we have contact.

This concludes our slightly lengthy adventure into the land of MPLS VPNs. I hope you were able to gain some knowledge and solid hands on experience from this article. In the future, we’ll take a look at some other aspects of MPLS VPNs and broaden our horizons!

References

BGP and MPLS-Based VPNs

RFC 2858 -Multiprotocol Extensions for BGP-4

Cisco IOS Multiprotocol Label Switching Configuration Guide, Release 12.4

Bookmark It

Hide Sites