Before beginning this blog post, make sure to read How-To: MPLS configuration from a service provider perspective | Part 1

Introduction

So after part 1 we have a running MPLS network. Congratulations! It may not seem any different from the OSPF-enabled routed networks you may have experience with before with what we’ve done so far, but now that we have MPLS running, we can make use of its content-agnostic design to transport some more interesting traffic via our topology.

Prerequisites

Customers

We’ll add a Cust1Site1 & Cust1Site2 to our topology; these are the customer-edge (CE) routers. Cust1Site{1,2} are connected to Access{1,2} respectively from their ether1 to the PE’s ether2

Screen Shot 2016-08-24 at 3.22.04 PM

BGP

We’ll need BGP up with some alternate address families to support L2 and L3 VPNs. For the purpose of this exercise, we’ll use AS65530. We want to create peerings between loopback interfaces so that OSPF can provide us with redundancy for our BGP applications.

On each router, set our BGP router-id the same way as we set the OSPF router-id. Hopefully we still have the global variable $lo0addr. If not, see Part 1 to find the command to create it.

# Set the router-id
/routing bgp instance set [/routing bgp instance find default] router-id=$lo0addr

Route Reflectors

First off, since we have a core portion of our network, let’s establish some route reflectors. Route reflectors eliminate the need for a full-mesh topology with iBGP. Without an RR, each iBGP speaker would need to be adjacent with every other in BGP. Turning on route reflection is as simple as specifying it when creating a peer.

On Core1:

# Core1 -> Core2
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.2 remote-as=65530 route-reflect=yes update-source=lo0

On Core2:

# Core2 -> Core1
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.1 remote-as=65530 route-reflect=yes update-source=lo0

On Core{1,2}:
# Core{x} -> Agg1
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.3 remote-as=65530 route-reflect=yes update-source=lo0
# Core{x} -> Agg2
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.4 remote-as=65530 route-reflect=yes update-source=lo0
# Core{x} -> Agg3
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.5 remote-as=65530 route-reflect=yes update-source=lo0
# Core{x} -> Access1
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.6 remote-as=65530 route-reflect=yes update-source=lo0
# Core{x} -> Access2
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.7 remote-as=65530 route-reflect=yes update-source=lo0
# Core{x} -> Access3
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.8 remote-as=65530 route-reflect=yes update-source=lo0

On each of Agg{1,2,3} and Access{1,2,3}:

# {Agg,Access}{x} -> Core1
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.1 remote-as=65530 update-source=lo0
# {Agg,Access}{x} -> Core2
/routing bgp peer add address-families=ip,l2vpn,vpnv4 multihop=yes name=peer1 remote-address=10.101.0.2 remote-as=65530 update-source=lo0

We’re now running a route-reflected BGP network using MPLS and OSPF to transport BGP just like any other application. Remember that BGP is just TCP 179, so we need to treat it like any other application.

L3 VPN

Modern business is often geographically distributed; in this reality, we as service providers can make use of our technology and knowledge to provide businesses which have built outward with coherent networks facilitated by a layer-3 interconnect.

So when Cust1Site1 and Cust1Site2 need to be connected, let’s make it happen.

On Access{1,2}, let’s add a VRF for Customer 1 on ether2. Here, I’ve chosen the route-distinguisher 65530:1 to signify that us, 65530 is connected to “them” who I’ve designated as 1 for Customer1. These numbers are completely arbitrary, but must be distinct for each customer.

/ip route vrf add routing-mark=Cust1 route-distinguisher=65530:1 export-route-targets=65530:1 import-route-targets=65530:1 interfaces=ether2 redistribute-other-bgp=yes

Now, let’s associate that vrf with a BGP instance:

# Create customer-facing BGP instance for
# Access1:
/routing bgp instance add name=Cust1 router-id=10.102.0.1 as=65530 routing-table=Cust1
# Access2:
/routing bgp instance add name=Cust1 router-id=10.102.0.2 as=65530 routing-table=Cust1

# Create VRF in instance
/routing bgp instance vrf add routing-mark=Cust1

At this point, we should have a ready-to-go PE for the L3 VPN between sites. All we need to do now is enable a BGP peering between the CE and PE, then configure the CE. Though this is possible and practiced with other routing protocols, I’m of the opinion that BGP is the right tool for the job as a PE-CE peering protocol, since it is much better suited as a protocol between companies or borders of responsibility.

Let’s start by assigning a /30 for the PE-CE link

On Access1:

/ip address add interface=ether2 address=172.21.0.1/30

On Access2:

/ip address add interface=ether2 address=172.21.0.5/30

On Cust1Site1:

/ip address add interface=ether1 address=172.21.0.2/30

On Cust1Site2:

/ip address add interface=ether1 address=172.21.0.6/30

Now we can set up our PE-CE BGP peerings. First, set the ASN on each Cust1Site{1,2} to something different than the provider. I’ll use 65531 and 65532 here.

# Cust1Site1
/routing bgp instance set [/routing bgp instance find default] as=65531
# Cust1Site1
/routing bgp instance set [/routing bgp instance find default] as=65532

Create the peers on the Access{1,2} routers respectively. We’ve set the next-hop-self flag to make sure we don’t need to advertise the point-to-point bits.

# Access1 -> Cust1Site1
/routing bgp peer add remote-as=65531 remote-address=172.21.0.2 update-source=ether2 nexthop-choice=force-self instance=Cust1

# Access2 -> Cust1Site2
/routing bgp peer add remote-as=65532 remote-address=172.21.0.6 update-source=ether2 nexthop-choice=force-self instance=Cust1

And finally, create the peers on Cust1Site{1,2}:

# Cust1Site1 -> Access1
/routing bgp peer add remote-as=65530 remote-address=172.21.0.1

# Cust1Site2 -> Access2
/routing bgp peer add remote-as=65530 remote-address=172.21.0.5

Now, let’s create a route between the two sites on each site’s router:

# Cust1Site1
/routing bgp network add network=192.168.1.0/24 synchronize=no
# Cust1Site2
/routing bgp network add network=192.168.2.0/24 synchronize=no

These routes should be propagated to the other side! We now have bidirectional communication through an MPLS-enabled L3VPN.

If you’d like to grab a copy of this project with a working L3VPN and play with it in GNS3, the ZIP file here contains everything you’ll need to import it.

L2 VPN

Sometimes enterprises want or need layer-2 connectivity between sites. By-and-large, we don’t want to run large layer-2 networks in the provider space. High-capacity links are expensive and there’s no reason to leave them in blocking state. Seeing xe-0/0/1.0 or even et-0/0/1.0 status: blocked by STP is disappointing to say the least.

Part 3 is now online, which discusses layer-2 Metro Ethernet E-LAN MPLS-enabled VPNs via VPLS and BGP.