Dealing With Backdoor Links on VOS Devices

By Radu Pavaloiu
  The purpose of this article is to show how to configure VOS devices to send the customer traffic using the SD-WAN network in case of a backdoor connectivity between customer’s sites. Running OSPF protocol between SD-WAN VOS appliances and customer’s routers present a special interest as OSPF prefers intra-area routes over inter-area routes over external routes for the same prefix. The VOS used in the testbed is 21.2.2 release. Figure 1 shows the topology used in this article.
Figure 1. OSPF between PE (VOS) devices and customer’s routers
  For now, let’s just ignore the GRE Tunnel (red link) in the above picture. To propagate the customer routes from site to site, OSPF is redistributed into MP-iBGP and vice versa on the VOS appliances. The downside of this is that all OSPF routes become external routes on the remote VOS appliance when the routes are redistributed back into OSPF. The result of this would be that all OSPF routes that transverse the SD-WAN network would be less preferable than the routes that did not transverse the SD-WAN but were sent via an intersite link (backdoor link) from one OSPF site to another. Checking on R5 the OSPF DB for external LSAs (type 5), We find the prefix coming from the other site (advertised by R4): 
admin@R5-cli> show ospf database external | match 192.168.14.0
External 192.168.14.0 5.5.5.5 0x80000001 329 0x000048FA
[ok][2021-09-28 01:29:21]
Now, checking on R5 the “router” LSA (type 1) advertised by R4: 
admin@R5-cli> show ospf database router adv-router 44.44.44.44 detail

OSPF Router with ID (55.55.55.55) Instance ID (3037)

Area 0.0.0.0; LSA type: router;
LSA ID ADV Router Sequence Age (secs) Checksum
------ ---------- -------- --------- --------
44.44.44.44 44.44.44.44 0x80000008 111 0x00005587
Age: 111 secs; Sequence number: 0x80000008; Checksum: 0x00005587
Bits: E, B Links: 3
Router links:
1. Link ID: 192.168.45.6; Link data: 192.168.45.6; Type: Transit
Num. of TOS: 0; metric: 1
2. Link ID: 192.168.45.2; Link data: 192.168.45.1; Type: Transit
Num. of TOS: 0; metric: 10
3. Link ID: 192.168.14.0; Link data: 255.255.255.0; Type: Stub
Num. of TOS: 0; metric: 1
R5 will prefer the intra-area (backdoor link) LSA vs the external LSA, and will build the routing table accordingly:   
admin@R5-cli> show route routing-instance Service-VNF-VR 192.168.14.0/24 exact

Routes for Routing instance : Service-VNF-VR AFI: ipv4 SAFI: unicast
[+] - Active Route

Routing entry for 192.168.14.0 (mask 255.255.255.0) [+]
Known via 'OSPF', distance 30, metric 11, type iA, forward metric 11
Redistributing via OSPF
Last update from 192.168.45.1 12m9s ago
Routing Descriptor Blocks:
* 192.168.45.1 , via vni-0/1.0 12m9s ago
The same is equally true from R4 perspective: 
admin@R4-cli> show route routing-instance Service-VNF-VR 192.168.15.0/24 exact

Routes for Routing instance : Service-VNF-VR AFI: ipv4 SAFI: unicast
[+] - Active Route

Routing entry for 192.168.15.0 (mask 255.255.255.0) [+]
Known via 'OSPF', distance 30, metric 11, type iA, forward metric 11
Redistributing via OSPF
Last update from 192.168.45.2 14m14s ago
Routing Descriptor Blocks:
* 192.168.45.2 , via vni-0/1.0 14m14s ago
We start to send traffic between the sites: 
gns3@Host1C2:~$ ip a s ens4
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 0c:86:9d:19:02:01 brd ff:ff:ff:ff:ff:ff
inet 192.168.14.2/24 brd 192.168.14.255 scope global noprefixroute ens4
valid_lft forever preferred_lft forever
inet6 fe80::95ad:3c3f:4ddd:ab5/64 scope link noprefixroute
valid_lft forever preferred_lft forever
gns3@Host1C2:~$ ping 192.168.15.2 -c 1
PING 192.168.15.2 (192.168.15.2) 56(84) bytes of data.
64 bytes from 192.168.15.2: icmp_seq=1 ttl=62 time=1.45 ms

--- 192.168.15.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.452/1.452/1.452/0.000 ms
As the traffic is not going over the SD-WAN network, but on the backdoor link there is no “icmp” session observed on the SD-WAN appliances: 
admin@Branch-4-cli> show orgs org Customer2 sessions brief
VSN VSN SESS DESTINATION SOURCE DESTINATION
ID VID ID SOURCE IP IP PORT PORT PROTOCOL NATTED SDWAN APPLICATION
---------------------------------------------------------------------------------------------------
0 2 1 11.0.0.10 11.0.0.0 1154 1234 6 No No -
The solution for this scenario is to emulate an OSPF intra-area link between the SD-WAN appliances over the SD-WAN network which results in prefixes being advertised as iA instead of E2 between the two respective sites. Basically, this is similar with the OSPF shamlink feature coming from the MPLS L3VPN world. Till the shamlink feature will be supported in the VOS this emulation could be built by using GRE tunnel between the branches and running OSPF on top of it. Now, lets move to the configuration side. The first step is to create 2 loopbacks on the branches (4.4.4.4 on Branch-4 and 5.5.5.5 on Branch-5) and add those to the LAN-VR and Orgs-Limit. These lookbacks addresses must be advertised in MP-BGP between the sites and as a result there will be reachability between them. Only Branch-4 is being shown but same operation will have to be performed on the remote Branch. These loopbacks will be used as support to build the GRE tunnel between the sites.
Figure2. Loopback interface configuration
Figure3. Loopback added to the LAN-VR interfaces
Figure 4. Loopback added to the Organization Limits
  The next step is to create a GRE tunnel between the 2 branches using the loopbacks as Source and Destination. Tunnel Type must be EoGRE as in the VOS point-to-point GRE tunnel interface is dropping the multicast traffic (OSPF is based on multicast and without supporting multicast even the OPSF adjacency would not come up)
Figure 5. GRE tunnel interface used to build OSPF neighborship
  Following the GRE tunnel interface creation, it should be added to the LAN-VR, Organization Limits and the OPSP process running in the LAN-VR.  
Figure 6. Create a network (Sham_Link) bound to the GRE interface
Figure  7. GRE tunnel interface added to the LAN-VR
 
Figure 8. GRE interface added to the Organization Limits
 
Figure 9. GRE interface added to the OSPF process
  Now, the communication between the sites should be possible and treated equally (OSPF iA routes) over both the backdoor link and the SD-WAN shamlink/virtual-link. However, for preferring one of them the OSPF cost of it should be lesser than the other one. For this reason, the OPSF backdoor links cost is increased artificially to be higher than the virtual link one.    
Figure 10. R4 backdoor link cost change
  Let’s move now to the verification task. All the verification will be done only on the Site-4/Branch-4 side. A similar process could be performed on the other site. As it can be seen on the bellow excerpt Branch-4 has two OSPF neighbors: one to the local LAN router and the second one with Branch-5 over the GRE tunnel. 
admin@Branch-4-cli> show ospf neighbor brief
State codes: atmpt - attempt, exchg - exchange, exst - exchange start,
load - loading, 2-way - two-way, full - full
Op codes: gdown - going down, gup - going up

Intf address Interface State Neighbor ID Pri Op
------------ --------- ----- ----------- --- --
45.45.45.2 tvi-0/1000.0 full 5.5.5.5 1 up
192.168.45.6 vni-0/2.0 full 44.44.44.44 1 up
[ok][2021-09-29 03:34:13]
Now we’ll check the route 192.168.15/24 from Site-4 perspective. – R4 is pointing to Branch-4 SD-WAN appliance: 
admin@R4-cli> show route routing-instance Service-VNF-VR 192.168.15.0/24 exact

Routes for Routing instance : Service-VNF-VR AFI: ipv4 SAFI: unicast
[+] - Active Route

Routing entry for 192.168.15.0 (mask 255.255.255.0) [+]
Known via 'OSPF', distance 30, metric 4, type iA, forward metric 4
Redistributing via OSPF
Last update from 192.168.45.5 40m43s ago
Routing Descriptor Blocks:
* 192.168.45.5 , via vni-0/0.0 40m43s ago
– Branch-4 is pointing to Branch-5 over the GRE tunnel: 
admin@Branch-4-cli> show route routing-instance Customer2-LAN-VR 192.168.15.0/24 exact

Routes for Routing instance : Customer2-LAN-VR AFI: ipv4 SAFI: unicast
[+] - Active Route

Routing entry for 192.168.15.0 (mask 255.255.255.0)
Known via 'BGP', distance 200,
Redistributing via BGP
Last update from 11.0.0.12 1h33m56s ago
Routing Descriptor Blocks:
* 11.0.0.12 , via Indirect 1h33m56s ago

Routing entry for 192.168.15.0 (mask 255.255.255.0) [+]
Known via 'OSPF', distance 30, metric 3, type iA, forward metric 3
Redistributing via OSPF
Last update from 45.45.45.2 42m44s ago
Routing Descriptor Blocks:
* 45.45.45.2 , via tvi-0/1000.0 42m44s ago <<<<<<
– and finally Branch-5 is pointing to R5: 
admin@Branch-5-cli> show route routing-instance Customer2-LAN-VR 192.168.15.0/24 exact

Routes for Routing instance : Customer2-LAN-VR AFI: ipv4 SAFI: unicast
[+] - Active Route

Routing entry for 192.168.15.0 (mask 255.255.255.0)
Known via 'BGP', distance 200,
Redistributing via BGP
Last update from 11.0.0.10 1h36m29s ago
Routing Descriptor Blocks:
* 11.0.0.10 , via Indirect 1h36m29s ago

Routing entry for 192.168.15.0 (mask 255.255.255.0) [+]
Known via 'OSPF', distance 30, metric 2, type iA, forward metric 2
Redistributing via OSPF
Last update from 192.168.45.10 1h39m23s ago
Routing Descriptor Blocks:
* 192.168.45.10 , via vni-0/2.0 1h39m23s ago <<<<<<
We’ll start again the ICMP test, probing if the traffic is now going over SD-WAN network. – Branch-4 appliance 
admin@Branch-4-cli> show orgs org Customer2 sessions brief
VSN VSN SESS DESTINATION SOURCE DESTINATION
ID VID ID SOURCE IP IP PORT PORT PROTOCOL NATTED SDWAN APPLICATION
---------------------------------------------------------------------------------------------------------
0 2 1 11.0.0.10 11.0.0.0 1154 1234 6 No No -
0 2 2 192.168.14.2 192.168.15.2 2346 2346 1 No No icmp/(predef)
– Branch-5 appliance 
admin@Branch-5-cli> show orgs org Customer2 sessions brief
VSN VSN SESS DESTINATION SOURCE DESTINATION
ID VID ID SOURCE IP IP PORT PORT PROTOCOL NATTED SDWAN APPLICATION
---------------------------------------------------------------------------------------------------------
0 2 1 11.0.0.12 11.0.0.0 1025 1234 6 No No -
0 2 2 192.168.14.2 192.168.15.2 2346 2346 1 No No icmp/(predef)
The above is a clear indication that traffic is flowing over the SD-WAN network. The backdoor link will be used only in case if all SD-WAN paths will fail. Regarding the presented solution there are few minor drawbacks that you should be aware of:
  • Configuration complexity increased a little bit.
  • The GRE header will contribute to the overlay header overhead (at least 24 bytes)
  • New addressed for the GRE tunnel are needed
  • Periodic LSA refresh flooding across the GRE interface
Starting with VOS release 22.1 OSPF sham link feature will be introduced. The sham link endpoint on each SD-WAN appliance will be a /32 IPv4 address from the specific Customer-LAN VR. The sham link endpoint is an unnumbered point-to-point intra-area link that is treated as a demand-circuit link. This means that LSAs are flooded across the sham link, but not periodic refresh flooding occurs over it. Also, the GRE header overhead would not be present. When the feature will be available a related article will be published on this blog. For more information regarding VOS routing capabilities please consult Versa Networks official documentation.