Troubleshooting: Reachability issue between Versa Director and Versa SD-WAN CPE

This document will provide the steps to be followed to narrow down and mitigate reachability issues between Versa Director and Versa SD-WAN CPE (Branch). This Communication forms a critical piece of the SD-WAN set up as the Director communicates with the Branch appliance over a netconf session – SSH on TCP port 2022.

It is important to note that the Director communicates to the appliance via the Controller.

Troubleshooting

Verifying reachability from Versa Director to Appliance has two parts –
I. Verifying IP reachability between Director and Branch
II. Verifying Netconf / SSH on the Director and Branch

I. Verifying IP reachability between Director and Branch

Step 1:
Verify IP reachability between the Versa Director and the Branch. We have to check this in two parts –
– IP reachability from Director to Branch
– IP reachability from DIrector to Controller

IP reachability from Director to Branch
From the Director Shell ping the Management IP of the Branch with a unique packet size.

IP reachability from DIrector to Controller
Verify IP reachability between Versa Director and Controller – the IP address of the Controller that connects to the Director.

Validate the session extensive output of this ICMP session between Versa Director to Branch to ensure that the Controller is forwarding the packets towards the Branch.
If sessions are not getting created on the Controller, it most likely is a case that the ICMP packets are not even reaching the Controller. At this point, we need to verify the IP connectivity between the Director and Controller.

Step 2:
Verify IP reachability between Controller and Branch.
In the controller, check if the Branch Management subnet is present in the Control-VR routing instance of the particular Tenant (in this case Tenant-2). This will be learnt over SD-WAN.

If the management route for the branch is present, check the reachability from the Controller to the Branch management IP over the respective Tenant Control-VR.

If the ping is unsuccessful, then check the Control Plane status between Controller and Branch – verify IPSec connectivity and BGP neighborship between the Controller and Branch. The IPsec should be established and the SLA’s between the Branch and the Controller over all the available WAN Transports which are a part of SD-WAN in the Branch.

The output below shows the SLA status from the Branch towards the controller and we see that the SLAs are up on all the Transport links.

If the the SLAs are UP, use TCPDUMP to verify the packets coming out of the controller on the WAN interface and at the Branch coming into the WAN interface matching the specific packet size used in Step 1.

There has to be a reverse route on the Control-VR for the source director subnet.

II. Verifying Netconf / SSH on the Director and Branch

Step 1:
Verify the hash values of the Director and Branch. These have to match for the Netconf session to establish even if there is IP reachability.

Step 2:
Check if the NetConf Port (2022) on the appliance is reachable. This can be done by running a netconf-check.sh script on the director. The expected result of the test is a ‘SUCCESS’ which confirms that NetConf / ssh sessions are up.

admin@director1:~$ /opt/versa/vnms/scripts/netconf-check.sh 10.1.64.101 versa123
PING 10.1.64.101 (10.1.64.101) 56(84) bytes of data.
64 bytes from 10.1.64.101: icmp_seq=1 ttl=63 time=2.70 ms
64 bytes from 10.1.64.101: icmp_seq=2 ttl=63 time=1.44 ms
64 bytes from 10.1.64.101: icmp_seq=3 ttl=63 time=1.12 ms
 
--- 10.1.64.101 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.120/1.756/2.707/0.685 ms
Warning: Permanently added '[10.1.64.101]:2022' (RSA) to the list of known hosts.
<?xml version="1.0" encoding="UTF-8"?>
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<capabilities>
<capability>urn:ietf:params:netconf:base:1.0</capability>
<capability>urn:ietf:params:netconf:base:1.1</capability>
===== SUCCESS: IPAddress and NetConf port is reachable and also able to do basic hand-shake====

Step 3:
Verify if the netconf session is open on the Branch and that intermediate device like a perimeter firewall is blocking it as the Branch will be listening on netconf port 2022 for the Director.

Step 4:
Verify if the Director Southbound IP address is present in ‘vnf-manager-ip addresses’ list and the relevant tvi interface is listed in the ‘vnf-manager vnf-mgmt-interfaces’.