Troubleshooting : IKE & IPSec Issues
Versa Secure SD-WAN architecture relies on secure IPSec-over-VXLAN overlay tunnels to transmit control-plane and data-plane traffic.
Versa Controller creates IPSec tunnels that carry control-plane information to the branches and hubs using the underlay transport network. Similarly, the branches, between themselves or between the hub and the branch form IPSec tunnels that carry data-plane traffic.
This article explains how to troubleshoot IKE & IPSec issues.
Troubleshooting
View IKE Session History
To troubleshoot an IKE session, we can view it from the session history. IKE session history can be viewed by executing the CLI command
show orgs org-services organization-name ipsec vpn-profile profile-name ike history
The command output displays the IKE session history information including the sessions that flapped and the reason for the flap.
versa@PoP3-Ten1-Branch2-cli> show orgs org-services My-Organization ipsec vpn-profile branch-cntrl1 ike history Local Gateway: 10.3.12.1 Remote Gateway: 10.1.1.121 Last Known State : Active (Rekey) Last State Timestamp : 2015-12-21T06:25:5518.101768-00:00 Event History: 0. Event : IKE Rekey Timestamp : 2015-12-21T06:25:5526.560768-00:00 Role : initiator Inbound SPI : 0x3fd02bbfd83d0002 Outbound SPI: 0xe28dedee106e0002 1. Event : IKE Rekey Timestamp : 2015-12-20T23:00:24534.391488-00:00 Role : initiator Inbound SPI : 0xaf42d13b41ae0002 Outbound SPI: 0x32dd83255e370002 2. Event : IKE Rekey Timestamp : 2015-12-20T15:34:53165.19972-00:00 Role : initiator Inbound SPI : 0x598148d4b880002 Outbound SPI: 0x8d396252e73a0002 3. Event : IKE Rekey Timestamp : 2015-12-20T08:09:21060.162088-00:00 Role : initiator Inbound SPI : 0xdd149fd165df0002 Outbound SPI: 0xcd3cd3f7e85d0002
In the output above, we see that the IKE session has been re-initiated as the IPSec rekey interval is about to expire.
View IKE Session
To display the IKE security association, run the following CLI command,
show orgs org-services organization-name ipsec vpn-profile profile-name ike security-associations brief
versa@PoP3-Ten1-Branch2-cli> show orgs org-services My-Organization ipsec vpn-profile branch-cntrl1 ike security-associations brief Flags: P - PSK C - Certificate N - NAT-T R - Responder I - Initiator Tunnel Ver Local Remote VPN Local Remote Flags ID Gateway Gateway Type SPI SPI ------ ---- --------------- --------------- ---------- ------------------ ------------------ ------- 2 v2 10.3.12.1 10.1.1.121 SDWAN-B 0xc311bdf523e40002 0x52cdbeb4963d0002 P,I [ok][2020-09-02 02:16:08] admin@T1-Branch2-cli>
View IPSec Tunnel Information
To display information about the IPsec tunnel, run the show orgs org-services organization-name ipsec vpn-profile profile-name security-associations brief CLI command.
versa@PoP3-Ten1-Branch2-cli> show orgs org-services My-Organization ipsec vpn-profile branch-cntrl1 security-associations brief Remote Gateway Transform Inbound SPI Bytes/sec Outbound SPI Bytes/sec Up Time Next Rekey Time --------------- --------- ----------- --------- ------------ --------- -------- --------------- 10.3.11.1 aes-gcm 0x2004150 115 0x2000de2 0 3w6d21h 02:07:27
Troubleshoot IPSec in Stage 1 and Stage 2
To identify any issues in Stage 1 & 2, check if the IPSec session is up between the branch and the Controller.
versa@PoP3-Ten1-Branch2-cli> show orgs org-services My-Organization ipsec vpn-profile branch-cntrl1 security-associations brief Remote Gateway Transform Inbound SPI Bytes/sec Outbound SPI Bytes/sec Up Time Next Rekey Time --------------- --------- ----------- --------- ------------ --------- -------- --------------- 10.3.11.1 aes-gcm 0x2004150 115 0x2000de2 0 3w6d21h 02:07:27
If the IPSec session is down, then
a. Check the IPSec configuration to ensure that the local and remote authentication parameters and that the local and remote IP are those for the VNI interfaces.
b. Ping from the local IP address to remote IP address in the IPsec profile to ensure that they are reachable.
If at this point the peer IP is not reachable, then issues with the data path.
If the IPSec session between branch and controller is up, then
a. Check if the TVI interfaces have been assigned an IP address by the staging server or Controller.
admin@PoP3-Ten2-Branch5-cli> show interfaces brief NAME IP MAC OPER ADMIN TNT VRF ------------- ---------------------- ----------------- ----- ----- --- --------------- tvi-0/3 n/a up up tvi-0/3.0 [ 10.3.1.113/24 ] n/a up up 1 mgmt vni-0/0 52:0a:30:be:05:02 up up vni-0/0.0 [ 113.1.1.5/24 ] 52:0a:30:be:05:02 up up 1 grt-vrf vni-0/1 52:0a:30:be:05:03 down down vni-0/2 52:0a:30:be:ce:04 down down
In the above output, tvi-0/3.0 is the IPSec tunnel interface and the IP from the staging pool has been assigned by the Controller.
b. Verify reachability by executing a ping from the Versa Director to branch device – the tvi interface IP address. If the ping fails, check if the route for the Director IP address is installed in the Branch routing table.
c. Confirm access by doing ssh into the branch device from the Director.
Troubleshoot IPSec Stage 3 : Branch-to-Controller Issues
Verify if the IPSec session is up between the branch and Controller.
versa@PoP3-Ten1-Branch2-cli> show orgs org-services My-Organization ipsec vpn-profile branch-cntrl1 security-associations br Remote Gateway Transform Inbound SPI Bytes/sec Outbound SPI Bytes/sec Tunnel Status Up Time --------------- --------- ----------- --------- ------------ --------- ------------- -------- 10.3.11.1 aes-cbc 0x20aebb9 0 0x20b5bba 0 UP 1071 sec >>>> First entry is between branch and Controller 10.3.13.1 aes-cbc 0x20adbbb 0 0x20adbba 0 UP 1113 sec >>>> Additional entries are for branch to branch 10.3.14.1 aes-cbc 0x20adbbc 0 0x20adbba 0 UP 339 sec 10.1.1.121 aes-cbc 0x20069de 0 0x2000a36 0 UP 9728 sec
Troubleshoot IPSec Stage 3 : Branch-to-Branch Issues
To check whether the IPsec sessions between the branch and all other branches are up, run the show orgs org-services organization-name ipsec vpn-profile profile-name branch-2-branch security-associations br CLI command.
In the command output below, the first entry is for the Controller and rest are for branches. All the IPsec sessions must be up.
versa@PoP3-Ten1-Branch2-cli> show orgs org-services Costco ipsec vpn-profile branch-cntrl1 branch-2-branch security-associations br Remote Gateway Transform Inbound SPI Bytes/sec Outbound SPI Bytes/sec Tunnel Status Up Time --------------- --------- ----------- --------- ------------ --------- ------------- ------- 10.3.11.1 aes-cbc 0x20aebb9 0 0x20b5bba 0 UP 1071 sec >>>> First entry is between branch and Controller 10.3.13.1 aes-cbc 0x20adbbb 0 0x20adbba 0 UP 1113 sec >>>> Subsequent entries are for branch to branch 10.3.14.1 aes-cbc 0x20adbbc 0 0x20adbba 0 UP 339 sec 10.1.1.121 aes-cbc 0x20069de 0 0x2000a36 0 UP 9728 sec
DHKEY protocol is used to exchange the IPsec keys for branch-to-branch communication. For every remote branch, one PTVI-ESP interface is created, and the DHKEY pair protocol generates and periodically refreshes the IPsec key pairs between any two branches.
This SPI is associated with the PTVI-ESP interface corresponding to the remote branch.
If the branch-2-branch IPSec is down, we have to run an esp-ping between the branches and if that fails, we need to enable IPSec debugging logs and verify if the correct SPI index is associated with the PTVI index.
Summary
In this article we saw how to troubleshoot IKE & IPSec issues on Versa VOS device and in turn check the connectivity between Branches or between Hub & branch sites.
