Troubleshooting: Configuration Sync Issues from Director to Branch
This article describes how to troubleshoot configuration sync issued between Director and Branch.
Sometimes, you will not be able to sync the Director configuration with the branch and will get the following error on the Director UI.
Remote Server Exception view details malformed-message : Failed to authenticate towards device GIMEC-LAB-CPE2: Bad private SSH key for local/remote user arun.c/admin
This error state will not allow you to commit any changes to the Branch device.
This issue is caused by one of the two following causes
- Host IP moves from one device to another and the Director updates the keys of old IPs and tries to authenticate using the same keys. This authentication fails as the IPs are already configured in another host.
- The appliance is missing the Versa Director Keys
To fix the issue, we have two steps that need to be followed
Step 1: Execute the shell command
sudo /opt/versa/vnms/scripts/push_keys_to_device.sh <IP-Address> <device-password>
This shell command pushes the key from the Versa Director to the affected Branch.
admin@DIRECTOR:~$ sudo /opt/versa/vnms/scripts/push_keys_to_device.sh 10.3.64.229 ****** [sudo] password for admin: Warning: Permanently added '10.3.64.229' (ECDSA) to the list of known hosts. 0 0 admin@DIRECTOR:~$Step 2: Run the CLI command
request devices fetch-ssh-host-keys device <device-name>
This command, fetches the Key form the Branch Device using Versa Director.
Administrator@Director-New> request devices fetch-ssh-host-keys device Branch-2 fetch-result { device Branch-2 result unchanged fingerprint { algorithm ssh-dss value 2a:5c:1b:d1:4c:31:cd:37:35:a7:27:25:be:dd:f3:b7 } }
Once these steps are done, you will be able to sync the configuration between the Director and Branch and commit any changes via the Versa Director.
Step 3:
If there are multiple entries found under /home/admin/.ssh/authorized_keys on the branch, then truncate the file and follow the step1.
This will delete the Old Keys and push the new Key from Versa Director.
For example: Issue the truncate -s 0 /home/admin/.ssh/authorized_keys
[admin@Ind-Spoke-1: ~] $ cat /home/admin/.ssh/authorized_keys
ssh-dss AAAAB3NzaC1kc3MAAACBAIxT+EyQT99QPlFhn4VtIIZCPwmt9mECZxxqdynqbskqXIyXiOrN4pu1gyvWDaebxpufcPlPkp4wZXuyZCTBh/hNmadEleQfO3jb2xwsosynOhPUF1GoUpgxuSJRCPH/RrCOKc9qo68uKAnuVxQSKj8JF/MWMzPGp1oATWyvgl51AAAAFQDzUVfx12JtW+dfvPWkUZTXyQL31wAAAIANXUvarYpC+fwKDSMJKFe/KRNWYT+i+hYNBcCPc6r0JwTt0bhaeaw04JS+++EuRWwOc78K6vend2mnMP/VLJxIcrAUmIdkU9tqdiiVTEup4J8F3+lmF5E1kwk59ILIRRy4oSi4F0pWYcJRW3D8iOpqkqzZ6EOh/rKOqAxgnxZScgAAAIAVC+XAv66b0kBaygsTs0Yz6mvAMya9jlHD3SFpyAsO4/fMZ+42ik9QeSGo2kB3zZNXZxeJXxJPaitZfrlF3crHZrJuYMdq7WPKOdFhorlf4W0weCavLToGoKuSzcZPkyJ2gYRdaqbY5svqzNAxyvSgt1Phssn/PCIj2yHRFrMGPA== root@versa-director
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlrJUGHhOoETMWhPaHlBJmZhGy6qzqCiuXBA/vakZv7LN7J0yd+Atp2Eppj9XbNdq8633n+wqbnORQ7R7iVq+E9rnNYyRBqP9FhZfHUkdPtiQy0qd3Tf9RHMIrjyYDblbhn68VUWVH6s1uvMDy5xrEa6dNVGFhJjYpD5Zea2pwVh4xr9pNmjiUs4xX8x7522Ov+ab3W4CUAY19NZb6kEIAx+3Dy44lQ/wyH0yQV5S6jvcKLGKX91WgRQpzUN3Eab3mtw3VwNFQWyEUAjrlkLGQKYtEADOz+DU5u1yfZeC3WnrfQ1KAZ+dFlqG1p0NUlZU9Gq37CHIilCVlJqyg5/JV root@versa-director1
ssh-dss AAAAB3NzaC1kc3MAAACBAIDJ7aj5A5qsSKOmWvKpO5sPs6Sflixy85uMnxOx6f/6IeMSNX5WaY76qEqMnEVP6cwa7ogajtM7PHQXVRAVCr8Cx3ZIf4VGSW+ekuHJxivgqO7SOSOPmSIoYo2ufQYy4sIr7A5+ZNCfK6lwf32u8xSBwIeGrkssNXMfBO39NXXXAAAAFQDzJ/Oq5BU9qfjVggVdmx+LbIt9jQAAAIBQA/F2oYU7yDc8qpnDuCqk+24NSczb4Jqcswzs2CjgRXHpRnaS+Rc//eOR3H1eIVhPbCbvGv/Gy8aEspdugrkvYQKQAxTE5sSyoBqepK/5uNzdSfjl2LkrwJErTVsbV1smqxbm2V3JPBVcI6fAFafEN8vwqNcv5OWiwn9VVfxotgAAAIBYE7Ygmtq7bH7mZs4kWVwwPUyvp7aoX5WTFFZxb+eVZYgp87mUxqicFo671kwE2X7Hvigx74EjJLRUsSlFTNg8lX3nmJTiaQhhR85Ix3rC8Ui8L1jMFZCvLWyOSQWzk3/Rog85WUk3ZiI+FkfcCa1WTnXT6/JbdwX4l5F9lpZvkQ== root@versa-director
ssh-dss AAAAB3NzaC1kc3MAAACBALX9lSLVaUmDvuXIdIA5+gpw0mUlNe/gXnkg4t8NZqyrziTsUPh53XsZbd3aP3Pxj7FcGW6g8c8P92EBh+opb9mBtz52oWofWb0tufCS6POBt5hSY6bCnERsOy9NAEQGd7QkRBvgRYkNpHJYBOlJiSTmJG2odmtqhh30RtdZEWZNAAAAFQDOsTsHVSWE6tKtui5nRdaBttCHIwAAAIAUwDlGpMfL41f/jGrG0MAd9ZBJOWaZ57WGX3TZ3ykiGqXKUUP1Zls17mB6ZC+Lt/S9u1LMHa5/ESZ/1gdGbW2oPqJSujqBDghvXpTXWbugjbSdCiuxQXoDt4GJsurksvtuqQXMX9h+4h5PVZd/OLblX8Bc45kqltAwdYTQ6t0IkQAAAIEAs5YqJ9b86jdhxbohec2POWDOBm6wL5HVIfOwgE8ePHIeLacgkklnb2AvizKaW7gDikT3lAQYVjqiVeGZYNOcPCoBOCFk+2vKiFPFsgp96aAgR6UIDnZHWkpMEahxZE5b255vK/9nAd/tUjt2B4Fl8KFA6RauwBXXnTwDSGD0FeQ= root@versa-director
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsKiPu7TBPxKLhrp87a1wsV6T069Vr3f5YHIJmOZLqMof0cfv3L8CjvW42Pfb1qtZ5kwNG9FBiwjegh0hoqktMrTG9sKEOvhhr4UFO60PNO4kp/P8w/u7P7G6EoVDcFSPk+Vo1PQdHlpWxqgPJEu5BAm3IzSe616KdAAyeH2UOHCR7XEuc712EoACsmbPlXhev/FxDhk88btAOhoCDwCp8m9OI+g+wtT7NhRdwpNyTwygg9Rxj8SlUDYO7+729bGnl21f1lHiovXRUDyx9OlqlRGtALJLgLz68Zw83kVOSORQ9u2NlMW0uukkq0J6k2EBFrwUhAUS1p2MknRXaYu0d root@VD-LIL-IN-01-STX-STG
To truncate the /home/admin/.ssh/authorized_keys:
[admin@Ind-Spoke-1: ~] $ truncate -s 0 /home/admin/.ssh/authorized_keys
Summary
In this article, we saw what causes the sync issue between Director and Branch configuration and how to fix it by updating the Host IP address and keys.
