13

Raised bug -> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1874257

SSH timeout issue, once connect to VPN.

Connecting via putty fine. No changes made before.

VPN established by open-connect. This is previously working. Reinstalled VPN packages and reboot still no luck

Environment

Dell XPS 9570
Ubuntu 16.04.6 Xenial Xerus)
kernel - 4.15.0-55-generic

$dpkg -l | grep -i openssh
ii openssh-client 1:7.2p2-4ubuntu2.8 -->
ii openssh-server 1:7.2p2-4ubuntu2.8
ii openssh-sftp-server 1:7.2p2-4ubuntu2.8

VPN tunnel info
====
vpn0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:IP P-t-P:xx Mask:255.255.252.0
          inet6 addr: fe80::b8e2:bea4:2e62:fe08/64 Scope:Link
          UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1406 Metric:1
          RX packets:962 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1029 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:87839 (87.8 KB) TX bytes:238740 (238.7 KB)

Issue

Unable to connect to any host via ssh or sftp after VPN connection

Tried

Reinstalled the openssh-client package and still no luck. May I know why the default cipher is not taking/hanging? Please let me know . There were no recent changes.

Workaround

Able to connect to ssh / sftp $ssh -c aes128-ctr user@IP

Below is the debug ssh client logs ===

$ssh -vvv user@ip
OpenSSH_7.2p2 Ubuntu-4ubuntu2.8, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "IP" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to IP [IP] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/user/.ssh/id_rsa-cert type -1



debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

<< Hangs here >>

Please shed some views

Thanks

Jay
  • 171
  • 1
  • 1
  • 7
  • 1
    Related: https://serverfault.com/questions/210408/cannot-ssh-debug1-expecting-ssh2-msg-kex-dh-gex-reply – Terrance Apr 22 '20 at 13:50
  • 1
    Tried with MTU lower to 1200, still not fixed. the above link is not useful. – Jay Apr 25 '20 at 05:28
  • There are several answers there in the link that you can try. Only the accepted one worked for the person who asked the question, but there's also trying MTU at 1400, or setting different certificate types, etc. Don't stop after only trying the one thing there. Look for what might fit yours the best. None of the answers there that have upvotes look destructive at all. – Terrance Apr 25 '20 at 14:15
  • 2
    Hey I tried the MTU to 1100 which fixed the problem. What is the reason behind this? – Jay Apr 26 '20 at 13:27
  • It could be a noisy or unreliable connection to the server. Probably that data corruption is happening to the packet when it is sent. See the trade offs in https://en.wikipedia.org/wiki/Maximum_transmission_unit Also, there is a possibility that forward error correction is not being used. See: https://en.wikipedia.org/wiki/Forward_error_correction These are only speculations that I have on it. – Terrance Apr 26 '20 at 14:41
  • -> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1874257 issue with openconnect package - MTU fails with detection handling on v7.06 . As per community this is fixed in upstream 18.04 and 20.04 openconnect package. – Jay May 12 '20 at 11:05
  • People say it is an [MTU](https://en.wikipedia.org/wiki/Maximum_transmission_unit) problem, but for me, it still failed with `1200` or `1400`. [Weirdly enough, `1500` does the job!](https://serverfault.com/questions/210408/cannot-ssh-debug1-expecting-ssh2-msg-kex-dh-gex-reply#comment1336469_670081) So, for the future me: try to play a bit with this number :) – schneiderfelipe Jul 31 '20 at 18:44

5 Answers5

9

As a temporary workaround, setting the KEX algorithm manually solves this problem for me.

Add KexAlgorithms ecdh-sha2-nistp521 to the corresponding SSH config, or add -oKexAlgorithms=ecdh-sha2-nistp521 to the command line args for one time use.

Reference (2019): https://superuser.com/a/1462220/555702

bfrguci
  • 624
  • 1
  • 6
  • 16
7

Setting MTU to 1200 solved it for me (Ubuntu 20.04).

with (replace {dev}):  
sudo ip li set mtu 1200 dev {dev}
houseofkraft
  • 580
  • 1
  • 5
  • 14
lesinto
  • 71
  • 1
  • 2
  • Had same issue with `git` (via `ssh`) inside WSL2, `sudo ip link set eth0 mtu 1300` also helped – user158037 Dec 06 '21 at 14:12
  • Note that in some cases you will need to limit the MTU for the other direction too (on the SSH server). You can also limit MTU per route `ip route ... mtu 1300`. It is not always needed to set MTU for the whole interface. – pabouk - Ukraine stay strong Oct 30 '22 at 18:01
4

The root cause of the issue is with the openconnect VPN client package, MTU handling fail to negotiate. Bug -> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1874257

Jay
  • 171
  • 1
  • 1
  • 7
1

A colleague of mine just experienced this hanging at "expecting SSH2_MSG_KEX_ECDH_REPLY" issue this morning. He couldn't ssh into an Ubuntu 20.04.2 LTS server via OpenVPN. At first I thought it was an MTU issue but it turns out that it was a file permissions issue. Our organization uses signed keys and the file containing the CA public key only had owner read/write permissions.

We fixed the problem by adding read permissions for all users:

chmod a+r ca.pub

The bizarre thing about this problem was that even with the highest level of debug enabled while trying to use the ssh client, there was no error reported relating to the inability to read the file. The only related debug message had to do with successfully finding the @cert-authority line in the known_hosts file.

Caleb Carroll
  • 31
  • 1
0

I've experienced the problem while trying to connect to a GCP Ubuntu 22 VM which is a part of a ZeroTier network. A ZeroTier interface zt<xxxx> defaults MTU to 2800. Setting it to 1300 solved the issue.

BMC
  • 11
  • 2