I am having a lot of trouble establishing a reliable ppp / tcpip link between two debian systems over radio modems.
My hardware topology is a little complex.
The system uses:
- Raspberry Pi 3B at each end of the radio link running raspbian stretch (RPI)
- RFDesign RFD900x radio modems (connected via FTDI cable via USB to the RPIs) (RFD900)
- A linksys wifi router that it NATing (WIFI) To a satelite service (SkyMuster - Australia) to an unknown POP in AUS to the Internet (SAT)
- A vpn (vpnc) over the SAT to another Aus ISP static IP terminted by a router. (which is the default route for the RPI3Bs)
- (VPN) The vpn endpoint is connected to the net with a static ip (END)
I beleive the problem to be over the RFD900x modems, related to TCP congestion fall backs that occur when the radio drops packets, though I provide the other details for context and in case I'm missing somthing silly.
The issues are reproducable between the RPIs over the RFD900.
From the end-point (with the most trouble) the link to the Internet is as follows:
RPI -> RFD900 -> RFD900 -> RPI -> VPN -> WIFI -> SAT -> END.
Again the above for context.
The RFD900s drop packets a lot with the distance and obsticles involved. I have tried all sorts of aerial configurations to no avail (omni, yaggi, direct vs bouncing off granite cliffs). I have tried tuning all sorts of parameters in the modems, mtu, ppp settings etc to acheive TCP/IP reliability to no avail.
Air speed is 64kb. Serial speed is 57kb.
Diag notes:
- On simple serial to serial comms over the RFD900 at various distances the radio MTU size of 131 bytes or 151 bytes has best throughput.
- The throughput is reliable, though is "bursty" -> burst, burst, burst, rather than continuous flow.
- I suspect this "burstiness" is a function of TCP seeing the radio packet dropouts as congestion which progresses to an inevitable retry saturation.
- When it saturates, sessions (ssh, scp, apt etc) just seem to freeze for variable amounts of extended time (seconds, often 2-3 minutes, sometimes > 10 minutes).
- apt will generally fail. scp and ssh tend to keep on going and get there eventually, though usually with multiple stalls and crazy delay times.
- Interactively over ssh, the link is usable, providing that no long responses are involved - eg a long ls -la.
- Flow control to the modems (none, RTSCTS, XONXOFF) seem inconsequential to my tests.
- Different forms of ppp payload compression seem inconsequential (BSD, Predictor, deflate etc).
- Van Jacobsen header compression increases the throughput per burst, but exacerbates the stalls and delays
- I've searched extensively for solutions (even going back and reading the RFCs).
- It seems that VJ header comp was identified as problematic for lossy links and there have been RFC advances on compression techniques eg ROHC - RObust Header Compression, including a ROHC work-group from seems to have emerged various proprietary compression protocols that are not available in open source.
- The problem seems well-solved for cellular links (both with ppp and RLP) - which rely on proprietary protocols.
I also post here my current script which runs pppd (including the various options I've tried - see #commented lines.):
# set up the pppd to allow the remote unit to connect as an IP peer
# additional options for remote end: usepeerdns defaultroute replacedefultroute
pppd /dev/ttyUSB0 57600 mru 131 mtu 131 noauth crtscts nocdtrcts lock passive 192.168.10.1:192.168.10.2 local maxfail 0 persist proxyarp updetach
#pppd /dev/ttyUSB0 57600 novj novjccomp mru 131 mtu 131 noauth crtscts nocdtrcts lock passive 192.168.10.1:192.168.10.2 local maxfail 0 persist proxyarp updetach
#pppd /dev/ttyUSB0 57600 192.168.10.1:192.168.10.2 mru 131 mtu 131 proxyarp noauth crtscts nocdtrcts noaccomp nobsdcomp nodeflate nopcomp nopredictor1 novj novjccomp lock mru 131 mtu 131 passive local maxfail 0 persist updetach
#debug ktune bsdcomp 12 xonxoff nocrtscts mru 296 mtu 296
#pppd /dev/ttyUSB0 57600 debug mru 131 mtu 131 noauth crtscts nocdtrcts lock passive 192.168.10.1:192.168.10.2 local maxfail 0 persist updetach proxyarp
#pppd /dev/ttyUSB0 57600 noaccomp nobsdcomp nodeflate nopcomp nopredictor1 novj novjccomp mru 131 mtu 131 noauth crtscts nocdtrcts lock passive 192.168.10.1:192.168.10.2 local maxfail 0 persist proxyarp updetach
#pppd /dev/ttyUSB0 57600 novjccomp mru 131 mtu 131 noauth crtscts nocdtrcts lock passive 192.168.10.1:192.168.10.2 local maxfail 0 persist proxyarp updetach
Has anyone solved this with open source pppd? Are there other options or technologies which would be an alternative?
Are kernel TCP congestion settings worth looking into?
