Requests from external network fail, though initial packets reach server

Question

I have an Ubuntu 16.04.1 LTS server that serves pages just fine to anything on the same network, but fails to respond to requests coming from outside.

Packets appear to reach the server OK (so I don't think the router / VM host is to blame). For example this is tcpdump from my phone's browser (not on wifi):

sudo tcpdump -n -nn -tttt -i enp0s8 port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes
2017-03-04 01:38:17.541225 IP 209.171.x.x.19235 > 192.168.100.100.80:Flags [S], seq 2831699911, win 32120, options [mss 1460,sackOK,TS val 3366582762 ecr 0,nop,wscale 7], length 0
2017-03-04 01:38:18.533486 IP 209.171.x.x.19235 > 192.168.100.100.80: Flags [S], seq 2831699911, win 32120, options [mss 1460,sackOK,TS val 3366583762 ecr 0,nop,wscale 7], length 0
2017-03-04 01:38:20.251207 IP 209.171.x.x.30518 > 192.168.100.100.80: Flags [S], seq 3262824289, win 32120, options [mss 1460,sackOK,TS val 3366585480 ecr 0,nop,wscale 7], length 0
2017-03-04 01:38:21.249853 IP 209.171.x.x.30518 > 192.168.100.100.80: Flags [S], seq 3262824289, win 32120, options [mss 1460,sackOK,TS val 3366586480 ecr 0,nop,wscale 7], length 0
2017-03-04 01:38:23.261555 IP 209.171.x.x.30518 > 192.168.100.100.80: Flags [S], seq 3262824289, win 32120, options [mss 1460,sackOK,TS val 3366588480 ecr 0,nop,wscale 7], length 0
2017-03-04 01:38:27.251426 IP 209.171.x.x.30518 > 192.168.100.100.80: Flags [S], seq 3262824289, win 32120, options [mss 1460,sackOK,TS val 3366592480 ecr 0,nop,wscale 7], length 0

For comparison this is another LAN computer loading (successfully) a page from this server:

sudo tcpdump -n -nn -tttt -i enp0s8 port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes
2017-03-04 01:52:12.740465 IP 192.168.0.11.50439 > 192.168.100.100.80: Flags [S], seq 3685185069, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
2017-03-04 01:52:12.740538 IP 192.168.100.100.80 > 192.168.0.11.50439: Flags [S.], seq 262599678, ack 3685185070, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
2017-03-04 01:52:12.741398 IP 192.168.0.11.50439 > 192.168.100.100.80: Flags [.], ack 1, win 16425, length 0
2017-03-04 01:52:12.742041 IP 192.168.0.11.50439 > 192.168.100.100.80: Flags [P.], seq 1:399, ack 1, win 16425, length 398: HTTP: GET / HTTP/1.1
2017-03-04 01:52:12.742081 IP 192.168.100.100.80 > 192.168.0.11.50439: Flags [.], ack 399, win 237, length 0
2017-03-04 01:52:12.742769 IP 192.168.100.100.80 > 192.168.0.11.50439: Flags [P.], seq 1:151, ack 399, win 237, length 150: HTTP: HTTP/1.1 200 OK
2017-03-04 01:52:12.804288 IP 192.168.0.11.50439 > 192.168.100.100.80: Flags [P.], seq 399:783, ack 151, win 16387, length 384: HTTP: GET /favicon.ico HTTP/1.1
2017-03-04 01:52:12.804985 IP 192.168.100.100.80 > 192.168.0.11.50439: Flags [P.], seq 151:301, ack 783, win 245, length 150: HTTP: HTTP/1.1 200 OK
2017-03-04 01:52:13.016970 IP 192.168.100.100.80 > 192.168.0.11.50439: Flags [P.], seq 151:301, ack 783, win 245, length 150: HTTP: HTTP/1.1 200 OK
2017-03-04 01:52:13.017937 IP 192.168.0.11.50439 > 192.168.100.100.80: Flags [.], ack 301, win 16350, options [nop,nop,sack 1 {151:301}], length 0
2017-03-04 01:52:14.927196 IP 192.168.0.11.50439 > 192.168.100.100.80: Flags [F.], seq 783, ack 301, win 16350, length 0
2017-03-04 01:52:14.927729 IP 192.168.100.100.80 > 192.168.0.11.50439: Flags [F.], seq 301, ack 784, win 245, length 0
2017-03-04 01:52:14.928142 IP 192.168.0.11.50439 > 192.168.100.100.80: Flags [.], ack 302, win 16350, length 0
2017-03-04 01:52:15.732564 IP 192.168.0.11.50437 > 192.168.100.100.80: Flags [S], seq 420429641, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
2017-03-04 01:52:15.732638 IP 192.168.100.100.80 > 192.168.0.11.50437: Flags [S.], seq 113394854, ack 420429642, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
2017-03-04 01:52:15.733329 IP 192.168.0.11.50437 > 192.168.100.100.80: Flags [.], ack 1, win 16425, length 0
2017-03-04 01:52:15.738538 IP 192.168.0.11.50438 > 192.168.100.100.80: Flags [S], seq 1232950050, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
2017-03-04 01:52:15.738644 IP 192.168.100.100.80 > 192.168.0.11.50438: Flags [S.], seq 2713178861, ack 1232950051, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0

ifconfig

enp0s3    Link encap:Ethernet  HWaddr 08:00:27:c3:0a:85
      inet addr:10.0.2.15  Bcast:10.0.2.255  Mask:255.255.255.0
      inet6 addr: fe80::a00:27ff:fec3:a85/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:32 errors:0 dropped:0 overruns:0 frame:0
      TX packets:33 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000
      RX bytes:3412 (3.4 KB)  TX bytes:3144 (3.1 KB)

enp0s8    Link encap:Ethernet  HWaddr 08:00:27:c3:18:c4
      inet addr:192.168.100.100  Bcast:192.168.255.255  Mask:255.255.0.0
      inet6 addr: fe80::a00:27ff:fec3:18c4/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:7138 errors:0 dropped:0 overruns:0 frame:0
      TX packets:2364 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000
      RX bytes:928468 (928.4 KB)  TX bytes:289133 (289.1 KB)

enp0s9    Link encap:Ethernet  HWaddr 08:00:27:bd:31:ae
      inet addr:10.10.100.100  Bcast:10.10.100.255  Mask:255.255.255.0
      inet6 addr: fe80::a00:27ff:febd:31ae/64 Scope:Link
      UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
      RX packets:45 errors:0 dropped:0 overruns:0 frame:0
      TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1000
      RX bytes:5535 (5.5 KB)  TX bytes:648 (648.0 B)

lo        Link encap:Local Loopback
      inet addr:127.0.0.1  Mask:255.0.0.0
      inet6 addr: ::1/128 Scope:Host
      UP LOOPBACK RUNNING  MTU:65536  Metric:1
      RX packets:0 errors:0 dropped:0 overruns:0 frame:0
      TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
      collisions:0 txqueuelen:1
      RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Firewall I removed UFW and cleared all its leftover chains from iptables, as well as cleared iptables

iptables -L -v
Chain INPUT (policy ACCEPT 66 packets, 6921 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 50 packets, 7337 bytes)
 pkts bytes target     prot opt in     out     source               destination

Router: Asus RT-AC56U with latest firmware Has port 80 forwarded to 192.168.100.100:80 Router internal IP is 192.168.0.1 and the LAN is 192.168.0.0/255.255.0.0

Vagrant/VirtualBox: The server is running in a Vagrant + VirtualBox VM on a Windows 10 host. Host's firewall is completely disabled. **The network corresponding to enp0s8 is bridged to the host's wired network adapter (which in turn is connected to a wired port on the router)

Edit: The listening server is a basic Node script (run as sudo for access to port 80) which just answers with plain text and prints to console the remote IP of connections. So no Apache/Nginx involved.

Anything in `/var/log/apache2/error.log` at times your phone browser attempted to connect? — Organic Marble, Mar 04 '17 at 02:13
The listening server is just a simple node.js app. It prints the remote IP on the connect event and has only printed local IPs. Nothing for the phone connection attempts. [This is the code](http://pastebin.com/hvBcQFuN) — Nova 9, Mar 04 '17 at 02:20
Your server doesn't appear to be listening to all IP addresses. By the way there is no "-nn" switch for tcpdump, although I must admit I spread the idea around a few years before I got corrected by someone. — Doug Smythies, Mar 04 '17 at 03:33
Also consider `sudo tcpdump -n -tttt -i port 80` so as to monitor all interfaces to determine if the return packets, if there are any, are being routed to another interface. — Doug Smythies, Mar 04 '17 at 07:01

Requests from external network fail, though initial packets reach server

0 Answers0