IP address removed from docker0 bridge on QEMU/KVM virtual machine

Question

I'm renting a virtualized server running Ubuntu 22.04 and I'm trying to run Docker containers within it. The server is virtualized with qemu-kvm.

Unfortunatly, the containers don't have network access.

host:~# docker run -it ubuntu /bin/bash
container:/# apt-get update

The APT update fails because the repositorys cannot be reached. Checking the repositorys from the host (QEMU VM), the repositorys are totally fine:

host:~# ping archive.ubuntu.com

I've already tried No internet connection inside Docker containers and it didn't work, simply restarting the Docker service isn't the solution neither.

The hardware firewall is deactivated and ufw is disabled.

How do I get internet access in my containers?

Edit 2

Placing this above because that seems more relevant to me.

Relating to the answer here I set up systemd-networkd accordingly. The docker0 interface keeps it's 172.17.0.1 address until I start a container. Then the IP is lost. As long as docker0 has it's IP addess, the following route exists: 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown

Edit 1

As far as I understand QEMU, the network interface has to be explicitly connected to the physical host network interface. Since Docker is creating own network interfaces for every network, or at lease docker0, this isn't connected to the internet.

Let's assume I have an eth0 interface as default network interface in my VM. Can I set the iptables to route the docker0's traffic through the eth0 interface?

System information:

ip route

default via 87.106.234.1 dev ens6 proto dhcp src [LOCAL_IP] metric 100 
87.106.234.1 dev ens6 proto dhcp scope link src [LOCAL_IP] metric 100 
212.227.123.16 via 87.106.234.1 dev ens6 proto dhcp src [LOCAL_IP] metric 100 
212.227.123.17 via 87.106.234.1 dev ens6 proto dhcp src [LOCAL_IP] metric 100

ifconfig

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        ether 02:42:74:19:f6:1a  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet [LOCAL_IP]  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 [LOCAL_IP_V6]  prefixlen 64  scopeid 0x20<link>
        ether 02:01:72:39:35:f9  txqueuelen 1000  (Ethernet)
        RX packets 14885  bytes 164716593 (164.7 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 11369  bytes 1415635 (1.4 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 184  bytes 19073 (19.0 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 184  bytes 19073 (19.0 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

nft list ruleset

table ip nat {
    chain DOCKER {
        iifname "docker0" counter packets 0 bytes 0 return
    }

    chain POSTROUTING {
        type nat hook postrouting priority srcnat; policy accept;
        oifname != "docker0" ip saddr 172.17.0.0/16 counter packets 0 bytes 0 masquerade 
    }

    chain PREROUTING {
        type nat hook prerouting priority dstnat; policy accept;
        fib daddr type local counter packets 3317 bytes 150836 jump DOCKER
    }

    chain OUTPUT {
        type nat hook output priority -100; policy accept;
        ip daddr != 127.0.0.0/8 fib daddr type local counter packets 0 bytes 0 jump DOCKER
    }
}
table ip filter {
    chain DOCKER {
    }

    chain DOCKER-ISOLATION-STAGE-1 {
        iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
        counter packets 0 bytes 0 return
    }

    chain DOCKER-ISOLATION-STAGE-2 {
        oifname "docker0" counter packets 0 bytes 0 drop
        counter packets 0 bytes 0 return
    }

    chain FORWARD {
        type filter hook forward priority filter; policy drop;
        counter packets 0 bytes 0 jump DOCKER-USER
        counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-1
        oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
        oifname "docker0" counter packets 0 bytes 0 jump DOCKER
        iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
        iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
    }

    chain DOCKER-USER {
        counter packets 0 bytes 0 return
    }
}

This is not a Docker or Docker Compose problem **absolutely**. I'd suggest removing those tags as irrelevant. This *is* the problem of virtual machine networking and general routing, and it's not QEMU-specific but this happens with any virtualization platform. No, you *can't* "set iptables to route" because iptables is not an interface to routing (rather, it's an interface to a packet filter/mangler), `ip route` is. The real answer will depend on what are current routes in the VM. — Nikita Kipriyanov, Jun 28 '23 at 08:06
And on how the networking in the VM is constructed in general. What is QEMU startup line? What is the `ip route` output in the VM? — Nikita Kipriyanov, Jun 28 '23 at 08:12
I'd also be interested in `nft list ruleset` or `iptables-save` on the VM — Jaromanda X, Jun 28 '23 at 08:20
@NikitaKipriyanov Thanks for your reply, I've added some more information. I know that's not a problem within Docker, but in combination with Qemu/KVM and docker. However, if that's not what tags are ment for here I'll remove them. — Eric, Jun 28 '23 at 12:35
@JaromandaX I did, too. That changes sometimes, since when I ran the command a little bit earlier, the interface had the typical `172.17.0.1` address. So with or without, the result was the same unfortunately. — Eric, Jun 28 '23 at 19:57
well, you'd expect a `route` as well, does that sometimes exist too? — Jaromanda X, Jun 29 '23 at 00:57
@JaromandaX Well now I realize that could be the problem indeed. When I launch a simple ubuntu container the according `verth...` network has no IPv4 neither. They just get an IPv6 addess which won't work with docker when IPv6 isn't enabled. — Eric, Jun 29 '23 at 04:22
Just tested if the internet connection would work with an IPv6 network instead, but that's not the case. — Eric, Jun 29 '23 at 04:26
@JaromandaX I tested some things related to the self removing IP. Please see my edit in the question. — Eric, Jun 29 '23 at 04:55

score 1 · Accepted Answer · answered Jul 13 '23 at 15:57

The host resolved this issue with a configuration change:

In /etc/netplan/50-cloud-init.yaml, replace

match:
  name: '*'

with

match:
  name: 'en*'

The root problem was, that the docker0 network lost it's IP address, so it had no internet connection.

IP address removed from docker0 bridge on QEMU/KVM virtual machine

1 Answers1