1

I have two ISP options where both provide DHCP leases. I want the secondary (and possibly a 3rd when they allow & if needed) as a failover if the first (or second) fails, eg:

diagram

After reviewing Netplan examples, it is a little confusing to see how to bridge these (or even if a bridge is necessary) while having the main interface receive IP config values via DHCP from the ISP and then to hand that off to a static output to the LAN at 192.168.10.1, for example. It appears that you might do something like this:

network:
  version: 2
  renderer: networkd
  ethernets:
    # LAN interface
    eth0:
      optional: true
      dhcp4: no
      addresses:
        - 192.168.10.1/24
      # Is a gateway necessary since it provides a bridge?
      gateway4: 192.168.1.1
      nameservers:
          addresses: [1.1.1.1, 192.168.1.1]
    # ISP 1 interface
    eth1:
      optional: true
      dhcp4: true
    # ISP 2 interface
    eth2:
      optional: true
      dhcp4: true
    # ISP 3interface
    eth3:
      optional: true
      dhcp4: true
  bridges:
    br0:
      addresses: [ 192.168.10.1/24 ]
      interfaces: [ eth0 eth1 eth2 eth3 ]

It also appears that metric values might need to be assigned to the ISP interfaces... But then again, I see other examples that make me think I may be going about this wrong.

What would or should be used in this situation as far as a Netplan conf goes?

UPDATE: heynnema's answer seems close below and localhost (on Ubuntu Server) get's to the Internet fine, however while a test system can ping to 192.168.10.1 just fine, it can't ping on out to 1.1.1.1 and thus it appears that there's still a routing issue between eth0 and eth(1|2|3).

As per request, ip addr and ip route ouptput: https://gist.github.com/ylluminate/6435840c37edc01e82c047c61f4c071b

ylluminate
  • 1,359
  • 2
  • 14
  • 24

2 Answers2

1

Maybe start with something closer to this...

Note: This doesn't take into account having to come up with routing tables for iptables.

network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      # local LAN
      optional: true
      addresses:
        - 192.168.10.1/24
    eth1:
      # ISP 1 is primary ISP
      optional: true
      dhcp4: true
      dhcp4-overrides:
        route-metric: 100
    eth2:
      # ISP 2 is incomplete (tether)
      optional: true
      dhcp4: true
      dhcp4-overrides:
        route-metric: 200
  wifis:
    wlan0:
      # ISP 3 is incomplete (wireless)
      dhcp4: true
      dhcp4-overrides:
        route-metric: 300
      access-points:
        "network_ssid_name":
          password: "**********"

sudo netplan --debug generate

sudo netplan apply

reboot

heynnema
  • 68,647
  • 15
  • 124
  • 180
  • Given that I don't know what IP subnet always comes from the ISPs, I'm curious as to why you included the `192.168.1.1` in the gateway4 and nameserver fields? Is there a way to properly determine what these should be when we don't know the lease value for `eth(1|2|3)`? I realize that I put those into my config above, but this is perhaps part of where I'm confused and part of the answer to your comment above... – ylluminate May 17 '20 at 16:53
  • @ylluminate I made some minor edits in my answer, based on slangasek's answer. I'm waiting for confirmation on the new .yaml. – heynnema May 17 '20 at 17:44
  • just tested and this is getting closer (I suspect .10.1 needs to be assigned to an interface / route in some fashion that chooses eth1 or eth2 based on availability and priority?) Screenshot: https://share.getcloudapp.com/rRu9A4zj – ylluminate May 17 '20 at 18:14
  • @ylluminate Try `ping -I eth0 1.1.1.1` and `ping -I eth1 1.1.1.1` and `ping -I eth2 1.1.1.1` and see which ones may respond, and which ones might not. Also make sure that .2 is valid for that network. – heynnema May 17 '20 at 18:58
  • `eth0` is complete loss. Both `eth1` and `eth2` reply correctly. This machine is isolated and only has ISP (`eth1` & `eth2`) connections with `eth0` (192.168.10.2) going out to a LAN so it shouldn't matter what IP is used here. It's simply that there is no .10.1 as far as I can tell since I thought this machine itself should / would be .10.1... – ylluminate May 17 '20 at 19:11
  • @ylluminate For eth0, there is no router/modem? If not, then how can it be the primary WAN? See if you can `ping -I eth0 192.168.10.x`, ping another device on the 192.168.10.x LAN. Only eth1 and eth2 have a router/modem? – heynnema May 17 '20 at 19:19
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/108147/discussion-between-ylluminate-and-heynnema). – ylluminate May 17 '20 at 19:34
  • it can't be correct to both have 192.168.10.1 as the host address on eth0 and to also have it as the gateway for a default route attached to eth1. What routes are presented to you by the dhcp server on eth1's network? – slangasek May 19 '20 at 19:59
  • you might want to just show the full output of 'ip addr' and 'ip route' when using this particular yaml, so we can look at exactly what's being presented by all the dhcp servers. – slangasek May 19 '20 at 20:00
  • @slangasek Thanks for your input. I'll let the OP respond, and I'll take a step back and let you see if you can figure out a way for this to work. Thanks! – heynnema May 19 '20 at 20:10
  • @ylluminate Please respond to slangasek's comments. Start comments with "@slangasek". – heynnema May 19 '20 at 20:11
  • @slangasek would you mind having a chat somewhere (maybe IRC if nothing else is simpler) so I can start with a fundamental review of what should actually be done? I've been told on Freenode (#netplan) that Netplan doesn't support NAT'ing and that I need to utilize UFW via https://devops.ionos.com/tutorials/deploy-outbound-nat-gateway-on-ubuntu/#enable-ip-forwarding (modified slightly: https://gist.github.com/ylluminate/29c4f2877e1ef4e6d8e5985e0c96b588) so I wanted to make sure I'm not going in the wrong direction in the first place and before investing more time... – ylluminate May 19 '20 at 20:18
  • @ylluminate Just use the chat room that we initiated 2 days ago. Edit your question with *"show the full output of 'ip addr' and 'ip route'"* when using my .yaml. – heynnema May 19 '20 at 20:20
  • @slangasek added `ip` output and [moved to chat](https://chat.stackexchange.com/rooms/108147/discussion-between-ylluminate-and-heynnema) in hopes of covering more ground. Also please note my UFW remarks: https://askubuntu.com/questions/1240452/netplan-configuration-for-a-router-with-2-or-more-input-isps-with-failover-for-s#comment2092035_1240618 – ylluminate May 19 '20 at 22:24
0

You cannot use a bridge or bond to treat upstream links from multiple ISPs the same, because none of your ISPs are going to route traffic that uses the IP of the other.

Since you say you only care about using the additional ISPs as failover, then you probably should simply set route metrics for each of the uplinks, and use NAT so that internal machines do not have to care about changes to the active uplink network.

slangasek
  • 5,293
  • 2
  • 18
  • 26
  • Based on some new requirements from OP, I've put together a new .yaml that is incomplete, and is probably wrong... but it's gotten over my head... – heynnema May 17 '20 at 20:15
  • @heynnema is close but still running into an issue and he says you probably can clarify the issue. Note the updates to the original question as requested by him. – ylluminate May 17 '20 at 20:32