How to correctly set up routing on machine with 4 interfaces so that three of interfaces are on the same subnet?

Question

So I have a linux machine with 4 interfaces:

1. enp1s0
2. enp2s0
3. enp3s0
4. enp4s0

What I would to do is have enp1s0 be the WAN interface and acquire its ip address, dns and gateway via DHCP.

For the other three interfaces, I would like them to have:

• IP addresses on the same subnet
• Have all hosts on one LAN interface be able to see all the other hosts on other LAN interface
• Run shorewall as a firewall
• Use DNS Masq

Here is what I've gotten so far:

 cat /etc/network/interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). source /etc/network/interfaces.d/* # The loopback network interface auto lo iface lo inet loopback # The primary network interface # listed as #1 # this is the wan interface auto enp1s0 iface enp1s0 inet dhcp # listed as #2 # this routes to upstairs auto enp3s0 iface enp3s0 inet static address 192.168.47.254 netmask 255.255.255.0 network 192.168.47.0 broadcast 192.168.47.255 # listed as #4 auto enp2s0 iface enp2s0 inet static address 192.168.47.253 netmask 255.255.255.0 network 192.168.47.0 broadcast 192.168.47.255 # listed as #3 auto enp4s0 iface enp4s0 inet static address 192.168.47.252 netmask 255.255.255.0 network 192.168.47.0 broadcast 192.168.47.255 

 cat /etc/dnsmasq.conf interface=enp2s0 interface=enp3s0 interface=enp4s0 listen-address=192.168.47.254 # Explicitly specify the address to listen on bind-interfaces # Bind to the interface to make sure we aren't sending things elsewhere server=8.8.8.8 # Forward DNS requests to Google DNS domain-needed # Don't forward short names bogus-priv # Never forward addresses in the non-routed address spaces. dhcp-range=192.168.47.100,192.168.47.250,12h # Assign IP addresses between 192.168.46.100-250 with a 12 hour lease time log-dhcp log-queries 

 interface=enp2s0 # Use interface wlan0 interface=enp3s0 # Use interface wlan0 interface=enp4s0 # Use interface wlan0 listen-address=192.168.47.254 # Explicitly specify the address to listen on bind-interfaces # Bind to the interface to make sure we aren't sending things elsewhere server=8.8.8.8 # Forward DNS requests to Google DNS domain-needed # Don't forward short names bogus-priv # Never forward addresses in the non-routed address spaces. dhcp-range=192.168.47.100,192.168.47.250,12h # Assign IP addresses between 192.168.46.100-250 with a 12 hour lease time log-dhcp log-queries 

And here is a sample ip route table from one of the client machines:

 ip -one addr 1: lo inet 127.0.0.1/8 scope host lo\ valid_lft forever preferred_lft forever 1: lo inet6 ::1/128 scope host \ valid_lft forever preferred_lft forever 2: eth0 inet 192.168.46.5/24 brd 192.168.46.255 scope global eth0\ valid_lft forever preferred_lft forever 2: eth0 inet6 fe80::82ee:73ff:fe5d:89d1/64 scope link \ valid_lft forever preferred_lft forever 3: eth1 inet 192.168.47.244/24 brd 192.168.47.255 scope global eth1\ valid_lft forever preferred_lft forever 3: eth1 inet6 fe80::82ee:73ff:fe5d:89d0/64 scope link \ valid_lft forever preferred_lft forever 

 route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.46.1 0.0.0.0 UG 0 0 0 eth0 192.168.46.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0 192.168.47.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1 

What works so far:

• Machines on different interfaces get IP address that are correct
• Machines can either ping all of the interfaces on the router or they can't. I think this might depend on which machine most recently DHCP'ed in
• If I can ping from a client machine, other things work too e.g. ssh
• I've never gotten two client machines to ping each other

After a lot of reading, I've determined that I need either static routes, different subnets for each lan interface or bridging.

Ideally, I would like to be set permissions in shorewall based on the interface and still use one subnet for all the LAN interfaces.

Answer 1

You're on the wrong track altogether, and it doesn't have anything to do with Shorewall (or really any other firewall package) but rather IP networking in general.

If you imagine that a given IP host has three interfaces in the same subnet and a packet arrives for that subnet from the fourth, how does that host know which of the three interfaces should receive the packet? Does it ARP in all three? Does it just send all the traffic to the first one that came up, thus preventing traffic bound for the other two? This is why if you tried to configure this from a shell you'd be given an error message about overlapping subnets.

So - you have three reasonable options, with a couple of sub-options.

1. Dump the other two interfaces and just hook up a single port to a switch and then hang the hosts (and possibly wifi in the same subnet) off of that switch. It's possible to join all three of your internal interfaces into a single bond or team but this basically takes all three physical interfaces and turns them into a single logical interface.

2. Create a Linux bridge interface. Give the IP, DHCP and other configuration elements to the bridge and then tie the three physical interfaces in as members of the bridge. Your firewall sees the single bridge interface as the inside (...even though it ties to three segments). Again - from an IP perspective you have one interface in the inside subnet, not three.

3. Put the three interfaces in three non-overlapping IP subnets and configure accordingly.