VPN on Linux - IP routing issue

Question

I got the following scenario:

an ubuntu server 20.04 lts, for simplicity named A server with the following network interfaces:

• loopback
• enp1s0 (wan) PUBLIC-IP/23
• enp8s0 (lan) 10.9.96.3/20
• ppp0 (l2tp) 192.168.42.1

with this routing table (public routes are intentionally omitted):

• 10.9.96.0/20 dev enp8s0 proto kernel scope link src 10.9.96.3
• 192.168.1.0/24 via 192.168.42.10 dev ppp0
• 192.168.42.10 dev ppp0 proto kernel scope link src 192.168.42.1

There's a remote vpn client connected with ip 192.168.42.10.
This client is a MikroTik router providing the remote lan network 192.168.1.0/24
Using the static route added by me (192.168.1.0/24 via 192.168.42.10 dev ppp0) I am able to reach out 192.168.1.0/24 devices.

---

another ubuntu server 20.04 lts, for simplicity named B server with the following network interfaces:

• loopback
• enp1s0 (wan) PUBLIC-IP/23
• enp8s0 (lan) 10.9.96.4/20

and this routing table (public routes are intentionally omitted):

• 10.9.96.0/20 dev enp8s0 proto kernel scope link src 10.9.96.4

Basically I need to access 192.168.1.0/24 devices from server B but I cannot make it work.
I've also tried to add a static route: 192.168.1.0/24 via 10.9.96.3 dev enp8s0
within this server without success, I see packets destined to 192.168.1.X reaching out server A but then they are not forwarded to ppp0 interface I guess (so I've tried also some iptables rules)

How to fix this issue?

Firewall is disabled in both servers.

Answer 1

NOTE: In this answer, the network 100.64.10.0/23 represents the public network. It's not particularly relevant but exists only to make the network configuration of these virtual nodes match what you've described in your question.

---

Your problem is that hosts on the 192.168.1.0/24 network don't know how to reach the 10.9.96.0/20 network.

Based on what you've shown us in your question, the routing table on the microtik router (192.168.42.10/192.168.1.1) probably looks something like this:

192.168.1.0/24 dev h1-eth0 proto kernel scope link src 192.168.1.1 192.168.42.0/24 dev h1-eth1 proto kernel scope link src 192.168.42.10 

And the routing table on serverA looks something like:

default via 100.64.10.1 dev serverA-eth0 10.9.96.0/20 dev serverA-eth1 proto kernel scope link src 10.9.96.3 100.64.10.0/23 dev serverA-eth0 proto kernel scope link src 100.64.10.10 192.168.1.0/24 via 192.168.42.10 dev serverA-eth2 192.168.42.0/24 dev serverA-eth2 proto kernel scope link src 192.168.42.1 

On serverA, when you ping an address on the 192.168.1.0/24 network, your source address is 192.168.42.1. The microtik router has a route for that network, so everything is great.

If on serverB you have a routing table like this:

default via 100.64.10.1 dev serverB-eth0 10.9.96.0/20 dev serverB-eth1 proto kernel scope link src 10.9.96.4 100.64.10.0/23 dev serverB-eth0 proto kernel scope link src 100.64.10.20 192.168.1.0/24 via 10.9.96.3 dev serverB-eth1 

Then when you try to connect to an address on the 192.168.1.0/24 network, your source address will be 10.9.96.4. The microtik router doesn't know how to reach that address (or it attempts to respond via the default gateway, which is inappropriate).

The solution is to add a route to the microtik router like this:

ip route add 10.9.96.0/20 via 192.168.42.1 

Now:

• ServerB can ping the microtik router because the microtik router has a valid route back to serverB
• ServerB can ping other hosts on the 192.168.1.0/24 network because those hosts use the microtik router (192.168.1.1) as their default gateway...and as established in the previous bullet point, the router has an appropriate route.

---

I tested all of this in a simulated network environment built using mininet; you can find the configuration here if you're curious.