DEV Community

Cover image for Supercharge Your Container Networking: Seamless Host Communication with VxLAN and Docker
Tanvir Rahman
Tanvir Rahman

Posted on

Supercharge Your Container Networking: Seamless Host Communication with VxLAN and Docker

#docker #devops #linux #cloud

In this hands-on demo, we will explore how to set up communication between two hosts using Virtual Extensible LAN (VxLAN) and Docker. The goal is to create a VxLAN overlay network tunnel between the hosts' containers, allowing them to communicate with each other. Let's dive into the steps involved in this process.

What are we going to cover in this hands-on demo?

  • Creating two VMs and installing Docker: We will use two virtual machines (VMs) and install Docker to run the containers.

  • Creating separate subnets and assigning static IP addresses: To simplify the setup, we'll create separate subnets for each VM and assign static IP addresses.

  • Creating a VxLAN bridge: We'll utilize the Linux "ip link vxlan" feature to create a VxLAN bridge.

  • Binding the VxLAN to the Docker bridge: We'll bind the VxLAN to the Docker bridge to establish the tunnel.

  • Verifying communication between containers: Finally, we'll test the communication between containers on different hosts.

Let's get started!
To facilitate effective communication between hosts, we need to deploy two VMs using any hypervisor or virtualization technology. It is crucial to ensure that both VMs are connected to the same network.

Step 1: Creating the VMs

We begin by creating two Lubuntu VMs using UTM / Multipass. These VMs will serve as the hosts for our containers.

Step 2: Installing necessary tools and Docker

In this step, we will install several necessary tools and Docker on our Ubuntu VMs. Here's a brief overview of the tools we are going to install:

  • net-tools: This package includes important network management tools such as ifconfig, netstat, route, etc.
  • iputils-ping: This package provides the ping command, which is used to test the reachability of a network host.
  • bridge-utils: This package provides utilities for configuring Ethernet bridging on Linux.

To install these tools and Docker, execute the following commands:

apt-get update apt-get install net-tools apt-get install iputils-ping apt-get install bridge-utils apt-get install -y docker.io
Enter fullscreen mode Exit fullscreen mode

Additionally, if the IP address of one of the VMs is the same as the other, you'll need to modify it to avoid conflicts. For example, you can change the IP address of one of the VMs to 192.168.64.7/24 using the following commands:

ifconfig eth0 192.168.64.7/24 route add default gw 192.168.64.1 eth0
Enter fullscreen mode Exit fullscreen mode

To configure DNS resolution, open the resolv.conf file:

nano /etc/resolv.conf
Enter fullscreen mode Exit fullscreen mode

Delete the existing content and add the following lines:

nameserver 8.8.8.8 nameserver 8.8.4.4
Enter fullscreen mode Exit fullscreen mode

Image 1

Now, let's proceed to set up the containers and their communication.

Step 3: Running Docker Containers on the VxLAN Network
Now for host1(amicable-hyena),

# create a separate docker bridge network  docker network create --subnet 172.18.0.0/16 vxlan-net # list all networks in docker docker network ls # The output should include the newly created vxlan-net network. NETWORK ID NAME DRIVER SCOPE 53be5ce8e682 bridge bridge local 757eb3a14b73 host host local c1c0e4f01fa6 none null local 08bdd2dc3a82 vxlan-net bridge local # Check interfaces ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 36:db:7c:3f:11:58 brd ff:ff:ff:ff:ff:ff inet 192.168.64.9/24 brd 192.168.64.255 scope global dynamic enp0s2 valid_lft 85395sec preferred_lft 85395sec inet6 fde4:cfbc:2cca:cd78:34db:7cff:fe3f:1158/64 scope global dynamic mngtmpaddr noprefixroute valid_lft 2591923sec preferred_lft 604723sec inet6 fe80::34db:7cff:fe3f:1158/64 scope link valid_lft forever preferred_lft forever 3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:aa:59:e4:e3 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever 4: br-08bdd2dc3a82: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:4b:42:67:6b brd ff:ff:ff:ff:ff:ff inet 172.18.0.1/16 brd 172.18.255.255 scope global br-08bdd2dc3a82 valid_lft forever preferred_lft forever
Enter fullscreen mode Exit fullscreen mode

For Host 2(affluent-toad),

docker network create --subnet 172.18.0.0/16 vxlan-net docker network ls cdc5ceed0de2 bridge bridge local 11ed966e63df host host local 5ea8856ad30c none null local 55316818ea9f vxlan-net bridge local ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether f2:5f:8d:d3:9f:c9 brd ff:ff:ff:ff:ff:ff inet 192.168.64.10/24 brd 192.168.64.255 scope global dynamic enp0s2 valid_lft 85276sec preferred_lft 85276sec inet6 fde4:cfbc:2cca:cd78:f05f:8dff:fed3:9fc9/64 scope global dynamic mngtmpaddr noprefixroute valid_lft 2591908sec preferred_lft 604708sec inet6 fe80::f05f:8dff:fed3:9fc9/64 scope link valid_lft forever preferred_lft forever 3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:b3:cb:9a:b9 brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 valid_lft forever preferred_lft forever 4: br-55316818ea9f: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default link/ether 02:42:2c:83:cd:df brd ff:ff:ff:ff:ff:ff inet 172.18.0.1/16 brd 172.18.255.255 scope global br-55316818ea9f valid_lft forever preferred_lft forever
Enter fullscreen mode Exit fullscreen mode

Step 4: Run docker container
Let's run docker container on top of newly created docker bridge network and try to ping docker bridge

For Host 1,

# running alpine container with "sleep 3000" and a static ip docker run -d --net vxlan-net --ip 172.18.0.11 alpine sleep 3000 # check the container running or not docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d2b96eddea7a alpine "sleep 3000" About a minute ago Up About a minute funny_jepsen # check the IPAddress to make sure that the ip assigned properly docker inspect d2 | grep IPAddress "SecondaryIPAddresses": null, "IPAddress": "", "IPAddress": "172.18.0.11", # ping the docker bridge ip to see whether the traffic can pass ping 172.18.0.1 -c 2 PING 172.18.0.1 (172.18.0.1) 56(84) bytes of data. 64 bytes from 172.18.0.1: icmp_seq=1 ttl=64 time=0.166 ms 64 bytes from 172.18.0.1: icmp_seq=2 ttl=64 time=0.050 ms --- 172.18.0.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1018ms rtt min/avg/max/mdev = 0.050/0.108/0.166/0.058 ms
Enter fullscreen mode Exit fullscreen mode

For Host 2,

docker run -d --net vxlan-net --ip 172.18.0.12 alpine sleep 3000 docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9d2e17598b8b alpine "sleep 3000" 3 seconds ago Up 2 seconds eloquent_black docker inspect 9d | grep IPAddress "SecondaryIPAddresses": null, "IPAddress": "", "IPAddress": "172.18.0.12", ping 172.18.0.1 -c 2 PING 172.18.0.1 (172.18.0.1) 56(84) bytes of data. 64 bytes from 172.18.0.1: icmp_seq=1 ttl=64 time=0.193 ms 64 bytes from 172.18.0.1: icmp_seq=2 ttl=64 time=0.067 ms --- 172.18.0.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1017ms rtt min/avg/max/mdev = 0.067/0.130/0.193/0.063 ms
Enter fullscreen mode Exit fullscreen mode

Step 5: Testing Communication
Now, let's access one of the running containers and test communication between hosts. Note that containers within the same host should communicate, but container-to-container communication between hosts will fail at this stage because there is no tunnel or anything to carry the traffic.

docker exec -it d2 sh apk update apk add net-tools apk add iputils-ping ping 172.18.0.12 -c 2 ping: -c: Try again ping -c 2 192.168.64.10 PING 192.168.64.10 (192.168.64.10) 56(84) bytes of data. 64 bytes from 192.168.64.10: icmp_seq=1 ttl=63 time=3.93 ms 64 bytes from 192.168.64.10: icmp_seq=2 ttl=63 time=0.967 ms --- 192.168.64.10 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 0.967/2.447/3.928/1.480 ms
Enter fullscreen mode Exit fullscreen mode

Step 6: Creating the VxLAN Tunnel
To establish communication between the containers, we need to create a VxLAN tunnel and attach it to the Docker bridge. Make sure the VNI (Virtual Network Identifier) ID is the same for both hosts.

For Host 1,

# check the bridges list on the hosts brctl show bridge name bridge id STP enabled interfaces br-08bdd2dc3a82 8000.02424b42676b no docker0 8000.0242aa59e4e3 no # create a vxlan # 'vxlan-demo' is the name of the interface, type should be vxlan # VNI ID is 100 # dstport should be 4789 which a udp standard port for vxlan communication # 192.168.64.10 is the ip of another host ip link add vxlan-demo type vxlan id 100 remote 192.168.64.10 dstport 4789 dev enp0s2 # check interface list if the vxlan interface created ip a | grep vxla 7: vxlan-demo: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000 # make the interface up ip link set vxlan-demo up # now attach the newly created vxlan interface to the docker bridge we created brctl addif br-08bdd2dc3a82 vxlan-demo # check the route to ensure everything is okay. here '172.18.0.0' part is our concern part. route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.64.1 0.0.0.0 UG 100 0 0 enp0s2 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-08bdd2dc3a82 192.168.64.0 0.0.0.0 255.255.255.0 U 0 0 0 enp0s2 192.168.64.1 0.0.0.0 255.255.255.255 UH 100 0 0 enp0s2
Enter fullscreen mode Exit fullscreen mode

For Host 2,

brctl show bridge name bridge id STP enabled interfaces br-55316818ea9f 8000.02422c83cddf no docker0 8000.0242b3cb9ab9 ip link add vxlan-demo type vxlan id 100 remote 192.168.64.9 dstport 4789 dev enp0s2 ip a | grep vxlan 7: vxlan-demo: <BROADCAST,MULTICAST> mtu 1450 qdisc noop state DOWN group default qlen 1000 ip link set vxlan-demo up brctl addif br-55316818ea9f vxlan-demo route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.64.1 0.0.0.0 UG 100 0 0 enp0s2 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-55316818ea9f 192.168.64.0 0.0.0.0 255.255.255.0 U 0 0 0 enp0s2 192.168.64.1 0.0.0.0 255.255.255.255 UH 100 0 0 enp0s2
Enter fullscreen mode Exit fullscreen mode

Step 7: Testing Communication between Containers
Now that the VxLAN overlay network tunnel has been created, let's test the communication between the containers on different hosts.

docker exec -it a9 bash ping 192.168.64.10 -c 2
Enter fullscreen mode Exit fullscreen mode

You can find result of ping from below images. [Note: Re ran docker container in both host as previous container already ran for 3000 seconds]

Image 3

You should see successful ping responses, indicating that communication between the containers on different hosts is now established.

Congratulations! You have successfully set up communication between hosts using VxLAN and Docker.

Reference:
The demonstration code and steps mentioned in this blog post were adapted from the following GitHub repository: vxlan-docker-hands-on

Feel free to explore the repository for more in-depth details and examples.

Thank you for reading this blog post, and I hope you found it informative and useful.

Top comments (0)