kokotap provides network tapping for Kubernetes Pod. kokotap creates VxLAN interface to target Pod/Container then do packet mirroring to the VxLAN interface by tc-mirred. kokotap can also create VxLAN interface to Kubernetes target node (e.g. 'kube-master') to capture the traffic or you can specify specific IP addresses for non Kubernetes node for capture.
kokotap supports following runtime:
- Docker runtime
- cri-o
See releases page.
Currently kokotap creates pod yaml file, so you can put it in kubectl to create pods.
[centos@kube-master ~]$ ./kokotap -h usage: kokotap --pod=POD --vxlan-id=VXLAN-ID [<flags>] kokotap Flags: -h, --help Show context-sensitive help (also try --help-long and --help-man). -v, --version Show application version. --pod=POD tap target pod name --pod-ifname="eth0" tap target interface name of pod (optional) --vxlan-id=VXLAN-ID VxLAN ID to encap tap traffic --vxlan-port=4789 VxLAN UDP port --ifname="mirror" Mirror interface name --mirrortype=both mirroring type {ingress|egress|both} --dest-node=DEST-NODE kubernetes node for tap interface --dest-ip=DEST-IP IP address for destination tap interface --namespace="default" namespace for pod/container (optional) --kubeconfig=KUBECONFIG kubeconfig file path (optional) --image="quay.io/s1061123/kokotap:latest" kokotap container image Example1 - Create a mirror interface for Pod 'centos' and receive interface "mirror" at kube-master.
This command creates two interfaces as following:
- VxLAN interface (name: mirror) at Pod to capture eth0 traffic
- VxLAN interface (name: mirror) at the kube-master (container host) to capture above Pod traffic
[centos@kube-master ~]$ ./kokotap --pod=centos --mirrortype=both \ --dest-node=kube-master --vxlan-id=100 | kubectl create -f - pod/kokotap-centos-sender created pod/kokotap-centos-receiver-kube-master created [centos@kube-master ~]$ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever (snip) 17: mirror: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN qlen 1000 link/ether 7e:3a:cb:bf:95:28 brd ff:ff:ff:ff:ff:ff inet6 fe80::7c3a:cbff:febf:9528/64 scope link valid_lft forever preferred_lft forever [centos@kube-master ~]$ ./kokotap --pod=centos --mirrortype=both \ --dest-node=kube-master --vxlan-id=100 | kubectl delete -f - pod "kokotap-centos-sender" deleted pod "kokotap-centos-receiver-kube-master" deleted [centos@kube-master ~]$ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever (snip) You can also delete mirror interface by removing two pods (begins with 'kokotap-', find by 'kubectl get pod')
This command create an interface as following:
- VxLAN interface (name: mirror) at Pod to capture eth0 traffic
You need to create VxLAN interface manually to receive mirror traffic in this case.
[centos@kube-master ~]$ ./kokotap --pod=centos --mirrortype=both \ --dest-ip=10.1.1.1 --vxlan-id=100 | kubectl create -f - pod/kokotap-centos-sender created pod/kokotap-centos-receiver-kube-master created [centos@10.1.1.1 ~]$ sudo ip link add mirror type vxlan id 192.168.1.1 dev eth0 dstport 4789 [centos@10.1.1.1 ~]$ sudo ip link set up mirror Same as Example1, but you need to delete receiver side by hand.
[centos@kube-master ~]$ ./kokotap --pod=centos --mirrortype=both \ --dest-ip=10.1.1.1 --vxlan-id=100 | kubectl delete -f - pod "kokotap-centos-sender" deleted pod "kokotap-centos-receiver-kube-master" deleted (snip) [centos@10.1.1.1 ~]$ sudo ip link set down mirror [centos@10.1.1.1 ~]$ sudo ip link delete mirror You can also delete mirror interface by removing two pods (begins with 'kokotap-', find by 'kubectl get pod')
- Add more usable feature (logging?)
- Document
- Test code
- Tomofumi Hayashi (s1061123)