1

tldr; bridge (see below) doesn't work if there is a matching drop in another table (like the default rules of firewalld).

Hello,
I'm building my own VM lib (kind of quickemu).
I have a problem with the firewall rules for bridging. I started with iptables-nft and it worked fine until I tested my scripts on fedora. No matter what I do, firewalld blocks everything. So I went with nftables to try to bypass firewalld rules directly, but even then I can't find a way to make it work without deleting firewalld rules.

It's my understanding that the priority only affects the order in which the rules are tested, if there is a drop, even at the end, the packet is dropped?

Is there a way to bypass this behavior that I don't know about? I tried looking into marks, but I'm not sure it's the right solution.

Here is the rules I use for the bridge (works perfectly is there is no other table in the ruleset):

#!/usr/bin/nft -f # vim:set ts=2 sw=2 et: table ip QEMU delete table ip QEMU table ip QEMU { chain input { type filter hook input priority filter - 1; ct state {established,related} iifname "virbr0" counter accept } chain forward { type filter hook forward priority filter - 1; ct state {established,related} iifname "virbr0" counter accept } chain postrouting { type nat hook postrouting priority srcnat; iifname "virbr0" counter masquerade } }
Improve this question
8
  • 1
    You are correct: a dropped packet stays dropped, wherever and whenever it was dropped. Only an accepted packet gets the chance to be ... dropped later. When multiple firewalling tools are competing between each others, things become complicated. Note that nowhere in what you showed bridging (as in: forwarding Ethernet frames) is involved: it's routing over the bridge interface itself, the side that participates in routing. If now you are also running Docker or Kubernetes, then interactions between bridge firewalling and routing firewalling can also happen (making it even more difficult). Commented Mar 13, 2024 at 18:45
  • 1
    Here's how libvirt handles it: libvirt.org/… Commented Mar 13, 2024 at 19:12
  • Well that sucks. How is that logical behavior...? What if there is another firewall in place... You really have to go through all the rules and modify them every time you need to add something like that? Yeah I guess this is not a bridge. I have a wireless card, so I tried NAT instead (seemed simpler). I guess I got lost in my research and confused the terms. Commented Mar 13, 2024 at 20:52
  • 1
    To be a bit pedantic: the interface virbr0 is really a bridge. but the firewall you are considering is not at the bridge path, so it's not a bridging firewall (unless despite itself because of running Docker and loading the br_netfilter kernel module) and thus no bridging is involved (for the firewall). The fact that a Linux bridge is often involved in routing (through the interface itself rather than a bridge port) makes things confusing. Commented Mar 13, 2024 at 20:57
  • For comparison: have you ever heard of a commercial firewall applicance were two competing firewalls are working at the same time on it? This doesn't happen. There might be instances, partitioning or whatever, but if there are multiple firewall instances on it, they are made to work together. One should have a single firewall, or be ready to cope with consequences if not. Commented Mar 13, 2024 at 21:01

1 Answer 1

Reset to default
3

Nitpicking: despite virbr0 being a bridge interface, this is about routing, not about bridging. The firewall happens on the routing side of the bridge: the bridge interface itself, not on the bridge ports (which would require a table bridge rather than a table ip firewall). So the word bridging won't be mentioned again below.

When a packet is dropped within netfilter, including nftables, it stays dropped. When a packet is accepted, it will then just continue to traverse additional hooks possibly making this packet dropped later, as reminded in nft(8):

accept

Terminate ruleset evaluation and accept the packet. The packet can still be dropped later by another hook, for instance accept in the forward hook still allows one to drop the packet later in the postrouting hook, or another forward base chain that has a higher priority number and is evaluated afterwards in the processing pipeline.

drop

Terminate ruleset evaluation and drop the packet. The drop occurs instantly, no further chains or hooks are evaluated. It is not possible to accept the packet in a later chain again, as those are not evaluated anymore for the packet.

This causes firewalling tools with a default drop behavior and accepting only some flows (which is a good behavior from a security standpoint) rather than tools with a default accept behavior and dropping some flows (which is not so good from a security standpoint) to occult the decision from other firewalling tools: they drop things that other tools would have chosen to accept.

So to have two tools co-exist, actions have usually to be done on both of them.

For this case, one can tell firewalld to ignore (ie: accept) flows related to one or more specific interface(s), so the handling of this specific interface remains in the control of an other tool. If such handling is not done elsewhere, this can possibly open a security hole instead, so this has to be considered properly.

This answer requires firewalld >= 0.9.0 to leverage firewalld policies as initially introduced:

Policies are applied to traffic flowing between zones in a stateful unidirectional manner. This allows different policies depending on the direction of traffic.

+----------+ policyA +----------+ | | <------------ | | | libvirt | | public | | | ------------> | | +----------+ policyB +----------+

The zone is used between a remote side and the host, policies are used for routing between zones. They will reuse interfaces attached to zones.

Create a new zone:

firewall-cmd --permanent --new-zone=local-ignore

Have this zone accept anything by default:

firewall-cmd --permanent --zone=local-ignore --set-target=ACCEPT

Add the target interface to this zone:

firewall-cmd --permanent --zone=local-ignore --add-interface=virbr0

This will take care of traffic between VMs and host (once rules are reloaded).

Add firewalld policies that will use this zone, once for ingress and once egress, with ANY as other side and also set them to a default behavior of ACCEPT:

firewall-cmd --permanent --new-policy=local-ignore-from firewall-cmd --permanent --policy=local-ignore-from --add-ingress-zone=local-ignore firewall-cmd --permanent --policy=local-ignore-from --add-egress-zone=ANY firewall-cmd --permanent --policy=local-ignore-from --set-target=ACCEPT firewall-cmd --permanent --new-policy=local-ignore-to firewall-cmd --permanent --policy=local-ignore-to --add-egress-zone=local-ignore firewall-cmd --permanent --policy=local-ignore-to --add-ingress-zone=ANY firewall-cmd --permanent --policy=local-ignore-to --set-target=ACCEPT

Finally reload the rules:

firewall-cmd --reload

This takes care of having traffic related to virbr0 unhindered by firewalld.

Now the (huge) nftables table inet firewalld should co-exist peacefully with OP's table ip QEMU. Whether firewalld is running or not (ie: its rules are removed) should not change the behavior for anything related to virbr0. All restrictions have to be handled in table ip QEMU.

Currently table ip QEMU doesn't restrict anything at all (there is no drop rule nor drop policy) so should be fixed properly. One should not use a drop policy or this will cause again the problem that has just been fixed, in the other direction. Just drop any unwanted traffic related to virbr0 (especially towards virbr0) in it.

Additional interfaces can be added to the ignore list, for example to add lxcbr0 (meaning: to leave LXC unhindered by firewalld):

firewall-cmd --permanent --zone=local-ignore --add-interface=lxcbr0 firewall-cmd --reload
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.