Traversal Iptables rules – Docker User Bridge

How can the docker-host reach an exported port without hitting the iptables DNAT-rule?

Bonus: Why are the iptables log messages only shown with curl 127.0.0.1:8082 but not with curl localhost:8082?

Setup

docker-compose.yml (mounts omitted)

services:
    webapp:
        image: webapp
        networks:
            - webapp_database
        ports:
            - "8082:80"

    database:
        networks:
            - webapp_database
        ports:
            - "3306"

networks:
    webapp_database:
        driver: bridge
        #internal: true <-- port mapping works (for testing)

$ docker network ls

NETWORK ID          NAME                     DRIVER              SCOPE
ebf4f0469f51        docker_webapp_database   bridge              local

Counters

*nat
Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8082 LOG flags 0 level 7 prefix "_nat_DOCKER_"
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0           
    0     0 RETURN     all  --  br-ebf4f0469f51 *       0.0.0.0/0            0.0.0.0/0           
    0     0 DNAT       tcp  --  !br-ebf4f0469f51 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:32768 to:172.18.0.2:3306
    0     0 DNAT       tcp  --  !br-ebf4f0469f51 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8082 to:172.18.0.3:80

*filter
Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 LOG flags 0 level 7 prefix "_filter_DOCKER_"
    0     0 ACCEPT     tcp  --  !br-ebf4f0469f51 br-ebf4f0469f51  0.0.0.0/0            172.18.0.2           tcp dpt:3306
    0     0 ACCEPT     tcp  --  !br-ebf4f0469f51 br-ebf4f0469f51  0.0.0.0/0            172.18.0.3           tcp dpt:80

Expected Behaviour

$ curl 127.0.0.1:8082

Although the nat-table is traversed only once per connection, the counters for the DNAT should not be zero; Traversal for first packet:

  1. Packet starts in OUTPUT chain
  2. Packet leaves unchanged (-> lo), arrives again (lo)
  3. Packet still unchanged: *natPREROUTING -jump-> *natDOCKER
  4. DNAT –to-destination 172.18.0.2:80
  5. Routing decision –> *filterFORWARD (destination is foreign host)
  6. *filterFORWARD -jump-> *filterDOCKER ang get accepted, further packets already accepted by conntrack target

Observed Behaviour

  1. Packet starts in OUTPUT chain
  2. Packet leaves (lo), arrives again (lo)
  3. Accepted by conntrack target in *filterINPUT chain

Active rules

:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [19:2601]
:POSTROUTING ACCEPT [19:2601]
:DOCKER - [0:0]
-A PREROUTING -p tcp -m tcp --dport 8082 -j LOG --log-prefix _nat_PREROUTING_ --log-level 7
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -p tcp -m tcp --dport 8082 -j LOG --log-prefix _nat_POSTROUTING_ --log-level 7
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.18.0.0/16 ! -o br-ebf4f0469f51 -j MASQUERADE
-A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 3306 -j MASQUERADE
-A POSTROUTING -s 172.18.0.3/32 -d 172.18.0.3/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A DOCKER -p tcp -m tcp --dport 8082 -j LOG --log-prefix _nat_DOCKER_ --log-level 7
-A DOCKER -i docker0 -j RETURN
-A DOCKER -i br-ebf4f0469f51 -j RETURN
-A DOCKER ! -i br-ebf4f0469f51 -p tcp -m tcp --dport 32768 -j DNAT --to-destination 172.18.0.2:3306
-A DOCKER ! -i br-ebf4f0469f51 -p tcp -m tcp --dport 8082 -j DNAT --to-destination 172.18.0.3:80
COMMIT
# Completed on Sat Feb 16 20:02:21 2019
# Generated by iptables-save v1.8.2 on Sat Feb 16 20:02:21 2019
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [152:41691]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A INPUT -p tcp -m tcp --dport 8082 -j LOG --log-prefix _filter_INPUT_ --log-level 7
-A INPUT -s 192.168.0.0/24 -p tcp -m tcp --dport 22000 -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
-A INPUT -i tun0 -p tcp -m tcp --dport 22000 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 80 -j LOG --log-prefix _filter_FORWARD_ --log-level 7
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o br-ebf4f0469f51 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-ebf4f0469f51 -j DOCKER
-A FORWARD -i br-ebf4f0469f51 ! -o br-ebf4f0469f51 -j ACCEPT
-A FORWARD -i br-ebf4f0469f51 -o br-ebf4f0469f51 -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 8082 -j LOG --log-prefix _filter_OUTPUT_ --log-level 7
-A OUTPUT ! -d 127.0.0.1/32 -p udp -m udp --dport 53 -j DROP
-A DOCKER -p tcp -m tcp --dport 80 -j LOG --log-prefix _filter_DOCKER_ --log-level 7
-A DOCKER -d 172.18.0.2/32 ! -i br-ebf4f0469f51 -o br-ebf4f0469f51 -p tcp -m tcp --dport 3306 -j ACCEPT
-A DOCKER -d 172.18.0.3/32 ! -i br-ebf4f0469f51 -o br-ebf4f0469f51 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-ebf4f0469f51 ! -o br-ebf4f0469f51 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-ebf4f0469f51 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT

Source: StackOverflow