Docker + AWS ElasticBeanstalk: "host.docker.internal: host-gateway": Connection refused

Related question: What is linux equivalent of "host.docker.internal" (the answer is not working here, hence this question)

So I have a multicontainer project running on AWS ElasticBeanstalk (latest Docker platform).

The containers share the same bridge network since they need to be able to communicate with each other (ports 3000, 8080, 9090, 9100). But one of those containers needs to be able to communicate with the host too (ports 55113, 56113).

This is normally possible by just adding extra_hosts: "host.docker.internal:host-gateway" to that container and then using host.docker.internal instead of localhost inside the container, i.e. host.docker.internal:55113.

But I just can’t figure out why this is not working on AWS ElasticBeanstalk (as you can see below).

Am I doing something wrong? Or is there any workaround?

AWS ElasticBeanstalk Platform:

Docker running on 64bit Amazon Linux 2/3.4.2

Host’s Docker & Docker-Compose versions:

$ sudo docker --version

Docker version 20.10.4, build d3cb89e

$ sudo docker-compose --version

docker-compose version 1.29.2, build unknown

Project’s docker-compose.yml: (removing all the other services and irrelevant settings)

version: "3.9"

services:
  prometheus:
    image: prom/prometheus:v2.28.1
    container_name: prometheus
    ports:
      - 9090:9090
    extra_hosts:
      - host.docker.internal:host-gateway # <-------- EXTRA HOST

networks:
  default:
    name: monitoring

Container debug:

$ sudo docker exec -it prometheus cat /etc/hosts

127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.1  host.docker.internal # <-------- the mapping is present
172.25.0.4  f01ffc6b2a8a

$ sudo docker exec -it prometheus wget host.docker.internal

Connecting to host.docker.internal (172.17.0.1:80)
wget: can't connect to remote host (172.17.0.1): Connection refused

$ sudo docker exec -it prometheus wget host.docker.internal:55113  # <-------- the port I want to reach

Connecting to host.docker.internal:56113 (172.17.0.1:56113)
wget: can't connect to remote host (172.17.0.1): Connection refused

Host’s iptables:

$ sudo iptables -S

-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o br-a8a58f19c627 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-a8a58f19c627 -j DOCKER
-A FORWARD -i br-a8a58f19c627 ! -o br-a8a58f19c627 -j ACCEPT
-A FORWARD -i br-a8a58f19c627 -o br-a8a58f19c627 -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.25.0.2/32 ! -i br-a8a58f19c627 -o br-a8a58f19c627 -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER -d 172.25.0.3/32 ! -i br-a8a58f19c627 -o br-a8a58f19c627 -p tcp -m tcp --dport 9100 -j ACCEPT
-A DOCKER -d 172.25.0.4/32 ! -i br-a8a58f19c627 -o br-a8a58f19c627 -p tcp -m tcp --dport 9090 -j ACCEPT
-A DOCKER -d 172.25.0.5/32 ! -i br-a8a58f19c627 -o br-a8a58f19c627 -p tcp -m tcp --dport 3000 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-a8a58f19c627 ! -o br-a8a58f19c627 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o br-a8a58f19c627 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN

Host’s open ports:

$ netstat -tuplen

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name
tcp        0      0 0.0.0.0:9090            0.0.0.0:*               LISTEN      0          200315     -
tcp        0      0 127.0.0.1:55113         0.0.0.0:*               LISTEN      1000       119886     -
tcp        0      0 0.0.0.0:9100            0.0.0.0:*               LISTEN      0          200280     -
tcp        0      0 127.0.0.1:22221         0.0.0.0:*               LISTEN      1001       197452     -
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      0          14121      -
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      0          201081     -
tcp        0      0 127.0.0.1:56113         0.0.0.0:*               LISTEN      1000       119890     -
tcp        0      0 127.0.0.1:38581         0.0.0.0:*               LISTEN      0          20325      -
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          18585      -
tcp        0      0 0.0.0.0:3000            0.0.0.0:*               LISTEN      0          201164     -
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      0          16235      -
tcp6       0      0 ::1:55113               :::*                    LISTEN      1000       119885     -
tcp6       0      0 :::111                  :::*                    LISTEN      0          14124      -
tcp6       0      0 ::1:56113               :::*                    LISTEN      1000       119889     -
tcp6       0      0 :::22                   :::*                    LISTEN      0          18594      -
udp        0      0 0.0.0.0:68              0.0.0.0:*                           0          15762      -
udp        0      0 0.0.0.0:111             0.0.0.0:*                           0          14119      -
udp        0      0 127.0.0.1:323           0.0.0.0:*                           0          14990      -
udp        0      0 0.0.0.0:979             0.0.0.0:*                           0          14120      -
udp6       0      0 fe80::ed:19ff:fe84::546 :::*                                0          16539      -
udp6       0      0 :::111                  :::*                                0          14122      -
udp6       0      0 ::1:323                 :::*                                0          14991      -
udp6       0      0 :::979                  :::*                                0          14123      -

Source: Docker Questions

LEAVE A COMMENT