docker-compose v3.1 elasticsearch v6.6.1 cluster ‘slave’ does not join the master

I have a docker-compose setup for elasticsearch on a local host (both containers on this my machine), with master, slave so so say. Idea is that the ‘slave’ joins the ‘master’. Both containers are running (for now) on one host, my local machine. Problem: the slave does not join the master. (of course, it is the plan to go on production on several machines)

Please see the following part of the docker-compose file, that contains the 2 elasticsearch services (other FE service taken out here). Idea is, to have the ES as an index for the FE, which can be scaled by:

docker-compose up scale elastic_slave=3

Result: all 4 containers are started and running, according to:
docker ps

Problem: however, when accessing the health of the cluster via

… _cluster/health?pretty

the “master” shows only:

“number_of_nodes” : 1,
“number_of_data_nodes” : 1,

Well – we could not find a running example for these versions: d-c 3.1, and ES 6.6.1. So, the ideas for these settings:

    depends_on:
      - elastic
    environment: 
      - cluster.name=docker-cluster
      - "discovery.zen.ping.unicast.hosts=elastic"

were taken from examples for d-c v2.

I tried to “leave out” one of these lines: no “depends_on”, no “cluster.name”, not “discover.zen …”
.. without results.

Actually, why they seems not to “see each other” automatically when they run on the same internal network “elk”?

What we also tried, was, to stop the running slaves, and then, to repeat docker-compose up, thought:
maybe the master must be setup totally running for that the slaves can see it. – the same result: all containers are running but not connected as a cluster.

  • is the scenario as thought about (scaling the slaves for an index), via docker-compose, possible at all – with these current versions?

  • or: which versions might be possible to use?

  • (well, we are not interested to migrate to ES 7 right now, it was difficult enough to get it to 6.6.1)

  • is there another ‘mechanism’, not docker-compose, that can make such scenario run, not sure about ‘swarm’ (deprecated?)

Thank you very much and please apologize if the solution is somewhere there, we have searched a lot however req are: d-c version > 2 (for 2, there are a lot of examples that might work or not, we haven’t tried)

version: '3.1'

services:
  elastic:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.6.1
    container_name: elastic
    networks:    
      - elk
    ports:
      - "9200"
      - "9300"
    expose:
      - "9300"
      - "9200"
    environment:
      - cluster.name=docker-cluster
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
      - "http.host=0.0.0.0"
      - "transport.host=127.0.0.1"
      - "xpack.security.enabled=false"
    ulimits:
      memlock:
        soft: -1
        hard: -1

    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9200"]
      interval: 10s
      timeout: 3s
      retries: 1

  elastic_slave:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.6.1
    networks:
      - elk
    depends_on:
      - elastic
    environment: 
      - cluster.name=docker-cluster
      - "discovery.zen.ping.unicast.hosts=elastic"

networks:
   elk:
    driver: bridge

please see part of the logs, of the slave. It says “0 nodes joined”, starting as a master. Whether the cache issue is important, i do not know, so show it also:

elastic_slave_1  | [2019-06-24T11:58:11,055][INFO ][o.e.t.TransportService   ] [yhWiQ_5] publish_address {172.21.0.3:9300}, bound_addresses {0.0.0.0:9300}
elastic_slave_1  | [2019-06-24T11:58:11,084][INFO ][o.e.b.BootstrapChecks    ] [yhWiQ_5] bound or publishing to a non-loopback address, enforcing bootstrap checks
elastic_slave_1  | [2019-06-24T11:58:14,187][INFO ][o.e.c.s.MasterService    ] [yhWiQ_5] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {yhWiQ_5}{yhWiQ_5QQ1iUm0blGbkatg}{6DOMrIVVSuiIqQdFgMLDOg}{172.21.0.3}{172.21.0.3:9300}{ml.machine_memory=7741763584, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
elastic_slave_1  | [2019-06-24T11:58:14,193][INFO ][o.e.c.s.ClusterApplierService] [yhWiQ_5] new_master {yhWiQ_5}{yhWiQ_5QQ1iUm0blGbkatg}{6DOMrIVVSuiIqQdFgMLDOg}{172.21.0.3}{172.21.0.3:9300}{ml.machine_memory=7741763584, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {yhWiQ_5}{yhWiQ_5QQ1iUm0blGbkatg}{6DOMrIVVSuiIqQdFgMLDOg}{172.21.0.3}{172.21.0.3:9300}{ml.machine_memory=7741763584, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
elastic_slave_1  | [2019-06-24T11:58:14,273][INFO ][o.e.h.n.Netty4HttpServerTransport] [yhWiQ_5] publish_address {172.21.0.3:9200}, bound_addresses {0.0.0.0:9200}
elastic_slave_1  | [2019-06-24T11:58:14,273][INFO ][o.e.n.Node               ] [yhWiQ_5] started
elastic_slave_1  | [2019-06-24T11:58:14,296][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [yhWiQ_5] Failed to clear cache for realms [[]]
elastic_slave_1  | [2019-06-24T11:58:14,380][INFO ][o.e.g.GatewayService     ] [yhWiQ_5] recovered [0] indices into cluster_state

Source: StackOverflow