Issue in accessing cosmos db linux docker emulator over public IP, how to use custom server certificate?

  azure, azure-cosmosdb, docker, java, networking

I am using cosmos DB emulator docker on linux. Java Application can access it when cosmos emulator and application is on same machine(localhost), but application can not access the cosmos emulator when it is on different machine(accessing via hostname or IP) because emulator issue a self signed certificate only for CN=localhost, java application fail to verify hostname as certificate is issue to localhost and get following on java application.

java.security.cert.CertificateException: No subject alternative names matching IP address <<PUBLIC_IP>> found

I want to use my own self signed certificate in cosmos db emulator so that I can change CN=<customehostname>. By using custom certificate I can avoid importing certificate every time when container recreated and also application doesn’t have to override hostname verifier for HTTPS calls.

I am using docker compose to run emulator, I tried to use custom self signed by providing environment AZURE_COSMOS_EMULATOR_CERTIFICATE

version: '2.4' 
services:
  cosmosdb:
    container_name: "azurecosmosemulator"
    hostname: "azurecosmosemulator"
    image: 'mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator'
    tty: true
    mem_limit: 3GB
    ports:
        - '8081:8081'
        - '8900:8900'
        - '8901:8901'
        - '8902:8902'
        - '10250:10250'
        - '10251:10251'
        - '10252:10252'
        - '10253:10253'
        - '10254:10254'
        - '10255:10255'
        - '10256:10256'
        - '10350:10350'
    environment:
      AZURE_COSMOS_EMULATOR_PARTITION_COUNT: 5
      AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE: "true"      
      AZURE_COSMOS_EMULATOR_CERTIFICATE: "/home/user1/emulator/cosmos/appdata/self-signed-cosmos-cert.pfx"

    network_mode: "host"
    volumes:
     - '/home/user1/emulator/cosmos/appdata/:/tmp/cosmos/appdata/'

With the above option cosmos emulator fail to start without giving any information on console.

azurecosmosemulator | This is an evaluation version.  There are [141] days left in the evaluation period.
azurecosmosemulator | Shutting Down
azurecosmosemulator | Shut Down
azurecosmosemulator exited with code 255

I also tried AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE: <<PUBLIC_IP>>, with this I do not face any issue in SSLHandshake but application does get any data from emulator. On the other hand emulator shows following error in gateway.log

Connection: <not connected> -> rntbd://<<PUBLIC_IP>>:10251/
DocDBTrace Information: 0 : RNTBD: ConnectReuseAddrAsync binding local endpoint 0.0.0.0:0
DocDBTrace Information: 0 : RNTBD: ConnectReuseAddrAsync connecting to rntbd://<<PUBLIC_IP>>:10251/ (address <<PUBLIC_IP>>)
DocDBTrace Warning: 0 : RNTBD open timed out on channel <not connected> -> rntbd://<<PUBLIC_IP>>:10251/. Error: ConnectTimeout
DocDBTrace Warning: 0 : Channel.InitializeAsync failed. Channel: <not connected> -> rntbd://<<PUBLIC_IP>>:10251/. TransportException: Microsoft.Azure.Documents.TransportException: A client transport error occurred: The connection attempt timed out. (Time: 2021-07-23T03:07:48.3889645Z, activity ID: 7d1a352e-a1ac-4261-ac70-4fae32fd4146, error code: ConnectTimeout [0x0006], base error: HRESULT 0x80131500, URI: rntbd://<<PUBLIC_IP>>:10251/, connection: <not connected> -> rntbd://<<PUBLIC_IP>>:10251/, payload sent: False, CPU history: not available, CPU count: 4)
   at Microsoft.Azure.Documents.Rntbd.Channel.<InitializeAsync>d__28.MoveNext()
DocDBTrace Warning: 0 : Channel initialization failed. Consuming the task exception asynchronously. Server URI: rntbd://<<PUBLIC_IP>>:10251/. Exception: A client transport error occurred: The connection attempt timed out. (Time: 2021-07-23T03:07:48.3889645Z, activity ID: 7d1a352e-a1ac-4261-ac70-4fae32fd4146, error code: ConnectTimeout [0x0006], base error: HRESULT 0x80131500, URI: rntbd://<<PUBLIC_IP>>:10251/, connection: <not connected> -> rntbd://<<PUBLIC_IP>>:10251/, payload sent: False, CPU history: not available, CPU count: 4)
DocDBTrace Information: 0 : RequestAsync failed: RID: , Resource Type: DatabaseAccount, Op: (operationType: Read, resourceType: DatabaseAccount), Address: rntbd://<<PUBLIC_IP>>:10251/apps/DocDbApp/services/DocDbMaster0/partitions/780e44f4-38c8-11e6-8106-8cdcd42c33be/replicas/1p/, Exception: Microsoft.Azure.Documents.TransportException: A client transport error occurred: The connection attempt timed out. (Time: 2021-07-23T03:07:48.3889645Z, activity ID: 7d1a352e-a1ac-4261-ac70-4fae32fd4146, error code: ConnectTimeout [0x0006], base error: HRESULT 0x80131500, URI: rntbd://<<PUBLIC_IP>>:10251/, connection: <not connected> -> rntbd://<<PUBLIC_IP>>:10251/, payload sent: False, CPU history: (2021-07-23T03:04:10.1134303Z 100.000), (2021-07-23T03:04:20.1135838Z 100.000), CPU count: 4)
   at Microsoft.Azure.Documents.Rntbd.Channel.<InitializeAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.Rntbd.Channel.<<Initialize>b__14_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.Rntbd.Channel.<RequestAsync>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Azure.Documents.Rntbd.LoadBalancingPartition.<RequestAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.Rntbd.TransportClient.<InvokeStoreAsync>d__11.MoveNext()
DocDBTrace Information: 0 : Converting to Gone (read-only request)
DocDBTrace Information: 0 : RequestAsync failed: RID: , Resource Type: DatabaseAccount, Op: (operationType: Read, resourceType: DatabaseAccount), Address: rntbd://<<PUBLIC_IP>>:10251/apps/DocDbApp/services/DocDbMaster0/partitions/780e44f4-38c8-11e6-8106-8cdcd42c33be/replicas/1p/, Exception: Microsoft.Azure.Documents.TransportException: A client transport error occurred: The connection attempt timed out. (Time: 2021-07-23T03:07:48.3889645Z, activity ID: 7d1a352e-a1ac-4261-ac70-4fae32fd4146, error code: ConnectTimeout [0x0006], base error: HRESULT 0x80131500, URI: rntbd://<<PUBLIC_IP>>:10251/, connection: <not connected> -> rntbd://<<PUBLIC_IP>>:10251/, payload sent: False, CPU history: (2021-07-23T03:04:10.1134303Z 100.000), (2021-07-23T03:04:20.1135838Z 100.000), CPU count: 4)
   at Microsoft.Azure.Documents.Rntbd.Channel.<InitializeAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.Rntbd.Channel.<<Initialize>b__14_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.Rntbd.Channel.<RequestAsync>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Azure.Documents.Rntbd.LoadBalancingPartition.<RequestAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Documents.Rntbd.TransportClient.<InvokeStoreAsync>d__11.MoveNext()
DocDBTrace Information: 0 : Converting to Gone (read-only request)
DocDBTrace Information: 0 : Exception Microsoft.Azure.Documents.GoneException: Message: The requested resource is no longer available at the server.

Q1. Can I use my own self signed certificate instead of using generated by emulator on every start?

Q2. Is there any other way to solve this problem? I have already tried overriding hostname verifier, Unable to override that because I am using azure-spring-boot-starter-cosmos and that internally uses reactor netty HttpClient.

Q3. Is the cosmos simulator docker image is created in that way, It can only be used on local machin not over public IP?

Any help would be great. Thanks

Source: Docker Questions

LEAVE A COMMENT