Always installing vscode plugin in docker container doesnt work

I am using vscode with docker container. I have following entry in user settings.json.

"remote.containers.defaultExtensions": [
        "ms-python.python",
        "ms-azuretools.vscode-docker",
        "ryanluker.vscode-coverage-gutters"
    ]

But when I build or rebuild container, these plugins don’t get installed automatically inside container.
Am I doing something wrong ?

Source: StackOverflow