Azure DevOps private nuget repo Unauthorized when building with docker

I am trying to create a docker image for my project that has a private Azure DevOps nuget repo. I am able to connect to the repo in visual studio with my credentials but when I try to build a docker image I get an error.

Unable to load the service index for source
https://{my_url}/nuget/v3/index.json Response status code does not
indicate success: 401 (Unauthorized)

I copied my Nuget.config file to the root of my project

nuget.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <disabledPackageSources>
    <add key="Microsoft and .NET" value="true" />
  </disabledPackageSources>
  <activePackageSource>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
  </activePackageSource>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="libraries" value="https://{my_url}/nuget/v3/index.json" />
  </packageSources>
  <packageSourceCredentials>
    <libraries>
      <add key="Username" value="{my_username}" />
      <add key="ClearTextPassword" value="{my_pass}" />
    </libraries>
  </packageSourceCredentials>
</configuration>

and this is my dockerfile

FROM mcr.microsoft.com/dotnet/core/aspnet:3.0.0-preview8-nanoserver-1903 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/core/sdk:3.0.100-preview8-nanoserver-1903 AS build
WORKDIR /src
COPY NuGet.config "MyProject/"
COPY ["MyProject/MyProject.csproj", "MyProject/"]
COPY . .
RUN dotnet restore "MyProject/MyProject.csproj"
WORKDIR "/src/MyProject"
RUN dotnet build "MyProject.csproj" -c Release -o /app

FROM build AS publish
RUN dotnet publish "MyProject.csproj" -c Release -o /app

FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "MyProject.dll"]

I am not sure what else to do for it to be able to connect to my repo

Source: StackOverflow