Azure DevOps private nuget repo Unauthorized when building with docker

I am trying to create a docker image for my project that has a private Azure DevOps nuget repo. I am able to connect to the repo in visual studio with my credentials but when I try to build a docker image I get an error.

Unable to load the service index for source
https://{my_url}/nuget/v3/index.json Response status code does not
indicate success: 401 (Unauthorized)

I copied my Nuget.config file to the root of my project


<?xml version="1.0" encoding="utf-8"?>
    <add key="Microsoft and .NET" value="true" />
    <add key="" value="" />
    <add key="" value="" protocolVersion="3" />
    <add key="libraries" value="https://{my_url}/nuget/v3/index.json" />
      <add key="Username" value="{my_username}" />
      <add key="ClearTextPassword" value="{my_pass}" />

and this is my dockerfile

FROM AS base

FROM AS build
COPY NuGet.config "MyProject/"
COPY ["MyProject/MyProject.csproj", "MyProject/"]
COPY . .
RUN dotnet restore "MyProject/MyProject.csproj"
WORKDIR "/src/MyProject"
RUN dotnet build "MyProject.csproj" -c Release -o /app

FROM build AS publish
RUN dotnet publish "MyProject.csproj" -c Release -o /app

FROM base AS final
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "MyProject.dll"]

I am not sure what else to do for it to be able to connect to my repo

Source: StackOverflow