Azure DevOps Pipelines push Docker image to Container Registry Denied

I am currently trying to use Azure Pipelines to build a Docker image and push it to the Azure Container Registry. I have a Service Connection setup, and but the build is failing with “denied.” I suspect the reason for this is because my Container Registry is setup to only allow from “selected networks” and is restricted to a few IPs. I validated this by temporarily allowing all networks, and then the build/push succeeded.

Is there any way to get Azure Pipelines to successfully push a Docker image to the Container Registry that is only allowing selected networks? I thought that was what the Service Connection was for?

Selected Networks image

Source: StackOverflow