So my workflow is gonna be a bit wonky, but I’m not permitted to use Docker, so I have Singularity instead.
I’m running some code that is giving me this error:
RuntimeError: nvrtc: error: failed to open libnvrtc-builtins.so.11.1. Make sure that libnvrtc-builtins.so.11.1 is installed correctly. nvrtc compilation failed:
If more details are needed, I can provide.
I’m operating in a Singularity container built from a docker_hub image I created here: https://hub.docker.com/repository/docker/elavizadeh/imaginaire_cuda_10.2/builds
(Ignore the fact that the title says cuda_10.2, I’ll fix this all up if and when this works.)
Which is based on this dockerfile I created here: https://github.com/EvanLavizadeh/imaginaire-1/blob/master/Dockerfile
In that Dockerfile, I specify a base image "FROM nvcr.io/nvidia/pytorch:20.12-py3"
I specifically chose that version of that base image because it claims it has Cuda toolkit 11.1.1 which I am hoping will contain
But when I spin up my container, cuda reports version 11.0.221:
Singularity> nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Wed_Jul_22_19:09:09_PDT_2020 Cuda compilation tools, release 11.0, V11.0.221 Build cuda_11.0_bu.TC445_37.28845127_0
Additionally, when I look at the files in
/usr/local/cuda/lib64, I see
libnvrtc-builtins.so.11.0 but not
I believe this is the source of my error but I don’t know how to correct it. I either need my code to use
libnvrtc-builtins.so.11.0 instead of
libnvrtc-builtins.so.11.1 or I need my container to build Cuda 11.1 and hopefully generate
Source: Docker Questions