I’m running airflow on a google VM server using docker and I have a bunch of tasks I want to run.
I have my scripts inside the dag folder and even if I’m using an absolute path to read a csv file (os.path.dirname(os.path.realpath(file))) I get : FileNotFoundError: [Errno 2] No such file or directory: ‘/opt/airflow/dags/bearish_abandonedbaby/updates3.csv’.
What is strange also is that I can’t see an airflow folder inside /opt/ on the server but I can see it inside the container, and inside the container the file is there.
Please let me know what am doing wrong.?
Source: Docker Questions