Dockerized Pyspark app Unable to clear output directory file error

  docker, docker-volume, pyspark

I wrote a pyspark app that reads stream from kafka and write as csv file with overwrite method. Code works regularly when host machine withıut dockerized. But when I dockerized pyspark app, docker gives below logs :

java.io.IOException: Unable to clear output directory file:/app/files prior to writing to it

Note : I mount /app/files with my host machine in docker-compose.yaml

Source: Docker Questions

LEAVE A COMMENT