Kubeflow: Can’t import from modules in docker image

Published

I’ve set up a kubeflow pipeline where I have an image and a function importing a module within the image:

import kfp
from kfp.components import func_to_container_op, InputPath, OutputPath

def process_data(experiment_name):
    import os

    print("Listing dir: ", os.system("ls"))
    import driver
    

# decorator with an arg
process_data = func_to_container_op(process_data, base_image=image)


@kfp.dsl.pipeline(name="", description="")
def pipeline(experiment_name):
    process_data(experiment_name)


if __name__ == "__main__":
    kfp.compiler.Compiler().compile(pipeline, __file__.replace(".py", ".yaml"))

However, I’m getting the error

ModuleNotFoundError: No module named 'driver'

even though driver.py exists in the image.
Any ideas on what’s wrong with my setup?

Source: Docker Questions

Published
Categorised as docker, import, kubeflow, kubeflow-pipelines, python Tagged , , , ,

Answers

Leave a Reply

Still Have Questions?


Our dedicated development team is here for you!

We can help you find answers to your question for as low as 5$.

Contact Us
faq