Category : dbt

Problem statement I have the following Dockerfile: ARG BASE_IMAGE=python:3.9-slim FROM $BASE_IMAGE as base RUN apt update && apt install –yes –no-install-recommends build-essential libsasl2-dev && apt clean RUN python -m venv /opt/venv ENV PATH="/opt/venv/bin:$PATH" RUN pip install –upgrade –no-cache-dir pip && pip install –no-cache-dir dbt-spark[PyHive]==0.20.1 FROM $BASE_IMAGE as runtime RUN apt update && apt install –yes ..

Read more

We’re running DBT in Airflow on a GCP Compute Engine using Docker and docker-compose. In the docker-compose.yml file for our Airflow deployment, the DBT repo is attached as a volume: webserver: … volumes: – ../our-dbt-repo:/usr/local/airflow/dbt Running dbt-run usually generates a /logs directory with DBT logs. However, running dbt-run from the docker container on the GCP ..

Read more