BigData Pipeline is a local testing environment for experimenting with various storage solutions (RDB, HDFS), query engines (Trino), schedulers (Airflow), and ETL/ELT tools (DBT). It supports MySQL, Hadoop, Hive, Kudu, and more.
mysql docker airflow big-data hive hadoop docker-compose kudu distributed-computing data-warehouse mysql-server data-lake dbt airflow-docker trino hiveql airflow-dags dbt-core trino-cli data-lakestore
-
Updated
Nov 2, 2024 - Dockerfile