A portable Datamart and Business Intelligence suite built with Docker, Mage, dbt, DuckDB and Superset
-
Updated
Nov 9, 2024 - Dockerfile
#
mage-ai
Here are 27 public repositories matching this topic...
This end-to-end data pipeline project processes and analyzes Maintenance Work Orders using Mage, Docker, Google BigQuery, MariaDB, and Looker Studio. It features a seamless integration of cloud and open-source tools for scalable data storage, transformation, and visualization.
python
bigquery
data
automation
cloud
sql
database
docker-compose
virtual-machine
excel
gcp
mariadb
orchestration
data-engineering
maintenance
database-schema
compute-engine
work-orders
visualization-dashboard
mage-ai
-
Updated
Nov 5, 2024 - Jupyter Notebook
In this project, I demonstrate creating a data pipeline from NYC Taxi data using Mage.ai for ETL, GCP for storage and BigQuery, and Looker Studio for analytics.
-
Updated
Oct 4, 2024
Project MLOps
flask
aws
streaming
monitoring
mongodb
model
grafana
orchestration
batch-processing
prefect
mlops
mlflow
mage-ai
-
Updated
Sep 28, 2024 - Jupyter Notebook
The CNPJ Data ETL Pipeline is designed to automate the download, processing, and storage of public CNPJ data from the Brazilian Federal Revenue. The pipeline is built with Mage.ai and AWS S3 to ensure efficient data management and scalability.
data-storage
data-engineering
cloud-computing
public-data
data-processing
data-pipeline
amazon-s3
etl-pipeline
script-automation
mage-ai
cnpj-data
-
Updated
Sep 1, 2024 - Python
mlops zoomcamp capstone project
-
Updated
Jun 25, 2024 - Jupyter Notebook
Personal project for Data Engineering Zoomcamp
-
Updated
Apr 13, 2024 - Jupyter Notebook
Data Engineering Zoomcamp 2024
python
docker
sql
apache-spark
etl
terraform
postgresql
jupyter-notebook
pipelines
pandas
ci-cd
data-warehouse
apache-kafka
dbt
google-cloud-platform
dlt
data-ingestion
etl-pipeline
pyarrow
mage-ai
-
Updated
Apr 8, 2024 - Jupyter Notebook
A data engineering project built around Smogon's Stats API.
-
Updated
Apr 2, 2024 - Python
Data Engineering on GCP and Mage AI
google-cloud-storage
fact-table
big-query
sales-analytics
dimensional-modeling
looker-studio
mage-ai
-
Updated
Mar 25, 2024 - Jupyter Notebook
This project showcases a data engineering pipeline using Google Cloud Platform (GCP) for analyzing taxi trips data from New York City. The pipeline includes steps such as data collection, storage setup, ETL processing, database creation, and data visualization.
python
bigquery
dashboard
google-cloud-storage
data-engineering
data-analysis
google-cloud-platform
data-modeling
google-compute-engine
etl-pipeline
google-looker-studio
mage-ai
-
Updated
Mar 5, 2024 - Jupyter Notebook
Data Engineering Pipeline for Uber Data Analysis using Google Cloud Platform, Mage-AI and Looker
-
Updated
Feb 23, 2024 - Jupyter Notebook
Solutions for @DataTalksClub's Data Engineering Zoomcamp 2024.
python
docker
workflow
data-science
database
etl
docker-compose
orchestration
datascience
data-engineering
data-analysis
etl-pipeline
de-zoomcamp
mage-ai
-
Updated
Mar 4, 2024 - Jupyter Notebook
Developed an interactive dashboard utilizing Google Looker Studio to analyze Uber usage patterns in NYC. Designed a robust data model and implemented an efficient data pipeline using Mage AI on GCP's VM infrastructure. Utilized Google BigQuery for in-depth analytics and leveraged Google Cloud Platform for data processing.
bigquery
google-cloud-storage
data-engineering
data-analytics
google-cloud-platform
etl-pipeline
data-modelling
gcp-vm
looker-studio
compute-instance
mage-ai
-
Updated
Feb 2, 2024 - Jupyter Notebook
An end to end Data Engineering Project
data-engineering
data-analytics
data-modeling
python-pandas
data-visualization-project
data-engineering-pipeline
gcp-project
looker-studio
mage-ai
-
Updated
Jan 23, 2024 - Jupyter Notebook
Engineered a robust data pipeline using Google Cloud Platform (GCP) services like Cloud Storage, Compute Engine, BigQuery, resulting in a 30% decrease in data processing and analysis time of the San Francisco Crime dataset.Designed an interactive dashboard using Looker Studio, empowering stakeholders to explore crime data visually.
-
Updated
Jan 20, 2024 - Jupyter Notebook
-
Updated
Jan 8, 2024 - Python
Improve this page
Add a description, image, and links to the mage-ai topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the mage-ai topic, visit your repo's landing page and select "manage topics."