PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows

An end-to-end data engineering pipeline that fetches data from Wikipedia, cleans and transforms it with Apache Airflow and saves it on Azure Data Lake. Other processing takes place on Azure Data Factory, Azure Synapse and Tableau.

Repository for Microsoft Databricks Training Events - Hosted by BlueGranite

[archived] A Python SDK for the Azure Databricks REST API 2.0

Free High-Quality Financial Data in Azure

Automated pipeline for energy consumption forecasting across Europe using Azure cloud and Databricks.

Reusable Python classes that extend open source PySpark capabilities. Examples of implementation is available under notebooks of repo https://github.com/bennyaustin/synapse-dataplatform

End-to-end ETL pipeline in the Microsoft Azure cloud - (Jun '24 - Jul '24)

A wrapper for the Azure Databricks REST API

ETL motor racing data project using Azure Databricks, Pyspark and Azure Date Lakes

Applying data engineering techniques to create data pipeline with Azure Cloud Computing

A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.

F1 Data Engineering Project on Azure Databricks!

A demand forecasting pipeline deployed on Azure and AWS

Article Repository for: Ensemble Machine Learning Modeling for the Prediction of Artemisinin Resistance in Malaria

In this project, I've created an end-to-end ETL pipeline and subsequently developed a machine learning model to predict the price of Amazon products based on several product-related features.

A repository to continue my education on Azure Data Services.

Customer churn prediction using Azure Databricks and Apache Spark, covering data preprocessing, model training, evaluation, and deployment.

This repository contains code for an end-to-end IoT data pipeline using Azure services. It ingests, processes, and stores IoT device data from AWS S3 to Azure Data Lake Storage and Azure SQL Database, leveraging Azure Data Factory and Azure Functions for seamless integration and automation.

"Explore Formula 1 data analytics with this project. Leveraging the Ergast API, it utilizes Databricks Spark for ingestion, transformation, and analysis. ADLS acts as the storage layer, while Power BI visualizes the ADLS presentation layer. Uncover insights in the world of Formula 1 through powerful data analytics."