ETL pipeline tailored for Olympics data
-
Updated
Jun 2, 2024 - Python
ETL pipeline tailored for Olympics data
Databricks ETL Pipeline for retrieving and processing NI TestStand test results, featuring a well-documented notebook for ETL operations, Data Lake for storage, Spark SQL+Python for transformations, and Power BI as the final visualization of factory metrics.
A polycloud .NET cloud storage abstraction layer. Provides Blob storage (AWS S3, GCP, FTP, SFTP, Azure Blob/File/Event Hub/Data Lake) and Messaging (AWS SQS, Azure Queue/ServiceBus). Supports .NET 5+ and .NET Standard 2.0+. Pure C#.
An application developed to give real-time insights on machine health using Iot sensors by tracking and monitoring parameters such as temperature, pressure, current and humidity.
A real-time application to guide cab drivers looking for ride towards the areas of the cities experiencing higher demand
This repository contains code for an end-to-end IoT data pipeline using Azure services. It ingests, processes, and stores IoT device data from AWS S3 to Azure Data Lake Storage and Azure SQL Database, leveraging Azure Data Factory and Azure Functions for seamless integration and automation.
Created a movie recommendation system on Azure utilizing Spark SQL by analyzing the MovieLens dataset.
Collection of Databricks and Jupyter Notebooks
Data Engineering & Software Blog
We have dataset of IPL from 2008 to 2020 and we have to visualize analytics on Power BI dashboard. We have to upload that dataset into data lake. After that we have to process that data through pipeline and produce modeled data in warehouse. So, that we will be able to analyze the data in Power BI through pre-defined dashboards.
a high-performance, POSIX-ish Amazon S3 file system written in Go
A list of samples for integration of a .NET application with various Azure cloud services.
An Akka Streams source of Azure Data Lake data
POC projects working on Cloud Platforms
An E2E solution of the Data Resources on Azure using the Snapshot Serengeti dataset. This E2E solution focuses Azure Synapse Analytics, Power Bi & the Azure Data Factory.
A comprehensive guide to understanding and implementing data management and analytics solutions in the Azure ecosystem using Azure Data Fundamentals.
Fluentd output plugin for Azure Datalake Storage Gen2 (append support)
Add a description, image, and links to the azure-data-lake topic page so that developers can more easily learn about it.
To associate your repository with the azure-data-lake topic, visit your repo's landing page and select "manage topics."