80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Jupyter magics and kernels for working with remote Spark clusters

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

High performance distributed data processing engine

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

Data Analysis Using Python: A Beginner’s Guide Featuring NYC Open Data

MCW Big data analytics and visualization

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

Cosmic Clone is a utility that can backup\clone\restore a azure Cosmos database Collection. It can also anonymize cosmos documents and helps hide personally identifiable data.

Apache Spark is a fast, in-memory data processing engine with elegant and expressive development API's to allow data workers to efficiently execute streaming, machine learning or SQL workloads that require fast iterative access to datasets.This project will have sample programs for Spark in Scala language .

Road to Azure Data Engineer Part-I: DP-200 - Implementing an Azure Data Solution

a high-performance, POSIX-ish Amazon S3 file system written in Go

An Azure Databricks workshop leveraging the New York Taxi and Limousine Commission Trip Records dataset

R interface to Azure storage accounts

Repository to store sample python programs for python learning

Albis: High-Performance File Format for Big Data Systems

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

SimpleSQLite is a Python library to simplify SQLite database operations: table creation, data insertion and get data as other data formats. Simple ORM functionality for SQLite.

Spark VCF data source implementation for Dataframes