2024 Open source projects that are alternatives of or similar to Geni

Fundamentals of Spark with Python (using PySpark), code examples

Koalas: pandas API on Apache Spark

The Accelerator is a tool for fast and reproducible processing of large amounts of data.

A simple Spark-powered ETL framework that just works 🍺

Feature Store for Machine Learning

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

RSparkling: Use H2O Sparkling Water from R (Spark + R + Machine Learning)

Fast and easy parallel mapreduce on HPC clusters

📊 📋 Dashboards using YAML or JSON files

Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive

A simplified, lightweight ETL Framework based on Apache Spark

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Example project implementing best practices for PySpark ETL jobs and applications.

VerticaPy is a Python library that exposes sci-kit like functionality to conduct data science projects on data stored in Vertica, thus taking advantage Vertica’s speed and built-in analytics and machine learning capabilities.

🎓 Jupyter notebooks from UFC data science course

Julia binding for Apache Spark

Tools and Docs on the Azure Data Science Virtual Machine (http://aka.ms/dsvm)

Apache Spark 官方文档中文版

Teaching materials in python at the @ENSAE

📢 Ready to learn or review your knowledge!

Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

Example end to end data engineering project.

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

Scalable stream processing platform for advanced realtime analytics on top of Kafka and Spark. LogIsland also supports MQTT and Kafka Streams (Flink being in the roadmap). The platform does complex event processing and is suitable for time series analysis. A large set of valuable ready to use processors, data sources and sinks are available.

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

Parallel Workflow extension for Rake, runs on multicores, clusters, clouds.

Research on distributed system

danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.

大数据入门指南 ⭐

A purely functional library to build distributed and event-driven systems

Word2Vec models with Twitter data using Spark. Blog:

Big Data for Data Engineers Coursera Specialization from Yandex

A Spark cluster setup running on Docker containers

Apache Spark Website

An R-focused pipeline toolkit for reproducibility and high-performance computing

The Data Engineering Cookbook

Open-source software for volunteer computing and grid computing.

Explore high-dimensional datasets and how your algo handles specific regions.

Example workflows for the drake R package

Two-day workshop that covers how to use R to interact databases and Spark

R on Apache Spark (SparkR) tutorials for Big Data analysis and Machine Learning as IPython / Jupyter notebooks

Distributed Deep learning with Keras & Spark

Pandas and Spark DataFrame comparison for humans

🐍 Quick reference guide to common patterns & functions in PySpark.

Data science and Big Data with Python

repo for code published on pythondata.com

A suite of benchmarks to test the sequential CPU and GPU performance of most popular high-performance libraries for Python.

Accelerate data science

A parallel application binary interface for Fortran 2018 compilers.

Apache Superset is a Data Visualization and Data Exploration Platform

Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

Collaborate on privacy-preserving policy for data science projects in Pandas and Apache Spark

Tools for computation on batch systems

Griffon Data Science Virtual Machine

PipelineX: Python package to build ML pipelines for experimentation with Kedro, MLflow, and more

A large-scale entity and relation database supporting aggregation of properties