421 Open source projects that are alternatives of or similar to MLBD

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

Big Data for Data Engineers Coursera Specialization from Yandex

大数据入门指南 ⭐

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Asakusa Framework

🍉基于Hadoop和HBase的大规模海量数据去重

DevOps pipeline for Real Time Social/Web Mining

An open-source, real-time Security Information & Event Management tool based on big data technologies, providing a scalable, advanced security analytics framework.

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Distributed stream processing model and its implementation

A distributed, fast open-source graph database featuring horizontal scalability and high availability

Service for automatically managing and cleaning up unreferenced data

Mirror of Apache Airavata Django Portal

Win probability predictions for League of Legends matches using neural networks

大数据生态圈学习

Mirror of Apache Airavata PHP Gateway

Ppts, códigos y videos de las meetups, data science days, videollamadas y workshops. Data Science Research es una organización sin fines de lucro que busca difundir, descentralizar y difundir los conocimientos en Ciencia de Datos e Inteligencia Artificial en el Perú, dando oportunidades a nuevos talentos mediante MeetUps, Workshops y Semilleros …

Data pipeline project

InsightEdge Core

🔥 Latest computer science e-books。提供最新技术类电子书下载， “我无非就是想卷死各位，或者被各位卷死！”

Deploy a secured, clustered, auto-scaling NiFi service in AWS.

Automile offers a simple, smart, cutting-edge telematics solution for businesses to track and manage their business vehicles.

Bulletproof Apache Spark jobs with fast root cause analysis of failures.

Information Inference Service of the OpenAIRE system

Families In the WIld: A Kinship Recogntion Toolbox.

A distributed in-memory key-value storage for billions of small objects.

A distributed block-based data storage and compute engine

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

Apache Arrow DataFusion SQL Query Engine

Matlab library for gradient descent algorithms: Version 1.0.1

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

A crawler and API server for Liquor Control Board of Ontario retail data

TalariaDB is a distributed, highly available, and low latency time-series database for Presto

Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.

rastercube is a python library for big data analysis of georeferenced time series data (e.g. MODIS NDVI)

Automile offers a simple, smart, cutting-edge telematics solution for businesses to track and manage their business vehicles.

MATLAB/Octave library for stochastic optimization algorithms: Version 1.0.20

High level linear algebra library for Dlang

A boilerplate project for Azure Big Data PaaS services

swissgeol.ch gives you insight in geoscientific data - above and below the surface.

基于Vue、three.js、echarts，数据可视化展示项目，包含三维模型导入交互、三维模型标注等功能

Scalable RNA-seq analysis

Apache Beam Site

RECSM-UPF Summer School: Social Media and Big Data Research

Event-Triggered Communication in Parallel Machine Learning

PredictionIO E-Commerce Recommendation Engine Template (Scala-based parallelized engine)

Fast and easy parallel mapreduce on HPC clusters

A High-Performance Data Science Toolkit for the Earth Sciences

a one-stop repo to lookup for code snippets of core java concepts, sql, data structures as well as big data. It also consists of interview questions asked in real-life.

Big Data Visualization

Toolkit for highly memory efficient analysis of single-cell RNA-Seq, scATAC-Seq and CITE-Seq data. Analyze atlas scale datasets with millions of cells on laptop.

PredictionIO Java SDK

A privacy-focused list of alternatives to mainstream services to help the competition.

Celeborn provides an elastic and high-performance service for shuffle and spilled data.

CSCE 585 - Machine Learning Systems

Erlang Parse Transforms Including Fold (MapReduce) comprehension, Elixir-like Pipeline, and default function arguments

Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.

This project aim is to provide free software to fetch data from plcs (Siemens S7-300/400/1200/1500) and store it. Used stack is completly opensource. I used InfluDB as data storage, so application principle is following Big Data paradigm.

🦄🌟 Graph Database layer on top of Google Bigtable