Toolkit for highly memory efficient analysis of single-cell RNA-Seq, scATAC-Seq and CITE-Seq data. Analyze atlas scale datasets with millions of cells on laptop.

RemoteShuffleService

Celeborn provides an elastic and high-performance service for shuffle and spilled data.

✭ 262

java scala shell big-data spark shuffle

IoT-system-PLC-data-to-InfluxDB

This project aim is to provide free software to fetch data from plcs (Siemens S7-300/400/1200/1500) and store it. Used stack is completly opensource. I used InfluDB as data storage, so application principle is following Big Data paradigm.

✭ 26

python iot big-data influxdb free plc snap7

terraform-aws-kinesis-firehose

This code creates a Kinesis Firehose in AWS to send CloudWatch log data to S3.

✭ 25

HCL python big-data analytics terraform kinesis-firehose cloudwatch-logs parquet terraform-provider etl-job terraform-aws big-data-processing

spark-root

Apache Spark Data Source for ROOT File Format

✭ 28

Jupyter Notebook scala big-data spark root histogrammar

dxram

A distributed in-memory key-value storage for billions of small objects.

✭ 25

java shell memcached big-data social-network distributed in-memory graph-processing low-latency in-memory-storage key-value-database key-value-store storage-nodes

nebula

A distributed, fast open-source graph database featuring horizontal scalability and high availability

✭ 8,196

C++Gherkin python CMake Yacc Thrift distributed-systems database big-data graph raft scalability distributed graph-database graphdb nebula nebula-graph nebulagraph

img2dataset

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

✭ 1,173

python Jupyter Notebook shell image big-data deep-learning dataset image-dataset download-images multimodal

Real Time Social Media Mining

DevOps pipeline for Real Time Social/Web Mining

✭ 22

HTML python java shell Dockerfile github jenkins devops crawler big-data social-network actions hdfs kdd social-network-analysis github-actions

GDLibrary

Matlab library for gradient descent algorithms: Version 1.0.1

✭ 50

matlab machine-learning big-data newton optimization svm linear-regression machine-learning-algorithms statistical-learning classification logistic-regression gradient gradient-descent softmax-regression optimization-algorithms matrix-completion multinomial-regression rosenbrock-problem

airavata-django-portal

Mirror of Apache Airavata Django Portal

✭ 20

python Vue javascript HTML SCSS Dockerfile cloud big-data network-server airavata

lcbo-api

A crawler and API server for Liquor Control Board of Ontario retail data

✭ 152

javascript HTML ruby CSS api docker redis big-data postgresql api-server learn-to-code ruby-on-rails learning-by-doing lcbo

datalake-etl-pipeline

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

✭ 39

python big-data spark apache-spark hadoop etl xml xml-parsing pyspark data-pipeline datalake hadoop-mapreduce spark-sql etl-framework hadoop-hdfs etl-pipeline etl-components

gan deeplearning4j

Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.

✭ 19

java Jupyter Notebook data-science machine-learning big-data spark apache-spark computer-vision deep-learning bigdata datascience generative-adversarial-network gan machinelearning deeplearning generative-adversarial-networks deeplearning4j

automile-php

Automile offers a simple, smart, cutting-edge telematics solution for businesses to track and manage their business vehicles.

FlameStream

Distributed stream processing model and its implementation

✭ 14

java HCL Jinja Puppet python shell ruby streaming big-data graph

lubeck

High level linear algebra library for Dlang

✭ 57

d Meson Makefile big-data high-performance numpy linear-algebra octave blas quantitative-finance numerical-methods ndslice native-code hedgefund symmetry-investments

ngm

swissgeol.ch gives you insight in geoscientific data - above and below the surface.

✭ 23

typescript javascript CSS rust HTML shell big-data globe geology cesiumjs 3d subsurface

nifi

Deploy a secured, clustered, auto-scaling NiFi service in AWS.

✭ 37

HCL shell python aws ansible automation cloud big-data kms ec2 pipeline ubuntu terraform s3 iam apache zookeeper rhel iac nifi apache-nifi

automile-net

Automile offers a simple, smart, cutting-edge telematics solution for businesses to track and manage their business vehicles.

big-data-upf

RECSM-UPF Summer School: Social Media and Big Data Research

✭ 21

HTML r social-media facebook twitter big-data rstudio text-analysis social-network-analysis scraping-websites

iis

Information Inference Service of the OpenAIRE system

✭ 16

java HTML PigLatin python scala shell HiveQL text-mining data-mining big-data spark hadoop iis openaire information-inference data-processing-system

predictionio-template-ecom-recommender

PredictionIO E-Commerce Recommendation Engine Template (Scala-based parallelized engine)

✭ 73

scala python big-data predictionio

FIW KRT

Families In the WIld: A Kinship Recogntion Toolbox.

✭ 18

python matlab big-data computer-vision deep-learning notebook pandas facial-recognition

predictionio-sdk-java

PredictionIO Java SDK

✭ 107

java big-data predictionio

shifting

A privacy-focused list of alternatives to mainstream services to help the competition.

✭ 31

microsoft open-source search-engine security services list youtube google privacy facebook browser big-data gmail data-collection privacy-enhancing-technologies privacy-protection privacy-tools

yildiz

🦄🌟 Graph Database layer on top of Google Bigtable

✭ 24

typescript javascript nodejs database big-data graph edges graph-database nodes bigtable hyper-relational yildiz event-relation

HadoopDedup

🍉基于Hadoop和HBase的大规模海量数据去重

✭ 27

java big-data mapreduce cdc dedup fsp

data-viz-utils

Functions for easily making publication-quality figures with matplotlib.

✭ 16

Jupyter Notebook python legend data-science big-data time-series eda data-visualization data-analysis matplotlib scatterplot

hazelcast-csharp-client

Hazelcast .NET Client

✭ 98

C#powershell caching big-data clustering hazelcast scalability distributed in-memory datagrid netframework hazelcast-client imdg netcore2 clscompilant

pyspark-algorithms

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

✭ 72

python shell data-science data big-data spark algorithms nosql transformations distributed-computing pyspark monoid mapreduce partitioning dataframe rdd graphframes data-abstractions pyspark-algorithms

lidbox

End-to-end spoken language identification out of the box.

✭ 39

python big-data deep-learning tensorflow speech audio-analysis language-recognition language-identification spoken-language-recognition spoken-language-identification

big-data-engineering-indonesia

A curated list of big data engineering tools, resources and communities.

✭ 26

big-data data-engineering

awesome-tools

curated list of awesome tools and libraries for specific domains

✭ 31

data-science streaming big-data apache-spark

merkle-db

High-scalability analytics database built on immutable merkle-trees

✭ 44

clojure HCL database big-data nosql merkle-dag

javaer-mind

Java 程序员进阶学习的思维导图

✭ 66

mysql distributed-systems big-data nosql mindmap

metriql

The metrics layer for your data. Join us at https://metriql.com/slack

✭ 227

kotlin java Jinja Vue javascript SCSS big-data analytics olap dbt

leetspeek

Open and collaborative content from leet hackers!

✭ 11

CSS HTML ruby javascript computer-science security devops cloud big-data ai ops ml dev pentest

dislib

The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.

✭ 39

python Jupyter Notebook machine-learning big-data hpc distributed-computing

awesome-coder-resources

编程路上加油站！------【持续更新中...欢迎star,欢迎常回来看看......】【内容：编程/学习/阅读资源，开源项目,面试题,网站,书,博客,教程等等】

✭ 54

awesome books big-data frontend backend bigdata resources interview awesome-list progamming

phoenix-queryserver

Apache Phoenix Query Server

✭ 33

python java shell Makefile Dockerfile phoenix sql database big-data

couchdb-pkg

Apache CouchDB Packaging support files

✭ 24

shell Makefile couchdb http cloud database big-data apache network-server network-client

cdp-service

cdp数据平台，帮助企业充分了解客户，实现千人千面的精准营销。

✭ 30

go Dockerfile big-data crm portrait dmp cdp

Quantitative-Big-Imaging-2018

(Latest semester at https://github.com/kmader/Quantitative-Big-Imaging-2019) The material for the Quantitative Big Imaging course at ETHZ for the Spring Semester 2018

✭ 50

Jupyter Notebook big-data knime image-processing kaggle jupyterlab image-analysis lecture-videos morphological-analysis radiomics mooc-website skimage texture-analysis binder-ready lecture-handout

sgd

An R package for large scale estimation with stochastic gradient descent

✭ 55

C++r c objective c statistics big-data data-analysis gradient-descent

mmtf-spark

Methods for the parallel and distributed analysis and mining of the Protein Data Bank using MMTF and Apache Spark.

✭ 20

java machine-learning big-data apache-spark protein-data-bank protein-structure scientific-computing protein-sequences protein-protein-interaction protein-ligand-interactions

ytpriv

YT metadata exporter

✭ 28

go json crawler youtube csv big-data video datascience

Clustering4Ever

C4E, a JVM friendly library written in Scala for both local and distributed (Spark) Clustering.

✭ 126

scala big-data ai spark clustering bigdata scalability artificial-intelligence clustering-algorithm clustering-evaluation

scikit-learn-intelex

Intel(R) Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application

✭ 887

python C++cython shell CMake c Batchfile machine-learning big-data analytics gpu scikit-learn machine-learning-algorithms intel data-analysis ai-training oneapi ai-inference swrepo ai-machine-learning

predictionio-sdk-python

PredictionIO Python SDK

✭ 199

python shell big-data predictionio

accumulo-docker

Apache Accumulo Docker

✭ 17

Dockerfile shell docker big-data accumulo hacktoberfest

twitter-archive-reader

Full featured TypeScript Twitter archive reader and browser

✭ 43

typescript twitter big-data tweets bigdata twitter-archives

bullet-core

Bullet is a streaming query engine that can be plugged into any singular data stream using a Stream Processing framework like Apache Storm, Spark or Flink.

✭ 36

java streaming real-time big-data sketches bullet querying

accumulo-testing

Apache Accumulo Testing

✭ 14

java shell python HCL perl Dockerfile big-data accumulo hacktoberfest

301-360 of 369 big-data projects

first

‹

›