Apache Liminals goal is to operationalise the machine learning process, allowing data scientists to quickly transition from a successful experiment to an automated pipeline of model training, validation, deployment and inference in production. Liminal provides a Domain Specific Language to build ML workflows on top of Apache Airflow.

Stars: ✭ 117 (-95.04%)

Mutual labels: big-data

siembol

An open-source, real-time Security Information & Event Management tool based on big data technologies, providing a scalable, advanced security analytics framework.

Stars: ✭ 153 (-93.52%)

Mutual labels: big-data

dataframe

Structured data processing in Kotlin

Stars: ✭ 319 (-86.48%)

Mutual labels: dataframe

matcha

🍵 SPARQL-like DSL for querying in memory Linked Data Models

Stars: ✭ 18 (-99.24%)

Mutual labels: query-engine

heidi

heidi : tidy data in Haskell

Stars: ✭ 24 (-98.98%)

Mutual labels: dataframe

IoT-system-PLC-data-to-InfluxDB

This project aim is to provide free software to fetch data from plcs (Siemens S7-300/400/1200/1500) and store it. Used stack is completly opensource. I used InfluDB as data storage, so application principle is following Big Data paradigm.

Stars: ✭ 26 (-98.9%)

Mutual labels: big-data

cloudberry

Big Data Visualization

Stars: ✭ 89 (-96.23%)

Mutual labels: big-data

Pointy

A jQuery plugin that dynamically points one element at another ~

Stars: ✭ 25 (-98.94%)

Mutual labels: arrow

sparkucx

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

Stars: ✭ 32 (-98.64%)

Mutual labels: big-data

dxram

A distributed in-memory key-value storage for billions of small objects.

Stars: ✭ 25 (-98.94%)

Mutual labels: big-data

LoL-Match-Prediction

Win probability predictions for League of Legends matches using neural networks

Stars: ✭ 34 (-98.56%)

Mutual labels: big-data

rastercube

rastercube is a python library for big data analysis of georeferenced time series data (e.g. MODIS NDVI)

Stars: ✭ 15 (-99.36%)

Mutual labels: big-data

nebula

A distributed, fast open-source graph database featuring horizontal scalability and high availability

Stars: ✭ 8,196 (+247.29%)

Mutual labels: big-data

img2dataset

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

Stars: ✭ 1,173 (-50.3%)

Mutual labels: big-data

hamilton

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

Stars: ✭ 612 (-74.07%)

Mutual labels: dataframe

airavata-php-gateway

Mirror of Apache Airavata PHP Gateway

Stars: ✭ 15 (-99.36%)

Mutual labels: big-data

Real Time Social Media Mining

DevOps pipeline for Real Time Social/Web Mining

Stars: ✭ 22 (-99.07%)

Mutual labels: big-data

GDLibrary

Matlab library for gradient descent algorithms: Version 1.0.1

Stars: ✭ 50 (-97.88%)

Mutual labels: big-data

datafusion-python

A Python library to run analytics workloads with the performance of Rust, the flexibility of Python and O(1) cost in moving data between the two. Uses Apache Arrow in-memory format and respective query engine DataFusion.

Stars: ✭ 56 (-97.63%)

Mutual labels: datafusion

airavata-django-portal

Mirror of Apache Airavata Django Portal

Stars: ✭ 20 (-99.15%)

Mutual labels: big-data

lcbo-api

A crawler and API server for Liquor Control Board of Ontario retail data

Stars: ✭ 152 (-93.56%)

Mutual labels: big-data

hood

The plugin to manage benchmarks on your CI

Stars: ✭ 17 (-99.28%)

Mutual labels: arrow

📺(tv) Tidy Viewer is a cross-platform CLI csv pretty printer that uses column styling to maximize viewer enjoyment.

Stars: ✭ 1,763 (-25.3%)

Mutual labels: dataframe

tooltip

[DEPRECATED] The tooltip that has all the right moves

Stars: ✭ 133 (-94.36%)

Mutual labels: arrow

spark-vcf

Spark VCF data source implementation for Dataframes

Stars: ✭ 15 (-99.36%)

Mutual labels: dataframe

datalake-etl-pipeline

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Stars: ✭ 39 (-98.35%)

Mutual labels: big-data

gan deeplearning4j

Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.

Stars: ✭ 19 (-99.19%)

Mutual labels: big-data

azure-big-data-starter

A boilerplate project for Azure Big Data PaaS services

Stars: ✭ 13 (-99.45%)

Mutual labels: big-data

automile-php

Automile offers a simple, smart, cutting-edge telematics solution for businesses to track and manage their business vehicles.