CalciteApache Calcite
Stars: ✭ 2,816 (+18673.33%)
GeopysparkGeoTrellis for PySpark
Stars: ✭ 167 (+1013.33%)
TrinoOfficial repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
Stars: ✭ 4,581 (+30440%)
TezApache Tez
Stars: ✭ 313 (+1986.67%)
IgniteApache Ignite
Stars: ✭ 4,027 (+26746.67%)
Eel SdkBig Data Toolkit for the JVM
Stars: ✭ 140 (+833.33%)
ibmpairsopen source tools for interaction with IBM PAIRS:
Stars: ✭ 23 (+53.33%)
Hadoop For GeoeventArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.
Stars: ✭ 5 (-66.67%)
DrillApache Drill is a distributed MPP query layer for self describing data
Stars: ✭ 1,619 (+10693.33%)
RichdemHigh-performance Terrain and Hydrology Analysis
Stars: ✭ 127 (+746.67%)
OrcApache ORC - the smallest, fastest columnar storage for Hadoop workloads
Stars: ✭ 389 (+2493.33%)
HiveApache Hive
Stars: ✭ 4,031 (+26773.33%)
big-data-liteSamples to the Oracle Big Data Lite VM
Stars: ✭ 41 (+173.33%)
big dataA collection of tutorials on Hadoop, MapReduce, Spark, Docker
Stars: ✭ 34 (+126.67%)
bigdata-funA complete (distributed) BigData stack, running in containers
Stars: ✭ 14 (-6.67%)
sparkucxA high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer
Stars: ✭ 32 (+113.33%)
Data Science Ipython NotebooksData science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
Stars: ✭ 22,048 (+146886.67%)
MagellanGeo Spatial Data Analytics on Spark
Stars: ✭ 507 (+3280%)
AsakusafwAsakusa Framework
Stars: ✭ 114 (+660%)
MoosefsMooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)
Stars: ✭ 1,025 (+6733.33%)
Spark With PythonFundamentals of Spark with Python (using PySpark), code examples
Stars: ✭ 150 (+900%)
PrestoThe official home of the Presto distributed SQL query engine for big data
Stars: ✭ 12,957 (+86280%)
Bigdata PlaygroundA complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
Stars: ✭ 177 (+1080%)
GafferA large-scale entity and relation database supporting aggregation of properties
Stars: ✭ 1,642 (+10846.67%)
CloudbreakA tool for provisioning and managing Apache Hadoop clusters in the cloud. Cloudbreak, as part of the Hortonworks Data Platform, makes it easy to provision, configure and elastically grow HDP clusters on cloud infrastructure. Cloudbreak can be used to provision Hadoop across cloud infrastructure providers including AWS, Azure, GCP and OpenStack.
Stars: ✭ 301 (+1906.67%)
H2o 3H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
Stars: ✭ 5,656 (+37606.67%)
leaflet heatmap简单的可视化湖州通话数据 假设数据量很大,没法用浏览器直接绘制热力图,把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后,再使用Apache Spark绘制热力图,然后用leafletjs加载OpenStreetMap图层和热力图图层,以达到良好的交互效果。现在使用Apache Spark实现绘制,可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法,并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .
Stars: ✭ 13 (-13.33%)
Hdfs ShellHDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS
Stars: ✭ 117 (+680%)
autThe Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
Stars: ✭ 111 (+640%)
clusterdockclusterdock is a framework for creating Docker-based container clusters
Stars: ✭ 26 (+73.33%)
datalake-etl-pipelineSimplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
Stars: ✭ 39 (+160%)
OzoneScalable, redundant, and distributed object store for Apache Hadoop
Stars: ✭ 330 (+2100%)
Griffon VmGriffon Data Science Virtual Machine
Stars: ✭ 128 (+753.33%)
SparkrdmaRDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark
Stars: ✭ 215 (+1333.33%)
iisInformation Inference Service of the OpenAIRE system
Stars: ✭ 16 (+6.67%)
census-loaderA quick way to get started with ABS Census 2016 data
Stars: ✭ 32 (+113.33%)
img2datasetEasily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.
Stars: ✭ 1,173 (+7720%)
disqA library for manipulating bioinformatics sequencing formats in Apache Spark
Stars: ✭ 29 (+93.33%)
geemap-appsInteractive web apps created using geemap and streamlit
Stars: ✭ 24 (+60%)
LogAnalyzeHelper论坛日志分析系统清洗程序(包含IP规则库,UDF开发,MapReduce程序,日志数据)
Stars: ✭ 33 (+120%)
tilematrixhelps handling tile pyramids
Stars: ✭ 15 (+0%)
GDLibraryMatlab library for gradient descent algorithms: Version 1.0.1
Stars: ✭ 50 (+233.33%)
geovoronoia package to create and plot Voronoi regions within geographic boundaries
Stars: ✭ 106 (+606.67%)
rgisPerformant, cross-platform (web, desktop) GIS app written in Rust
Stars: ✭ 79 (+426.67%)
corcAn ORC File Scheme for the Cascading data processing platform.
Stars: ✭ 14 (-6.67%)
aruco-geobitsgeobits: ArUco Ground Control Point Targets and Detection for Aerial Imagery (UAV/MAV).
Stars: ✭ 32 (+113.33%)
lcbo-apiA crawler and API server for Liquor Control Board of Ontario retail data
Stars: ✭ 152 (+913.33%)
skeinA tool and library for easily deploying applications on Apache YARN
Stars: ✭ 128 (+753.33%)