Hydrospheredata / Mist
Licence: apache-2.0
Serverless proxy for Spark cluster
Stars: ✭ 309
Programming Languages
scala
5932 projects
Projects that are alternatives of or similar to Mist
Spark On Lambda
Apache Spark on AWS Lambda
Stars: ✭ 137 (-55.66%)
Mutual labels: serverless, big-data, apache-spark
datalake-etl-pipeline
Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
Stars: ✭ 39 (-87.38%)
Mutual labels: big-data, apache-spark
gan deeplearning4j
Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.
Stars: ✭ 19 (-93.85%)
Mutual labels: big-data, apache-spark
SynapseML
Simple and Distributed Machine Learning
Stars: ✭ 3,355 (+985.76%)
Mutual labels: big-data, apache-spark
Detecting-Malicious-URL-Machine-Learning
No description or website provided.
Stars: ✭ 47 (-84.79%)
Mutual labels: big-data, apache-spark
mmtf-spark
Methods for the parallel and distributed analysis and mining of the Protein Data Bank using MMTF and Apache Spark.
Stars: ✭ 20 (-93.53%)
Mutual labels: big-data, apache-spark
sparkucx
A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer
Stars: ✭ 32 (-89.64%)
Mutual labels: big-data, apache-spark
Spider Less
Web spider as a service, spider on serverless
Stars: ✭ 172 (-44.34%)
Mutual labels: api, serverless
pyspark-cheatsheet
PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster
Stars: ✭ 115 (-62.78%)
Mutual labels: big-data, apache-spark
leaflet heatmap
简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .
Stars: ✭ 13 (-95.79%)
Mutual labels: big-data, apache-spark
aut
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
Stars: ✭ 111 (-64.08%)
Mutual labels: big-data, apache-spark
Dialetus Service
API to Informal dictionary for the idiomatic expressions that each Brazilian region It has
Stars: ✭ 202 (-34.63%)
Mutual labels: api, serverless
Aws Lambda Fastify
Insipired by aws-serverless-express to work with Fastify with inject functionality.
Stars: ✭ 190 (-38.51%)
Mutual labels: api, serverless
awesome-tools
curated list of awesome tools and libraries for specific domains
Stars: ✭ 31 (-89.97%)
Mutual labels: big-data, apache-spark
Autoserver
Create a full-featured REST/GraphQL API from a configuration file
Stars: ✭ 188 (-39.16%)
Mutual labels: api, serverless
spark-records
Bulletproof Apache Spark jobs with fast root cause analysis of failures.
Stars: ✭ 67 (-78.32%)
Mutual labels: big-data, apache-spark
Morpheus
Morpheus brings the leading graph query language, Cypher, onto the leading distributed processing platform, Spark.
Stars: ✭ 303 (-1.94%)
Mutual labels: big-data, apache-spark
Graphql Genie
Simply pass in your GraphQL type defintions and get a fully featured GraphQL API with referential integrity, inverse updates, subscriptions and role based access control that can be used client side or server side.
Stars: ✭ 147 (-52.43%)
Mutual labels: api, serverless
SparkProgrammingInScala
Apache Spark Course Material
Stars: ✭ 57 (-81.55%)
Mutual labels: big-data, apache-spark
Hydrosphere Mist
Hydrosphere Mist is a serverless proxy for Spark cluster. Mist provides a new functional programming framework and deployment model for Spark applications.
Please see our quick start guide and documentation
Features:
- Spark Function as a Service. Deploy Spark functions rather than notebooks or scripts.
- Spark Cluster and Session management. Fully managed Spark sessions backed by on-demand EMR, Hortonworks, Cloudera, DC/OS and vanilla Spark clusters.
- Typesafe programming framework that clearly defines inputs and outputs of every Spark job.
- REST HTTP & Messaging (MQTT, Kafka) API for Scala & Python Spark jobs.
- Multi-cluster mode: Seamless Spark cluster on-demand provisioning, autoscaling and termination(pending)
It creates a unified API layer for building enterprise solutions and microservices on top of a Spark functions.
High Level Architecture
Contact
Please report bugs/problems to: https://github.com/Hydrospheredata/mist/issues.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].