458 Open source projects that are alternatives of or similar to Cc Pyspark

pyspark🍒🥭 is delicious，just eat it!😋😋

State of the Art Natural Language Processing

Apache Spark Connector for Azure Cosmos DB

LearningApacheSpark

Training models with Apache Spark, PySpark for Titanic Kaggle competition

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Jupyter magics and kernels for working with remote Spark clusters

Fundamentals of Spark with Python (using PySpark), code examples

A library that provides useful extensions to Apache Spark and PySpark.

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

🐍 Quick reference guide to common patterns & functions in PySpark.

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

My presentation at ODSC India 2018 about Deep Learning with Apache Spark

Big Data Processing Framework - Unified Data API or SQL on Any Storage

🌐 Interactive Workshop on GeoAnalysis using PySpark

🚚 Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

🎼 Docker compose files for various kafka stacks

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Example project implementing best practices for PySpark ETL jobs and applications.

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

Word2Vec models with Twitter data using Spark. Blog:

Spark 2.0 Python Machine Learning examples

A simple Spark TDD example

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

Simple and Distributed Machine Learning

HandySpark - bringing pandas-like capabilities to Spark dataframes

Isolation Forest on Spark

Some class materials for a data processing course using PySpark

Apache Spark (PySpark) Practice on Real Data

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

Relation Extraction using Deep learning(CNN)

Updated repository

Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

There are pieces of scala code that explain Scala syntax and related things - like what you can do with all this

A repository to keep track of all the code that I end up writing for my blog posts.

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

This package contains a generic implementation of greedy Information Theoretic Feature Selection (FS) methods. The implementation is based on the common theoretic framework presented by Gavin Brown. Implementations of mRMR, InfoGain, JMI and other commonly used FS filters are provided.

Collection of open-source Spark tools & frameworks that have made the data engineering and data science teams at Swoop highly productive

BERT, AWS RDS, AWS Forecast, EMR Spark Cluster, Hive, Serverless, Google Assistant + Raspberry Pi, Infrared, Google Cloud Platform Natural Language, Anomaly detection, Tensorflow, Mathematics

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

汇总java生态圈常用技术框架、开源中间件，系统架构、数据库、大公司架构案例、常用三方类库、项目管理、线上问题排查、个人成长、思考等知识

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

Boiler plate framework to use Spark and ZIO together.

Apache Spark and Apache Kafka integration example

Avro SerDe for Apache Spark structured APIs.

Spark Streaming监控平台，支持任务部署与告警、自启动

Kinesis Connector for Structured Streaming

商品大数据实时推荐系统。前端：Vue + TypeScript + ElementUI，后端 Spring + Spark

An encrypted data analytics platform

Elassandra = Elasticsearch + Apache Cassandra

A pandas-like deferred expression system, with first-class SQL support

Jupyter kernel for scala and spark

📊 Cube — Open-Source Analytics API for Building Data Apps

Spark RDD with Lucene's query and entity linkage capabilities