4589 Open source projects that are alternatives of or similar to Devops Python Tools

macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

DevOps Toolkit for Every Cloud on Every Cloud

🎼 Docker compose files for various kafka stacks

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

Reusable Terraform modules

CloudFormation generic custom resource provider

Build better AWS infrastructure

Spark 2.0 Python Machine Learning examples

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

A big data platform for analyzing web access logs

Machine Learning Platform and Recommendation Engine built on Kubernetes

Sample AWS CloudFormation templates

GCS support for avro-tools, parquet-tools and protobuf

A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.

Kafka Connect FileSystem Connector

A collection of Apache Parquet add-on modules

同步Hive数据仓库数据到Elasticsearch的小工具

Hive Storage Handler for interoperability between BigQuery and Apache Hive

基于Hadoop的分布式云存储系统 🌴

Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )

一个大数据架构师应该掌握的技能

基于hadoop+hbase+springboot实现分布式网盘系统

Pyspark in Google Colab: A simple machine learning (Linear Regression) model

Make record oriented data to columnar format.

大数据学习笔记，学习路线，技术案例整理。

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

Repair hbase metadata table from hdfs.

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL

Collection of Databricks and Jupyter Notebooks

Hot sync two Redis servers using dumps.

A tool and library for easily deploying applications on Apache YARN

Hadoop utility to compact small files

tools for bigData

Collective Knowledge framework (CK) helps to organize black-box research software as a database of reusable components and micro-services with common APIs, automation actions and extensible meta descriptions. See real-world use cases from Arm, General Motors, ACM, Raspberry Pi foundation and others:

大数据相关内容汇总，包括分布式存储引擎、分布式计算引擎、数仓建设等。关键词：Hadoop、HBase、ES、Kudu、Hive、Presto、Spark、Flink、Kylin、ClickHouse

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

Hadoop splittable InputFormat for ROS. Process rosbag with Hadoop Spark and other HDFS compatible systems.

Fast desktop client for Hadoop Distributed File System

No description or website provided.

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。

Mount HDFS with fuse, works with kerberos!

Code examples for my blog posts

Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services

[DEPRECATED] Argo CD Operator (Helm) installs Argo CD in OpenShift and Kubernetes.

Some class materials for a data processing course using PySpark

Learn and understand Docker technologies, with real DevOps practice!

云计算之hadoop、hive、hue、oozie、sqoop、hbase、zookeeper环境搭建及配置文件

My presentation at ODSC India 2018 about Deep Learning with Apache Spark

low-level helpers for Apache Spark libraries and tests

Building Large-Scale AI Applications for Distributed Big Data

Autoscaling for Google Cloud Dataproc

Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.

Perl Utility Library for my other repos

O'Reilly Book: [Data Algorithms with Spark] by Mahmoud Parsian

A library that provides useful extensions to Apache Spark and PySpark.

Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.

How to use Parquet in Flink