Devops Python Tools80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Stars: ✭ 406 (+600%)
qweryA SQL-like language for performing ETL transformations.
Stars: ✭ 28 (-51.72%)
ChoetlETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
Stars: ✭ 372 (+541.38%)
StoragetapperStorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service
Stars: ✭ 232 (+300%)
Vscode Data PreviewData Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files
Stars: ✭ 245 (+322.41%)
SchemerSchema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
Stars: ✭ 97 (+67.24%)
OctosqlOctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.
Stars: ✭ 2,579 (+4346.55%)
Bigdata File ViewerA cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.
Stars: ✭ 86 (+48.28%)
DaFlowApache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Stars: ✭ 24 (-58.62%)
RoapiCreate full-fledged APIs for static datasets without writing a single line of code.
Stars: ✭ 253 (+336.21%)
TiledbThe Universal Storage Engine
Stars: ✭ 1,072 (+1748.28%)
Ps Webapi(Migrated from CodePlex) Let PowerShell Script serve or command-line process as WebAPI. PSWebApi is a simple library for building ASP.NET Web APIs (RESTful Services) by PowerShell Scripts or batch/executable files out of the box.
Stars: ✭ 24 (-58.62%)
PucketBucketing and partitioning system for Parquet
Stars: ✭ 29 (-50%)
Elasticsearch loaderA tool for batch loading data files (json, parquet, csv, tsv) into ElasticSearch
Stars: ✭ 300 (+417.24%)
Just Dashboard📊 📋 Dashboards using YAML or JSON files
Stars: ✭ 1,511 (+2505.17%)
IcebergIceberg is a table format for large, slow-moving tabular data
Stars: ✭ 393 (+577.59%)
SpecsTechnical specifications and guidelines for implementing Frictionless Data.
Stars: ✭ 403 (+594.83%)
Data Science Ipython NotebooksData science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
Stars: ✭ 22,048 (+37913.79%)
RioA Swiss-Army Knife for Data I/O
Stars: ✭ 467 (+705.17%)
SpartaReal Time Analytics and Data Pipelines based on Spark Streaming
Stars: ✭ 513 (+784.48%)
ParsrsCSV, JSON, XML text parsers and generators written in pure POSIX shellscript
Stars: ✭ 56 (-3.45%)
DatasetteAn open source multi-tool for exploring and publishing data
Stars: ✭ 5,640 (+9624.14%)
TrdsqlCLI tool that can execute SQL queries on CSV, LTSV, JSON and TBLN. Can output to various formats.
Stars: ✭ 593 (+922.41%)
CountriesWorld countries in JSON, CSV, XML and Yaml. Any help is welcome!
Stars: ✭ 5,379 (+9174.14%)
Fsharp.dataF# Data: Library for Data Access
Stars: ✭ 631 (+987.93%)
Pyspark Example ProjectExample project implementing best practices for PySpark ETL jobs and applications.
Stars: ✭ 633 (+991.38%)
Pulsar SparkWhen Apache Pulsar meets Apache Spark
Stars: ✭ 55 (-5.17%)
Python Ml CourseCurso de Introducción a Machine Learning con Python
Stars: ✭ 442 (+662.07%)
God Of Bigdata专注大数据学习面试,大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
Stars: ✭ 6,008 (+10258.62%)
AwsA collection of bash shell scripts for automating various tasks with Amazon Web Services using the AWS CLI and jq.
Stars: ✭ 493 (+750%)
Pytablewriterpytablewriter is a Python library to write a table in various formats: CSV / Elasticsearch / HTML / JavaScript / JSON / LaTeX / LDJSON / LTSV / Markdown / MediaWiki / NumPy / Excel / Pandas / Python / reStructuredText / SQLite / TOML / TSV.
Stars: ✭ 422 (+627.59%)
ApiOur Database
Stars: ✭ 568 (+879.31%)
ServicestackThoughtfully architected, obscenely fast, thoroughly enjoyable web services for all
Stars: ✭ 4,976 (+8479.31%)
World countriesConstantly updated lists of world countries and their associated alpha-2, alpha-3 and numeric country codes as defined by the ISO 3166 standard, available in CSV, JSON , PHP and SQL formats, in multiple languages and with national flags included
Stars: ✭ 598 (+931.03%)
Agile data code 2Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
Stars: ✭ 413 (+612.07%)
Boltons🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.
Stars: ✭ 5,671 (+9677.59%)
DataprooferA proofreader for your data
Stars: ✭ 628 (+982.76%)
CountriesCountries, Languages & Continents data (capital and currency, native name, calling codes).
Stars: ✭ 656 (+1031.03%)
H2o 3H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
Stars: ✭ 5,656 (+9651.72%)
Sheetjs📗 SheetJS Community Edition -- Spreadsheet Data Toolkit
Stars: ✭ 28,479 (+49001.72%)
Pmacctpmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].
Stars: ✭ 677 (+1067.24%)
Nano SqlUniversal database layer for the client, server & mobile devices. It's like Lego for databases.
Stars: ✭ 717 (+1136.21%)
RowsA common, beautiful interface to tabular data, no matter the format
Stars: ✭ 739 (+1174.14%)
SqlitebiterA CLI tool to convert CSV / Excel / HTML / JSON / Jupyter Notebook / LDJSON / LTSV / Markdown / SQLite / SSV / TSV / Google-Sheets to a SQLite database file.
Stars: ✭ 601 (+936.21%)
Structured Text ToolsA list of command line tools for manipulating structured text data
Stars: ✭ 6,180 (+10555.17%)
Kafka Storm StarterCode examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.
Stars: ✭ 728 (+1155.17%)
Json2csvcommand line tool to convert json to csv
Stars: ✭ 742 (+1179.31%)
Goodreads etl pipelineAn end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
Stars: ✭ 793 (+1267.24%)
PgbackrestReliable PostgreSQL Backup & Restore
Stars: ✭ 766 (+1220.69%)
Cluster PackA library on top of either pex or conda-pack to make your Python code easily available on a cluster
Stars: ✭ 23 (-60.34%)
KaluluUganda Elections Tools and Resources
Stars: ✭ 24 (-58.62%)
Bigdata Interview🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
Stars: ✭ 857 (+1377.59%)
Tiledb VcfEfficient variant-call data storage and retrieval library using the TileDB storage library.
Stars: ✭ 26 (-55.17%)
ClevercsvCleverCSV is a Python package for handling messy CSV files. It provides a drop-in replacement for the builtin CSV module with improved dialect detection, and comes with a handy command line application for working with CSV files.
Stars: ✭ 887 (+1429.31%)
PortabletextPortable Text is a JSON based rich text specification for modern content editing platforms.
Stars: ✭ 759 (+1208.62%)
S3proxyAccess other storage backends via the S3 API
Stars: ✭ 952 (+1541.38%)
Gcs ToolsGCS support for avro-tools, parquet-tools and protobuf
Stars: ✭ 57 (-1.72%)
SnappydataProject SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster
Stars: ✭ 995 (+1615.52%)