845 Open source projects that are alternatives of or similar to jobAnalytics_and_search

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

Personal Data Engineering Projects

Udacity Data Engineering Nano Degree (DEND)

Example end to end data engineering project.

Blog post on ETL pipelines with Airflow

Dev jobs in the sustainability sector

Some SWE/PM/Designer related career resources for students

A job board data collector

Bulk Stash is a docker rclone service to sync, or copy, files between different storage services. For example, you can copy files either to or from a remote storage services like Amazon S3 to Google Cloud Storage, or locally from your laptop to a remote storage.

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

This repo contains commands that data engineers use in day to day work.

Astro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Airflow Deployment on AWS ECS Fargate Using Cloudformation

a collaborative collection of interview questions collected from both sides of the game: Interviewer(s) and Interviewee.

JS Jobs search telegram channel

Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

A list of useful resources to learn Data Engineering from scratch

locopy: Loading/Unloading to Redshift and Snowflake using Python.

Mural de vagas para desenvolvedor Android.

Example project implementing best practices for PySpark ETL jobs and applications.

A library on top of either pex or conda-pack to make your Python code easily available on a cluster

A curated list of awesome Amazon Web Services (AWS) libraries, open source repos, guides, blogs, and other resources. Featuring the Fiery Meter of AWSome.

A Data Engineering & Machine Learning Knowledge Hub

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Continuously synchronize directories from remote object store to local filesystem

Metric collection, data testing and monitoring for SQL accessible data

A collection of Airflow operators, hooks, and utilities to elevate dbt to a first-class citizen of Airflow.

🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript

A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract data from S3, apply a series of transformations and load into S3 and Redshift.

Viewflow is an Airflow-based framework that allows data scientists to create data models without writing Airflow code.

Example of an ETL Pipeline using Airflow

Road to Azure Data Engineer Part-I: DP-200 - Implementing an Azure Data Solution

Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Real estate dagster pipeline

Open Source Feature Flagging and A/B Testing Platform

Tellery lets you build metrics using SQL and bring them to your team. As easy as using a document. As powerful as a data modeling tool.

Free and open source schema versioning and database migration made natively with .NET Core.

A list of semi to fully remote-friendly companies (jobs) in tech.

A tool for building feature stores.

Pluggable foundation blocks for building distributed apps.

Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes

Go Wrapper for using localstack

Configuration and definitions of Airflow for OpenTrials

A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.

A schema-aware Scala library for data transformation

Learn to code and get your first job in tech 🐣

Notes & HowTo's covering the Raspberry Pi, Arduino, ESP8266, ESP32, etc.

A simple collection of tools to help you get started with Amazon Redshift from node.js

GoZeit

S3 Reverse Proxy with GET, PUT and DELETE methods and authentication (OpenID Connect and Basic Auth)

Simple rails plugin that makes it easy to store uploaded files on Amazon S3

Links to the Repos and Sections in our Dive into AWS Course.

💼 É dev? É devops? É bom? Quer mexer com muita tecnologia e desafios? Vem pro match!

支持秒级分布式定时任务系统, A high performance distributed task scheduling system, Support multi protocol scheduling tasks

Vagas e empresas que ativamente contratam pessoas desenvolvedoras Clojure no Brasil

Herd-MDL, a turnkey managed data lake in the cloud. See https://finraos.github.io/herd-mdl/ for more information.

Terraform module to provision an AWS CloudTrail and an encrypted S3 bucket with versioning to store CloudTrail logs

CodeCareer is seeking core contributors to take the lead on this project.

A wrapper around AWS Kinesis Firehose with retry logic and custom queuing behavior. Requires node >= 6.0.0