GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → huseinzol05 → Gather Deployment

huseinzol05 / Gather Deployment

Licence: mit
Gathers scalable tensorflow and infrastructure deployment

Programming Languages

python
139335 projects - #7 most used programming language

Labels

flasknginxkafkaface-detectiontext-classificationhadoopceleryinception

Projects that are alternatives of or similar to Gather Deployment

Incepiton Mysql
🍭A web platform designed for mysql inception
Stars: ✭ 90 (-72.39%)
Mutual labels:  flask, inception, celery
Autoops
linux资产管理,cmdb,django, webssh,运维管理平台，数据库操作平台 本项目已停止开发！因长时间未对代码进行维护，可能会造成项目在不同环境上无法部署、运行BUG等问题，请知晓！项目仅供参考！
Stars: ✭ 340 (+4.29%)
Mutual labels:  inception, celery, nginx
Camus
Mirror of Linkedin's Camus
Stars: ✭ 81 (-75.15%)
Mutual labels:  kafka, hadoop
Repository
个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。
Stars: ✭ 92 (-71.78%)
Mutual labels:  kafka, hadoop
Aliyun Emapreduce Datasources
Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.
Stars: ✭ 132 (-59.51%)
Mutual labels:  kafka, hadoop
Bigdata Interview
🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
Stars: ✭ 857 (+162.88%)
Mutual labels:  kafka, hadoop
Cdc Kafka Hadoop
MySQL to NoSQL real time dataflow
Stars: ✭ 13 (-96.01%)
Mutual labels:  kafka, hadoop
Bigdata Notebook
Stars: ✭ 100 (-69.33%)
Mutual labels:  kafka, hadoop
God Of Bigdata
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
Stars: ✭ 6,008 (+1742.94%)
Mutual labels:  kafka, hadoop
Bigdata Playground
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
Stars: ✭ 177 (-45.71%)
Mutual labels:  kafka, hadoop
Bigdata practice
大数据分析可视化实践
Stars: ✭ 166 (-49.08%)
Mutual labels:  kafka, nginx
Seconds Kill
基于 Springboot + Redis + Kafka 的秒杀系统，乐观锁 + 缓存 + 限流 + 异步，TPS 从 500 优化到 3000
Stars: ✭ 180 (-44.79%)
Mutual labels:  kafka, nginx
Dockerfiles
50+ DockerHub public images for Docker & Kubernetes - Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak, TeamCity and DevOps tools built on the major Linux distros: Alpine, CentOS, Debian, Fedora, Ubuntu
Stars: ✭ 847 (+159.82%)
Mutual labels:  kafka, hadoop
Szt Bigdata
深圳地铁大数据客流分析系统🚇🚄🌟
Stars: ✭ 826 (+153.37%)
Mutual labels:  kafka, hadoop
Nagios Plugins
450+ AWS, Hadoop, Cloud, Kafka, Docker, Elasticsearch, RabbitMQ, Redis, HBase, Solr, Cassandra, ZooKeeper, HDFS, Yarn, Hive, Presto, Drill, Impala, Consul, Spark, Jenkins, Travis CI, Git, MySQL, Linux, DNS, Whois, SSL Certs, Yum Security Updates, Kubernetes, Cloudera etc...
Stars: ✭ 1,000 (+206.75%)
Mutual labels:  kafka, hadoop
Bigdataguide
大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料
Stars: ✭ 817 (+150.61%)
Mutual labels:  kafka, hadoop
Bigdata Notes
大数据入门指南 ⭐
Stars: ✭ 10,991 (+3271.47%)
Mutual labels:  kafka, hadoop
Devops Bash Tools
550+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Kafka, Docker, APIs, Hadoop, SQL, PostgreSQL, MySQL, Hive, Impala, Travis CI, Jenkins, Concourse, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, .tmux.conf, .psqlrc ...
Stars: ✭ 226 (-30.67%)
Mutual labels:  kafka, hadoop
Wedatasphere
WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!
Stars: ✭ 372 (+14.11%)
Mutual labels:  kafka, hadoop
Kafka Connect Hdfs
Kafka Connect HDFS connector
Stars: ✭ 400 (+22.7%)
Mutual labels:  kafka, hadoop
View All Similar Projects ➔

Gather-Deployment

Gathers scalable tensorflow and infrastructure deployment, reusable purpose in the future.

Table of contents

Tensorflow deployment

  1. Object Detection. Flask SocketIO + WebRTC
  • Stream from webcam using WebRTC -> Flask SocketIO to detect objects -> WebRTC -> Website.
  1. Object Detection. Flask SocketIO + opencv
  • Stream from OpenCV -> Flask SocketIO to detect objects -> OpenCV.
  1. Speech streaming. Flask SocketIO
  • Stream speech from microphone -> Flask SocketIO to do realtime speech recognition.
  1. Text classification. Flask + Gunicorn
  • Serve Tensorflow text model using Flask multiworker + Gunicorn.
  1. Image classification. TF Serving
  • Serve image classification model using TF Serving.
  1. Image Classification using Inception. Flask SocketIO
  • Stream image using SocketIO -> Flask SocketIO to classify.
  1. Object Detection. Flask + opencv
  • Webcam -> Opencv -> Flask -> web dashboard.
  1. Face-detection using MTCNN. Flask SocketIO + opencv
  • Stream from OpenCV -> Flask SocketIO to detect faces -> OpenCV.
  1. Face-detection using MTCNN. opencv
  • Webcam -> Opencv.
  1. Image classification using Inception. Flask + Docker
  • Serve Tensorflow image model using Flask multiworker + Gunicorn on Docker container.
  1. Image classification using Inception. Flask + EC2 Docker Swarm + Nginx load balancer
  • Serve inception on multiple AWS EC2, scale using Docker Swarm, balancing using Nginx.
  1. Text classification. Hadoop streaming MapReduce
  • Batch processing to classify texts using Tensorflow text model on Hadoop MapReduce.
  1. Text classification. Kafka
  • Stream text to Kafka producer and classify using Kafka consumer.
  1. Text classification. Distributed TF using Flask + Gunicorn + Eventlet
  • Serve text model on multiple machines using Distributed TF + Flask + Gunicorn + Eventlet. Means that, Distributed TF will split a single neural network model to multiple machines to do feed-forward.
  1. Text classification. Tornado + Gunicorn
  • Serve Tensorflow text model using Tornado + Gunicorn.
  1. Text classification. Flask + Celery + Hadoop
  • Submit large texts using Flask, signal queue celery job to process using Hadoop, delay Hadoop MapReduce.
  1. Text classification. Luigi scheduler + Hadoop
  • Submit large texts on Luigi scheduler, run Hadoop inside Luigi, event based Hadoop MapReduce.
  1. Text classification. Luigi scheduler + Distributed Celery
  • Submit large texts on Luigi scheduler, run Hadoop inside Luigi, delay processing.
  1. Text classification. Airflow scheduler + elasticsearch + Flask
  • Scheduling based processing using Airflow, store inside elasticsearch, serve it using Flask.
  1. Text classification. Apache Kafka + Apache Storm
  • Stream from twitter -> Kafka Producer -> Apache Storm, to do distributed minibatch realtime processing.
  1. Text classification. Dask
  • Batch processing to classify texts using Tensorflow text model on Dask.
  1. Text classification. Pyspark
  • Batch processing to classify texts using Tensorflow text model on Pyspark.
  1. Text classification. Pyspark streaming + Kafka
  • Stream texts to Kafka Producer -> Pyspark Streaming, to do minibatch realtime processing.
  1. Text classification. PyFlink
  • Batch processing to classify texts using Tensorflow text model on Flink batch processing.
  1. Text classification. Streamz + Dask + Kafka
  • Stream texts to Kafka Producer -> Streamz -> Dask, to do minibatch realtime processing.
  1. Text classification. FastAPI + Streamz + Water Healer
  • Change concurrent requests into mini-batch realtime processing to speed up text classification.

Printscreen

All folders contain print screens, logs and instructions on how to start.

Notes

  1. Deploy them on a server, change local in code snippets to your own IP.
  2. WebRTC chrome only can tested on HTTPS server.
  3. When come to real deployment, always prepare for up-scaling architectures. Learn about DevOps.
  4. Please aware with your cloud cost!
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].