Use Cases

Sample Projects for Data Scientists in Training by V Granville, 2018

3D Recognition

https://github.com/IsaacGuan/PointNet-Plane-Detection
- accuracy around 85% for 100 epochs using TensorFlow
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, 2017
- https://github.com/charlesq34/pointnet

Semantic Segmentation

SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks by John McCormac et al, 2016

Audio Recognition

Speech to Text

Data Augmentation

Design

Games

Gesture Recognition

Using wearable sensors (phones, watches etc.)

Apps

Code repositories

Hyperparameter Tuning

Image Recognition

Face Recognition

Вы и Брэд Питт похожи на 99%
- telegram bot telling you which celebrity your face is similar to
- dlib + resnet + nmslib
Умные фотографии ВКонтакте, 2018 (Smart photos in Vkontakte)
FaceNet: A Unified Embedding for Face Recognition and Clustering by Florian Schroff et al, 2015
- the model: FaceNet

Food Recognition

Semantic Segmentation

What do we learn from region based object detectors (Faster R-CNN, R-FCN, FPN)? 2018
What do we learn from single shot object detectors (SSD, YOLOv3), FPN & Focal loss (RetinaNet)? 2018
Design choices, lessons learned and trends for object detections?
Semantic Image Segmentation with DeepLab in Tensorflow, 2018
- model DeepLab-v3+ built on top of CNN
- https://github.com/tensorflow/models/tree/master/research/deeplab
- has Checkpoints and frozen inference graphs
- Deeplab demo on python
- support adopting MobileNetv2 for mobile devices and Xception for server-side deployment
- evaluates results in terms of mIOU (mean intersection-over-union)
- use PASCAL VOC 2012 and Cityscapes semantic segmentation benchmarks as an example in the code
- https://github.com/lankastersky/deeplab_background_segmentation (not working android app)
Rethinking Atrous Convolution for Semantic Image Segmentation by Liang-Chieh Chen et al, 2017 MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features by Liang-Chieh Chen et al, 2017](https://arxiv.org/abs/1712.04837)
- present a model, called MaskLab, which produces three outputs: box detection, semantic segmentation, and direction prediction
- built on top of the Faster-RCNN object detector
- evaluated on the COCO instance segmentation benchmark and shows comparable performance with other state-of-art models
Mask R-CNN by Kaiming He et al, 2017
- https://github.com/facebookresearch/Detectron
- see links to articles at the end of the page
- extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition
- simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps
- easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework
- outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners
- uses the area under the precision recall curve (AP) metrics
A Brief History of CNNs in Image Segmentation: From R-CNN to Mask R-CNN, 2017

Interpretability

The Building Blocks of Interpretability, 2018
GoogleNet for image classification is used as an example
Attributing a deep network’s prediction to its input features by MUKUND SUNDARARAJAN, 2017
- Integrated Gradients method
A unified approach to interpreting model predictions by Scott M Lundberg et al, 2017
"Why Should I Trust You?": Explaining the Predictions of Any Classifier by Marco Tulio Ribeiro et al, 2016
- Lime Framework: Explaining the predictions of any machine learning classifier
Monotonic Calibrated Interpolated Look-Up Tables by Maya Gupta et al, 2016
see Decision trees
see Distillation

Programming and ML

Predict defects

To type or not to type: quantifying detectable bugs in JavaScript by Gao et al, 2017
Predicting Defects for Eclipse by T Zimmermann at al, 2007
- used code complexity metrics as features and logistic regression for classification (if file/module has defects) and linear regression for ranking (how many defects)
Predicting Component Failures at Design Time by Adrian Schroter et al, 2006
- showed that design data such as import relationships can predict failures
- used the number of failures in a component as dependent variable and the imported resources used from this component as input features
Mining Version Histories to Guide Software Changes by T Zimmermann at al, 2004
- used apriory algorithm to predict likely changes in files/modules

Predict performance

https://codescene.io
3 ways AI will change project management for the better, 2017
A deep learning model for estimating story points by Morakot Choetkiertikul et al, 2016
- estimating story points based on long short-term memory and recurrent highway network

Searching code

Deep code search by Xiaodong Gu1 et al, 2018
- blog post
How To Create Natural Language Semantic Search For Arbitrary Objects With Deep Learning, 2018
- https://github.com/hamelsmu/code_search
- you can use similar techniques to search video, audio, and other objects

Writing code

NLP

Improving Language Understanding with Unsupervised Learning - OpenAI
SentEval: An Evaluation Toolkit for Universal Sentence Representations by A. Conneau et al, 2018
- https://github.com/facebookresearch/SentEval
- the benchmarks may not be appropriate for domain-specific problems
Text Embedding Models Contain Bias. Here's Why That Matters, 2018
How to Clean Text for Machine Learning with Python

Chatbots

https://ipavlov.ai/ - open-source conversational AI framework built on TensorFlow and Keras (En, Ru)
- https://github.com/deepmipt/DeepPavlov
Behind the Chat: How E-commerce Robot Assistant AliMe Works, 2018
How I Used Deep Learning To Train A Chatbot To Talk Like Me (Sorta), 2017
- Short-Text Conversations generative model based on Tensorflow’s embedding_rnn_seq2seq() with custom dataset. Deployed as a Facebook chatbot using heroku (hosting)+express(frontend)+flask(backend)
Deep Learning for Chatbots, Part 1 – Introduction, 2016
Deep Learning for Chatbots, Part 2 – Implementing a Retrieval-Based Model in Tensorflow, 2016
https://github.com/gunthercox/ChatterBot
- Retrieval-based model based on naive Bayesian classification and search algorithms
- see Sequence to sequence
A Persona-Based Neural Conversation Model by Jiwei Li et al, 2016
Smart reply
- Smart Reply: Automated Response Suggestion for Emai by Anjuli Kannan et al, 2016
- Computer, respond to this email, 2015
Chatbot projects: https://github.com/fendouai/Awesome-Chatbot
see Chatbot platforms

Crossword question answerers

see Reverse dictionaries

Database queries

LEARNING A NATURAL LANGUAGE INTERFACE WITH NEURAL PROGRAMMER by Arvind Neelakantan et al, 2017
- weakly supervised, end-to-end neural network model mapping natural language queries to logical forms or programs that provide the desired response when executed on the database

Named entity resolution

Also known as deduplication and record linkage (but not entity recognition which is picking up the names and classifying them in running text)

Collective Entity Resolution in Familial Networks by Pigi Kouki et al, 2017
- combines machine learning (although not NNs) with collective inference
Entity Resolution Using Convolutional Neural Network by Ram DeepakGottapu et al, 2016
Adaptive Blocking: Learning to Scale Up Record Linkage by Mikhail Bilenko et al, 2006
- extremely high recall but low precision
https://stats.stackexchange.com/questions/136755/popular-named-entity-resolution-software

Reverse dictionaries

Other name is concept finders Return the name of a concept given a definition or description:

Learning to Understand Phrases by Embedding the Dictionary by Felix Hill et al, 2016
- used models: Bag-of-Words NLMs and LSTM
comparing definitions in a database to the input query, and returning the word whose definitionis ‘closest’ to that query
see RNNs (with LSTMs)
see bag-of-word

Sequence to sequence

Smart Compose: Using Neural Networks to Help Write Emails, 2018
Introducing Semantic Experiences with Talk to Books and Semantris by Rey Kurzweil et al, 2018
- Universal Sentence Encoder Daniel Cer et al, 2018
- Pretrained semantic TensorFlow module
Keras LSTM tutorial – How to easily build a powerful deep learning language model by Andy, 2018
Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models by Louis Shao et al, 2017
- trained on a combined data set of over 2.3B conversation messages mined from the web
- The model: LSTM on tensorflow
Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features by Matteo Pagliardini et al, 2017
- the model: Sent2Vec based on vec2vec
Skip-Thought Vectors by Ryan Kiros et al, 2015
- based on RNN encoder-decoder models
Sequence to Sequence Learning with Neural Networks by Ilya Sutskever et al, 2014
- the model: seq2seq based on LSTM
Distributed Representations of Sentences and Documents by Quoc V. Le, Mikolov, 2014
- gensim's doc2vec
- python example to train doc2vec model (with or without pre-trained word embeddings)
Distributed Representations of Words and Phrases and their Compositionality by Tomas Mikolov et al, 2013
- word2vec based on Mikolov's Skip-gram model
Learning Continuous Phrase Representations and Syntactic Parsing with Recursive Neural Networks by Richard Socher et al, 2010
- based on context-sensitive recursive neural networks (CRNN)
see Reverse dictionaries
How to calculate the sentence similarity using word2vec model
- Doc2Vec
- Average w2v vectors
- Weighted average w2v vectors (e.g. tf-idf)
- RNN-based embeddings (e.g. deep LSTM networks)
- Document Similarity With Word Movers Distance
  - From Word Embeddings To Document Distances by Matt J. Kusner et al, 2015
- A SIMPLE BUT TOUGH-TO-BEAT BASELINE FOR SENTENCE EMBEDDINGS by Sanjeev Arora et al, 2017
  - uses smooth inverse frequency
  - computing the weighted average of word vectors in the sentence and then remove the projections of the average vectors on their first principal component
  - example
  - https://github.com/peter3125/sentence2vec - requires writing the get_word_frequency() method which can be easily accomplished by using Python's Counter() and returning a dict with keys: unique words w, values: #w/#total doc len

Semantic analysis

Advances in Semantic Textual Similarity, 2018
- Learning Semantic Textual Similarity from Conversations by Yinfei Yang et al, 2018
- Universal Sentence Encoder by Daniel Cer et al, 2018
Semantic Textual Similarity Wiki, 2017
A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks by Soujanya Poria et al, 2017
- Blog post: Detecting Sarcasm with Deep Convolutional Neural Networks
Twitter Sentiment Analysis Using Combined LSTM-CNN Models by SOSAVPM, 2018
- https://github.com/pmsosa/CS291K
- used pre-trained embeddings with LSTM-CNN model with dropouts
- 75.2% accuracy for binary classification (positive-negative tweet)
doc2vec example, 2015

Spelling

How to Write a Spelling Corrector

Summarization

How To Create Data Products That Are Magical Using Sequence-to-Sequence Models
- A tutorial on how to summarize text and generate features from Github Issues using deep learning with Keras and TensorFlow
- https://github.com/hamelsmu/Seq2Seq_Tutorial
Generating Wikipedia by Summarizing Long Sequences by Peter J. Liu et al, 2018

Text classification

Universal Language Model Fine-tuning for Text Classification by Jeremy Howard et al, 2018
- outperforms the state-of-the-art on six text classification tasks, reducing the error by 18-24% on the majority of datasets.
- with only 100 labeled examples, it matches the performance of training from scratch on 100× more data
- http://nlp.fast.ai/ulmfit

Text to Image

Text to Speech

Personality recognition

Mining Facebook Data for Predictive Personality Modeling (Dejan Markovikj,Sonja Gievska, Michal Kosinski, David Stillwell)
Personality Traits Recognition on Social Network — Facebook (Firoj Alam, Evgeny A. Stepanov, Giuseppe Riccardi)
The Relationship Between Dimensions of Love, Personality, and Relationship Length (Gorkan Ahmetoglu, Viren Swami, Tomas Chamorro-Premuzic)

Robotics

Grasp2Vec: Learning Object Representations from Self-Supervised Grasping
- Achieved a success rate of 80 percent on objects seen during data collection and 59% on novel objects the robot hasn’t encountered before

Search

Can word2vec be used for search?
- alternative search queries can be built using approximate nearest neighbors in embedding vectors space of terms (using https://github.com/spotify/annoy e.g.)
- Improving Document Ranking with Dual Word Embeddings by Eric Nalisnick et al, 2016

Transfer Learning

Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies by Alessandro Achille et al, 2018
- possible solution of catastrophic forgetting
Deep Learning & Art: Neural Style Transfer – An Implementation with Tensorflow in Python, 2018
Image Classification using Flowers dataset on Cloud ML Enginge, 2018
Android & TensorFlow: Artistic Style Transfer, 2018 codelab
The TensorFlow Poet tutorial shows how to retrain a tensorflow graph to classify images of flowers.

Uber

Engineering More Reliable Transportation with Machine Learning and AI at Uber, 2017

Video recognition

Pose recognition

Object detection

Here are video-specific methods. See also Semantic Segmentation.

Scene Segmentation

Detects when one video (shot/scene/chapter) ends and another begins

Video Captioning

Video Classification

Learnable pooling with Context Gating for video classification by Antoine Miech et al, 2018
- Rank #1 at Google Cloud & YouTube-8M Video Understanding Challenge
- Slow for inference/training
- NOT a sequential problem
- Needs lots of data for training
- not clear about very long videos
The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge, 2017
- Rank #2 at Google Cloud & YouTube-8M Video Understanding Challenge
Hierarchical Deep Recurrent Architecture for Video Understanding by Luming Tang et al, 2017
Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet? by Kensho Hara et al, 2017
- https://github.com/kenshohara/video-classification-3d-cnn-pytorch
  - trained on the Kinetics dataset from scratch using only RGB input
  - pretrained ResNeXt-101 achieved 94.5% and 70.2% on UCF-101 and HMDB-51
Appearance-and-Relation Networks for Video Classification by Limin Wang et al, 2017
- https://github.com/wanglimin/ARTNet
  - trained on the Kinetics dataset from scratch using only RGB input
  - 70.9% and 94.3% on HMDB51 UCF101
Five video classification methods implemented in Keras and TensorFlow by Matt Harvey, 2017
- https://github.com/harvitronix/five-video-classification-methods
Video Understanding: From Video Classification to Captioning by Jiajun Sun et al, 2017
Video Classification using Two Stream CNNs, 2016 code based on articles below
- Two-Stream Convolutional Networks for Action Recognition in Videos
- Fusing Multi-Stream Deep Networks for Video Classification
- Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
- Towards Good Practices for Very Deep Two-Stream ConvNets
Beyond Short Snippets: Deep Networks for Video Classification by Joe Yue-Hei Ng et al, 2015
- In order to learn a global description of the video while maintaining a low computational footprint, we propose processing only one frame per second
Large-scale Video Classification with Convolutional Neural Networks by Andrej Karpathy et al, 2014
- 63.3% on UCF-101

Visualization

Google Datalab
Google Dataprep
Tableau
Google Brain: Big Picture Group
Deeplearn.js - open source hardware-accelerated machine intelligence library for the web
Facets - open source visualizations for machine learning datasets
- Facets: An Open Source Visualization Tool for Machine Learning Training Data, 2017
- https://github.com/PAIR-code/facets
Embedding Projector - an open source, visualization tool for high-dimensional data

Multiple Modalities

Multimodal Classification for Analysing Social Media by Chi Thang Duong et al, 2017
- Blog post: Detecting Emotions with CNN Fusion Models
- https://emoclassifier.github.io/

Open problems

Recycled goods (not solved, no dataset)
- Recycling symbols explained
- similar to traffic signs recognition
Safety symbols on cardboard boxes (not solved, no dataset)

Tools

50+ Useful Machine Learning & Prediction APIs, 2018
- Face and Image Recognition
- Text Analysis, NLP, Sentiment Analysis
- Language Translation
- Machine Learning and prediction
Command-line tricks data scientists
Deep Video Analytics
- Data-centric platform for Computer Vision
- https://github.com/akshayubhat/deepvideoanalytics

Amazon SageMaker

Distributed Training: You can’t choose the number of workers and parameter servers independently
Job Startup Latency: Up to 5 minutes single node
Hyper Parameters Tuning: In-Preview, and only supports the built-in algorithms
Batch Prediction: Not supported
GPU readiness: Bring your own docker image with CUDA installed
Auto-scale Online Serving: You need to specify the number of nodes
Training Job Monitoring: No monitoring

Apple Core ML

iOS framework from Apple to integrate machine learning models into your app.

Apple Create ML

Apple framework used with familiar tools like Swift and macOS playgrounds to create and train custom machine learning models on your Mac.

Introducing Create ML on wwdc2018

Apple Natural Language Framework

Introducing Natural Language Framework on wwdc2018

Firebase ML Kit

ML Kit: Machine Learning SDK for mobile developers (Google I/O '18)
Uses Google Cloud APIs under the hood
Uses custom TensorFlow Lite models
Can compress TensorFlow to TensorFlow Lite models
Runs on a device (fast, inaccurate) or on a cloud
Examples and codelabs

Google AutoML

Pros:

let users train their own custom machine learning algorithms from scratch, without having to write a single line of code
uses Transfer Learning (the more data and customers, the better results)
is fully integrated with other Google Cloud services (Google Cloud Storage to store data, use Cloud ML or Vision API to customize the model etc.)

Cons:

limited to image recognition (2018-Q1)
doesn't allow to download a trained model

Google Datalab

Powerful interactive tool created to explore, analyze, transform and visualize data and build machine learning models on Google Cloud Platform. It runs on Google Compute Engine and connects to multiple cloud services easily so you can focus on your data science tasks.
Built on Jupyter (formerly IPython), which boasts a thriving ecosystem of modules and a robust knowledge base.
Enables analysis of your data on Google BigQuery, Cloud Machine Learning Engine, Google Compute Engine, and Google Cloud Storage using Python, SQL, and JavaScript (for BigQuery user-defined functions).

Intelligent data service for visually exploring, cleaning, and preparing structured and unstructured data for analysis. Cloud Dataprep is serverless and works at any scale. Easy data preparation with clicks and no code.

Google ML Engine

Samples & Tutorials
Samples for usage
Distributed Training: Specify number of nodes, types, (workers/PS), associated accelerators, and sizes
Job Startup Latency: 90 seconds for single node
Hyper Parameters Tuning: Grid Search, Random Search, and Bayesian Optimisation
Batch Prediction: You can submit a batch prediction job for high throughputs
GPU readiness: Out-of-the box, either via scale-tier, or config file
Auto-scale Online Serving: Scaled up to your specified maximum number of nodes, down to 0 nodes if no requests for 5 minutes
Training Job Monitoring: Full monitoring to the cluster nodes (CPU, Memory, etc.)
Automation of ML: AutoML - Vision, NLP, Speech, etc.
Specialised Hardware: Tensor Processing Units (TPUs)
SQL-supported ML: BQML

Google Natural language

entiry recognition: extract information about people, places, events, and much more mentioned in text documents, news articles, or blog posts
sentiment analysis: understand the overall sentiment expressed in a block of text
multilingual support
syntax analysis: extract tokens and sentences, identify parts of speech (PoS) and create dependency parse trees for each sentence

Google Deep Learning Virtual Machine

VMs with CPU and GPU

Google Mobile Vision

Detect Faces (finds facial landmarks such as the eyes, nose, and mouth; doesn't identifies a person)
Scan barcodes
Recognize Text

Google Speech API

speech recognition
word hints: Can provide context hints for improved accuracy. Especially useful for device and app use cases.
noise robustness: No need for signal processing or noise cancellation before calling API; can handle noisy audio from a variety of environments
realtime results: can stream text results, returning partial recognition results as they become available. Can also be run on buffered or archived audio files.
over 80 languages
can also filter inappropriate content in text results

Google Translation API

Supports more than 100 languages and thousands of language pairs
automatic language detection
continuous updates: Translation API is learning from logs analysis and human translation examples. Existing language pairs improve and new language pairs come online at no additional cost

Google Video Intelligence

Label Detection - Detect entities within the video, such as "dog", "flower" or "car"
Shot Change Detection - Detect scene changes within the video
Explicit Content Detection - Detect adult content within a video
Video Transcription - Automatically transcribes video content in English

Google Vision API

Object recognition: detect broad sets of categories within an image, ranging from modes of transportation to animals
Facial sentiment and logos: Analyze facial features to detect emotions: joy, sorrow, anger; detect logos
Extract text: detect and extract text within an image, with support of many languages and automatic language identification
Detect inapropriate content: fetect different types of inappropriate content from adult to violent content

Experiments Frameworks

Tools to help you configure, organize, log and reproduce experiments

https://www.reddit.com/r/MachineLearning/comments/5gyzqj/d_how_do_you_keep_track_of_your_experiments/, 2017
How to Plan and Run Machine Learning Experiments Systematically by Jason Brownlee, 2017
- using a speadsheet with a template
https://github.com/IDSIA/sacred

Jupyter Notebook

Lobe

Lobe is an easy-to-use visual tool (no coding required) that lets you build custom deep learning models, quickly train them, and ship them directly in your app without writing any code.

Microsoft Azure Bot Service

Microsoft Azure Machine Learning

Microsoft Cognitive Services

Microsoft Cognitive Toolkit

Supervisely

Annotate images for computer vision tasks using AI
https://github.com/supervisely/supervisely

Syn Bot Oscova

finds similarity between the expressions
https://github.com/SynHub/syn-bot-samples
MS Visual Studio is required (doesn't work with VS Code)
activating Deep Learning feature requires license activating
number of requests to the server is limited by the license

Tableau

Data visualization tool created by Tableau Software.
Connects to files, relational and Big Data sources, allows transforming data into dashboards that look amazing and are also interactive.

TensorFlow

TensorFlow Hub
https://github.com/tensorflow/models/tree/master/research
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples
- https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android
  - TF Classify
  - TF Detect
  - TF Stylize
  - TF Speech
- https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/examples
  - TF Classify
  - TF Detect
  - TF Speech
- https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/java/demo
  - TF classify using tflite model
- Freeze tensorflow model graph
TensorFlow Estimator APIs Tutorials

Turi Create

Apple python framework that simplifies the development of custom machine learning models. You don't have to be a machine learning expert to add recommendations, object detection, image classification, image similarity or activity classification to your app.

Export models to Core ML for use in iOS, macOS, watchOS, and tvOS apps.
A Guide to Turi Create from wwdc2018

Playgrounds

Training Data Analyst - Labs and demos for Google Cloud Platform courses
SEEDBANK - Collection of Interactive Machine Learning Examples
AI Lab: Learn to Code with the Cutting-Edge Microsoft AI Platform, 2018
Teachable Machine by Google

Google AIY

Vision Kit - Do-it-yourself intelligent camera. Experiment with image recognition using neural networks on Raspberry Pi.
Voice Kit - Do-it-yourself intelligent speaker. Experiment with voice recognition and the Google Assistant on Raspberry Pi.

IDEs

Repositories

https://github.com/bulutyazilim/awesome-datascience

Models

Decision Trees

Pros:

can model nonlinearities
are highly interpretable
do not require extensive feature preprocessing
do not require enormous data sets

Cons:

tend to overfit
- fixed by building a decision forest with boosting
unstable/undeterministic (generate different results while trained on the same data)
- fixed by using bootstrap aggregation/bagging (a boosted forest)
do mapping directly from the raw input to the label
- better use neural nets that can learn intermediate representations

Hyperparameters:

tree depth
maximum number of leaf nodes

Distillation

trains a model to mimic the behavior of a pretrained model so it can work independently of the pretrained model
can train the smaller model with unlabeled examples
not all target classes need to be represented in the distillation training set
reduces the need for regularization
Distilling the Knowledge in a Neural Network by Geoffrey Hinton et al, 2015
“Why Should I Trust You?” Explaining the Predictions of Any Classifier by Marco Tulio Ribeiro et al, 2016
Detecting Bias in Black-Box Models Using Transparent Model Distillation by Sarah Tan et al, 2017

Embedding models

https://github.com/Hironsan/awesome-embedding-models
gensim's word2vec (embedded words and phrases)
- online vocaburary update tutorial
- How to Develop Word Embeddings in Python with Gensim
gensim's doc2vec
https://github.com/jhlau/doc2vec
see recursive autoencoders
see bag-of-words models

Evolutionary Algorithms

Metrics of dataset quality

Statistical metrics
- descriptive statistics: dimensionality, unique subject counts, systematic replicates counts, pdfs, cdfs (probability and cumulative distribution fx's)
- cohort design
- power analysis
- sensitivity analysis
- multiple testing correction analysis
- dynamic range sensitivity
Numerical analysis metrics
- number of clusters
- PCA dimensions
- MDS space dimensions/distances/curves/surfaces
- variance between buckets/bags/trees/branches
- informative/discriminative indices (i.e. how much does the top 10 features differ from one another and the group)
- feature engineering differnetiators

Neural Networks

Approaches when our model doesn’t work:

Fetch more data
Add more layers to Neural Network
Try some new approach in Neural Network
Train longer (increase the number of iterations)
Change batch size
Try Regularisation
Check Bias Variance trade-off to avoid under and overfitting
Use more GPUs for faster computation

Back-propagation problems:

it requires labeled training data; while almost all data is unlabeled
the learning time does not scale well, which means it is very slow in networks with multiple hidden layers
it can get stuck in poor local optima, so for deep nets they are far from optimal.

Capsule Networks

Convolutional Neural Networks

Deep Residual Networks

Understand Deep Residual Networks — a simple, modular learning framework that has redefined state-of-the-art, 2017

Distributed Neural Networks

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer by Jeff Dean et al
PathNet: Evolution Channels Gradient Descent in Super Neural Networks by deepmind
Feature extraction - uses layers of a pretrained model as inputs to another model, effectively chaining two models together

Feed-Forward Neural Networks

Perceptrons

Gated Recurrent Neural Networks

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling by Junyoung Chung et al, 2014

Generative Adversarial Networks

Progressive Growing of GANs for Improved Quality, Stability, and Variation by Tero Karras et al, 2017

Long-Short Term Memory Networks

Exploring LSTMs, 2017
Understanding LSTM Networks by Christopher Olah, 2015
- “Almost all exciting results based on recurrent neural networks are achieved with [LSTMs].”
Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks by Graves & Schmidhuber, 2009
- showed that RNNs with LSTM are currently the best systems for reading cursive writing
LONG SHORT-TERM MEMORY by Hochreiter & Schmidhuber, 1997

Recurrent Neural Networks

Symmetrically Connected Networks

Hopfield Nets (without hidden units)
- Neural networks and physical systems with emergent collective computational abilities by Hopfield, 1982
Boltzmann machines (stochastic recurrent neural network with hidden units)
Restricted Boltzmann Machines by Salakhutdinov and Hinton, 2014
Deep Boltzmann Machines by Salakhutdinov and Hinton, 2012

Reinforcement Learning

Guidelines

Stanford CS-230 cheatsheets
- The top concepts of Deep Learning, CNNs and RNNs summarized in 3 short pages
AI Transformation Playbook by Andrew Ng, 2018
- Steps for transforming your enterprise with AI, which I will explain in this playbook:
  - Execute pilot projects to gain momentum
  - Build an in-house AI team
  - Provide broad AI training
  - Develop an AI strategy
  - Develop internal and external communications
AI at Google: our principles, 2018
Rules of Machine Learning: Best Practices for ML Engineering by Martin Zinkevich, 2018
Practical advice for analysis of large, complex data sets by PATRICK RILEY, 2016
What’s your ML test score? A rubric for ML production systems by Eric Breck, 2016
Machine Learning: The High Interest Credit Card of Technical Debt by D. Sculley et al, 2014
- Complex Models Erode Boundaries
  - Entanglement
  - Hidden Feedback Loops
  - Undeclared Consumers
- Data Dependencies Cost More than Code Dependencies
  - Unstable Data Dependencies
  - Underutilized Data Dependencies
  - Static Analysis of Data Dependencies
  - Correction Cascades
- System-level Spaghetti
  - Glue Code
  - Pipeline Jungles
  - Dead Experimental Codepaths
  - Configuration Debt
- Dealing with Changes in the External World
  - Fixed Thresholds in Dynamic Systems
  - When Correlations No Longer Correlate
  - Monitoring and Testing
Principles of Research Code by Charles Sutton, 2012
Patterns for Research in Machine Learning by Ali Eslami, 2012
Lessons learned developing a practical large scale machine learning system by Simon Tong, 2010
The Professional Data Science Manifesto
Machine Learning Glossary

Deep learning

Deep Learning: A Critical Appraisal by Gary Marcus, 2018
- Deep learning thus far is data hungry
- Deep learning thus far is shallow and has limited capacity for transfer
- Deep learning thus far has no natural way to deal with hierarchical structure
- Deep learning thus far has struggled with open-ended inference
- Deep learning thus far is not sufficiently transparent
- Deep learning thus far has not been well integrated with prior knowledge
- Deep learning thus far cannot inherently distinguish causation from correlation
- Deep learning presumes a largely stable world, in ways that may be problematic
- Deep learning thus far works well as an approximation, but its answers often cannot be fully trusted
- Deep learning thus far is difficult to engineer with
Software 2.0 by Andrej Karpathy, 2017

Interview preparation

MOOC

Google oriented courses

https://developers.google.com/machine-learning/crash-course/
- for beginners, explains hard things with simple words
- from google gurus
- uses TensorFlow and codelabs
https://www.coursera.org/specializations/gcp-data-machine-learning
- shows how to use GCP for machine learning

Books

NLP

Natural Language Processing with Python by Steven Bird et al, 2014

Statistics

Datasets

https://ai.google/tools/datasets/
https://toolbox.google.com/datasetsearch
- Making it easier to discover datasets, 2018
Microsoft Research Open Data
- users can also copy datasets directly to an Azure based Data Science virtual machine

3D

ScanNet - RGB-D video dataset annotated with 3D camera poses, surface reconstructions, and instance-level semantic segmentations
SceneNet - Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth

Audios

The VU sound corpus - based on https://freesound.org/ database
- See article The VU Sound Corpus by Emiel van Miltenburg et al
AudioSet - consists of an expanding ontology of 632 audio event classes and a collection of 2,084,320 human-labeled 10-second sound clips drawn from YouTube videos

Images

Conceptual Captions: A New Dataset and Challenge for Image Captioning, 2018
Landmarks 2018
ImageNet
COCO
SUN
Caltech 256
Pascal
CIFAR-10 - 60000 32x32 colour images in 10 classes, with 6000 images per class
- commonly used to train image classifiers

Videos

Microsoft multimedia challenge dataset, 2017
- largest dataset in terms of sentence and vocabulary
- challenge: to automatically generate a complete and natural sentence to describe video content
Kinetics, 2017
YouTube-8M, 2017
- large, but annotations are slightly noisy and only video-level labels have been assigned (include frames that do not relate to target actions)
- youtube-dl - Command-line program to download videos from YouTube.com and other video sites
Sports-1M by A. Karpathy, 2016
- large, but annotations are slightly noisy and only video-level labels have been assigned (include frames that do not relate to target actions)
FCVID
ActivityNet
http://crcv.ucf.edu/data/UCF101.php 2013
Hollywood2
HMDB-51
CCV

Research Groups

Cartoons

The Browser of a Data Scientist

Jokes

A statistician drowned crossing a river that was only three feet deep on average

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

lankastersky / neuromantic

Labels

Projects that are alternatives of or similar to neuromantic

TOC

Use Cases

3D Recognition

Semantic Segmentation

Audio Recognition

Speech to Text

Data Augmentation

Design

Games

Gesture Recognition

Using wearable sensors (phones, watches etc.)

Hyperparameter Tuning

Image Recognition

Face Recognition

Food Recognition

Image Captioning

Performance

Person Detection

Semantic Segmentation

Interpretability

Programming and ML

Predict defects

Predict performance

Searching code

Writing code

NLP

Chatbots

Crossword question answerers

Database queries

Named entity resolution

Reverse dictionaries

Sequence to sequence

Semantic analysis

Spelling

Summarization

Text classification

Text to Image

Text to Speech

Personality recognition

Robotics

Search

Transfer Learning

Uber

Video recognition

Pose recognition

Object detection

Scene Segmentation

Video Captioning

Video Classification

Visualization

Multiple Modalities

Open problems

Tools

Experiments Frameworks

Jupyter Notebook

Playgrounds

IDEs

Repositories

Models

Decision Trees

Distillation

Embedding models

Evolutionary Algorithms

Metrics of dataset quality

Neural Networks

Capsule Networks

Convolutional Neural Networks

Deep Residual Networks

Distributed Neural Networks

Feed-Forward Neural Networks

Gated Recurrent Neural Networks

Generative Adversarial Networks

Long-Short Term Memory Networks

Recurrent Neural Networks

Symmetrically Connected Networks

Reinforcement Learning