Deep-blogs

Many well-curated lists of deep learning resources are available through repositories. They included course materials, papers and others. However, researchers' well-written blogs are lacked of information. I also study deep learning through online courses and papers, moreover blog materials have brought easily understanding of algorithms and implemenetations. In this repository, I especially listed blogs of researchers related to deep learning. I also recommend to go over open-project website: GitXiv.

Online Courses

• IPAM Summer School [[web]] (http://www.ipam.ucla.edu/programs/summer-schools/graduate-summer-school-deep-learning-feature-learning/?tab=schedule)
  • Popular researchers and professors in deep learning field teaches as a summer school. Although the materials are not the latest, tutorial contents are very beneficial.
• IPAM Summer School: Tutorials on Theano/Torch [[web]] (https://github.com/clementfarabet/ipam-tutorials)
  • Opensource of deep learning frameworks are introduced. They are introduced at one of the programs in above summer school.
• Deep Learning: Yann LeCun (2015-2016) [[web]] (https://www.college-de-france.fr/site/en-yann-lecun/course-2015-2016.htm)
  • Deep learning courses taught by Yann LeCun. English and French are provided. Lectures cover MLP, CNN, NLP, and unsupervised learning.
• University of Toronto: CSC2541 - Differential Inference and Generative Models [[web]] (http://www.cs.toronto.edu/~duvenaud/courses/csc2541/index.html)
  • The latest course related to generative models is introduced. Unfortunately, lecture video is not provided. However, its selected papers and several lecture notes are very beneficial. I think trying to read papers shown this material is good start point to study generative models.
• ELEG 5040: Advanced Topics in Signal Processing [[web]] (https://piazza.com/cuhk.edu.hk/spring2015/eleg5040/resources)
• Resources related to deep learning are provided.

Blogs

• Andrej Karpathy: Hacker's guide to Neural Networks [[web]] (http://karpathy.github.io/neuralnets/)
  • This is one of the posts of Andrej Karpathy's. I think this should be in must-read list.
• Brandon Amos: Ph.D students in CMU [[web]] (http://bamos.github.io/)
  • Blog post: <Image Completion with Deep Learning in TensorFlow> was very beneficial.
• Clément thorey: What does the gradient flowing through batch normalization looks like ? [[web]] (http://cthorey.github.io/backpropagation/)
  • Review and explanation of batch normalization
• Two Sigma: A Survey of Selected Papers on Deep Learning at ICML 2016 [[web]] (https://www.twosigma.com/insights/a-survey-of-selected-papers-on-deep-learning-at-icml-2016)
  • To grasp latest research on deep learning, recent papers are needed to be read. To save time, these kinds of paper survey can be a reference.
• Abubakar Abid: Introduction to Deep Learning in Medicine and Biology [[web]] (http://a12d.com/deep-learning-biomedicine)
  • Interesting post of using deep learning in medicine and biology fields.
• Yarin Gal: University of Cambridge (http://mlg.eng.cam.ac.uk/yarin/blog.html)

Computer Vision

• Adit Deshpande: CS Undergrad at UCLA ('19) [[web]] (https://adeshpande3.github.io/adeshpande3.github.io/)
  • Introduction of the state-of-the-art ConvNets, GANS, and Reinforcement learnings are posted.
• Aaditya Prakash: One by One [ 1 x 1 ] Convolution [[web]] (http://iamaaditya.github.io/2016/03/one-by-one-convolution/)
  • Why use 1x1 convolution in deep learning architecture is explained.
• Arthur JulianiFollow: Cognitive Scientist & AI Researcher [[web]] (https://chatbotslife.com/resnets-highwaynets-and-densenets-oh-my-9bb15918ee32#.n12cjesup)
  • State-of-the-art deep learning networks including ResNet and HighwayNets are posted with detailed explanation and implementation using TensorFlow. Other posts of this blog are also very beneficial.
• Distill: Deconvolution and Checkerboard Artifacts [[web]] (http://distill.pub/2016/deconv-checkerboard/)
  • Great post of deconvolution comparison. The post is written by Google Brain researcher. I think this post is in must-read list.
• Explained Visually: Image Kernels [[web]] (http://setosa.io/ev/image-kernels/)
  • Although deep learning is not explained in this post, included content of image kernels is beneficial.
• Understanding Neural Networks Through Deep Visualization [[web]] (http://yosinski.com/deepvis)
  • This post explains about visualizing outputs of inner-layers in ConvNets. The paper is released from ICML.

Unsupervised Learning

• OpenAI: Generative Models [[web]] (https://openai.com/blog/generative-models/)
• Nice explanation of generative model and its various versions.
• Mind, etc.: Generating Faces with Deconvolution Networks [[web]] (https://zo7.github.io/blog/2016/09/25/generating-faces.html)
  • Generating faces is one of the applications of generative models.
• Generative Adversarial Networks Explained with a Classic Spongebob Squarepants Episode [[web]] (https://medium.com/@awjuliani/generative-adversarial-networks-explained-with-a-classic-spongebob-squarepants-episode-54deab2fce39#.jiv4fnrno)
  • This post explains generative adversarial nets using Spongebob metaphor. It has a tutorial implementing GAN on Tensorflow.
• Learning Interpretable Latent Representations with InfoGAN [[web]] (https://medium.com/emergent-future/learning-interpretable-latent-representations-with-infogan-dd710852db46#.8b1gr8jmc)
  • This post explains extension of GANs called InfoGAN. It has a tutorial implementing InfoGAN on Tensorflow.
• Mark Swarbrick Jones: Generative adversarial autoencoders in Theano [[web]] (https://swarbrickjones.wordpress.com/2016/01/24/generative-adversarial-autoencoders-in-theano/)
  • Brief information of generative adversarial autoencoders
• FAIR: The eyescream project [[web]] (http://soumith.ch/eyescream/)
• Facebook genearting natural images project
• Jan Hendrik Metzen: Variational Autoencoder in TensorFlow [[web]] (https://jmetzen.github.io/2015-11-27/vae.html)
  • Implementation of VAE using TensorFlow is explained step by step.
• Fast Forward Labs: Under the Hood of the Variational Autoencoder [[web]] (http://blog.fastforwardlabs.com/post/149329060653/under-the-hood-of-the-variational-autoencoder-in)
  • Explanation and implementation of VAE using TensorFlow are posted by machine intelligence research company called Fast Forward Labs.
• Introducing Neural Dream Videos [[web]] (https://medium.com/@awjuliani/introducing-neural-dream-videos-5d517b3cc804#.skva5r1zp)
  • This post introduce about neural deam videos, which is combined with variational autoencoder and recurrent neural network. Implementation of it is included as Github address.
• What is variational autoencoders? [[web]] (https://jaan.io/what-is-variational-autoencoder-vae/)
  • This post explicitly explains about VAE. Excellent introduction to study VAE.
• Introduction to Autoencoders [[web]] (https://pgaleone.eu/neural-networks/2016/11/18/introduction-to-autoencoders/)
• Convolutional Autoencoders [[web]] (https://pgaleone.eu/neural-networks/2016/11/24/convolutional-autoencoders/)

Natural Language Processing

• Vered Shwartz: Probably Approximately a Scientific Blog [[web]] (http://veredshwartz.blogspot.kr/2016/01/representing-words.html)
  • This post explains methods of representing words which is mostly used in natural language processing.

Reinforcement Learning

• Ruben Fiszel's website: Reinforcement Learning and DQN, learning to play from pixels [[web]] (https://rubenfiszel.github.io/posts/rl4j/2016-08-24-Reinforcement-Learning-and-DQN.html)
  • Post of reinforcement learning and DQN with detailed explanation and implementation.

Papers

• Patrick Emami: Paper Summaries [[web]] (http://pemami4911.github.io/)
  • Summaries of papers related to machine learning and various topics concerning artificial intelligence are posted.

Frameworks

• Quantitative Journey: Beginner Tutorial: Neural Nets in Theano [[web]] (http://outlace.com/Beginner-Tutorial-Theano/)
  • Theano tutorial of neural nets
• NVIDIA: Understanding Natural Language with Deep Neural Networks Using Torch [[web]] (https://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/)
  • Explanation and implementation of natural language processing are posted. Torch is used.

Others

• Jeremy D. Jackson: Deep Learning [[web]] (http://www.jeremydjacksonphd.com/category/deep-learning/)
  • Curated list of deep learning resources.
• Handong1587: Image Generation [[web]] (https://handong1587.github.io/deep_learning/2015/10/09/image-generation.html)
  • Curated list of image generation by Handong University.
• Rodrigob: Classification datasets results [[web]] (http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html)
  • Current state of the art in object classification is collected. Well-known datasets: MNIST, CIFAR-10, CIFAR-100, STL-10, SVHN, ILSVRC2012 task 1 are shown.
• SciRate: Computer Vision and Pattern Recognition (cs.CV) [[web]] (https://scirate.com/arxiv/cs.CV)
  • Lists of CVPR papers with abstracts.
• Adam Harley: 2D Visualization of a Convolutional Neural Network [[web]] (http://scs.ryerson.ca/~aharley/vis/conv/flat.html)
  • Visual intuition of convolutional neural network
• Reddit: Machine Learning [[web]] (https://www.reddit.com/r/MachineLearning/)
  • Machine learning channel in Reddit.