All Projects → hangtwenty → Dive Into Machine Learning

hangtwenty / Dive Into Machine Learning

Licence: cc-by-4.0
Dive into Machine Learning with Python Jupyter notebook and scikit-learn! First posted in 2016, maintained as of 2021. Pull requests welcome.

Projects that are alternatives of or similar to Dive Into Machine Learning

Virgilio
Virgilio is developed and maintained by these awesome people. You can email us virgilio.datascience (at) gmail.com or join the Discord chat.
Stars: ✭ 13,200 (+22.11%)
Mutual labels:  jupyter-notebook, data-science, learning, scikit-learn
Ds and ml projects
Data Science & Machine Learning projects and tutorials in python from beginner to advanced level.
Stars: ✭ 56 (-99.48%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Minerva Training Materials
Learn advanced data science on real-life, curated problems
Stars: ✭ 37 (-99.66%)
Mutual labels:  jupyter-notebook, data-science, ipython-notebook
Dat8
General Assembly's 2015 Data Science course in Washington, DC
Stars: ✭ 1,516 (-85.98%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Crime Analysis
Association Rule Mining from Spatial Data for Crime Analysis
Stars: ✭ 20 (-99.81%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Python for ml
brief introduction to Python for machine learning
Stars: ✭ 29 (-99.73%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Python Hierarchical Clustering Exercises
Exercises for hierarchical clustering with Python 3 and scipy as Jupyter Notebooks
Stars: ✭ 62 (-99.43%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Data Analysis And Machine Learning Projects
Repository of teaching materials, code, and data for my data analysis and machine learning projects.
Stars: ✭ 5,166 (-52.21%)
Mutual labels:  jupyter-notebook, data-science, ipython-notebook
Scikit Learn Tips
🤖⚡️ scikit-learn tips
Stars: ✭ 1,203 (-88.87%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Notebooks
A collection of Jupyter/IPython notebooks
Stars: ✭ 78 (-99.28%)
Mutual labels:  jupyter-notebook, data-science, ipython-notebook
Pymc Example Project
Example PyMC3 project for performing Bayesian data analysis using a probabilistic programming approach to machine learning.
Stars: ✭ 90 (-99.17%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Notebooks
Learn Python for free using open-source notebooks in Hebrew.
Stars: ✭ 877 (-91.89%)
Mutual labels:  jupyter-notebook, learning, learning-by-doing
Python Machine Learning Book 2nd Edition
The "Python Machine Learning (2nd edition)" book code repository and info resource
Stars: ✭ 6,422 (-40.59%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Machinelearningcourse
A collection of notebooks of my Machine Learning class written in python 3
Stars: ✭ 35 (-99.68%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Learn Julia The Hard Way
Learn Julia the hard way!
Stars: ✭ 679 (-93.72%)
Mutual labels:  data-science, learning, learning-by-doing
Data Science Cookbook
🎓 Jupyter notebooks from UFC data science course
Stars: ✭ 60 (-99.44%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Tutorials
All of the code for my Medium articles
Stars: ✭ 92 (-99.15%)
Mutual labels:  jupyter-notebook, learning, learning-by-doing
Thesemicolon
This repository contains Ipython notebooks and datasets for the data analytics youtube tutorials on The Semicolon.
Stars: ✭ 345 (-96.81%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Data Science Portfolio
Portfolio of data science projects completed by me for academic, self learning, and hobby purposes.
Stars: ✭ 559 (-94.83%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn
Hyperlearn
50% faster, 50% less RAM Machine Learning. Numba rewritten Sklearn. SVD, NNMF, PCA, LinearReg, RidgeReg, Randomized, Truncated SVD/PCA, CSR Matrices all 50+% faster
Stars: ✭ 1,204 (-88.86%)
Mutual labels:  jupyter-notebook, data-science, scikit-learn

Dive into Machine Learning Creative Commons License Awesome

Hi there! This guide is for you if:

If that's you, join me in getting a bit ahead of yourself, and see if you want to learn more about the field. (For alternatives, jump to the end or check out Nam Vu's guide, Machine Learning for Software Engineers.)

Of course, there is no easy path to expertise. Also, I'm not an expert! I just want to connect you with some great resources from experts. Applications of ML are all around us. I think it's in the public interest that more of us learn more about ML.

Whatever motivates you to dive into machine learning, if you know a bit of Python, these days you can get hands-on with a machine learning "Hello World!" in less than an hour.

Let's get started

Tools you'll need

If you prefer local installation

  • Python. Python 3 is the best option.
  • Jupyter Notebook. (Formerly known as IPython Notebook.)
  • Some scientific computing packages:
    • numpy
    • pandas
    • scikit-learn
    • matplotlib

You can install Python 3 and all of these packages in a few clicks with the Anaconda Python distribution. Anaconda is popular in Data Science and Machine Learning communities. (Use whichever tool you want.)

Cloud-based options

Some options you can use from your browser:

For other options, see:

Let's go!

Learn how to use Jupyter Notebook (5-10 minutes). (You can learn by screencast instead.)

Now, follow along with this brief exercise: An introduction to machine learning with scikit-learn. Do it in ipython or Jupyter Notebook. It'll really boost your confidence.

I'll wait.

What just happened?

You just classified some hand-written digits using scikit-learn. Neat huh?

Dive in

A Visual Introduction to Machine Learning

Let's learn a bit more about Machine Learning, and a couple of common ideas and concerns. Read "A Visual Introduction to Machine Learning, Part 1" by Stephanie Yee and Tony Chu.

A Visual Introduction to Machine Learning, Part 1

It won't take long. It's a beautiful introduction ... Try not to drool too much!

A Few Useful Things to Know about Machine Learning

OK. Let's dive deeper.

Read "A Few Useful Things to Know about Machine Learning" by Prof. Pedro Domingos. It's densely packed with valuable information, but not opaque.

Take your time with this one. Take notes. Don't worry if you don't understand it all yet.

The whole paper is packed with value, but I want to call out two points:

  • Data alone is not enough. This is where science meets art in machine-learning. Quoting Domingos: "... the need for knowledge in learning should not be surprising. Machine learning is not magic; it can’t get something from nothing. What it does is get more from less. Programming, like all engineering, is a lot of work: we have to build everything from scratch. Learning is more like farming, which lets nature do most of the work. Farmers combine seeds with nutrients to grow crops. Learners combine knowledge with data to grow programs."
  • More data can beat a cleverer algorithm. Listen up, programmers. We like cool tools. Resist the temptation to reinvent the wheel, or to over-engineer solutions. Your starting point is to Do the Simplest Thing that Could Possibly Work. Quoting Domingos: "Suppose you’ve constructed the best set of features you can, but the classifiers you’re getting are still not accurate enough. What can you do now? There are two main choices: design a better learning algorithm, or gather more data. [...] As a rule of thumb, a dumb algorithm with lots and lots of data beats a clever one with modest amounts of it. (After all, machine learning is all about letting data do the heavy lifting.)"

When you work on a real Machine Learning problem, you should focus your efforts on your domain knowledge and data before optimizing your choice of algorithms. Prefer to do simple things until you have to increase complexity. You should not rush into neural networks because you think they're cool. To improve your model, get more data. Then use your knowledge of the problem to explore and process the data. You should only optimize the choice of algorithms after you have gathered enough data, and you've processed it well.

Jargon note

Just about time for a break...

Totally optional: some podcast episodes of note

First, download an interview with Prof. Domingos on the _Data Skeptic_podcast (2018). Prof. Domingos wrote the paper we read earlier. You might also start reading his book, The Master Algorithm by Prof. Pedro Domingos, a clear and accessible overview of machine learning. (It's available as an audiobook too.)

Next, subscribe to more machine learning and data science podcasts! These are great, low-effort resources that you can casually learn more from. To learn effectively, listen over time, with plenty of headspace. By the way, don't speed up technical podcasts, that can hinder your comprehension.

Subscribe to Talking Machines.

I suggest this listening order:

  • Download the "Starting Simple" episode, and listen to that soon. It supports what we read from Domingos. Ryan Adams talks about starting simple, as we discussed above. Adams also stresses the importance of feature engineering. Feature engineering is an exercise of the "knowledge" Domingos writes about. In a later episode, they share many concrete tips for feature engineering.
  • Then, over time, you can listen to the entire podcast series (start from the beginning).

Want to subscribe to more podcasts? Here's a good listicle of suggestions, and another.

OK! Take a break, come back refreshed.


Play to learn

Next, play along from one or more of notebooks.

  • Dr. Randal Olson's Example Machine Learning notebook: "let's pretend we're working for a startup that just got funded to create a smartphone app that automatically identifies species of flowers from pictures taken on the smartphone. We've been tasked by our head of data science to create a demo machine learning model that takes four measurements from the flowers (sepal length, sepal width, petal length, and petal width) and identifies the species based on those measurements alone."
  • Various notebooks by topic:
  • Notebooks in a series:
    • ageron/handson-ml2 - "Jupyter notebooks that walk you through the fundamentals of Machine Learning and Deep Learning in Python." Scikit-Learn, Keras, TensorFlow 2.

Find more great Jupyter Notebooks when you're ready:


Immerse yourself

Now you should be hooked, and hungry to learn more. Pick one of the courses below and start on your way.

Recommended course: Prof. Andrew Ng's Machine Learning on Coursera

Prof. Andrew Ng's Machine Learning is a popular and esteemed free online course. I've seen it recommended often. And emphatically.

You might like to have a pet project to play with, on the side. When you are ready for that, you could explore one of these Awesome Public Datasets.

Also, you should grab an in-depth textbook to use as a reference. The two recommendations I saw repeatedly were Understanding Machine Learning and Elements of Statistical Learning. You only need to use one of the two options as your main reference; here's some context/comparison to help you pick which one is right for you. You can download each book free as PDFs at those links - so grab them!

Tips for studying

Take my tips with a grain of salt

I work as a software developer, and I am not a machine learning expert. Ultimately I decided not to pursue specializing in ML, and instead it's a side interest. I don't regret investing time in learning about ML and I don't think you will, either.

Other courses

Here are some other free online courses I've seen recommended. (Machine Learning, Data Science, and related topics.)

Getting Help: Questions, Answers, Chats

Start with the support forums and chats related to the course(s) you're taking.

Check out datascience.stackexchange.com and stats.stackexchange.com – such as the tag, machine-learning. There are some subreddits, like /r/LearningMachineLearning and /r/MachineLearning.

Don't forget about meetups. Also, nowadays there are many active and helpful online communities around the ML ecosystem. Look for chat invitations on project pages and so on.

Supplement: Learning Pandas well

You'll want to get more familiar with Pandas.

Supplement: Cheat Sheets

Some good cheat sheets I've come across. (Please submit a Pull Request to add other useful cheat sheets.)

Assorted Tips and Resources

Risks

"Machine learning systems automatically learn programs from data." Pedro Domingos, in "A Few Useful Things to Know about Machine Learning." The programs you generate will require maintenance. Like any way of creating programs faster, you can rack up technical debt.

Here is the abstract of Machine Learning: The High-Interest Credit Card of Technical Debt:

Machine learning offers a fantastically powerful toolkit for building complex systems quickly. This paper argues that it is dangerous to think of these quick wins as coming for free. Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning. The goal of this paper is highlight several machine learning specific risk factors and design patterns to be avoided or refactored where possible. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, changes in the external world, and a variety of system-level anti-patterns.

If you're following this guide, you should read that paper. You can also listen to a podcast episode interviewing one of the authors of this paper.

Welcome to the Danger Zone

So you are dabbling with Machine Learning. You've got Hacking Skills. Maybe you've got some "knowledge" in Domingos' sense (some "Substantive Expertise" or "Domain Knowledge"). This diagram is modified slightly from Drew Conway's "Data Science Venn Diagram." It isn't a perfect fit for our purposes here, but it might get the point across:

Drew Conway's Data Science Venn Diagram, modified slightly

Please don't sell yourself as a Machine Learning expert while you're still in the Danger Zone. Don't build bad products or publish junk science. Please do bookmark the Institute for Ethical AI & Machine Learning's Responsible Machine Learning Principles. Thank you, you're awesome.

Skilling up

What are some ways to practice?

One way: competitions and challenges

You need practice. On Hacker News, user olympus commented to say you could use competitions to practice and evaluate yourself. Kaggle and ChaLearn are hubs for Machine Learning competitions. (You can find more competitions here or here.)

You also need understanding. You should review what Kaggle competition winners say about their solutions, for example, the "No Free Hunch" blog. These might be over your head at first but once you're starting to understand and appreciate these, you know you're getting somewhere.

Competitions and challenges are just one way to practice. Remember, Machine Learning isn't just about Kaggle competitions.

Another way: try doing some practice studies

Here's a complementary way to practice: do practice studies.

  1. Ask a question. Start exploring some data. The "most important thing in data science is the question" (Dr. Jeff T. Leek). So start with a question. Then, find real data. Analyze it. Then ...
  2. Communicate results. When you think you have a novel finding, ask for review. When you're still learning, ask in informal communities (some are linked below).
  3. Learn from feedback. Consider learning in public, it works great for some folks. (Don't pressure yourself though! Do what works for you.)

How can you come up with interesting questions? Here's one way. Pick a day each week to look for public datasets and write down some questions that come to mind. Also, sign up for Data is Plural, a newsletter of interesting datasets. When a question inspires you, try exploring it with the skills you're learning.

This advice, to do practice studies and learn from review, is based on a conversation with Dr. Randal S. Olson. Here's more advice from Olson, quoted with permission:

I think the best advice is to tell people to always present their methods clearly and to avoid over-interpreting their results. Part of being an expert is knowing that there's rarely a clear answer, especially when you're working with real data.

As you repeat this process, your practice studies will become more scientific, interesting, and focused. Also, here's a video about the scientific method in data science.)

Some communities to know about

Great places to ask for feedback informally, and while you are learning:

Peer review

OpenReview.net "aims to promote openness in scientific communication, particularly the peer review process."

  • Open Peer Review: We provide a configurable platform for peer review that generalizes over many subtle gradations of openness, allowing conference organizers, journals, and other "reviewing entities" to configure the specific policy of their choice. We intend to act as a testbed for different policies, to help scientific communities experiment with open scholarship while addressing legitimate concerns regarding confidentiality, attribution, and bias.
  • Open Publishing: Track submissions, coordinate the efforts of editors, reviewers and authors, and host… Sharded and distributed for speed and reliability.
  • Open Access: Free access to papers for all, free paper submissions. No fees.
More about OpenReview.net
  • Open Discussion: Hosting of accepted papers, with their reviews, comments. Continued discussion forum associated with the paper post acceptance. Publication venue chairs/editors can control structure of review/comment forms, read/write access, and its timing.
  • Open Directory: Collection of people, with conflict-of-interest information, including institutions and relations, such as co-authors, co-PIs, co-workers, advisors/advisees, and family connections.
  • Open Recommendations: Models of scientific topics and expertise. Directory of people includes scientific expertise. Reviewer-paper matching for conferences with thousands of submissions, incorporating expertise, bidding, constraints, and reviewer balancing of various sorts. Paper recommendation to users.
  • Open API: We provide a simple REST API [...]
  • Open Source: We are committed to open source. Many parts of OpenReview are already in the OpenReview organization on GitHub. Some further releases are pending a professional security review of the codebase.
  • OpenReview.net is created by Andrew McCallum’s Information Extraction and Synthesis Laboratory in the College of Information and Computer Sciences at University of Massachusetts Amherst

  • OpenReview.net is built over an earlier version described in the paper Open Scholarship and Peer Review: a Time for Experimentation published in the ICML 2013 Peer Review Workshop.

  • OpenReview is a long-term project to advance science through improved peer review, with legal nonprofit status through Code for Science & Society. We gratefully acknowledge the support of the great diversity of OpenReview Sponsors––scientific peer review is sacrosanct, and should not be owned by any one sponsor.

Production, Deployment, MLOps

If you are learning about MLOps but find it overwhelming, these resources might help you get your bearings:

Recommended awesomelists to save/star/watch:


Deep Learning

In early editions of this guide, there was no specific "Deep Learning" section. There are experts in the field who warn against jumping too far ahead.

Maybe this is a way to check your progress: ask yourself, does Deep Learning seem like magic? If so, take that as a sign that you aren't ready to work with it professionally, and let the fascination motivate you to learn more. I have read some argue you can learn Deep Learning in isolation; I have read others recommend it's best to master traditional Machine Learning first. Why not start with traditional Machine Learning, and develop your reasoning and intuition there? You'll only have an easier time learning Deep Learning after that. After all of it, you'll able to tackle all sorts of interesting problems.

In any case, when you're ready to dive into Deep Learning, here are some helpful resources.

Easier sharing of deep learning models and demos

  • replicate.ai "makes it easy to share a running machine learning model" for the sake of reproducible research.
    • For beginners, you can try some models using GANs and other Deep Learning concepts.
    • Join/bookmark so you can contribute a model of your own later on
    • Open source tools: cog, keepsake

Collaborate with Domain Experts

Machine Learning can be powerful, but it is not magic.

Whenever you apply Machine Learning to solve a problem, you are going to be working in some specific problem domain. To get good results, you or your team will need "substantive expertise" AKA "domain knowledge." Learn what you can, for yourself... But you should also collaborate. You'll have better results if you collaborate with domain experts.

Machine Learning and User Experience (UX)

I couldn't say it better:

Machine learning won’t figure out what problems to solve. If you aren’t aligned with a human need, you’re just going to build a very powerful system to address a very small—or perhaps nonexistent—problem.

That quote is from "The UX of AI" by Josh Lovejoy. In other words, You Are Not The User. Suggested reading: Martin Zinkevich's "Rules of ML Engineering", Rule #23: "You are not a typical end user"


Big data

Here are some useful links regarding Big Data and ML.

See also: the MLOps section!

If you are working with data-intensive applications at all, I'll recommend this book:

  • Designing Data-Intensive Applications by Martin Kleppman. (You can start reading it online, free, via Safari Books.) It's not specific to Machine Learning, but you can bridge that gap yourself.

More Data Science materials

Here are some additional Data Science resources:

Aside: Bayesian Statistics and Machine Learning

From the "Bayesian Machine Learning" overview on Metacademy:

... Bayesian ideas have had a big impact in machine learning in the past 20 years or so because of the flexibility they provide in building structured models of real world phenomena. Algorithmic advances and increasing computational resources have made it possible to fit rich, highly structured models which were previously considered intractable.

Here are some awesome resources for learning Bayesian methods.

Finding Open-Source Libraries

(↑ Back to top)

Non-sequitur

These next two resources are not about ML, but since you're here I know you love to learn, too.


More ways to "Dive into Machine Learning"

Here are some other guides to learning Machine Learning. They can be alternatives or supplements to this guide.

(↑ Back to top)

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].