GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → umbertogriffo → Minimalistic-Multiple-Layer-Neural-Network-from-Scratch-in-Python

umbertogriffo / Minimalistic-Multiple-Layer-Neural-Network-from-Scratch-in-Python

Licence: other
Minimalistic Multiple Layer Neural Network from Scratch in Python.

Programming Languages

python
139335 projects - #7 most used programming language

Labels

deep-neural-networksdeep-learningneural-networkregressionneural-networksdeep-learning-algorithmsclassificationperceptrongradient-descentbackpropagation-learning-algorithmbackpropagationneural-network-exampledepplearning

Projects that are alternatives of or similar to Minimalistic-Multiple-Layer-Neural-Network-from-Scratch-in-Python

Deep-Learning-Coursera
Projects from the Deep Learning Specialization from deeplearning.ai provided by Coursera
Stars: ✭ 123 (+412.5%)
Mutual labels:  gradient-descent, backpropagation-learning-algorithm, backpropagation
Go Deep
Artificial Neural Network
Stars: ✭ 303 (+1162.5%)
Mutual labels:  regression, classification, backpropagation
Machine Learning Algorithms
A curated list of almost all machine learning algorithms and deep learning algorithms grouped by category.
Stars: ✭ 92 (+283.33%)
Mutual labels:  regression, deep-learning-algorithms, classification
Machine-Learning-in-Python-Workshop
My workshop on machine learning using python language to implement different algorithms
Stars: ✭ 89 (+270.83%)
Mutual labels:  gradient-descent, backpropagation
machine learning from scratch matlab python
Vectorized Machine Learning in Python 🐍 From Scratch
Stars: ✭ 28 (+16.67%)
Mutual labels:  regression, classification
onelearn
Online machine learning methods
Stars: ✭ 14 (-41.67%)
Mutual labels:  regression, classification
R-Machine-Learning
D-Lab's 6 hour introduction to machine learning in R. Learn the fundamentals of machine learning, regression, and classification, using tidymodels in R.
Stars: ✭ 27 (+12.5%)
Mutual labels:  regression, classification
Python-Machine-Learning-Fundamentals
D-Lab's 6 hour introduction to machine learning in Python. Learn how to perform classification, regression, clustering, and do model selection using scikit-learn and TPOT.
Stars: ✭ 46 (+91.67%)
Mutual labels:  regression, classification
Image-Classifier
Final Project of the Udacity AI Programming with Python Nanodegree
Stars: ✭ 63 (+162.5%)
Mutual labels:  gradient-descent, backpropagation-learning-algorithm
stg
Python/R library for feature selection in neural nets. ("Feature selection using Stochastic Gates", ICML 2020)
Stars: ✭ 47 (+95.83%)
Mutual labels:  regression, classification
wymlp
tiny fast portable real-time deep neural network for regression and classification within 50 LOC.
Stars: ✭ 36 (+50%)
Mutual labels:  regression, classification
ugtm
ugtm: a Python package for Generative Topographic Mapping
Stars: ✭ 34 (+41.67%)
Mutual labels:  regression, classification
Python-Machine-Learning
Python Machine Learning Algorithms
Stars: ✭ 80 (+233.33%)
Mutual labels:  regression, classification
data-science-notes
Open-source project hosted at https://makeuseofdata.com to crowdsource a robust collection of notes related to data science (math, visualization, modeling, etc)
Stars: ✭ 52 (+116.67%)
Mutual labels:  regression, classification
InstantDL
InstantDL: An easy and convenient deep learning pipeline for image segmentation and classification
Stars: ✭ 33 (+37.5%)
Mutual labels:  regression, classification
Machine-Learning-Specialization
Project work and Assignments for Machine learning specialization course on Coursera by University of washington
Stars: ✭ 27 (+12.5%)
Mutual labels:  regression, classification
Predictive-Maintenance-of-Aircraft-Engine
In this project I aim to apply Various Predictive Maintenance Techniques to accurately predict the impending failure of an aircraft turbofan engine.
Stars: ✭ 48 (+100%)
Mutual labels:  regression, classification
pywedge
Makes Interactive Chart Widget, Cleans raw data, Runs baseline models, Interactive hyperparameter tuning & tracking
Stars: ✭ 49 (+104.17%)
Mutual labels:  regression, classification
Regression
Multiple Regression Package for PHP
Stars: ✭ 88 (+266.67%)
Mutual labels:  regression, gradient-descent
Character-recognition-by-neural-network
Back Propagation, Python
Stars: ✭ 32 (+33.33%)
Mutual labels:  backpropagation-learning-algorithm, backpropagation
View All Similar Projects ➔

Minimalistic Multiple Layer Neural Network from Scratch in Python

  • Author: Umberto Griffo

Inspired by [1] and [2] I implemented a Minimalistic Multiple Layer Neural Network from Scratch in Python. You can use It to better understand the core concepts of Neural Networks.

Software Environment

  • Python 3.0 - 3.5

Features

  • Backpropagation Algorithm With Stochastic Gradient Descent. During training we are using single training examples for one forward/backward pass.
  • Supporting multiple hidden layers.
  • Classification (MultilayerNnClassifier.py).
  • Regression (MultilayerNnRegressor.py).
  • Activation Function: Linear, ReLU, Sigmoid, Tanh.
  • Classification Evaluator: Accuracy.
  • Regression Evaluator: Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Coefficient of Determination (R^2).

Demo

If you run Test.py you can see the following textual menu:

Please enter one of following numbers: 
 0 - Classification on Seed Dataset
 1 - Classification on Wine Red Dataset
 2 - Classification on Pokemon Dataset
 3 - Regression on Wine White Dataset
 4 - Regression on Wine Red Dataset

If you choose 2 will be performed a classification task on Pokemon Dataset:

2
You entered 2
>epoch=0, lrate=0.100, error=0.396
>epoch=100, lrate=0.100, error=0.087
>epoch=200, lrate=0.100, error=0.083
>epoch=300, lrate=0.100, error=0.081
>epoch=400, lrate=0.100, error=0.081
>epoch=500, lrate=0.100, error=0.080
>accuracy=95.450
>epoch=0, lrate=0.100, error=0.353
>epoch=100, lrate=0.100, error=0.092
>epoch=200, lrate=0.100, error=0.085
>epoch=300, lrate=0.100, error=0.083
>epoch=400, lrate=0.100, error=0.082
>epoch=500, lrate=0.100, error=0.081
>accuracy=95.400
>epoch=0, lrate=0.100, error=0.415
>epoch=100, lrate=0.100, error=0.087
>epoch=200, lrate=0.100, error=0.083
>epoch=300, lrate=0.100, error=0.082
>epoch=400, lrate=0.100, error=0.081
>epoch=500, lrate=0.100, error=0.080
>accuracy=95.520
>epoch=0, lrate=0.100, error=0.401
>epoch=100, lrate=0.100, error=0.089
>epoch=200, lrate=0.100, error=0.084
>epoch=300, lrate=0.100, error=0.083
>epoch=400, lrate=0.100, error=0.082
>epoch=500, lrate=0.100, error=0.081
>accuracy=95.280
>epoch=0, lrate=0.100, error=0.395
>epoch=100, lrate=0.100, error=0.093
>epoch=200, lrate=0.100, error=0.087
>epoch=300, lrate=0.100, error=0.085
>epoch=400, lrate=0.100, error=0.084
>epoch=500, lrate=0.100, error=0.083
>accuracy=94.900
Scores: [95.45, 95.39999999999999, 95.52000000000001, 95.28, 94.89999999999999]
Mean Accuracy: 95.310%

Possible Extensions:

  • Early stopping.
  • Experiment with different weight initialization techniques (such as small random numbers).
  • Batch Gradient Descent. Change the training procedure from online to batch gradient descent and update the weights only at the end of each epoch.
  • Mini-Batch Gradient Descent. More info here.
  • Momentum. More info here.
  • Annealing the learning rate. More info here.
  • Dropout Regularization, Batch Normalization. More info here.
  • Model Ensembles. More info here.

References:

  • [1] How to Implement Backpropagation Algorithm from scratch in Python here.
  • [2] Implementing Multiple Layer Neural Network from Scratch here.
  • [3] Andrew Ng Lecture on Gradient Descent here.
  • [4] Andrew Ng Lecture on Backpropagation Algorithm here.
  • [5] (P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis. Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4):547-553, 2009.) here
  • [6] seeds Data Set here
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].