kimhc6028 / Forward Thinking Pytorch
Licence: bsd-3-clause
Pytorch implementation of "Forward Thinking: Building and Training Neural Networks One Layer at a Time"
Stars: ✭ 66
Programming Languages
python
139335 projects - #7 most used programming language
forward-thinking-pytorch
Pytorch implementation of Forward Thinking: Building and Training Neural Networks One Layer at a Time
Requirements
- Python 2.7
- numpy
- pytorch
- Matplotlib
Usage
$ ./run.sh
to run forward-thinking & forward-thinking (deep) & backpropagation & backpropagation (deep).
For forward-thinking experiment (5 layers),
$ python forward_thinking.py
If you want to do this with very deep model (20 layers),
$ python forward_thinking.py --epoch 200 --deep
As a comparison, learn by backpropagation:
$ python backpropagate.py
Backpropagate with very deep model (20 layers),
$ python backpropagate.py --epoch 200 --deep
Result
For 5 layers, forward-thinking learns slightly faster than backpropagation. Dips are observed when new layers are added.
When the model becomes very deep (20 layers), backpropagation cannot train the model. On the other hand, forward-thinking can train the model as usual.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].