All Projects → kalekiu → Easyesn

kalekiu / Easyesn

Python library for Reservoir Computing using Echo State Networks

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Easyesn

sgrnn
Tensorflow implementation of Synthetic Gradient for RNN (LSTM)
Stars: ✭ 40 (-56.99%)
Mutual labels:  recurrent-neural-networks, rnn
Rmdl
RMDL: Random Multimodel Deep Learning for Classification
Stars: ✭ 375 (+303.23%)
Mutual labels:  rnn, recurrent-neural-networks
EdgarAllanPoetry
Computer-generated poetry
Stars: ✭ 22 (-76.34%)
Mutual labels:  recurrent-neural-networks, rnn
python-machine-learning-book-2nd-edition
<머신러닝 교과서 with 파이썬, 사이킷런, 텐서플로>의 코드 저장소
Stars: ✭ 60 (-35.48%)
Mutual labels:  recurrent-neural-networks, rnn
Deepseqslam
The Official Deep Learning Framework for Route-based Place Recognition
Stars: ✭ 49 (-47.31%)
Mutual labels:  rnn, recurrent-neural-networks
tiny-rnn
Lightweight C++11 library for building deep recurrent neural networks
Stars: ✭ 41 (-55.91%)
Mutual labels:  recurrent-neural-networks, rnn
Rnnsharp
RNNSharp is a toolkit of deep recurrent neural network which is widely used for many different kinds of tasks, such as sequence labeling, sequence-to-sequence and so on. It's written by C# language and based on .NET framework 4.6 or above versions. RNNSharp supports many different types of networks, such as forward and bi-directional network, sequence-to-sequence network, and different types of layers, such as LSTM, Softmax, sampled Softmax and others.
Stars: ✭ 277 (+197.85%)
Mutual labels:  rnn, recurrent-neural-networks
VariationalNeuralAnnealing
A variational implementation of classical and quantum annealing using recurrent neural networks for the purpose of solving optimization problems.
Stars: ✭ 21 (-77.42%)
Mutual labels:  recurrent-neural-networks, rnn
Predicting Myers Briggs Type Indicator With Recurrent Neural Networks
Stars: ✭ 43 (-53.76%)
Mutual labels:  rnn, recurrent-neural-networks
Theano Kaldi Rnn
THEANO-KALDI-RNNs is a project implementing various Recurrent Neural Networks (RNNs) for RNN-HMM speech recognition. The Theano Code is coupled with the Kaldi decoder.
Stars: ✭ 31 (-66.67%)
Mutual labels:  rnn, recurrent-neural-networks
automatic-personality-prediction
[AAAI 2020] Modeling Personality with Attentive Networks and Contextual Embeddings
Stars: ✭ 43 (-53.76%)
Mutual labels:  recurrent-neural-networks, rnn
Codegan
[Deprecated] Source Code Generation using Sequence Generative Adversarial Networks
Stars: ✭ 73 (-21.51%)
Mutual labels:  rnn, recurrent-neural-networks
sequence-rnn-py
Sequence analyzing using Recurrent Neural Networks (RNN) based on Keras
Stars: ✭ 28 (-69.89%)
Mutual labels:  recurrent-neural-networks, rnn
rindow-neuralnetworks
Neural networks library for machine learning on PHP
Stars: ✭ 37 (-60.22%)
Mutual labels:  recurrent-neural-networks, rnn
SpeakerDiarization RNN CNN LSTM
Speaker Diarization is the problem of separating speakers in an audio. There could be any number of speakers and final result should state when speaker starts and ends. In this project, we analyze given audio file with 2 channels and 2 speakers (on separate channels).
Stars: ✭ 56 (-39.78%)
Mutual labels:  recurrent-neural-networks, rnn
Lstm Human Activity Recognition
Human Activity Recognition example using TensorFlow on smartphone sensors dataset and an LSTM RNN. Classifying the type of movement amongst six activity categories - Guillaume Chevalier
Stars: ✭ 2,943 (+3064.52%)
Mutual labels:  rnn, recurrent-neural-networks
Probabilistic-RNN-DA-Classifier
Probabilistic Dialogue Act Classification for the Switchboard Corpus using an LSTM model
Stars: ✭ 22 (-76.34%)
Mutual labels:  recurrent-neural-networks, rnn
Human-Activity-Recognition
Human activity recognition using TensorFlow on smartphone sensors dataset and an LSTM RNN. Classifying the type of movement amongst six categories (WALKING, WALKING_UPSTAIRS, WALKING_DOWNSTAIRS, SITTING, STANDING, LAYING).
Stars: ✭ 16 (-82.8%)
Mutual labels:  recurrent-neural-networks, rnn
Tensorflow Char Rnn
Char-RNN implemented using TensorFlow.
Stars: ✭ 429 (+361.29%)
Mutual labels:  rnn, recurrent-neural-networks
Bitcoin Price Prediction Using Lstm
Bitcoin price Prediction ( Time Series ) using LSTM Recurrent neural network
Stars: ✭ 67 (-27.96%)
Mutual labels:  rnn, recurrent-neural-networks

easyesn

easyesn is a library for recurrent neural networks using echo state networks (ESNs, also called reservoir computing) with a high level easy to use API that is closely oriented towards sklearn. It aims to make the use of ESN as easy as possible, by providing algorithms for automatic gradient based hyperparameter tuning (of ridge regression penalty, spectral radius, leaking rate and feedback scaling), as well as transient time estimation. Furtheremore, it incorporates the ability to use spatio temporal ESNs for prediction and classification of geometrically extended input signals.

easyesn can either use the CPU or make use of the GPU thanks to cupy. At the moment the use of the CPU is recommended though!

This project is based on research results for the gradient based hyperparameter optimization and transient time estimation of Luca Thiede and the spatio temporal prediction and classification techniques of Roland Zimmermann.

Getting started

Installation

The easyesn library is built using python 3. You cannot use it in a python 2.x environment. The recommended way to install easyesn at the moment is via pip. Nevertheless, you can also install easyesn on your own without pip.

pip

You can install easyesn via pip by executing

pip install easyesn

from a valid python 3.x environment, which will automatically install easyesn and its dependencies.

manually

To install the library without pip, there are four steps you have to perform:

  1. Go to the pypi page of easyesn and download the latest version as a *.tar.gz archive.
  2. Extract the archive.
  3. Open your command line/terminal and cd into the directory containing the setup.py.
  4. Execute python setup.py install to start the installation.

First steps

As already mentioned, the API is very similar to the one of sklearn, which makes it as easy as possible for beginners. For every task there is a specialized module, e.g. ClassificationESN for the classification of input signals, PredictionESN for the prediction or generation (that is a several step ahead prediction by always feeding the previous prediction back in), RegressionESN for mapping a signal to a real number, or SpatioTemporalESN for the prediction of geometrically extended input signals (for example the electric excitation on the heart surface or video frames).

Import the module typing

from easyesn import PredictionESN

Now simply fit the esn using

esn.fit(x_train, y_train, transientTime=100, verbose=1)

and predict by using

y_test_pred = esn.predict(x_test, transientTime=100, verbose=1)

For automatic hyperparamter optimization import

from easyesn.optimizers import GradientOptimizer
from easyesn.optimizers import GridSearchOptimizer

Next create a new object

esn = PredictionESN(n_input=1, n_output=1, n_reservoir=500, leakingRate=0.2, spectralRadius=0.2, regressionParameters=[1e-2])

whith a penalty 1e-2 for the ridge regression. To optimize the hyperparameter also create an optimizer object

opt = GradientOptimizer(esn, learningRate=0.001)

and use it with

validationLosses, fitLosses, inputScalings, spectralRadiuses, leakingRates, learningRates = opt.optimizeParameterForTrainError(inputDataTraining, outputDataTraining, inputDataValidation, outputDataValidation, epochs=150, transientTime=100)
validationLosses, fitLosses, penalties = opt.optimizePenalty(inputDataTraining, outputDataTraining, inputDataValidation, outputDataValidation, epochs=150, transientTime=100)

More extensive examples can be found in the examples directory.

Backends

As already mentioned in the beginning, easyesn can be used either on the CPU or on the GPU. To achieve this, all low level calculations are outsourced into a backend (similiar to the backend technology of keras). To change the backend to another backend named backendName, there are currently two ways:

  1. Modify the settings file, stored at ~/.easyesn/easyesn.json to contain to look like this:

    {
        "backend": "backendName"
    }
    

    and use easyesn without any further modification inside your code.

  2. Set the EASYESN_BACKEND environment variable to backendName and use easyesn without any further modification inside your code.

At the moment, these are supported backend names:

backend name backend type Notes
numpy numpy (CPU)
np numpy (CPU)
cupy cupy (GPU) no eig & arange function which is limiting the speed
cp cupy (GPU) no eig & arange function which is limiting the speed
torch torch (CPU/GPU) experimental (Blasting fast but tested/developed for only on PredictionESN)

To set which device the torch backend should use, use the following easyesn.json config:

{
 "backend": "torch",
 "backend_config": {
     "device": "cpu"
    }
}

where cpu can be replaced with any valid torch.device, e.g. cuda.

Notes

As of right now, the GradientOptimizer does not fully work - we are looking into this and try to fix the issue.

Documentation

Develop

Todo

At the moment the easyesn library covers not only all basic features of reservoir computing but also some new, state of the art methods for its application. Nevertheless, there are still some more things which should be implemented in future versions. In the following, these feature requests and ideas are listed together which their progress:

  • Ensemble ESNs (25%)
  • Adding support for deep learning methods as the output method (still open)
  • Improved GPU computing performance (still partially open, predictionESN done)
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].