SimplePythonCNN
This is Convolutional Neural Network only in python & numpy. It is simple and slow but will get the job done
Specification
Weight Initialization : HE Normal
Weight Update Policy : ADAM, NAG, Momentum, Vanila
Active Function : ReLU, Sigmoid
Regulization : Droupout(only on fc), L2
Pooling : Max, Average
Loss Function : Softmax, Logistic
Prerequisites
numpy (+ mkl for intel processors. recommend anaconda)
Used sklearn for LabelEncoder & utils.shuffle on examples.
Example
AND gate and CIFAR-10 examples are included.
lr = 1e-4
l2_reg = 8e-6
cnn = NeuralNetwork(train_images.shape[1:],
[
{'type': 'conv', 'k': 16, 'u_type': 'nag', 'f': 5, 's': 1, 'p': 2},
{'type': 'pool', 'method': 'average'},
{'type': 'conv', 'k': 20, 'u_type': 'nag', 'f': 5, 's': 1, 'p': 2},
{'type': 'pool', 'method': 'average'},
{'type': 'conv', 'k': 20, 'u_type': 'nag', 'f': 5, 's': 1, 'p': 2},
{'type': 'pool', 'method': 'average'},
{'type': 'output', 'k': len(le.classes_), 'u_type': 'adam'}
]
, lr, l2_reg=l2_reg)CIFAR-10 example gets ~72% test accuracy in 20 epoch.
API Reference
classes.NeuralNetwork(self, input_shape, layer_list, lr, l2_reg=0, loss='softmax'):| Parameter | Description |
|---|---|
| input_shape | Data's numpy shape. |
| layer_list | List of layers you want to be networked. All of properties goes to **kwargs. |
| lr | Learning rate. |
| l2_reg | L2 regularization |
| loss | Loss function. 'softmax', 'logistic' |
# type fc, output
classes.NeuralLayer(input_size, k, f=3, s=1, p=1, u_type='adam', a_type='relu', dropout=1)
# type pool
classes.PoolLayer(input_size, f=2, s=2, method='max', dropout=1):
# type conv
classes.ConvLayer(input_size, k, f=3, s=1, p=1, u_type='adam', a_type='relu', dropout=1)| Update Policy | u_type |
|---|---|
| ADAM | 'adam' |
| Momentum | 'm' |
| Vanilla | 'v' |
| NAG | 'nag' |
| RMSProp | 'rmsprop' |
| Activation Function | a_type |
|---|---|
| ReLU | 'relu' |
| Sigmoid | 'sigmoid' |
| Pooling | method |
|---|---|
| Max | 'max' |
| Avverage | 'average' |
License
MIT