gonnp
Deep learning from scratch using Go. Specializes in natural language processing
What
Package gonnp is the library of neural network components specialized in natural language processing in Go. You can assemble a neural network with the necessary components.
Dependencies
This component depends on gonum.org/v1/gonum/mat
https://github.com/gonum/gonum/
Components
The number of components will increase in the future.
Layers
- Affine
- MatuMul
- Embedding
- EmbeddingDot
- Relu
- Sigmoid
- Softmax with Loss
- Sigmoid with Loss
Optimizer
- SDG
- Adam
Sampler
Unigram Sampler
Directory
.
├── layers ---( Package layers impliments various layer for neural network. )
├── matutil ---( Package matutil has utility functions of gonum matrix. )
├── models ---( Package models has some of neural netwark models. )
├── optimizers ---( Package optimizers updates prams (ex. weight, bias ...) using various algorism. )
├── params ---( Package params has common parametors type. )
├── store ---( Package store lets you to store trained data. )
├── testdata
│ ├── ptb ---( Package ptb provides load PTB data functions. )
├── trainer ---( Package trainer impliments shorhand of training for deep lerning. )
└── word ---( Package word is functions of text processing. )
Example
Word2Vec
with EmbeddingDot Layers & Negative Sampling
package e2e_test
import (
"fmt"
"io/ioutil"
"log"
"os"
"github.com/po3rin/gonnp/matutil"
"github.com/po3rin/gonnp/models"
"github.com/po3rin/gonnp/optimizers"
"github.com/po3rin/gonnp/trainer"
"github.com/po3rin/gonnp/word"
)
func main() {
windowSize := 5
hiddenSize := 100
batchSize := 100
maxEpoch := 10
// prepare one-hot matrix from text data.
file, err := os.Open("../testdata/golang.txt")
if err != nil {
log.Fatal(err)
}
defer file.Close()
text, err := ioutil.ReadAll(file)
if err != nil {
log.Fatal(err)
}
corpus, w2id, id2w := word.PreProcess(string(text))
vocabSize := len(w2id)
contexts, target := word.CreateContextsAndTarget(corpus, windowSize)
// Inits model
model := models.InitCBOW(vocabSize, hiddenSize, windowSize, corpus)
// choses optimizer
optimizer := optimizers.InitAdam(0.001, 0.9, 0.999)
// inits trainer with model & optimizer.
trainer := trainer.InitTrainer(model, optimizer)
// training !!
trainer.Fit(contexts, target, maxEpoch, batchSize)
// checks outputs
dist := trainer.GetWordDist()
w2v := word.GetWord2VecFromDist(dist, id2w)
for w, v := range w2v {
fmt.Printf("=== %v ===\n", w)
matutil.PrintMat(v)
}
}outputs
=== you ===
⎡ -0.983712641282964⎤
⎢ 0.9633828650811918⎥
⎢-0.7253396760955725⎥
⎢-0.9927919148802162⎥
.
.
.MNIST
package main
import (
"github.com/po3rin/gomnist"
"github.com/po3rin/gonnp/models"
"github.com/po3rin/gonnp/optimizers"
"github.com/po3rin/gonnp/trainer"
)
func main() {
model := models.NewTwoLayerNet(784, 100, 10)
optimizer := optimizers.InitSDG(0.01)
trainer := trainer.InitTrainer(model, optimizer, trainer.EvalInterval(20))
// load MNIST data using github.com/po3rin/gomnist package
l := gomnist.NewLoader("./../testdata", gomnist.OneHotLabel(true), gomnist.Normalization(true))
mnist, _ := l.Load()
trainer.Fit(mnist.TestData, mnist.TestLabels, 10, 100)
}Reference
https://github.com/oreilly-japan/deep-learning-from-scratch-2
TODO
- Impliments RNN