19 open source projects by CyberZHG

Implementation of BERT that could load official pre-trained models for feature extraction and prediction

A wrapper layer for stacking layers horizontally

Implementation of XLNet that can load pretrained checkpoints

Load GPT-2 checkpoint and generate texts

Attention mechanism for processing sequential data that considers the context for each timestamp.

https://cyberzhg.github.io/toolbox/

RAdam implemented in Keras & TensorFlow

Some exercises and problems in Introduction to Algorithms 3rd edition.

Transformer implemented in Keras

Concatenate word and character embeddings in Keras

Remove and restore masks for layers that do not support masking

Layer normalization implemented in Keras

Octave convolution

Transformer-XL with checkpoint loader

Multi-head attention in PyTorch

Ordered Neurons LSTM

Gradient accumulation for Keras

Lookahead mechanism for optimizers in Keras.

Position embedding layers in Keras