7 open source projects by EleutherAI

An implementation of model parallel GPT2& GPT3-like models, with the ability to scale up to full GPT3 sizes (and possibly more!), using the mesh-tensorflow library.

An implementation of model parallel GPT-3-like models on GPUs, based on the DeepSpeed library. Designed to be able to train models in the hundreds of billions of parameters or larger.

Open-AI's DALL-E for large scale training in mesh-tensorflow.

A library for finding knowledge neurons in pretrained transformer models.

See the issue board for the current status of active and prospective projects!

No description, website, or topics provided.