Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Ongoing research training transformer models at scale
Minimalistic large language model 3D-parallelism training
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
Open-source pre-training implementation of Google's LaMDA in PyTorch. Adding RLHF similar to ChatGPT.
1 capture since 2026-05-27
