Lit-Parrot is an implementation of the nanoGPT-based StableLM/Pythia/INCITE language model. Supports flash attention, LLaMA-Adapter fine-tuning, and pre-training.

Hackable implementations of state-of-the-art open source large language models:

This implementation is based onLit-LLaMAandnanoGPT, powered byLightning Fabricprovide support.

The weights can be downloaded as follows:

Design Principles

This repository follows openness through clarity main principles.

Lit-Parrot yes:

• Simple:Single-file implementation with no boilerplate.
• correct:Numerically equivalent to the original model.
• optimization:Run on consumer hardware or at scale.
• Open source:No strings attached.

avoiding code duplication is notTarget.readabilityandCrackabilityyes.