nanoGPT claims to be the easiest and fastest database for training/tuning medium-sized GPT, and is still under active development, but the documentation train.pyGPT-2 was reproduced on OpenWebText (trained for about 4 days on an 8XA100 40GB node).

Because the code is so simple, it’s easy to modify it to suit your needs, train a new model from scratch, or fine-tune a pretrained one.

Install

dependencies:

  • pytorch <3
  • numpy <3
  • pip install transformers
  • pip install datasets
  • pip install tiktoken
  • pip install wandb
  • pip install tqdm

#nanoGPT #Homepage #Documentation #Downloads #Fast #lightweight #mediumsized #GPT #project #News Fast Delivery

Leave a Comment

Your email address will not be published. Required fields are marked *