BLOOMChat is a 176 billion parameter multilingual chat model. It is guided by BLOOM(176B) on an assistant-style dialogue dataset and supports dialogue, question answering, and generative answers in multiple languages.

BLOOMChat is a new, open, multilingual chat LLM. SambaNova and Together trained BLOOMChat on the SambaNova DataScale system using SambaNova’s unique reconfigurable dataflow architecture; built on BigScience group’s BLOOM and fine-tuned on OpenChatKit, Dolly 2.0 and OASST1’s OIG.

BLOOM is already the largest multilingual open model, trained on 46 languages, and was developed by an international collaboration of more than 1,000 researchers. SambaNova releases BLOOMChat under a modified version of the Apache 2.0 license that includes the usage-based restrictions in BLOOM’s RAIL license.

Example:

method

data collection

In developing BLOOMChat, the project team aimed to combine the advantages of synthetic conversation data and high-quality human-written examples.First leverage a large synthetic dialogue dataset OpenChatKit Lay the groundwork for the model’s chat functionality.In order to enhance the performance of the model and manually align, the artificially generated dataset is subsequently Dolly 2.0 and OASST1 It was trained on , with a small number of high-quality examples.

train

Use RDU (Reconfigurable Data Flow Unit) in BLOOMChat is trained on a SambaNova DataScale system.Started with BLOOM-176B, then tuned BLOOM-176B on OpenChatKit with each data source subsampled to 100k in one epoch, then on combined OpenChatKit and Dolly-v2 superiorDo 3 rounds of fine tune.

All the code for preparing the dataset and the scripts for running training and inference are open source and available at GitHub for SambaNovaGet it for free.