LiveFR

Pretraining

Definition: Pretraining is a model's first learning phase, over vast corpora, to acquire a general understanding of language.

It is the most compute-intensive stage. It is then followed by fine-tuning and alignment (such as RLHF) that make the model useful and safe.

See also

← Full AI glossary · AI news