Loading…
Glossary term
A 16-bit number format that keeps the wide range of 32-bit numbers, widely used in AI training.
Bfloat16 is a 16-bit number format that keeps the same range as standard 32-bit floats but with fewer digits of precision. For neural networks the range matters more than the exact digits, so it trains reliably.
It was developed at Google and is widely supported by modern accelerators.
It became a default for AI training because it halves memory use without the instability of other 16-bit formats.