Loading…
Glossary term
An 8-bit integer format used to run models with less memory and faster speed.
INT8 means storing and computing with 8-bit integers. After quantizing a model's weights, and sometimes its activations, to INT8, it needs a quarter of the memory of 32-bit floats and runs faster on supporting hardware. Accuracy loss is usually small if done carefully.
It is the classic target for fast, low-power inference.