Quantized Arithmetic
I learned that in quantized models, the integer operations are not truly in low-bit arithmetic — the accumulations happen in higher precision (usually int32). The scale and zero-point conversions make the math approximate, but safe. It’s a subtle design that preserves speed without fully abandoning numerical stability.
2025-10-05