From Dev.to
Cider Quantization: Engineering Trade-offs in On-Device Inference
Most conversations about on-device models start and end with one question: how much did quantization speed things up? That is the easiest question to answer and the least important. The trade-offs tha...