Quantization is a powerful technique for reducing memory usage and speeding up AI applications built with LLMs, diffusion models, CNNs, and other architectures. In fact, quantization is fundamental to all data compression—from JPEG and GIF to MP3 and MP4 (HEVC)! In this episode, we'll cover the basics of neural network quantization, laying the groundwork for future episodes where we'll dive into specific quantization algorithms.

The AI Podcast is hosted by Kirill, CEO of TheStage AI. With his team's deep scientific and industrial expertise in neural network acceleration and deployment, they'll show you how to run AI anywhere and everywhere!

OUTLINE:

00:00 - Jingle!

01:24 - Structure of Podcast

01:46 - When and How to Use Quantization?

03:11 - Speedup or reduce memory? Or Both?

04:18 - Hardware with quantization support

05:28 - DNN compilers to run quantized networks

06:01 - What is quantization mathematically?

07:22 - Fake Quantized Tensors

08:43 - Symmetric, asymmetric, per-tensor, per-channel, per-group

09:43 - Quantized matrix multiplication

11:31 - Quantization algorithms

13:23 - Examples of PTQ and QAT

16:11 - Quantized parameters exists not in discrete space! Is it manifold?

18:08 - Details of the next episode!