TrainingFoundations Updated 2026

Batch Normalization

Definition

A training technique that normalizes the inputs to each layer, allowing deeper networks to train faster and more stably.

Ioffe & Szegedy (2015) introduced batch normalization, which standardizes each layer's inputs using the statistics of the current mini-batch. This reduces the shifting of internal distributions during training, letting practitioners use higher learning rates and train much deeper networks reliably.

Related normalization schemes (such as layer normalization) are a standard component inside Transformers.

References

Primary, peer-reviewed and archival sources for this definition.

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015). Proceedings of the 32nd International Conference on Machine Learning (ICML 2015).

Dictionary & encyclopedic entries

Cite this entry

MultipleChat. "Batch Normalization." MultipleChat AI & LLM Glossary, 2026. https://multiple.chat/ai-glossary/batch-normalization

Related terms

See this in practice

Run the same prompt across ChatGPT, Claude, Gemini, Grok and Perplexity side by side — with your own files as context and answers you can cross-check.

Try MultipleChat Free

Continue learning

See paid plans