Skip to content

Files

Latest commit

b495cf6 · Oct 26, 2022

History

History
5 lines (3 loc) · 406 Bytes

Weight initialization.md

File metadata and controls

5 lines (3 loc) · 406 Bytes

Weight initialization. When you instantiate a [[neural network]], you have to consider what the initial weight values are.

Example. Xavier initialization uses a normal distribution with mean 0 and variance 1 d where d is the number of incoming connections.

Example. He/Kaiming initialization is used for [[ReLU]] and uses a normal distribution with mean 0 and variance 2 d .