The KL divergence $$D_{KL}(P \| Q) = \sum P(x) \log\frac{P(x)}{Q(x)}$$ is used in VAEs. Which property is TRUE?

Question

MLQuiz · Accepted Answer

KL divergence is **not symmetric** and is **always non-negative** (Gibbs' inequality). $$D_{KL}=0$$ only when $$P=Q$$. In VAEs, it regularizes the latent space toward a standard normal $$\mathcal{N}(0,1)$$.