The explicit reparameterization trick is often used to train various latent variable models due to the ease of calculating gradients of continuous random variables. However, due to its peculiarities, explicit reparameterization trick is not applicable to several important continuous standard distributions, such as mixture, Gamma, Beta and Dirichlet.
An alternative method for calculating reparameterization gradients relies on implicit differentiation of cumulative distribution functions. The implicit reparameterization trick is much more expressive and applicable to a wider class of distributions
This article provides an overview of various reparameterization tricks and announces a new Python library, irt.distributions
, for sampling from various distributions using the implicit reparameterization trick.