Importance Sampling

02 Nov 2024

🚧 Work in progress…

This article will cover Importance Sampling, a Monte Carlo technique for estimating expectations under one distribution using samples from another.

Topics to cover:

• Motivation: when direct sampling is difficult
• The importance sampling principle
• Importance weights and the proposal distribution
• Mathematical formulation
• Estimating expectations with importance sampling
• Choosing good proposal distributions
• Effective sample size
• Self-normalized importance sampling
• Variance of importance sampling estimators
• When importance sampling works well (and when it doesn’t)
• Applications in Bayesian inference
• Sequential importance sampling
• Relationship to other Monte Carlo methods
• Practical examples and implementation