This week’s chapter introduces Mean Field approaches to approximating . The general idea is to optimize the search for the best approximation from a tractable graph rather than the whole graph .

According to Proposition 5.1, any mean parameter gives a lower bound on . Mean Field approaches try to find the best (tightest) lower bound as shown in Equation 5.8. They also present an alternative view of this approximation using KL divergence where minimizing (bottom of page 133) is the same as Equation 5.8.

The Naive Mean Field Algorithm chooses the product distribution as the tractable distribution (that is, a graph with no edges). Section 5.4 describes the noncovexity of the Mean Field approach which may give local minima. Structured Mean Field approaches use a graph with more structure as the tractable distribution (such as a tree).

Some things to examine closely:

- Proposition 5.1 and equation 5.8 and the relationship between 5.8 and KL divergence on the bottom of page 133.
- Why is Mean Field guaranteed to give a lower bound to ?
- The update rules in Example 5.2
- Example 5.4. Note that on the bottom of page 138, and not
- Understand how Naive Mean Field is a special case of Structured Mean Field as well as understand Structured Mean Field in general.

## Leave a Reply