# UMass LIVING

## July 8, 2009

### (Jul 9 3-5 PM) Pre-Meeting Overview of Chapter 5 – Mean Field Method

Filed under: Uncategorized — umassliving @ 4:59 pm
Tags: ,

This week’s chapter introduces Mean Field approaches to approximating $A(\theta)$. The general idea is to optimize the search for the best approximation from a tractable graph $F$ rather than the whole graph $G$.

According to Proposition 5.1, any mean parameter $\mu$ gives a lower bound on $A(\theta)$. Mean Field approaches try to find the best (tightest) lower bound as shown in Equation 5.8. They also present an alternative view of this approximation using KL divergence where minimizing $D(\mu || \theta)$ (bottom of page 133) is the same as Equation 5.8.

The Naive Mean Field Algorithm chooses the product distribution as the tractable distribution (that is, a graph with no edges). Section 5.4 describes the noncovexity of the Mean Field approach which may give local minima. Structured Mean Field approaches use a graph with more structure as the tractable distribution (such as a tree).

Some things to examine closely:

• Proposition 5.1 and equation 5.8 and the relationship between 5.8 and KL divergence on the bottom of page 133.
• Why is Mean Field guaranteed to give a lower bound to $A(\theta)$ ?
• The update rules in Example 5.2
• Example 5.4. Note that on the bottom of page 138, $\theta_{12} = \frac{1}{2}$ and not $\frac{1}{4}$
• Understand how Naive Mean Field is a special case of Structured Mean Field as well as understand Structured Mean Field in general.