July 8, 2009

(Jul 9 3-5 PM) Pre-Meeting Overview of Chapter 5 – Mean Field Method

Filed under: Uncategorized — umassliving @ 4:59 pm
Tags: ,

This week’s chapter introduces Mean Field approaches to approximating A(\theta). The general idea is to optimize the search for the best approximation from a tractable graph F rather than the whole graph G.

According to Proposition 5.1, any mean parameter \mu gives a lower bound on A(\theta). Mean Field approaches try to find the best (tightest) lower bound as shown in Equation 5.8. They also present an alternative view of this approximation using KL divergence where minimizing D(\mu || \theta) (bottom of page 133) is the same as Equation 5.8.

The Naive Mean Field Algorithm chooses the product distribution as the tractable distribution (that is, a graph with no edges). Section 5.4 describes the noncovexity of the Mean Field approach which may give local minima. Structured Mean Field approaches use a graph with more structure as the tractable distribution (such as a tree).

Some things to examine closely:

  • Proposition 5.1 and equation 5.8 and the relationship between 5.8 and KL divergence on the bottom of page 133.
  • Why is Mean Field guaranteed to give a lower bound to A(\theta) ?
  • The update rules in Example 5.2
  • Example 5.4. Note that on the bottom of page 138, \theta_{12} = \frac{1}{2} and not \frac{1}{4}
  • Understand how Naive Mean Field is a special case of Structured Mean Field as well as understand Structured Mean Field in general.

Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at

%d bloggers like this: