August 20, 2009

(Aug 20 3-5PM) Meeting Summary : Wainwright – Estimating the “Wrong” Graphical Model

Filed under: Uncategorized — umassliving @ 5:08 pm
Tags: ,

Things we covered today:

  1. In the regularized surrogate likelihood (Eq 16) \ell_B(\theta), we use B(\theta), from the Bethe surrogate formulation covered in the survey paper. We then went through the steps in the joint estimation and prediction in section 4.2. We noted that the noise model is localized for each element of y : p(y|z) = \prod_{s = 1}^N p(y_s | z_s). Erik wondered whether this was done for simplicity or by necessity and we thought it was for simplicity.
  2. The parameter estimate based on the surrogate likelihood \hat{\theta^n} is asymptotically normal.
  3. We had a long discussion about the relationship between globally stable, Lipschitz, and strongly convex.  Any variational method that has a strongly convex entropy approximation is globally stable. Also, any variational method based on a convex optimization is Lipschitz stable.  I’m not sure if there’s a difference between Lipschitz stable and globally stable…
  4. None of us knew how to derive eq 22, but we got some intuition by observing that if \alpha = 0, then this is when the SNR is pure noise and the observation Y is useless. This is reflected in eq 22 by removing the term involving Y. Similarly, if \alpha = 1, then Z_s = Y_s, which is intuitive.
  5. When the SNR \alpha = 0, in section 6.2 they show that \Delta B(\hat{\theta}) = \mu^* = \Delta A(\theta) which means \Delta B(\theta) will be different.
  6. In Figure 4, the curve marked with the red diamonds (approximate model) is upper bounded, as stated in Theorem 7.  Figure 4 also illustrates that performing approximate inference (TRW method) using the approximate model (parameters) can be superior to performing approximate inference using the true model (black circles).

1 Comment »

  1. Added a clarification to point 6.

    Comment by Gary — August 27, 2009 @ 10:22 am | Reply

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create a free website or blog at

%d bloggers like this: