UMass LIVING

August 20, 2009

(Aug 20 3-5PM) Meeting Summary : Wainwright – Estimating the “Wrong” Graphical Model

Filed under: Uncategorized — umassliving @ 5:08 pm
Tags: ,

Things we covered today:

1. In the regularized surrogate likelihood (Eq 16) $\ell_B(\theta)$, we use $B(\theta)$, from the Bethe surrogate formulation covered in the survey paper. We then went through the steps in the joint estimation and prediction in section 4.2. We noted that the noise model is localized for each element of $y$ : $p(y|z) = \prod_{s = 1}^N p(y_s | z_s)$. Erik wondered whether this was done for simplicity or by necessity and we thought it was for simplicity.
2. The parameter estimate based on the surrogate likelihood $\hat{\theta^n}$ is asymptotically normal.
3. We had a long discussion about the relationship between globally stable, Lipschitz, and strongly convex.  Any variational method that has a strongly convex entropy approximation is globally stable. Also, any variational method based on a convex optimization is Lipschitz stable.  I’m not sure if there’s a difference between Lipschitz stable and globally stable…
4. None of us knew how to derive eq 22, but we got some intuition by observing that if $\alpha = 0$, then this is when the SNR is pure noise and the observation $Y$ is useless. This is reflected in eq 22 by removing the term involving $Y$. Similarly, if $\alpha = 1$, then $Z_s = Y_s$, which is intuitive.
5. When the SNR $\alpha = 0$, in section 6.2 they show that $\Delta B(\hat{\theta}) = \mu^* = \Delta A(\theta)$ which means $\Delta B(\theta)$ will be different.
6. In Figure 4, the curve marked with the red diamonds (approximate model) is upper bounded, as stated in Theorem 7.  Figure 4 also illustrates that performing approximate inference (TRW method) using the approximate model (parameters) can be superior to performing approximate inference using the true model (black circles).