UMass LIVING

June 22, 2009

(Jun 12) Meeting Summary: Wainwright / Jordan, ch 4 – 4.1.5 (incl)

Filed under: Uncategorized — umassliving @ 6:01 am
Tags: ,

Discussed the problems with solving the variational form of $A(\theta)$, namely that the space of mean parameters is hard to characterize explicitly and for most distributions, $A^*(\mu)$ does not have an explicit form. Noted that the Bethe Variational Principle (4.16) is an approximation to $A(\theta)$ by utilizing a simple-to-characterize polyhedral outer-bound $\mathbb{L}(G)$ (to approximate $\mathbb{M}(G)$ and applying the Bethe entropy approximation (to approximate $A^*(\mu)$). Theorem 4.2 shows that a Lagragian method for solving the BVP is equivalent to the sum-product updates, which provides a principled basis for applying the sum-product algorithm to graph with cycles, namely as a means to approximate the Bethe objective function. Note that, as expected, $A_{\mathrm{Bethe}}(\theta) = A(\theta)$ for tree-structured graphs, since sum-product is an exact inference algorithm for tree-structured graphs.

Note that the right hand side of Equation 4.17 should be reversed, $\sum \tau_s - 1$, to be consistent with a positive $\lambda_{SS}$ in Equation 4.21a.