Discussed the problems with solving the variational form of , namely that the space of mean parameters is hard to characterize explicitly and for most distributions, does not have an explicit form. Noted that the Bethe Variational Principle (4.16) is an approximation to by utilizing a simple-to-characterize polyhedral outer-bound (to approximate and applying the Bethe entropy approximation (to approximate ). Theorem 4.2 shows that a Lagragian method for solving the BVP is equivalent to the sum-product updates, which provides a principled basis for applying the sum-product algorithm to graph with cycles, namely as a means to approximate the Bethe objective function. Note that, as expected, for tree-structured graphs, since sum-product is an exact inference algorithm for tree-structured graphs.

Note that the right hand side of Equation 4.17 should be reversed, , to be consistent with a positive in Equation 4.21a.

## Leave a Reply