BFE is Infinity when data connects to deterministic nodes

[HoangMHNguyen]

When an observation connects to a deterministic node, ReactiveMP counts its entropy but doesn't subtract it, leading to infinity value for BFE since the entropy of a PointMass variable is infinity. This problem doesn't happen to stochastic nodes.

[bvdmitri]

That happens because the BFE procedure is divided into three steps:

• Count entropies of all edges (including the clamped ones), this step adds a couple of infinities (from clamped PointMass) to the final value (we store them in a separate field) with the assumption that those infinities will cancel out in the second step
• Compute factor bound free energy which includes computing an average energy term minus entropies of associated cluster. At this step we assume that the "minus entropy" will cancel out all infinities.
• Check that all infinities have been cancelled out and if not, return Inf

For deterministic nodes the second step in the procedure differs, however. Instead of computing an average energy term minus entropies of clusters we simply compute the minus entropy of the inbound edges. This is according to the theory paper. This, however, does not account for the case when observation is connected directly to the outbound edge of the deterministic node. If that happens, the minus entropy of the inbound edges cannot not cancel out the entropy of the outbound edge and that causes problems in the third step. The assumption that all "problematic" entropies will be cancelled out does not hold anymore.

@ThijsvdLaar @ismailsenoz please discuss

What should we do in case if an outbound edge of a deterministic node is connected directly to an observation? Should it be special-cased? As far as I can recall the theory paper does not include this case.

[albertpod]

@ismailsenoz @ThijsvdLaar, any thoughts on this one?