A two-minute RBM experiment

Can a hidden cause create visible correlation?

Anna and Lukas have no direct connection. Tune a one-hidden-unit Boltzmann machine and watch their joint distribution change.

Challenge

Make Anna-only and Lukas-only the two most likely observations.

Use Bruno as a hidden cause: reward Anna when Bruno is present and discourage Lukas at the same time.

Restricted architecture

No Anna-Lukas edge

positive weight negative weight

+4.0 -4.0 Anna bias -2.0 Lukas bias +2.0 Bruno bias +0.0 visible units hidden unit

Exact visible distribution

What the model predicts

p(v) = sumh exp(score(v,h)) / Z

Hidden inference

What does each observation imply about Bruno?

Alternating Gibbs sampling

Sample one visible-hidden-visible step

Choose a starting observation, then sample Bruno and the next visible state.

Takeaway

An RBM uses hidden variables to shape visible correlations.

The visible units are conditionally independent once Bruno is fixed. After Bruno is hidden again, Anna and Lukas can appear correlated.