Example: Camera Image€¦ · x i x i PD Dr. Rudolph Triebel Computer Vision Group Machine Learning...

xi xi

PD Dr. Rudolph TriebelComputer Vision Group

Machine Learning for Computer Vision

Example: Camera Image

• directions are counter-intuitive for images • Markov blanket is not just the direct neighbors

when using a directed model!1



Markov Random Fields

All paths from A to B go through C, i.e. C blocks all paths.

Markov Blanket

We only need to condition on the direct neighbors of x to get c.i., because these already block every path from x to any other node.

!2



Factorization of MRFs

Any two nodes xi and xj that are not connected in an MRF are conditionally independent given all other nodes:

This means: each factor contains only nodes that are connected This motivates the consideration of cliques in the graph:

• A clique is a fully connected subgraph.

• A maximal clique can not be extendedwith another node without loosing the property of full connectivity.

Clique

Maximal Clique

!3

p(xi, xj | x\{i,j}) = p(xi | x\{i,j})p(xj | x\{i,j})



Factorization of MRFsIn general, a Markov Random Field is factorized as

where is the set of all (maximal) cliques and is

a positive function of a given clique xC of nodes, called the clique potential. Z is called the partition function. Theorem (Hammersley/Clifford): Any undirected model with associated clique potentials is a perfect map for the probability distribution defined by Equation (4.1). As a conclusion, all probability distributions that can be factorized as in (4.1), can be represented as an MRF.

!4

p(x) =

Qc2C c(xc)P

x0Q

c2C c(x0c)

=1

Z

Y

c2C c(xc) (4.1)

<latexit sha1_base64="kSt+tWU/Q41kNcIZWo+i/MhMj7Y=">AAADaXicnVJLaxsxENbafaTuI3Z7Ke1F1JjYUMyuCbSXQmguvQRSiJPQyCxardYW0WMjaYtdoVP7J/sbCv0N1TpL6ziUQgckDTOab755ZCVnxsbx96jVvnP33v2dB52Hjx4/2e32np4aVWlCp0Rxpc8zbChnkk4ts5yel5pikXF6ll0e1v6zz1QbpuSJXZV0JvBcsoIRbIMp7f6EJRwige0iK9zSwxF8B1GhMXGo1CpPHUFMrv0Ec3foPUSZ4rlZifCg0rCUbISnZOQdMpVI3R/jnv8fqL0aK5Bp2CTeffobzj8YQXR1VeF8uD9ORmm3H4/jtcDbStIofdDIcdqLCMoVqQSVlnBszEUSl3bmsLaMcOo7qDK0xOQSz+lFUCUW1MzcejAeDoIlh4XS4UgL19bNCIeFqYmHnzVjs+2rjb99g02nXdaIZiu/Ld7OHJNlZakk1+mLikOrYD17mDNNieWroGCiWagAkgUO7bVhQ24g1euUv9YVDzdRoq7eyEpkVNM8FM/nKoQvxIR6CAcwtCU0yMCGMWxQIJPGYh4wOshQS1QlQyJ3hJdH2Gq2DNtpvJvEvhPGkmwP4bZyOhkn8Tj5uN8/eN8MaAe8BK/AECTgDTgAH8AxmAISnURfoq/Rt9aPdq/9vP3i+msramKegRvS7v8CMUke3w==</latexit><latexit sha1_base64="kSt+tWU/Q41kNcIZWo+i/MhMj7Y=">AAADaXicnVJLaxsxENbafaTuI3Z7Ke1F1JjYUMyuCbSXQmguvQRSiJPQyCxardYW0WMjaYtdoVP7J/sbCv0N1TpL6ziUQgckDTOab755ZCVnxsbx96jVvnP33v2dB52Hjx4/2e32np4aVWlCp0Rxpc8zbChnkk4ts5yel5pikXF6ll0e1v6zz1QbpuSJXZV0JvBcsoIRbIMp7f6EJRwige0iK9zSwxF8B1GhMXGo1CpPHUFMrv0Ec3foPUSZ4rlZifCg0rCUbISnZOQdMpVI3R/jnv8fqL0aK5Bp2CTeffobzj8YQXR1VeF8uD9ORmm3H4/jtcDbStIofdDIcdqLCMoVqQSVlnBszEUSl3bmsLaMcOo7qDK0xOQSz+lFUCUW1MzcejAeDoIlh4XS4UgL19bNCIeFqYmHnzVjs+2rjb99g02nXdaIZiu/Ld7OHJNlZakk1+mLikOrYD17mDNNieWroGCiWagAkgUO7bVhQ24g1euUv9YVDzdRoq7eyEpkVNM8FM/nKoQvxIR6CAcwtCU0yMCGMWxQIJPGYh4wOshQS1QlQyJ3hJdH2Gq2DNtpvJvEvhPGkmwP4bZyOhkn8Tj5uN8/eN8MaAe8BK/AECTgDTgAH8AxmAISnURfoq/Rt9aPdq/9vP3i+msramKegRvS7v8CMUke3w==</latexit><latexit sha1_base64="kSt+tWU/Q41kNcIZWo+i/MhMj7Y=">AAADaXicnVJLaxsxENbafaTuI3Z7Ke1F1JjYUMyuCbSXQmguvQRSiJPQyCxardYW0WMjaYtdoVP7J/sbCv0N1TpL6ziUQgckDTOab755ZCVnxsbx96jVvnP33v2dB52Hjx4/2e32np4aVWlCp0Rxpc8zbChnkk4ts5yel5pikXF6ll0e1v6zz1QbpuSJXZV0JvBcsoIRbIMp7f6EJRwige0iK9zSwxF8B1GhMXGo1CpPHUFMrv0Ec3foPUSZ4rlZifCg0rCUbISnZOQdMpVI3R/jnv8fqL0aK5Bp2CTeffobzj8YQXR1VeF8uD9ORmm3H4/jtcDbStIofdDIcdqLCMoVqQSVlnBszEUSl3bmsLaMcOo7qDK0xOQSz+lFUCUW1MzcejAeDoIlh4XS4UgL19bNCIeFqYmHnzVjs+2rjb99g02nXdaIZiu/Ld7OHJNlZakk1+mLikOrYD17mDNNieWroGCiWagAkgUO7bVhQ24g1euUv9YVDzdRoq7eyEpkVNM8FM/nKoQvxIR6CAcwtCU0yMCGMWxQIJPGYh4wOshQS1QlQyJ3hJdH2Gq2DNtpvJvEvhPGkmwP4bZyOhkn8Tj5uN8/eN8MaAe8BK/AECTgDTgAH8AxmAISnURfoq/Rt9aPdq/9vP3i+msramKegRvS7v8CMUke3w==</latexit><latexit sha1_base64="kSt+tWU/Q41kNcIZWo+i/MhMj7Y=">AAADaXicnVJLaxsxENbafaTuI3Z7Ke1F1JjYUMyuCbSXQmguvQRSiJPQyCxardYW0WMjaYtdoVP7J/sbCv0N1TpL6ziUQgckDTOab755ZCVnxsbx96jVvnP33v2dB52Hjx4/2e32np4aVWlCp0Rxpc8zbChnkk4ts5yel5pikXF6ll0e1v6zz1QbpuSJXZV0JvBcsoIRbIMp7f6EJRwige0iK9zSwxF8B1GhMXGo1CpPHUFMrv0Ec3foPUSZ4rlZifCg0rCUbISnZOQdMpVI3R/jnv8fqL0aK5Bp2CTeffobzj8YQXR1VeF8uD9ORmm3H4/jtcDbStIofdDIcdqLCMoVqQSVlnBszEUSl3bmsLaMcOo7qDK0xOQSz+lFUCUW1MzcejAeDoIlh4XS4UgL19bNCIeFqYmHnzVjs+2rjb99g02nXdaIZiu/Ld7OHJNlZakk1+mLikOrYD17mDNNieWroGCiWagAkgUO7bVhQ24g1euUv9YVDzdRoq7eyEpkVNM8FM/nKoQvxIR6CAcwtCU0yMCGMWxQIJPGYh4wOshQS1QlQyJ3hJdH2Gq2DNtpvJvEvhPGkmwP4bZyOhkn8Tj5uN8/eN8MaAe8BK/AECTgDTgAH8AxmAISnURfoq/Rt9aPdq/9vP3i+msramKegRvS7v8CMUke3w==</latexit>

C<latexit sha1_base64="pBtXnSyJlrIounj4TnOYKTpyTsI=">AAAChHicZVFbaxNRED5ZtdZ4a/XRl4Mh4EMJu1HRJyn2xZdCBdMWklBmz06SQ89lOWdWEpb9Gb7q7/LfOJsu0qYD5/Axl29mvslLoyOl6d9e8uDho73H+0/6T589f/Hy4PDVefRVUDhR3vhwmUNEox1OSJPByzIg2NzgRX590sYvfmKI2rsftClxbmHp9EIrIHZNZxZopcDUJ83VwSAdpVuT90HWgYHo7OzqsKdmhVeVRUfKQIzTLC1pXkMgrQw2/VkVsQR1DUucMnRgMc7r7cyNHLKnkAsf+DmSW+/tihpsjBubc2Y7Y9yNtc7/seHtIK1bxrjTnxaf57V2ZUXo1E37RWUkednKIgsdUJHZMAAVNG8g1QoCKGLx7jC1ShdHoTL8K2/b7aOrbI4BC17eLD2Xr+wYGymHkmVhgaLsJpYdi9QuEhjm6M8ikvKV40b1KaxPgYJe8+FiU4/Tps9nyXaPcB+cj0dZOsq+fxgcf+0OtC/eiLfincjEJ3EsvokzMRFKePFL/BZ/kr3kKHmffLxJTXpdzWtxx5Iv/wDt3cbP</latexit><latexit sha1_base64="pBtXnSyJlrIounj4TnOYKTpyTsI=">AAAChHicZVFbaxNRED5ZtdZ4a/XRl4Mh4EMJu1HRJyn2xZdCBdMWklBmz06SQ89lOWdWEpb9Gb7q7/LfOJsu0qYD5/Axl29mvslLoyOl6d9e8uDho73H+0/6T589f/Hy4PDVefRVUDhR3vhwmUNEox1OSJPByzIg2NzgRX590sYvfmKI2rsftClxbmHp9EIrIHZNZxZopcDUJ83VwSAdpVuT90HWgYHo7OzqsKdmhVeVRUfKQIzTLC1pXkMgrQw2/VkVsQR1DUucMnRgMc7r7cyNHLKnkAsf+DmSW+/tihpsjBubc2Y7Y9yNtc7/seHtIK1bxrjTnxaf57V2ZUXo1E37RWUkednKIgsdUJHZMAAVNG8g1QoCKGLx7jC1ShdHoTL8K2/b7aOrbI4BC17eLD2Xr+wYGymHkmVhgaLsJpYdi9QuEhjm6M8ikvKV40b1KaxPgYJe8+FiU4/Tps9nyXaPcB+cj0dZOsq+fxgcf+0OtC/eiLfincjEJ3EsvokzMRFKePFL/BZ/kr3kKHmffLxJTXpdzWtxx5Iv/wDt3cbP</latexit><latexit sha1_base64="pBtXnSyJlrIounj4TnOYKTpyTsI=">AAAChHicZVFbaxNRED5ZtdZ4a/XRl4Mh4EMJu1HRJyn2xZdCBdMWklBmz06SQ89lOWdWEpb9Gb7q7/LfOJsu0qYD5/Axl29mvslLoyOl6d9e8uDho73H+0/6T589f/Hy4PDVefRVUDhR3vhwmUNEox1OSJPByzIg2NzgRX590sYvfmKI2rsftClxbmHp9EIrIHZNZxZopcDUJ83VwSAdpVuT90HWgYHo7OzqsKdmhVeVRUfKQIzTLC1pXkMgrQw2/VkVsQR1DUucMnRgMc7r7cyNHLKnkAsf+DmSW+/tihpsjBubc2Y7Y9yNtc7/seHtIK1bxrjTnxaf57V2ZUXo1E37RWUkednKIgsdUJHZMAAVNG8g1QoCKGLx7jC1ShdHoTL8K2/b7aOrbI4BC17eLD2Xr+wYGymHkmVhgaLsJpYdi9QuEhjm6M8ikvKV40b1KaxPgYJe8+FiU4/Tps9nyXaPcB+cj0dZOsq+fxgcf+0OtC/eiLfincjEJ3EsvokzMRFKePFL/BZ/kr3kKHmffLxJTXpdzWtxx5Iv/wDt3cbP</latexit><latexit sha1_base64="pBtXnSyJlrIounj4TnOYKTpyTsI=">AAAChHicZVFbaxNRED5ZtdZ4a/XRl4Mh4EMJu1HRJyn2xZdCBdMWklBmz06SQ89lOWdWEpb9Gb7q7/LfOJsu0qYD5/Axl29mvslLoyOl6d9e8uDho73H+0/6T589f/Hy4PDVefRVUDhR3vhwmUNEox1OSJPByzIg2NzgRX590sYvfmKI2rsftClxbmHp9EIrIHZNZxZopcDUJ83VwSAdpVuT90HWgYHo7OzqsKdmhVeVRUfKQIzTLC1pXkMgrQw2/VkVsQR1DUucMnRgMc7r7cyNHLKnkAsf+DmSW+/tihpsjBubc2Y7Y9yNtc7/seHtIK1bxrjTnxaf57V2ZUXo1E37RWUkednKIgsdUJHZMAAVNG8g1QoCKGLx7jC1ShdHoTL8K2/b7aOrbI4BC17eLD2Xr+wYGymHkmVhgaLsJpYdi9QuEhjm6M8ikvKV40b1KaxPgYJe8+FiU4/Tps9nyXaPcB+cj0dZOsq+fxgcf+0OtC/eiLfincjEJ3EsvokzMRFKePFL/BZ/kr3kKHmffLxJTXpdzWtxx5Iv/wDt3cbP</latexit>

c(xc)<latexit sha1_base64="VpRNga+lfFONIXE83OPJfcV6S9g=">AAACj3icZVHbattAEF2rt9S9xG771pelxpBCMZIppE/FtC/tQyCFOglERqzGI3vJXsTuqNgV+pe+tn/Uv+nKESVxBnY5zOXMzJm8VNJTHP/tRffuP3j46OBx/8nTZ88PB8MXZ95WDnAOVll3kQuPShqckySFF6VDoXOF5/nV5zZ+/gOdl9Z8p22JCy1WRhYSBAVXNniVll5mcJRqQeu8qDdNBm+zwSiexDvjd0HSgRHr7DQb9iBdWqg0GgIlvL9M4pIWtXAkQWHTTyuPpYArscLLAI3Q6Bf1bvyGj4NnyQvrwjPEd96bFbXQ3m91HjLbKf1+rHX+j41vBmnTMvq9/lR8WNTSlBWhgev2RaU4Wd4qxJfSIZDaBiDAybABh7VwAijoeIupFX35zlUq/GB1u703lc7R4TIsr1Y2lK/1FBvOxzzIEgTyvJuYdyxcGk9CBY5+6pHAViY0qk/E5kSQk5twQ9/U07jph7Mk+0e4C86mkySeJN/ej2afugMdsNfsDTtiCTtmM/aFnbI5A/aT/WK/2Z9oGB1HH6PZdWrU62peslsWff0H2tfKpA==</latexit><latexit sha1_base64="VpRNga+lfFONIXE83OPJfcV6S9g=">AAACj3icZVHbattAEF2rt9S9xG771pelxpBCMZIppE/FtC/tQyCFOglERqzGI3vJXsTuqNgV+pe+tn/Uv+nKESVxBnY5zOXMzJm8VNJTHP/tRffuP3j46OBx/8nTZ88PB8MXZ95WDnAOVll3kQuPShqckySFF6VDoXOF5/nV5zZ+/gOdl9Z8p22JCy1WRhYSBAVXNniVll5mcJRqQeu8qDdNBm+zwSiexDvjd0HSgRHr7DQb9iBdWqg0GgIlvL9M4pIWtXAkQWHTTyuPpYArscLLAI3Q6Bf1bvyGj4NnyQvrwjPEd96bFbXQ3m91HjLbKf1+rHX+j41vBmnTMvq9/lR8WNTSlBWhgev2RaU4Wd4qxJfSIZDaBiDAybABh7VwAijoeIupFX35zlUq/GB1u703lc7R4TIsr1Y2lK/1FBvOxzzIEgTyvJuYdyxcGk9CBY5+6pHAViY0qk/E5kSQk5twQ9/U07jph7Mk+0e4C86mkySeJN/ej2afugMdsNfsDTtiCTtmM/aFnbI5A/aT/WK/2Z9oGB1HH6PZdWrU62peslsWff0H2tfKpA==</latexit><latexit sha1_base64="VpRNga+lfFONIXE83OPJfcV6S9g=">AAACj3icZVHbattAEF2rt9S9xG771pelxpBCMZIppE/FtC/tQyCFOglERqzGI3vJXsTuqNgV+pe+tn/Uv+nKESVxBnY5zOXMzJm8VNJTHP/tRffuP3j46OBx/8nTZ88PB8MXZ95WDnAOVll3kQuPShqckySFF6VDoXOF5/nV5zZ+/gOdl9Z8p22JCy1WRhYSBAVXNniVll5mcJRqQeu8qDdNBm+zwSiexDvjd0HSgRHr7DQb9iBdWqg0GgIlvL9M4pIWtXAkQWHTTyuPpYArscLLAI3Q6Bf1bvyGj4NnyQvrwjPEd96bFbXQ3m91HjLbKf1+rHX+j41vBmnTMvq9/lR8WNTSlBWhgev2RaU4Wd4qxJfSIZDaBiDAybABh7VwAijoeIupFX35zlUq/GB1u703lc7R4TIsr1Y2lK/1FBvOxzzIEgTyvJuYdyxcGk9CBY5+6pHAViY0qk/E5kSQk5twQ9/U07jph7Mk+0e4C86mkySeJN/ej2afugMdsNfsDTtiCTtmM/aFnbI5A/aT/WK/2Z9oGB1HH6PZdWrU62peslsWff0H2tfKpA==</latexit><latexit sha1_base64="VpRNga+lfFONIXE83OPJfcV6S9g=">AAACj3icZVHbattAEF2rt9S9xG771pelxpBCMZIppE/FtC/tQyCFOglERqzGI3vJXsTuqNgV+pe+tn/Uv+nKESVxBnY5zOXMzJm8VNJTHP/tRffuP3j46OBx/8nTZ88PB8MXZ95WDnAOVll3kQuPShqckySFF6VDoXOF5/nV5zZ+/gOdl9Z8p22JCy1WRhYSBAVXNniVll5mcJRqQeu8qDdNBm+zwSiexDvjd0HSgRHr7DQb9iBdWqg0GgIlvL9M4pIWtXAkQWHTTyuPpYArscLLAI3Q6Bf1bvyGj4NnyQvrwjPEd96bFbXQ3m91HjLbKf1+rHX+j41vBmnTMvq9/lR8WNTSlBWhgev2RaU4Wd4qxJfSIZDaBiDAybABh7VwAijoeIupFX35zlUq/GB1u703lc7R4TIsr1Y2lK/1FBvOxzzIEgTyvJuYdyxcGk9CBY5+6pHAViY0qk/E5kSQk5twQ9/U07jph7Mk+0e4C86mkySeJN/ej2afugMdsNfsDTtiCTtmM/aFnbI5A/aT/WK/2Z9oGB1HH6PZdWrU62peslsWff0H2tfKpA==</latexit>

c<latexit sha1_base64="a60x/Nz3mEHj8BMWDOmUVovubTU=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNoewvtpZdACrUTiI0Zrcb2NvsQO6NiI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73k3v0HDx8dPO4/efrs+YvDo5dT8lVQOFHe+HCRA6HRDies2eBFGRBsbvA8v/rSxs9/YiDt3Xfelji3sHJ6qRVwdE1nJemFWhwO0lG6M3kXZB0YiM7OFkc9NSu8qiw6VgaILrO05HkNgbUy2PRnFWEJ6gpWeBmhA4s0r3fjNnIYPYVc+hCfY7nz3qyowRJtbR4zLfCa9mOt839seDPIm5aR9vrz8uO81q6sGJ26br+sjGQvW0VkoQMqNtsIQAUdN5BqDQEUR91uMbUiF+9DZeKvvG23J1fZHAMWcXmz8rF8bcfYSDmUUZYoEMluYtmxSO2IwUSO/oyQla9cbFSfwuYUOOhNvBk19Tht+vEs2f4R7oLpeJSlo+zb8eDkc3egA/FavBHvRCY+iBPxVZyJiVDih/glfos/SS95m4yS9Do16XU1r8QtSz79A+d5xJY=</latexit><latexit sha1_base64="a60x/Nz3mEHj8BMWDOmUVovubTU=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNoewvtpZdACrUTiI0Zrcb2NvsQO6NiI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73k3v0HDx8dPO4/efrs+YvDo5dT8lVQOFHe+HCRA6HRDies2eBFGRBsbvA8v/rSxs9/YiDt3Xfelji3sHJ6qRVwdE1nJemFWhwO0lG6M3kXZB0YiM7OFkc9NSu8qiw6VgaILrO05HkNgbUy2PRnFWEJ6gpWeBmhA4s0r3fjNnIYPYVc+hCfY7nz3qyowRJtbR4zLfCa9mOt839seDPIm5aR9vrz8uO81q6sGJ26br+sjGQvW0VkoQMqNtsIQAUdN5BqDQEUR91uMbUiF+9DZeKvvG23J1fZHAMWcXmz8rF8bcfYSDmUUZYoEMluYtmxSO2IwUSO/oyQla9cbFSfwuYUOOhNvBk19Tht+vEs2f4R7oLpeJSlo+zb8eDkc3egA/FavBHvRCY+iBPxVZyJiVDih/glfos/SS95m4yS9Do16XU1r8QtSz79A+d5xJY=</latexit><latexit sha1_base64="a60x/Nz3mEHj8BMWDOmUVovubTU=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNoewvtpZdACrUTiI0Zrcb2NvsQO6NiI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73k3v0HDx8dPO4/efrs+YvDo5dT8lVQOFHe+HCRA6HRDies2eBFGRBsbvA8v/rSxs9/YiDt3Xfelji3sHJ6qRVwdE1nJemFWhwO0lG6M3kXZB0YiM7OFkc9NSu8qiw6VgaILrO05HkNgbUy2PRnFWEJ6gpWeBmhA4s0r3fjNnIYPYVc+hCfY7nz3qyowRJtbR4zLfCa9mOt839seDPIm5aR9vrz8uO81q6sGJ26br+sjGQvW0VkoQMqNtsIQAUdN5BqDQEUR91uMbUiF+9DZeKvvG23J1fZHAMWcXmz8rF8bcfYSDmUUZYoEMluYtmxSO2IwUSO/oyQla9cbFSfwuYUOOhNvBk19Tht+vEs2f4R7oLpeJSlo+zb8eDkc3egA/FavBHvRCY+iBPxVZyJiVDih/glfos/SS95m4yS9Do16XU1r8QtSz79A+d5xJY=</latexit><latexit sha1_base64="a60x/Nz3mEHj8BMWDOmUVovubTU=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNoewvtpZdACrUTiI0Zrcb2NvsQO6NiI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73k3v0HDx8dPO4/efrs+YvDo5dT8lVQOFHe+HCRA6HRDies2eBFGRBsbvA8v/rSxs9/YiDt3Xfelji3sHJ6qRVwdE1nJemFWhwO0lG6M3kXZB0YiM7OFkc9NSu8qiw6VgaILrO05HkNgbUy2PRnFWEJ6gpWeBmhA4s0r3fjNnIYPYVc+hCfY7nz3qyowRJtbR4zLfCa9mOt839seDPIm5aR9vrz8uO81q6sGJ26br+sjGQvW0VkoQMqNtsIQAUdN5BqDQEUR91uMbUiF+9DZeKvvG23J1fZHAMWcXmz8rF8bcfYSDmUUZYoEMluYtmxSO2IwUSO/oyQla9cbFSfwuYUOOhNvBk19Tht+vEs2f4R7oLpeJSlo+zb8eDkc3egA/FavBHvRCY+iBPxVZyJiVDih/glfos/SS95m4yS9Do16XU1r8QtSz79A+d5xJY=</latexit>



Representability• As for DAGs, we can define an I-map, a D-map

and a perfect map for MRFs.

• The set of all distributions for which a DAG exists that is a perfect map is different from that for MRFs.

Distributions with a DAG as perfect map

Distributions with an MRF as

perfect map

All distributions

!5



A Simple Example

If a distribution p satisfies all conditional independence relationships of this graph, then we can write p as

!6

x2<latexit sha1_base64="vSKLpqANOkSIiN4QGo86Du5ka/s=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSenkrSbF+GrhpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+tQUrTP73k1u07d+/t3e8/ePjo8ZP9g6cnGKqoYaKDDfEsVwjWeJiQIQtnZQTlcgun+cXnNn76AyKa4L/TpoSZU0tvFkYrYte39Xw83x+ko3Rr8ibIOjAQnR3PD3p6WgRdOfCkrUI8z9KSZrWKZLSFpj+tEEqlL9QSzhl65QBn9XbWRg7ZU8hFiPw8ya33akWtHOLG5ZzpFK1wN9Y6/8eGV4O0bhlxpz8tPs5q48uKwOvL9ovKSgqylUMWJoImu2GgdDS8gdQrFZUmFu0aU6tw8SZWln8dXLs9+srlEKHg5e0ycPnKjaGRcihZFhYIZTex7Fik8UjKMkd/ikA6VJ4b1UdqfaQomjUfDJt6nDZ9Pku2e4Sb4GQ8ytJR9vXd4PBTd6A98Vy8EK9EJj6IQ/FFHIuJ0GIpfopf4nfvb/IyeZ28vUxNel3NM3HNkvf/ALiTxBY=</latexit><latexit sha1_base64="vSKLpqANOkSIiN4QGo86Du5ka/s=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSenkrSbF+GrhpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+tQUrTP73k1u07d+/t3e8/ePjo8ZP9g6cnGKqoYaKDDfEsVwjWeJiQIQtnZQTlcgun+cXnNn76AyKa4L/TpoSZU0tvFkYrYte39Xw83x+ko3Rr8ibIOjAQnR3PD3p6WgRdOfCkrUI8z9KSZrWKZLSFpj+tEEqlL9QSzhl65QBn9XbWRg7ZU8hFiPw8ya33akWtHOLG5ZzpFK1wN9Y6/8eGV4O0bhlxpz8tPs5q48uKwOvL9ovKSgqylUMWJoImu2GgdDS8gdQrFZUmFu0aU6tw8SZWln8dXLs9+srlEKHg5e0ycPnKjaGRcihZFhYIZTex7Fik8UjKMkd/ikA6VJ4b1UdqfaQomjUfDJt6nDZ9Pku2e4Sb4GQ8ytJR9vXd4PBTd6A98Vy8EK9EJj6IQ/FFHIuJ0GIpfopf4nfvb/IyeZ28vUxNel3NM3HNkvf/ALiTxBY=</latexit><latexit sha1_base64="vSKLpqANOkSIiN4QGo86Du5ka/s=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSenkrSbF+GrhpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+tQUrTP73k1u07d+/t3e8/ePjo8ZP9g6cnGKqoYaKDDfEsVwjWeJiQIQtnZQTlcgun+cXnNn76AyKa4L/TpoSZU0tvFkYrYte39Xw83x+ko3Rr8ibIOjAQnR3PD3p6WgRdOfCkrUI8z9KSZrWKZLSFpj+tEEqlL9QSzhl65QBn9XbWRg7ZU8hFiPw8ya33akWtHOLG5ZzpFK1wN9Y6/8eGV4O0bhlxpz8tPs5q48uKwOvL9ovKSgqylUMWJoImu2GgdDS8gdQrFZUmFu0aU6tw8SZWln8dXLs9+srlEKHg5e0ycPnKjaGRcihZFhYIZTex7Fik8UjKMkd/ikA6VJ4b1UdqfaQomjUfDJt6nDZ9Pku2e4Sb4GQ8ytJR9vXd4PBTd6A98Vy8EK9EJj6IQ/FFHIuJ0GIpfopf4nfvb/IyeZ28vUxNel3NM3HNkvf/ALiTxBY=</latexit><latexit sha1_base64="vSKLpqANOkSIiN4QGo86Du5ka/s=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSenkrSbF+GrhpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+tQUrTP73k1u07d+/t3e8/ePjo8ZP9g6cnGKqoYaKDDfEsVwjWeJiQIQtnZQTlcgun+cXnNn76AyKa4L/TpoSZU0tvFkYrYte39Xw83x+ko3Rr8ibIOjAQnR3PD3p6WgRdOfCkrUI8z9KSZrWKZLSFpj+tEEqlL9QSzhl65QBn9XbWRg7ZU8hFiPw8ya33akWtHOLG5ZzpFK1wN9Y6/8eGV4O0bhlxpz8tPs5q48uKwOvL9ovKSgqylUMWJoImu2GgdDS8gdQrFZUmFu0aU6tw8SZWln8dXLs9+srlEKHg5e0ycPnKjaGRcihZFhYIZTex7Fik8UjKMkd/ikA6VJ4b1UdqfaQomjUfDJt6nDZ9Pku2e4Sb4GQ8ytJR9vXd4PBTd6A98Vy8EK9EJj6IQ/FFHIuJ0GIpfopf4nfvb/IyeZ28vUxNel3NM3HNkvf/ALiTxBY=</latexit>

x1<latexit sha1_base64="yxoXJf1YLSVLJJZWDJUMUYGrXYQ=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSenkrSbF+GrhpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+tQUrTP73k1u07d+/t3e8/ePjo8ZP9g6cnGKqoYaKDDfEsVwjWeJiQIQtnZQTlcgun+cXnNn76AyKa4L/TpoSZU0tvFkYrYte39Tyb7w/SUbo1eRNkHRiIzo7nBz09LYKuHHjSViGeZ2lJs1pFMtpC059WCKXSF2oJ5wy9coCzejtrI4fsKeQiRH6e5NZ7taJWDnHjcs50ila4G2ud/2PDq0Fat4y4058WH2e18WVF4PVl+0VlJQXZyiELE0GT3TBQOhreQOqVikoTi3aNqVW4eBMry78Ort0efeVyiFDw8nYZuHzlxtBIOZQsCwuEsptYdizSeCRlmaM/RSAdKs+N6iO1PlIUzZoPhk09Tps+nyXbPcJNcDIeZeko+/pucPipO9CeeC5eiFciEx/EofgijsVEaLEUP8Uv8bv3N3mZvE7eXqYmva7mmbhmyft/tn3EFQ==</latexit><latexit sha1_base64="yxoXJf1YLSVLJJZWDJUMUYGrXYQ=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSenkrSbF+GrhpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+tQUrTP73k1u07d+/t3e8/ePjo8ZP9g6cnGKqoYaKDDfEsVwjWeJiQIQtnZQTlcgun+cXnNn76AyKa4L/TpoSZU0tvFkYrYte39Tyb7w/SUbo1eRNkHRiIzo7nBz09LYKuHHjSViGeZ2lJs1pFMtpC059WCKXSF2oJ5wy9coCzejtrI4fsKeQiRH6e5NZ7taJWDnHjcs50ila4G2ud/2PDq0Fat4y4058WH2e18WVF4PVl+0VlJQXZyiELE0GT3TBQOhreQOqVikoTi3aNqVW4eBMry78Ort0efeVyiFDw8nYZuHzlxtBIOZQsCwuEsptYdizSeCRlmaM/RSAdKs+N6iO1PlIUzZoPhk09Tps+nyXbPcJNcDIeZeko+/pucPipO9CeeC5eiFciEx/EofgijsVEaLEUP8Uv8bv3N3mZvE7eXqYmva7mmbhmyft/tn3EFQ==</latexit><latexit sha1_base64="yxoXJf1YLSVLJJZWDJUMUYGrXYQ=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSenkrSbF+GrhpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+tQUrTP73k1u07d+/t3e8/ePjo8ZP9g6cnGKqoYaKDDfEsVwjWeJiQIQtnZQTlcgun+cXnNn76AyKa4L/TpoSZU0tvFkYrYte39Tyb7w/SUbo1eRNkHRiIzo7nBz09LYKuHHjSViGeZ2lJs1pFMtpC059WCKXSF2oJ5wy9coCzejtrI4fsKeQiRH6e5NZ7taJWDnHjcs50ila4G2ud/2PDq0Fat4y4058WH2e18WVF4PVl+0VlJQXZyiELE0GT3TBQOhreQOqVikoTi3aNqVW4eBMry78Ort0efeVyiFDw8nYZuHzlxtBIOZQsCwuEsptYdizSeCRlmaM/RSAdKs+N6iO1PlIUzZoPhk09Tps+nyXbPcJNcDIeZeko+/pucPipO9CeeC5eiFciEx/EofgijsVEaLEUP8Uv8bv3N3mZvE7eXqYmva7mmbhmyft/tn3EFQ==</latexit><latexit sha1_base64="yxoXJf1YLSVLJJZWDJUMUYGrXYQ=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSenkrSbF+GrhpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+tQUrTP73k1u07d+/t3e8/ePjo8ZP9g6cnGKqoYaKDDfEsVwjWeJiQIQtnZQTlcgun+cXnNn76AyKa4L/TpoSZU0tvFkYrYte39Tyb7w/SUbo1eRNkHRiIzo7nBz09LYKuHHjSViGeZ2lJs1pFMtpC059WCKXSF2oJ5wy9coCzejtrI4fsKeQiRH6e5NZ7taJWDnHjcs50ila4G2ud/2PDq0Fat4y4058WH2e18WVF4PVl+0VlJQXZyiELE0GT3TBQOhreQOqVikoTi3aNqVW4eBMry78Ort0efeVyiFDw8nYZuHzlxtBIOZQsCwuEsptYdizSeCRlmaM/RSAdKs+N6iO1PlIUzZoPhk09Tps+nyXbPcJNcDIeZeko+/pucPipO9CeeC5eiFciEx/EofgijsVEaLEUP8Uv8bv3N3mZvE7eXqYmva7mmbhmyft/tn3EFQ==</latexit>

x3<latexit sha1_base64="6/Adn5T2xpeKy+57uPL2wVqKY2c=">AAACfHicZVFbSxtBFJ6sbbXpRW0f+zI0Bgptw24stY9SX/oiWGpUMCGcnT1JBueyzJyVhGV/gq/62/wz4mxcisYDM3ycy3fO+U6aK+kpjm9b0dqLl6/WN16337x9935za/vDibeFEzgQVll3loJHJQ0OSJLCs9wh6FThaXpxUMdPL9F5ac0xLXIcaZgaOZECKLj+zce7461O3IuXxp+DpAEd1tjReLslhpkVhUZDQoH350mc06gER1IorNrDwmMO4gKmeB6gAY1+VC5nrXg3eDI+sS48Q3zpfVxRgvZ+odOQqYFmfjVWO//Huo+DNK8Z/Up/mvwaldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4eybK1T4hdX19t4UOkWHWVheTW0on+k+Vpx3eZAlCOR5MzFvWLg0nkAFjvbQIwlbmNCoPIT5IZCT83AwX5X9uGqHsySrR3gOTvq9JO4lf3909n83B9pgn9hn9oUlbI/tsz/siA2YYFN2xa7ZTesu2om+Rt8fUqNWU/ORPbHo5z26qcQX</latexit><latexit sha1_base64="6/Adn5T2xpeKy+57uPL2wVqKY2c=">AAACfHicZVFbSxtBFJ6sbbXpRW0f+zI0Bgptw24stY9SX/oiWGpUMCGcnT1JBueyzJyVhGV/gq/62/wz4mxcisYDM3ycy3fO+U6aK+kpjm9b0dqLl6/WN16337x9935za/vDibeFEzgQVll3loJHJQ0OSJLCs9wh6FThaXpxUMdPL9F5ac0xLXIcaZgaOZECKLj+zce7461O3IuXxp+DpAEd1tjReLslhpkVhUZDQoH350mc06gER1IorNrDwmMO4gKmeB6gAY1+VC5nrXg3eDI+sS48Q3zpfVxRgvZ+odOQqYFmfjVWO//Huo+DNK8Z/Up/mvwaldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4eybK1T4hdX19t4UOkWHWVheTW0on+k+Vpx3eZAlCOR5MzFvWLg0nkAFjvbQIwlbmNCoPIT5IZCT83AwX5X9uGqHsySrR3gOTvq9JO4lf3909n83B9pgn9hn9oUlbI/tsz/siA2YYFN2xa7ZTesu2om+Rt8fUqNWU/ORPbHo5z26qcQX</latexit><latexit sha1_base64="6/Adn5T2xpeKy+57uPL2wVqKY2c=">AAACfHicZVFbSxtBFJ6sbbXpRW0f+zI0Bgptw24stY9SX/oiWGpUMCGcnT1JBueyzJyVhGV/gq/62/wz4mxcisYDM3ycy3fO+U6aK+kpjm9b0dqLl6/WN16337x9935za/vDibeFEzgQVll3loJHJQ0OSJLCs9wh6FThaXpxUMdPL9F5ac0xLXIcaZgaOZECKLj+zce7461O3IuXxp+DpAEd1tjReLslhpkVhUZDQoH350mc06gER1IorNrDwmMO4gKmeB6gAY1+VC5nrXg3eDI+sS48Q3zpfVxRgvZ+odOQqYFmfjVWO//Huo+DNK8Z/Up/mvwaldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4eybK1T4hdX19t4UOkWHWVheTW0on+k+Vpx3eZAlCOR5MzFvWLg0nkAFjvbQIwlbmNCoPIT5IZCT83AwX5X9uGqHsySrR3gOTvq9JO4lf3909n83B9pgn9hn9oUlbI/tsz/siA2YYFN2xa7ZTesu2om+Rt8fUqNWU/ORPbHo5z26qcQX</latexit><latexit sha1_base64="6/Adn5T2xpeKy+57uPL2wVqKY2c=">AAACfHicZVFbSxtBFJ6sbbXpRW0f+zI0Bgptw24stY9SX/oiWGpUMCGcnT1JBueyzJyVhGV/gq/62/wz4mxcisYDM3ycy3fO+U6aK+kpjm9b0dqLl6/WN16337x9935za/vDibeFEzgQVll3loJHJQ0OSJLCs9wh6FThaXpxUMdPL9F5ac0xLXIcaZgaOZECKLj+zce7461O3IuXxp+DpAEd1tjReLslhpkVhUZDQoH350mc06gER1IorNrDwmMO4gKmeB6gAY1+VC5nrXg3eDI+sS48Q3zpfVxRgvZ+odOQqYFmfjVWO//Huo+DNK8Z/Up/mvwaldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4eybK1T4hdX19t4UOkWHWVheTW0on+k+Vpx3eZAlCOR5MzFvWLg0nkAFjvbQIwlbmNCoPIT5IZCT83AwX5X9uGqHsySrR3gOTvq9JO4lf3909n83B9pgn9hn9oUlbI/tsz/siA2YYFN2xa7ZTesu2om+Rt8fUqNWU/ORPbHo5z26qcQX</latexit>

x4<latexit sha1_base64="ymYlsYey5dDiZTvFzaA03SYXcOA=">AAACfHicZVFbSxtBFJ5sbavpxUsf+zIYA4W2YTdI9VHsS18ERaOCCWF29iQZnMsy56wkLPsT+mp/W/9M6WxcROOBGT7O5TvnfCfNtUKK47+t6NXa6zdv1zfa795/+Li5tb1zia7wEgbSaeevU4GglYUBKdJwnXsQJtVwld7+rONXd+BROXtBixxGRkytmigpKLjO5+P98VYn7sVL4y9B0oAOa+x0vN2Sw8zJwoAlqQXiTRLnNCqFJyU1VO1hgZALeSumcBOgFQZwVC5nrXg3eDI+cT48S3zpfVpRCoO4MGnINIJmuBqrnY+x7tMgzWtGXOlPk8NRqWxeEFj50H5SaE6O13LwTHmQpBcBCOlV2IDLmfBCUhDtGVOtcPbNFzr80pl6e7SFScFDFpbXUxfKZ6YPFeddHmQJAiFvJuYNC1cWSejA0R4ikHSFDY3KEzE/EeTVPBwMq7IfV+1wlmT1CC/BZb+XxL3kbL9zdNwcaJ19ZrvsC0vYATtiv9gpGzDJpuw3u2d/Wv+ivehr9P0hNWo1NZ/YM4t+/Ae8v8QY</latexit><latexit sha1_base64="ymYlsYey5dDiZTvFzaA03SYXcOA=">AAACfHicZVFbSxtBFJ5sbavpxUsf+zIYA4W2YTdI9VHsS18ERaOCCWF29iQZnMsy56wkLPsT+mp/W/9M6WxcROOBGT7O5TvnfCfNtUKK47+t6NXa6zdv1zfa795/+Li5tb1zia7wEgbSaeevU4GglYUBKdJwnXsQJtVwld7+rONXd+BROXtBixxGRkytmigpKLjO5+P98VYn7sVL4y9B0oAOa+x0vN2Sw8zJwoAlqQXiTRLnNCqFJyU1VO1hgZALeSumcBOgFQZwVC5nrXg3eDI+cT48S3zpfVpRCoO4MGnINIJmuBqrnY+x7tMgzWtGXOlPk8NRqWxeEFj50H5SaE6O13LwTHmQpBcBCOlV2IDLmfBCUhDtGVOtcPbNFzr80pl6e7SFScFDFpbXUxfKZ6YPFeddHmQJAiFvJuYNC1cWSejA0R4ikHSFDY3KEzE/EeTVPBwMq7IfV+1wlmT1CC/BZb+XxL3kbL9zdNwcaJ19ZrvsC0vYATtiv9gpGzDJpuw3u2d/Wv+ivehr9P0hNWo1NZ/YM4t+/Ae8v8QY</latexit><latexit sha1_base64="ymYlsYey5dDiZTvFzaA03SYXcOA=">AAACfHicZVFbSxtBFJ5sbavpxUsf+zIYA4W2YTdI9VHsS18ERaOCCWF29iQZnMsy56wkLPsT+mp/W/9M6WxcROOBGT7O5TvnfCfNtUKK47+t6NXa6zdv1zfa795/+Li5tb1zia7wEgbSaeevU4GglYUBKdJwnXsQJtVwld7+rONXd+BROXtBixxGRkytmigpKLjO5+P98VYn7sVL4y9B0oAOa+x0vN2Sw8zJwoAlqQXiTRLnNCqFJyU1VO1hgZALeSumcBOgFQZwVC5nrXg3eDI+cT48S3zpfVpRCoO4MGnINIJmuBqrnY+x7tMgzWtGXOlPk8NRqWxeEFj50H5SaE6O13LwTHmQpBcBCOlV2IDLmfBCUhDtGVOtcPbNFzr80pl6e7SFScFDFpbXUxfKZ6YPFeddHmQJAiFvJuYNC1cWSejA0R4ikHSFDY3KEzE/EeTVPBwMq7IfV+1wlmT1CC/BZb+XxL3kbL9zdNwcaJ19ZrvsC0vYATtiv9gpGzDJpuw3u2d/Wv+ivehr9P0hNWo1NZ/YM4t+/Ae8v8QY</latexit><latexit sha1_base64="ymYlsYey5dDiZTvFzaA03SYXcOA=">AAACfHicZVFbSxtBFJ5sbavpxUsf+zIYA4W2YTdI9VHsS18ERaOCCWF29iQZnMsy56wkLPsT+mp/W/9M6WxcROOBGT7O5TvnfCfNtUKK47+t6NXa6zdv1zfa795/+Li5tb1zia7wEgbSaeevU4GglYUBKdJwnXsQJtVwld7+rONXd+BROXtBixxGRkytmigpKLjO5+P98VYn7sVL4y9B0oAOa+x0vN2Sw8zJwoAlqQXiTRLnNCqFJyU1VO1hgZALeSumcBOgFQZwVC5nrXg3eDI+cT48S3zpfVpRCoO4MGnINIJmuBqrnY+x7tMgzWtGXOlPk8NRqWxeEFj50H5SaE6O13LwTHmQpBcBCOlV2IDLmfBCUhDtGVOtcPbNFzr80pl6e7SFScFDFpbXUxfKZ6YPFeddHmQJAiFvJuYNC1cWSejA0R4ikHSFDY3KEzE/EeTVPBwMq7IfV+1wlmT1CC/BZb+XxL3kbL9zdNwcaJ19ZrvsC0vYATtiv9gpGzDJpuw3u2d/Wv+ivehr9P0hNWo1NZ/YM4t+/Ae8v8QY</latexit> x5

<latexit sha1_base64="zcChtJTWIqDRnoNtwIAfDSKBozk=">AAACfHicZVFbSxtBFJ6sbbXpRW0f+zI0Bgptw26wtY9SX/oiWGpUMCGcnT1JBueyzJyVhGV/gq/62/wz4mxcisYDM3ycy3fO+U6aK+kpjm9b0dqLl6/WN16337x9935za/vDibeFEzgQVll3loJHJQ0OSJLCs9wh6FThaXpxUMdPL9F5ac0xLXIcaZgaOZECKLj+zcc/xluduBcvjT8HSQM6rLGj8XZLDDMrCo2GhALvz5M4p1EJjqRQWLWHhcccxAVM8TxAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NPk1KqXJC0IjHtpPCsXJ8loOnkmHgtQiABBOhg24mIEDQUG0J0y1wtk3V6jwC6vr7b0pdIoOs7C8mtpQPtN9rDjv8iBLEMjzZmLesHBpPIEKHO2hRxK2MKFReQjzQyAn5+Fgvir7cdUOZ0lWj/AcnPR7SdxL/u529n83B9pgn9hn9oUlbI/tsz/siA2YYFN2xa7ZTesu2om+Rt8fUqNWU/ORPbHo5z2+1cQZ</latexit><latexit sha1_base64="zcChtJTWIqDRnoNtwIAfDSKBozk=">AAACfHicZVFbSxtBFJ6sbbXpRW0f+zI0Bgptw26wtY9SX/oiWGpUMCGcnT1JBueyzJyVhGV/gq/62/wz4mxcisYDM3ycy3fO+U6aK+kpjm9b0dqLl6/WN16337x9935za/vDibeFEzgQVll3loJHJQ0OSJLCs9wh6FThaXpxUMdPL9F5ac0xLXIcaZgaOZECKLj+zcc/xluduBcvjT8HSQM6rLGj8XZLDDMrCo2GhALvz5M4p1EJjqRQWLWHhcccxAVM8TxAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NPk1KqXJC0IjHtpPCsXJ8loOnkmHgtQiABBOhg24mIEDQUG0J0y1wtk3V6jwC6vr7b0pdIoOs7C8mtpQPtN9rDjv8iBLEMjzZmLesHBpPIEKHO2hRxK2MKFReQjzQyAn5+Fgvir7cdUOZ0lWj/AcnPR7SdxL/u529n83B9pgn9hn9oUlbI/tsz/siA2YYFN2xa7ZTesu2om+Rt8fUqNWU/ORPbHo5z2+1cQZ</latexit><latexit sha1_base64="zcChtJTWIqDRnoNtwIAfDSKBozk=">AAACfHicZVFbSxtBFJ6sbbXpRW0f+zI0Bgptw26wtY9SX/oiWGpUMCGcnT1JBueyzJyVhGV/gq/62/wz4mxcisYDM3ycy3fO+U6aK+kpjm9b0dqLl6/WN16337x9935za/vDibeFEzgQVll3loJHJQ0OSJLCs9wh6FThaXpxUMdPL9F5ac0xLXIcaZgaOZECKLj+zcc/xluduBcvjT8HSQM6rLGj8XZLDDMrCo2GhALvz5M4p1EJjqRQWLWHhcccxAVM8TxAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NPk1KqXJC0IjHtpPCsXJ8loOnkmHgtQiABBOhg24mIEDQUG0J0y1wtk3V6jwC6vr7b0pdIoOs7C8mtpQPtN9rDjv8iBLEMjzZmLesHBpPIEKHO2hRxK2MKFReQjzQyAn5+Fgvir7cdUOZ0lWj/AcnPR7SdxL/u529n83B9pgn9hn9oUlbI/tsz/siA2YYFN2xa7ZTesu2om+Rt8fUqNWU/ORPbHo5z2+1cQZ</latexit><latexit sha1_base64="zcChtJTWIqDRnoNtwIAfDSKBozk=">AAACfHicZVFbSxtBFJ6sbbXpRW0f+zI0Bgptw26wtY9SX/oiWGpUMCGcnT1JBueyzJyVhGV/gq/62/wz4mxcisYDM3ycy3fO+U6aK+kpjm9b0dqLl6/WN16337x9935za/vDibeFEzgQVll3loJHJQ0OSJLCs9wh6FThaXpxUMdPL9F5ac0xLXIcaZgaOZECKLj+zcc/xluduBcvjT8HSQM6rLGj8XZLDDMrCo2GhALvz5M4p1EJjqRQWLWHhcccxAVM8TxAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NPk1KqXJC0IjHtpPCsXJ8loOnkmHgtQiABBOhg24mIEDQUG0J0y1wtk3V6jwC6vr7b0pdIoOs7C8mtpQPtN9rDjv8iBLEMjzZmLesHBpPIEKHO2hRxK2MKFReQjzQyAn5+Fgvir7cdUOZ0lWj/AcnPR7SdxL/u529n83B9pgn9hn9oUlbI/tsz/siA2YYFN2xa7ZTesu2om+Rt8fUqNWU/ORPbHo5z2+1cQZ</latexit>

p(x) =1

Z 123(x1, x2, x3) 234(x2, x3, x4) 35(x3, x5)

<latexit sha1_base64="/rYlOHZIsgBXtbd4AlhTG6ybBfo=">AAAC1nicZVLNahsxEJa3f6n757THXkSNwYZgdtcOzSUQ2ksvgRTqxNRrFq1Wa4voZ5G0ZY3Y3kqvfaU+Rp+g1/YNKtlLSZyBER/faGY03ygrGdUmDH91gnv3Hzx8dPC4++Tps+cveocvL7WsFCYzLJlU8wxpwqggM0MNI/NSEcQzRq6y6/c+fvWFKE2l+GQ2JVlytBK0oBgZR6W9RTlMODLrrLB1M4KnMCkUwjZq7OcmKTVNbRRPmmGdRkd1GjufjHZ0PJl6eks5n7b05Niznjkepb1+OA63Bu+CqAV90NpFetjBSS5xxYkwmCGtF1FYmqVFylDMSNNNKk1KhK/RiiwcFIgTvbRbFRo4cEwOC6mcCwO37M0Mi7jWG565m35ivR/z5P/Y4GbQ1L6i3utvipOlpaKsDBF4176oGDQSeqFhThXBhm0cQFhRNwHEa+TENW4dtyr53eVHqmLuxJL76bWoeEYUyd3wbCVd+prHpIFwAJ0sTiAN2xfDtgqkQhvEXI1uoonBshKukT1H9TkyitbuK+jGxmHTdWuJ9pdwF1zG4ygcRx+n/bN37YIOwGvwBgxBBN6CM/ABXIAZwOAn+A3+gL/BPPgafAu+764GnTbnFbhlwY9/dYHkUA==</latexit><latexit sha1_base64="/rYlOHZIsgBXtbd4AlhTG6ybBfo=">AAAC1nicZVLNahsxEJa3f6n757THXkSNwYZgdtcOzSUQ2ksvgRTqxNRrFq1Wa4voZ5G0ZY3Y3kqvfaU+Rp+g1/YNKtlLSZyBER/faGY03ygrGdUmDH91gnv3Hzx8dPC4++Tps+cveocvL7WsFCYzLJlU8wxpwqggM0MNI/NSEcQzRq6y6/c+fvWFKE2l+GQ2JVlytBK0oBgZR6W9RTlMODLrrLB1M4KnMCkUwjZq7OcmKTVNbRRPmmGdRkd1GjufjHZ0PJl6eks5n7b05Niznjkepb1+OA63Bu+CqAV90NpFetjBSS5xxYkwmCGtF1FYmqVFylDMSNNNKk1KhK/RiiwcFIgTvbRbFRo4cEwOC6mcCwO37M0Mi7jWG565m35ivR/z5P/Y4GbQ1L6i3utvipOlpaKsDBF4176oGDQSeqFhThXBhm0cQFhRNwHEa+TENW4dtyr53eVHqmLuxJL76bWoeEYUyd3wbCVd+prHpIFwAJ0sTiAN2xfDtgqkQhvEXI1uoonBshKukT1H9TkyitbuK+jGxmHTdWuJ9pdwF1zG4ygcRx+n/bN37YIOwGvwBgxBBN6CM/ABXIAZwOAn+A3+gL/BPPgafAu+764GnTbnFbhlwY9/dYHkUA==</latexit><latexit sha1_base64="/rYlOHZIsgBXtbd4AlhTG6ybBfo=">AAAC1nicZVLNahsxEJa3f6n757THXkSNwYZgdtcOzSUQ2ksvgRTqxNRrFq1Wa4voZ5G0ZY3Y3kqvfaU+Rp+g1/YNKtlLSZyBER/faGY03ygrGdUmDH91gnv3Hzx8dPC4++Tps+cveocvL7WsFCYzLJlU8wxpwqggM0MNI/NSEcQzRq6y6/c+fvWFKE2l+GQ2JVlytBK0oBgZR6W9RTlMODLrrLB1M4KnMCkUwjZq7OcmKTVNbRRPmmGdRkd1GjufjHZ0PJl6eks5n7b05Niznjkepb1+OA63Bu+CqAV90NpFetjBSS5xxYkwmCGtF1FYmqVFylDMSNNNKk1KhK/RiiwcFIgTvbRbFRo4cEwOC6mcCwO37M0Mi7jWG565m35ivR/z5P/Y4GbQ1L6i3utvipOlpaKsDBF4176oGDQSeqFhThXBhm0cQFhRNwHEa+TENW4dtyr53eVHqmLuxJL76bWoeEYUyd3wbCVd+prHpIFwAJ0sTiAN2xfDtgqkQhvEXI1uoonBshKukT1H9TkyitbuK+jGxmHTdWuJ9pdwF1zG4ygcRx+n/bN37YIOwGvwBgxBBN6CM/ABXIAZwOAn+A3+gL/BPPgafAu+764GnTbnFbhlwY9/dYHkUA==</latexit><latexit sha1_base64="/rYlOHZIsgBXtbd4AlhTG6ybBfo=">AAAC1nicZVLNahsxEJa3f6n757THXkSNwYZgdtcOzSUQ2ksvgRTqxNRrFq1Wa4voZ5G0ZY3Y3kqvfaU+Rp+g1/YNKtlLSZyBER/faGY03ygrGdUmDH91gnv3Hzx8dPC4++Tps+cveocvL7WsFCYzLJlU8wxpwqggM0MNI/NSEcQzRq6y6/c+fvWFKE2l+GQ2JVlytBK0oBgZR6W9RTlMODLrrLB1M4KnMCkUwjZq7OcmKTVNbRRPmmGdRkd1GjufjHZ0PJl6eks5n7b05Niznjkepb1+OA63Bu+CqAV90NpFetjBSS5xxYkwmCGtF1FYmqVFylDMSNNNKk1KhK/RiiwcFIgTvbRbFRo4cEwOC6mcCwO37M0Mi7jWG565m35ivR/z5P/Y4GbQ1L6i3utvipOlpaKsDBF4176oGDQSeqFhThXBhm0cQFhRNwHEa+TENW4dtyr53eVHqmLuxJL76bWoeEYUyd3wbCVd+prHpIFwAJ0sTiAN2xfDtgqkQhvEXI1uoonBshKukT1H9TkyitbuK+jGxmHTdWuJ9pdwF1zG4ygcRx+n/bN37YIOwGvwBgxBBN6CM/ABXIAZwOAn+A3+gL/BPPgafAu+764GnTbnFbhlwY9/dYHkUA==</latexit>



A Simple Example

How to define the potentials? • Intuitively, the potential of a

clique should be high, iff the joint probability of the corres-ponding random variables is high.

!7






p(x) =1

Z 123(x1, x2, x3) 234(x2, x3, x4) 35(x3, x5)

<latexit sha1_base64="/rYlOHZIsgBXtbd4AlhTG6ybBfo=">AAAC1nicZVLNahsxEJa3f6n757THXkSNwYZgdtcOzSUQ2ksvgRTqxNRrFq1Wa4voZ5G0ZY3Y3kqvfaU+Rp+g1/YNKtlLSZyBER/faGY03ygrGdUmDH91gnv3Hzx8dPC4++Tps+cveocvL7WsFCYzLJlU8wxpwqggM0MNI/NSEcQzRq6y6/c+fvWFKE2l+GQ2JVlytBK0oBgZR6W9RTlMODLrrLB1M4KnMCkUwjZq7OcmKTVNbRRPmmGdRkd1GjufjHZ0PJl6eks5n7b05Niznjkepb1+OA63Bu+CqAV90NpFetjBSS5xxYkwmCGtF1FYmqVFylDMSNNNKk1KhK/RiiwcFIgTvbRbFRo4cEwOC6mcCwO37M0Mi7jWG565m35ivR/z5P/Y4GbQ1L6i3utvipOlpaKsDBF4176oGDQSeqFhThXBhm0cQFhRNwHEa+TENW4dtyr53eVHqmLuxJL76bWoeEYUyd3wbCVd+prHpIFwAJ0sTiAN2xfDtgqkQhvEXI1uoonBshKukT1H9TkyitbuK+jGxmHTdWuJ9pdwF1zG4ygcRx+n/bN37YIOwGvwBgxBBN6CM/ABXIAZwOAn+A3+gL/BPPgafAu+764GnTbnFbhlwY9/dYHkUA==</latexit><latexit sha1_base64="/rYlOHZIsgBXtbd4AlhTG6ybBfo=">AAAC1nicZVLNahsxEJa3f6n757THXkSNwYZgdtcOzSUQ2ksvgRTqxNRrFq1Wa4voZ5G0ZY3Y3kqvfaU+Rp+g1/YNKtlLSZyBER/faGY03ygrGdUmDH91gnv3Hzx8dPC4++Tps+cveocvL7WsFCYzLJlU8wxpwqggM0MNI/NSEcQzRq6y6/c+fvWFKE2l+GQ2JVlytBK0oBgZR6W9RTlMODLrrLB1M4KnMCkUwjZq7OcmKTVNbRRPmmGdRkd1GjufjHZ0PJl6eks5n7b05Niznjkepb1+OA63Bu+CqAV90NpFetjBSS5xxYkwmCGtF1FYmqVFylDMSNNNKk1KhK/RiiwcFIgTvbRbFRo4cEwOC6mcCwO37M0Mi7jWG565m35ivR/z5P/Y4GbQ1L6i3utvipOlpaKsDBF4176oGDQSeqFhThXBhm0cQFhRNwHEa+TENW4dtyr53eVHqmLuxJL76bWoeEYUyd3wbCVd+prHpIFwAJ0sTiAN2xfDtgqkQhvEXI1uoonBshKukT1H9TkyitbuK+jGxmHTdWuJ9pdwF1zG4ygcRx+n/bN37YIOwGvwBgxBBN6CM/ABXIAZwOAn+A3+gL/BPPgafAu+764GnTbnFbhlwY9/dYHkUA==</latexit><latexit sha1_base64="/rYlOHZIsgBXtbd4AlhTG6ybBfo=">AAAC1nicZVLNahsxEJa3f6n757THXkSNwYZgdtcOzSUQ2ksvgRTqxNRrFq1Wa4voZ5G0ZY3Y3kqvfaU+Rp+g1/YNKtlLSZyBER/faGY03ygrGdUmDH91gnv3Hzx8dPC4++Tps+cveocvL7WsFCYzLJlU8wxpwqggM0MNI/NSEcQzRq6y6/c+fvWFKE2l+GQ2JVlytBK0oBgZR6W9RTlMODLrrLB1M4KnMCkUwjZq7OcmKTVNbRRPmmGdRkd1GjufjHZ0PJl6eks5n7b05Niznjkepb1+OA63Bu+CqAV90NpFetjBSS5xxYkwmCGtF1FYmqVFylDMSNNNKk1KhK/RiiwcFIgTvbRbFRo4cEwOC6mcCwO37M0Mi7jWG565m35ivR/z5P/Y4GbQ1L6i3utvipOlpaKsDBF4176oGDQSeqFhThXBhm0cQFhRNwHEa+TENW4dtyr53eVHqmLuxJL76bWoeEYUyd3wbCVd+prHpIFwAJ0sTiAN2xfDtgqkQhvEXI1uoonBshKukT1H9TkyitbuK+jGxmHTdWuJ9pdwF1zG4ygcRx+n/bN37YIOwGvwBgxBBN6CM/ABXIAZwOAn+A3+gL/BPPgafAu+764GnTbnFbhlwY9/dYHkUA==</latexit><latexit sha1_base64="/rYlOHZIsgBXtbd4AlhTG6ybBfo=">AAAC1nicZVLNahsxEJa3f6n757THXkSNwYZgdtcOzSUQ2ksvgRTqxNRrFq1Wa4voZ5G0ZY3Y3kqvfaU+Rp+g1/YNKtlLSZyBER/faGY03ygrGdUmDH91gnv3Hzx8dPC4++Tps+cveocvL7WsFCYzLJlU8wxpwqggM0MNI/NSEcQzRq6y6/c+fvWFKE2l+GQ2JVlytBK0oBgZR6W9RTlMODLrrLB1M4KnMCkUwjZq7OcmKTVNbRRPmmGdRkd1GjufjHZ0PJl6eks5n7b05Niznjkepb1+OA63Bu+CqAV90NpFetjBSS5xxYkwmCGtF1FYmqVFylDMSNNNKk1KhK/RiiwcFIgTvbRbFRo4cEwOC6mcCwO37M0Mi7jWG565m35ivR/z5P/Y4GbQ1L6i3utvipOlpaKsDBF4176oGDQSeqFhThXBhm0cQFhRNwHEa+TENW4dtyr53eVHqmLuxJL76bWoeEYUyd3wbCVd+prHpIFwAJ0sTiAN2xfDtgqkQhvEXI1uoonBshKukT1H9TkyitbuK+jGxmHTdWuJ9pdwF1zG4ygcRx+n/bN37YIOwGvwBgxBBN6CM/ABXIAZwOAn+A3+gL/BPPgafAu+764GnTbnFbhlwY9/dYHkUA==</latexit>



A Simple Example



• In most cases the potential is defined using a log-linear model:

!8






log c(xc) = �c(xc)T✓c

<latexit sha1_base64="7nocstFofSq12/N1J5fFqQZpLx4=">AAACzHicZVLLattAFB0rfaTuy2mX3Qw1hhSKkUyg3QRCu8kmJYE4CUSuGI2urCHzEDNXwUZo21/Kt2TRbfsbHTmi2M4FSYd7z30epaUUDsPwvhfsPHn67Pnui/7LV6/fvB3svbtwprIcptxIY69S5kAKDVMUKOGqtMBUKuEyvfnexi9vwTph9DkuS5gpNtciF5yhdyWDs1iaOY1LJxJO92PFsEjzetEk/BM9pHFqZOaWyn/isvCUTcbP83UCFoAs4clgGI7DldHHIOrAkHR2muz1eJwZXinQyCVz7joKS5zVzKLgEpp+XDkoGb9hc7j2UDMFblavdm/oyHsymhvrH4105V3PqJly7YCe2Y7utmOt839stB7ERVvRbfXH/OusFrqsEDR/aJ9XkqKh7XlpJixwlEsPGLfCb0B5wSzj6EXYqNQqln22lfRvblS7vdOVSsFC5peXc+PTCzWBhtIR9WfxB3K0m5h2VajQDpn0NfqxA+Sm0r5RfcIWJwytWPgfwDX1JGz6XpZoW4TH4GIyjsJxdHYwPPrWCbRLPpCPZJ9E5As5IsfklEwJJ3fkN/lD/gY/AgzqoHmgBr0u5z3ZsODXP3tW5B0=</latexit><latexit sha1_base64="7nocstFofSq12/N1J5fFqQZpLx4=">AAACzHicZVLLattAFB0rfaTuy2mX3Qw1hhSKkUyg3QRCu8kmJYE4CUSuGI2urCHzEDNXwUZo21/Kt2TRbfsbHTmi2M4FSYd7z30epaUUDsPwvhfsPHn67Pnui/7LV6/fvB3svbtwprIcptxIY69S5kAKDVMUKOGqtMBUKuEyvfnexi9vwTph9DkuS5gpNtciF5yhdyWDs1iaOY1LJxJO92PFsEjzetEk/BM9pHFqZOaWyn/isvCUTcbP83UCFoAs4clgGI7DldHHIOrAkHR2muz1eJwZXinQyCVz7joKS5zVzKLgEpp+XDkoGb9hc7j2UDMFblavdm/oyHsymhvrH4105V3PqJly7YCe2Y7utmOt839stB7ERVvRbfXH/OusFrqsEDR/aJ9XkqKh7XlpJixwlEsPGLfCb0B5wSzj6EXYqNQqln22lfRvblS7vdOVSsFC5peXc+PTCzWBhtIR9WfxB3K0m5h2VajQDpn0NfqxA+Sm0r5RfcIWJwytWPgfwDX1JGz6XpZoW4TH4GIyjsJxdHYwPPrWCbRLPpCPZJ9E5As5IsfklEwJJ3fkN/lD/gY/AgzqoHmgBr0u5z3ZsODXP3tW5B0=</latexit><latexit sha1_base64="7nocstFofSq12/N1J5fFqQZpLx4=">AAACzHicZVLLattAFB0rfaTuy2mX3Qw1hhSKkUyg3QRCu8kmJYE4CUSuGI2urCHzEDNXwUZo21/Kt2TRbfsbHTmi2M4FSYd7z30epaUUDsPwvhfsPHn67Pnui/7LV6/fvB3svbtwprIcptxIY69S5kAKDVMUKOGqtMBUKuEyvfnexi9vwTph9DkuS5gpNtciF5yhdyWDs1iaOY1LJxJO92PFsEjzetEk/BM9pHFqZOaWyn/isvCUTcbP83UCFoAs4clgGI7DldHHIOrAkHR2muz1eJwZXinQyCVz7joKS5zVzKLgEpp+XDkoGb9hc7j2UDMFblavdm/oyHsymhvrH4105V3PqJly7YCe2Y7utmOt839stB7ERVvRbfXH/OusFrqsEDR/aJ9XkqKh7XlpJixwlEsPGLfCb0B5wSzj6EXYqNQqln22lfRvblS7vdOVSsFC5peXc+PTCzWBhtIR9WfxB3K0m5h2VajQDpn0NfqxA+Sm0r5RfcIWJwytWPgfwDX1JGz6XpZoW4TH4GIyjsJxdHYwPPrWCbRLPpCPZJ9E5As5IsfklEwJJ3fkN/lD/gY/AgzqoHmgBr0u5z3ZsODXP3tW5B0=</latexit><latexit sha1_base64="7nocstFofSq12/N1J5fFqQZpLx4=">AAACzHicZVLLattAFB0rfaTuy2mX3Qw1hhSKkUyg3QRCu8kmJYE4CUSuGI2urCHzEDNXwUZo21/Kt2TRbfsbHTmi2M4FSYd7z30epaUUDsPwvhfsPHn67Pnui/7LV6/fvB3svbtwprIcptxIY69S5kAKDVMUKOGqtMBUKuEyvfnexi9vwTph9DkuS5gpNtciF5yhdyWDs1iaOY1LJxJO92PFsEjzetEk/BM9pHFqZOaWyn/isvCUTcbP83UCFoAs4clgGI7DldHHIOrAkHR2muz1eJwZXinQyCVz7joKS5zVzKLgEpp+XDkoGb9hc7j2UDMFblavdm/oyHsymhvrH4105V3PqJly7YCe2Y7utmOt839stB7ERVvRbfXH/OusFrqsEDR/aJ9XkqKh7XlpJixwlEsPGLfCb0B5wSzj6EXYqNQqln22lfRvblS7vdOVSsFC5peXc+PTCzWBhtIR9WfxB3K0m5h2VajQDpn0NfqxA+Sm0r5RfcIWJwytWPgfwDX1JGz6XpZoW4TH4GIyjsJxdHYwPPrWCbRLPpCPZJ9E5As5IsfklEwJJ3fkN/lD/gY/AgzqoHmgBr0u5z3ZsODXP3tW5B0=</latexit>

Feature function Parameters



A Simple Example



• In most cases the potential is defined using a log-linear model

making the parameters explicit:

!9






log c(xc) = �c(xc)T✓c

<latexit sha1_base64="7nocstFofSq12/N1J5fFqQZpLx4=">AAACzHicZVLLattAFB0rfaTuy2mX3Qw1hhSKkUyg3QRCu8kmJYE4CUSuGI2urCHzEDNXwUZo21/Kt2TRbfsbHTmi2M4FSYd7z30epaUUDsPwvhfsPHn67Pnui/7LV6/fvB3svbtwprIcptxIY69S5kAKDVMUKOGqtMBUKuEyvfnexi9vwTph9DkuS5gpNtciF5yhdyWDs1iaOY1LJxJO92PFsEjzetEk/BM9pHFqZOaWyn/isv CUTcbP83UCFoAs4clgGI7DldHHIOrAkHR2muz1eJwZXinQyCVz7joKS5zVzKLgEpp+XDkoGb9hc7j2UDMFblavdm/oyHsymhvrH4105V3PqJly7YCe2Y7utmOt839stB7ERVvRbfXH/OusFrqsEDR/aJ9XkqKh7XlpJixwlEsPGLfCb0B5wSzj6EXYqNQqln22lfRvblS7vdOVSsFC5peXc+PTCzWBhtIR9WfxB3K0m5h2VajQDpn0NfqxA+Sm0r5RfcIWJwytWPgfwDX1JGz6XpZoW4TH4GIyjsJxdHYwPPrWCbRLPpCPZJ9E5As5IsfklEwJJ3fkN/lD/gY/AgzqoHmgBr0u5z3ZsODXP3tW5B0=</latexit><latexit sha1_base64="7nocstFofSq12/N1J5fFqQZpLx4=">AAACzHicZVLLattAFB0rfaTuy2mX3Qw1hhSKkUyg3QRCu8kmJYE4CUSuGI2urCHzEDNXwUZo21/Kt2TRbfsbHTmi2M4FSYd7z30epaUUDsPwvhfsPHn67Pnui/7LV6/fvB3svbtwprIcptxIY69S5kAKDVMUKOGqtMBUKuEyvfnexi9vwTph9DkuS5gpNtciF5yhdyWDs1iaOY1LJxJO92PFsEjzetEk/BM9pHFqZOaWyn/isv CUTcbP83UCFoAs4clgGI7DldHHIOrAkHR2muz1eJwZXinQyCVz7joKS5zVzKLgEpp+XDkoGb9hc7j2UDMFblavdm/oyHsymhvrH4105V3PqJly7YCe2Y7utmOt839stB7ERVvRbfXH/OusFrqsEDR/aJ9XkqKh7XlpJixwlEsPGLfCb0B5wSzj6EXYqNQqln22lfRvblS7vdOVSsFC5peXc+PTCzWBhtIR9WfxB3K0m5h2VajQDpn0NfqxA+Sm0r5RfcIWJwytWPgfwDX1JGz6XpZoW4TH4GIyjsJxdHYwPPrWCbRLPpCPZJ9E5As5IsfklEwJJ3fkN/lD/gY/AgzqoHmgBr0u5z3ZsODXP3tW5B0=</latexit><latexit sha1_base64="7nocstFofSq12/N1J5fFqQZpLx4=">AAACzHicZVLLattAFB0rfaTuy2mX3Qw1hhSKkUyg3QRCu8kmJYE4CUSuGI2urCHzEDNXwUZo21/Kt2TRbfsbHTmi2M4FSYd7z30epaUUDsPwvhfsPHn67Pnui/7LV6/fvB3svbtwprIcptxIY69S5kAKDVMUKOGqtMBUKuEyvfnexi9vwTph9DkuS5gpNtciF5yhdyWDs1iaOY1LJxJO92PFsEjzetEk/BM9pHFqZOaWyn/isv CUTcbP83UCFoAs4clgGI7DldHHIOrAkHR2muz1eJwZXinQyCVz7joKS5zVzKLgEpp+XDkoGb9hc7j2UDMFblavdm/oyHsymhvrH4105V3PqJly7YCe2Y7utmOt839stB7ERVvRbfXH/OusFrqsEDR/aJ9XkqKh7XlpJixwlEsPGLfCb0B5wSzj6EXYqNQqln22lfRvblS7vdOVSsFC5peXc+PTCzWBhtIR9WfxB3K0m5h2VajQDpn0NfqxA+Sm0r5RfcIWJwytWPgfwDX1JGz6XpZoW4TH4GIyjsJxdHYwPPrWCbRLPpCPZJ9E5As5IsfklEwJJ3fkN/lD/gY/AgzqoHmgBr0u5z3ZsODXP3tW5B0=</latexit><latexit sha1_base64="7nocstFofSq12/N1J5fFqQZpLx4=">AAACzHicZVLLattAFB0rfaTuy2mX3Qw1hhSKkUyg3QRCu8kmJYE4CUSuGI2urCHzEDNXwUZo21/Kt2TRbfsbHTmi2M4FSYd7z30epaUUDsPwvhfsPHn67Pnui/7LV6/fvB3svbtwprIcptxIY69S5kAKDVMUKOGqtMBUKuEyvfnexi9vwTph9DkuS5gpNtciF5yhdyWDs1iaOY1LJxJO92PFsEjzetEk/BM9pHFqZOaWyn/isv CUTcbP83UCFoAs4clgGI7DldHHIOrAkHR2muz1eJwZXinQyCVz7joKS5zVzKLgEpp+XDkoGb9hc7j2UDMFblavdm/oyHsymhvrH4105V3PqJly7YCe2Y7utmOt839stB7ERVvRbfXH/OusFrqsEDR/aJ9XkqKh7XlpJixwlEsPGLfCb0B5wSzj6EXYqNQqln22lfRvblS7vdOVSsFC5peXc+PTCzWBhtIR9WfxB3K0m5h2VajQDpn0NfqxA+Sm0r5RfcIWJwytWPgfwDX1JGz6XpZoW4TH4GIyjsJxdHYwPPrWCbRLPpCPZJ9E5As5IsfklEwJJ3fkN/lD/gY/AgzqoHmgBr0u5z3ZsODXP3tW5B0=</latexit>

) log p(x | ✓) =X

c2C�c(xc)

T✓c � logZ(✓)<latexit sha1_base64="/JZXOoWv9G/jQKi3ZzilkndyxZQ=">AAADG3icZVLLbtNAFB0bKCW8UliyuSKKlEolsiNeG6SKbthUKqhpKzrBGo8n8ajzsGbGkMjypyDxL+wQWxb8DePEQnlcyZ6j+zh37j2TFoJbF0V/g/DW7Tt7d/fvde4/ePjocffgyYXVpaFsTLXQ5iollgmu2NhxJ9hVYRiRqWCX6c1JE7/8yozlWp27RcEmkswUn3JKnHcl3R/4E5/ljhijvwEWegYFDLAkLk+n1bwGLHkGONUiswvpD+xy5sghvANsS5lUFHO1TKdEVCd1vZFb5Dyha2wJPfxyvkOWUHixav15sNsp6faiYbQ02AVxC3qotbPkIKA407SUTDkqiLXXcVS4SUWM41SwuoNLywpCb8iMXXuoiGR2Ui1XWUPfezKYauM/5WDpXa+oiLTN/XxmM5bdjjXO/7H+etDNG0a71d9N304qrorSMUVX7aelAKehUQsybhh1YuEBoYb7CYDmxBDqvKYbTM0DyI5MKfyfatlMb1UpU2ZY5ocXM+3LczliNUAf/Fr8giy0N4aWBbiyjgjP0cGWOapL5RtVp2R+Spzhc/+ebF2NorrjZYm3RdgFF6NhHA3jjy97x+9bgfbRM/QcDVCM3qBj9AGdoTGiwV5wFLwKXoffw5/hr/D3KjUM2pqnaMPCP/8A/N8BFA==</latexit><latexit sha1_base64="/JZXOoWv9G/jQKi3ZzilkndyxZQ=">AAADG3icZVLLbtNAFB0bKCW8UliyuSKKlEolsiNeG6SKbthUKqhpKzrBGo8n8ajzsGbGkMjypyDxL+wQWxb8DePEQnlcyZ6j+zh37j2TFoJbF0V/g/DW7Tt7d/fvde4/ePjocffgyYXVpaFsTLXQ5iollgmu2NhxJ9hVYRiRqWCX6c1JE7/8yozlWp27RcEmkswUn3JKnHcl3R/4E5/ljhijvwEWegYFDLAkLk+n1bwGLHkGONUiswvpD+xy5sghvANsS5lUFHO1TKdEVCd1vZFb5Dyha2wJPfxyvkOWUHixav15sNsp6faiYbQ02AVxC3qotbPkIKA407SUTDkqiLXXcVS4SUWM41SwuoNLywpCb8iMXXuoiGR2Ui1XWUPfezKYauM/5WDpXa+oiLTN/XxmM5bdjjXO/7H+etDNG0a71d9N304qrorSMUVX7aelAKehUQsybhh1YuEBoYb7CYDmxBDqvKYbTM0DyI5MKfyfatlMb1UpU2ZY5ocXM+3LczliNUAf/Fr8giy0N4aWBbiyjgjP0cGWOapL5RtVp2R+Spzhc/+ebF2NorrjZYm3RdgFF6NhHA3jjy97x+9bgfbRM/QcDVCM3qBj9AGdoTGiwV5wFLwKXoffw5/hr/D3KjUM2pqnaMPCP/8A/N8BFA==</latexit><latexit sha1_base64="/JZXOoWv9G/jQKi3ZzilkndyxZQ=">AAADG3icZVLLbtNAFB0bKCW8UliyuSKKlEolsiNeG6SKbthUKqhpKzrBGo8n8ajzsGbGkMjypyDxL+wQWxb8DePEQnlcyZ6j+zh37j2TFoJbF0V/g/DW7Tt7d/fvde4/ePjocffgyYXVpaFsTLXQ5iollgmu2NhxJ9hVYRiRqWCX6c1JE7/8yozlWp27RcEmkswUn3JKnHcl3R/4E5/ljhijvwEWegYFDLAkLk+n1bwGLHkGONUiswvpD+xy5sghvANsS5lUFHO1TKdEVCd1vZFb5Dyha2wJPfxyvkOWUHixav15sNsp6faiYbQ02AVxC3qotbPkIKA407SUTDkqiLXXcVS4SUWM41SwuoNLywpCb8iMXXuoiGR2Ui1XWUPfezKYauM/5WDpXa+oiLTN/XxmM5bdjjXO/7H+etDNG0a71d9N304qrorSMUVX7aelAKehUQsybhh1YuEBoYb7CYDmxBDqvKYbTM0DyI5MKfyfatlMb1UpU2ZY5ocXM+3LczliNUAf/Fr8giy0N4aWBbiyjgjP0cGWOapL5RtVp2R+Spzhc/+ebF2NorrjZYm3RdgFF6NhHA3jjy97x+9bgfbRM/QcDVCM3qBj9AGdoTGiwV5wFLwKXoffw5/hr/D3KjUM2pqnaMPCP/8A/N8BFA==</latexit><latexit sha1_base64="/JZXOoWv9G/jQKi3ZzilkndyxZQ=">AAADG3icZVLLbtNAFB0bKCW8UliyuSKKlEolsiNeG6SKbthUKqhpKzrBGo8n8ajzsGbGkMjypyDxL+wQWxb8DePEQnlcyZ6j+zh37j2TFoJbF0V/g/DW7Tt7d/fvde4/ePjocffgyYXVpaFsTLXQ5iollgmu2NhxJ9hVYRiRqWCX6c1JE7/8yozlWp27RcEmkswUn3JKnHcl3R/4E5/ljhijvwEWegYFDLAkLk+n1bwGLHkGONUiswvpD+xy5sghvANsS5lUFHO1TKdEVCd1vZFb5Dyha2wJPfxyvkOWUHixav15sNsp6faiYbQ02AVxC3qotbPkIKA407SUTDkqiLXXcVS4SUWM41SwuoNLywpCb8iMXXuoiGR2Ui1XWUPfezKYauM/5WDpXa+oiLTN/XxmM5bdjjXO/7H+etDNG0a71d9N304qrorSMUVX7aelAKehUQsybhh1YuEBoYb7CYDmxBDqvKYbTM0DyI5MKfyfatlMb1UpU2ZY5ocXM+3LczliNUAf/Fr8giy0N4aWBbiyjgjP0cGWOapL5RtVp2R+Spzhc/+ebF2NorrjZYm3RdgFF6NhHA3jjy97x+9bgfbRM/QcDVCM3qBj9AGdoTGiwV5wFLwKXoffw5/hr/D3KjUM2pqnaMPCP/8A/N8BFA==</latexit>



A Simple Example

Using numbers, e.g.: • Let all variables be binary:

• We can define features

!10






xi 2 {0, 1}<latexit sha1_base64="mFCz3MOySQq4/Jggl2gAH55kZnI=">AAACinicZVFLa9tAEF6rj6Ru2jjtsZelxtBDMJIptKWX0PbQSyCFOglExqxWY3vJPsTObLAR+iW9tj+q/6YrR5TEGdjlYx7fzHxTVFohpenfXvLo8ZOne/vP+s8PXrw8HBy9OkcXvISpdNr5y0IgaGVhSoo0XFYehCk0XBTXX9v4xQ14VM7+pE0FMyOWVi2UFBRd88Hheq54rizP6/Q4y5v5YJiO063xhyDrwJB1djY/6sm8dDIYsCS1QLzK0opmtfCkpIamnweESshrsYSrCK0wgLN6O3nDR9FT8oXz8VniW+/diloYxI0pYqYRtMLdWOv8HxvdDdK6ZcSd/rT4OKuVrQKBlbftF0FzcrwVh5fKgyS9iUBIr+IGXK6EF5KihPeYWr3LYx90/KUz7fZogynAQxmX10sXy1dmAg3nIx5liQIh7ybmHQtXFknoyNHPEUi6YGOj+lSsTwV5tY7nw6aepE0/niXbPcJDcD4ZZ+k4+/F+ePKlO9A+e8PesncsYx/YCfvOztiUSRbYL/ab/UkOkknyKfl8m5r0uprX7J4l3/4BJGPHpw==</latexit><latexit sha1_base64="mFCz3MOySQq4/Jggl2gAH55kZnI=">AAACinicZVFLa9tAEF6rj6Ru2jjtsZelxtBDMJIptKWX0PbQSyCFOglExqxWY3vJPsTObLAR+iW9tj+q/6YrR5TEGdjlYx7fzHxTVFohpenfXvLo8ZOne/vP+s8PXrw8HBy9OkcXvISpdNr5y0IgaGVhSoo0XFYehCk0XBTXX9v4xQ14VM7+pE0FMyOWVi2UFBRd88Hheq54rizP6/Q4y5v5YJiO063xhyDrwJB1djY/6sm8dDIYsCS1QLzK0opmtfCkpIamnweESshrsYSrCK0wgLN6O3nDR9FT8oXz8VniW+/diloYxI0pYqYRtMLdWOv8HxvdDdK6ZcSd/rT4OKuVrQKBlbftF0FzcrwVh5fKgyS9iUBIr+IGXK6EF5KihPeYWr3LYx90/KUz7fZogynAQxmX10sXy1dmAg3nIx5liQIh7ybmHQtXFknoyNHPEUi6YGOj+lSsTwV5tY7nw6aepE0/niXbPcJDcD4ZZ+k4+/F+ePKlO9A+e8PesncsYx/YCfvOztiUSRbYL/ab/UkOkknyKfl8m5r0uprX7J4l3/4BJGPHpw==</latexit><latexit sha1_base64="mFCz3MOySQq4/Jggl2gAH55kZnI=">AAACinicZVFLa9tAEF6rj6Ru2jjtsZelxtBDMJIptKWX0PbQSyCFOglExqxWY3vJPsTObLAR+iW9tj+q/6YrR5TEGdjlYx7fzHxTVFohpenfXvLo8ZOne/vP+s8PXrw8HBy9OkcXvISpdNr5y0IgaGVhSoo0XFYehCk0XBTXX9v4xQ14VM7+pE0FMyOWVi2UFBRd88Hheq54rizP6/Q4y5v5YJiO063xhyDrwJB1djY/6sm8dDIYsCS1QLzK0opmtfCkpIamnweESshrsYSrCK0wgLN6O3nDR9FT8oXz8VniW+/diloYxI0pYqYRtMLdWOv8HxvdDdK6ZcSd/rT4OKuVrQKBlbftF0FzcrwVh5fKgyS9iUBIr+IGXK6EF5KihPeYWr3LYx90/KUz7fZogynAQxmX10sXy1dmAg3nIx5liQIh7ybmHQtXFknoyNHPEUi6YGOj+lSsTwV5tY7nw6aepE0/niXbPcJDcD4ZZ+k4+/F+ePKlO9A+e8PesncsYx/YCfvOztiUSRbYL/ab/UkOkknyKfl8m5r0uprX7J4l3/4BJGPHpw==</latexit><latexit sha1_base64="mFCz3MOySQq4/Jggl2gAH55kZnI=">AAACinicZVFLa9tAEF6rj6Ru2jjtsZelxtBDMJIptKWX0PbQSyCFOglExqxWY3vJPsTObLAR+iW9tj+q/6YrR5TEGdjlYx7fzHxTVFohpenfXvLo8ZOne/vP+s8PXrw8HBy9OkcXvISpdNr5y0IgaGVhSoo0XFYehCk0XBTXX9v4xQ14VM7+pE0FMyOWVi2UFBRd88Hheq54rizP6/Q4y5v5YJiO063xhyDrwJB1djY/6sm8dDIYsCS1QLzK0opmtfCkpIamnweESshrsYSrCK0wgLN6O3nDR9FT8oXz8VniW+/diloYxI0pYqYRtMLdWOv8HxvdDdK6ZcSd/rT4OKuVrQKBlbftF0FzcrwVh5fKgyS9iUBIr+IGXK6EF5KihPeYWr3LYx90/KUz7fZogynAQxmX10sXy1dmAg3nIx5liQIh7ybmHQtXFknoyNHPEUi6YGOj+lSsTwV5tY7nw6aepE0/niXbPcJDcD4ZZ+k4+/F+ePKlO9A+e8PesncsYx/YCfvOztiUSRbYL/ab/UkOkknyKfl8m5r0uprX7J4l3/4BJGPHpw==</latexit>

�ijk(xi, xj , xk) =

⇢1 if xn = 1 8n 2 {i, j, k}0 otherwise

<latexit sha1_base64="ZQB/iHSvQLu2mVc17B/KINeJ32g=">AAADCHicZVLNatwwEJbdv3T7t2lPpZchy5YUlsVeCu0lENpLL4EUukkgWowsj9falWUjya0X4xco9F16K732LfocfYHKGxPyMzDSx/xq5lNcSmFsEPz1/Dt3791/sPNw8Ojxk6fPhrvPT0xRaY5zXshCn8XMoBQK51ZYiWelRpbHEk/j9cfOf/oVtRGF+mI3JS5ytlQiFZxZZ4qGP2iZiagRq3W7X0diUkcrp+s3cABUYmppAzTGpVAN05pt2obzFkJ4DTSPi7oRaQsU6kgdhO6maaGZlKCACqeNmKwma+oiKASXKYXNUH8TBluKKunLUi2WmZ1Gw1EwDbYCt0HYgxHp5Tja9ThNCl7lqCyXzJjzMCjtwlW1gktsB7QyWDK+Zks8d1CxHM2i2e6thbGzJOAe7VRZ2FqvZjQsN2aTxy4yZzYzN32d8dI3vuq0dVfR3Ohv0/eLRqiysqj4Rfu0kmAL6KiBRGjkVm4cYFwLNwHwjGnGrSPwWqWO7WSiK+lOXuTd9EZVeYwaEze8XBYuPctn2AKMwa3FLchA/2Loq4BQxjq6MBlQg5YXlXKNmiNWHzGrRe0+j2mbWdAOHC3hTRJug5PZNAym4ee3o8MPPUE75BXZI/skJO/IIflEjsmccPLPe+nteSP/u//T/+X/vgj1vT7nBbkm/p//fJv3Vw==</latexit><latexit sha1_base64="ZQB/iHSvQLu2mVc17B/KINeJ32g=">AAADCHicZVLNatwwEJbdv3T7t2lPpZchy5YUlsVeCu0lENpLL4EUukkgWowsj9falWUjya0X4xco9F16K732LfocfYHKGxPyMzDSx/xq5lNcSmFsEPz1/Dt3791/sPNw8Ojxk6fPhrvPT0xRaY5zXshCn8XMoBQK51ZYiWelRpbHEk/j9cfOf/oVtRGF+mI3JS5ytlQiFZxZZ4qGP2iZiagRq3W7X0diUkcrp+s3cABUYmppAzTGpVAN05pt2obzFkJ4DTSPi7oRaQsU6kgdhO6maaGZlKCACqeNmKwma+oiKASXKYXNUH8TBluKKunLUi2WmZ1Gw1EwDbYCt0HYgxHp5Tja9ThNCl7lqCyXzJjzMCjtwlW1gktsB7QyWDK+Zks8d1CxHM2i2e6thbGzJOAe7VRZ2FqvZjQsN2aTxy4yZzYzN32d8dI3vuq0dVfR3Ohv0/eLRqiysqj4Rfu0kmAL6KiBRGjkVm4cYFwLNwHwjGnGrSPwWqWO7WSiK+lOXuTd9EZVeYwaEze8XBYuPctn2AKMwa3FLchA/2Loq4BQxjq6MBlQg5YXlXKNmiNWHzGrRe0+j2mbWdAOHC3hTRJug5PZNAym4ee3o8MPPUE75BXZI/skJO/IIflEjsmccPLPe+nteSP/u//T/+X/vgj1vT7nBbkm/p//fJv3Vw==</latexit><latexit sha1_base64="ZQB/iHSvQLu2mVc17B/KINeJ32g=">AAADCHicZVLNatwwEJbdv3T7t2lPpZchy5YUlsVeCu0lENpLL4EUukkgWowsj9falWUjya0X4xco9F16K732LfocfYHKGxPyMzDSx/xq5lNcSmFsEPz1/Dt3791/sPNw8Ojxk6fPhrvPT0xRaY5zXshCn8XMoBQK51ZYiWelRpbHEk/j9cfOf/oVtRGF+mI3JS5ytlQiFZxZZ4qGP2iZiagRq3W7X0diUkcrp+s3cABUYmppAzTGpVAN05pt2obzFkJ4DTSPi7oRaQsU6kgdhO6maaGZlKCACqeNmKwma+oiKASXKYXNUH8TBluKKunLUi2WmZ1Gw1EwDbYCt0HYgxHp5Tja9ThNCl7lqCyXzJjzMCjtwlW1gktsB7QyWDK+Zks8d1CxHM2i2e6thbGzJOAe7VRZ2FqvZjQsN2aTxy4yZzYzN32d8dI3vuq0dVfR3Ohv0/eLRqiysqj4Rfu0kmAL6KiBRGjkVm4cYFwLNwHwjGnGrSPwWqWO7WSiK+lOXuTd9EZVeYwaEze8XBYuPctn2AKMwa3FLchA/2Loq4BQxjq6MBlQg5YXlXKNmiNWHzGrRe0+j2mbWdAOHC3hTRJug5PZNAym4ee3o8MPPUE75BXZI/skJO/IIflEjsmccPLPe+nteSP/u//T/+X/vgj1vT7nBbkm/p//fJv3Vw==</latexit><latexit sha1_base64="ZQB/iHSvQLu2mVc17B/KINeJ32g=">AAADCHicZVLNatwwEJbdv3T7t2lPpZchy5YUlsVeCu0lENpLL4EUukkgWowsj9falWUjya0X4xco9F16K732LfocfYHKGxPyMzDSx/xq5lNcSmFsEPz1/Dt3791/sPNw8Ojxk6fPhrvPT0xRaY5zXshCn8XMoBQK51ZYiWelRpbHEk/j9cfOf/oVtRGF+mI3JS5ytlQiFZxZZ4qGP2iZiagRq3W7X0diUkcrp+s3cABUYmppAzTGpVAN05pt2obzFkJ4DTSPi7oRaQsU6kgdhO6maaGZlKCACqeNmKwma+oiKASXKYXNUH8TBluKKunLUi2WmZ1Gw1EwDbYCt0HYgxHp5Tja9ThNCl7lqCyXzJjzMCjtwlW1gktsB7QyWDK+Zks8d1CxHM2i2e6thbGzJOAe7VRZ2FqvZjQsN2aTxy4yZzYzN32d8dI3vuq0dVfR3Ohv0/eLRqiysqj4Rfu0kmAL6KiBRGjkVm4cYFwLNwHwjGnGrSPwWqWO7WSiK+lOXuTd9EZVeYwaEze8XBYuPctn2AKMwa3FLchA/2Loq4BQxjq6MBlQg5YXlXKNmiNWHzGrRe0+j2mbWdAOHC3hTRJug5PZNAym4ee3o8MPPUE75BXZI/skJO/IIflEjsmccPLPe+nteSP/u//T/+X/vgj1vT7nBbkm/p//fJv3Vw==</latexit>

�<latexit sha1_base64="Xo1sIJ0SPxMC8hnShjOX1qrpSWY=">AAACfXicZVFLa9tAEF6rr9R9Je2xl6XG0EMwkgmkx9BeegmkUCeB2ITRamwt2YfYGRUbob/Qa/vX+mvalSNK4gzs8jGPb2a+ySujidP0zyB59PjJ02d7z4cvXr56/Wb/4O05+ToonClvfLjMgdBohzPWbPCyCgg2N3iR33zp4hc/MJD27jtvKlxYWDm91Aq4c82rUl/vj9JJujX5EGQ9GInezq4PBmpeeFVbdKwMEF1lacWLBgJrZbAdzmvCCtQNrPAqQgcWadFsh23lOHoKufQhPsdy671b0YAl2tg8ZlrgknZjnfN/bHw3yOuOkXb68/LTotGuqhmdum2/rI1kLzs9ZKEDKjabCEAFHTeQqoQAiqNq95g6iYvDUJv4K2+77cnVNseARVzerHwsL+0UWynHMsoSBSLZTyx7FqkdMZjIMZwTsvK1i42aU1ifAge9jhejtpmm7TCeJds9wkNwPp1k6ST7djQ6+dwfaE+8Fx/ER5GJY3EivoozMRNKlOKn+CV+D/4m4+QwmdymJoO+5p24Z8nxPxPwxLQ=</latexit><latexit sha1_base64="Xo1sIJ0SPxMC8hnShjOX1qrpSWY=">AAACfXicZVFLa9tAEF6rr9R9Je2xl6XG0EMwkgmkx9BeegmkUCeB2ITRamwt2YfYGRUbob/Qa/vX+mvalSNK4gzs8jGPb2a+ySujidP0zyB59PjJ02d7z4cvXr56/Wb/4O05+ToonClvfLjMgdBohzPWbPCyCgg2N3iR33zp4hc/MJD27jtvKlxYWDm91Aq4c82rUl/vj9JJujX5EGQ9GInezq4PBmpeeFVbdKwMEF1lacWLBgJrZbAdzmvCCtQNrPAqQgcWadFsh23lOHoKufQhPsdy671b0YAl2tg8ZlrgknZjnfN/bHw3yOuOkXb68/LTotGuqhmdum2/rI1kLzs9ZKEDKjabCEAFHTeQqoQAiqNq95g6iYvDUJv4K2+77cnVNseARVzerHwsL+0UWynHMsoSBSLZTyx7FqkdMZjIMZwTsvK1i42aU1ifAge9jhejtpmm7TCeJds9wkNwPp1k6ST7djQ6+dwfaE+8Fx/ER5GJY3EivoozMRNKlOKn+CV+D/4m4+QwmdymJoO+5p24Z8nxPxPwxLQ=</latexit><latexit sha1_base64="Xo1sIJ0SPxMC8hnShjOX1qrpSWY=">AAACfXicZVFLa9tAEF6rr9R9Je2xl6XG0EMwkgmkx9BeegmkUCeB2ITRamwt2YfYGRUbob/Qa/vX+mvalSNK4gzs8jGPb2a+ySujidP0zyB59PjJ02d7z4cvXr56/Wb/4O05+ToonClvfLjMgdBohzPWbPCyCgg2N3iR33zp4hc/MJD27jtvKlxYWDm91Aq4c82rUl/vj9JJujX5EGQ9GInezq4PBmpeeFVbdKwMEF1lacWLBgJrZbAdzmvCCtQNrPAqQgcWadFsh23lOHoKufQhPsdy671b0YAl2tg8ZlrgknZjnfN/bHw3yOuOkXb68/LTotGuqhmdum2/rI1kLzs9ZKEDKjabCEAFHTeQqoQAiqNq95g6iYvDUJv4K2+77cnVNseARVzerHwsL+0UWynHMsoSBSLZTyx7FqkdMZjIMZwTsvK1i42aU1ifAge9jhejtpmm7TCeJds9wkNwPp1k6ST7djQ6+dwfaE+8Fx/ER5GJY3EivoozMRNKlOKn+CV+D/4m4+QwmdymJoO+5p24Z8nxPxPwxLQ=</latexit><latexit sha1_base64="Xo1sIJ0SPxMC8hnShjOX1qrpSWY=">AAACfXicZVFLa9tAEF6rr9R9Je2xl6XG0EMwkgmkx9BeegmkUCeB2ITRamwt2YfYGRUbob/Qa/vX+mvalSNK4gzs8jGPb2a+ySujidP0zyB59PjJ02d7z4cvXr56/Wb/4O05+ToonClvfLjMgdBohzPWbPCyCgg2N3iR33zp4hc/MJD27jtvKlxYWDm91Aq4c82rUl/vj9JJujX5EGQ9GInezq4PBmpeeFVbdKwMEF1lacWLBgJrZbAdzmvCCtQNrPAqQgcWadFsh23lOHoKufQhPsdy671b0YAl2tg8ZlrgknZjnfN/bHw3yOuOkXb68/LTotGuqhmdum2/rI1kLzs9ZKEDKjabCEAFHTeQqoQAiqNq95g6iYvDUJv4K2+77cnVNseARVzerHwsL+0UWynHMsoSBSLZTyx7FqkdMZjIMZwTsvK1i42aU1ifAge9jhejtpmm7TCeJds9wkNwPp1k6ST7djQ6+dwfaE+8Fx/ER5GJY3EivoozMRNKlOKn+CV+D/4m4+QwmdymJoO+5p24Z8nxPxPwxLQ=</latexit>



A Simple Example


• We can define features • and determine weights

!11







�ijk(xi, xj , xk) =

⇢1 if xn = 1 8n 2 {i, j, k}0 otherwise

<latexit sha1_base64="ZQB/iHSvQLu2mVc17B/KINeJ32g=">AAADCHicZVLNatwwEJbdv3T7t2lPpZchy5YUlsVeCu0lENpLL4EUukkgWowsj9falWUjya0X4xco9F16K732LfocfYHKGxPyMzDSx/xq5lNcSmFsEPz1/Dt3791/sPNw8Ojxk6fPhrvPT0xRaY5zXshCn8XMoBQK51ZYiWelRpbHEk/j9cfOf/oVtRGF+mI3JS5ytlQiFZxZZ4qGP2iZiagRq3W7X0diUkcrp+s3cABUYmppAzTGpVAN05pt2obzFkJ4DTSPi7oRaQsU6kgdhO6maaGZlKCACqeNmKwma+oiKASXKYXNUH8TBluKKunLUi2WmZ1Gw1EwDbYCt0HYgxHp5Tja9ThNCl7lqCyXzJjzMCjtwlW1gktsB7QyWDK+Zks8d1CxHM2i2e6thbGzJOAe7VRZ2FqvZjQsN2aTxy4yZzYzN32d8dI3vuq0dVfR3Ohv0/eLRqiysqj4Rfu0kmAL6KiBRGjkVm4cYFwLNwHwjGnGrSPwWqWO7WSiK+lOXuTd9EZVeYwaEze8XBYuPctn2AKMwa3FLchA/2Loq4BQxjq6MBlQg5YXlXKNmiNWHzGrRe0+j2mbWdAOHC3hTRJug5PZNAym4ee3o8MPPUE75BXZI/skJO/IIflEjsmccPLPe+nteSP/u//T/+X/vgj1vT7nBbkm/p//fJv3Vw==</latexit><latexit sha1_base64="ZQB/iHSvQLu2mVc17B/KINeJ32g=">AAADCHicZVLNatwwEJbdv3T7t2lPpZchy5YUlsVeCu0lENpLL4EUukkgWowsj9falWUjya0X4xco9F16K732LfocfYHKGxPyMzDSx/xq5lNcSmFsEPz1/Dt3791/sPNw8Ojxk6fPhrvPT0xRaY5zXshCn8XMoBQK51ZYiWelRpbHEk/j9cfOf/oVtRGF+mI3JS5ytlQiFZxZZ4qGP2iZiagRq3W7X0diUkcrp+s3cABUYmppAzTGpVAN05pt2obzFkJ4DTSPi7oRaQsU6kgdhO6maaGZlKCACqeNmKwma+oiKASXKYXNUH8TBluKKunLUi2WmZ1Gw1EwDbYCt0HYgxHp5Tja9ThNCl7lqCyXzJjzMCjtwlW1gktsB7QyWDK+Zks8d1CxHM2i2e6thbGzJOAe7VRZ2FqvZjQsN2aTxy4yZzYzN32d8dI3vuq0dVfR3Ohv0/eLRqiysqj4Rfu0kmAL6KiBRGjkVm4cYFwLNwHwjGnGrSPwWqWO7WSiK+lOXuTd9EZVeYwaEze8XBYuPctn2AKMwa3FLchA/2Loq4BQxjq6MBlQg5YXlXKNmiNWHzGrRe0+j2mbWdAOHC3hTRJug5PZNAym4ee3o8MPPUE75BXZI/skJO/IIflEjsmccPLPe+nteSP/u//T/+X/vgj1vT7nBbkm/p//fJv3Vw==</latexit><latexit sha1_base64="ZQB/iHSvQLu2mVc17B/KINeJ32g=">AAADCHicZVLNatwwEJbdv3T7t2lPpZchy5YUlsVeCu0lENpLL4EUukkgWowsj9falWUjya0X4xco9F16K732LfocfYHKGxPyMzDSx/xq5lNcSmFsEPz1/Dt3791/sPNw8Ojxk6fPhrvPT0xRaY5zXshCn8XMoBQK51ZYiWelRpbHEk/j9cfOf/oVtRGF+mI3JS5ytlQiFZxZZ4qGP2iZiagRq3W7X0diUkcrp+s3cABUYmppAzTGpVAN05pt2obzFkJ4DTSPi7oRaQsU6kgdhO6maaGZlKCACqeNmKwma+oiKASXKYXNUH8TBluKKunLUi2WmZ1Gw1EwDbYCt0HYgxHp5Tja9ThNCl7lqCyXzJjzMCjtwlW1gktsB7QyWDK+Zks8d1CxHM2i2e6thbGzJOAe7VRZ2FqvZjQsN2aTxy4yZzYzN32d8dI3vuq0dVfR3Ohv0/eLRqiysqj4Rfu0kmAL6KiBRGjkVm4cYFwLNwHwjGnGrSPwWqWO7WSiK+lOXuTd9EZVeYwaEze8XBYuPctn2AKMwa3FLchA/2Loq4BQxjq6MBlQg5YXlXKNmiNWHzGrRe0+j2mbWdAOHC3hTRJug5PZNAym4ee3o8MPPUE75BXZI/skJO/IIflEjsmccPLPe+nteSP/u//T/+X/vgj1vT7nBbkm/p//fJv3Vw==</latexit><latexit sha1_base64="ZQB/iHSvQLu2mVc17B/KINeJ32g=">AAADCHicZVLNatwwEJbdv3T7t2lPpZchy5YUlsVeCu0lENpLL4EUukkgWowsj9falWUjya0X4xco9F16K732LfocfYHKGxPyMzDSx/xq5lNcSmFsEPz1/Dt3791/sPNw8Ojxk6fPhrvPT0xRaY5zXshCn8XMoBQK51ZYiWelRpbHEk/j9cfOf/oVtRGF+mI3JS5ytlQiFZxZZ4qGP2iZiagRq3W7X0diUkcrp+s3cABUYmppAzTGpVAN05pt2obzFkJ4DTSPi7oRaQsU6kgdhO6maaGZlKCACqeNmKwma+oiKASXKYXNUH8TBluKKunLUi2WmZ1Gw1EwDbYCt0HYgxHp5Tja9ThNCl7lqCyXzJjzMCjtwlW1gktsB7QyWDK+Zks8d1CxHM2i2e6thbGzJOAe7VRZ2FqvZjQsN2aTxy4yZzYzN32d8dI3vuq0dVfR3Ohv0/eLRqiysqj4Rfu0kmAL6KiBRGjkVm4cYFwLNwHwjGnGrSPwWqWO7WSiK+lOXuTd9EZVeYwaEze8XBYuPctn2AKMwa3FLchA/2Loq4BQxjq6MBlQg5YXlXKNmiNWHzGrRe0+j2mbWdAOHC3hTRJug5PZNAym4ee3o8MPPUE75BXZI/skJO/IIflEjsmccPLPe+nteSP/u//T/+X/vgj1vT7nBbkm/p//fJv3Vw==</latexit>


✓<latexit sha1_base64="fIJ8viYXNeSTUqBRhCJv53FE61M=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNocwvtpZdACrUTiE0Yrcb2NvsQO6NgI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73kwcNHj5/sPe0/e/7i5av9g9dT8lVQOFHe+HCeA6HRDies2eB5GRBsbvAsv/raxs+uMZD27gdvSpxbWDq90Ao4uqYzXiHD5f4gHaVbk/dB1oGB6Oz08qCnZoVXlUXHygDRRZaWPK8hsFYGm/6sIixBXcESLyJ0YJHm9XbcRg6jp5ALH+JzLLfe2xU1WKKNzWOmBV7Rbqx1/o8Nbwd53TLSTn9efJ7X2pUVo1M37ReVkexlq4gsdEDFZhMBqKDjBlKtIIDiqNsdplbk4mOoTPyVt+325CqbY8AiLm+WPpav7BgbKYcyyhIFItlNLDsWqR0xmMjRnxGy8pWLjeoTWJ8AB72ON6OmHqdNP54l2z3CfTAdj7J0lH0/HBx/6Q60J96Kd+KDyMQncSy+iVMxEUr8FL/Eb/En6SXvk1GS3qQmva7mjbhjydE/+CXEng==</latexit><latexit sha1_base64="fIJ8viYXNeSTUqBRhCJv53FE61M=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNocwvtpZdACrUTiE0Yrcb2NvsQO6NgI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73kwcNHj5/sPe0/e/7i5av9g9dT8lVQOFHe+HCeA6HRDies2eB5GRBsbvAsv/raxs+uMZD27gdvSpxbWDq90Ao4uqYzXiHD5f4gHaVbk/dB1oGB6Oz08qCnZoVXlUXHygDRRZaWPK8hsFYGm/6sIixBXcESLyJ0YJHm9XbcRg6jp5ALH+JzLLfe2xU1WKKNzWOmBV7Rbqx1/o8Nbwd53TLSTn9efJ7X2pUVo1M37ReVkexlq4gsdEDFZhMBqKDjBlKtIIDiqNsdplbk4mOoTPyVt+325CqbY8AiLm+WPpav7BgbKYcyyhIFItlNLDsWqR0xmMjRnxGy8pWLjeoTWJ8AB72ON6OmHqdNP54l2z3CfTAdj7J0lH0/HBx/6Q60J96Kd+KDyMQncSy+iVMxEUr8FL/Eb/En6SXvk1GS3qQmva7mjbhjydE/+CXEng==</latexit><latexit sha1_base64="fIJ8viYXNeSTUqBRhCJv53FE61M=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNocwvtpZdACrUTiE0Yrcb2NvsQO6NgI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73kwcNHj5/sPe0/e/7i5av9g9dT8lVQOFHe+HCeA6HRDies2eB5GRBsbvAsv/raxs+uMZD27gdvSpxbWDq90Ao4uqYzXiHD5f4gHaVbk/dB1oGB6Oz08qCnZoVXlUXHygDRRZaWPK8hsFYGm/6sIixBXcESLyJ0YJHm9XbcRg6jp5ALH+JzLLfe2xU1WKKNzWOmBV7Rbqx1/o8Nbwd53TLSTn9efJ7X2pUVo1M37ReVkexlq4gsdEDFZhMBqKDjBlKtIIDiqNsdplbk4mOoTPyVt+325CqbY8AiLm+WPpav7BgbKYcyyhIFItlNLDsWqR0xmMjRnxGy8pWLjeoTWJ8AB72ON6OmHqdNP54l2z3CfTAdj7J0lH0/HBx/6Q60J96Kd+KDyMQncSy+iVMxEUr8FL/Eb/En6SXvk1GS3qQmva7mjbhjydE/+CXEng==</latexit><latexit sha1_base64="fIJ8viYXNeSTUqBRhCJv53FE61M=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNocwvtpZdACrUTiE0Yrcb2NvsQO6NgI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73kwcNHj5/sPe0/e/7i5av9g9dT8lVQOFHe+HCeA6HRDies2eB5GRBsbvAsv/raxs+uMZD27gdvSpxbWDq90Ao4uqYzXiHD5f4gHaVbk/dB1oGB6Oz08qCnZoVXlUXHygDRRZaWPK8hsFYGm/6sIixBXcESLyJ0YJHm9XbcRg6jp5ALH+JzLLfe2xU1WKKNzWOmBV7Rbqx1/o8Nbwd53TLSTn9efJ7X2pUVo1M37ReVkexlq4gsdEDFZhMBqKDjBlKtIIDiqNsdplbk4mOoTPyVt+325CqbY8AiLm+WPpav7BgbKYcyyhIFItlNLDsWqR0xmMjRnxGy8pWLjeoTWJ8AB72ON6OmHqdNP54l2z3CfTAdj7J0lH0/HBx/6Q60J96Kd+KDyMQncSy+iVMxEUr8FL/Eb/En6SXvk1GS3qQmva7mjbhjydE/+CXEng==</latexit>

✓ = (0 0 0 0 0 0 0 1)T<latexit sha1_base64="R/BLV7f4tWsgfXNIlnl6pvvHt4I=">AAACsHicdVHdahNBFJ5sq9b4l+qlN0NDoIKE3aDojVD0xptChaYNdGM4O3uSTDs/y8xZSVj2Sfo03uoT+DbOpou0KR6YmY/zO9/5skJJT3H8pxPt7D54+GjvcffJ02fPX/T2X555WzqBY2GVdZMMPCppcEySFE4Kh6AzhefZ1Zcmfv4DnZfWnNK6wKmGhZFzKYCCa9Z7n2ZW5X6tw5PSEgn4J34Y85T/7yRvvp/Oev14GG+M3wdJC/qstZPZfkekuRWlRkNCgfcXSVzQtAJHUiisu2npsQBxBQu8CNCARj+tNvxqPgienM+tC8cQ33hvV1SgfUMhZGqgpd+ONc5/scHtIK2ajn5rPs0/TitpipLQiJvx81JxsrxZIc+lQ0FqHQAIJwMDLpbgQFBY9J1OjSr5W1eqcAurG/belDpDh3kgrxY2lC/1CGvOBzysJSzI8/bHvO3CpfEEKvToph5J2NKEQdUxrI6BnFwFkX1djeK6G2RJtkW4D85GwyQeJt/e9Y8+twLtsdfsgB2yhH1gR+wrO2FjJtg1+8l+sd/RKJpEswhuUqNOW/OK3bHo8i/OhNJ9</latexit><latexit sha1_base64="R/BLV7f4tWsgfXNIlnl6pvvHt4I=">AAACsHicdVHdahNBFJ5sq9b4l+qlN0NDoIKE3aDojVD0xptChaYNdGM4O3uSTDs/y8xZSVj2Sfo03uoT+DbOpou0KR6YmY/zO9/5skJJT3H8pxPt7D54+GjvcffJ02fPX/T2X555WzqBY2GVdZMMPCppcEySFE4Kh6AzhefZ1Zcmfv4DnZfWnNK6wKmGhZFzKYCCa9Z7n2ZW5X6tw5PSEgn4J34Y85T/7yRvvp/Oev14GG+M3wdJC/qstZPZfkekuRWlRkNCgfcXSVzQtAJHUiisu2npsQBxBQu8CNCARj+tNvxqPgienM+tC8cQ33hvV1SgfUMhZGqgpd+ONc5/scHtIK2ajn5rPs0/TitpipLQiJvx81JxsrxZIc+lQ0FqHQAIJwMDLpbgQFBY9J1OjSr5W1eqcAurG/belDpDh3kgrxY2lC/1CGvOBzysJSzI8/bHvO3CpfEEKvToph5J2NKEQdUxrI6BnFwFkX1djeK6G2RJtkW4D85GwyQeJt/e9Y8+twLtsdfsgB2yhH1gR+wrO2FjJtg1+8l+sd/RKJpEswhuUqNOW/OK3bHo8i/OhNJ9</latexit><latexit sha1_base64="R/BLV7f4tWsgfXNIlnl6pvvHt4I=">AAACsHicdVHdahNBFJ5sq9b4l+qlN0NDoIKE3aDojVD0xptChaYNdGM4O3uSTDs/y8xZSVj2Sfo03uoT+DbOpou0KR6YmY/zO9/5skJJT3H8pxPt7D54+GjvcffJ02fPX/T2X555WzqBY2GVdZMMPCppcEySFE4Kh6AzhefZ1Zcmfv4DnZfWnNK6wKmGhZFzKYCCa9Z7n2ZW5X6tw5PSEgn4J34Y85T/7yRvvp/Oev14GG+M3wdJC/qstZPZfkekuRWlRkNCgfcXSVzQtAJHUiisu2npsQBxBQu8CNCARj+tNvxqPgienM+tC8cQ33hvV1SgfUMhZGqgpd+ONc5/scHtIK2ajn5rPs0/TitpipLQiJvx81JxsrxZIc+lQ0FqHQAIJwMDLpbgQFBY9J1OjSr5W1eqcAurG/belDpDh3kgrxY2lC/1CGvOBzysJSzI8/bHvO3CpfEEKvToph5J2NKEQdUxrI6BnFwFkX1djeK6G2RJtkW4D85GwyQeJt/e9Y8+twLtsdfsgB2yhH1gR+wrO2FjJtg1+8l+sd/RKJpEswhuUqNOW/OK3bHo8i/OhNJ9</latexit><latexit sha1_base64="R/BLV7f4tWsgfXNIlnl6pvvHt4I=">AAACsHicdVHdahNBFJ5sq9b4l+qlN0NDoIKE3aDojVD0xptChaYNdGM4O3uSTDs/y8xZSVj2Sfo03uoT+DbOpou0KR6YmY/zO9/5skJJT3H8pxPt7D54+GjvcffJ02fPX/T2X555WzqBY2GVdZMMPCppcEySFE4Kh6AzhefZ1Zcmfv4DnZfWnNK6wKmGhZFzKYCCa9Z7n2ZW5X6tw5PSEgn4J34Y85T/7yRvvp/Oev14GG+M3wdJC/qstZPZfkekuRWlRkNCgfcXSVzQtAJHUiisu2npsQBxBQu8CNCARj+tNvxqPgienM+tC8cQ33hvV1SgfUMhZGqgpd+ONc5/scHtIK2ajn5rPs0/TitpipLQiJvx81JxsrxZIc+lQ0FqHQAIJwMDLpbgQFBY9J1OjSr5W1eqcAurG/belDpDh3kgrxY2lC/1CGvOBzysJSzI8/bHvO3CpfEEKvToph5J2NKEQdUxrI6BnFwFkX1djeK6G2RJtkW4D85GwyQeJt/e9Y8+twLtsdfsgB2yhH1gR+wrO2FjJtg1+8l+sd/RKJpEswhuUqNOW/OK3bHo8i/OhNJ9</latexit>



A Simple Example


• We can define features • and determine weights • Then, we can compute the

(log of the ) joint probability for each realisation of the

!12








✓<latexit sha1_base64="fIJ8viYXNeSTUqBRhCJv53FE61M=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNocwvtpZdACrUTiE0Yrcb2NvsQO6NgI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73kwcNHj5/sPe0/e/7i5av9g9dT8lVQOFHe+HCeA6HRDies2eB5GRBsbvAsv/raxs+uMZD27gdvSpxbWDq90Ao4uqYzXiHD5f4gHaVbk/dB1oGB6Oz08qCnZoVXlUXHygDRRZaWPK8hsFYGm/6sIixBXcESLyJ0YJHm9XbcRg6jp5ALH+JzLLfe2xU1WKKNzWOmBV7Rbqx1/o8Nbwd53TLSTn9efJ7X2pUVo1M37ReVkexlq4gsdEDFZhMBqKDjBlKtIIDiqNsdplbk4mOoTPyVt+325CqbY8AiLm+WPpav7BgbKYcyyhIFItlNLDsWqR0xmMjRnxGy8pWLjeoTWJ8AB72ON6OmHqdNP54l2z3CfTAdj7J0lH0/HBx/6Q60J96Kd+KDyMQncSy+iVMxEUr8FL/Eb/En6SXvk1GS3qQmva7mjbhjydE/+CXEng==</latexit><latexit sha1_base64="fIJ8viYXNeSTUqBRhCJv53FE61M=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNocwvtpZdACrUTiE0Yrcb2NvsQO6NgI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73kwcNHj5/sPe0/e/7i5av9g9dT8lVQOFHe+HCeA6HRDies2eB5GRBsbvAsv/raxs+uMZD27gdvSpxbWDq90Ao4uqYzXiHD5f4gHaVbk/dB1oGB6Oz08qCnZoVXlUXHygDRRZaWPK8hsFYGm/6sIixBXcESLyJ0YJHm9XbcRg6jp5ALH+JzLLfe2xU1WKKNzWOmBV7Rbqx1/o8Nbwd53TLSTn9efJ7X2pUVo1M37ReVkexlq4gsdEDFZhMBqKDjBlKtIIDiqNsdplbk4mOoTPyVt+325CqbY8AiLm+WPpav7BgbKYcyyhIFItlNLDsWqR0xmMjRnxGy8pWLjeoTWJ8AB72ON6OmHqdNP54l2z3CfTAdj7J0lH0/HBx/6Q60J96Kd+KDyMQncSy+iVMxEUr8FL/Eb/En6SXvk1GS3qQmva7mjbhjydE/+CXEng==</latexit><latexit sha1_base64="fIJ8viYXNeSTUqBRhCJv53FE61M=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNocwvtpZdACrUTiE0Yrcb2NvsQO6NgI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73kwcNHj5/sPe0/e/7i5av9g9dT8lVQOFHe+HCeA6HRDies2eB5GRBsbvAsv/raxs+uMZD27gdvSpxbWDq90Ao4uqYzXiHD5f4gHaVbk/dB1oGB6Oz08qCnZoVXlUXHygDRRZaWPK8hsFYGm/6sIixBXcESLyJ0YJHm9XbcRg6jp5ALH+JzLLfe2xU1WKKNzWOmBV7Rbqx1/o8Nbwd53TLSTn9efJ7X2pUVo1M37ReVkexlq4gsdEDFZhMBqKDjBlKtIIDiqNsdplbk4mOoTPyVt+325CqbY8AiLm+WPpav7BgbKYcyyhIFItlNLDsWqR0xmMjRnxGy8pWLjeoTWJ8AB72ON6OmHqdNP54l2z3CfTAdj7J0lH0/HBx/6Q60J96Kd+KDyMQncSy+iVMxEUr8FL/Eb/En6SXvk1GS3qQmva7mjbhjydE/+CXEng==</latexit><latexit sha1_base64="fIJ8viYXNeSTUqBRhCJv53FE61M=">AAACf3icZVFLa9tAEF6rr9R9Je2xl6XGtIdiJBNocwvtpZdACrUTiE0Yrcb2NvsQO6NgI/Qfem3/Wf9NV44oiTOwy8c8vpn5Ji+NJk7Tv73kwcNHj5/sPe0/e/7i5av9g9dT8lVQOFHe+HCeA6HRDies2eB5GRBsbvAsv/raxs+uMZD27gdvSpxbWDq90Ao4uqYzXiHD5f4gHaVbk/dB1oGB6Oz08qCnZoVXlUXHygDRRZaWPK8hsFYGm/6sIixBXcESLyJ0YJHm9XbcRg6jp5ALH+JzLLfe2xU1WKKNzWOmBV7Rbqx1/o8Nbwd53TLSTn9efJ7X2pUVo1M37ReVkexlq4gsdEDFZhMBqKDjBlKtIIDiqNsdplbk4mOoTPyVt+325CqbY8AiLm+WPpav7BgbKYcyyhIFItlNLDsWqR0xmMjRnxGy8pWLjeoTWJ8AB72ON6OmHqdNP54l2z3CfTAdj7J0lH0/HBx/6Q60J96Kd+KDyMQncSy+iVMxEUr8FL/Eb/En6SXvk1GS3qQmva7mjbhjydE/+CXEng==</latexit>

xi<latexit sha1_base64="rxBVHaA+hw0CFRz2ybgNdJiZSik=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSankrSbF+G7hpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+NjpSmf3rJrdt37t7bu99/8PDR4yf7B09Poq+CwonyxoezHCIa7XBCmgyelQHB5gZP84vPbfz0B4aovftOmxJnFpZOL7QCYte39VzP9wfpKN2avAmyDgxEZ8fzg56aFl5VFh0pAzGeZ2lJsxoCaWWw6U+riCWoC1jiOUMHFuOs3s7ayCF7CrnwgZ8jufVerajBxrixOWdaoFXcjbXO/7Hh1SCtW8a4058WH2e1dmVF6NRl+0VlJHnZyiELHVCR2TAAFTRvINUKAihi0a4xtQoXb0Jl+FfetttHV9kcAxa8vFl6Ll/ZMTZSDiXLwgJF2U0sOxapXSQwzNGfRiTlK8eN6iNYHwEFveaDxaYep02fz5LtHuEmOBmPsnSUfX03OPzUHWhPPBcvxCuRiQ/iUHwRx2IilFiKn+KX+N37m7xMXidvL1OTXlfzTFyz5P0/K1zETQ==</latexit><latexit sha1_base64="rxBVHaA+hw0CFRz2ybgNdJiZSik=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSankrSbF+G7hpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+NjpSmf3rJrdt37t7bu99/8PDR4yf7B09Poq+CwonyxoezHCIa7XBCmgyelQHB5gZP84vPbfz0B4aovftOmxJnFpZOL7QCYte39VzP9wfpKN2avAmyDgxEZ8fzg56aFl5VFh0pAzGeZ2lJsxoCaWWw6U+riCWoC1jiOUMHFuOs3s7ayCF7CrnwgZ8jufVerajBxrixOWdaoFXcjbXO/7Hh1SCtW8a4058WH2e1dmVF6NRl+0VlJHnZyiELHVCR2TAAFTRvINUKAihi0a4xtQoXb0Jl+FfetttHV9kcAxa8vFl6Ll/ZMTZSDiXLwgJF2U0sOxapXSQwzNGfRiTlK8eN6iNYHwEFveaDxaYep02fz5LtHuEmOBmPsnSUfX03OPzUHWhPPBcvxCuRiQ/iUHwRx2IilFiKn+KX+N37m7xMXidvL1OTXlfzTFyz5P0/K1zETQ==</latexit><latexit sha1_base64="rxBVHaA+hw0CFRz2ybgNdJiZSik=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSankrSbF+G7hpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+NjpSmf3rJrdt37t7bu99/8PDR4yf7B09Poq+CwonyxoezHCIa7XBCmgyelQHB5gZP84vPbfz0B4aovftOmxJnFpZOL7QCYte39VzP9wfpKN2avAmyDgxEZ8fzg56aFl5VFh0pAzGeZ2lJsxoCaWWw6U+riCWoC1jiOUMHFuOs3s7ayCF7CrnwgZ8jufVerajBxrixOWdaoFXcjbXO/7Hh1SCtW8a4058WH2e1dmVF6NRl+0VlJHnZyiELHVCR2TAAFTRvINUKAihi0a4xtQoXb0Jl+FfetttHV9kcAxa8vFl6Ll/ZMTZSDiXLwgJF2U0sOxapXSQwzNGfRiTlK8eN6iNYHwEFveaDxaYep02fz5LtHuEmOBmPsnSUfX03OPzUHWhPPBcvxCuRiQ/iUHwRx2IilFiKn+KX+N37m7xMXidvL1OTXlfzTFyz5P0/K1zETQ==</latexit><latexit sha1_base64="rxBVHaA+hw0CFRz2ybgNdJiZSik=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSankrSbF+G7hpJGOYTfNVv82fEmuwgu9mCbg51OVV1Ki+NjpSmf3rJrdt37t7bu99/8PDR4yf7B09Poq+CwonyxoezHCIa7XBCmgyelQHB5gZP84vPbfz0B4aovftOmxJnFpZOL7QCYte39VzP9wfpKN2avAmyDgxEZ8fzg56aFl5VFh0pAzGeZ2lJsxoCaWWw6U+riCWoC1jiOUMHFuOs3s7ayCF7CrnwgZ8jufVerajBxrixOWdaoFXcjbXO/7Hh1SCtW8a4058WH2e1dmVF6NRl+0VlJHnZyiELHVCR2TAAFTRvINUKAihi0a4xtQoXb0Jl+FfetttHV9kcAxa8vFl6Ll/ZMTZSDiXLwgJF2U0sOxapXSQwzNGfRiTlK8eN6iNYHwEFveaDxaYep02fz5LtHuEmOBmPsnSUfX03OPzUHWhPPBcvxCuRiQ/iUHwRx2IilFiKn+KX+N37m7xMXidvL1OTXlfzTFyz5P0/K1zETQ==</latexit>

) log p(x | ✓) =X

c2C�c(xc)

T✓c � logZ(✓)<latexit sha1_base64="/JZXOoWv9G/jQKi3ZzilkndyxZQ=">AAADG3icZVLLbtNAFB0bKCW8UliyuSKKlEolsiNeG6SKbthUKqhpKzrBGo8n8ajzsGbGkMjypyDxL+wQWxb8DePEQnlcyZ6j+zh37j2TFoJbF0V/g/DW7Tt7d/fvde4/ePjocffgyYXVpaFsTLXQ5iollgmu2NhxJ9hVYRiRqWCX6c1JE7/8yozlWp27RcEmkswUn3JKnHcl3R/4E5/ljhijvwEWegYFDLAkLk+n1bwGLHkGONUiswvpD+xy5sghvANsS5lUFHO1TKdEVCd1vZFb5Dyha2wJPfxyvkOWUHixav15sNsp6faiYbQ02AVxC3qotbPkIKA407SUTDkqiLXXcVS4SUWM41SwuoNLywpCb8iMXXuoiGR2Ui1XWUPfezKYauM/5WDpXa+oiLTN/XxmM5bdjjXO/7H+etDNG0a71d9N304qrorSMUVX7aelAKehUQsybhh1YuEBoYb7CYDmxBDqvKYbTM0DyI5MKfyfatlMb1UpU2ZY5ocXM+3LczliNUAf/Fr8giy0N4aWBbiyjgjP0cGWOapL5RtVp2R+Spzhc/+ebF2NorrjZYm3RdgFF6NhHA3jjy97x+9bgfbRM/QcDVCM3qBj9AGdoTGiwV5wFLwKXoffw5/hr/D3KjUM2pqnaMPCP/8A/N8BFA==</latexit><latexit sha1_base64="/JZXOoWv9G/jQKi3ZzilkndyxZQ=">AAADG3icZVLLbtNAFB0bKCW8UliyuSKKlEolsiNeG6SKbthUKqhpKzrBGo8n8ajzsGbGkMjypyDxL+wQWxb8DePEQnlcyZ6j+zh37j2TFoJbF0V/g/DW7Tt7d/fvde4/ePjocffgyYXVpaFsTLXQ5iollgmu2NhxJ9hVYRiRqWCX6c1JE7/8yozlWp27RcEmkswUn3JKnHcl3R/4E5/ljhijvwEWegYFDLAkLk+n1bwGLHkGONUiswvpD+xy5sghvANsS5lUFHO1TKdEVCd1vZFb5Dyha2wJPfxyvkOWUHixav15sNsp6faiYbQ02AVxC3qotbPkIKA407SUTDkqiLXXcVS4SUWM41SwuoNLywpCb8iMXXuoiGR2Ui1XWUPfezKYauM/5WDpXa+oiLTN/XxmM5bdjjXO/7H+etDNG0a71d9N304qrorSMUVX7aelAKehUQsybhh1YuEBoYb7CYDmxBDqvKYbTM0DyI5MKfyfatlMb1UpU2ZY5ocXM+3LczliNUAf/Fr8giy0N4aWBbiyjgjP0cGWOapL5RtVp2R+Spzhc/+ebF2NorrjZYm3RdgFF6NhHA3jjy97x+9bgfbRM/QcDVCM3qBj9AGdoTGiwV5wFLwKXoffw5/hr/D3KjUM2pqnaMPCP/8A/N8BFA==</latexit><latexit sha1_base64="/JZXOoWv9G/jQKi3ZzilkndyxZQ=">AAADG3icZVLLbtNAFB0bKCW8UliyuSKKlEolsiNeG6SKbthUKqhpKzrBGo8n8ajzsGbGkMjypyDxL+wQWxb8DePEQnlcyZ6j+zh37j2TFoJbF0V/g/DW7Tt7d/fvde4/ePjocffgyYXVpaFsTLXQ5iollgmu2NhxJ9hVYRiRqWCX6c1JE7/8yozlWp27RcEmkswUn3JKnHcl3R/4E5/ljhijvwEWegYFDLAkLk+n1bwGLHkGONUiswvpD+xy5sghvANsS5lUFHO1TKdEVCd1vZFb5Dyha2wJPfxyvkOWUHixav15sNsp6faiYbQ02AVxC3qotbPkIKA407SUTDkqiLXXcVS4SUWM41SwuoNLywpCb8iMXXuoiGR2Ui1XWUPfezKYauM/5WDpXa+oiLTN/XxmM5bdjjXO/7H+etDNG0a71d9N304qrorSMUVX7aelAKehUQsybhh1YuEBoYb7CYDmxBDqvKYbTM0DyI5MKfyfatlMb1UpU2ZY5ocXM+3LczliNUAf/Fr8giy0N4aWBbiyjgjP0cGWOapL5RtVp2R+Spzhc/+ebF2NorrjZYm3RdgFF6NhHA3jjy97x+9bgfbRM/QcDVCM3qBj9AGdoTGiwV5wFLwKXoffw5/hr/D3KjUM2pqnaMPCP/8A/N8BFA==</latexit><latexit sha1_base64="/JZXOoWv9G/jQKi3ZzilkndyxZQ=">AAADG3icZVLLbtNAFB0bKCW8UliyuSKKlEolsiNeG6SKbthUKqhpKzrBGo8n8ajzsGbGkMjypyDxL+wQWxb8DePEQnlcyZ6j+zh37j2TFoJbF0V/g/DW7Tt7d/fvde4/ePjocffgyYXVpaFsTLXQ5iollgmu2NhxJ9hVYRiRqWCX6c1JE7/8yozlWp27RcEmkswUn3JKnHcl3R/4E5/ljhijvwEWegYFDLAkLk+n1bwGLHkGONUiswvpD+xy5sghvANsS5lUFHO1TKdEVCd1vZFb5Dyha2wJPfxyvkOWUHixav15sNsp6faiYbQ02AVxC3qotbPkIKA407SUTDkqiLXXcVS4SUWM41SwuoNLywpCb8iMXXuoiGR2Ui1XWUPfezKYauM/5WDpXa+oiLTN/XxmM5bdjjXO/7H+etDNG0a71d9N304qrorSMUVX7aelAKehUQsybhh1YuEBoYb7CYDmxBDqvKYbTM0DyI5MKfyfatlMb1UpU2ZY5ocXM+3LczliNUAf/Fr8giy0N4aWBbiyjgjP0cGWOapL5RtVp2R+Spzhc/+ebF2NorrjZYm3RdgFF6NhHA3jjy97x+9bgfbRM/QcDVCM3qBj9AGdoTGiwV5wFLwKXoffw5/hr/D3KjUM2pqnaMPCP/8A/N8BFA==</latexit>



A Simple Example

Using numbers, e.g.: • The same graph can also be

interpreted as a binary MRF • This a more specific

representation, but it is less complex (and therefore more efficient)

• In Computer Vision, we almost always use binary MRFs; they are a specific case of general MRFs:

!13






p(x) =1

Z 12(x1, x2) 13(x1, x3) 23(x2, x3) 24(x2, x4) 34(x3, x4) 35(x3, x5)

<latexit sha1_base64="AJJf4Ll3p3A+5hWRnN3UGvxNc2U=">AAADAnicZZLNattAEMdX6lfqfjnNsS0sNQYbgpFkh/ZSCO2ll0AKdRIaGbFarewl+yF2V0Vm0a3Qd+mt9NoX6UP0HbpS5OI4A1qG3+zMaOa/acGoNkHwx/Pv3L13/8Hew96jx0+ePuvvPz/TslSYzLFkUl2kSBNGBZkbahi5KBRBPGXkPL360MTPvxKlqRSfzbogC46WguYUI+NQ0v9ejGKOzCrNbVWP4TsY5wphG9b2Sx0XmiY2jOpRlYSHVRKNOzLdkGlHopZE22S2IbOOTFsy3SZHG3I0TvqDYBK0Bm87YecMQGenyb6H40zikhNhMENaX4ZBYRYWKUMxI3UvLjUpEL5CS3LpXIE40QvbbqyGQ0cymEvlPmFgS7czLOJar3nqbjbb0buxBv6PDbeDpmoq6p3+Jn+7sFQUpSECX7fPSwaNhI0oMKOKYMPWzkFYUTcBxCvkhDBOuhuVGp2zQ1Uyd2LJm+m1KHlKFMnc8GwpXfqKR6SGcAjdWtyCNOz+GHZVIBXaIOZq9GJNDJalcI3sCapOkFG0cs9G1zYK6p6TJdwV4bZzFk3CYBJ+mg2O33cC7YEX4DUYgRC8AcfgIzgFc4DBX+/Ae+m98r/5P/yf/q/rq77X5RyAG+b//gemZ/IX</latexit><latexit sha1_base64="AJJf4Ll3p3A+5hWRnN3UGvxNc2U=">AAADAnicZZLNattAEMdX6lfqfjnNsS0sNQYbgpFkh/ZSCO2ll0AKdRIaGbFarewl+yF2V0Vm0a3Qd+mt9NoX6UP0HbpS5OI4A1qG3+zMaOa/acGoNkHwx/Pv3L13/8Hew96jx0+ePuvvPz/TslSYzLFkUl2kSBNGBZkbahi5KBRBPGXkPL360MTPvxKlqRSfzbogC46WguYUI+NQ0v9ejGKOzCrNbVWP4TsY5wphG9b2Sx0XmiY2jOpRlYSHVRKNOzLdkGlHopZE22S2IbOOTFsy3SZHG3I0TvqDYBK0Bm87YecMQGenyb6H40zikhNhMENaX4ZBYRYWKUMxI3UvLjUpEL5CS3LpXIE40QvbbqyGQ0cymEvlPmFgS7czLOJar3nqbjbb0buxBv6PDbeDpmoq6p3+Jn+7sFQUpSECX7fPSwaNhI0oMKOKYMPWzkFYUTcBxCvkhDBOuhuVGp2zQ1Uyd2LJm+m1KHlKFMnc8GwpXfqKR6SGcAjdWtyCNOz+GHZVIBXaIOZq9GJNDJalcI3sCapOkFG0cs9G1zYK6p6TJdwV4bZzFk3CYBJ+mg2O33cC7YEX4DUYgRC8AcfgIzgFc4DBX+/Ae+m98r/5P/yf/q/rq77X5RyAG+b//gemZ/IX</latexit><latexit sha1_base64="AJJf4Ll3p3A+5hWRnN3UGvxNc2U=">AAADAnicZZLNattAEMdX6lfqfjnNsS0sNQYbgpFkh/ZSCO2ll0AKdRIaGbFarewl+yF2V0Vm0a3Qd+mt9NoX6UP0HbpS5OI4A1qG3+zMaOa/acGoNkHwx/Pv3L13/8Hew96jx0+ePuvvPz/TslSYzLFkUl2kSBNGBZkbahi5KBRBPGXkPL360MTPvxKlqRSfzbogC46WguYUI+NQ0v9ejGKOzCrNbVWP4TsY5wphG9b2Sx0XmiY2jOpRlYSHVRKNOzLdkGlHopZE22S2IbOOTFsy3SZHG3I0TvqDYBK0Bm87YecMQGenyb6H40zikhNhMENaX4ZBYRYWKUMxI3UvLjUpEL5CS3LpXIE40QvbbqyGQ0cymEvlPmFgS7czLOJar3nqbjbb0buxBv6PDbeDpmoq6p3+Jn+7sFQUpSECX7fPSwaNhI0oMKOKYMPWzkFYUTcBxCvkhDBOuhuVGp2zQ1Uyd2LJm+m1KHlKFMnc8GwpXfqKR6SGcAjdWtyCNOz+GHZVIBXaIOZq9GJNDJalcI3sCapOkFG0cs9G1zYK6p6TJdwV4bZzFk3CYBJ+mg2O33cC7YEX4DUYgRC8AcfgIzgFc4DBX+/Ae+m98r/5P/yf/q/rq77X5RyAG+b//gemZ/IX</latexit><latexit sha1_base64="AJJf4Ll3p3A+5hWRnN3UGvxNc2U=">AAADAnicZZLNattAEMdX6lfqfjnNsS0sNQYbgpFkh/ZSCO2ll0AKdRIaGbFarewl+yF2V0Vm0a3Qd+mt9NoX6UP0HbpS5OI4A1qG3+zMaOa/acGoNkHwx/Pv3L13/8Hew96jx0+ePuvvPz/TslSYzLFkUl2kSBNGBZkbahi5KBRBPGXkPL360MTPvxKlqRSfzbogC46WguYUI+NQ0v9ejGKOzCrNbVWP4TsY5wphG9b2Sx0XmiY2jOpRlYSHVRKNOzLdkGlHopZE22S2IbOOTFsy3SZHG3I0TvqDYBK0Bm87YecMQGenyb6H40zikhNhMENaX4ZBYRYWKUMxI3UvLjUpEL5CS3LpXIE40QvbbqyGQ0cymEvlPmFgS7czLOJar3nqbjbb0buxBv6PDbeDpmoq6p3+Jn+7sFQUpSECX7fPSwaNhI0oMKOKYMPWzkFYUTcBxCvkhDBOuhuVGp2zQ1Uyd2LJm+m1KHlKFMnc8GwpXfqKR6SGcAjdWtyCNOz+GHZVIBXaIOZq9GJNDJalcI3sCapOkFG0cs9G1zYK6p6TJdwV4bZzFk3CYBJ+mg2O33cC7YEX4DUYgRC8AcfgIzgFc4DBX+/Ae+m98r/5P/yf/q/rq77X5RyAG+b//gemZ/IX</latexit>



Using Graphical Models

We can use a graphical model to do inference:

• We want to find • Some nodes in the graph are observed, for others

we want to find the posterior distribution

• Also, computing the local marginal distribution p(xn) at any node xn can be done using inference.

Question: How can inference be done with a graphical model?

We will see that, when exploiting conditional independences, we can do efficient inference.

!14

argmaxx

p(x)<latexit sha1_base64="wpE+fo2E2RivgYtFxJCz/z7Phg8=">AAACn3icZVHbbtNAEN2YWwmXpvDIy0KIVCQU2RESPFbwAC9FQSJtUR1F6/UkWXUv1u4YObL8A3wNr/An/A3j1II2HWlHZ+c+c7JCq4Bx/KcX3bp95+69vfv9Bw8fPd4fHDw5Ca70EmbSaefPMhFAKwszVKjhrPAgTKbhNLv40PpPv4MPytmvuClgbsTKqqWSAsm0GLxMhV/x1IhqUZPGdbasq6YpDv9/XvHFYBiP463wmyDpwJB1Ml0c9GSaO1kasCi1COE8iQuc18KjkhqafloGKIS8ECs4J2iFgTCvt+s0fESWnC+dp2eRb61XM2phQtiYjCLbIcOurzX+842uOrFqK4ad/rh8N6+VLUoEKy/bL0vN0fH2YjxXHiTqDQEhvaINuFwLLyTSXa9VaknIX/tSk5bOtNsHW5oMPOS0vF45Sl+bCTScjzidhQ4UeDcx76pwZQMKTTX6aQCUrrTUqD4W1bFAryriNDT1JG76REuyS8JNcDIZJ/E4+fJmePS+I2iPPWMv2CFL2Ft2xD6xKZsxyX6wn+wX+x09jz5Gn6PpZWjU63KesmsSffsLE33RlQ==</latexit><latexit sha1_base64="wpE+fo2E2RivgYtFxJCz/z7Phg8=">AAACn3icZVHbbtNAEN2YWwmXpvDIy0KIVCQU2RESPFbwAC9FQSJtUR1F6/UkWXUv1u4YObL8A3wNr/An/A3j1II2HWlHZ+c+c7JCq4Bx/KcX3bp95+69vfv9Bw8fPd4fHDw5Ca70EmbSaefPMhFAKwszVKjhrPAgTKbhNLv40PpPv4MPytmvuClgbsTKqqWSAsm0GLxMhV/x1IhqUZPGdbasq6YpDv9/XvHFYBiP463wmyDpwJB1Ml0c9GSaO1kasCi1COE8iQuc18KjkhqafloGKIS8ECs4J2iFgTCvt+s0fESWnC+dp2eRb61XM2phQtiYjCLbIcOurzX+842uOrFqK4ad/rh8N6+VLUoEKy/bL0vN0fH2YjxXHiTqDQEhvaINuFwLLyTSXa9VaknIX/tSk5bOtNsHW5oMPOS0vF45Sl+bCTScjzidhQ4UeDcx76pwZQMKTTX6aQCUrrTUqD4W1bFAryriNDT1JG76REuyS8JNcDIZJ/E4+fJmePS+I2iPPWMv2CFL2Ft2xD6xKZsxyX6wn+wX+x09jz5Gn6PpZWjU63KesmsSffsLE33RlQ==</latexit><latexit sha1_base64="wpE+fo2E2RivgYtFxJCz/z7Phg8=">AAACn3icZVHbbtNAEN2YWwmXpvDIy0KIVCQU2RESPFbwAC9FQSJtUR1F6/UkWXUv1u4YObL8A3wNr/An/A3j1II2HWlHZ+c+c7JCq4Bx/KcX3bp95+69vfv9Bw8fPd4fHDw5Ca70EmbSaefPMhFAKwszVKjhrPAgTKbhNLv40PpPv4MPytmvuClgbsTKqqWSAsm0GLxMhV/x1IhqUZPGdbasq6YpDv9/XvHFYBiP463wmyDpwJB1Ml0c9GSaO1kasCi1COE8iQuc18KjkhqafloGKIS8ECs4J2iFgTCvt+s0fESWnC+dp2eRb61XM2phQtiYjCLbIcOurzX+842uOrFqK4ad/rh8N6+VLUoEKy/bL0vN0fH2YjxXHiTqDQEhvaINuFwLLyTSXa9VaknIX/tSk5bOtNsHW5oMPOS0vF45Sl+bCTScjzidhQ4UeDcx76pwZQMKTTX6aQCUrrTUqD4W1bFAryriNDT1JG76REuyS8JNcDIZJ/E4+fJmePS+I2iPPWMv2CFL2Ft2xD6xKZsxyX6wn+wX+x09jz5Gn6PpZWjU63KesmsSffsLE33RlQ==</latexit><latexit sha1_base64="wpE+fo2E2RivgYtFxJCz/z7Phg8=">AAACn3icZVHbbtNAEN2YWwmXpvDIy0KIVCQU2RESPFbwAC9FQSJtUR1F6/UkWXUv1u4YObL8A3wNr/An/A3j1II2HWlHZ+c+c7JCq4Bx/KcX3bp95+69vfv9Bw8fPd4fHDw5Ca70EmbSaefPMhFAKwszVKjhrPAgTKbhNLv40PpPv4MPytmvuClgbsTKqqWSAsm0GLxMhV/x1IhqUZPGdbasq6YpDv9/XvHFYBiP463wmyDpwJB1Ml0c9GSaO1kasCi1COE8iQuc18KjkhqafloGKIS8ECs4J2iFgTCvt+s0fESWnC+dp2eRb61XM2phQtiYjCLbIcOurzX+842uOrFqK4ad/rh8N6+VLUoEKy/bL0vN0fH2YjxXHiTqDQEhvaINuFwLLyTSXa9VaknIX/tSk5bOtNsHW5oMPOS0vF45Sl+bCTScjzidhQ4UeDcx76pwZQMKTTX6aQCUrrTUqD4W1bFAryriNDT1JG76REuyS8JNcDIZJ/E4+fJmePS+I2iPPWMv2CFL2Ft2xD6xKZsxyX6wn+wX+x09jz5Gn6PpZWjU63KesmsSffsLE33RlQ==</latexit>



Example Application: Denoising

Aim: Recover the noise-free image from the noisy one

We model the original image with variables and the noisy image with pixel values

!15

Noise-free image Noisy image (observation)

xi 2 {�1, 1}<latexit sha1_base64="R82+9CmUS5QEr/XfBrpZ/o53jBM=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31llPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+ud8ff</latexit><latexit sha1_base64="R82+9CmUS5QEr/XfBrpZ/o53jBM=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31llPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+ud8ff</latexit><latexit sha1_base64="R82+9CmUS5QEr/XfBrpZ/o53jBM=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31llPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+ud8ff</latexit><latexit sha1_base64="R82+9CmUS5QEr/XfBrpZ/o53jBM=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31llPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+ud8ff</latexit>

yi 2 {�1, 1}<latexit sha1_base64="GzsNihGvaDzpf3ZLFzgLqkLFF68=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31VlPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+wnMfg</latexit><latexit sha1_base64="GzsNihGvaDzpf3ZLFzgLqkLFF68=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31VlPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+wnMfg</latexit><latexit sha1_base64="GzsNihGvaDzpf3ZLFzgLqkLFF68=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31VlPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+wnMfg</latexit><latexit sha1_base64="GzsNihGvaDzpf3ZLFzgLqkLFF68=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31VlPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+wnMfg</latexit>

?




Aim: Recover the noise-free image from the noisy one

We model the original image with variables and the noisy image with pixel values

We consider the true pixel vales as hidden or latent

!16


xi 2 {�1, 1}<latexit sha1_base64="R82+9CmUS5QEr/XfBrpZ/o53jBM=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31llPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+ud8ff</latexit><latexit sha1_base64="R82+9CmUS5QEr/XfBrpZ/o53jBM=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31llPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+ud8ff</latexit><latexit sha1_base64="R82+9CmUS5QEr/XfBrpZ/o53jBM=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31llPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+ud8ff</latexit><latexit sha1_base64="R82+9CmUS5QEr/XfBrpZ/o53jBM=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31llPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+ud8ff</latexit>

yi 2 {�1, 1}<latexit sha1_base64="GzsNihGvaDzpf3ZLFzgLqkLFF68=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31VlPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+wnMfg</latexit><latexit sha1_base64="GzsNihGvaDzpf3ZLFzgLqkLFF68=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31VlPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+wnMfg</latexit><latexit sha1_base64="GzsNihGvaDzpf3ZLFzgLqkLFF68=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31VlPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+wnMfg</latexit><latexit sha1_base64="GzsNihGvaDzpf3ZLFzgLqkLFF68=">AAACi3icZVFNa9tAEF2raZs6Teu0x16WGEMPqZHcQkvpITQUegkkECeByJjVamwv2Q+xOwo2Qv+k1/Y/5d905IiQOAO7PObjzcybrNAqYBzfdqJnW89fvNx+1d15vfvmbW/v3XlwpZcwlk47f5mJAFpZGKNCDZeFB2EyDRfZ9VETv7gBH5SzZ7gqYGLE3KqZkgLJNe31VlPFU2V5Wn1KDpK0nvb68TBeG38Kkhb0WWsn072OTHMnSwMWpRYhXCVxgZNKeFRSQ91NywCFkNdiDlcErTAQJtV69JoPyJPzmfP0LPK192FFJUwIK5NRphG4CJuxxnkfGzwM4rJhDBv9cfZtUilblAhW3rWflZqj4406PFceJOoVASG9og24XAgvJJKGj5gawfMDX2r6pTPN9sGWJgMPOS2v547KF2YENecDTrKQQIG3E/OWhSsbUGji6KYBULrSUqPqWCyPBXq1pPuFuhrFdZfOkmwe4Sk4Hw2TeJicfukf/mwPtM0+sH32kSXsKztkv9kJGzPJbtgf9pf9i3ajz9H36MddatRpa96zRxb9+g+wnMfg</latexit>

?




!17

We define two simple edge features:

These are multiplied by parameters and :


yi<latexit sha1_base64="T4bOcyBkSaDwgzBUUPvBcr4MpBQ=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSankrSbF+GrhpJGOYTfNVv82fEnuwgu9mCbg51OVV1Ki+NJk7TP73k1u07d+/t3e8/ePjo8ZP9g6cn5KugcKK88eEsB0KjHU5Ys8GzMiDY3OBpfvG5jZ/+wEDau++8KXFmYen0Qivg6Pq2mev5/iAdpVuTN0HWgYHo7Hh+0FPTwqvKomNlgOg8S0ue1RBYK4NNf1oRlqAuYInnETqwSLN6O2sjh9FTyIUP8TmWW+/Vihos0cbmMdMCr2g31jr/x4ZXg7xuGWmnPy8+zmrtyorRqcv2i8pI9rKVQxY6oGKziQBU0HEDqVYQQHEU7RpTq3DxJlQm/srbdntylc0xYBGXN0sfy1d2jI2UQxlliQKR7CaWHYvUjhhM5OhPCVn5ysVG9RGsj4CDXseDUVOP06Yfz5LtHuEmOBmPsnSUfX03OPzUHWhPPBcvxCuRiQ/iUHwRx2IilFiKn+KX+N37m7xMXidvL1OTXlfzTFyz5P0/LXTETg==</latexit><latexit sha1_base64="T4bOcyBkSaDwgzBUUPvBcr4MpBQ=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSankrSbF+GrhpJGOYTfNVv82fEnuwgu9mCbg51OVV1Ki+NJk7TP73k1u07d+/t3e8/ePjo8ZP9g6cn5KugcKK88eEsB0KjHU5Ys8GzMiDY3OBpfvG5jZ/+wEDau++8KXFmYen0Qivg6Pq2mev5/iAdpVuTN0HWgYHo7Hh+0FPTwqvKomNlgOg8S0ue1RBYK4NNf1oRlqAuYInnETqwSLN6O2sjh9FTyIUP8TmWW+/Vihos0cbmMdMCr2g31jr/x4ZXg7xuGWmnPy8+zmrtyorRqcv2i8pI9rKVQxY6oGKziQBU0HEDqVYQQHEU7RpTq3DxJlQm/srbdntylc0xYBGXN0sfy1d2jI2UQxlliQKR7CaWHYvUjhhM5OhPCVn5ysVG9RGsj4CDXseDUVOP06Yfz5LtHuEmOBmPsnSUfX03OPzUHWhPPBcvxCuRiQ/iUHwRx2IilFiKn+KX+N37m7xMXidvL1OTXlfzTFyz5P0/LXTETg==</latexit><latexit sha1_base64="T4bOcyBkSaDwgzBUUPvBcr4MpBQ=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSankrSbF+GrhpJGOYTfNVv82fEnuwgu9mCbg51OVV1Ki+NJk7TP73k1u07d+/t3e8/ePjo8ZP9g6cn5KugcKK88eEsB0KjHU5Ys8GzMiDY3OBpfvG5jZ/+wEDau++8KXFmYen0Qivg6Pq2mev5/iAdpVuTN0HWgYHo7Hh+0FPTwqvKomNlgOg8S0ue1RBYK4NNf1oRlqAuYInnETqwSLN6O2sjh9FTyIUP8TmWW+/Vihos0cbmMdMCr2g31jr/x4ZXg7xuGWmnPy8+zmrtyorRqcv2i8pI9rKVQxY6oGKziQBU0HEDqVYQQHEU7RpTq3DxJlQm/srbdntylc0xYBGXN0sfy1d2jI2UQxlliQKR7CaWHYvUjhhM5OhPCVn5ysVG9RGsj4CDXseDUVOP06Yfz5LtHuEmOBmPsnSUfX03OPzUHWhPPBcvxCuRiQ/iUHwRx2IilFiKn+KX+N37m7xMXidvL1OTXlfzTFyz5P0/LXTETg==</latexit><latexit sha1_base64="T4bOcyBkSaDwgzBUUPvBcr4MpBQ=">AAACfHicZVHbihNBEO2MtzXedvXRl8YYENQwE0R9XPTFl4UVze7CJoSankrSbF+GrhpJGOYTfNVv82fEnuwgu9mCbg51OVV1Ki+NJk7TP73k1u07d+/t3e8/ePjo8ZP9g6cn5KugcKK88eEsB0KjHU5Ys8GzMiDY3OBpfvG5jZ/+wEDau++8KXFmYen0Qivg6Pq2mev5/iAdpVuTN0HWgYHo7Hh+0FPTwqvKomNlgOg8S0ue1RBYK4NNf1oRlqAuYInnETqwSLN6O2sjh9FTyIUP8TmWW+/Vihos0cbmMdMCr2g31jr/x4ZXg7xuGWmnPy8+zmrtyorRqcv2i8pI9rKVQxY6oGKziQBU0HEDqVYQQHEU7RpTq3DxJlQm/srbdntylc0xYBGXN0sfy1d2jI2UQxlliQKR7CaWHYvUjhhM5OhPCVn5ysVG9RGsj4CDXseDUVOP06Yfz5LtHuEmOBmPsnSUfX03OPzUHWhPPBcvxCuRiQ/iUHwRx2IilFiKn+KX+N37m7xMXidvL1OTXlfzTFyz5P0/LXTETg==</latexit>

xj<latexit sha1_base64="/uiTXGk2yRxVSM8cHktxwgEDOwo=">AAACfHicZVHbattAEF2rSZu6aXN77MtSx1BIaiQTmjyG9qUvgZTWSSA2ZrQa25vsReyOgo3QJ/S1/bb8TOnKESVxBnY5zOXMzJk0V9JTHN+3ohdr6y9fbbxuv9l8+25re2f3wtvCCRwIq6y7SsGjkgYHJEnhVe4QdKrwMr39Wscv79B5ac1PWuQ40jA1ciIFUHD9mI9vxtuduBcvjT8HSQM6rLHz8U5LDDMrCo2GhALvr5M4p1EJjqRQWLWHhcccxC1M8TpAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NDkZldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4ezQFSr8wup6e28KnaLDLCyvpjaUz3QfK867PMgSBPK8mZg3LFwaT6ACR3vokYQtTGhUnsH8DMjJeTiYr8p+XLXDWZLVIzwHF/1eEveS70ed0y/NgTbYe/aBfWQJO2an7Bs7ZwMm2JT9Yr/Zn9bfaD86iD49pEatpmaPPbHo8z8tcsRO</latexit><latexit sha1_base64="/uiTXGk2yRxVSM8cHktxwgEDOwo=">AAACfHicZVHbattAEF2rSZu6aXN77MtSx1BIaiQTmjyG9qUvgZTWSSA2ZrQa25vsReyOgo3QJ/S1/bb8TOnKESVxBnY5zOXMzJk0V9JTHN+3ohdr6y9fbbxuv9l8+25re2f3wtvCCRwIq6y7SsGjkgYHJEnhVe4QdKrwMr39Wscv79B5ac1PWuQ40jA1ciIFUHD9mI9vxtuduBcvjT8HSQM6rLHz8U5LDDMrCo2GhALvr5M4p1EJjqRQWLWHhcccxC1M8TpAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NDkZldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4ezQFSr8wup6e28KnaLDLCyvpjaUz3QfK867PMgSBPK8mZg3LFwaT6ACR3vokYQtTGhUnsH8DMjJeTiYr8p+XLXDWZLVIzwHF/1eEveS70ed0y/NgTbYe/aBfWQJO2an7Bs7ZwMm2JT9Yr/Zn9bfaD86iD49pEatpmaPPbHo8z8tcsRO</latexit><latexit sha1_base64="/uiTXGk2yRxVSM8cHktxwgEDOwo=">AAACfHicZVHbattAEF2rSZu6aXN77MtSx1BIaiQTmjyG9qUvgZTWSSA2ZrQa25vsReyOgo3QJ/S1/bb8TOnKESVxBnY5zOXMzJk0V9JTHN+3ohdr6y9fbbxuv9l8+25re2f3wtvCCRwIq6y7SsGjkgYHJEnhVe4QdKrwMr39Wscv79B5ac1PWuQ40jA1ciIFUHD9mI9vxtuduBcvjT8HSQM6rLHz8U5LDDMrCo2GhALvr5M4p1EJjqRQWLWHhcccxC1M8TpAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NDkZldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4ezQFSr8wup6e28KnaLDLCyvpjaUz3QfK867PMgSBPK8mZg3LFwaT6ACR3vokYQtTGhUnsH8DMjJeTiYr8p+XLXDWZLVIzwHF/1eEveS70ed0y/NgTbYe/aBfWQJO2an7Bs7ZwMm2JT9Yr/Zn9bfaD86iD49pEatpmaPPbHo8z8tcsRO</latexit><latexit sha1_base64="/uiTXGk2yRxVSM8cHktxwgEDOwo=">AAACfHicZVHbattAEF2rSZu6aXN77MtSx1BIaiQTmjyG9qUvgZTWSSA2ZrQa25vsReyOgo3QJ/S1/bb8TOnKESVxBnY5zOXMzJk0V9JTHN+3ohdr6y9fbbxuv9l8+25re2f3wtvCCRwIq6y7SsGjkgYHJEnhVe4QdKrwMr39Wscv79B5ac1PWuQ40jA1ciIFUHD9mI9vxtuduBcvjT8HSQM6rLHz8U5LDDMrCo2GhALvr5M4p1EJjqRQWLWHhcccxC1M8TpAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NDkZldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4ezQFSr8wup6e28KnaLDLCyvpjaUz3QfK867PMgSBPK8mZg3LFwaT6ACR3vokYQtTGhUnsH8DMjJeTiYr8p+XLXDWZLVIzwHF/1eEveS70ed0y/NgTbYe/aBfWQJO2an7Bs7ZwMm2JT9Yr/Zn9bfaD86iD49pEatpmaPPbHo8z8tcsRO</latexit>

�(xi, xj) = xixj<latexit sha1_base64="vh1mweHcLGRGTudW1ePmPptylxQ=">AAACknicZVHbattAEF2rt9S9OW2hD31ZagwpBCOZQkuhkDYvfWgghToJxEasVmN7k72I3VGREfqavrYf1L/pyBElcQZ2OZyzM7NzJiu0ChjHf3vRnbv37j/Yedh/9PjJ02eD3ecnwZVewlQ67fxZJgJoZWGKCjWcFR6EyTScZpeHrX76E3xQzv7AdQFzI5ZWLZQUSFQ6eDUrVmqvStV+lV685Z84J0wwHQzjcbwJfhskHRiyLo7T3Z6c5U6WBixKLUI4T+IC57XwqKSGpj8rAxRCXoolnBO0wkCY15sJGj4iJucL5+lY5Bv2ekYtTAhrk9FLI3AVtrWW/K+NrotYtRXDVn9cfJjXyhYlgpVX7Rel5uh4axLPlQeJek1ASK9oAi5XwguJZOWNSq3v+b4vNd3SmXb6YEuTgYechtdLR+krM4GG8xEnW8igwLsf864KVzag0FSjPwuA0pWWGtVHojoS6FVFawxNPYmbPq0l2V7CbXAyGSfxOPn+bnjwpVvQDnvN3rA9lrD37IB9ZcdsyiRr2C/2m/2JXkYfo8/R4dXTqNflvGA3Ivr2D1P9ysA=</latexit><latexit sha1_base64="vh1mweHcLGRGTudW1ePmPptylxQ=">AAACknicZVHbattAEF2rt9S9OW2hD31ZagwpBCOZQkuhkDYvfWgghToJxEasVmN7k72I3VGREfqavrYf1L/pyBElcQZ2OZyzM7NzJiu0ChjHf3vRnbv37j/Yedh/9PjJ02eD3ecnwZVewlQ67fxZJgJoZWGKCjWcFR6EyTScZpeHrX76E3xQzv7AdQFzI5ZWLZQUSFQ6eDUrVmqvStV+lV685Z84J0wwHQzjcbwJfhskHRiyLo7T3Z6c5U6WBixKLUI4T+IC57XwqKSGpj8rAxRCXoolnBO0wkCY15sJGj4iJucL5+lY5Bv2ekYtTAhrk9FLI3AVtrWW/K+NrotYtRXDVn9cfJjXyhYlgpVX7Rel5uh4axLPlQeJek1ASK9oAi5XwguJZOWNSq3v+b4vNd3SmXb6YEuTgYechtdLR+krM4GG8xEnW8igwLsf864KVzag0FSjPwuA0pWWGtVHojoS6FVFawxNPYmbPq0l2V7CbXAyGSfxOPn+bnjwpVvQDnvN3rA9lrD37IB9ZcdsyiRr2C/2m/2JXkYfo8/R4dXTqNflvGA3Ivr2D1P9ysA=</latexit><latexit sha1_base64="vh1mweHcLGRGTudW1ePmPptylxQ=">AAACknicZVHbattAEF2rt9S9OW2hD31ZagwpBCOZQkuhkDYvfWgghToJxEasVmN7k72I3VGREfqavrYf1L/pyBElcQZ2OZyzM7NzJiu0ChjHf3vRnbv37j/Yedh/9PjJ02eD3ecnwZVewlQ67fxZJgJoZWGKCjWcFR6EyTScZpeHrX76E3xQzv7AdQFzI5ZWLZQUSFQ6eDUrVmqvStV+lV685Z84J0wwHQzjcbwJfhskHRiyLo7T3Z6c5U6WBixKLUI4T+IC57XwqKSGpj8rAxRCXoolnBO0wkCY15sJGj4iJucL5+lY5Bv2ekYtTAhrk9FLI3AVtrWW/K+NrotYtRXDVn9cfJjXyhYlgpVX7Rel5uh4axLPlQeJek1ASK9oAi5XwguJZOWNSq3v+b4vNd3SmXb6YEuTgYechtdLR+krM4GG8xEnW8igwLsf864KVzag0FSjPwuA0pWWGtVHojoS6FVFawxNPYmbPq0l2V7CbXAyGSfxOPn+bnjwpVvQDnvN3rA9lrD37IB9ZcdsyiRr2C/2m/2JXkYfo8/R4dXTqNflvGA3Ivr2D1P9ysA=</latexit><latexit sha1_base64="vh1mweHcLGRGTudW1ePmPptylxQ=">AAACknicZVHbattAEF2rt9S9OW2hD31ZagwpBCOZQkuhkDYvfWgghToJxEasVmN7k72I3VGREfqavrYf1L/pyBElcQZ2OZyzM7NzJiu0ChjHf3vRnbv37j/Yedh/9PjJ02eD3ecnwZVewlQ67fxZJgJoZWGKCjWcFR6EyTScZpeHrX76E3xQzv7AdQFzI5ZWLZQUSFQ6eDUrVmqvStV+lV685Z84J0wwHQzjcbwJfhskHRiyLo7T3Z6c5U6WBixKLUI4T+IC57XwqKSGpj8rAxRCXoolnBO0wkCY15sJGj4iJucL5+lY5Bv2ekYtTAhrk9FLI3AVtrWW/K+NrotYtRXDVn9cfJjXyhYlgpVX7Rel5uh4axLPlQeJek1ASK9oAi5XwguJZOWNSq3v+b4vNd3SmXb6YEuTgYechtdLR+krM4GG8xEnW8igwLsf864KVzag0FSjPwuA0pWWGtVHojoS6FVFawxNPYmbPq0l2V7CbXAyGSfxOPn+bnjwpVvQDnvN3rA9lrD37IB9ZcdsyiRr2C/2m/2JXkYfo8/R4dXTqNflvGA3Ivr2D1P9ysA=</latexit>

�<latexit sha1_base64="4S/SCOBrObd0drPevhFgUbuyfv8=">AAACfnicZVFLa9tAEF6rr8R9Je0xl6XGpYfWlUyhOYb20ksggToJxCaMVmN7yT7EzqjYCP2GXtuf1n/TlSNC4gzs8jGPb2a+yUujidP0Xy959PjJ02c7u/3nL16+er23/+aMfBUUTpQ3PlzkQGi0wwlrNnhRBgSbGzzPr7+38fNfGEh795PXJc4sLJyeawUcXZNpjgxXe4N0lG5MPgRZBwais5Or/Z6aFl5VFh0rA0SXWVryrIbAWhls+tOKsAR1DQu8jNCBRZrVm2kbOYyeQs59iM+x3HjvVtRgidY2j5kWeEnbsdZ5GxveDfKqZaSt/jw/nNXalRWjUzft55WR7GUriCx0QMVmHQGooOMGUi0hgOIo2z2mVuPiY6hM/JW37fbkKptjwCIubxY+li/tGBsphzLKEgUi2U0sOxapHTGYyNGfErLylYuN6mNYHQMHvYono6Yep00/niXbPsJDcDYeZekoO/0yOPrWHWhHHIh34oPIxFdxJH6IEzERSmjxW/wRfxORvE8+JZ9vUpNeV/NW3LPk8D/R3sQa</latexit><latexit sha1_base64="4S/SCOBrObd0drPevhFgUbuyfv8=">AAACfnicZVFLa9tAEF6rr8R9Je0xl6XGpYfWlUyhOYb20ksggToJxCaMVmN7yT7EzqjYCP2GXtuf1n/TlSNC4gzs8jGPb2a+yUujidP0Xy959PjJ02c7u/3nL16+er23/+aMfBUUTpQ3PlzkQGi0wwlrNnhRBgSbGzzPr7+38fNfGEh795PXJc4sLJyeawUcXZNpjgxXe4N0lG5MPgRZBwais5Or/Z6aFl5VFh0rA0SXWVryrIbAWhls+tOKsAR1DQu8jNCBRZrVm2kbOYyeQs59iM+x3HjvVtRgidY2j5kWeEnbsdZ5GxveDfKqZaSt/jw/nNXalRWjUzft55WR7GUriCx0QMVmHQGooOMGUi0hgOIo2z2mVuPiY6hM/JW37fbkKptjwCIubxY+li/tGBsphzLKEgUi2U0sOxapHTGYyNGfErLylYuN6mNYHQMHvYono6Yep00/niXbPsJDcDYeZekoO/0yOPrWHWhHHIh34oPIxFdxJH6IEzERSmjxW/wRfxORvE8+JZ9vUpNeV/NW3LPk8D/R3sQa</latexit><latexit sha1_base64="4S/SCOBrObd0drPevhFgUbuyfv8=">AAACfnicZVFLa9tAEF6rr8R9Je0xl6XGpYfWlUyhOYb20ksggToJxCaMVmN7yT7EzqjYCP2GXtuf1n/TlSNC4gzs8jGPb2a+yUujidP0Xy959PjJ02c7u/3nL16+er23/+aMfBUUTpQ3PlzkQGi0wwlrNnhRBgSbGzzPr7+38fNfGEh795PXJc4sLJyeawUcXZNpjgxXe4N0lG5MPgRZBwais5Or/Z6aFl5VFh0rA0SXWVryrIbAWhls+tOKsAR1DQu8jNCBRZrVm2kbOYyeQs59iM+x3HjvVtRgidY2j5kWeEnbsdZ5GxveDfKqZaSt/jw/nNXalRWjUzft55WR7GUriCx0QMVmHQGooOMGUi0hgOIo2z2mVuPiY6hM/JW37fbkKptjwCIubxY+li/tGBsphzLKEgUi2U0sOxapHTGYyNGfErLylYuN6mNYHQMHvYono6Yep00/niXbPsJDcDYeZekoO/0yOPrWHWhHHIh34oPIxFdxJH6IEzERSmjxW/wRfxORvE8+JZ9vUpNeV/NW3LPk8D/R3sQa</latexit><latexit sha1_base64="4S/SCOBrObd0drPevhFgUbuyfv8=">AAACfnicZVFLa9tAEF6rr8R9Je0xl6XGpYfWlUyhOYb20ksggToJxCaMVmN7yT7EzqjYCP2GXtuf1n/TlSNC4gzs8jGPb2a+yUujidP0Xy959PjJ02c7u/3nL16+er23/+aMfBUUTpQ3PlzkQGi0wwlrNnhRBgSbGzzPr7+38fNfGEh795PXJc4sLJyeawUcXZNpjgxXe4N0lG5MPgRZBwais5Or/Z6aFl5VFh0rA0SXWVryrIbAWhls+tOKsAR1DQu8jNCBRZrVm2kbOYyeQs59iM+x3HjvVtRgidY2j5kWeEnbsdZ5GxveDfKqZaSt/jw/nNXalRWjUzft55WR7GUriCx0QMVmHQGooOMGUi0hgOIo2z2mVuPiY6hM/JW37fbkKptjwCIubxY+li/tGBsphzLKEgUi2U0sOxapHTGYyNGfErLylYuN6mNYHQMHvYono6Yep00/niXbPsJDcDYeZekoO/0yOPrWHWhHHIh34oPIxFdxJH6IEzERSmjxW/wRfxORvE8+JZ9vUpNeV/NW3LPk8D/R3sQa</latexit>

⌘<latexit sha1_base64="dnn+0zb/dBVyW3r8zSyldrgTiFg=">AAACfXicZVFLa9tAEF4rfaTuK0mPvSw1hh6CkUwgPYb2kksghToJxCaMVmN7yT7EzqjYCP2FXpu/1l/TrhxREmdgl495fDPzTV4aTZymf3rJzrPnL17uvuq/fvP23fu9/YML8lVQOFHe+HCVA6HRDies2eBVGRBsbvAyv/3Wxi9/YiDt3Q9elzizsHB6rhVw65oiw83eIB2lG5NPQdaBgejs/Ga/p6aFV5VFx8oA0XWWljyrIbBWBpv+tCIsQd3CAq8jdGCRZvVm2EYOo6eQcx/icyw33ocVNViitc1jpgVe0nasdf6PDR8GedUy0lZ/nn+Z1dqVFaNT9+3nlZHsZauHLHRAxWYdAaig4wZSLSGA4qjaI6ZW4uIwVCb+ytt2e3KVzTFgEZc3Cx/Ll3aMjZRDGWWJApHsJpYdi9SOGEzk6E8JWfnKxUb1GazOgINexYtRU4/Tph/Pkm0f4Sm4GI+ydJR9PxqcfO0OtCs+ik/is8jEsTgRp+JcTIQSS/FL/BZ3vb/JMDlMRvepSa+r+SAeWXL8DwVMxK0=</latexit><latexit sha1_base64="dnn+0zb/dBVyW3r8zSyldrgTiFg=">AAACfXicZVFLa9tAEF4rfaTuK0mPvSw1hh6CkUwgPYb2kksghToJxCaMVmN7yT7EzqjYCP2FXpu/1l/TrhxREmdgl495fDPzTV4aTZymf3rJzrPnL17uvuq/fvP23fu9/YML8lVQOFHe+HCVA6HRDies2eBVGRBsbvAyv/3Wxi9/YiDt3Q9elzizsHB6rhVw65oiw83eIB2lG5NPQdaBgejs/Ga/p6aFV5VFx8oA0XWWljyrIbBWBpv+tCIsQd3CAq8jdGCRZvVm2EYOo6eQcx/icyw33ocVNViitc1jpgVe0nasdf6PDR8GedUy0lZ/nn+Z1dqVFaNT9+3nlZHsZauHLHRAxWYdAaig4wZSLSGA4qjaI6ZW4uIwVCb+ytt2e3KVzTFgEZc3Cx/Ll3aMjZRDGWWJApHsJpYdi9SOGEzk6E8JWfnKxUb1GazOgINexYtRU4/Tph/Pkm0f4Sm4GI+ydJR9PxqcfO0OtCs+ik/is8jEsTgRp+JcTIQSS/FL/BZ3vb/JMDlMRvepSa+r+SAeWXL8DwVMxK0=</latexit><latexit sha1_base64="dnn+0zb/dBVyW3r8zSyldrgTiFg=">AAACfXicZVFLa9tAEF4rfaTuK0mPvSw1hh6CkUwgPYb2kksghToJxCaMVmN7yT7EzqjYCP2FXpu/1l/TrhxREmdgl495fDPzTV4aTZymf3rJzrPnL17uvuq/fvP23fu9/YML8lVQOFHe+HCVA6HRDies2eBVGRBsbvAyv/3Wxi9/YiDt3Q9elzizsHB6rhVw65oiw83eIB2lG5NPQdaBgejs/Ga/p6aFV5VFx8oA0XWWljyrIbBWBpv+tCIsQd3CAq8jdGCRZvVm2EYOo6eQcx/icyw33ocVNViitc1jpgVe0nasdf6PDR8GedUy0lZ/nn+Z1dqVFaNT9+3nlZHsZauHLHRAxWYdAaig4wZSLSGA4qjaI6ZW4uIwVCb+ytt2e3KVzTFgEZc3Cx/Ll3aMjZRDGWWJApHsJpYdi9SOGEzk6E8JWfnKxUb1GazOgINexYtRU4/Tph/Pkm0f4Sm4GI+ydJR9PxqcfO0OtCs+ik/is8jEsTgRp+JcTIQSS/FL/BZ3vb/JMDlMRvepSa+r+SAeWXL8DwVMxK0=</latexit><latexit sha1_base64="dnn+0zb/dBVyW3r8zSyldrgTiFg=">AAACfXicZVFLa9tAEF4rfaTuK0mPvSw1hh6CkUwgPYb2kksghToJxCaMVmN7yT7EzqjYCP2FXpu/1l/TrhxREmdgl495fDPzTV4aTZymf3rJzrPnL17uvuq/fvP23fu9/YML8lVQOFHe+HCVA6HRDies2eBVGRBsbvAyv/3Wxi9/YiDt3Q9elzizsHB6rhVw65oiw83eIB2lG5NPQdaBgejs/Ga/p6aFV5VFx8oA0XWWljyrIbBWBpv+tCIsQd3CAq8jdGCRZvVm2EYOo6eQcx/icyw33ocVNViitc1jpgVe0nasdf6PDR8GedUy0lZ/nn+Z1dqVFaNT9+3nlZHsZauHLHRAxWYdAaig4wZSLSGA4qjaI6ZW4uIwVCb+ytt2e3KVzTFgEZc3Cx/Ll3aMjZRDGWWJApHsJpYdi9SOGEzk6E8JWfnKxUb1GazOgINexYtRU4/Tph/Pkm0f4Sm4GI+ydJR9PxqcfO0OtCs+ik/is8jEsTgRp+JcTIQSS/FL/BZ3vb/JMDlMRvepSa+r+SAeWXL8DwVMxK0=</latexit>

�(xi, yi) = xiyi<latexit sha1_base64="GPLQqt4Rn/5/6KkEgkborzGLo/E=">AAACknicZVHbahsxEJW3l6TuzWkLfeiLqDGkEMyuKSQUCmnz0ocGUqiTQGwWrXZsi+iySLPBZtmvyWv6Qf2bzjpLSZwBicM5mhnNmazQKmAc/+1Ejx4/ebq1/az7/MXLV697O29Ogyu9hLF02vnzTATQysIYFWo4LzwIk2k4yy6PGv3sCnxQzv7GVQFTI+ZWzZQUSFTaez8pFmp3maq9Vao+8a+cEyaY9vrxMF4HfwiSFvRZGyfpTkdOcidLAxalFiFcJHGB00p4VFJD3Z2UAQohL8UcLghaYSBMq/UENR8Qk/OZ83Qs8jV7N6MSJoSVyeilEbgIm1pD/tcGd0VcNhXDRn+cHUwrZYsSwcrb9rNSc3S8MYnnyoNEvSIgpFc0AZcL4YVEsvJepcb3fM+Xmm7pTDN9sKXJwENOw+u5o/SFGUHN+YCTLWRQ4O2PeVuFKxtQaKrRnQRA6UpLjapjsTwW6NWS1hjqahTXXVpLsrmEh+B0NEziYfLrc//we7ugbfaBfWS7LGH77JD9YCdszCSr2TW7YX+id9GX6Ft0dPs06rQ5b9m9iH7+A1QBysA=</latexit><latexit sha1_base64="GPLQqt4Rn/5/6KkEgkborzGLo/E=">AAACknicZVHbahsxEJW3l6TuzWkLfeiLqDGkEMyuKSQUCmnz0ocGUqiTQGwWrXZsi+iySLPBZtmvyWv6Qf2bzjpLSZwBicM5mhnNmazQKmAc/+1Ejx4/ebq1/az7/MXLV697O29Ogyu9hLF02vnzTATQysIYFWo4LzwIk2k4yy6PGv3sCnxQzv7GVQFTI+ZWzZQUSFTaez8pFmp3maq9Vao+8a+cEyaY9vrxMF4HfwiSFvRZGyfpTkdOcidLAxalFiFcJHGB00p4VFJD3Z2UAQohL8UcLghaYSBMq/UENR8Qk/OZ83Qs8jV7N6MSJoSVyeilEbgIm1pD/tcGd0VcNhXDRn+cHUwrZYsSwcrb9rNSc3S8MYnnyoNEvSIgpFc0AZcL4YVEsvJepcb3fM+Xmm7pTDN9sKXJwENOw+u5o/SFGUHN+YCTLWRQ4O2PeVuFKxtQaKrRnQRA6UpLjapjsTwW6NWS1hjqahTXXVpLsrmEh+B0NEziYfLrc//we7ugbfaBfWS7LGH77JD9YCdszCSr2TW7YX+id9GX6Ft0dPs06rQ5b9m9iH7+A1QBysA=</latexit><latexit sha1_base64="GPLQqt4Rn/5/6KkEgkborzGLo/E=">AAACknicZVHbahsxEJW3l6TuzWkLfeiLqDGkEMyuKSQUCmnz0ocGUqiTQGwWrXZsi+iySLPBZtmvyWv6Qf2bzjpLSZwBicM5mhnNmazQKmAc/+1Ejx4/ebq1/az7/MXLV697O29Ogyu9hLF02vnzTATQysIYFWo4LzwIk2k4yy6PGv3sCnxQzv7GVQFTI+ZWzZQUSFTaez8pFmp3maq9Vao+8a+cEyaY9vrxMF4HfwiSFvRZGyfpTkdOcidLAxalFiFcJHGB00p4VFJD3Z2UAQohL8UcLghaYSBMq/UENR8Qk/OZ83Qs8jV7N6MSJoSVyeilEbgIm1pD/tcGd0VcNhXDRn+cHUwrZYsSwcrb9rNSc3S8MYnnyoNEvSIgpFc0AZcL4YVEsvJepcb3fM+Xmm7pTDN9sKXJwENOw+u5o/SFGUHN+YCTLWRQ4O2PeVuFKxtQaKrRnQRA6UpLjapjsTwW6NWS1hjqahTXXVpLsrmEh+B0NEziYfLrc//we7ugbfaBfWS7LGH77JD9YCdszCSr2TW7YX+id9GX6Ft0dPs06rQ5b9m9iH7+A1QBysA=</latexit><latexit sha1_base64="GPLQqt4Rn/5/6KkEgkborzGLo/E=">AAACknicZVHbahsxEJW3l6TuzWkLfeiLqDGkEMyuKSQUCmnz0ocGUqiTQGwWrXZsi+iySLPBZtmvyWv6Qf2bzjpLSZwBicM5mhnNmazQKmAc/+1Ejx4/ebq1/az7/MXLV697O29Ogyu9hLF02vnzTATQysIYFWo4LzwIk2k4yy6PGv3sCnxQzv7GVQFTI+ZWzZQUSFTaez8pFmp3maq9Vao+8a+cEyaY9vrxMF4HfwiSFvRZGyfpTkdOcidLAxalFiFcJHGB00p4VFJD3Z2UAQohL8UcLghaYSBMq/UENR8Qk/OZ83Qs8jV7N6MSJoSVyeilEbgIm1pD/tcGd0VcNhXDRn+cHUwrZYsSwcrb9rNSc3S8MYnnyoNEvSIgpFc0AZcL4YVEsvJepcb3fM+Xmm7pTDN9sKXJwENOw+u5o/SFGUHN+YCTLWRQ4O2PeVuFKxtQaKrRnQRA6UpLjapjsTwW6NWS1hjqahTXXVpLsrmEh+B0NEziYfLrc//we7ugbfaBfWS7LGH77JD9YCdszCSr2TW7YX+id9GX6Ft0dPs06rQ5b9m9iH7+A1QBysA=</latexit>

log (xi, yi) = ⌘xiyi<latexit sha1_base64="RRO0FK7ILH6GVFMPyMIbSquC0kQ=">AAACnHicZVFbaxNBFJ6stxovTfVRkNEQqFDCbhDaF6GoD4IEKpi00A3L7OQkGTqXZeasJCz77K/xVX+L/8az6SJtemCGb75zm3O+vNAqYBz/7UT37j94+GjvcffJ02fP93sHL6bBlV7CRDrt/EUuAmhlYYIKNVwUHoTJNZznV58a//kP8EE5+x03BcyMWFq1UFIgUVnvTardkqdFUIfrTB1tMvWOf+A8BRScCHpnvX48jLfG74KkBX3W2ll20JHp3MnSgEWpRQiXSVzgrBIeldRQd9MyQCHklVjCJUErDIRZtZ2l5gNi5nzhPB2LfMvezKiECWFjcoo0Aldh19eQ/32Dm05cNxXDTn9cnMwqZYsSwcrr9otSc3S8WRefKw8S9YaAkF7RBFyuhBcSaam3KjUKzI98qemWzjTTB1uaHDzMaXi9dJS+MiOoOR9wWgstKPD2x7ytwpUNKDTV6KYBULrSUqNqLNZjgV6tSdBQV6O47pIsya4Id8F0NEziYfLtff/0YyvQHnvF3rJDlrBjdsq+sDM2YZL9ZL/Yb/Yneh19jr5G4+vQqNPmvGS3LJr+A3duzqM=</latexit><latexit sha1_base64="RRO0FK7ILH6GVFMPyMIbSquC0kQ=">AAACnHicZVFbaxNBFJ6stxovTfVRkNEQqFDCbhDaF6GoD4IEKpi00A3L7OQkGTqXZeasJCz77K/xVX+L/8az6SJtemCGb75zm3O+vNAqYBz/7UT37j94+GjvcffJ02fP93sHL6bBlV7CRDrt/EUuAmhlYYIKNVwUHoTJNZznV58a//kP8EE5+x03BcyMWFq1UFIgUVnvTardkqdFUIfrTB1tMvWOf+A8BRScCHpnvX48jLfG74KkBX3W2ll20JHp3MnSgEWpRQiXSVzgrBIeldRQd9MyQCHklVjCJUErDIRZtZ2l5gNi5nzhPB2LfMvezKiECWFjcoo0Aldh19eQ/32Dm05cNxXDTn9cnMwqZYsSwcrr9otSc3S8WRefKw8S9YaAkF7RBFyuhBcSaam3KjUKzI98qemWzjTTB1uaHDzMaXi9dJS+MiOoOR9wWgstKPD2x7ytwpUNKDTV6KYBULrSUqNqLNZjgV6tSdBQV6O47pIsya4Id8F0NEziYfLtff/0YyvQHnvF3rJDlrBjdsq+sDM2YZL9ZL/Yb/Yneh19jr5G4+vQqNPmvGS3LJr+A3duzqM=</latexit><latexit sha1_base64="RRO0FK7ILH6GVFMPyMIbSquC0kQ=">AAACnHicZVFbaxNBFJ6stxovTfVRkNEQqFDCbhDaF6GoD4IEKpi00A3L7OQkGTqXZeasJCz77K/xVX+L/8az6SJtemCGb75zm3O+vNAqYBz/7UT37j94+GjvcffJ02fP93sHL6bBlV7CRDrt/EUuAmhlYYIKNVwUHoTJNZznV58a//kP8EE5+x03BcyMWFq1UFIgUVnvTardkqdFUIfrTB1tMvWOf+A8BRScCHpnvX48jLfG74KkBX3W2ll20JHp3MnSgEWpRQiXSVzgrBIeldRQd9MyQCHklVjCJUErDIRZtZ2l5gNi5nzhPB2LfMvezKiECWFjcoo0Aldh19eQ/32Dm05cNxXDTn9cnMwqZYsSwcrr9otSc3S8WRefKw8S9YaAkF7RBFyuhBcSaam3KjUKzI98qemWzjTTB1uaHDzMaXi9dJS+MiOoOR9wWgstKPD2x7ytwpUNKDTV6KYBULrSUqNqLNZjgV6tSdBQV6O47pIsya4Id8F0NEziYfLtff/0YyvQHnvF3rJDlrBjdsq+sDM2YZL9ZL/Yb/Yneh19jr5G4+vQqNPmvGS3LJr+A3duzqM=</latexit><latexit sha1_base64="RRO0FK7ILH6GVFMPyMIbSquC0kQ=">AAACnHicZVFbaxNBFJ6stxovTfVRkNEQqFDCbhDaF6GoD4IEKpi00A3L7OQkGTqXZeasJCz77K/xVX+L/8az6SJtemCGb75zm3O+vNAqYBz/7UT37j94+GjvcffJ02fP93sHL6bBlV7CRDrt/EUuAmhlYYIKNVwUHoTJNZznV58a//kP8EE5+x03BcyMWFq1UFIgUVnvTardkqdFUIfrTB1tMvWOf+A8BRScCHpnvX48jLfG74KkBX3W2ll20JHp3MnSgEWpRQiXSVzgrBIeldRQd9MyQCHklVjCJUErDIRZtZ2l5gNi5nzhPB2LfMvezKiECWFjcoo0Aldh19eQ/32Dm05cNxXDTn9cnMwqZYsSwcrr9otSc3S8WRefKw8S9YaAkF7RBFyuhBcSaam3KjUKzI98qemWzjTTB1uaHDzMaXi9dJS+MiOoOR9wWgstKPD2x7ytwpUNKDTV6KYBULrSUqNqLNZjgV6tSdBQV6O47pIsya4Id8F0NEziYfLtff/0YyvQHnvF3rJDlrBjdsq+sDM2YZL9ZL/Yb/Yneh19jr5G4+vQqNPmvGS3LJr+A3duzqM=</latexit>

log (xi, yi) = ⌘xiyi<latexit sha1_base64="RRO0FK7ILH6GVFMPyMIbSquC0kQ=">AAACnHicZVFbaxNBFJ6stxovTfVRkNEQqFDCbhDaF6GoD4IEKpi00A3L7OQkGTqXZeasJCz77K/xVX+L/8az6SJtemCGb75zm3O+vNAqYBz/7UT37j94+GjvcffJ02fP93sHL6bBlV7CRDrt/EUuAmhlYYIKNVwUHoTJNZznV58a//kP8EE5+x03BcyMWFq1UFIgUVnvTardkqdFUIfrTB1tMvWOf+A8BRScCHpnvX48jLfG74KkBX3W2ll20JHp3MnSgEWpRQiXSVzgrBIeldRQd9MyQCHklVjCJUErDIRZtZ2l5gNi5nzhPB2LfMvezKiECWFjcoo0Aldh19eQ/32Dm05cNxXDTn9cnMwqZYsSwcrr9otSc3S8WRefKw8S9YaAkF7RBFyuhBcSaam3KjUKzI98qemWzjTTB1uaHDzMaXi9dJS+MiOoOR9wWgstKPD2x7ytwpUNKDTV6KYBULrSUqNqLNZjgV6tSdBQV6O47pIsya4Id8F0NEziYfLtff/0YyvQHnvF3rJDlrBjdsq+sDM2YZL9ZL/Yb/Yneh19jr5G4+vQqNPmvGS3LJr+A3duzqM=</latexit><latexit sha1_base64="RRO0FK7ILH6GVFMPyMIbSquC0kQ=">AAACnHicZVFbaxNBFJ6stxovTfVRkNEQqFDCbhDaF6GoD4IEKpi00A3L7OQkGTqXZeasJCz77K/xVX+L/8az6SJtemCGb75zm3O+vNAqYBz/7UT37j94+GjvcffJ02fP93sHL6bBlV7CRDrt/EUuAmhlYYIKNVwUHoTJNZznV58a//kP8EE5+x03BcyMWFq1UFIgUVnvTardkqdFUIfrTB1tMvWOf+A8BRScCHpnvX48jLfG74KkBX3W2ll20JHp3MnSgEWpRQiXSVzgrBIeldRQd9MyQCHklVjCJUErDIRZtZ2l5gNi5nzhPB2LfMvezKiECWFjcoo0Aldh19eQ/32Dm05cNxXDTn9cnMwqZYsSwcrr9otSc3S8WRefKw8S9YaAkF7RBFyuhBcSaam3KjUKzI98qemWzjTTB1uaHDzMaXi9dJS+MiOoOR9wWgstKPD2x7ytwpUNKDTV6KYBULrSUqNqLNZjgV6tSdBQV6O47pIsya4Id8F0NEziYfLtff/0YyvQHnvF3rJDlrBjdsq+sDM2YZL9ZL/Yb/Yneh19jr5G4+vQqNPmvGS3LJr+A3duzqM=</latexit><latexit sha1_base64="RRO0FK7ILH6GVFMPyMIbSquC0kQ=">AAACnHicZVFbaxNBFJ6stxovTfVRkNEQqFDCbhDaF6GoD4IEKpi00A3L7OQkGTqXZeasJCz77K/xVX+L/8az6SJtemCGb75zm3O+vNAqYBz/7UT37j94+GjvcffJ02fP93sHL6bBlV7CRDrt/EUuAmhlYYIKNVwUHoTJNZznV58a//kP8EE5+x03BcyMWFq1UFIgUVnvTardkqdFUIfrTB1tMvWOf+A8BRScCHpnvX48jLfG74KkBX3W2ll20JHp3MnSgEWpRQiXSVzgrBIeldRQd9MyQCHklVjCJUErDIRZtZ2l5gNi5nzhPB2LfMvezKiECWFjcoo0Aldh19eQ/32Dm05cNxXDTn9cnMwqZYsSwcrr9otSc3S8WRefKw8S9YaAkF7RBFyuhBcSaam3KjUKzI98qemWzjTTB1uaHDzMaXi9dJS+MiOoOR9wWgstKPD2x7ytwpUNKDTV6KYBULrSUqNqLNZjgV6tSdBQV6O47pIsya4Id8F0NEziYfLtff/0YyvQHnvF3rJDlrBjdsq+sDM2YZL9ZL/Yb/Yneh19jr5G4+vQqNPmvGS3LJr+A3duzqM=</latexit><latexit sha1_base64="RRO0FK7ILH6GVFMPyMIbSquC0kQ=">AAACnHicZVFbaxNBFJ6stxovTfVRkNEQqFDCbhDaF6GoD4IEKpi00A3L7OQkGTqXZeasJCz77K/xVX+L/8az6SJtemCGb75zm3O+vNAqYBz/7UT37j94+GjvcffJ02fP93sHL6bBlV7CRDrt/EUuAmhlYYIKNVwUHoTJNZznV58a//kP8EE5+x03BcyMWFq1UFIgUVnvTardkqdFUIfrTB1tMvWOf+A8BRScCHpnvX48jLfG74KkBX3W2ll20JHp3MnSgEWpRQiXSVzgrBIeldRQd9MyQCHklVjCJUErDIRZtZ2l5gNi5nzhPB2LfMvezKiECWFjcoo0Aldh19eQ/32Dm05cNxXDTn9cnMwqZYsSwcrr9otSc3S8WRefKw8S9YaAkF7RBFyuhBcSaam3KjUKzI98qemWzjTTB1uaHDzMaXi9dJS+MiOoOR9wWgstKPD2x7ytwpUNKDTV6KYBULrSUqNqLNZjgV6tSdBQV6O47pIsya4Id8F0NEziYfLtff/0YyvQHnvF3rJDlrBjdsq+sDM2YZL9ZL/Yb/Yneh19jr5G4+vQqNPmvGS3LJr+A3duzqM=</latexit>

“Ising model"


!18

With this, we can compute the joint:

and its log:



xj<latexit sha1_base64="/uiTXGk2yRxVSM8cHktxwgEDOwo=">AAACfHicZVHbattAEF2rSZu6aXN77MtSx1BIaiQTmjyG9qUvgZTWSSA2ZrQa25vsReyOgo3QJ/S1/bb8TOnKESVxBnY5zOXMzJk0V9JTHN+3ohdr6y9fbbxuv9l8+25re2f3wtvCCRwIq6y7SsGjkgYHJEnhVe4QdKrwMr39Wscv79B5ac1PWuQ40jA1ciIFUHD9mI9vxtuduBcvjT8HSQM6rLHz8U5LDDMrCo2GhALvr5M4p1EJjqRQWLWHhcccxC1M8TpAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NDkZldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4ezQFSr8wup6e28KnaLDLCyvpjaUz3QfK867PMgSBPK8mZg3LFwaT6ACR3vokYQtTGhUnsH8DMjJeTiYr8p+XLXDWZLVIzwHF/1eEveS70ed0y/NgTbYe/aBfWQJO2an7Bs7ZwMm2JT9Yr/Zn9bfaD86iD49pEatpmaPPbHo8z8tcsRO</latexit><latexit sha1_base64="/uiTXGk2yRxVSM8cHktxwgEDOwo=">AAACfHicZVHbattAEF2rSZu6aXN77MtSx1BIaiQTmjyG9qUvgZTWSSA2ZrQa25vsReyOgo3QJ/S1/bb8TOnKESVxBnY5zOXMzJk0V9JTHN+3ohdr6y9fbbxuv9l8+25re2f3wtvCCRwIq6y7SsGjkgYHJEnhVe4QdKrwMr39Wscv79B5ac1PWuQ40jA1ciIFUHD9mI9vxtuduBcvjT8HSQM6rLHz8U5LDDMrCo2GhALvr5M4p1EJjqRQWLWHhcccxC1M8TpAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NDkZldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4ezQFSr8wup6e28KnaLDLCyvpjaUz3QfK867PMgSBPK8mZg3LFwaT6ACR3vokYQtTGhUnsH8DMjJeTiYr8p+XLXDWZLVIzwHF/1eEveS70ed0y/NgTbYe/aBfWQJO2an7Bs7ZwMm2JT9Yr/Zn9bfaD86iD49pEatpmaPPbHo8z8tcsRO</latexit><latexit sha1_base64="/uiTXGk2yRxVSM8cHktxwgEDOwo=">AAACfHicZVHbattAEF2rSZu6aXN77MtSx1BIaiQTmjyG9qUvgZTWSSA2ZrQa25vsReyOgo3QJ/S1/bb8TOnKESVxBnY5zOXMzJk0V9JTHN+3ohdr6y9fbbxuv9l8+25re2f3wtvCCRwIq6y7SsGjkgYHJEnhVe4QdKrwMr39Wscv79B5ac1PWuQ40jA1ciIFUHD9mI9vxtuduBcvjT8HSQM6rLHz8U5LDDMrCo2GhALvr5M4p1EJjqRQWLWHhcccxC1M8TpAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NDkZldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4ezQFSr8wup6e28KnaLDLCyvpjaUz3QfK867PMgSBPK8mZg3LFwaT6ACR3vokYQtTGhUnsH8DMjJeTiYr8p+XLXDWZLVIzwHF/1eEveS70ed0y/NgTbYe/aBfWQJO2an7Bs7ZwMm2JT9Yr/Zn9bfaD86iD49pEatpmaPPbHo8z8tcsRO</latexit><latexit sha1_base64="/uiTXGk2yRxVSM8cHktxwgEDOwo=">AAACfHicZVHbattAEF2rSZu6aXN77MtSx1BIaiQTmjyG9qUvgZTWSSA2ZrQa25vsReyOgo3QJ/S1/bb8TOnKESVxBnY5zOXMzJk0V9JTHN+3ohdr6y9fbbxuv9l8+25re2f3wtvCCRwIq6y7SsGjkgYHJEnhVe4QdKrwMr39Wscv79B5ac1PWuQ40jA1ciIFUHD9mI9vxtuduBcvjT8HSQM6rLHz8U5LDDMrCo2GhALvr5M4p1EJjqRQWLWHhcccxC1M8TpAAxr9qFzOWvFu8GR8Yl14hvjS+7iiBO39QqchUwPN/Gqsdv6PdR8HaV4z+pX+NDkZldLkBaERD+0nheJkeS0Hz6RDQWoRAAgnwwZczMCBoCDaE6Za4ezQFSr8wup6e28KnaLDLCyvpjaUz3QfK867PMgSBPK8mZg3LFwaT6ACR3vokYQtTGhUnsH8DMjJeTiYr8p+XLXDWZLVIzwHF/1eEveS70ed0y/NgTbYe/aBfWQJO2an7Bs7ZwMm2JT9Yr/Zn9bfaD86iD49pEatpmaPPbHo8z8tcsRO</latexit>



p(x,y | ⌘,�) = 1

Z

Y

i

exp(⌘xiyi)Y

i,j

exp(�xixj)

<latexit sha1_base64="A2q+UUgCJ+0lbdXj7CIGZFHw/uA=">AAAC73icZVJLa9tAEF6pr8R9Oe2xl6XGEIMxkim0l0JoL70EUqiT0MiI1WrlbLIPdXdUZIR+R2+l1/6k9td05KghcQa0+phvHjvzbVYq6SGK/gThvfsPHj7a2R08fvL02fPh3otjbyvHxYJbZd1pxrxQ0ogFSFDitHSC6UyJk+zyY8effBfOS2u+wLoUS81WRhaSM0BXOoRyP9EMzrOiqdvpf7huaaJlThMBbEqTDH8T+p4mhWO8idvmK/Kls3nayJZiVI1VMIbWqVyncnJNTi/ans16uk4vJjQdjqJZtDF6F8Q9GJHejtK9gCe55ZUWBrhi3p/FUQnLhjmQXIl2kFRelIxfspU4Q2iYFn7ZbNbT0jF6clpYh58BuvHezGiY9n6tM4zs5vfbXOe85sY3Sai7in6rPxTvlo00ZQXC8Kv2RaUoWNopQHPpBAe1RsC4kzgB5ecMNwuo061Knaj51FUKT251N703lc6EEzkOr1YW08/1XKAIY4prwQV52t+Y9lWoNB6YwhqDxAvgtjLYqDlk9SEDJ2t8I75t5lE7QFnibRHuguP5LI5m8ec3o4MPvUA75BV5TfZJTN6SA/KJHJEF4eRvQILdYBB+C3+EP8NfV6Fh0Oe8JLcs/P0P45Ptcg==</latexit><latexit sha1_base64="A2q+UUgCJ+0lbdXj7CIGZFHw/uA=">AAAC73icZVJLa9tAEF6pr8R9Oe2xl6XGEIMxkim0l0JoL70EUqiT0MiI1WrlbLIPdXdUZIR+R2+l1/6k9td05KghcQa0+phvHjvzbVYq6SGK/gThvfsPHj7a2R08fvL02fPh3otjbyvHxYJbZd1pxrxQ0ogFSFDitHSC6UyJk+zyY8effBfOS2u+wLoUS81WRhaSM0BXOoRyP9EMzrOiqdvpf7huaaJlThMBbEqTDH8T+p4mhWO8idvmK/Kls3nayJZiVI1VMIbWqVyncnJNTi/ans16uk4vJjQdjqJZtDF6F8Q9GJHejtK9gCe55ZUWBrhi3p/FUQnLhjmQXIl2kFRelIxfspU4Q2iYFn7ZbNbT0jF6clpYh58BuvHezGiY9n6tM4zs5vfbXOe85sY3Sai7in6rPxTvlo00ZQXC8Kv2RaUoWNopQHPpBAe1RsC4kzgB5ecMNwuo061Knaj51FUKT251N703lc6EEzkOr1YW08/1XKAIY4prwQV52t+Y9lWoNB6YwhqDxAvgtjLYqDlk9SEDJ2t8I75t5lE7QFnibRHuguP5LI5m8ec3o4MPvUA75BV5TfZJTN6SA/KJHJEF4eRvQILdYBB+C3+EP8NfV6Fh0Oe8JLcs/P0P45Ptcg==</latexit><latexit sha1_base64="A2q+UUgCJ+0lbdXj7CIGZFHw/uA=">AAAC73icZVJLa9tAEF6pr8R9Oe2xl6XGEIMxkim0l0JoL70EUqiT0MiI1WrlbLIPdXdUZIR+R2+l1/6k9td05KghcQa0+phvHjvzbVYq6SGK/gThvfsPHj7a2R08fvL02fPh3otjbyvHxYJbZd1pxrxQ0ogFSFDitHSC6UyJk+zyY8effBfOS2u+wLoUS81WRhaSM0BXOoRyP9EMzrOiqdvpf7huaaJlThMBbEqTDH8T+p4mhWO8idvmK/Kls3nayJZiVI1VMIbWqVyncnJNTi/ans16uk4vJjQdjqJZtDF6F8Q9GJHejtK9gCe55ZUWBrhi3p/FUQnLhjmQXIl2kFRelIxfspU4Q2iYFn7ZbNbT0jF6clpYh58BuvHezGiY9n6tM4zs5vfbXOe85sY3Sai7in6rPxTvlo00ZQXC8Kv2RaUoWNopQHPpBAe1RsC4kzgB5ecMNwuo061Knaj51FUKT251N703lc6EEzkOr1YW08/1XKAIY4prwQV52t+Y9lWoNB6YwhqDxAvgtjLYqDlk9SEDJ2t8I75t5lE7QFnibRHuguP5LI5m8ec3o4MPvUA75BV5TfZJTN6SA/KJHJEF4eRvQILdYBB+C3+EP8NfV6Fh0Oe8JLcs/P0P45Ptcg==</latexit><latexit sha1_base64="A2q+UUgCJ+0lbdXj7CIGZFHw/uA=">AAAC73icZVJLa9tAEF6pr8R9Oe2xl6XGEIMxkim0l0JoL70EUqiT0MiI1WrlbLIPdXdUZIR+R2+l1/6k9td05KghcQa0+phvHjvzbVYq6SGK/gThvfsPHj7a2R08fvL02fPh3otjbyvHxYJbZd1pxrxQ0ogFSFDitHSC6UyJk+zyY8effBfOS2u+wLoUS81WRhaSM0BXOoRyP9EMzrOiqdvpf7huaaJlThMBbEqTDH8T+p4mhWO8idvmK/Kls3nayJZiVI1VMIbWqVyncnJNTi/ans16uk4vJjQdjqJZtDF6F8Q9GJHejtK9gCe55ZUWBrhi3p/FUQnLhjmQXIl2kFRelIxfspU4Q2iYFn7ZbNbT0jF6clpYh58BuvHezGiY9n6tM4zs5vfbXOe85sY3Sai7in6rPxTvlo00ZQXC8Kv2RaUoWNopQHPpBAe1RsC4kzgB5ecMNwuo061Knaj51FUKT251N703lc6EEzkOr1YW08/1XKAIY4prwQV52t+Y9lWoNB6YwhqDxAvgtjLYqDlk9SEDJ2t8I75t5lE7QFnibRHuguP5LI5m8ec3o4MPvUA75BV5TfZJTN6SA/KJHJEF4eRvQILdYBB+C3+EP8NfV6Fh0Oe8JLcs/P0P45Ptcg==</latexit>

log p(x,y | ⌘,�) = ⌘X

i

xiyi + �X

i,j

xixj � log(Z)<latexit sha1_base64="x8vhrlNeaXVkx9xAOXPlE9E93ZY=">AAAC5nicZVLLattAFB2pr9R9xGmX3VxqDA51jWQK7aYQ2k02gRTqJDQyYjQa2RPPQ8yMioTQL3RXus0vZZkv6bYjWymJc2Gko3PuY+69SnLOjA2Ca89/8PDR4yc7T3vPnr94udvfe3ViVKEJnRHFlT5LsKGcSTqzzHJ6lmuKRcLpabL62uqnP6k2TMnvtsrpXOCFZBkj2Doq7q8irhaQjyKB7TLJ6rIZ38CqgUiwFCJq8RiixL324TOsvyEyhYhr1kAZsypm8G7jcMOPLxpopTK+gPfQ1hj92I/7g2ASrA3ug7ADA9TZcbznkShVpBBUWsKxMedhkNt5jbVlhNOmFxWG5pis8IKeOyixoGZer6fSwNAxKWRKuyMtrNnbETUWxlQicZ5tx2Zba8n/2vC2aMs2o9mqb7NP85rJvLBUkk35rOBgFbSDh5RpSiyvHMBEM9cBkCXWmFi3njuZ2l2mY11w9yRKtN0bWYiEapq65vlCufClmFI34yG4sbgBGehuDF0WYNJYzF2OXmSoJaqQrlB9hMsjbDUr3a9hmnoaND23lnB7CffByXQSBpPw24fBwZduQTvoDXqLRihEH9EBOkTHaIYIukJ/PeR5/tL/5f/2/2xcfa+LeY3umH/5D66H6Y8=</latexit><latexit sha1_base64="x8vhrlNeaXVkx9xAOXPlE9E93ZY=">AAAC5nicZVLLattAFB2pr9R9xGmX3VxqDA51jWQK7aYQ2k02gRTqJDQyYjQa2RPPQ8yMioTQL3RXus0vZZkv6bYjWymJc2Gko3PuY+69SnLOjA2Ca89/8PDR4yc7T3vPnr94udvfe3ViVKEJnRHFlT5LsKGcSTqzzHJ6lmuKRcLpabL62uqnP6k2TMnvtsrpXOCFZBkj2Doq7q8irhaQjyKB7TLJ6rIZ38CqgUiwFCJq8RiixL324TOsvyEyhYhr1kAZsypm8G7jcMOPLxpopTK+gPfQ1hj92I/7g2ASrA3ug7ADA9TZcbznkShVpBBUWsKxMedhkNt5jbVlhNOmFxWG5pis8IKeOyixoGZer6fSwNAxKWRKuyMtrNnbETUWxlQicZ5tx2Zba8n/2vC2aMs2o9mqb7NP85rJvLBUkk35rOBgFbSDh5RpSiyvHMBEM9cBkCXWmFi3njuZ2l2mY11w9yRKtN0bWYiEapq65vlCufClmFI34yG4sbgBGehuDF0WYNJYzF2OXmSoJaqQrlB9hMsjbDUr3a9hmnoaND23lnB7CffByXQSBpPw24fBwZduQTvoDXqLRihEH9EBOkTHaIYIukJ/PeR5/tL/5f/2/2xcfa+LeY3umH/5D66H6Y8=</latexit><latexit sha1_base64="x8vhrlNeaXVkx9xAOXPlE9E93ZY=">AAAC5nicZVLLattAFB2pr9R9xGmX3VxqDA51jWQK7aYQ2k02gRTqJDQyYjQa2RPPQ8yMioTQL3RXus0vZZkv6bYjWymJc2Gko3PuY+69SnLOjA2Ca89/8PDR4yc7T3vPnr94udvfe3ViVKEJnRHFlT5LsKGcSTqzzHJ6lmuKRcLpabL62uqnP6k2TMnvtsrpXOCFZBkj2Doq7q8irhaQjyKB7TLJ6rIZ38CqgUiwFCJq8RiixL324TOsvyEyhYhr1kAZsypm8G7jcMOPLxpopTK+gPfQ1hj92I/7g2ASrA3ug7ADA9TZcbznkShVpBBUWsKxMedhkNt5jbVlhNOmFxWG5pis8IKeOyixoGZer6fSwNAxKWRKuyMtrNnbETUWxlQicZ5tx2Zba8n/2vC2aMs2o9mqb7NP85rJvLBUkk35rOBgFbSDh5RpSiyvHMBEM9cBkCXWmFi3njuZ2l2mY11w9yRKtN0bWYiEapq65vlCufClmFI34yG4sbgBGehuDF0WYNJYzF2OXmSoJaqQrlB9hMsjbDUr3a9hmnoaND23lnB7CffByXQSBpPw24fBwZduQTvoDXqLRihEH9EBOkTHaIYIukJ/PeR5/tL/5f/2/2xcfa+LeY3umH/5D66H6Y8=</latexit><latexit sha1_base64="x8vhrlNeaXVkx9xAOXPlE9E93ZY=">AAAC5nicZVLLattAFB2pr9R9xGmX3VxqDA51jWQK7aYQ2k02gRTqJDQyYjQa2RPPQ8yMioTQL3RXus0vZZkv6bYjWymJc2Gko3PuY+69SnLOjA2Ca89/8PDR4yc7T3vPnr94udvfe3ViVKEJnRHFlT5LsKGcSTqzzHJ6lmuKRcLpabL62uqnP6k2TMnvtsrpXOCFZBkj2Doq7q8irhaQjyKB7TLJ6rIZ38CqgUiwFCJq8RiixL324TOsvyEyhYhr1kAZsypm8G7jcMOPLxpopTK+gPfQ1hj92I/7g2ASrA3ug7ADA9TZcbznkShVpBBUWsKxMedhkNt5jbVlhNOmFxWG5pis8IKeOyixoGZer6fSwNAxKWRKuyMtrNnbETUWxlQicZ5tx2Zba8n/2vC2aMs2o9mqb7NP85rJvLBUkk35rOBgFbSDh5RpSiyvHMBEM9cBkCXWmFi3njuZ2l2mY11w9yRKtN0bWYiEapq65vlCufClmFI34yG4sbgBGehuDF0WYNJYzF2OXmSoJaqQrlB9hMsjbDUr3a9hmnoaND23lnB7CffByXQSBpPw24fBwZduQTvoDXqLRihEH9EBOkTHaIYIukJ/PeR5/tL/5f/2/2xcfa+LeY3umH/5D66H6Y8=</latexit>


!19

Our aim now is to find the hidden states such that this log of the joint is maximal (or at least very high). Simple approach is Iterated Conditional Modes (ICM): 1. Initialize all by corresponding 2. For all nodes : 1. set to +1 and to -1 and evaluate 2. keep the value that gives higher log joint

This will keep or increase the joint in every step The nodes can be visited in order or randomly

log p(x,y | ⌘,�) = ⌘X

i

xiyi + �X

i,j

xixj � log(Z)<latexit sha1_base64="x8vhrlNeaXVkx9xAOXPlE9E93ZY=">AAAC5nicZVLLattAFB2pr9R9xGmX3VxqDA51jWQK7aYQ2k02gRTqJDQyYjQa2RPPQ8yMioTQL3RXus0vZZkv6bYjWymJc2Gko3PuY+69SnLOjA2Ca89/8PDR4yc7T3vPnr94udvfe3ViVKEJnRHFlT5LsKGcSTqzzHJ6lmuKRcLpabL62uqnP6k2TMnvtsrpXOCFZBkj2Doq7q8irhaQjyKB7TLJ6rIZ38CqgUiwFCJq8RiixL324TOsvyEyhYhr1kAZsypm8G7jcMOPLxpopTK+gPfQ1hj92I/7g2ASrA3ug7ADA9TZcbznkShVpBBUWsKxMedhkNt5jbVlhNOmFxWG5pis8IKeOyixoGZer6fSwNAxKWRKuyMtrNnbETUWxlQicZ5tx2Zba8n/2vC2aMs2o9mqb7NP85rJvLBUkk35rOBgFbSDh5RpSiyvHMBEM9cBkCXWmFi3njuZ2l2mY11w9yRKtN0bWYiEapq65vlCufClmFI34yG4sbgBGehuDF0WYNJYzF2OXmSoJaqQrlB9hMsjbDUr3a9hmnoaND23lnB7CffByXQSBpPw24fBwZduQTvoDXqLRihEH9EBOkTHaIYIukJ/PeR5/tL/5f/2/2xcfa+LeY3umH/5D66H6Y8=</latexit><latexit sha1_base64="x8vhrlNeaXVkx9xAOXPlE9E93ZY=">AAAC5nicZVLLattAFB2pr9R9xGmX3VxqDA51jWQK7aYQ2k02gRTqJDQyYjQa2RPPQ8yMioTQL3RXus0vZZkv6bYjWymJc2Gko3PuY+69SnLOjA2Ca89/8PDR4yc7T3vPnr94udvfe3ViVKEJnRHFlT5LsKGcSTqzzHJ6lmuKRcLpabL62uqnP6k2TMnvtsrpXOCFZBkj2Doq7q8irhaQjyKB7TLJ6rIZ38CqgUiwFCJq8RiixL324TOsvyEyhYhr1kAZsypm8G7jcMOPLxpopTK+gPfQ1hj92I/7g2ASrA3ug7ADA9TZcbznkShVpBBUWsKxMedhkNt5jbVlhNOmFxWG5pis8IKeOyixoGZer6fSwNAxKWRKuyMtrNnbETUWxlQicZ5tx2Zba8n/2vC2aMs2o9mqb7NP85rJvLBUkk35rOBgFbSDh5RpSiyvHMBEM9cBkCXWmFi3njuZ2l2mY11w9yRKtN0bWYiEapq65vlCufClmFI34yG4sbgBGehuDF0WYNJYzF2OXmSoJaqQrlB9hMsjbDUr3a9hmnoaND23lnB7CffByXQSBpPw24fBwZduQTvoDXqLRihEH9EBOkTHaIYIukJ/PeR5/tL/5f/2/2xcfa+LeY3umH/5D66H6Y8=</latexit><latexit sha1_base64="x8vhrlNeaXVkx9xAOXPlE9E93ZY=">AAAC5nicZVLLattAFB2pr9R9xGmX3VxqDA51jWQK7aYQ2k02gRTqJDQyYjQa2RPPQ8yMioTQL3RXus0vZZkv6bYjWymJc2Gko3PuY+69SnLOjA2Ca89/8PDR4yc7T3vPnr94udvfe3ViVKEJnRHFlT5LsKGcSTqzzHJ6lmuKRcLpabL62uqnP6k2TMnvtsrpXOCFZBkj2Doq7q8irhaQjyKB7TLJ6rIZ38CqgUiwFCJq8RiixL324TOsvyEyhYhr1kAZsypm8G7jcMOPLxpopTK+gPfQ1hj92I/7g2ASrA3ug7ADA9TZcbznkShVpBBUWsKxMedhkNt5jbVlhNOmFxWG5pis8IKeOyixoGZer6fSwNAxKWRKuyMtrNnbETUWxlQicZ5tx2Zba8n/2vC2aMs2o9mqb7NP85rJvLBUkk35rOBgFbSDh5RpSiyvHMBEM9cBkCXWmFi3njuZ2l2mY11w9yRKtN0bWYiEapq65vlCufClmFI34yG4sbgBGehuDF0WYNJYzF2OXmSoJaqQrlB9hMsjbDUr3a9hmnoaND23lnB7CffByXQSBpPw24fBwZduQTvoDXqLRihEH9EBOkTHaIYIukJ/PeR5/tL/5f/2/2xcfa+LeY3umH/5D66H6Y8=</latexit><latexit sha1_base64="x8vhrlNeaXVkx9xAOXPlE9E93ZY=">AAAC5nicZVLLattAFB2pr9R9xGmX3VxqDA51jWQK7aYQ2k02gRTqJDQyYjQa2RPPQ8yMioTQL3RXus0vZZkv6bYjWymJc2Gko3PuY+69SnLOjA2Ca89/8PDR4yc7T3vPnr94udvfe3ViVKEJnRHFlT5LsKGcSTqzzHJ6lmuKRcLpabL62uqnP6k2TMnvtsrpXOCFZBkj2Doq7q8irhaQjyKB7TLJ6rIZ38CqgUiwFCJq8RiixL324TOsvyEyhYhr1kAZsypm8G7jcMOPLxpopTK+gPfQ1hj92I/7g2ASrA3ug7ADA9TZcbznkShVpBBUWsKxMedhkNt5jbVlhNOmFxWG5pis8IKeOyixoGZer6fSwNAxKWRKuyMtrNnbETUWxlQicZ5tx2Zba8n/2vC2aMs2o9mqb7NP85rJvLBUkk35rOBgFbSDh5RpSiyvHMBEM9cBkCXWmFi3njuZ2l2mY11w9yRKtN0bWYiEapq65vlCufClmFI34yG4sbgBGehuDF0WYNJYzF2OXmSoJaqQrlB9hMsjbDUr3a9hmnoaND23lnB7CffByXQSBpPw24fBwZduQTvoDXqLRihEH9EBOkTHaIYIukJ/PeR5/tL/5f/2/2xcfa+LeY3umH/5D66H6Y8=</latexit>








log p(x,y | ⌘,�)<latexit sha1_base64="SLhgcGOrXpNfQnZ5byfyZF58J3o=">AAACqnicZVHbbtNAEN2YWwm3FB55GRFFKlIU7AgJHit44aVSQaQtiqNovR4nq+7F2h2jRJY/g6/hFT6Cv2GdGtSmI+3u0ZnbzpysVNJTHP/pRXfu3rv/4OBh/9HjJ0+fDQ6fn3lbOYEzYZV1Fxn3qKTBGUlSeFE65DpTeJ5dfmz959/ReWnNV9qWuNB8ZWQhBadALQdvUmVXUB6lmtM6K+pNM/4Htw2kWuaQIvExpFl4XsNyMIwn8c7gNkg6MGSdnS4PeyLNrag0GhKKez9P4pIWNXckhcKmn1YeSy4u+QrnARqu0S/q3WQNjAKTQ2FdOIZgx17PqLn2fquzENn+2u/7WvK/b3TdSZu2ot/rT8X7RS1NWREacdW+qBSQhXZ5kEuHgtQ2AC6cDBOAWHPHBYUV36jU6pGPXaXCLaxup/em0hk6zMPwamVD+lpPsQEYQVhLWJCH7sfQVQFpPHEVavRTjyRsZUKj+oRvTjg5uQny+qaexk0/yJLsi3AbnE0nSTxJPr8dHn/oBDpgL9krdsQS9o4ds0/slM2YYD/YT/aL/Y7G0ZfoWzS/Co16Xc4LdsOi/C9q5NTv</latexit><latexit sha1_base64="SLhgcGOrXpNfQnZ5byfyZF58J3o=">AAACqnicZVHbbtNAEN2YWwm3FB55GRFFKlIU7AgJHit44aVSQaQtiqNovR4nq+7F2h2jRJY/g6/hFT6Cv2GdGtSmI+3u0ZnbzpysVNJTHP/pRXfu3rv/4OBh/9HjJ0+fDQ6fn3lbOYEzYZV1Fxn3qKTBGUlSeFE65DpTeJ5dfmz959/ReWnNV9qWuNB8ZWQhBadALQdvUmVXUB6lmtM6K+pNM/4Htw2kWuaQIvExpFl4XsNyMIwn8c7gNkg6MGSdnS4PeyLNrag0GhKKez9P4pIWNXckhcKmn1YeSy4u+QrnARqu0S/q3WQNjAKTQ2FdOIZgx17PqLn2fquzENn+2u/7WvK/b3TdSZu2ot/rT8X7RS1NWREacdW+qBSQhXZ5kEuHgtQ2AC6cDBOAWHPHBYUV36jU6pGPXaXCLaxup/em0hk6zMPwamVD+lpPsQEYQVhLWJCH7sfQVQFpPHEVavRTjyRsZUKj+oRvTjg5uQny+qaexk0/yJLsi3AbnE0nSTxJPr8dHn/oBDpgL9krdsQS9o4ds0/slM2YYD/YT/aL/Y7G0ZfoWzS/Co16Xc4LdsOi/C9q5NTv</latexit><latexit sha1_base64="SLhgcGOrXpNfQnZ5byfyZF58J3o=">AAACqnicZVHbbtNAEN2YWwm3FB55GRFFKlIU7AgJHit44aVSQaQtiqNovR4nq+7F2h2jRJY/g6/hFT6Cv2GdGtSmI+3u0ZnbzpysVNJTHP/pRXfu3rv/4OBh/9HjJ0+fDQ6fn3lbOYEzYZV1Fxn3qKTBGUlSeFE65DpTeJ5dfmz959/ReWnNV9qWuNB8ZWQhBadALQdvUmVXUB6lmtM6K+pNM/4Htw2kWuaQIvExpFl4XsNyMIwn8c7gNkg6MGSdnS4PeyLNrag0GhKKez9P4pIWNXckhcKmn1YeSy4u+QrnARqu0S/q3WQNjAKTQ2FdOIZgx17PqLn2fquzENn+2u/7WvK/b3TdSZu2ot/rT8X7RS1NWREacdW+qBSQhXZ5kEuHgtQ2AC6cDBOAWHPHBYUV36jU6pGPXaXCLaxup/em0hk6zMPwamVD+lpPsQEYQVhLWJCH7sfQVQFpPHEVavRTjyRsZUKj+oRvTjg5uQny+qaexk0/yJLsi3AbnE0nSTxJPr8dHn/oBDpgL9krdsQS9o4ds0/slM2YYD/YT/aL/Y7G0ZfoWzS/Co16Xc4LdsOi/C9q5NTv</latexit><latexit sha1_base64="SLhgcGOrXpNfQnZ5byfyZF58J3o=">AAACqnicZVHbbtNAEN2YWwm3FB55GRFFKlIU7AgJHit44aVSQaQtiqNovR4nq+7F2h2jRJY/g6/hFT6Cv2GdGtSmI+3u0ZnbzpysVNJTHP/pRXfu3rv/4OBh/9HjJ0+fDQ6fn3lbOYEzYZV1Fxn3qKTBGUlSeFE65DpTeJ5dfmz959/ReWnNV9qWuNB8ZWQhBadALQdvUmVXUB6lmtM6K+pNM/4Htw2kWuaQIvExpFl4XsNyMIwn8c7gNkg6MGSdnS4PeyLNrag0GhKKez9P4pIWNXckhcKmn1YeSy4u+QrnARqu0S/q3WQNjAKTQ2FdOIZgx17PqLn2fquzENn+2u/7WvK/b3TdSZu2ot/rT8X7RS1NWREacdW+qBSQhXZ5kEuHgtQ2AC6cDBOAWHPHBYUV36jU6pGPXaXCLaxup/em0hk6zMPwamVD+lpPsQEYQVhLWJCH7sfQVQFpPHEVavRTjyRsZUKj+oRvTjg5uQny+qaexk0/yJLsi3AbnE0nSTxJPr8dHn/oBDpgL9krdsQS9o4ds0/slM2YYD/YT/aL/Y7G0ZfoWzS/Co16Xc4LdsOi/C9q5NTv</latexit>



Result of ICM

!20


Result of ICM



General Inference in MRFs

• In general, we do not have such an easy model • Therefore, we need more general inference

methods for MRF • The major aim is to exploit sparsity in the

graphical model to make inference efficient

!21



Inference on a Chain

The joint probability is given by

The marginal at x3 is

In the general case with N nodes we have

and

!22




• This would mean KN computations! A more efficient way is obtained by rearranging:

Vectors of size K

!23




In general, we have

!24




The messages µα and µβ can be computed recursively:

Computation of µα starts at the first node and computation of µβ starts at the last node.

!25




• The first values of µα and µβ are:

• The partition function can be computed at any node:

• Overall, we have O(NK2) operations to compute the marginal

!26




To compute local marginals: •Compute and store all forward messages, .

•Compute and store all backward messages,

•Compute Z once at a node xm:

•Computefor all variables required.

!27



More General Graphs

The message-passing algorithm can be extended to more general graphs:

Directed Tree PolytreeUndirected

Tree

It is then known as the sum-product algorithm. A special case of this is belief propagation.

!28

f(x1, x2, x3) = p(x1)p(x2)p(x3 | x1, x2)



Factor Graphs

• The Sum-product algorithm can be used to do inference on undirected and directed graphs. • A representation that generalizes directed and

undirected models is the factor graph.

Directed graph Factor graph

!29



Factor Graphs

• The Sum-product algorithm can be used to do inference on undirected and directed graphs. • A representation that generalizes directed and

undirected models is the factor graph.

Undirected graph Factor graph

!30

fa

fb



Factor Graphs

Factor graphs • can contain multiple factors

for the same nodes • are more general than

undirected graphs • are bipartite, i.e. they consist

of two kinds of nodes and all edges connect nodes of different kind

!31

x1 x3

x4

fa



Factor Graphs

• Directed trees convert to tree-structured factor graphs

• The same holds for undirected trees

• Also: directed polytrees convert to tree-structured factor graphs

• And: Local cycles in a directed graph can be removed by converting to a factor graph

!32

x1 x3

x4

x1 x3

x4



The Sum-Product Algorithm

Assumptions: • all variables are discrete • the factor graph has a tree structure The factor graph represents the joint distribution as a product of factor nodes:

The marginal distribution at a given node x is

!33

p(x) =Y

s

fs(xs)

p(x) =X

x\x

p(x)



p(x) =X

x\x

Y

s2ne(x)Fs(x,Xs)

p(x) =Y

s2ne(x)

X

Xs

Fs(x,Xs) =Y

s2ne(x)µfs!x(x)


For a given node x the joint can be written as

!34

Product of all factors associated with fs

p(x) =Y

s2ne(x)Fs(x,Xs)

Thus, we have

Key insight: Sum and product can be exchanged!

“Messages from factors to node x”



µfs!x(x) =X

x1

· · ·X

xM

fs(x, x1, . . . , xM )Y

m2ne(fs)\x

X

Xsm

Gm(xm, Xsm)

=X

x1

· · ·X

xM

fs(x, x1, . . . , xM )Y

m2ne(fs)\xµxm!fs(xm)


The factors in the messages can be factorized further:

!35

The messages can then be computed as

“Messages from nodes to factors”

Fs(x,Xs) = fs(x, x1, . . . , xM )G1(x1, Xs1) . . . GM (xM , XsM )



GM (xm, Xsm) =Y

l2ne(xm)\fs

Fl(xm, Xml)

µxm!fs(xm) =Y

l2ne(xm)\fs

X

Xml

Fl(xm, Xml)


The factors G of the neighboring nodes can again be factorized further:

!36

This results in the exact same situation as above! We can now recursively apply the derived rules:

=Y

l2ne(xm)\fs

µfl!xm(xm)




!37

Summary marginalization:

1.Consider the node x as a root note 2.Initialize the recursion at the leaf nodes as:

(var) or (fac) 3.Propagate the messages from the leaves to the

root x 4.Propagate the messages back from the root to

the leaves 5.We can get the marginals at every node in the

graph by multiplying all incoming messages

µf!x(x) = 1 µx!f (x) = f(x)



The Max-Sum Algorithm

Sum-product is used to find the marginal distributions at every node, but: How can we find the setting of all variables that maximizes the joint probability? And what is the value of that maximal probability? Idea: use sum-product to find all marginals and then report the value for each node x that maximizes the marginal p(x) However: this does not give the overall maximum of the joint probability

!38



=1

Zmaxx1

[ 1,2(x1, x2) [. . .max N�1,N (xN�1, xN )]]


Observation: the max-operator is distributive, just like the multiplication used in sum-product:

Idea: use max instead of sum as above and exchange it with the product Chain example:

Message passing can be used as above!

!39

max(ab, ac) = amax(b, c) if a � 0

maxx

p(x) =1

Zmaxx1

. . .max[ 1,2(x1, x2) . . . N�1,N (xN�1, xN )]




To find the maximum value of p(x), we start again at the leaves and propagate to the root. Two problems: • no summation, but many multiplications; this

leads to numerical instability (very small values) • when propagating back, multiple configurations

of x can maximize p(x), leading to wrong assignments of the overall maximum

Solution to the first: Transform everything into log-space and use sums

!40




Solution to the second problem: Keep track of the arg max in the forward step, i.e. store at each node which value was responsible for the maximum:

Then, in the back-tracking step we can recover the arg max by recursive substitution of:

!41

�(xn) = argmaxxn�1

[ln fn�1,n(xn�1, xn) + µxn�1!fn�1,n(xn)]

xmaxn�1 = �(xmax

n )



Sum-Product Inference in General Graphical Models

1.Convert graph (directed or undirected) into a factor graph (there are no cycles)

2.If the goal is to marginalize at node x, then consider x as a root node

3.Initialize the recursion at the leaf nodes as: (var) or (fac)

4.Propagate messages from the leaves to x 5.Propagate messages from x to the leaves 6.Obtain marginals at every node by multiplying all incoming messages

!42

µf!x(x) = 1 µx!f (x) = f(x)



Other Inference Algorithms• Max-Sum algorithm: used to maximize the joint

probability of all variables (no marginalization) • Junction Tree algorithm: exact inference for

general graphs (even with loops) • Loopy belief propagation: approximate

inference on general graphs (more efficient)

Special kind of undirected GM: • Conditional Random fields (e.g.: classification)

!43



Conditional Random Fields

• Another kind of undirected graphical model is known as Conditional Random Field (CRF). • CRFs are used for classification where labels are

represented as discrete random variables y and features as continuous random variables x • A CRF represents the conditional probability where w are parameters learned from training data.

• CRFs are discriminative and MRFs are generative

!44



Conditional Random Fields

Derivation of the formula for CRFs:

In the training phase, we compute parameters w that maximize the posterior:

where (x,y) is the training data and p(w) is a Gaussian prior. In the inference phase we maximize

!45

w = argmaxw

p(w | x,y) = argmaxw

p(y | x,w)p(w)

argmaxy⇤

p(y⇤ | x⇤, w)



CRF Training

We minimize the negative log-posterior:

Computing the likelihood is intractable, as we have to compute the partition function for each w. We can approximate the likelihood using pseudo-likelihood:

whereMarkov blanket Ci: All cliques containing yi

!46



Pseudo Likelihood

!47



Pseudo Likelihood

Pseudo-likelihood is computed only on the Markov blanket of yi and its corresp. feature nodes.

!48



Potential Functions

• The only requirement for the potential functions is that they are positive. We achieve that with:where f is a compatibility function that is large if the labels yC fit well to the features xC.

• This is called the log-linear model.

• The function f can be, e.g. a local classifier

!49



CRF Training and Inference

Training: • Using pseudo-likelihood, training is efficient. We have

to minimize:

• This is a convex function that can be minimized using gradient descent

Inference: • Only approximatively, e.g. using loopy belief

propagation

Log-pseudo-likelihood Gaussian prior

!50



Summary• Undirected models (aka Markov random fields)

provide an intuitive representation of conditional independence

• An MRF is defined as a factorization over clique potentials and normalized globally

• Directed and undirected models have different representative power (no simple “containment”)

• Inference on undirected Markov chains is efficient using message passing

• Factor graphs are more general; exact inference can be done efficiently using sum-product

!51

Example: Camera Image€¦ · x i x i PD Dr. Rudolph Triebel Computer Vision Group Machine Learning...

Documents

Transcript of Example: Camera Image€¦ · x i x i PD Dr. Rudolph Triebel Computer Vision Group Machine Learning...