FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte...

19
WESLEY MADDOX JOINT WORK WITH TIMUR GARIPOV, PAVEL IZMAILOV, DMITRY VETROV, ANDREW GORDON WILSON FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL AVERAGING OF DNNS 1

Transcript of FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte...

Page 1: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

WESLEY MADDOX

JOINT WORK WITH TIMUR GARIPOV, PAVEL IZMAILOV, DMITRY VETROV, ANDREW GORDON WILSON

FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL AVERAGING OF DNNS

�1

Page 2: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

SUMMARY‣ Stochastic Weight Averaging (Izmailov et al, UAI, 2018) computes first

moment of weights given from SGD iterates with a modified learning rate schedule.

‣ We propose to keep the variance as well to form a Gaussian approximation in weight space.

‣ Sample from Gaussian to compute Bayesian model averages and estimate uncertainty.

‣ Theoretically motivated from results on SGD & relation of iterates to Gaussian distribution (Ruppert, 1992 and Mandt et al, 2017).

�2

Page 3: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

APPROXIMATE BAYESIAN INFERENCE‣ Why?

‣ Compute intractable integrals

‣ Uncertainty quantification

‣ How?

‣ Laplace:

‣ Variational Bayes:

‣ Markov Chain Monte Carlo

�3

p(✓|y) ⇡ N(✓MAP , (H(✓MAP ) + �I)�1)<latexit sha1_base64="Mf9KAZJ8VrK1OBmaJH2llqNu3TI=">AAACMHicbZDLSgMxFIYz3q23qks3wSK0qGVGBF16WagLpYJVoVPLmTS1wcxMSM6IZewjufFRdKOgiFufwrTOwtsPgZ/vnMPJ+QMlhUHXfXYGBoeGR0bHxnMTk1PTM/nZuVMTJ5rxKotlrM8DMFyKiFdRoOTnSnMIA8nPgqvdXv3smmsj4ugEO4rXQ7iMREswQIsa+T1V9LHNEW47JeqDUjq+oUcZa6SH25XuCi3u/wAlukx9aXc0gR6ULtJVr1tq5Atu2e2L/jVeZgokU6WRf/CbMUtCHiGTYEzNcxXWU9AomOTdnJ8YroBdwSWvWRtByE097R/cpUuWNGkr1vZFSPv0+0QKoTGdMLCdIWDb/K714H+1WoKtzXoqIpUgj9jXolYiKca0lx5tCs0Zyo41wLSwf6WsDRoY2oxzNgTv98l/zela2XPL3vF6YWsni2OMLJBFUiQe2SBbZJ9USJUwckceyQt5de6dJ+fNef9qHXCymXnyQ87HJ0gRpzc=</latexit><latexit sha1_base64="Mf9KAZJ8VrK1OBmaJH2llqNu3TI=">AAACMHicbZDLSgMxFIYz3q23qks3wSK0qGVGBF16WagLpYJVoVPLmTS1wcxMSM6IZewjufFRdKOgiFufwrTOwtsPgZ/vnMPJ+QMlhUHXfXYGBoeGR0bHxnMTk1PTM/nZuVMTJ5rxKotlrM8DMFyKiFdRoOTnSnMIA8nPgqvdXv3smmsj4ugEO4rXQ7iMREswQIsa+T1V9LHNEW47JeqDUjq+oUcZa6SH25XuCi3u/wAlukx9aXc0gR6ULtJVr1tq5Atu2e2L/jVeZgokU6WRf/CbMUtCHiGTYEzNcxXWU9AomOTdnJ8YroBdwSWvWRtByE097R/cpUuWNGkr1vZFSPv0+0QKoTGdMLCdIWDb/K714H+1WoKtzXoqIpUgj9jXolYiKca0lx5tCs0Zyo41wLSwf6WsDRoY2oxzNgTv98l/zela2XPL3vF6YWsni2OMLJBFUiQe2SBbZJ9USJUwckceyQt5de6dJ+fNef9qHXCymXnyQ87HJ0gRpzc=</latexit><latexit sha1_base64="Mf9KAZJ8VrK1OBmaJH2llqNu3TI=">AAACMHicbZDLSgMxFIYz3q23qks3wSK0qGVGBF16WagLpYJVoVPLmTS1wcxMSM6IZewjufFRdKOgiFufwrTOwtsPgZ/vnMPJ+QMlhUHXfXYGBoeGR0bHxnMTk1PTM/nZuVMTJ5rxKotlrM8DMFyKiFdRoOTnSnMIA8nPgqvdXv3smmsj4ugEO4rXQ7iMREswQIsa+T1V9LHNEW47JeqDUjq+oUcZa6SH25XuCi3u/wAlukx9aXc0gR6ULtJVr1tq5Atu2e2L/jVeZgokU6WRf/CbMUtCHiGTYEzNcxXWU9AomOTdnJ8YroBdwSWvWRtByE097R/cpUuWNGkr1vZFSPv0+0QKoTGdMLCdIWDb/K714H+1WoKtzXoqIpUgj9jXolYiKca0lx5tCs0Zyo41wLSwf6WsDRoY2oxzNgTv98l/zela2XPL3vF6YWsni2OMLJBFUiQe2SBbZJ9USJUwckceyQt5de6dJ+fNef9qHXCymXnyQ87HJ0gRpzc=</latexit><latexit sha1_base64="Mf9KAZJ8VrK1OBmaJH2llqNu3TI=">AAACMHicbZDLSgMxFIYz3q23qks3wSK0qGVGBF16WagLpYJVoVPLmTS1wcxMSM6IZewjufFRdKOgiFufwrTOwtsPgZ/vnMPJ+QMlhUHXfXYGBoeGR0bHxnMTk1PTM/nZuVMTJ5rxKotlrM8DMFyKiFdRoOTnSnMIA8nPgqvdXv3smmsj4ugEO4rXQ7iMREswQIsa+T1V9LHNEW47JeqDUjq+oUcZa6SH25XuCi3u/wAlukx9aXc0gR6ULtJVr1tq5Atu2e2L/jVeZgokU6WRf/CbMUtCHiGTYEzNcxXWU9AomOTdnJ8YroBdwSWvWRtByE097R/cpUuWNGkr1vZFSPv0+0QKoTGdMLCdIWDb/K714H+1WoKtzXoqIpUgj9jXolYiKca0lx5tCs0Zyo41wLSwf6WsDRoY2oxzNgTv98l/zela2XPL3vF6YWsni2OMLJBFUiQe2SBbZJ9USJUwckceyQt5de6dJ+fNef9qHXCymXnyQ87HJ0gRpzc=</latexit>

p(✓|y) ⇡ N(µ, S)<latexit sha1_base64="uGLxWtCmfm/tnJsHfMg2JigS3sc=">AAACBnicbVDLSgNBEJyNrxhfqx5FGAxCAhJ2RdBj0IsniWgekF3C7GSSDJndHWZ6xSXm5MVf8eJBEa9+gzf/xsnjoIkFDUVVN91dgRRcg+N8W5mFxaXllexqbm19Y3PL3t6p6ThRlFVpLGLVCIhmgkesChwEa0jFSBgIVg/6FyO/fseU5nF0C6lkfki6Ee9wSsBILXtfFjzoMSAPaRF7REoV3+OrghcmR/im2LLzTskZA88Td0ryaIpKy/7y2jFNQhYBFUTrputI8AdEAaeCDXNeopkktE+6rGloREKm/cH4jSE+NEobd2JlKgI8Vn9PDEiodRoGpjMk0NOz3kj8z2sm0DnzBzySCbCIThZ1EoEhxqNMcJsrRkGkhhCquLkV0x5RhIJJLmdCcGdfnie145LrlNzrk3z5fBpHFu2hA1RALjpFZXSJKqiKKHpEz+gVvVlP1ov1bn1MWjPWdGYX/YH1+QM4sZen</latexit><latexit sha1_base64="uGLxWtCmfm/tnJsHfMg2JigS3sc=">AAACBnicbVDLSgNBEJyNrxhfqx5FGAxCAhJ2RdBj0IsniWgekF3C7GSSDJndHWZ6xSXm5MVf8eJBEa9+gzf/xsnjoIkFDUVVN91dgRRcg+N8W5mFxaXllexqbm19Y3PL3t6p6ThRlFVpLGLVCIhmgkesChwEa0jFSBgIVg/6FyO/fseU5nF0C6lkfki6Ee9wSsBILXtfFjzoMSAPaRF7REoV3+OrghcmR/im2LLzTskZA88Td0ryaIpKy/7y2jFNQhYBFUTrputI8AdEAaeCDXNeopkktE+6rGloREKm/cH4jSE+NEobd2JlKgI8Vn9PDEiodRoGpjMk0NOz3kj8z2sm0DnzBzySCbCIThZ1EoEhxqNMcJsrRkGkhhCquLkV0x5RhIJJLmdCcGdfnie145LrlNzrk3z5fBpHFu2hA1RALjpFZXSJKqiKKHpEz+gVvVlP1ov1bn1MWjPWdGYX/YH1+QM4sZen</latexit><latexit sha1_base64="uGLxWtCmfm/tnJsHfMg2JigS3sc=">AAACBnicbVDLSgNBEJyNrxhfqx5FGAxCAhJ2RdBj0IsniWgekF3C7GSSDJndHWZ6xSXm5MVf8eJBEa9+gzf/xsnjoIkFDUVVN91dgRRcg+N8W5mFxaXllexqbm19Y3PL3t6p6ThRlFVpLGLVCIhmgkesChwEa0jFSBgIVg/6FyO/fseU5nF0C6lkfki6Ee9wSsBILXtfFjzoMSAPaRF7REoV3+OrghcmR/im2LLzTskZA88Td0ryaIpKy/7y2jFNQhYBFUTrputI8AdEAaeCDXNeopkktE+6rGloREKm/cH4jSE+NEobd2JlKgI8Vn9PDEiodRoGpjMk0NOz3kj8z2sm0DnzBzySCbCIThZ1EoEhxqNMcJsrRkGkhhCquLkV0x5RhIJJLmdCcGdfnie145LrlNzrk3z5fBpHFu2hA1RALjpFZXSJKqiKKHpEz+gVvVlP1ov1bn1MWjPWdGYX/YH1+QM4sZen</latexit><latexit sha1_base64="uGLxWtCmfm/tnJsHfMg2JigS3sc=">AAACBnicbVDLSgNBEJyNrxhfqx5FGAxCAhJ2RdBj0IsniWgekF3C7GSSDJndHWZ6xSXm5MVf8eJBEa9+gzf/xsnjoIkFDUVVN91dgRRcg+N8W5mFxaXllexqbm19Y3PL3t6p6ThRlFVpLGLVCIhmgkesChwEa0jFSBgIVg/6FyO/fseU5nF0C6lkfki6Ee9wSsBILXtfFjzoMSAPaRF7REoV3+OrghcmR/im2LLzTskZA88Td0ryaIpKy/7y2jFNQhYBFUTrputI8AdEAaeCDXNeopkktE+6rGloREKm/cH4jSE+NEobd2JlKgI8Vn9PDEiodRoGpjMk0NOz3kj8z2sm0DnzBzySCbCIThZ1EoEhxqNMcJsrRkGkhhCquLkV0x5RhIJJLmdCcGdfnie145LrlNzrk3z5fBpHFu2hA1RALjpFZXSJKqiKKHpEz+gVvVlP1ov1bn1MWjPWdGYX/YH1+QM4sZen</latexit>

p(y⇤|y) = Ep(✓|y)(p(y|✓))<latexit sha1_base64="tR6QRGiHr0gv/IN6f3L8/n2xj4c=">AAACEHicbVDLSgMxFM34rPVVdekmWMTWRZkRQTdCUQSXFewD2jpk0rQNzcyE5I4wTPsJbvwVNy4UcevSnX9j2s5CWw9cODnnXnLv8aTgGmz721pYXFpeWc2sZdc3Nre2czu7NR1GirIqDUWoGh7RTPCAVYGDYA2pGPE9were4Grs1x+Y0jwM7iCWrO2TXsC7nBIwkps7koX4/ngYF/EFvnYTWWhBnwExwqhgrOH0WSy6ubxdsifA88RJSR6lqLi5r1YnpJHPAqCCaN10bAnthCjgVLBRthVpJgkdkB5rGhoQn+l2MjlohA+N0sHdUJkKAE/U3xMJ8bWOfc90+gT6etYbi/95zQi65+2EBzICFtDpR91IYAjxOB3c4YpRELEhhCpudsW0TxShYDLMmhCc2ZPnSe2k5Ngl5/Y0X75M48igfXSACshBZ6iMblAFVRFFj+gZvaI368l6sd6tj2nrgpXO7KE/sD5/AITVm6I=</latexit><latexit sha1_base64="tR6QRGiHr0gv/IN6f3L8/n2xj4c=">AAACEHicbVDLSgMxFM34rPVVdekmWMTWRZkRQTdCUQSXFewD2jpk0rQNzcyE5I4wTPsJbvwVNy4UcevSnX9j2s5CWw9cODnnXnLv8aTgGmz721pYXFpeWc2sZdc3Nre2czu7NR1GirIqDUWoGh7RTPCAVYGDYA2pGPE9were4Grs1x+Y0jwM7iCWrO2TXsC7nBIwkps7koX4/ngYF/EFvnYTWWhBnwExwqhgrOH0WSy6ubxdsifA88RJSR6lqLi5r1YnpJHPAqCCaN10bAnthCjgVLBRthVpJgkdkB5rGhoQn+l2MjlohA+N0sHdUJkKAE/U3xMJ8bWOfc90+gT6etYbi/95zQi65+2EBzICFtDpR91IYAjxOB3c4YpRELEhhCpudsW0TxShYDLMmhCc2ZPnSe2k5Ngl5/Y0X75M48igfXSACshBZ6iMblAFVRFFj+gZvaI368l6sd6tj2nrgpXO7KE/sD5/AITVm6I=</latexit><latexit sha1_base64="tR6QRGiHr0gv/IN6f3L8/n2xj4c=">AAACEHicbVDLSgMxFM34rPVVdekmWMTWRZkRQTdCUQSXFewD2jpk0rQNzcyE5I4wTPsJbvwVNy4UcevSnX9j2s5CWw9cODnnXnLv8aTgGmz721pYXFpeWc2sZdc3Nre2czu7NR1GirIqDUWoGh7RTPCAVYGDYA2pGPE9were4Grs1x+Y0jwM7iCWrO2TXsC7nBIwkps7koX4/ngYF/EFvnYTWWhBnwExwqhgrOH0WSy6ubxdsifA88RJSR6lqLi5r1YnpJHPAqCCaN10bAnthCjgVLBRthVpJgkdkB5rGhoQn+l2MjlohA+N0sHdUJkKAE/U3xMJ8bWOfc90+gT6etYbi/95zQi65+2EBzICFtDpR91IYAjxOB3c4YpRELEhhCpudsW0TxShYDLMmhCc2ZPnSe2k5Ngl5/Y0X75M48igfXSACshBZ6iMblAFVRFFj+gZvaI368l6sd6tj2nrgpXO7KE/sD5/AITVm6I=</latexit><latexit sha1_base64="tR6QRGiHr0gv/IN6f3L8/n2xj4c=">AAACEHicbVDLSgMxFM34rPVVdekmWMTWRZkRQTdCUQSXFewD2jpk0rQNzcyE5I4wTPsJbvwVNy4UcevSnX9j2s5CWw9cODnnXnLv8aTgGmz721pYXFpeWc2sZdc3Nre2czu7NR1GirIqDUWoGh7RTPCAVYGDYA2pGPE9were4Grs1x+Y0jwM7iCWrO2TXsC7nBIwkps7koX4/ngYF/EFvnYTWWhBnwExwqhgrOH0WSy6ubxdsifA88RJSR6lqLi5r1YnpJHPAqCCaN10bAnthCjgVLBRthVpJgkdkB5rGhoQn+l2MjlohA+N0sHdUJkKAE/U3xMJ8bWOfc90+gT6etYbi/95zQi65+2EBzICFtDpR91IYAjxOB3c4YpRELEhhCpudsW0TxShYDLMmhCc2ZPnSe2k5Ngl5/Y0X75M48igfXSACshBZ6iMblAFVRFFj+gZvaI368l6sd6tj2nrgpXO7KE/sD5/AITVm6I=</latexit>

Page 4: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

STOCHASTIC WEIGHT AVERAGING (IZMAILOV ET AL 2018)

�4

✓1<latexit sha1_base64="zw++W+WFFXmKl2onW11SbJCIbXA=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHHsaDcoW49TrxSBVnhNTOc1L3sOeSDBWUozkov/eHMUsjroBJakzPIwn4M6pBMMnnpX5qeELZhI54z1JFI278WXbxHJ9YZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JBKNoSvT/H/pH3mesT1bqqVxmUeRxEdoWN0ijxUQw10jZqohRhS6AE9oWfHOI/Oi/O6bC04+cwh+gHn7RPzyJBx</latexit><latexit sha1_base64="zw++W+WFFXmKl2onW11SbJCIbXA=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHHsaDcoW49TrxSBVnhNTOc1L3sOeSDBWUozkov/eHMUsjroBJakzPIwn4M6pBMMnnpX5qeELZhI54z1JFI278WXbxHJ9YZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JBKNoSvT/H/pH3mesT1bqqVxmUeRxEdoWN0ijxUQw10jZqohRhS6AE9oWfHOI/Oi/O6bC04+cwh+gHn7RPzyJBx</latexit><latexit sha1_base64="zw++W+WFFXmKl2onW11SbJCIbXA=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHHsaDcoW49TrxSBVnhNTOc1L3sOeSDBWUozkov/eHMUsjroBJakzPIwn4M6pBMMnnpX5qeELZhI54z1JFI278WXbxHJ9YZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JBKNoSvT/H/pH3mesT1bqqVxmUeRxEdoWN0ijxUQw10jZqohRhS6AE9oWfHOI/Oi/O6bC04+cwh+gHn7RPzyJBx</latexit><latexit sha1_base64="zw++W+WFFXmKl2onW11SbJCIbXA=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHHsaDcoW49TrxSBVnhNTOc1L3sOeSDBWUozkov/eHMUsjroBJakzPIwn4M6pBMMnnpX5qeELZhI54z1JFI278WXbxHJ9YZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JBKNoSvT/H/pH3mesT1bqqVxmUeRxEdoWN0ijxUQw10jZqohRhS6AE9oWfHOI/Oi/O6bC04+cwh+gHn7RPzyJBx</latexit>

✓2<latexit sha1_base64="hHhYIh+EK7pKduJSoxvXPATbyMg=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LdlSqb0VvXisYD+wXUo2zbah2eySzAql9F948aCIV/+NN/+N6XYFFX0w8Hhvhpl5QSKFAUI+nJXVtfWNzcJWcXtnd2+/dHDYNnGqGW+xWMa6G1DDpVC8BQIk7yaa0yiQvBNMrhZ+555rI2J1C9OE+xEdKREKRsFKd30Yc6CDCsaDUpm49TrxSBVnhNTOc1L3sOeSDGWUozkovfeHMUsjroBJakzPIwn4M6pBMMnnxX5qeELZhI54z1JFI278WXbxHJ9aZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JCKNoSvT/H/pF1xPeJ6N9Vy4zKPo4CO0Qk6Qx6qoQa6Rk3UQgwp9ICe0LNjnEfnxXldtq44+cwR+gHn7RP1TpBy</latexit><latexit sha1_base64="hHhYIh+EK7pKduJSoxvXPATbyMg=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LdlSqb0VvXisYD+wXUo2zbah2eySzAql9F948aCIV/+NN/+N6XYFFX0w8Hhvhpl5QSKFAUI+nJXVtfWNzcJWcXtnd2+/dHDYNnGqGW+xWMa6G1DDpVC8BQIk7yaa0yiQvBNMrhZ+555rI2J1C9OE+xEdKREKRsFKd30Yc6CDCsaDUpm49TrxSBVnhNTOc1L3sOeSDGWUozkovfeHMUsjroBJakzPIwn4M6pBMMnnxX5qeELZhI54z1JFI278WXbxHJ9aZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JCKNoSvT/H/pF1xPeJ6N9Vy4zKPo4CO0Qk6Qx6qoQa6Rk3UQgwp9ICe0LNjnEfnxXldtq44+cwR+gHn7RP1TpBy</latexit><latexit sha1_base64="hHhYIh+EK7pKduJSoxvXPATbyMg=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LdlSqb0VvXisYD+wXUo2zbah2eySzAql9F948aCIV/+NN/+N6XYFFX0w8Hhvhpl5QSKFAUI+nJXVtfWNzcJWcXtnd2+/dHDYNnGqGW+xWMa6G1DDpVC8BQIk7yaa0yiQvBNMrhZ+555rI2J1C9OE+xEdKREKRsFKd30Yc6CDCsaDUpm49TrxSBVnhNTOc1L3sOeSDGWUozkovfeHMUsjroBJakzPIwn4M6pBMMnnxX5qeELZhI54z1JFI278WXbxHJ9aZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JCKNoSvT/H/pF1xPeJ6N9Vy4zKPo4CO0Qk6Qx6qoQa6Rk3UQgwp9ICe0LNjnEfnxXldtq44+cwR+gHn7RP1TpBy</latexit><latexit sha1_base64="hHhYIh+EK7pKduJSoxvXPATbyMg=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LdlSqb0VvXisYD+wXUo2zbah2eySzAql9F948aCIV/+NN/+N6XYFFX0w8Hhvhpl5QSKFAUI+nJXVtfWNzcJWcXtnd2+/dHDYNnGqGW+xWMa6G1DDpVC8BQIk7yaa0yiQvBNMrhZ+555rI2J1C9OE+xEdKREKRsFKd30Yc6CDCsaDUpm49TrxSBVnhNTOc1L3sOeSDGWUozkovfeHMUsjroBJakzPIwn4M6pBMMnnxX5qeELZhI54z1JFI278WXbxHJ9aZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JCKNoSvT/H/pF1xPeJ6N9Vy4zKPo4CO0Qk6Qx6qoQa6Rk3UQgwp9ICe0LNjnEfnxXldtq44+cwR+gHn7RP1TpBy</latexit>

✓3<latexit sha1_base64="yy6UzEciob+ejIkAE1tl9er8rvA=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LVmt1N6KXjxWsB/YLiWbZtvQbHZJZoVS+i+8eFDEq//Gm//GdLuCij4YeLw3w8y8IJHCACEfztLyyuraemGjuLm1vbNb2ttvmTjVjDdZLGPdCajhUijeBAGSdxLNaRRI3g7GV3O/fc+1EbG6hUnC/YgOlQgFo2Clux6MOND+Gcb9Upm4tRrxSAVnhFTPc1LzsOeSDGWUo9EvvfcGMUsjroBJakzXIwn4U6pBMMlnxV5qeELZmA5511JFI278aXbxDB9bZYDDWNtSgDP1+8SURsZMosB2RhRG5rc3F//yuimEF/5UqCQFrthiUZhKDDGev48HQnMGcmIJZVrYWzEbUU0Z2JCKNoSvT/H/pHXqesT1birl+mUeRwEdoiN0gjxURXV0jRqoiRhS6AE9oWfHOI/Oi/O6aF1y8pkD9APO2yf21JBz</latexit><latexit sha1_base64="yy6UzEciob+ejIkAE1tl9er8rvA=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LVmt1N6KXjxWsB/YLiWbZtvQbHZJZoVS+i+8eFDEq//Gm//GdLuCij4YeLw3w8y8IJHCACEfztLyyuraemGjuLm1vbNb2ttvmTjVjDdZLGPdCajhUijeBAGSdxLNaRRI3g7GV3O/fc+1EbG6hUnC/YgOlQgFo2Clux6MOND+Gcb9Upm4tRrxSAVnhFTPc1LzsOeSDGWUo9EvvfcGMUsjroBJakzXIwn4U6pBMMlnxV5qeELZmA5511JFI278aXbxDB9bZYDDWNtSgDP1+8SURsZMosB2RhRG5rc3F//yuimEF/5UqCQFrthiUZhKDDGev48HQnMGcmIJZVrYWzEbUU0Z2JCKNoSvT/H/pHXqesT1birl+mUeRwEdoiN0gjxURXV0jRqoiRhS6AE9oWfHOI/Oi/O6aF1y8pkD9APO2yf21JBz</latexit><latexit sha1_base64="yy6UzEciob+ejIkAE1tl9er8rvA=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LVmt1N6KXjxWsB/YLiWbZtvQbHZJZoVS+i+8eFDEq//Gm//GdLuCij4YeLw3w8y8IJHCACEfztLyyuraemGjuLm1vbNb2ttvmTjVjDdZLGPdCajhUijeBAGSdxLNaRRI3g7GV3O/fc+1EbG6hUnC/YgOlQgFo2Clux6MOND+Gcb9Upm4tRrxSAVnhFTPc1LzsOeSDGWUo9EvvfcGMUsjroBJakzXIwn4U6pBMMlnxV5qeELZmA5511JFI278aXbxDB9bZYDDWNtSgDP1+8SURsZMosB2RhRG5rc3F//yuimEF/5UqCQFrthiUZhKDDGev48HQnMGcmIJZVrYWzEbUU0Z2JCKNoSvT/H/pHXqesT1birl+mUeRwEdoiN0gjxURXV0jRqoiRhS6AE9oWfHOI/Oi/O6aF1y8pkD9APO2yf21JBz</latexit><latexit sha1_base64="yy6UzEciob+ejIkAE1tl9er8rvA=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LVmt1N6KXjxWsB/YLiWbZtvQbHZJZoVS+i+8eFDEq//Gm//GdLuCij4YeLw3w8y8IJHCACEfztLyyuraemGjuLm1vbNb2ttvmTjVjDdZLGPdCajhUijeBAGSdxLNaRRI3g7GV3O/fc+1EbG6hUnC/YgOlQgFo2Clux6MOND+Gcb9Upm4tRrxSAVnhFTPc1LzsOeSDGWUo9EvvfcGMUsjroBJakzXIwn4U6pBMMlnxV5qeELZmA5511JFI278aXbxDB9bZYDDWNtSgDP1+8SURsZMosB2RhRG5rc3F//yuimEF/5UqCQFrthiUZhKDDGev48HQnMGcmIJZVrYWzEbUU0Z2JCKNoSvT/H/pHXqesT1birl+mUeRwEdoiN0gjxURXV0jRqoiRhS6AE9oWfHOI/Oi/O6aF1y8pkD9APO2yf21JBz</latexit>

✓4<latexit sha1_base64="ZJWMqr9mpBruyYLQFS+dO1s2dHc=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHVYwH5Qpx63XikSrOCKmd56TuYc8lGSooR3NQfu8PY5ZGXAGT1JieRxLwZ1SDYJLPS/3U8ISyCR3xnqWKRtz4s+ziOT6xyhCHsbalAGfq94kZjYyZRoHtjCiMzW9vIf7l9VIIL/yZUEkKXLHlojCVGGK8eB8PheYM5NQSyrSwt2I2ppoysCGVbAhfn+L/SfvM9Yjr3VQrjcs8jiI6QsfoFHmohhroGjVRCzGk0AN6Qs+OcR6dF+d12Vpw8plD9APO2yf4WpB0</latexit><latexit sha1_base64="ZJWMqr9mpBruyYLQFS+dO1s2dHc=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHVYwH5Qpx63XikSrOCKmd56TuYc8lGSooR3NQfu8PY5ZGXAGT1JieRxLwZ1SDYJLPS/3U8ISyCR3xnqWKRtz4s+ziOT6xyhCHsbalAGfq94kZjYyZRoHtjCiMzW9vIf7l9VIIL/yZUEkKXLHlojCVGGK8eB8PheYM5NQSyrSwt2I2ppoysCGVbAhfn+L/SfvM9Yjr3VQrjcs8jiI6QsfoFHmohhroGjVRCzGk0AN6Qs+OcR6dF+d12Vpw8plD9APO2yf4WpB0</latexit><latexit sha1_base64="ZJWMqr9mpBruyYLQFS+dO1s2dHc=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHVYwH5Qpx63XikSrOCKmd56TuYc8lGSooR3NQfu8PY5ZGXAGT1JieRxLwZ1SDYJLPS/3U8ISyCR3xnqWKRtz4s+ziOT6xyhCHsbalAGfq94kZjYyZRoHtjCiMzW9vIf7l9VIIL/yZUEkKXLHlojCVGGK8eB8PheYM5NQSyrSwt2I2ppoysCGVbAhfn+L/SfvM9Yjr3VQrjcs8jiI6QsfoFHmohhroGjVRCzGk0AN6Qs+OcR6dF+d12Vpw8plD9APO2yf4WpB0</latexit><latexit sha1_base64="ZJWMqr9mpBruyYLQFS+dO1s2dHc=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHVYwH5Qpx63XikSrOCKmd56TuYc8lGSooR3NQfu8PY5ZGXAGT1JieRxLwZ1SDYJLPS/3U8ISyCR3xnqWKRtz4s+ziOT6xyhCHsbalAGfq94kZjYyZRoHtjCiMzW9vIf7l9VIIL/yZUEkKXLHlojCVGGK8eB8PheYM5NQSyrSwt2I2ppoysCGVbAhfn+L/SfvM9Yjr3VQrjcs8jiI6QsfoFHmohhroGjVRCzGk0AN6Qs+OcR6dF+d12Vpw8plD9APO2yf4WpB0</latexit>

Cycle the learning rate of SGD.

Page 5: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

STOCHASTIC WEIGHT AVERAGING (IZMAILOV ET AL 2018)

�5

Average models at end of cycles.

✓1<latexit sha1_base64="zw++W+WFFXmKl2onW11SbJCIbXA=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHHsaDcoW49TrxSBVnhNTOc1L3sOeSDBWUozkov/eHMUsjroBJakzPIwn4M6pBMMnnpX5qeELZhI54z1JFI278WXbxHJ9YZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JBKNoSvT/H/pH3mesT1bqqVxmUeRxEdoWN0ijxUQw10jZqohRhS6AE9oWfHOI/Oi/O6bC04+cwh+gHn7RPzyJBx</latexit><latexit sha1_base64="zw++W+WFFXmKl2onW11SbJCIbXA=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHHsaDcoW49TrxSBVnhNTOc1L3sOeSDBWUozkov/eHMUsjroBJakzPIwn4M6pBMMnnpX5qeELZhI54z1JFI278WXbxHJ9YZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JBKNoSvT/H/pH3mesT1bqqVxmUeRxEdoWN0ijxUQw10jZqohRhS6AE9oWfHOI/Oi/O6bC04+cwh+gHn7RPzyJBx</latexit><latexit sha1_base64="zw++W+WFFXmKl2onW11SbJCIbXA=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHHsaDcoW49TrxSBVnhNTOc1L3sOeSDBWUozkov/eHMUsjroBJakzPIwn4M6pBMMnnpX5qeELZhI54z1JFI278WXbxHJ9YZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JBKNoSvT/H/pH3mesT1bqqVxmUeRxEdoWN0ijxUQw10jZqohRhS6AE9oWfHOI/Oi/O6bC04+cwh+gHn7RPzyJBx</latexit><latexit sha1_base64="zw++W+WFFXmKl2onW11SbJCIbXA=">AAAB8XicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv3AdinZNNuGZrNLMiuU0n/hxYMiXv033vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHHsaDcoW49TrxSBVnhNTOc1L3sOeSDBWUozkov/eHMUsjroBJakzPIwn4M6pBMMnnpX5qeELZhI54z1JFI278WXbxHJ9YZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JBKNoSvT/H/pH3mesT1bqqVxmUeRxEdoWN0ijxUQw10jZqohRhS6AE9oWfHOI/Oi/O6bC04+cwh+gHn7RPzyJBx</latexit>

✓2<latexit sha1_base64="hHhYIh+EK7pKduJSoxvXPATbyMg=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LdlSqb0VvXisYD+wXUo2zbah2eySzAql9F948aCIV/+NN/+N6XYFFX0w8Hhvhpl5QSKFAUI+nJXVtfWNzcJWcXtnd2+/dHDYNnGqGW+xWMa6G1DDpVC8BQIk7yaa0yiQvBNMrhZ+555rI2J1C9OE+xEdKREKRsFKd30Yc6CDCsaDUpm49TrxSBVnhNTOc1L3sOeSDGWUozkovfeHMUsjroBJakzPIwn4M6pBMMnnxX5qeELZhI54z1JFI278WXbxHJ9aZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JCKNoSvT/H/pF1xPeJ6N9Vy4zKPo4CO0Qk6Qx6qoQa6Rk3UQgwp9ICe0LNjnEfnxXldtq44+cwR+gHn7RP1TpBy</latexit><latexit sha1_base64="hHhYIh+EK7pKduJSoxvXPATbyMg=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LdlSqb0VvXisYD+wXUo2zbah2eySzAql9F948aCIV/+NN/+N6XYFFX0w8Hhvhpl5QSKFAUI+nJXVtfWNzcJWcXtnd2+/dHDYNnGqGW+xWMa6G1DDpVC8BQIk7yaa0yiQvBNMrhZ+555rI2J1C9OE+xEdKREKRsFKd30Yc6CDCsaDUpm49TrxSBVnhNTOc1L3sOeSDGWUozkovfeHMUsjroBJakzPIwn4M6pBMMnnxX5qeELZhI54z1JFI278WXbxHJ9aZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JCKNoSvT/H/pF1xPeJ6N9Vy4zKPo4CO0Qk6Qx6qoQa6Rk3UQgwp9ICe0LNjnEfnxXldtq44+cwR+gHn7RP1TpBy</latexit><latexit sha1_base64="hHhYIh+EK7pKduJSoxvXPATbyMg=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LdlSqb0VvXisYD+wXUo2zbah2eySzAql9F948aCIV/+NN/+N6XYFFX0w8Hhvhpl5QSKFAUI+nJXVtfWNzcJWcXtnd2+/dHDYNnGqGW+xWMa6G1DDpVC8BQIk7yaa0yiQvBNMrhZ+555rI2J1C9OE+xEdKREKRsFKd30Yc6CDCsaDUpm49TrxSBVnhNTOc1L3sOeSDGWUozkovfeHMUsjroBJakzPIwn4M6pBMMnnxX5qeELZhI54z1JFI278WXbxHJ9aZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JCKNoSvT/H/pF1xPeJ6N9Vy4zKPo4CO0Qk6Qx6qoQa6Rk3UQgwp9ICe0LNjnEfnxXldtq44+cwR+gHn7RP1TpBy</latexit><latexit sha1_base64="hHhYIh+EK7pKduJSoxvXPATbyMg=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LdlSqb0VvXisYD+wXUo2zbah2eySzAql9F948aCIV/+NN/+N6XYFFX0w8Hhvhpl5QSKFAUI+nJXVtfWNzcJWcXtnd2+/dHDYNnGqGW+xWMa6G1DDpVC8BQIk7yaa0yiQvBNMrhZ+555rI2J1C9OE+xEdKREKRsFKd30Yc6CDCsaDUpm49TrxSBVnhNTOc1L3sOeSDGWUozkovfeHMUsjroBJakzPIwn4M6pBMMnnxX5qeELZhI54z1JFI278WXbxHJ9aZYjDWNtSgDP1+8SMRsZMo8B2RhTG5re3EP/yeimEF/5MqCQFrthyUZhKDDFevI+HQnMGcmoJZVrYWzEbU00Z2JCKNoSvT/H/pF1xPeJ6N9Vy4zKPo4CO0Qk6Qx6qoQa6Rk3UQgwp9ICe0LNjnEfnxXldtq44+cwR+gHn7RP1TpBy</latexit>

✓3<latexit sha1_base64="yy6UzEciob+ejIkAE1tl9er8rvA=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LVmt1N6KXjxWsB/YLiWbZtvQbHZJZoVS+i+8eFDEq//Gm//GdLuCij4YeLw3w8y8IJHCACEfztLyyuraemGjuLm1vbNb2ttvmTjVjDdZLGPdCajhUijeBAGSdxLNaRRI3g7GV3O/fc+1EbG6hUnC/YgOlQgFo2Clux6MOND+Gcb9Upm4tRrxSAVnhFTPc1LzsOeSDGWUo9EvvfcGMUsjroBJakzXIwn4U6pBMMlnxV5qeELZmA5511JFI278aXbxDB9bZYDDWNtSgDP1+8SURsZMosB2RhRG5rc3F//yuimEF/5UqCQFrthiUZhKDDGev48HQnMGcmIJZVrYWzEbUU0Z2JCKNoSvT/H/pHXqesT1birl+mUeRwEdoiN0gjxURXV0jRqoiRhS6AE9oWfHOI/Oi/O6aF1y8pkD9APO2yf21JBz</latexit><latexit sha1_base64="yy6UzEciob+ejIkAE1tl9er8rvA=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LVmt1N6KXjxWsB/YLiWbZtvQbHZJZoVS+i+8eFDEq//Gm//GdLuCij4YeLw3w8y8IJHCACEfztLyyuraemGjuLm1vbNb2ttvmTjVjDdZLGPdCajhUijeBAGSdxLNaRRI3g7GV3O/fc+1EbG6hUnC/YgOlQgFo2Clux6MOND+Gcb9Upm4tRrxSAVnhFTPc1LzsOeSDGWUo9EvvfcGMUsjroBJakzXIwn4U6pBMMlnxV5qeELZmA5511JFI278aXbxDB9bZYDDWNtSgDP1+8SURsZMosB2RhRG5rc3F//yuimEF/5UqCQFrthiUZhKDDGev48HQnMGcmIJZVrYWzEbUU0Z2JCKNoSvT/H/pHXqesT1birl+mUeRwEdoiN0gjxURXV0jRqoiRhS6AE9oWfHOI/Oi/O6aF1y8pkD9APO2yf21JBz</latexit><latexit sha1_base64="yy6UzEciob+ejIkAE1tl9er8rvA=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LVmt1N6KXjxWsB/YLiWbZtvQbHZJZoVS+i+8eFDEq//Gm//GdLuCij4YeLw3w8y8IJHCACEfztLyyuraemGjuLm1vbNb2ttvmTjVjDdZLGPdCajhUijeBAGSdxLNaRRI3g7GV3O/fc+1EbG6hUnC/YgOlQgFo2Clux6MOND+Gcb9Upm4tRrxSAVnhFTPc1LzsOeSDGWUo9EvvfcGMUsjroBJakzXIwn4U6pBMMlnxV5qeELZmA5511JFI278aXbxDB9bZYDDWNtSgDP1+8SURsZMosB2RhRG5rc3F//yuimEF/5UqCQFrthiUZhKDDGev48HQnMGcmIJZVrYWzEbUU0Z2JCKNoSvT/H/pHXqesT1birl+mUeRwEdoiN0gjxURXV0jRqoiRhS6AE9oWfHOI/Oi/O6aF1y8pkD9APO2yf21JBz</latexit><latexit sha1_base64="yy6UzEciob+ejIkAE1tl9er8rvA=">AAAB8XicdVBNSwMxEM36WetX1aOXYBE8LVmt1N6KXjxWsB/YLiWbZtvQbHZJZoVS+i+8eFDEq//Gm//GdLuCij4YeLw3w8y8IJHCACEfztLyyuraemGjuLm1vbNb2ttvmTjVjDdZLGPdCajhUijeBAGSdxLNaRRI3g7GV3O/fc+1EbG6hUnC/YgOlQgFo2Clux6MOND+Gcb9Upm4tRrxSAVnhFTPc1LzsOeSDGWUo9EvvfcGMUsjroBJakzXIwn4U6pBMMlnxV5qeELZmA5511JFI278aXbxDB9bZYDDWNtSgDP1+8SURsZMosB2RhRG5rc3F//yuimEF/5UqCQFrthiUZhKDDGev48HQnMGcmIJZVrYWzEbUU0Z2JCKNoSvT/H/pHXqesT1birl+mUeRwEdoiN0gjxURXV0jRqoiRhS6AE9oWfHOI/Oi/O6aF1y8pkD9APO2yf21JBz</latexit>

✓i<latexit sha1_base64="7MtAShdCU97XGBvGDB5yo3o0icA=">AAAB8HicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv2QdinZNNuGZrNLMiuU0l/hxYMiXv053vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHAg/KFeLW68QjVZwRUjvPSd3DnksyVFCO5qD83h/GLI24AiapMT2PJODPqAbBJJ+X+qnhCWUTOuI9SxWNuPFn2cFzfGKVIQ5jbUsBztTvEzMaGTONAtsZURib395C/MvrpRBe+DOhkhS4YstFYSoxxHjxPR4KzRnIqSWUaWFvxWxMNWVgMyrZEL4+xf+T9pnrEde7qVYal3kcRXSEjtEp8lANNdA1aqIWYihCD+gJPTvaeXRenNdla8HJZw7RDzhvn/BKkH8=</latexit><latexit sha1_base64="7MtAShdCU97XGBvGDB5yo3o0icA=">AAAB8HicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv2QdinZNNuGZrNLMiuU0l/hxYMiXv053vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHAg/KFeLW68QjVZwRUjvPSd3DnksyVFCO5qD83h/GLI24AiapMT2PJODPqAbBJJ+X+qnhCWUTOuI9SxWNuPFn2cFzfGKVIQ5jbUsBztTvEzMaGTONAtsZURib395C/MvrpRBe+DOhkhS4YstFYSoxxHjxPR4KzRnIqSWUaWFvxWxMNWVgMyrZEL4+xf+T9pnrEde7qVYal3kcRXSEjtEp8lANNdA1aqIWYihCD+gJPTvaeXRenNdla8HJZw7RDzhvn/BKkH8=</latexit><latexit sha1_base64="7MtAShdCU97XGBvGDB5yo3o0icA=">AAAB8HicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv2QdinZNNuGZrNLMiuU0l/hxYMiXv053vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHAg/KFeLW68QjVZwRUjvPSd3DnksyVFCO5qD83h/GLI24AiapMT2PJODPqAbBJJ+X+qnhCWUTOuI9SxWNuPFn2cFzfGKVIQ5jbUsBztTvEzMaGTONAtsZURib395C/MvrpRBe+DOhkhS4YstFYSoxxHjxPR4KzRnIqSWUaWFvxWxMNWVgMyrZEL4+xf+T9pnrEde7qVYal3kcRXSEjtEp8lANNdA1aqIWYihCD+gJPTvaeXRenNdla8HJZw7RDzhvn/BKkH8=</latexit><latexit sha1_base64="7MtAShdCU97XGBvGDB5yo3o0icA=">AAAB8HicdVBNSwMxEM3Wr1q/qh69BIvgaclKpfZW9OKxgv2QdinZNNuGZrNLMiuU0l/hxYMiXv053vw3ptsVVPTBwOO9GWbmBYkUBgj5cAorq2vrG8XN0tb2zu5eef+gbeJUM95isYx1N6CGS6F4CwRI3k00p1EgeSeYXC38zj3XRsTqFqYJ9yM6UiIUjIKV7vow5kAHAg/KFeLW68QjVZwRUjvPSd3DnksyVFCO5qD83h/GLI24AiapMT2PJODPqAbBJJ+X+qnhCWUTOuI9SxWNuPFn2cFzfGKVIQ5jbUsBztTvEzMaGTONAtsZURib395C/MvrpRBe+DOhkhS4YstFYSoxxHjxPR4KzRnIqSWUaWFvxWxMNWVgMyrZEL4+xf+T9pnrEde7qVYal3kcRXSEjtEp8lANNdA1aqIWYihCD+gJPTvaeXRenNdla8HJZw7RDzhvn/BKkH8=</latexit>

Page 6: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

STOCHASTIC WEIGHT AVERAGING (IZMAILOV ET AL 2018)

�6

Average models at end of cycles.

✓swa<latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit>

Page 7: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

STOCHASTIC WEIGHT AVERAGING (IZMAILOV ET AL 2018)

�7

✓swa<latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit>

Average models at end of cycles.

Page 8: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

STOCHASTIC WEIGHT AVERAGING (IZMAILOV ET AL 2018)

�8

✓swa<latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit>

Average models at end of cycles.

Page 9: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

STOCHASTIC WEIGHT AVERAGING (IZMAILOV ET AL 2018)

�9

✓swa<latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit>

✓1<latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit>

✓2<latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit>

✓3<latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit>

✓4<latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit>

X = [✓i � ✓swa]i=1..s<latexit sha1_base64="zwV6122H41Kqx4DgbR59L21lWMA=">AAACDXicbZDLSsNAFIYnXmu9RV26GayCG0Migm4KRTcuK9gLpCFMptN26OTCzIlSQl7Aja/ixoUibt27822cthG09YeBj/+cw5nzB4ngCmz7y1hYXFpeWS2tldc3Nre2zZ3dpopTSVmDxiKW7YAoJnjEGsBBsHYiGQkDwVrB8Gpcb90xqXgc3cIoYV5I+hHvcUpAW7552K66HRgwIH7G85MfVPck97RTdSxL5b5ZsS17IjwPTgEVVKjum5+dbkzTkEVABVHKdewEvIxI4FSwvNxJFUsIHZI+czVGJGTKyybX5PhIO13ci6V+EeCJ+3siI6FSozDQnSGBgZqtjc3/am4KvQsv41GSAovodFEvFRhiPI4Gd7lkFMRIA6GS679iOiCSUNABlnUIzuzJ89A8tRzbcm7OKrXLIo4S2kcH6Bg56BzV0DWqowai6AE9oRf0ajwaz8ab8T5tXTCKmT30R8bHN4JWm9A=</latexit><latexit sha1_base64="zwV6122H41Kqx4DgbR59L21lWMA=">AAACDXicbZDLSsNAFIYnXmu9RV26GayCG0Migm4KRTcuK9gLpCFMptN26OTCzIlSQl7Aja/ixoUibt27822cthG09YeBj/+cw5nzB4ngCmz7y1hYXFpeWS2tldc3Nre2zZ3dpopTSVmDxiKW7YAoJnjEGsBBsHYiGQkDwVrB8Gpcb90xqXgc3cIoYV5I+hHvcUpAW7552K66HRgwIH7G85MfVPck97RTdSxL5b5ZsS17IjwPTgEVVKjum5+dbkzTkEVABVHKdewEvIxI4FSwvNxJFUsIHZI+czVGJGTKyybX5PhIO13ci6V+EeCJ+3siI6FSozDQnSGBgZqtjc3/am4KvQsv41GSAovodFEvFRhiPI4Gd7lkFMRIA6GS679iOiCSUNABlnUIzuzJ89A8tRzbcm7OKrXLIo4S2kcH6Bg56BzV0DWqowai6AE9oRf0ajwaz8ab8T5tXTCKmT30R8bHN4JWm9A=</latexit><latexit sha1_base64="zwV6122H41Kqx4DgbR59L21lWMA=">AAACDXicbZDLSsNAFIYnXmu9RV26GayCG0Migm4KRTcuK9gLpCFMptN26OTCzIlSQl7Aja/ixoUibt27822cthG09YeBj/+cw5nzB4ngCmz7y1hYXFpeWS2tldc3Nre2zZ3dpopTSVmDxiKW7YAoJnjEGsBBsHYiGQkDwVrB8Gpcb90xqXgc3cIoYV5I+hHvcUpAW7552K66HRgwIH7G85MfVPck97RTdSxL5b5ZsS17IjwPTgEVVKjum5+dbkzTkEVABVHKdewEvIxI4FSwvNxJFUsIHZI+czVGJGTKyybX5PhIO13ci6V+EeCJ+3siI6FSozDQnSGBgZqtjc3/am4KvQsv41GSAovodFEvFRhiPI4Gd7lkFMRIA6GS679iOiCSUNABlnUIzuzJ89A8tRzbcm7OKrXLIo4S2kcH6Bg56BzV0DWqowai6AE9oRf0ajwaz8ab8T5tXTCKmT30R8bHN4JWm9A=</latexit><latexit sha1_base64="zwV6122H41Kqx4DgbR59L21lWMA=">AAACDXicbZDLSsNAFIYnXmu9RV26GayCG0Migm4KRTcuK9gLpCFMptN26OTCzIlSQl7Aja/ixoUibt27822cthG09YeBj/+cw5nzB4ngCmz7y1hYXFpeWS2tldc3Nre2zZ3dpopTSVmDxiKW7YAoJnjEGsBBsHYiGQkDwVrB8Gpcb90xqXgc3cIoYV5I+hHvcUpAW7552K66HRgwIH7G85MfVPck97RTdSxL5b5ZsS17IjwPTgEVVKjum5+dbkzTkEVABVHKdewEvIxI4FSwvNxJFUsIHZI+czVGJGTKyybX5PhIO13ci6V+EeCJ+3siI6FSozDQnSGBgZqtjc3/am4KvQsv41GSAovodFEvFRhiPI4Gd7lkFMRIA6GS679iOiCSUNABlnUIzuzJ89A8tRzbcm7OKrXLIo4S2kcH6Bg56BzV0DWqowai6AE9oRf0ajwaz8ab8T5tXTCKmT30R8bHN4JWm9A=</latexit>

Page 10: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

STOCHASTIC WEIGHT AVERAGING (IZMAILOV ET AL 2018)

�10

✓swa<latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit>

✓1<latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit>

✓2<latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit>

✓3<latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit>

✓4<latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit>

X = [✓i � ✓swa]i=1..s<latexit sha1_base64="zwV6122H41Kqx4DgbR59L21lWMA=">AAACDXicbZDLSsNAFIYnXmu9RV26GayCG0Migm4KRTcuK9gLpCFMptN26OTCzIlSQl7Aja/ixoUibt27822cthG09YeBj/+cw5nzB4ngCmz7y1hYXFpeWS2tldc3Nre2zZ3dpopTSVmDxiKW7YAoJnjEGsBBsHYiGQkDwVrB8Gpcb90xqXgc3cIoYV5I+hHvcUpAW7552K66HRgwIH7G85MfVPck97RTdSxL5b5ZsS17IjwPTgEVVKjum5+dbkzTkEVABVHKdewEvIxI4FSwvNxJFUsIHZI+czVGJGTKyybX5PhIO13ci6V+EeCJ+3siI6FSozDQnSGBgZqtjc3/am4KvQsv41GSAovodFEvFRhiPI4Gd7lkFMRIA6GS679iOiCSUNABlnUIzuzJ89A8tRzbcm7OKrXLIo4S2kcH6Bg56BzV0DWqowai6AE9oRf0ajwaz8ab8T5tXTCKmT30R8bHN4JWm9A=</latexit><latexit sha1_base64="zwV6122H41Kqx4DgbR59L21lWMA=">AAACDXicbZDLSsNAFIYnXmu9RV26GayCG0Migm4KRTcuK9gLpCFMptN26OTCzIlSQl7Aja/ixoUibt27822cthG09YeBj/+cw5nzB4ngCmz7y1hYXFpeWS2tldc3Nre2zZ3dpopTSVmDxiKW7YAoJnjEGsBBsHYiGQkDwVrB8Gpcb90xqXgc3cIoYV5I+hHvcUpAW7552K66HRgwIH7G85MfVPck97RTdSxL5b5ZsS17IjwPTgEVVKjum5+dbkzTkEVABVHKdewEvIxI4FSwvNxJFUsIHZI+czVGJGTKyybX5PhIO13ci6V+EeCJ+3siI6FSozDQnSGBgZqtjc3/am4KvQsv41GSAovodFEvFRhiPI4Gd7lkFMRIA6GS679iOiCSUNABlnUIzuzJ89A8tRzbcm7OKrXLIo4S2kcH6Bg56BzV0DWqowai6AE9oRf0ajwaz8ab8T5tXTCKmT30R8bHN4JWm9A=</latexit><latexit sha1_base64="zwV6122H41Kqx4DgbR59L21lWMA=">AAACDXicbZDLSsNAFIYnXmu9RV26GayCG0Migm4KRTcuK9gLpCFMptN26OTCzIlSQl7Aja/ixoUibt27822cthG09YeBj/+cw5nzB4ngCmz7y1hYXFpeWS2tldc3Nre2zZ3dpopTSVmDxiKW7YAoJnjEGsBBsHYiGQkDwVrB8Gpcb90xqXgc3cIoYV5I+hHvcUpAW7552K66HRgwIH7G85MfVPck97RTdSxL5b5ZsS17IjwPTgEVVKjum5+dbkzTkEVABVHKdewEvIxI4FSwvNxJFUsIHZI+czVGJGTKyybX5PhIO13ci6V+EeCJ+3siI6FSozDQnSGBgZqtjc3/am4KvQsv41GSAovodFEvFRhiPI4Gd7lkFMRIA6GS679iOiCSUNABlnUIzuzJ89A8tRzbcm7OKrXLIo4S2kcH6Bg56BzV0DWqowai6AE9oRf0ajwaz8ab8T5tXTCKmT30R8bHN4JWm9A=</latexit><latexit sha1_base64="zwV6122H41Kqx4DgbR59L21lWMA=">AAACDXicbZDLSsNAFIYnXmu9RV26GayCG0Migm4KRTcuK9gLpCFMptN26OTCzIlSQl7Aja/ixoUibt27822cthG09YeBj/+cw5nzB4ngCmz7y1hYXFpeWS2tldc3Nre2zZ3dpopTSVmDxiKW7YAoJnjEGsBBsHYiGQkDwVrB8Gpcb90xqXgc3cIoYV5I+hHvcUpAW7552K66HRgwIH7G85MfVPck97RTdSxL5b5ZsS17IjwPTgEVVKjum5+dbkzTkEVABVHKdewEvIxI4FSwvNxJFUsIHZI+czVGJGTKyybX5PhIO13ci6V+EeCJ+3siI6FSozDQnSGBgZqtjc3/am4KvQsv41GSAovodFEvFRhiPI4Gd7lkFMRIA6GS679iOiCSUNABlnUIzuzJ89A8tRzbcm7OKrXLIo4S2kcH6Bg56BzV0DWqowai6AE9oRf0ajwaz8ab8T5tXTCKmT30R8bHN4JWm9A=</latexit>

✓swag ⇠ N (✓swa, XX>)<latexit sha1_base64="Tiy6ViiSF/4q/WetR7vFofQbnN0=">AAACIHicbVDJSgNBEO1xjXGLevTSGIQIEmZEiMegF08SwSyQiaGm00ma9Cx01yhhmE/x4q948aCI3vRr7CyHmPig4PFeFVX1vEgKjbb9bS0tr6yurWc2sptb2zu7ub39mg5jxXiVhTJUDQ80lyLgVRQoeSNSHHxP8ro3uBr59QeutAiDOxxGvOVDLxBdwQCN1M6VXOxzhHaiH6GXUlcLn7o+YJ+BTG7SwoydntJG4z5xMYzSk3YubxftMegicaYkT6aotHNfbidksc8DZBK0bjp2hK0EFAomeZp1Y80jYAPo8aahAfhct5Lxgyk9NkqHdkNlKkA6VmcnEvC1Hvqe6Rzdrue9kfif14yxe9FKRBDFyAM2WdSNJcWQjtKiHaE4Qzk0BJgS5lbK+qCAock0a0Jw5l9eJLWzomMXndvzfPlyGkeGHJIjUiAOKZEyuSYVUiWMPJEX8kberWfr1fqwPietS9Z05oD8gfXzCwCppBM=</latexit><latexit sha1_base64="Tiy6ViiSF/4q/WetR7vFofQbnN0=">AAACIHicbVDJSgNBEO1xjXGLevTSGIQIEmZEiMegF08SwSyQiaGm00ma9Cx01yhhmE/x4q948aCI3vRr7CyHmPig4PFeFVX1vEgKjbb9bS0tr6yurWc2sptb2zu7ub39mg5jxXiVhTJUDQ80lyLgVRQoeSNSHHxP8ro3uBr59QeutAiDOxxGvOVDLxBdwQCN1M6VXOxzhHaiH6GXUlcLn7o+YJ+BTG7SwoydntJG4z5xMYzSk3YubxftMegicaYkT6aotHNfbidksc8DZBK0bjp2hK0EFAomeZp1Y80jYAPo8aahAfhct5Lxgyk9NkqHdkNlKkA6VmcnEvC1Hvqe6Rzdrue9kfif14yxe9FKRBDFyAM2WdSNJcWQjtKiHaE4Qzk0BJgS5lbK+qCAock0a0Jw5l9eJLWzomMXndvzfPlyGkeGHJIjUiAOKZEyuSYVUiWMPJEX8kberWfr1fqwPietS9Z05oD8gfXzCwCppBM=</latexit><latexit sha1_base64="Tiy6ViiSF/4q/WetR7vFofQbnN0=">AAACIHicbVDJSgNBEO1xjXGLevTSGIQIEmZEiMegF08SwSyQiaGm00ma9Cx01yhhmE/x4q948aCI3vRr7CyHmPig4PFeFVX1vEgKjbb9bS0tr6yurWc2sptb2zu7ub39mg5jxXiVhTJUDQ80lyLgVRQoeSNSHHxP8ro3uBr59QeutAiDOxxGvOVDLxBdwQCN1M6VXOxzhHaiH6GXUlcLn7o+YJ+BTG7SwoydntJG4z5xMYzSk3YubxftMegicaYkT6aotHNfbidksc8DZBK0bjp2hK0EFAomeZp1Y80jYAPo8aahAfhct5Lxgyk9NkqHdkNlKkA6VmcnEvC1Hvqe6Rzdrue9kfif14yxe9FKRBDFyAM2WdSNJcWQjtKiHaE4Qzk0BJgS5lbK+qCAock0a0Jw5l9eJLWzomMXndvzfPlyGkeGHJIjUiAOKZEyuSYVUiWMPJEX8kberWfr1fqwPietS9Z05oD8gfXzCwCppBM=</latexit><latexit sha1_base64="Tiy6ViiSF/4q/WetR7vFofQbnN0=">AAACIHicbVDJSgNBEO1xjXGLevTSGIQIEmZEiMegF08SwSyQiaGm00ma9Cx01yhhmE/x4q948aCI3vRr7CyHmPig4PFeFVX1vEgKjbb9bS0tr6yurWc2sptb2zu7ub39mg5jxXiVhTJUDQ80lyLgVRQoeSNSHHxP8ro3uBr59QeutAiDOxxGvOVDLxBdwQCN1M6VXOxzhHaiH6GXUlcLn7o+YJ+BTG7SwoydntJG4z5xMYzSk3YubxftMegicaYkT6aotHNfbidksc8DZBK0bjp2hK0EFAomeZp1Y80jYAPo8aahAfhct5Lxgyk9NkqHdkNlKkA6VmcnEvC1Hvqe6Rzdrue9kfif14yxe9FKRBDFyAM2WdSNJcWQjtKiHaE4Qzk0BJgS5lbK+qCAock0a0Jw5l9eJLWzomMXndvzfPlyGkeGHJIjUiAOKZEyuSYVUiWMPJEX8kberWfr1fqwPietS9Z05oD8gfXzCwCppBM=</latexit>

Page 11: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

STOCHASTIC WEIGHT AVERAGING (IZMAILOV ET AL 2018)

�11

✓swa<latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit><latexit sha1_base64="L4FvxMNw/6olvznC9uH/6lbihZc=">AAAB83icbVBNS8NAEN3Ur1q/qh69LBbBU0lE0GPRi8cK9gOaUDbbSbt0swm7E6WE/g0vHhTx6p/x5r9x2+agrQ8GHu/NMDMvTKUw6LrfTmltfWNzq7xd2dnd2z+oHh61TZJpDi2eyER3Q2ZACgUtFCihm2pgcSihE45vZ37nEbQRiXrASQpBzIZKRIIztJLv4wiQ9XPzxKb9as2tu3PQVeIVpEYKNPvVL3+Q8CwGhVwyY3qem2KQM42CS5hW/MxAyviYDaFnqWIxmCCf3zylZ1YZ0CjRthTSufp7ImexMZM4tJ0xw5FZ9mbif14vw+g6yIVKMwTFF4uiTFJM6CwAOhAaOMqJJYxrYW+lfMQ042hjqtgQvOWXV0n7ou65de/+sta4KeIokxNySs6JR65Ig9yRJmkRTlLyTF7Jm5M5L86787FoLTnFzDH5A+fzB5SXkgY=</latexit>

✓1<latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit><latexit sha1_base64="UwdldS9bd+OkY0zCIsd9H8BgKNQ=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJf501654lW9OdxV4uekAjnqvfJXtx/zNEJFXDJjOr6XUJAxTYJLnJa6qcGE8TEbYsdSxSI0QTa/eOqeWaXvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUsmG4C+/vEqaF1Xfq/r3l5XaTR5HEU7gFM7BhyuowR3UoQEcFDzDK7w5xnlx3p2PRWvByWeO4Q+czx+WFJDY</latexit>

✓2<latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit><latexit sha1_base64="HdFqMHr303Ap8KrhKU4gRIidvIE=">AAAB8XicbVBNS8NAEJ34WetX1aOXYBE8laQIeix68VjBfmAbymY7bZduNmF3IpTQf+HFgyJe/Tfe/Ddu2xy09cHA470ZZuaFiRSGPO/bWVvf2NzaLuwUd/f2Dw5LR8dNE6eaY4PHMtbtkBmUQmGDBElsJxpZFEpshePbmd96Qm1ErB5okmAQsaESA8EZWemxSyMk1suq016p7FW8OdxV4uekDDnqvdJXtx/zNEJFXDJjOr6XUJAxTYJLnBa7qcGE8TEbYsdSxSI0QTa/eOqeW6XvDmJtS5E7V39PZCwyZhKFtjNiNDLL3kz8z+ukNLgOMqGSlFDxxaJBKl2K3dn7bl9o5CQnljCuhb3V5SOmGScbUtGG4C+/vEqa1YrvVfz7y3LtJo+jAKdwBhfgwxXU4A7q0AAOCp7hFd4c47w4787HonXNyWdO4A+czx+XmZDZ</latexit>

✓3<latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit><latexit sha1_base64="QPL9eaf63Job+ENvWAFFgaiKmdw=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQq6LHoxWMFW4ttKZvtpF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJFIY8rxvp7Cyura+UdwsbW3v7O6V9w+aJk41xwaPZaxbATMohcIGCZLYSjSyKJD4EIxupv7DE2ojYnVP4wS7ERsoEQrOyEqPHRoisV52PumVK17Vm8FdJn5OKpCj3it/dfoxTyNUxCUzpu17CXUzpklwiZNSJzWYMD5iA2xbqliEppvNLp64J1bpu2GsbSlyZ+rviYxFxoyjwHZGjIZm0ZuK/3ntlMKrbiZUkhIqPl8UptKl2J2+7/aFRk5ybAnjWthbXT5kmnGyIZVsCP7iy8ukeVb1vap/d1GpXedxFOEIjuEUfLiEGtxCHRrAQcEzvMKbY5wX5935mLcWnHzmEP7A+fwBmR6Q2g==</latexit>

✓4<latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit><latexit sha1_base64="RMNjLwtkHaTQHloW+Ync+0WAMA8=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69BIvgqSQi6LHoxWMF+4FtKJvttF262YTdiVBC/4UXD4p49d9489+4bXPQ1gcDj/dmmJkXJlIY8rxvp7C2vrG5Vdwu7ezu7R+UD4+aJk41xwaPZazbITMohcIGCZLYTjSyKJTYCse3M7/1hNqIWD3QJMEgYkMlBoIzstJjl0ZIrJddTnvlilf15nBXiZ+TCuSo98pf3X7M0wgVccmM6fheQkHGNAkucVrqpgYTxsdsiB1LFYvQBNn84ql7ZpW+O4i1LUXuXP09kbHImEkU2s6I0cgsezPxP6+T0uA6yIRKUkLFF4sGqXQpdmfvu32hkZOcWMK4FvZWl4+YZpxsSCUbgr/88ippXlR9r+rfX1ZqN3kcRTiBUzgHH66gBndQhwZwUPAMr/DmGOfFeXc+Fq0FJ585hj9wPn8AmqOQ2w==</latexit>

Diagonal version: ✓swa,diag ⇠ N⇣✓swa,

X

i=1

✓2i � ✓2swa

<latexit sha1_base64="QMjFHWRa3KLAsdbnu0Ru4fKf2oU=">AAACU3icbVHPSxtBGJ1dtbWxtqkeexkMggUNu0Gwl4LoxVNRMCpk4/Lt5EsydWZ3mfm2NSz7P4rgwX+kFw86iXtQ44OBx3vvmx9vklxJS0Fw7/kLi0sfPi5/aqx8Xv3ytflt7cxmhRHYFZnKzEUCFpVMsUuSFF7kBkEnCs+Tq8Opf/4XjZVZekqTHPsaRqkcSgHkpLj5J6IxEsRlRHhNpf0H2wMJo6rikZWa/44O5GhrLlNV284vdFzKX2FV2/Kyw3f4O9nLznSXH3GzFbSDGfg8CWvSYjWO4+ZtNMhEoTElocDaXhjk1C/BkBQKq0ZUWMxBXMEIe46moNH2y1knFd90yoAPM+NWSnymvpwoQVs70YlLaqCxfetNxfe8XkHDn/1SpnlBmIrng4aF4pTxacF8IA0KUhNHQBjp7srFGAwIct/QcCWEb588T8467TBohye7rf2Duo5l9p1tsC0Wsj22z47YMesywW7Yf/boMe/Oe/B9f/E56nv1zDp7BX/1CZDntW0=</latexit><latexit sha1_base64="QMjFHWRa3KLAsdbnu0Ru4fKf2oU=">AAACU3icbVHPSxtBGJ1dtbWxtqkeexkMggUNu0Gwl4LoxVNRMCpk4/Lt5EsydWZ3mfm2NSz7P4rgwX+kFw86iXtQ44OBx3vvmx9vklxJS0Fw7/kLi0sfPi5/aqx8Xv3ytflt7cxmhRHYFZnKzEUCFpVMsUuSFF7kBkEnCs+Tq8Opf/4XjZVZekqTHPsaRqkcSgHkpLj5J6IxEsRlRHhNpf0H2wMJo6rikZWa/44O5GhrLlNV284vdFzKX2FV2/Kyw3f4O9nLznSXH3GzFbSDGfg8CWvSYjWO4+ZtNMhEoTElocDaXhjk1C/BkBQKq0ZUWMxBXMEIe46moNH2y1knFd90yoAPM+NWSnymvpwoQVs70YlLaqCxfetNxfe8XkHDn/1SpnlBmIrng4aF4pTxacF8IA0KUhNHQBjp7srFGAwIct/QcCWEb588T8467TBohye7rf2Duo5l9p1tsC0Wsj22z47YMesywW7Yf/boMe/Oe/B9f/E56nv1zDp7BX/1CZDntW0=</latexit><latexit sha1_base64="QMjFHWRa3KLAsdbnu0Ru4fKf2oU=">AAACU3icbVHPSxtBGJ1dtbWxtqkeexkMggUNu0Gwl4LoxVNRMCpk4/Lt5EsydWZ3mfm2NSz7P4rgwX+kFw86iXtQ44OBx3vvmx9vklxJS0Fw7/kLi0sfPi5/aqx8Xv3ytflt7cxmhRHYFZnKzEUCFpVMsUuSFF7kBkEnCs+Tq8Opf/4XjZVZekqTHPsaRqkcSgHkpLj5J6IxEsRlRHhNpf0H2wMJo6rikZWa/44O5GhrLlNV284vdFzKX2FV2/Kyw3f4O9nLznSXH3GzFbSDGfg8CWvSYjWO4+ZtNMhEoTElocDaXhjk1C/BkBQKq0ZUWMxBXMEIe46moNH2y1knFd90yoAPM+NWSnymvpwoQVs70YlLaqCxfetNxfe8XkHDn/1SpnlBmIrng4aF4pTxacF8IA0KUhNHQBjp7srFGAwIct/QcCWEb588T8467TBohye7rf2Duo5l9p1tsC0Wsj22z47YMesywW7Yf/boMe/Oe/B9f/E56nv1zDp7BX/1CZDntW0=</latexit><latexit sha1_base64="QMjFHWRa3KLAsdbnu0Ru4fKf2oU=">AAACU3icbVHPSxtBGJ1dtbWxtqkeexkMggUNu0Gwl4LoxVNRMCpk4/Lt5EsydWZ3mfm2NSz7P4rgwX+kFw86iXtQ44OBx3vvmx9vklxJS0Fw7/kLi0sfPi5/aqx8Xv3ytflt7cxmhRHYFZnKzEUCFpVMsUuSFF7kBkEnCs+Tq8Opf/4XjZVZekqTHPsaRqkcSgHkpLj5J6IxEsRlRHhNpf0H2wMJo6rikZWa/44O5GhrLlNV284vdFzKX2FV2/Kyw3f4O9nLznSXH3GzFbSDGfg8CWvSYjWO4+ZtNMhEoTElocDaXhjk1C/BkBQKq0ZUWMxBXMEIe46moNH2y1knFd90yoAPM+NWSnymvpwoQVs70YlLaqCxfetNxfe8XkHDn/1SpnlBmIrng4aF4pTxacF8IA0KUhNHQBjp7srFGAwIct/QcCWEb588T8467TBohye7rf2Duo5l9p1tsC0Wsj22z47YMesywW7Yf/boMe/Oe/B9f/E56nv1zDp7BX/1CZDntW0=</latexit>

See also Liu et al, 2018, UDL Workshop

Page 12: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

BAYESIAN MODEL AVERAGING WITH SWAG‣ Monte Carlo estimates of predictions:

‣ Results in approximate Bayesian inference.

�12

Dataset (epochs) SGD, point SWA SGD, empirical SWAG, 30 samples

CIFAR-10 (300) 93.19± 0.22 93.44± 0.09 93.64± 0.14 93.57± 0.15CIFAR-10.1 84.93± 0.32 86.14± 0.59 85.78± 0.20 86.24± 0.67CIFAR-100 (300) 73.29± 0.38 74.04± 0.25 74.74± 0.26 74.57± 0.39

p(y⇤|y) ⇡ 1

K

KX

i=1

p(y⇤|✓i), ✓i ⇠ qSWAG(✓|y)<latexit sha1_base64="mFD65uXY6kdITXtbzipmANluiJk=">AAACRHicbVDLThsxFPXQ8gqvtF2ysRohJQhFMwgJNkiBLorEhgpCkDLJyON4Eiv2jLHvVIyG+bhu+gHd8QVsWICqbiucBxKvI1k6Oufca/uESnADrnvjzHz4ODs3v7BYWlpeWV0rf/p8bpJUU9akiUj0RUgMEzxmTeAg2IXSjMhQsFY4/DbyWz+ZNjyJzyBTrCNJP+YRpwSsFJTbqpp1N6+zGvaJUjq5wn6kCc29Ij8usG9SGeR83yu6x3iS9GHAgAS8toWfqI1xiS+D/LR18L2oTmS7MihX3Lo7Bn5LvCmpoClOgvIfv5fQVLIYqCDGtD1XQScnGjgVrCj5qWGK0CHps7alMZHMdPJxCQXesEoPR4m2JwY8Vp9P5EQak8nQJiWBgXntjcT3vHYK0V4n57FKgcV0clGUCgwJHjWKe1wzCiKzhFDN7VsxHRBbItjeS7YE7/WX35Lz7brn1r0fO5XG4bSOBbSOvqIq8tAuaqAjdIKaiKJf6Bbdowfnt3Pn/HX+TaIzznTmC3oB5/8jEP2w5w==</latexit><latexit sha1_base64="mFD65uXY6kdITXtbzipmANluiJk=">AAACRHicbVDLThsxFPXQ8gqvtF2ysRohJQhFMwgJNkiBLorEhgpCkDLJyON4Eiv2jLHvVIyG+bhu+gHd8QVsWICqbiucBxKvI1k6Oufca/uESnADrnvjzHz4ODs3v7BYWlpeWV0rf/p8bpJUU9akiUj0RUgMEzxmTeAg2IXSjMhQsFY4/DbyWz+ZNjyJzyBTrCNJP+YRpwSsFJTbqpp1N6+zGvaJUjq5wn6kCc29Ij8usG9SGeR83yu6x3iS9GHAgAS8toWfqI1xiS+D/LR18L2oTmS7MihX3Lo7Bn5LvCmpoClOgvIfv5fQVLIYqCDGtD1XQScnGjgVrCj5qWGK0CHps7alMZHMdPJxCQXesEoPR4m2JwY8Vp9P5EQak8nQJiWBgXntjcT3vHYK0V4n57FKgcV0clGUCgwJHjWKe1wzCiKzhFDN7VsxHRBbItjeS7YE7/WX35Lz7brn1r0fO5XG4bSOBbSOvqIq8tAuaqAjdIKaiKJf6Bbdowfnt3Pn/HX+TaIzznTmC3oB5/8jEP2w5w==</latexit><latexit sha1_base64="mFD65uXY6kdITXtbzipmANluiJk=">AAACRHicbVDLThsxFPXQ8gqvtF2ysRohJQhFMwgJNkiBLorEhgpCkDLJyON4Eiv2jLHvVIyG+bhu+gHd8QVsWICqbiucBxKvI1k6Oufca/uESnADrnvjzHz4ODs3v7BYWlpeWV0rf/p8bpJUU9akiUj0RUgMEzxmTeAg2IXSjMhQsFY4/DbyWz+ZNjyJzyBTrCNJP+YRpwSsFJTbqpp1N6+zGvaJUjq5wn6kCc29Ij8usG9SGeR83yu6x3iS9GHAgAS8toWfqI1xiS+D/LR18L2oTmS7MihX3Lo7Bn5LvCmpoClOgvIfv5fQVLIYqCDGtD1XQScnGjgVrCj5qWGK0CHps7alMZHMdPJxCQXesEoPR4m2JwY8Vp9P5EQak8nQJiWBgXntjcT3vHYK0V4n57FKgcV0clGUCgwJHjWKe1wzCiKzhFDN7VsxHRBbItjeS7YE7/WX35Lz7brn1r0fO5XG4bSOBbSOvqIq8tAuaqAjdIKaiKJf6Bbdowfnt3Pn/HX+TaIzznTmC3oB5/8jEP2w5w==</latexit><latexit sha1_base64="mFD65uXY6kdITXtbzipmANluiJk=">AAACRHicbVDLThsxFPXQ8gqvtF2ysRohJQhFMwgJNkiBLorEhgpCkDLJyON4Eiv2jLHvVIyG+bhu+gHd8QVsWICqbiucBxKvI1k6Oufca/uESnADrnvjzHz4ODs3v7BYWlpeWV0rf/p8bpJUU9akiUj0RUgMEzxmTeAg2IXSjMhQsFY4/DbyWz+ZNjyJzyBTrCNJP+YRpwSsFJTbqpp1N6+zGvaJUjq5wn6kCc29Ij8usG9SGeR83yu6x3iS9GHAgAS8toWfqI1xiS+D/LR18L2oTmS7MihX3Lo7Bn5LvCmpoClOgvIfv5fQVLIYqCDGtD1XQScnGjgVrCj5qWGK0CHps7alMZHMdPJxCQXesEoPR4m2JwY8Vp9P5EQak8nQJiWBgXntjcT3vHYK0V4n57FKgcV0clGUCgwJHjWKe1wzCiKzhFDN7VsxHRBbItjeS7YE7/WX35Lz7brn1r0fO5XG4bSOBbSOvqIq8tAuaqAjdIKaiKJf6Bbdowfnt3Pn/HX+TaIzznTmC3oB5/8jEP2w5w==</latexit>

Page 13: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

MODEL CALIBRATION‣ Can Bayesian methods help fix

the model calibration problem for DNNs?

‣ Expected calibration error (Naeini et al 2015)

�13

Method ECE

Laplace 0.7604SWA 0.7650SWAG-Diagonal 0.7093SWAG 0.6001

VGG16 on CIFAR100.

Page 14: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

OUT OF DISTRIBUTION UNCERTAINTY

‣ Train DNN on 5 classes of CIFAR 10 (in-class), test on in-class test set and other examples (out-of-class).

‣ How well can SWAG tell them apart?

�14

Page 15: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

CONCLUSIONS & FUTURE WORK▸ Code at: https://github.com/wjmaddox/swa_uncertainties

▸ Theory: connection to Polyak-Ruppert Averaging (see Chen et al 2016).

▸ Comparisons with other methods for approximate Bayesian inference.

▸ Laplace, Variational Bayes, etc…

▸ Adversarial attacks and defenses.

▸ Check out our poster…

�15

Page 16: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

TEXT

REFERENCES

▸ X. Chen, J. D. Lee, X. T. Tong, and Y. Zhang. Statistical Inference for Model Parameters in Stochastic Gradient Descent. arXiv: 1610.08637, Oct. 2016.

▸ P. Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov, and A. G. Wilson. Averaging Weights Leads to Wider Optima and Better Generalization. In UAI , 2018.

▸ J. Liu, S. Tripathi, U. Kurup, M. Shah, Make (Nearly) Every Neural Network Better: Generating Neural Network Ensembles by Weight Parameter Resampling. In UAI Workshop on Uncertainty in Deep Learning, 2018.

▸ M. P. Naeini, G. F. Cooper, and M. Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In AAAI , pages 2901–2907, 2015

▸ B. T. Polyak and A. B. Juditsky. Acceleration on Stochastic Approximation by Averaging. SIAM Journal on Control and Optimization , 30(4):838–855, July 1992.

▸ D. Ruppert. Efficient Estimators from a Slowly Convergent Robbins-Munro Process. Technical Report 781, Cornell University, School of Operations Research and Industrial Engineering, 1988.

�16

Page 17: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

TEXT

ASYMPTOTIC MOTIVATION OF SWAG

�17

‣ Polyak-Ruppert Averaging (Ruppert 1988; Polyak & Juditsky, 1992)

‣ Average the iterates of SGD

‣ Asymptotic distribution (around stationary point):

‣ Laplace approximation uses Gaussian around MAP with covariance H(\theta)

1T

T

∑i=1

θi ≈ N(θ, H(θ)−1SH(θ)−1)

Page 18: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

SWA WITH GAUSSIANS (SWAG): FORMULAS‣ (diagonal) Laplace: compute Hessian diagonals

‣ SWAG: diagonal approx.

‣ SWAG-LR: SWA + covariance

‣ SWAG-Hessian: compute Hessian diagonal + covariance

�18

N(θSWA, θ2 − θSWA)

N(θSWA, XX′�), Xj = (θj − θSWAj)

N (θSWA, H−1ii XX′�H−1

ii )

Page 19: FAST UNCERTAINTY ESTIMATES AND BAYESIAN MODEL … · BAYESIAN MODEL AVERAGING WITH SWAG ‣ Monte Carlo estimates of predictions: ‣ Results in approximate Bayesian inference. 12

EXPECTED CALIBRATION ERROR

�19

ECE =MX

m=1

|Bm|n

���acc(Bm)� conf(Bm)���

<latexit sha1_base64="gMn6iA2puPfmjgOWE+oRP/kaJZ0=">AAACKHicbZDLSsNAFIYnXmu9RV26GSxCXVgSEXRTLBXBjaBga6GJYTKdtENnJmFmIpQ0j+PGV3Ejoohbn8Rp7MLbgYGP/z+HM+cPE0aVdpx3a2Z2bn5hsbRUXl5ZXVu3NzbbKk4lJi0cs1h2QqQIo4K0NNWMdBJJEA8ZuQmHpxP/5o5IRWNxrUcJ8TnqCxpRjLSRAvvk7PSsDj2V8iDjdTe/vYBeJBHOxs2Aj/NM5F6T9scI46oR9uA+xLGICi6MwK44Naco+BfcKVTAtC4D+9nrxTjlRGjMkFJd10m0nyGpKWYkL3upIgnCQ9QnXYMCcaL8rDg0h7tG6cEoluYJDQv1+0SGuFIjHppOjvRA/fYm4n9eN9XRsZ9RkaSaCPy1KEoZ1DGcpAZ7VBKs2cgAwpKav0I8QCYnbbItmxDc3yf/hfZBzXVq7tVhpdGcxlEC22AHVIELjkADnINL0AIY3INH8AJerQfryXqz3r9aZ6zpzBb4UdbHJ7WapSE=</latexit><latexit sha1_base64="gMn6iA2puPfmjgOWE+oRP/kaJZ0=">AAACKHicbZDLSsNAFIYnXmu9RV26GSxCXVgSEXRTLBXBjaBga6GJYTKdtENnJmFmIpQ0j+PGV3Ejoohbn8Rp7MLbgYGP/z+HM+cPE0aVdpx3a2Z2bn5hsbRUXl5ZXVu3NzbbKk4lJi0cs1h2QqQIo4K0NNWMdBJJEA8ZuQmHpxP/5o5IRWNxrUcJ8TnqCxpRjLSRAvvk7PSsDj2V8iDjdTe/vYBeJBHOxs2Aj/NM5F6T9scI46oR9uA+xLGICi6MwK44Naco+BfcKVTAtC4D+9nrxTjlRGjMkFJd10m0nyGpKWYkL3upIgnCQ9QnXYMCcaL8rDg0h7tG6cEoluYJDQv1+0SGuFIjHppOjvRA/fYm4n9eN9XRsZ9RkaSaCPy1KEoZ1DGcpAZ7VBKs2cgAwpKav0I8QCYnbbItmxDc3yf/hfZBzXVq7tVhpdGcxlEC22AHVIELjkADnINL0AIY3INH8AJerQfryXqz3r9aZ6zpzBb4UdbHJ7WapSE=</latexit><latexit sha1_base64="gMn6iA2puPfmjgOWE+oRP/kaJZ0=">AAACKHicbZDLSsNAFIYnXmu9RV26GSxCXVgSEXRTLBXBjaBga6GJYTKdtENnJmFmIpQ0j+PGV3Ejoohbn8Rp7MLbgYGP/z+HM+cPE0aVdpx3a2Z2bn5hsbRUXl5ZXVu3NzbbKk4lJi0cs1h2QqQIo4K0NNWMdBJJEA8ZuQmHpxP/5o5IRWNxrUcJ8TnqCxpRjLSRAvvk7PSsDj2V8iDjdTe/vYBeJBHOxs2Aj/NM5F6T9scI46oR9uA+xLGICi6MwK44Naco+BfcKVTAtC4D+9nrxTjlRGjMkFJd10m0nyGpKWYkL3upIgnCQ9QnXYMCcaL8rDg0h7tG6cEoluYJDQv1+0SGuFIjHppOjvRA/fYm4n9eN9XRsZ9RkaSaCPy1KEoZ1DGcpAZ7VBKs2cgAwpKav0I8QCYnbbItmxDc3yf/hfZBzXVq7tVhpdGcxlEC22AHVIELjkADnINL0AIY3INH8AJerQfryXqz3r9aZ6zpzBb4UdbHJ7WapSE=</latexit><latexit sha1_base64="gMn6iA2puPfmjgOWE+oRP/kaJZ0=">AAACKHicbZDLSsNAFIYnXmu9RV26GSxCXVgSEXRTLBXBjaBga6GJYTKdtENnJmFmIpQ0j+PGV3Ejoohbn8Rp7MLbgYGP/z+HM+cPE0aVdpx3a2Z2bn5hsbRUXl5ZXVu3NzbbKk4lJi0cs1h2QqQIo4K0NNWMdBJJEA8ZuQmHpxP/5o5IRWNxrUcJ8TnqCxpRjLSRAvvk7PSsDj2V8iDjdTe/vYBeJBHOxs2Aj/NM5F6T9scI46oR9uA+xLGICi6MwK44Naco+BfcKVTAtC4D+9nrxTjlRGjMkFJd10m0nyGpKWYkL3upIgnCQ9QnXYMCcaL8rDg0h7tG6cEoluYJDQv1+0SGuFIjHppOjvRA/fYm4n9eN9XRsZ9RkaSaCPy1KEoZ1DGcpAZ7VBKs2cgAwpKav0I8QCYnbbItmxDc3yf/hfZBzXVq7tVhpdGcxlEC22AHVIELjkADnINL0AIY3INH8AJerQfryXqz3r9aZ6zpzBb4UdbHJ7WapSE=</latexit>

EP

h���P�Y = Y |P = p

�� p

���i

<latexit sha1_base64="T7OFUjZPqWTXY4P7nqI9Yow45XU=">AAACMXicbVDLSgMxFM3UV62vqks3wSLUhWVGBN0IpSJ0WcG+mBmGTJq2oZkHSUYo0/klN/6JuOlCEbf+hJnpLLT1Qi4n55xLco8bMiqkrs+1wtr6xuZWcbu0s7u3f1A+POqIIOKYtHHAAt5zkSCM+qQtqWSkF3KCPJeRrju5S/XuE+GCBv6jnIbE9tDIp0OKkVSUU27eO7E1RjJuJYnVoCMzbbOW5dJRNeP7CbyFfTiDuUvdwlQ9hxcwhJk7bbZTrug1PSu4CowcVEBeLaf8ag0CHHnEl5ghIUxDD6UdIy4pZiQpWZEgIcITNCKmgj7yiLDjbOMEnilmAIcBV8eXMGN/T8TIE2LqucrpITkWy1pK/qeZkRze2DH1w0gSHy8eGkYMygCm8cEB5QRLNlUAYU7VXyEeI46wVCGXVAjG8sqroHNZM/Sa8XBVqTfyOIrgBJyCKjDANaiDJmiBNsDgGbyBd/ChvWhz7VP7WlgLWj5zDP6U9v0DXAuoZA==</latexit><latexit sha1_base64="T7OFUjZPqWTXY4P7nqI9Yow45XU=">AAACMXicbVDLSgMxFM3UV62vqks3wSLUhWVGBN0IpSJ0WcG+mBmGTJq2oZkHSUYo0/klN/6JuOlCEbf+hJnpLLT1Qi4n55xLco8bMiqkrs+1wtr6xuZWcbu0s7u3f1A+POqIIOKYtHHAAt5zkSCM+qQtqWSkF3KCPJeRrju5S/XuE+GCBv6jnIbE9tDIp0OKkVSUU27eO7E1RjJuJYnVoCMzbbOW5dJRNeP7CbyFfTiDuUvdwlQ9hxcwhJk7bbZTrug1PSu4CowcVEBeLaf8ag0CHHnEl5ghIUxDD6UdIy4pZiQpWZEgIcITNCKmgj7yiLDjbOMEnilmAIcBV8eXMGN/T8TIE2LqucrpITkWy1pK/qeZkRze2DH1w0gSHy8eGkYMygCm8cEB5QRLNlUAYU7VXyEeI46wVCGXVAjG8sqroHNZM/Sa8XBVqTfyOIrgBJyCKjDANaiDJmiBNsDgGbyBd/ChvWhz7VP7WlgLWj5zDP6U9v0DXAuoZA==</latexit><latexit sha1_base64="T7OFUjZPqWTXY4P7nqI9Yow45XU=">AAACMXicbVDLSgMxFM3UV62vqks3wSLUhWVGBN0IpSJ0WcG+mBmGTJq2oZkHSUYo0/klN/6JuOlCEbf+hJnpLLT1Qi4n55xLco8bMiqkrs+1wtr6xuZWcbu0s7u3f1A+POqIIOKYtHHAAt5zkSCM+qQtqWSkF3KCPJeRrju5S/XuE+GCBv6jnIbE9tDIp0OKkVSUU27eO7E1RjJuJYnVoCMzbbOW5dJRNeP7CbyFfTiDuUvdwlQ9hxcwhJk7bbZTrug1PSu4CowcVEBeLaf8ag0CHHnEl5ghIUxDD6UdIy4pZiQpWZEgIcITNCKmgj7yiLDjbOMEnilmAIcBV8eXMGN/T8TIE2LqucrpITkWy1pK/qeZkRze2DH1w0gSHy8eGkYMygCm8cEB5QRLNlUAYU7VXyEeI46wVCGXVAjG8sqroHNZM/Sa8XBVqTfyOIrgBJyCKjDANaiDJmiBNsDgGbyBd/ChvWhz7VP7WlgLWj5zDP6U9v0DXAuoZA==</latexit><latexit sha1_base64="T7OFUjZPqWTXY4P7nqI9Yow45XU=">AAACMXicbVDLSgMxFM3UV62vqks3wSLUhWVGBN0IpSJ0WcG+mBmGTJq2oZkHSUYo0/klN/6JuOlCEbf+hJnpLLT1Qi4n55xLco8bMiqkrs+1wtr6xuZWcbu0s7u3f1A+POqIIOKYtHHAAt5zkSCM+qQtqWSkF3KCPJeRrju5S/XuE+GCBv6jnIbE9tDIp0OKkVSUU27eO7E1RjJuJYnVoCMzbbOW5dJRNeP7CbyFfTiDuUvdwlQ9hxcwhJk7bbZTrug1PSu4CowcVEBeLaf8ag0CHHnEl5ghIUxDD6UdIy4pZiQpWZEgIcITNCKmgj7yiLDjbOMEnilmAIcBV8eXMGN/T8TIE2LqucrpITkWy1pK/qeZkRze2DH1w0gSHy8eGkYMygCm8cEB5QRLNlUAYU7VXyEeI46wVCGXVAjG8sqroHNZM/Sa8XBVqTfyOIrgBJyCKjDANaiDJmiBNsDgGbyBd/ChvWhz7VP7WlgLWj5zDP6U9v0DXAuoZA==</latexit>

From Naeini et al 2015, also Guo et al ICML 2017 “On Calibration of Modern Neural Networks”