One Laptop Per Child - BSDCan Laptop Per Child BSDCan May 2007 Andrew Clunis
Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed...
Transcript of Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed...
![Page 1: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/1.jpg)
Reproducible Builds in FreeBSD
BSDCan 2016 (Ottawa, Canada)2016-06-11
![Page 2: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/2.jpg)
About Me
I FreeBSD user since early 2000sI FreeBSD committer since 2005I FreeBSD Foundation since 2011I Reproducible builds since 2015
![Page 3: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/3.jpg)
About You
I BSD src contributor or committer?I BSD ports maintainer or committer?I Contributed to Free / Open Source Software?I Have heard about reproducible builds?I Have worked on making software reproducible?
![Page 4: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/4.jpg)
About You
I BSD src contributor or committer?
I BSD ports maintainer or committer?I Contributed to Free / Open Source Software?I Have heard about reproducible builds?I Have worked on making software reproducible?
![Page 5: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/5.jpg)
About You
I BSD src contributor or committer?I BSD ports maintainer or committer?
I Contributed to Free / Open Source Software?I Have heard about reproducible builds?I Have worked on making software reproducible?
![Page 6: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/6.jpg)
About You
I BSD src contributor or committer?I BSD ports maintainer or committer?I Contributed to Free / Open Source Software?
I Have heard about reproducible builds?I Have worked on making software reproducible?
![Page 7: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/7.jpg)
About You
I BSD src contributor or committer?I BSD ports maintainer or committer?I Contributed to Free / Open Source Software?I Have heard about reproducible builds?
I Have worked on making software reproducible?
![Page 8: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/8.jpg)
About You
I BSD src contributor or committer?I BSD ports maintainer or committer?I Contributed to Free / Open Source Software?I Have heard about reproducible builds?I Have worked on making software reproducible?
![Page 9: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/9.jpg)
Reproducible Builds
Build software twice
and get the same result
![Page 10: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/10.jpg)
Reproducible Builds
Build software twiceand get the same result
![Page 11: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/11.jpg)
Reproducible Builds
Build binary artifacts twiceand get the same result
![Page 12: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/12.jpg)
Source code
![Page 13: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/13.jpg)
Source code
Binary artifact
Build process
![Page 14: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/14.jpg)
![Page 15: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/15.jpg)
=
![Page 16: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/16.jpg)
=
![Page 17: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/17.jpg)
=
=
![Page 18: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/18.jpg)
Why?
I Software Integrity / AssuranceI Practical Reasons
![Page 19: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/19.jpg)
Practical Reasons
I Reduce package repository storage size (keepingall builds)
I Reduce bandwidth when mirroring, updatingI Create smallest binary patchesI Allow retroactive debug data creation
I Accurate exp-runs – track:I Changing toolchain componentsI Packages impacted by a change in a headers or
macrosI Packages using static librariesI Improved packaging Q/A
![Page 20: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/20.jpg)
Practical Reasons
I Reduce package repository storage size (keepingall builds)
I Reduce bandwidth when mirroring, updatingI Create smallest binary patchesI Allow retroactive debug data creationI Accurate exp-runs
– track:I Changing toolchain componentsI Packages impacted by a change in a headers or
macrosI Packages using static librariesI Improved packaging Q/A
![Page 21: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/21.jpg)
Practical Reasons
I Reduce package repository storage size (keepingall builds)
I Reduce bandwidth when mirroring, updatingI Create smallest binary patchesI Allow retroactive debug data creationI Accurate exp-runs – track:
I Changing toolchain componentsI Packages impacted by a change in a headers or
macrosI Packages using static librariesI Improved packaging Q/A
![Page 22: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/22.jpg)
Software Assurance
“Reproducible builds are a set of softwaredevelopment practices which create a verifiablepath from human readable source code to the
binary code used by computers.”
https://reproducible-builds.org/
![Page 23: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/23.jpg)
Software Assurance
Can be verified
Can be used
![Page 24: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/24.jpg)
XCode Malware
![Page 25: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/25.jpg)
31c3 Perry and Schoen
https://media.cc.de "Moving Beyond Single Points of Failure"
![Page 26: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/26.jpg)
Reflections on Trusting Trust
![Page 27: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/27.jpg)
Who’s Involved
![Page 28: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/28.jpg)
Who’s Involved
![Page 29: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/29.jpg)
Who’s InvolvedI F-DroidI BitcoinI TorI SignalI OpenSUSEI UbuntuI GuixI NixOSI ElectroBSDI QubesI TAILSI Subgraph
![Page 30: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/30.jpg)
Components of Reproducible Builds
I Deterministic build systemI Reproducible build environmentI Distributing the build environmentI Rebuilding and checking the results
![Page 31: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/31.jpg)
Deterministic build system
I Stable inputsI Stable outputsI Capture as little as possible from the
environment
![Page 32: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/32.jpg)
Sources of nonreproducibility
I Embedded build informationI Input file ordering (filesystem, locale)I Archive metadataI Unstable output ordering (e.g. hashes)I Intentional randomnessI DWARF debug info pathsI Threaded producersI OptimizationsI Value initializationI Embedded signatures
![Page 33: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/33.jpg)
VariationsI hostname, domainnameI Environment: TZ, LANG, LC_ALL, USERI Timestamp, Y/M/D H:M:S
I Year or dateI uid, gidI Kernel versionI 32- or 64-bit kernelI shellI umaskI CPU typeI filesystem
![Page 34: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/34.jpg)
VariationsI hostname, domainnameI Environment: TZ, LANG, LC_ALL, USERI Timestamp, Y/M/D H:M:SI Year or dateI uid, gidI Kernel versionI 32- or 64-bit kernelI shellI umaskI CPU typeI filesystem
![Page 35: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/35.jpg)
Reproducible FreeBSD
I BaseI Ports
I Doc
![Page 36: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/36.jpg)
Reproducible FreeBSD
I BaseI PortsI Doc
![Page 37: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/37.jpg)
FreeBSD base
I Under our controlI Almost done
I ReproducibleBuilds wiki page created in 2013I Prompted by FreeBSD-update
![Page 38: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/38.jpg)
FreeBSD base - fixed
I Build date in /usr/include/osreldate.hI Build host and user in /usr/sbin/amdI Build date and time in /usr/sbin/bhyveI Build date and time in /etc/mail/*.cfI Build date in/usr/share/doc/psd/13.rcs/paper.ascii.gz
![Page 39: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/39.jpg)
FreeBSD base - TODO
Build host, user, path and time in/var/db/mergemaster.mtree
# user: emaste# machine: example# tree: /var/tmp/temproot.fFki3iM9# date: Sun Apr 10 12:19:52 2016
# ./set type=file. type=dir
aliases type=linkamd.map size=208 md5digest=e24ec9e1b9da870742a17669e69309a6apmd.conf size=1233 md5digest=ad61867e7088f15356ce7a123b909859auth.conf size=230 md5digest=5ced0a5986b19b6e60dcf25ca6d860b0crontab size=723 md5digest=26d10036869afb3fd0569d9c09d44c4ccsh.cshrc size=106 md5digest=0fb9d8e625dcdaa81f70ee308c8135d6...
![Page 40: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/40.jpg)
FreeBSD base - TODO
Build user, date and time in /boot/loader/loaderand other loaders
BTX loader 1.00 BTX version is 1.02Consoles: internal video/keyboardBIOS drive C: is disk0BIOS 638kB/1046464KB available memory
FreeBSD/i386 bootstrap loader, Revision 1.1([email protected], Thu Jan 1 09:55:10 UTC 2009)Loading /boot/defaults/loader.conf
![Page 41: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/41.jpg)
FreeBSD base - TODO
Build user, date and time in /boot/kernel/kernel
% uname -vFreeBSD 9.1-RELEASE #0 r243825: Tue Dec 4 09:23:10 UTC 2012root at farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
![Page 42: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/42.jpg)
FreeBSD base - TODO
I Full paths in non-debug sections in kernelmodules
I Other sporadic kernel module differencesI Date or other metadata in filesystem image
produced by makefs in/tests/sys/geom/class/uzip
I makewhatis output depends on man page deviceand inode numbers
![Page 43: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/43.jpg)
FreeBSD ports
I Ports do not enforce build environment(user, host name, path, ...)
I Variation good for identifying nonreproducibilityI We want to facilitate reproducibilityI Poudrière
![Page 44: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/44.jpg)
FreeBSD packages
I PortsReproducibleBuilds wiki page created inMarch 2015 (swills@)
I Initial patch sets stage dir timestamps to that ofthe newest distfile
I Builds vary the hostname, time, and dateI 15164 of 23599 packages reproducible (64.25%)
![Page 45: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/45.jpg)
FreeBSD packages
I Second iteration by bapt@I Record timestamp in make makesum during port
updateI Ports r415078 by emaste@
I Use timestamp for pkg archive metadataI Use timestamp as SOURCE_DATE_EPOCH in build
environment
![Page 46: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/46.jpg)
Use timestamp for pkg archive metadataNon-reproducible 5162Reproducible 20009Failed 4
I Example non-reproducible packages:GraphicsMagick, R-* (181), ansible, apache-openoffice, apache24,aspell, autoconf, automake, avrdude, bind910, busybox, clamav,cmake, couchdb, courier, crashmail, cyrus-imapd25, dbus, distcc,dpkg, elixir-* (61), erlang-* (63), exim, fpc-* (90), ficl, firefox,gcc, git, go, hadoop2, inkscape, kBuild, kde-runtime, lastpass-cli,libav, libdjbdns, libgcrypt, libgpg-error, libidn, liblz4, libtool, libxul,llvm38, m4, mongodb, node, octave-* (91), openjdk,openldap-server, openvpn, owncloudclient, p5-* (747), php56,py27-* (195), python27, python34, qemu, ruby, rubygem-* (1175),samba44, squid, subversion, tcl86, tex-luatex, thunderbird,u-boot-* (11), valgrind, virtualbox-ose, vlc, wine, yacc, yasm
![Page 47: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/47.jpg)
Set SOURCE_DATE_EPOCH in buildenvironment
Non-reproducible 3534Reproducible 5062Failed 4
I Example packages of 116 more nowreproducible:calibre, cherivis-devel, cmake, easydiff, freetar, gforth, gnumail,gnustep-wrapper, kbreakout, net-snmp, mongodb32, terminal.app
![Page 48: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/48.jpg)
Set SOURCE_DATE_EPOCH in buildenvironment + Clang patch
Non-reproducible 4011Reproducible 4549Failed 5
I Example packages of 514 more nowreproducible:abiword, analog, apache22, audacity, avrdude, bind99, bind910,clamav, crashmail, ctags, distcc, efax, exim, inkscape, libcaca,liblz4, llvm-cheri128, nagios4, nasm, rrdtool, subversion, x264,xchat, unzip
![Page 49: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/49.jpg)
SOURCE_DATE_EPOCHD2032 Stock pkg +build env +Clang
Non-reproducible 15164 25222 5162 3534 4011Reproducible 8435 0 20009 5062 4549Failed 824 875 4 5Reproducible % 64.26% 0% 79.49% 79.89% 81.92%
![Page 50: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/50.jpg)
FreeBSD packages
![Page 51: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/51.jpg)
Investigating Changes
![Page 52: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/52.jpg)
What’s changed?
I 1: nginx-1.10.0_3,2.txz bb31ba88b568...
I 2: nginx-1.10.0_3,2.txz bb31ba88b568...
I 1: avrdude-6.1_1.txz 0a3811a78a03...
I 2: avrdude-6.1_1.txz 7336a6d85dd6...
![Page 53: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/53.jpg)
What’s changed?
I 1: nginx-1.10.0_3,2.txz bb31ba88b568...
I 2: nginx-1.10.0_3,2.txz bb31ba88b568...
I 1: avrdude-6.1_1.txz 0a3811a78a03...
I 2: avrdude-6.1_1.txz 7336a6d85dd6...
![Page 54: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/54.jpg)
What’s changed?
I 1: nginx-1.10.0_3,2.txz bb31ba88b568...
I 2: nginx-1.10.0_3,2.txz bb31ba88b568...
I 1: avrdude-6.1_1.txz 0a3811a78a03...
I 2: avrdude-6.1_1.txz 7336a6d85dd6...
![Page 55: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/55.jpg)
Diffoscope
I Examine differences in depthI Output HTML or plain textI Recursively unpack archivesI Human readable output
I Uncompress .PDF, disassemble binaries
I https://diffoscope.orgI FreeBSD port sysutils/diffoscopeI Online version https://try.diffoscope.org
![Page 56: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/56.jpg)
Diffoscope
I ArchivesI bzip2, .cpio, .deb, gzip, .ipk, iso9660, RPM,
squashfs, .tar, .xz, .zipI Formats
I Debian .changes, TrueType & OpenType fonts,gettext .mo, .class, Mono .exe, PDF, PNG,sqlite3 databases, text files
I Maintainers wanted!
![Page 57: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/57.jpg)
Diffoscope
I ArchivesI bzip2, .cpio, .deb, gzip, .ipk, iso9660, RPM,
squashfs, .tar, .xz, .zipI Formats
I Debian .changes, TrueType & OpenType fonts,gettext .mo, .class, Mono .exe, PDF, PNG,sqlite3 databases, text files
I Maintainers wanted!
![Page 58: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/58.jpg)
Nonreproducible binaryExample
% cat hello.c#include <stdio.h>
intmain(int argc, char *argv[]){
puts("Hello, World compiled at "__TIME__ " on " __DATE__ "\n");
}% cc -o hello_1 hello.c% cc -o hello_2 hello.c
![Page 59: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/59.jpg)
Diffoscope output
![Page 60: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/60.jpg)
Diffoscope archives
Example
% cc -o hello hello.c% tar -cJxf hello_1.txz hello% cc -o hello hello.c% tar -cJxf hello_2.txz hello
![Page 61: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/61.jpg)
Diffoscope HTML output
![Page 62: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/62.jpg)
Diffoscope
I Diffoscope is a debugging / diagnostic tool
I “Reproducible” means bit-for-bit identicalI cmp / diff / sha256 test for reproducibility
![Page 63: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/63.jpg)
Diffoscope
I Diffoscope is a debugging / diagnostic toolI “Reproducible” means bit-for-bit identicalI cmp / diff / sha256 test for reproducibility
![Page 64: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/64.jpg)
Fixing nonreproducibility
![Page 65: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/65.jpg)
Stable order for inputs
I Process inputs in the same orderI Directory order is not stable!
I Solutions:I List inputs explicitly
I Sort explicitly
Exampletar -cf archive.tar src
\src/util.c src/helper.c src/main.c
![Page 66: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/66.jpg)
Stable order for inputs
I Process inputs in the same orderI Directory order is not stableI Solutions:
I List inputs explicitly
I Sort explicitly
Exampletar -cf archive.tar \
src/util.c src/helper.c src/main.c
![Page 67: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/67.jpg)
Stable order for inputs
I Process inputs in the same orderI Directory order is not stableI Solutions:
I List inputs explicitlyI Sort explicitly
Examplefind src -print0 | sort -z |
tar –null -T - –no-recursion -cf archive.tar
![Page 68: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/68.jpg)
Stable order for inputs
I Process inputs in the same orderI Directory order is not stableI Solutions:
I List inputs explicitlyI Sort explicitly with explicit locale
Examplefind src -print0 | LC_ALL=C sort -z |
tar –null -T - –no-recursion -cf archive.tar
![Page 69: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/69.jpg)
Deterministic version information
I Don’t generate a version number on each buildI Extract information from the source:
I Version control system revisionI Hash of the source codeI Changelog entry
Example (newvers.sh)
svn=‘cd ${SYSDIR} && $svnversion 2>/dev/null‘...#define VERSTR "...${svn}${git}${hg}..."
![Page 70: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/70.jpg)
Deterministic version information
I Don’t generate a version number on each buildI Extract information from the source:
I Version control system revisionI Hash of the source codeI Changelog entry
Example (newvers.sh)
svn=‘cd ${SYSDIR} && $svnversion 2>/dev/null‘...#define VERSTR "...${svn}${git}${hg}..."
![Page 71: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/71.jpg)
Eliminate build information
I "Compiled by emaste on 2 May 2016 at14:12:03"
I Just don’t record build metadata:I Date and timeI User nameI PathI Hostname
I If the build is reproducible thisinformation does not matter
![Page 72: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/72.jpg)
Eliminate build information
I "Compiled by emaste on 2 May 2016 at14:12:03"
I Just don’t record build metadata:I Date and timeI User nameI PathI Hostname
I If the build is reproducible thisinformation does not matter
![Page 73: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/73.jpg)
Eliminate build information
I "Compiled by emaste on 2 May 2016 at14:12:03"
I Just don’t record build metadata:I Date and timeI User nameI PathI Hostname
I If the build is reproducible thisinformation does not matter
![Page 74: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/74.jpg)
Don’t record the current date and time
I Avoid timestamps
I But if one is needed,I Record a suitable timestamp in the buildI Use date of last commit in VCSI Extract from changelogI Implement SOURCE_DATE_EPOCH
![Page 75: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/75.jpg)
Don’t record the current date and time
I Avoid timestampsI But if one is needed,
I Record a suitable timestamp in the buildI Use date of last commit in VCSI Extract from changelogI Implement SOURCE_DATE_EPOCH
![Page 76: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/76.jpg)
SOURCE_DATE_EPOCH
I Environment variable with a reference time touse instead of “current” time
I Number of seconds since UNIX Epoch(1970-01-01 00:00:00 UTC)
I Specification availableI Implemented in help2man, Epydoc, Doxygen,
text2man, Ghostscript, groff, texlive, GCC,maven, ant, u-boot, vgabios, ...
I Clang patch in progress
![Page 77: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/77.jpg)
Don’t record the current time (cont’d)I Archive metadata inclues modification timesI Storing a file can record build time
I Solutions:I Store an arbitrary value
I Pre-process file modification timeI Post-process archive
Example
tar -cf pkg.tar build
![Page 78: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/78.jpg)
Don’t record the current time (cont’d)I Archive metadata inclues modification timesI Storing a file can record build timeI Solutions:
I Store an arbitrary value
I Pre-process file modification timeI Post-process archive
Example
tar –mtime=’2015-08-13 00:00Z’ -cf pkg.tar build
![Page 79: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/79.jpg)
Don’t record the current time (cont’d)I Archive metadata inclues modification timesI Storing a file can record build timeI Solutions:
I Store an arbitrary value
I Pre-process file modification timeI Post-process archive
Example
tar –mtime=’2015-08-13 00:00Z’ -cf pkg.tar buildbsdtar does not support –mtime
![Page 80: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/80.jpg)
Don’t record the current time (cont’d)I Archive metadata inclues modification timesI Storing a file can record build timeI Solutions:
I Store an arbitrary valueI Pre-process file modification time
I Post-process archive
Example
touch –date="2015-08-13 00:00Z" build/* (GNU)tar -cf pkg.tar build
![Page 81: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/81.jpg)
Don’t record the current time (cont’d)I Archive metadata inclues modification timesI Storing a file can record build timeI Solutions:
I Store an arbitrary valueI Pre-process file modification time
I Post-process archive
Example
touch -d "2016-06-11 00:00:00Z" build/*tar -cf pkg.tar build
![Page 82: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/82.jpg)
Don’t record the current time (cont’d)I Archive metadata inclues modification timesI Storing a file can record build timeI Solutions:
I Store an arbitrary valueI Pre-process file modification timeI Post-process archive
Example
# zip has no equivalent of –mtimezip pkg.zip buildstrip-nondeterminism pkg.zip
![Page 83: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/83.jpg)
Explicitly set environment variables
I Some environment variables affect build output:I LC_CTIME (time strings)I LC_CTYPE (text encoding)I TZ (timezone)
I Set them to a controlled valueI Please don’t override LANG (upstream)
![Page 84: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/84.jpg)
Explicitly set environment variables
I Some environment variables affect build output:I LC_CTIME (time strings)I LC_CTYPE (text encoding)I TZ (timezone)
I Set them to a controlled value
I Please don’t override LANG (upstream)
![Page 85: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/85.jpg)
Explicitly set environment variables
I Some environment variables affect build output:I LC_CTIME (time strings)I LC_CTYPE (text encoding)I TZ (timezone)
I Set them to a controlled valueI Please don’t override LANG (upstream)
![Page 86: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/86.jpg)
Stable order for outputs
I Output lists in consistent orderI Typical issue: key order with hash tables
perldoc.perl.org/perlsec.html#Algorithmic-Complexity-Attacks
I Explicitly sort
Example
for module in dependencies.keys():version = dependencies[module]print(’%s (>= %s)’ % (module, version))
![Page 87: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/87.jpg)
Stable order for outputs
I Output lists in consistent orderI Typical issue: key order with hash tables
perldoc.perl.org/perlsec.html#Algorithmic-Complexity-Attacks
I Explicitly sort
Example
for module in sorted(dependencies.keys()):version = dependencies[module]print(’%s (>= %s)’ % (module, version))
![Page 88: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/88.jpg)
Avoid true randomness
Sources of randomness:I Temporary file namesI Generated UUIDsI Filesystem imagesI Protection against complexity attacksI LTO (symbol names)I Unique stamps in coverage data files
![Page 89: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/89.jpg)
Avoid true randomness
I Compilers use a PRNG
I Seed with a known valueI Use a fixed value
I Extract from source code (filename, content hash)I Have make provide it
Example
$ gcc -flto -c utils.c$ nm -a utils.o | grep inline0000000000000000 n .gnu.lto_.inline.381a277a0b6d
![Page 90: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/90.jpg)
Avoid true randomness
I Compilers use a PRNGI Seed with a known value
I Use a fixed value
I Extract from source code (filename, content hash)I Have make provide it
Example
$ gcc -flto -c -frandom-seed=0 utils.c$ nm -a utils.o | grep inline0000000000000000 n .gnu.lto_.inline.0
![Page 91: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/91.jpg)
Avoid true randomness
I Compilers use a PRNGI Seed with a known value
I Use a fixed valueI Extract from source code (filename, content hash)
I Have make provide it
Example
$ gcc -flto -c -frandom-seed=utils.o utils.c$ nm -a utils.o | grep inline0000000000000000 n .gnu.lto_.inline.a108e942
![Page 92: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/92.jpg)
Avoid true randomness
I Compilers use a PRNGI Seed with a known value
I Use a fixed valueI Extract from source code (filename, content hash)I Have make provide it
Example
$ gcc -flto -c -frandom-seed=${VAR:hash} utils.c$ nm -a utils.o | grep inline0000000000000000 n .gnu.lto_.inline.17b6a3ee
![Page 93: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/93.jpg)
Volatile inputs can change or disappear
I npm left-padnpm ERR! 404 Registry returned 404 for GET on
https://registry.npmjs.org/left-pad
I Don’t rely on the network (at all)I But if you must,
I Verify content using a cryptographic checksumI Have a backup under your control
I Like FreeBSD ports since r5390 (1995)
![Page 94: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/94.jpg)
Volatile inputs can change or disappear
I npm left-padnpm ERR! 404 Registry returned 404 for GET on
https://registry.npmjs.org/left-pad
I Don’t rely on the network (at all)
I But if you must,I Verify content using a cryptographic checksumI Have a backup under your control
I Like FreeBSD ports since r5390 (1995)
![Page 95: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/95.jpg)
Volatile inputs can change or disappear
I npm left-padnpm ERR! 404 Registry returned 404 for GET on
https://registry.npmjs.org/left-pad
I Don’t rely on the network (at all)I But if you must,
I Verify content using a cryptographic checksumI Have a backup under your control
I Like FreeBSD ports since r5390 (1995)
![Page 96: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/96.jpg)
Controlled value initialization
I Don’t record memory by accident
I Always initialize to a known value
Example
static int write_binary(FILE *out, FILE *in, struct bimg_header *hdr){
static uint8_t file_buf[MAX_RECORD_BYTES];struct bimg_data_header data_hdr;size_t n_written;
data_hdr.dest_addr = hdr->entry_addr;
![Page 97: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/97.jpg)
Controlled value initialization
I Don’t record memory by accidentI Always initialize to a known value
Example
static int write_binary(FILE *out, FILE *in, struct bimg_header *hdr){
static uint8_t file_buf[MAX_RECORD_BYTES];struct bimg_data_header data_hdr = { 0 };size_t n_written;
data_hdr.dest_addr = hdr->entry_addr;
![Page 98: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/98.jpg)
Reproducing the build environment
I Build tools and versionsI Build architectureI Operating systemI Build pathI Build date and timeI ...
I FreeBSD base system provides a shortcut
![Page 99: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/99.jpg)
Reproducing the build environment
I Build tools and versionsI Build architectureI Operating systemI Build pathI Build date and timeI ...I FreeBSD base system provides a shortcut
![Page 100: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/100.jpg)
Distributing the build environment
I Fetch and build known toolchainI Integrated toolchain sourceI Packaged toolchainI GitianI ContainersI VM imagesI ...
I FreeBSD base system provides a shortcut
![Page 101: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/101.jpg)
Distributing the build environment
I Fetch and build known toolchainI Integrated toolchain sourceI Packaged toolchainI GitianI ContainersI VM imagesI ...I FreeBSD base system provides a shortcut
![Page 102: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/102.jpg)
Debian’s Experience - CI
![Page 103: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/103.jpg)
Debian’s Experience - Bugs
![Page 104: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/104.jpg)
Debian’s Experience - Buy-in
I Provide good arguments on why reproduciblebuilds matter
I ROI on security-related work not always apparent
I Continuous integration to track progressI Reproducibility bugs come with patches
![Page 105: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/105.jpg)
Debian’s Experience - Policy
I Debian Policy ManualI Structure and contents of the Debian archiveI Design issues of the operating systemI Technical requirements packages must satisfy to be
included
I Section 4.15:“Source must build in a reproducible manner”
![Page 106: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/106.jpg)
Debian’s Experience - Policy
I Debian Policy ManualI Structure and contents of the Debian archiveI Design issues of the operating systemI Technical requirements packages must satisfy to be
included
I Proposed section 4.15:“Source must build in a reproducible manner”
![Page 107: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/107.jpg)
OK - Now what?
I Builders / RebuildersI Distribution of reproduction resultsI User interface
![Page 108: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/108.jpg)
Thank you
![Page 109: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/109.jpg)
Links
I Reproducible Builds https://reproducible-builds.org/
I Diffoscope https://diffoscope.org/
I FreeBSD Reproducible Builds wikihttps://wiki.freebsd.org/ReproducibleBuilds
I FreeBSD Reproducible Ports wikihttps://wiki.freebsd.org/PortsReproducibleBuilds
I Debian Reproducible Builds wikihttps://wiki.debian.org/ReproducibleBuilds
I Diverse Double-Compilationhttp://www.dwheeler.com/trusting-trust/
![Page 110: Reproducible Builds in FreeBSD - BSDCan 2018 · 2018-05-13 · Reproducible Builds in FreeBSD Ed Maste emaste@freebsd.org BSDCan 2016 (Ottawa, Canada) 2016-06-11](https://reader030.fdocuments.us/reader030/viewer/2022040904/5e77b3cf0a64ba7af829be57/html5/thumbnails/110.jpg)
Copyright c© 2016Ed Maste [email protected].
Copyright c© 2014–2016Holger Levsen [email protected] and others.
Copyright of images included in this document are held by their respective owners.
This work is licensed under the Creative Commons Attribution-Share Alike 3.0 License. To view a copyof this license, visit http://creativecommons.org/licenses/by-sa/3.0/ or send a letter to CreativeCommons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.