aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change!...
Transcript of aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change!...
![Page 1: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/1.jpg)
aptly - Debian repository management tool
Andrey Smirnov,DevOps Meetup Moscow Jan’14
![Page 2: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/2.jpg)
Repeatability!
![Page 3: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/3.jpg)
Change!
![Page 4: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/4.jpg)
Environment
![Page 5: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/5.jpg)
Repeatable Environment
![Page 6: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/6.jpg)
Environment Change
![Page 7: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/7.jpg)
Repetable Environment Change
![Page 8: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/8.jpg)
Environment is a set of packages
![Page 9: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/9.jpg)
Environment Versions
testing
development
staging
production
![Page 10: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/10.jpg)
Environment Control
apt-get
apt-sources
apt-preferences
Debianrepo
backports
3rd-partyrepo
![Page 11: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/11.jpg)
Wrong Version!
![Page 12: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/12.jpg)
apt pinning should help!
![Page 13: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/13.jpg)
No…
![Page 14: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/14.jpg)
Our Goal
• Fixed set of packages (repeatability)
• Update packet versions in controlled way (change)
![Page 15: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/15.jpg)
aptly* isn’t a replacement for apt, dpkg!
![Page 16: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/16.jpg)
Core Conceptsmirror – remote repository mirror
snapshot – immutable set of packages
publishedrepo – snapshot published as package repo
![Page 17: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/17.jpg)
snapshot
mirror
publishedrepo
take snapshot
localrepo
update
take snapshot snapshot
snapshot
merge
snapshotpull
publish
diff
verify
![Page 18: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/18.jpg)
aptly mirror
$ aptly mirror create wheezy-main http://mirror.yandex.ru/debian/ wheezy mainDownloading http://mirror.yandex.ru/debian/dists/wheezy/Release...!Mirror [mirror1]: http://mirror.yandex.ru/debian/ wheezy successfully added.You can run 'aptly mirror update mirror1' to download repository contents.
![Page 19: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/19.jpg)
aptly mirror$ aptly mirror update wheezy-mainDownloading http://mirror.yandex.ru/debian/dists/wheezy/Release...Downloading & parsing package files...Downloading http://mirror.yandex.ru/debian/dists/wheezy/main/binary-i386/Packages.bz2...Downloading http://mirror.yandex.ru/debian/dists/wheezy/main/binary-amd64/Packages.bz2...Saving packages to database...Building download queue...Download queue: 0 items, 0.00 GiB size!Mirror `wheezy-main` has been successfully updated.
![Page 20: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/20.jpg)
mirror
update
![Page 21: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/21.jpg)
aptly snapshot
$ aptly snapshot create deb-main-250114 from mirror wheezy-main!Snapshot deb-main-250114 successfully created.You can run 'aptly publish snapshot deb-main-250114' to publish snapshot as Debian repository.
![Page 22: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/22.jpg)
snapshot
mirror take snapshot
update
![Page 23: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/23.jpg)
aptly snapshot pull$ aptly snapshot pull deb-main-250114 percona-250114 deb-w-percona 'percona-server-server (>=5.5)’…[+] percona-server-server-5.5.35-rel33.0-611.wheezy_amd64 added[+] percona-server-server-5.5-5.5.35-rel33.0-611.wheezy_amd64 added[+] percona-server-client-5.5-5.5.35-rel33.0-611.wheezy_amd64 added[+] percona-server-common-5.5-5.5.35-rel33.0-611.wheezy_amd64 added[-] libmysqlclient18-5.5.31+dfsg-0+wheezy1_amd64 removed[+] libmysqlclient18-5.5.35-rel33.0-611.wheezy_amd64 added[+] percona-server-server-5.5.35-rel33.0-611.wheezy_i386 added[+] percona-server-server-5.5-5.5.35-rel33.0-611.wheezy_i386 added[+] percona-server-client-5.5-5.5.35-rel33.0-611.wheezy_i386 added[+] percona-server-common-5.5-5.5.35-rel33.0-611.wheezy_i386 added[-] libmysqlclient18-5.5.31+dfsg-0+wheezy1_i386 removed[+] libmysqlclient18-5.5.35-rel33.0-611.wheezy_i386 added
![Page 24: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/24.jpg)
aptly snapshot pull$ aptly snapshot pull aptly snapshot pull deb-main-250114 back-250114 deb-w-xorg rsyslog…[-] rsyslog-5.8.11-3_amd64 removed[+] rsyslog-7.4.4-1~bpo70+1_amd64 added[-] libestr0-0.1.1-2_amd64 removed[+] libestr0-0.1.9-1~bpo70+1_amd64 added[+] init-system-helpers-1.11~bpo70.1_all added[-] rsyslog-5.8.11-3_i386 removed[+] rsyslog-7.4.4-1~bpo70+1_i386 added[-] libestr0-0.1.1-2_i386 removed[+] libestr0-0.1.9-1~bpo70+1_i386 added
![Page 25: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/25.jpg)
snapshot
mirror take snapshot
update
snapshot
snapshotpull
![Page 26: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/26.jpg)
aptly snapshot diff
$ aptly snapshot diff deb-main-250114 deb-w-percona Arch | Package | Version in A | Version in B! amd64 | libmysqlclient18 | 5.5.31+dfsg-0+wheezy1 | 5.5.35-rel33.0-611.wheezy+ amd64 | percona-server-client-5.5 | - | 5.5.35-rel33.0-611.wheezy+ amd64 | percona-server-common-5.5 | - | 5.5.35-rel33.0-611.wheezy+ amd64 | percona-server-server | - | 5.5.35-rel33.0-611.wheezy+ amd64 | percona-server-server-5.5 | - | 5.5.35-rel33.0-611.wheezy! i386 | libmysqlclient18 | 5.5.31+dfsg-0+wheezy1 | 5.5.35-rel33.0-611.wheezy+ i386 | percona-server-client-5.5 | - | 5.5.35-rel33.0-611.wheezy+ i386 | percona-server-common-5.5 | - | 5.5.35-rel33.0-611.wheezy+ i386 | percona-server-server | - | 5.5.35-rel33.0-611.wheezy+ i386 | percona-server-server-5.5 | - | 5.5.35-rel33.0-611.wheezy
![Page 27: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/27.jpg)
snapshot
mirror take snapshot
update
snapshot
snapshotpull
diff
![Page 28: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/28.jpg)
aptly snapshot verify$ aptly snapshot verify deb-w-perconaMissing dependencies (13): fenix [amd64] fenix-plugins-system [amd64] ia32-libs-i386 [amd64] ia32-libs-gtk-i386 [amd64] vidcontrol [amd64] kbdcontrol [amd64] libmysqlclient18 (= 5.5.31+dfsg-0+wheezy1) [amd64] scsh-0.6 (>= 0.6.6) [amd64] scsh-0.6 [amd64] mozart (>= 1.4.0) [amd64] libmysqlclient18 (= 5.5.31+dfsg-0+wheezy1) [i386] vidcontrol [i386] kbdcontrol [i386]
![Page 29: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/29.jpg)
snapshot
mirror take snapshot
update
snapshot
snapshotpull
diff
verify
![Page 30: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/30.jpg)
aptly snapshot merge
$ aptly snapshot merge deb-w-percona-250114 deb-main-250114 percona-250114!Snapshot deb-w-percona-250114 successfully created.You can run 'aptly publish snapshot deb-w-percona-250114' to publish snapshot as Debian repository.
![Page 31: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/31.jpg)
snapshot
mirror take snapshot
update
snapshot
snapshot
merge
snapshotpull
diff
verify
![Page 32: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/32.jpg)
aptly publish snapshot$ aptly publish snapshot -distribution=wheezy percona-250114Signing file '/Users/smira/.aptly/public/dists/wheezy/Release' with gpg, please enter your passphrase when prompted:!…!Snapshot percona-250114 has been successfully published.Please setup your webserver to serve directory '/Users/smira/.aptly/public' with autoindexing.Now you can add following line to apt sources: deb http://your-server/ wheezy mainDon't forget to add your GPG key to apt with apt-key.
![Page 33: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/33.jpg)
snapshot
mirror
publishedrepo
take snapshot
update
snapshot
snapshot
merge
snapshotpull
publish
diff
verify
![Page 34: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/34.jpg)
Plans• Searching
• Filtering
• Managing your own package repositories
• Support for source packages and translations
• Verifying remote repositories
![Page 35: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/35.jpg)
What doesn’t work yet
• deletion
• bash-completion
• flat-repositories
• download progress
![Page 36: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/36.jpg)
Competition
• mirror: ftpsync, reprepro, debmirror, apt-mirror, apt-cache…
• archive generation: dak, mini-dak, reprepro, …
![Page 37: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/37.jpg)
Internals
• Written in Go
• Embedded database - LevelDB
• Package files are stored only once
![Page 38: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/38.jpg)
Internals• ~/.aptly.conf - configuration file
• ~/.aptly/ - default work directory
• db/ - database
• pool/ - package file storage
• public/ - published repositories
![Page 39: aptly - Debian repository management toolDevOps Meetup Moscow Jan’14 Repeatability! Change! Environment Repeatable Environment Environment Change Repetable Environment Change Environment](https://reader034.fdocuments.us/reader034/viewer/2022051913/6004b3ef55981f52871fb3d8/html5/thumbnails/39.jpg)
Links• http://www.aptly.info/
• https://github.com/smira/aptly/
• @smira
• Skype: smirnov.andrey