Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans
-
Upload
bacongobbler -
Category
Technology
-
view
114 -
download
0
Transcript of Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans
![Page 1: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/1.jpg)
Contain(erize)The Tests@markeijsermans
![Page 2: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/2.jpg)
Software Engineer
Love DevOps
Love Docker
![Page 3: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/3.jpg)
When you're a hammer...
![Page 4: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/4.jpg)
Our unit test problemsmonolithic code basecomplex setupfew mockshits databasesslow
![Page 5: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/5.jpg)
Jenkins WoesJenkins was bottlenecking our server configurationsPhp 5.5.x roll-out took months
![Page 6: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/6.jpg)
...the nail
![Page 7: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/7.jpg)
Parallelize the testssolved a problemlow risk
![Page 8: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/8.jpg)
Jenkins x Jenkinsrun tests on 10 (or more) slavesgoal: Docker and git are the only dependencies
![Page 9: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/9.jpg)
unittest.sh kickoff scriptprivate docker registrycontainer for running testlinked container for dbs
![Page 10: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/10.jpg)
Results25min down to 3min (could still be lower)...but, not all is rosy
![Page 11: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/11.jpg)
Findings
![Page 12: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/12.jpg)
Configuration Management
How to integrate Ansbile?
sshd adds +40MB (seems wrong)installed Ansible in base imagemaybe provision with chroot? (but lose layering)Packer.io? isn't quite there yetCM integration with Docker rumored
[soon to be fixed]
![Page 13: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/13.jpg)
Why not just use Ansibledirectly on slaves?
continuous integration is continually integrating itself!automatic refresh (pull) before running testseasy to scale more slavesslaves re-usable for other tasks - not polluted with dependenciespart of the experiment!
![Page 14: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/14.jpg)
The "PID 1" problemneeds to watch sub-processes if they die and re-launchcan't be SIGKILL'dsome hacks needed to quiet upstartwe used supervisord - works wellinit integration coming (systemd, SysV, etc.)
[soon to be fixed]
![Page 15: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/15.jpg)
libcontainernew in version 0.9nsinit allows you to attachmight be implemented as "docker exec"lxc namespacing still has lxc-attach
![Page 16: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/16.jpg)
Registry is slow from the cloudpush/pulling from local network to EC2 is painful+1GB containers!how about a local mirror?EDIT: added to the docker-registry a few weeks ago
[fixed]
![Page 17: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/17.jpg)
Trimming the fatinstalling packages dumps a TON of stuff on the systemgreat way to find out what a package doesdpkg --no-install-recommendsfilter out /usr/share/{man | doc | info}trash apt cache (+100MB)flatten: docker export -> import
[solvable]
![Page 18: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/18.jpg)
Container layering questionsbase -> Ansible -> runtime -> app -> unit test extras
![Page 19: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/19.jpg)
Linking containerscumbersome.lots of verbose ENV VARS
ambassador container (hacky)?service discovery - serf, etcd, etc?
DB_PORT_3306_TCP=tcp://127.0.0.1
DB_PORT_3306_TCP_ADDR=127.0.0.1
DB_PORT_3306_TCP_PORT=3306
DB_PORT_3306_TCP_PROTO=tcp
...
[unknown]
![Page 20: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/20.jpg)
Unknown maintenance profilenot all *nix tools ready for namespacingfree & top are broken (shows full system memory allocation)logs (on aufs, inside the container - good/bad?)problem? copy container locally, inspect?
![Page 21: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/21.jpg)
API first design*except a some commands like attach
![Page 22: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/22.jpg)
Great for open-sourcinga projectdocker run ... done!
![Page 23: Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans](https://reader033.fdocuments.us/reader033/viewer/2022052821/554a0714b4c9055b7a8b5634/html5/thumbnails/23.jpg)
Thank [email protected]@hootsuite.com