Continuous Integration with Docker and Jenkins
-
Upload
francesco-bruni -
Category
Software
-
view
1.693 -
download
7
Transcript of Continuous Integration with Docker and Jenkins
![Page 1: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/1.jpg)
CI with Docker and Jenkins
Francesco Bruni
Club degli sviluppatori - Bari, May ‘19
@brunifrancesco
![Page 2: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/2.jpg)
Who am I•MSc Telecommunication Engineer @ Poliba •Despite a Java background, I prefer Python whenever
possible •I'm not a computer scientist
![Page 3: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/3.jpg)
Continuous Integration•Keep integrating multiple times a day the central code
repository; •Run as more tests as you can to ensure nothing broke
down; •Be ready to replicate environment.
![Page 4: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/4.jpg)
Why CI matters•Continuous testing/building for all teammates; •Let someone do trivial jobs.
![Page 5: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/5.jpg)
CI advantage
“Continuous Integration doesn’t get rid of bugs, but it does make them
dramatically easier to find and remove” M. Fowler
![Page 6: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/6.jpg)
CI requirements•Single source repo; •Automated building; •Self testing build; •Build fast; •Run tests in a production env clone; •Automate deployment.
![Page 7: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/7.jpg)
CI cycle
Develop Feature
Integrate in the main code baseTest/build/label
Checkout codeCorrect detected
iussues
![Page 8: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/8.jpg)
CI and TDD
TDD
CI
![Page 9: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/9.jpg)
CI cycle•Integrating code should be automated; •Multiple levels testing; •Devs must not checkout bugged code; •Devs must not deploy untested code;
![Page 10: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/10.jpg)
CI drawbacks•Setting files; •Setting all the stuff up requires time and patience.
![Page 11: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/11.jpg)
CI natural evolution: Continuous Delivery
•If a commit builds (all tests succeed), deliver it; •Keep delivery time at minimum; •Multiple deliveries for multiple versions.
![Page 12: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/12.jpg)
Know your tools•Test everything; •Keep your sources/setting files versioned; •Containerize everything; •Automate as much as you can.
![Page 13: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/13.jpg)
Test everything•Test your sources, checking coverage; •Test for writing new features; •Test your application settings; •If you want to deliver software, check external
requirements.
![Page 14: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/14.jpg)
Smarter use of VCs•Keep code versioned: application settings/sources/dep
indexes; •Keep env settings versioned: scripts and configuration
files; •Enjoy version control well defined ‘flows’ :)
![Page 15: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/15.jpg)
Be containerized•Develop solutions in well separated envs; •Ship software as main part of env; •Don’t play with global deps. •‘Restore’ basic envs to speed up env creation process.
![Page 16: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/16.jpg)
What’s Docker•Wrap many services in many
containers; •Create, play with and destroy
containers; •Save and easily replicate
isolated containers.
![Page 17: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/17.jpg)
Dockerfiles# pull base image. FROM java:8
# maintainer details MAINTAINER Francesco Bruni "[email protected]"
# update packages and install maven RUN \ export DEBIAN_FRONTEND=noninteractive && \ sed -i 's/# \(.*multiverse$\)/\1/g' /etc/apt/sources.list && \ apt-get update && \ apt-get -y upgrade && \ apt-get install -y vim wget curl git maven ssh nano python-pip # attach volumes VOLUME /volume/git
# create working directory and set some stuff up RUN mkdir -p /local/git WORKDIR /local/git RUN mkdir -p /root/.ssh ADD id_rsa /root/.ssh/id_rsa RUN chmod 700 /root/.ssh/id_rsa RUN echo "Host <ip>\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config COPY .m2 /root/.m2 COPY run.sh run.sh RUN chmod a+x run.sh
# run this when container starts CMD ["./run.sh"]
![Page 18: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/18.jpg)
Automate procedures•Repetitive tasks should be performed by automated
agents; •Once you teach the agent what to do, it’s up to him to
fulfill the job.
![Page 19: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/19.jpg)
What’s Jenkins•Automated procedures worker; •Run tasks, report final status,
log activities; •Multiple plugin for multiple
features; •Use it just for playing too :)
![Page 20: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/20.jpg)
Deploy w/ Docker w/o Jenkins
•Test code (or at least run a set of tests); •Push code to repo; •SSH in the server, enter in container; •Pull changes and restart required services. •Check if it works! :)
![Page 21: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/21.jpg)
Practical session•Integrate new code into the
main branch; •Test all the stuff; •Deploy it :)
![Page 22: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/22.jpg)
What we’ll need•A GIT web hook; •Set private “credentials”; •A tool to test configuration settings; •A container to run the test suite; •A container to run the new application; •A pipeline.
![Page 23: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/23.jpg)
GIT web hook• Bind an action (tipically run a script) to GIT events; • Notify Jenkins new code has been pushed over the
‘develop’ branch.
#! /bin/bash
read oldrev newrev _branch tail=$(git log -1 --pretty=format:'%h %cn: %s%b' $newrev) curl http://<ip>:8080/job/QW/build?token=vH3YKmrRR5KOT4aeAOAV&commitMessage=$tail&cause=JenkinsDeploy
![Page 24: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/24.jpg)
Merge/Test configurations
import osmain_path = os.path.join("src", "main","resources","application.properties")test_path = os.path.join("src", "test","resources","application.properties")for element in (main_path, test_path,): with open(element) as old_main: data = map(lambda line: line.strip(), old_main.readlines()) data[6] = data[6].replace("192.168.1.75", "mysql") data[6] = data[6].replace("<myIp>", "mysql") with open(element, "wb") as new_main: new_main.write(“\n".join(data))
Connection().quick_connect("user", "password", “db_test", "<myIp>") Connection().quick_connect("user", "password", "db", "mysql")
![Page 25: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/25.jpg)
Create the container to run test suite
Run tests and see if they fails :)
![Page 26: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/26.jpg)
Deploy the new version
• Create the new container and run the new version; • The new container should replace an existing one.
![Page 27: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/27.jpg)
Going on productionNo way, do some manual testing and start a new CD build
![Page 28: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/28.jpg)
All togheternode { stage 'Stage 1' sh 'docker run --rm --link mysqlC:mysql --name sms-checklink-container sms-checklink' stage 'Stage 2' sh 'docker rm -f sms-test-container || echo \'no container to delete\'' sh 'docker run --rm --name sms-test-container --link mysqlC:mysql sms-test' stage 'Stage 3' sh 'docker run --rm --name sms-merge-container sms-merge' stage 'Stage 4' sh 'docker rm -f sms-staging-container || echo \'no container to delete\'' sh 'docker run -d -p 8433:8080 --name sms-staging-container --link mysqlC:mysql sms-staging' }
![Page 29: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/29.jpg)
Conclusions• Improve configuration settings handling; • Reduce testing time; • Bind Jenkins job to commit messages; • Improve git-flow integration; • Docker Compose :( • Deploy project on production via manual tests.
![Page 30: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/30.jpg)
References
Continuous Delivery Reliable Software Releases through Build, Test, and Deployment Automation
by Jez Humble and David Farley
Continuous Integration
by Martin Fowler http://martinfowler.com/articles/continuousIntegration.html
![Page 31: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/31.jpg)
References
Docker for beginners
https://scotch.io/tutorials/getting-started-with-docker
Meet Jenkins https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins
Docker Explained: Using Dockerfiles to Automate Building of Images
https://www.digitalocean.com/community/tutorials/docker-explained-using-dockerfiles-to-automate-building-of-images
![Page 32: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/32.jpg)
Let’s see it @ work
![Page 33: Continuous Integration with Docker and Jenkins](https://reader034.fdocuments.us/reader034/viewer/2022052122/5875bb921a28ab33128b46dd/html5/thumbnails/33.jpg)
Questions?@brunifrancesco