Hudson at FISL 2009
-
Upload
arun-gupta -
Category
Technology
-
view
3.535 -
download
2
description
Transcript of Hudson at FISL 2009
![Page 1: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/1.jpg)
Continuous Integrationwith Hudson
Arun GuptaSun Microsystems, Inc.
![Page 2: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/2.jpg)
Originally presented byKohsuke Kawaguchi & Jesse Glick
at JavaOne
Scheduled to be presented byFabiane Nardon
At FISL 10
Acknowledgements
![Page 3: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/3.jpg)
Rise of Continuous Integration> Offload from people, push to computers
3
$
time
computers
us
![Page 4: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/4.jpg)
“Never use humans for the job of a machine”
![Page 5: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/5.jpg)
Spend more CPU power to help you> … even if it only helps a little
> First on your laptops and workstations IDEs are at the forefront
> And then to the servers a.k.a. “Continuous Integration” More frequent build/test executions Static code analysis tools And more to come
5
![Page 6: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/6.jpg)
Hudson> Open-source CI server at java.net
> Emphasis on ease of installation and use “java -jar hudson.war” execution Configure everything from browsers
> Extensibility 140+ community-developed public plugins By 150+ contributors
> Estimated 13,000 installations6
https://hudson.dev.java.net/
![Page 7: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/7.jpg)
It basically does builds and tests> Check out the source code
Subversion, Perforce, Git, Mercurial, CVS, …> Do builds and/or tests
Ant, Maven, MSBuild, shell script, …> Record results
Binary, test results, code coverage, static analysis> Notify people
E-mail, IM, RSS, tray apps, IDEs
7
![Page 8: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/8.jpg)
Hudson Plugin Ecosystem
8
New Plugins
2008: 552009: 44 so far
![Page 9: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/9.jpg)
Localized to 8 languages
9
![Page 10: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/10.jpg)
And hopefully more to come…
10
![Page 11: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/11.jpg)
11
Adoption in all kinds of businesses
![Page 12: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/12.jpg)
Eclipse Community Survey
12
![Page 13: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/13.jpg)
13
![Page 14: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/14.jpg)
Why Distributed Builds?> You need to use multiple computers because…
You need different environments You need isolation
> There’s only so much you can do with 1 computer
14
![Page 15: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/15.jpg)
Installing new slaves> For first 20 or so slaves, we did it manually
Insert CD, click, type, click, type, click, … But that doesn’t scale
> Then we automated Available as “Hudson PXE Plugin”
15
![Page 16: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/16.jpg)
Automated System Installations
16
> Slaves Power on, hit F12 PC boots from network (PXE)
> Hudson + PXE plugin ISO images of OS
![Page 17: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/17.jpg)
Automated System Installations
17
> Slaves Power on, hit F12 PC boots from network (PXE) Choose OS from menu Installs non-interactively
> Hudson + PXE plugin ISO images of OS
Your corporate IT guy & his DHCP server
![Page 18: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/18.jpg)
Automated System Installations> Supports OpenSolaris, Ubuntu, CentOS, Fedora
Trivial with most Linux> Cooperate with Windows, too
> Quite useful outside Hudson, too No more broken CD drives No more CD-Rs
18
![Page 19: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/19.jpg)
Distributed builds with Hudson> Master
Serves HTTP requests Stores all important info
> Slaves 170KB single JAR Assumed to be unreliable Scale to at least 100
> Link Single bi-di byte stream No other requirements
19
![Page 20: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/20.jpg)
Heterogeneous Cluster Challenge> Your builds/tests need to run in specific
environment> Dependency on individual nodes hurts utilization
20
WombatWindows test
Hudson Windows test
Windows #1
jobs slaves
GlassFishWindows test
Windows #2
Solaris#1
Hudson Solaris test
![Page 21: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/21.jpg)
Labels to rescue> Label is a group of slaves> Tie jobs to labels
21
WombatWindows test
Hudson Windows test
Windows #1
jobs slaves
GlassFishWindows test
Windows #2
Solaris#1
Hudson Solaris test
Windows
SolarisWindow
s #3
![Page 22: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/22.jpg)
Making builds sticky> We want jobs to be mostly on the same slave
Faster check out Consistent results Minimizes disk consumption
> But does it softly
> Hudson uses consistent hash* to achieve this> More schedule controls become possible:
Use faster machines more frequently Slowly ramp up newly installed slaves
22
Coming to your
Hudson soon
* http://en.wikipedia.org/wiki/Consistent_hashing
![Page 23: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/23.jpg)
Forecasting failures> Hudson monitors key health metrics of slaves
Low disk space, insufficient swap Clock out of synch Extensible
> Slaves go offline automatically> Catch problems before they break builds
23
![Page 24: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/24.jpg)
Load Statistics Monitoring
24
![Page 25: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/25.jpg)
When it’s time to add more slaves
25
![Page 26: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/26.jpg)
Hudson made this extensible> Hudson detects excessive workload> Hudson notifies plugins> Plugins can provision more slaves
… assuming that you have that infrastructure
26
![Page 27: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/27.jpg)
27
![Page 28: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/28.jpg)
Amazon EC2: The Good> Pay as you go (10¢/h or so)
Loads on Hudson tend to be spiky> Programmable API> Instances launch at machine-speed> EC2 instances are forgetful
28
![Page 29: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/29.jpg)
Amazon EC2: The Bad> Your data is still inside your firewall
Takes time to check out code … or to archive build artifacts Some data just can’t be moved
> EC2 instances are forgetful> Can your tests run in parallel?
29
![Page 30: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/30.jpg)
Hudson EC2 plugin> Built on top of typica*> What does it do?
Automatically provisions slaves on EC2 on demand Picks the right AMI depending on demand Starts slave agent Shuts down unused instances
30* http://code.google.com/p/typica/
![Page 31: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/31.jpg)
Hudson “Appliance” on EC2> Run the master in the cloud too, if you like
Hudson on stock OpenSolaris AMI Data stored persistently in Elastic Block Storage
Dynamically expandable thanks to ZFS Online, too
> Packaged as a wizard
31
![Page 32: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/32.jpg)
Hudson EC2 plugin usage> Tell Hudson what AMIs you want to start
32
![Page 33: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/33.jpg)
33
![Page 34: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/34.jpg)
Conclusion> CI is here to stay
We’ll continue to push more workload to servers> Hudson makes this easy for you> Reap the benefit of a cluster in multiple ways
34
![Page 36: Hudson at FISL 2009](https://reader034.fdocuments.us/reader034/viewer/2022051412/5481ced5b4af9faf0d8b4601/html5/thumbnails/36.jpg)
36
Arun Gupta
http://blogs.sun.com/arunguptahttp://hudson.dev.java.net/