Continuous Integrationwith Hudson
Arun GuptaSun Microsystems, Inc.
Originally presented byKohsuke Kawaguchi & Jesse Glick
at JavaOne
Scheduled to be presented byFabiane Nardon
At FISL 10
Acknowledgements
Rise of Continuous Integration> Offload from people, push to computers
3
$
time
computers
us
“Never use humans for the job of a machine”
Spend more CPU power to help you> … even if it only helps a little
> First on your laptops and workstations IDEs are at the forefront
> And then to the servers a.k.a. “Continuous Integration” More frequent build/test executions Static code analysis tools And more to come
5
Hudson> Open-source CI server at java.net
> Emphasis on ease of installation and use “java -jar hudson.war” execution Configure everything from browsers
> Extensibility 140+ community-developed public plugins By 150+ contributors
> Estimated 13,000 installations6
https://hudson.dev.java.net/
It basically does builds and tests> Check out the source code
Subversion, Perforce, Git, Mercurial, CVS, …> Do builds and/or tests
Ant, Maven, MSBuild, shell script, …> Record results
Binary, test results, code coverage, static analysis> Notify people
E-mail, IM, RSS, tray apps, IDEs
7
Hudson Plugin Ecosystem
8
New Plugins
2008: 552009: 44 so far
Localized to 8 languages
9
And hopefully more to come…
10
11
Adoption in all kinds of businesses
Eclipse Community Survey
12
13
Why Distributed Builds?> You need to use multiple computers because…
You need different environments You need isolation
> There’s only so much you can do with 1 computer
14
Installing new slaves> For first 20 or so slaves, we did it manually
Insert CD, click, type, click, type, click, … But that doesn’t scale
> Then we automated Available as “Hudson PXE Plugin”
15
Automated System Installations
16
> Slaves Power on, hit F12 PC boots from network (PXE)
> Hudson + PXE plugin ISO images of OS
Automated System Installations
17
> Slaves Power on, hit F12 PC boots from network (PXE) Choose OS from menu Installs non-interactively
> Hudson + PXE plugin ISO images of OS
Your corporate IT guy & his DHCP server
Automated System Installations> Supports OpenSolaris, Ubuntu, CentOS, Fedora
Trivial with most Linux> Cooperate with Windows, too
> Quite useful outside Hudson, too No more broken CD drives No more CD-Rs
18
Distributed builds with Hudson> Master
Serves HTTP requests Stores all important info
> Slaves 170KB single JAR Assumed to be unreliable Scale to at least 100
> Link Single bi-di byte stream No other requirements
19
Heterogeneous Cluster Challenge> Your builds/tests need to run in specific
environment> Dependency on individual nodes hurts utilization
20
WombatWindows test
Hudson Windows test
Windows #1
jobs slaves
GlassFishWindows test
Windows #2
Solaris#1
Hudson Solaris test
Labels to rescue> Label is a group of slaves> Tie jobs to labels
21
WombatWindows test
Hudson Windows test
Windows #1
jobs slaves
GlassFishWindows test
Windows #2
Solaris#1
Hudson Solaris test
Windows
SolarisWindow
s #3
Making builds sticky> We want jobs to be mostly on the same slave
Faster check out Consistent results Minimizes disk consumption
> But does it softly
> Hudson uses consistent hash* to achieve this> More schedule controls become possible:
Use faster machines more frequently Slowly ramp up newly installed slaves
22
Coming to your
Hudson soon
* http://en.wikipedia.org/wiki/Consistent_hashing
Forecasting failures> Hudson monitors key health metrics of slaves
Low disk space, insufficient swap Clock out of synch Extensible
> Slaves go offline automatically> Catch problems before they break builds
23
Load Statistics Monitoring
24
When it’s time to add more slaves
25
Hudson made this extensible> Hudson detects excessive workload> Hudson notifies plugins> Plugins can provision more slaves
… assuming that you have that infrastructure
26
27
Amazon EC2: The Good> Pay as you go (10¢/h or so)
Loads on Hudson tend to be spiky> Programmable API> Instances launch at machine-speed> EC2 instances are forgetful
28
Amazon EC2: The Bad> Your data is still inside your firewall
Takes time to check out code … or to archive build artifacts Some data just can’t be moved
> EC2 instances are forgetful> Can your tests run in parallel?
29
Hudson EC2 plugin> Built on top of typica*> What does it do?
Automatically provisions slaves on EC2 on demand Picks the right AMI depending on demand Starts slave agent Shuts down unused instances
30* http://code.google.com/p/typica/
Hudson “Appliance” on EC2> Run the master in the cloud too, if you like
Hudson on stock OpenSolaris AMI Data stored persistently in Elastic Block Storage
Dynamically expandable thanks to ZFS Online, too
> Packaged as a wizard
31
Hudson EC2 plugin usage> Tell Hudson what AMIs you want to start
32
33
Conclusion> CI is here to stay
We’ll continue to push more workload to servers> Hudson makes this easy for you> Reap the benefit of a cluster in multiple ways
34
36
Arun Gupta
http://blogs.sun.com/arunguptahttp://hudson.dev.java.net/
Top Related