<Insert Picture Here>

Improving Engineering ProcessUsing Hudson

Arun Gupta, Java EE & GlassFish GuyOracle Corphttp://blogs.sun.com/arungupta, @arungupta

Never send a human to do a machine's job.

Throughout human history, we have been dependent on machines to survive.

Rise of Continuous Integration

Offload from people, push to computers

$

Time

computers

people

Spend more CPU power to help you

●First on your laptops and workstations● IDEs are at the forefront

●And then to the servers● a.k.a. “Continuous Integration”● More frequent build/test executions● Static code analysis tools● And more to come

Hudson (http://hudson-ci.org)

● Open-source CI server● Emphasis on ease of installation and use

● “java -jar hudson.war” execution● Or your OS-specific package● Configure everything from browser

● Extensibility● 230+ community-developed public plugins● By 230+ contributors

● Estimated 13,000 installations

"Das

hboard [H

udson]"

Basic Feature Set

● Hudson notices that there is a code change● Checks out the source code

●CVS, Subversion, ClearCase, Mercurial, Accurev, Perforce, Git, StarTeam, …

● Builds Ant, Maven, shell script, NAnt, …● Record and publish results● Tool Integration

● Findbugs, Corbertura, ...

Provides Feedback

● RSS feeds

● Email/IM notification● Tray application to notify developers

● Linux/Windows (Toolbar), Mac (Growl)

● . . .

Demo

Why do I care?●Life before Hudson

● Dev forgets to commit a file● Nightly build breaks, Testing blocked● Resources not utilized efficiently

●Life after Hudson● Hudson will tell you in 5 mins if a build broke● No one else notices that you forgot to commit a

file ;-)

●Life before Hudson● Dev makes a change Monday AM● Test nightly finds a bug Monday night● Dev fixes it Tuesday● Again, under utilization of resources

Why do I care?

●Life after Hudson● Dev makes a change Monday AM● Hudson finds a regression few mins later● Dev can fix it before lunch

●Life before Hudson● Test runs every night, results in e-mail● Excitement fades after 1 week● Regressions go unnoticed until it’s too late

Why do I care?

●Life after Hudson● Tests run Hudson after every commit● E-mail sent out only when tests start failing

● So it manages to keep people’s attention

Matrix projects●Run the same thing on

different environments● JDKs● Databases● OSes● App servers● XML parsers● ...

●Results aggregated

FindBugs integration

Tracking changes

16

… and more Browse workspaces Build time trend report

Inter-Project Support

●Host many projects on 1 Hudson● 1+1 > 2

●You can…● Execute tests separately and correlate them

with builds● Mark builds as “promoted” based on test

outcome● Track which build of libraries are used where● . . .

Test/Build Separation●Tests often dominate the build time●Multiple tests are run in parallel

● Improves turn-around time

JAX-WS #10

Unit test with GF

Unit test with Tomcat

SQE test

JAX-WS #11 JAX-WS #12 …

Test w/o container 3OS x 2JDK = 6 tests

time

Build Promotion●CI produces a lot of “successful” builds

● Often overwhelming to downstream consumers

●Run tests as fast as you can●If a build passes tests, promote it

20

Got the idea?

Automation Reduce turn-around time Make things transparent Remove people from the loop Save people’s time Push jobs to servers, keep workstations

idle for you

Doing Distributed BuildsDoing Distributed Builds

22

Distributed builds with Hudson

●Master● Serves HTTP requests● Stores all important info

●Slaves● 170KB single JAR● Assumed to be

unreliable● Scale to at least 100

●Link● Single bi-di byte stream● No other requirements

Master

slave1

slave2

slave3

slave4

slave5

slave6

slave7

slave8

Automated System Installations

●Slaves● Power on, hit F12● PC boots from network (PXE)● Choose OS from menu● Installs non-interactively

●Hudson + PXE plugin● ISO images of OS

Your corporate IT guy & his DHCP server

Automated Tool Installation - JDK

●JDK from http://java.sun.com/● Automatically chooses the right bundle● Always up to date with new releases

25

Automated Tool Installation - Apache

Ant and Maven from Apache

26

Heterogeneous Cluster Challenge Builds/tests need to run in specific environment Dependency on individual nodes hurts utilization

WombatWindows test

Hudson Windows test

Windows #1

jobs slaves

GlassFishWindows test

Windows #2

Solaris#1

Hudson Solaris test

Labels to rescue Label is a group of slaves Tie jobs to labels

WombatWindows test

Hudson Windows test

Windows #1

jobs slaves

GlassFishWindows test

Windows #2

Solaris#1

Hudson Solaris test

Windows

SolarisWindows

#3

Forecasting failures

●Hudson monitors key health metrics of slaves

● Low disk space, insufficient swap● Clock out of sync● Extensible

●Slaves go offline automatically

Load Statistics Monitoring

When it’s time to add more slaves

31

Hudson EC2 plugin

●Automatically provisions slaves on EC2 on demand

● Based on the current load●Picks the right AMI depending on demand●Starts slave agent●Shuts down unused instances

●Can run Hudson master in the cloud too

And a lot more . . .

IDE plugins iPhone/Android apps REST API and CLI Deployment Automation Hadoop, Selenium …

HUMANS: ALL YOUR COMPUTER ARE BELONGTO US

Conclusions

●CI is here to stay● Using lots of PCs is a key

●Hudson is very easy to get started● Once started, there is a lot you can do

●hudson-ci.org