Hudson Dev Workshop

Kohsuke Kawaguchikohsuke.kawaguchi@cloudbees.comCloudBees, Inc.

2

Why a Hudson plugin?

300+ people have done it. Can’t be that hard Integrate Hudson with your favorite tools, systems,

and etc. Make Hudson speak in domain specific terms▪ Instead of batch files or shell scripts

3

Pre-requisite

Maven 2 Terminal and/or command prompt IDE of your choice The following entry in your .m2/settings.xml

Windows user will find this in %USERPROFILE%\.m2\settings.xml

<settings> <pluginGroups> <pluginGroup>org.jvnet.hudson.tools</pluginGroup> </pluginGroups></settings>

4

Create a skeleton

CUI wizard to create a skeleton This will download a lot from the internet if this is

the first time

Download seed.zip and unzip it in ~/.m2/repository for faster bootstrap From my USB key

$ mvn -cpu hpi:create

And while we wait for Maven to download the internet…

6

Hudson Plugin Community

https://github.com/hudson/ Developer resources

IRC channel: #hudson Mailing list: hudson-dev@googlegroups.com Wiki:

http://wiki.hudson-ci.org/display/HUDSON/Extend+Hudson

7

Build your Hudson plugin

Update Hudson version to 1.387 In reference to parent POM

Hudson plugin follows a Maven standard

And while you wait for Maven to build…

$ mvn package$ ls target/*.hpi

8

Let’s look at source files

POM Source files Jelly files Help HTML files

9

Debug a Hudson plugin

Start your plugin in an embedded Hudson If port 8080 is occupied, use –Dport=12345

Point your browser to http://localhost:8080/

$ mvn hpi:run

10

Play with the hello world builder

Configure a new job Add a “Say hello world” build step Run the build and see hello world Where is HUDSON_HOME?

Hint: system configuration

11

Interactive Development

Edit help HTML file, save Just reload the browser and see the change

Edit Jelly files, save Just reload the browser and see the change

12

Interactive Development

For Eclipse Edit source code, save Wait for Hudson to auto reload

For other IDEs Edit source code, compile Wait for Hudson to auto reload

13

Sometimes auto-reload is annoying

Most often useful with “mvnDebug” So that you can reload classes from debugger

Add –DscanIntervalSeconds=0 to Maven

14

When your plugin is ready…

“Manage Hudson” > “Manage Plugins” > “Advancced”

15

Let’s look at the source codemore carefully

16

Key ingredient check list

@DataBoundConstructor @Extension DescriptorImpl Package structure for Jelly files Help files by convention

17

Basic pattern of plugins

Pick an extension point http://wiki.hudson-ci.org/display/HUDSON/

Extension+points Implement it Create a descriptor with @Extension

Or in some cases there’s no descriptor and the implementation class itself gets @Extension

18

UI / Data binding

Getter or public field

@field in config.jelly

Parameter name in

data bound constructor

19

Exercise: add a new config field

Add another text field to accept another name, then say hello to both

Add a help file for this new text field

20

UI Samples

Unfortunately, much of config.jelly editing is monkey-see-monkey-do

UI samples = our attempt to fix this

21

Exercise: add a new config field

Add auto-completion to the name field Note the method name has to match with @field

Extra Credits Try other UI samples and see if you can copy them

22

Form Validation

Write doCheckXyz method for xyz field “value” parameter to receive the current value Can also retrieve nearby sibling values Return/throw FormValidation instance for status

public FormValidation doCheckPort(@QueryParameter String value) {if(looksOk(value)) return FormValidation.ok();else return FormValidation.error("There's a problem here");

}

23

Different datatypes available

URL int boolean Enum hudson.util.Secret

Persisted and sent to browser in encrypted form Can be extended by adding converter

24

Exercise: play with Secret

Add another configuration field whose type is hudson.util.Secret

Inspect the persisted config.xml ./work/jobs/JOBNAME/config.xml

Inspect the value set in the config page Trace the execution in the debugger

25

Playing with Launcher

Abstraction for forking processes Works even when a build is on a slave

Fluent API launcher.launch().doThis().doThat().start()▪ or join()

26

Playing with BuildListener

Used to write to build console getLogger() : PrintStream

Used to report error e.printStackTrace(listener.error(“Failed to do X”));

27

Exercise: Launcher&BuildListener

Tweak HelloWorldBuilder to fork a process and send its output to build console E.g., “uname –a” or “cmd.exe /?”

28

FilePath: working with files

java.io.File on steroid Represents a file or a directory Lots of convenient file operations Works transparently on files on slaves

29

Exercise: FilePath

Tweak HelloWorldBuilder to play with FilePath Use AbstractBuild.getWorkspace()

Create several text files and create a zip file from them

Compute md5 checksums of all the files Hint: getDigest()

30

Descriptor : Describable

Descriptor : Describable = Class : Object Describable

Created from UI through Descriptor Live as long as the configuration remains the same

Descriptor Singletons Capture behaviors and operations that are “static” Global configuration, form validation … is a static nested type of Describable by convention

31

@Extension

Enumerated during compile time to generate index Sometimes “mvn compile” is needed for this Used at runtime to find extensions

View Technology

33

Side track: RootAction

Useful for experimenting with views Extension point to add a menu item to the top

page Descriptor-less, because it’s not configured from

anywhere

34

Apache Commons Jelly

XML based template engine Think of it as JSPX+JSTL+custom taglibs

Used to render HTML And other XMLs, such as RSS, JNLP, …

Most people start by monkey-see-monkey-do References

http://commons.apache.org/jelly/tags.html http://hudson-ci.org/maven-site/hudson-core/

jelly-taglib-ref.html

35

Apache Commons JEXL

EL on steroid ${foo.bar[5]+”abc”} Allow method calls XML friendly operators▪ “and”/”or”,”lt”,”ge” etc in addition to &&, ||, <, >=

a?b:c and a?:b (=a?a:b)

36

Object-oriented views

Views are like methods Co-located to classes via naming convention foo.jelly for org.acme.Bar should be in

org/acme/Bar/foo.jelly Views are inherited to subtypes In JEXL, “it” variable refers to the “this” object “index.jelly” plays the special role

37

Exercise: Jelly, JEXL, and RootAction

Create a new root action “MyRootAction” See javadoc for what your methods need to return

Add index.jelly Start from this:

Play with Jelly and JEXL Add methods to your class and invoke it from view Use <j:forEach> to say hello 10 times Otherwise be creative!

<j:jelly xmlns:j=“jelly:core”> <html><body><h1> Hello from ${it.getClass()} </h1></body></html></j:jelly>

38

Object-oriented Views

In Hudson, URLs are mapped to object graph And this determines how request is eventually

handled IOW, controller layer is implicit More precisely, this is in Stapler

See HTTP headers for evaluation trace So if you add “bar.jelly” to MyRootAction,

39

Exercise: Object-oriented views

Add “bar.jelly” to MyRootAction Modify index.jelly to add a hyperlink to the

new page Refactor MyRootAction and introduce a base

type Push down bar.jelly to the new base class. See

how the UI behaves

40

URL → Object graph mapping

41

URL → Object graph mapping

Typical traversal methods getFoo() → …/foo/ getFoo(“bar”) → …/foo/bar/ getDynamic(“foo”) → …/foo/…

Exact rules are in https://stapler.dev.java.net/reference.html

42

Exercise: URL → Object graph

Define immutable Employee class And populate several of them from MyRootAction

Define index.jelly on Employee and create navigable UI

public class Employee { public int number; public String name; public Employee boss; public List<Employee> reports; …}

public class MyRootAction { List<Employee> employees;}

43

Programmatic request handling

Instead of “foo.jelly”, define “doFoo” method Parameters are injected

StaplerRequest (<: HttpServletRequest) StaplerResponse (<: HttpServletResponse) @QueryParameter @AncestorInPath

Return value / exception becomes HTTP response (or void) Or void and control it yourself org.kohsuke.stapler.HttpResponse

44

Demo

Employee.doTerminate()

45

Exercise: doXyz method

Define a form that takes one text field and update the employee name

Persistence

47

XStream

This is how Hudson persists most data Characteristics

Clean almost human-readable XML No mapping required / semi-transparent to code Supports arbitrary object graph Customizable enough Robust in the face of partial unmarshalling failure

48

XStream: naïve example

public class Employee { public int number; public String name;}

<org.acme.Employee><number>1</number><name>Kohskue</name>

</org.acme.Employee>

49

Persistence and extension points

If/how your extensions are persisted is described in javadoc E.g., Builder is persisted as a part of

AbstractProject SecurityRealm is persisted as a part of Hudson etc.

50

XmlFile

Used if you need to persist things on your own

Represents a XML file that contains persisted objects

Important methods write(Object) Object read() void unmarshal(Object)

51

Data format evolution

Fields can be added to class Field is left to VM default value when reading old data

Fields can be deleted from class Extra data is thrown away when reading old data

Fields can change type But only if the XML remains compatible

Transient fields are read but not written Convenient for migration that involves computation

(in conjunction with readObject)

52

Exercise: persistence

Creates MyRootAction.doTest method And use this method to experiment with XmlFile

Test data format evolution Define a class that has a boolean field Save a file Evolve this boolean to 3-state enum Define a data migration logic▪ Must be able to read old data▪ But write 3-stete enum, not boolean

53

Exercise: persistence

Add a new boolean config option to HelloWorldBuilder And it should default to true

Remoting

55

Remoting overview

How does Hudson execute code on slaves, when slaves only has 17KB slave.jar?Q

56

The answer

By sending class files and loading them into classloaders on-demand

Master serializes a Callable, slave execute that and the result is send back to master

All in all, not so far away from what RMI did

57

Basics

A pair of Channels in 2 JVMs Connected by Input/OutputStream pair

Not sure how to get to Channel? Computer.getChannel() Computer.currentComputer()

JVM1 JVM2

Channel Channel

58

Careful!

Serialization sucks in everything referencible Convenient but can be dangerous Beware of anonymous Callable▪ static is your friend

Don’t mask InterruptedException

59

Exercise: remoting basics

Connect a few slaves Define a few slaves with JNLP launcher Launch them from your own computer▪ You now have several VMs (but on the same machine)

Use Channel.call and retrieve RuntimeMXBean.getName()

60

More ways to pass data

Pipe Mechanism for creating stream between local and

remote

RemoteInputStream/RemoteOutputStream Not symmetric in performance

61

Moving computation vs moving data

Execute on master Data is remote Easier to write Mostly transparent

Thanks to FilePath, Launcher, etc.

Execute on slave Data is local Scales better Performs better Explicit use of remoting

Start from left and shift to rightas you gain more experience

Unit Testing

63

Introducing HudsonTestCase

Our attempt at making testing easier JUnit extension Also a great sandbox environment to

experiment

64

Embedded Jetty

Hudson is started inside the JVM with Jetty Random available port for HTTP Create a real deployed environment Number of annotations to control initial state

Integrated HtmlUnit to emulate browser access Bunch of convenience methods

65

In-memory access

All Hudson objects accessible directly Drastically simplifies test set up and side-effect

verifications A number of convenience methods

66

Demo

Create/setup a project Perform a build Check the result Create a slave Use HtmlUnit Interactive Break JavaScript Debugger

67

Exercise: write unit tests

Makes sure that HelloWorldBuilder did a proper greeting

Hints HudsonTestCase.createFreeStyleProject() FreeStyleProject.getBuildersList().add() HudsonTestCase.assertBuildStatusSuccess() FreeStyleBuild.getLog()

68

Exercise: write unit tests

Make sure MyRootAction shows in UI Hints

HudsonTestCase.createWebClient()

Internationalization

70

i18n Basics

i18n in Jelly files i18n in Java source files i18n in help HTML files

71

i18n in Jelly

${%...} notation for default English locale

Translations

Glued together by naming convention foo.jelly and foo_ja.properties, side-by-side

<h1>${%Output from Maven}</h1>

Output\ from\ Maven=Mavenからの出力

72

i18n in Jelly with parameters

${%...(…)}

Translations in MessageFormat syntax

<h1>${%output(it.name)}</h1>

foo.properties:output=Output from {0}

foo_ja.properties:Output={0}からの出力

73

i18n in Java source code

Hudson expects one Messages.properties per package Build generates type-safe Messages class to use

resources

74

i18n in help files

Glued together by naming convention help-foo_ja.html for help-foo.html

Ditto for Jelly-based help files

Phew

76

Where to go from here?

Keep in touch with us at hudson-dev@googlegroups.com and IRC #hudson

Look for plugins that does something similar, use it as the basis

Host the plugins with the Hudson community

Q&A

Kohsuke Kawaguchikohsuke.kawaguchi@cloudbees.comCloudBees, Inc.