Bci for Beginners

IBM Brand Template

Iain Lewis 30 Sept 2013

BCI for Beginners

Important Disclaimers

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBMS CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS

About me

Iain Lewis

QA Engineer, IBM Java Technology Center, IBM Hursley, U.K.

13 years experience developing and deploying Java SDKs

Contact [email protected]

Visit the IBM booth #5112 and meet other IBM developers at JavaOne 2013

Agenda

Who am I, and why am I here

What is BCI?

BCI and the JVM

Practical BCI

Who am I?

Test IBM's Java SDKI break Java

Focus on system testing in QAMulti-threaded load testing

Third-party applications

System testing the Java SDK means writing Java

Can the VM handle any application or Java code?

Recently wrote some BCI based testing

Why are you here?

Have heard of BCI

Would like to know more aboutWhat BCI is

What BCI can do

Would like practical help onHow to get started

Where to get help

What tools are available

As the title of the session suggests, the idea was to get together people who are interested in BCI, but don't yet know where they are going with it.

I'm not a world expert!Can help you avoid some of the pitfalls(new lines in jar manifest files)

Suggestions for tools to help you

What is BCI?

Bytecode Instrumentation is modifying a Java class without the source

Can be done offline, at load time, or at any point afterwards, and multiple times

Application and most SDK classes can be changed

Class File

BCI

Modified ClassJVM

Disk/Network

Runtime

Bytecode technically refers to the opcodes in a class, that is the method bodies. However, the whole class file can be modified

BCI can be added/removed at will while an application is running

Why BCI?

Support for BCI at runtime is built into the Java SDK

Java bytecodes are well documented and easy to work with

BCI is supported by many tools

BCI is widely used

Not a hack,Been available since java 5

What can you do with BCI?

Offline/at load timeReplace the entire class with a different one!

Add/Remove fields and methods

At any timeModify the logic of existing methods

Insert new logic into existing methods

Allows you to doProfiling/monitoring

Code coverage

Code generation

Runtime logging

At load time can do almost anything:Restrictions: You can't go mad! If you removed a method from a class which implements an interface, the class would no longer verify, and would fail to load

Some things can't be done whilst running

maybe allow you to replace a newer version if for some reason it was difficult to get a new version/recompile

What can you do with BCI? (cont)

public void service(ServletRequest req, ServletResponse res) throws ServletException, java.io.IOException{// insert new logic here

// existing method logic}

Examples: Monitoring, logging/tracing, profiling

Servlets running in an application server (tomcat, websphere, glassfish)

Running the server, no idea what the application doesCan inject instrumentation at the top of this method to recordURL, time taken, where the request has come from..

Why BCI?

Doesn't have to be compiled in to a class

Doesn't need access to the source

Can be turned on and off

Can be applied to an arbitrary application

When turned off, has no performance impact

There are other mechanisms for doing some of the things mentioned previously

Why do them using BCI?

How do you do it? JVM interface

BCI is supported at the JVM level

Supported via 'agents' and the Instrumentation interface

java -javaagent:/path/to/agent.jar[=options] MyClass

Native agents also possible via JVMTI

My main focus is dynamic BCI, where the classes are modified either as the JVM loads the classes, or whilst an application is running.

Agents can be written in c, or java.Java agents are easier to deal withJVMTI has a larger interface, allows your agent to do more than just BCI

How do you do it? JVM interface (cont.)

import java.lang.instrument.Instrumentation;public class MyJavaAgent { public static void premain(String agentArgument, Instrumentation instrumentation) {} }

Manifest-Version: 1.0Premain-Class: MyJavaAgentCan-Retransform-Classes: true

Remember the new line at the end of the manifest!

How do you do it? JVM interface (cont.)

Allows your agent to:Register as a ClassFileTransformer

Transformers get a call back when a class is loaded, in order to transform it

Can also trigger retransformations via retransformClasses()

Documentation of Instrumentation :http://docs.oracle.com/javase/7/docs/api/index.html?java/lang/instrument/Instrumentation.html

voidaddTransformer(ClassFileTransformer transformer)

A class file transformer gets a call back whenver a new class is loadedRetransform is how you trigger modifcation of an already running class

How do you do it? JVM Interface (cont)

A class is an array of bytes

A transformer returns a modified copy of the byte array

Class byte arrayfrom diskTransformer 1Copied/Modifiedbyte arrayTransformer 2Finalbyte arraypublic static byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) {}

All transformations boil down to:Take an array of bytes, copy and modify the array, give it back to the JVM

How do you do it? Modifying Bytes (the hard way)

Read the Java Virtual Machine Specificationhttp://docs.oracle.com/javase/specs/jvms/se7/html/

Parse the class byte arrayhttp://docs.oracle.com/javase/7/docs/api/java/io/DataInputStream.html

Return a modified version

Offline- all of the above would be done by reading in a class file into a byte array

How do you do it? Modifying Bytes (the hard way)

DataInputStream is = new DataInputStream(new ByteArrayInputStream(classFileBuffer))

int magic = is.readInt();System.out.println("Magic number is " + magic); // 0xCAFEBABE

int minor = is.readShort();System.out.println("minor number is " + minor); // 0 for Java 7

int major = is.readShort();System.out.println("major number is " + major); // 51 for Java 7

SummaryJava SDK provides a mechanism for transforming classesGives you a a byte arrayNot very much other helpNow, need some help to actually twiddle the bytes

How do you do it? Modifying Bytes (the easier way)

Use a frameworke.g. ASM http://asm.ow2.org

Use javac/javap

Use a bytecode plugin for your IDEBytecode outline plugin for Eclipse http://andrei.gmxhome.de/bytecode/index.html

Write as much of your code in Java as possible

The following slides focus on making it easy to add new logic to an existing classOther frameworks are availableJavassist, bcel

n.b A basic understanding of how a JVM works is still required

Using a Framework (ASM)

Provides an abstraction layer to hide the details of a class

Easier to learn (provides symbolic constants, helper APIs)

Widely used

Under active development

Is an additional dependency

Doesn't tell you what bytecodes to create!

Easier to learn, but non-trivial java api

Does provide helper methods to do make common tasks easier(e.g. local variable sorter)

Now give brief overview, No details emphasis, this is just java, and there are only 3 classes and three methods

A transformer in 3 classes and 3 methods class 1

/** Implementation of transform() in ClassFileTransformer */public byte[]transform(...) throws IllegalClassFormatException{ClassWriter writer = new ClassWriter(ClassWriter.COMPUTE_FRAMES);ClassAdapter1 adapter = new ClassAdapter1(writer);ClassReader cr = new ClassReader(classfileBuffer);cr.accept(adapter, ClassReader.SKIP_FRAMES);return writer.toByteArray();}

This is class one, create a new instance of a method adapter


/** Override of visitMethod in ClassVisitor */public MethodVisitor visitMethod(...) {MethodVisitor mv = cv.visitMethod(...);return new MethodAdapter1(mv);}

In method adapter, override one method, create a MethodAdapter class


/** Override of visitCode in MethodVisitor */public void visitCode() {mv.visitCode();// Use ASM api calls to add profiler logic here}

This gets you to the point of:My class is parsed, I have a method body to create/modifyASM provides methods to add bytecodes butDoesn't tell you what bytecodes to add!Knowledge of the JVM architecture is still required

Next step, make creating new logic easier

Writing bytecodes with javac/javap

Writing logic in bytecode is possible, but not simple

Get your computer to do it for you!

Write your logic in Java

javac turns Java code into bytecode, packaged up in a class file

javap -v turns a class file into a human-readable format

Why javap? (and why not)Shipped with the Java SDK

Easy to use

Shows you the whole of the class, not just methods (helpful for learning)

Not easy to translate javap output into the Java code to produce it

Javap on HelloWorld

public static void main(String[] args) {System.out.println(Hello World);}

javap -v HelloWorld.class

Javap on HelloWorld

(...)public static void main(java.lang.String[]); flags: ACC_PUBLIC, ACC_STATIC Code: stack=2, locals=1, args_size=1 0: getstatic #16 // Field java/lang/System.out:Ljava/io/PrintStream; 3: ldc #22 // String Hello World 5: invokevirtual #24 // Method java/io/PrintStream.println:(Ljava/lang/String;)V 8: return LineNumberTable: line 7: 0 line 9: 8 LocalVariableTable: Start Length Slot Name Signature 0 9 0 args [Ljava/lang/String;(...)

This is just part of the output of javapAlso includes the whole of the constant poolEvery method in the classAll the meta data (stack maps, line number tabel, variable table)Not much help to go rebuild a similar method in your agentBut I do recommend it as a help to understanding how the JVM works

Writing bytecodes with the Bytecode Outline Eclipse Plugin

Plugin for Eclipse

Turns Java source code into bytecodes

Also shows you the Java code using ASM API calls to create the equivalent bytecode

(demo)

Demo

Very simple counting profiler

Prints out the number of times each method in your application was called

(demo)

Show the profiler logic in java show the bci logic in java Show byte code plugin to turn that into ASM (show view, other, byte code, use switch to change between asm and non-asm) Paste ASM code into boiler plate method transformer

Recap

Use a frameworke.g. ASM http://asm.ow2.org

Get your computer to write your bytecode for youUse javac and javap

Use the Bytecode Outline Plugin for Eclipse (or similar)

Write as much of your code in Java as possible

BCI is very powerful and used in many products

JVM knowledge is required but...

There are many tools available to make it easier

The following slides focus on the task of - adding new logic to an existing class

n.b A basic understanding of how a JVM works is still required

http://ibm.co/JavaOne2013

IBM booth #5112Wednesday 1pm - 3pm

References

Eclipse Bytecode Outline plugin (allows you to use asmifier in Eclipse)http://andrei.gmxhome.de/bytecode/index.html

ASMhttp://asm.ow2.org/index.html

java.lang.instrument package documentationhttp://docs.oracle.com/javase/7/docs/api/index.html?java/lang/instrument/package-summary.html

References

Inside the Java Virtual Machine Bill Vennershttp://www.artima.com/insidejvm/ed2/jvm.html

Java VM Spechttp://docs.oracle.com/javase/specs/jvms/se7/html/index.html

blue-logo

2013 IBM Corporation

IBM Confidential

23 September 2013

blue-logo 2013 IBM Corporation

37-degree-pos-tri-logo

blue-logo 2013 IBM Corporation

BCI for Beginners

23 September 2013

Click to edit the title text format

Click to edit the outline text formatSecond Outline LevelThird Outline Level


blue-logo


23 September 2013

Bci for Beginners

Technology

Transcript of Bci for Beginners