Challenges of resource management in an os gi environment chris gray + christer larsson

COPYRIGHT © 2008-2011 OSGi Alliance. All Rights Reserved

Challenges of Resource Management in an OSGi Environment

Chris Gray, /K/ Embedded Java SolutionsChrister Larsson, Makewave

20 September 2011

OSGi Alliance Marketing © 2008-2010 . All Rights Reserved

/K/ Embedded Java Solutions

onsdag den 21 september 2011

OSGi Community Event 2011 Darmstadt - Chris Gray / Christer Larsson

The Problem

• OSGi is a fine dynamic modular system which allows many independent modules to co-exist within a single JVM

• Supports the business objective of offering 3rd-party services on top of supplier-provided services

• BUT: there is a need to protect the platform against badly-written / malicious bundles



Types of Misbehavior

Operating System

JVM Process

Other OS Process

Other OS Process

Other OS ProcessOSGi Framework

Bundle

Bundle

Bundle Bundle

Bundle

Bundle




Operating System

JVM Process

Other OS Process

Other OS Process


Bundle

Bundle

Bundle Bundle

Bundle

Bundle

Excessive Memory Allocation




Operating System

JVM Process

Other OS Process

Other OS Process


Bundle

Bundle

Bundle Bundle

Bundle

Bundle

Excessive Memory Allocation

Excessive CPU usage, or using too many Threads




Operating System

Other OS Process

Other OS Process

Other OS Process

JVM / OSGi Rebooting. Please Wait.....



Potential solutions to the problem

• Use Resource Management inside the VM

• Run multiple VM instances to isolate critical OSGi applications from user application

• Careful and rigorous testing of all bundles before the are installed

• These three items can of course be combined



A Mika / Knopflerfish solution - resource constrain the bundles

Add a Resource Manager that can account for Bundle Resources using built-in VM features

Operating System

Mika Max

Other OS Process

Other OS Process

Other OS ProcessKnopflerfish OSGi Framework

Bundle

Bundle

Bundle Bundle

Bundle

Bundle

Resource Manager



Identifying the culprit (1)

• The first step is to associate resource allocations with modules.

• Various systems have been devised for this, but generally they are not transparent

• Certain APIs must be called for its resource consumption to be monitored.

• But there is no guarantee all code is written in this way

• And legacy code, or 3rd party libs are certainly not!

• Instead we propose to gather resource usage information by instrumenting the Java runtime itself.



Identifying the culprit - (2)

• OSGi helps us by allocating a unique classloader to each bundle. We define the “current bundle class loader” of a thread T as follows:

• If T is the system startup thread then the current bundle class loader of T's base frame is the system class loader.

• Otherwise the bundle class loader of T's base frame is inherited from the thread frame which called T.start().

• The bundle class loader of any other frame is the class loader of the method it is executing, or if this is the system class loader then it is inherited from the parent frame.



Identifying the culprit - (3)

• For the purpose of this paper the bootstrap class loader, extension class loader, and system class loader are all lumped together as “system”. Note that this includes resources consumed by the framework itself.

• The overhead is much less than for the “stack inspection” required by the Java2 fine-grained security model, because we only need to follow a few pointers (frame→method→class→loader) and test for equality. Could be further optimised using auxiliary data structures.



Resource Accounting

• For each class loader we maintain a data structure with counters for each monitored resource.

• Count up when resource is allocated

• Count down when released, BUT

• Resource is often released by another thread, e.g. Garbage collector or finalizer thread. Therefore we need to remember which class loader allocated which resource! Furthermore the resource may outlive the classloader which allocated it, so we use a “handle” instead of a direct pointer to the class loader.

• CPU usage is tricky, as depends on OS and on threading model.



Operating System

Mika Max

Class Loader

Class Loader

Bundle Resource AccountingFor each monitored bundle resources are accounted

and associated with the bundle’s Class Loader

Other OS Process

Other OS Process

Other OS ProcessKnopflerfish OSGi Framework

Bundle

Bundle

Bundle Bundle

Bundle

Bundle

Resource Manager



Resource Accounting (2)

• Knopflerfish Pro has a Resource Manager

• For every bundle quota can be set on CPU, memory and number of Threads used

• When a Bundle has exceeded his quota of some resource the Resource Manager is notified and can take action.

• But it is not always obvious what to do!



Dealing with Resource Overconsumption (1)

• The default action when the bundle over-consumes is as follows

• Overconsumption of Memory

• Throw an OutOfMemoryError

• Overconsumption of Threads or CPU

• Throw an InternalError

• Alternatively a user-defined callback can be invoked at that point



Dealing with Resource Overconsumption (2)

• The Resource Manager can supply information on over-consuming bundles to a Management Agent.

• The Management Agent can also query the Resource Manager for current bundle resources use and take action before quota is reached

• The Management Agent can then decide to send an alarm and / or stop the bundle.

• For a well-behaving bundle this is no problem

• But for a malicious, or badly written bundle that has gone bananas there are new problems on the horizon...



Difficulties with Java Threads

• Java threads are not OS processes, i.e.

• Java threads can not be destroyed, stopped or suspended. Attempting to do that is unsafe and may lead to unpredictable results. See javadoc for java.lang.Thread for more details.

• This means Activator start() and stop() methods that do not return can not be stopped or killed like an OS process.

• If this happens in Knopflerfish we:

• invalidate the Bundle after a timeout period so that bundle no longer is usable. All its services are unregistered and it can not get new services.

• Set the threads prio to lowest possible.



Showtime!



Thank you

Chris Gray - /K/ Embedded Java Solutions - www.k-embedded-java.comChrister Larsson - Makewave - www.makewave.com


http://www.k-embedded-java.com

http://www.k-embedded-java.com

http://www.makewave.com

http://www.makewave.com

Challenges of resource management in an os gi environment chris gray + christer larsson

Technology

Transcript of Challenges of resource management in an os gi environment chris gray + christer larsson