A Comprehensive Experimental Comparison of Event Driven ...

A Comprehensive Experimental Comparison ofEvent Driven and Multi-Threaded Sensor Node

Operating SystemsCormac Duffy1, Utz Roedig2, John Herbert1, Cormac Sreenan1

1Computer Science Department, University College Cork, Ireland

Email: {c.duffy,j.herbert,c.sreenan}@cs.ucc.ie2InfoLab21, Lancaster University, UK

Email: [email protected]

Abstract— The capabilities of a sensor network arestrongly influenced by the operating system used on thesensor nodes. In general, two different sensor network oper-ating system types are currently considered:event driven andmulti-threaded. It is commonly assumed that event drivenoperating systems are more suited to sensor networks asthey use less memory and processing resources. However, iffactors other than resource usage are considered important,a multi-threaded system might be preferred. This papercompares the resource needs of multi-threaded and eventdriven sensor network operating systems. The resourcesconsidered are memory usage and power consumption.Additionally, the event handling capabilities of event drivenand multi-threaded operating systems are analyzed andcompared. The results presented in this paper show thatfor a number of application areas a thread-based sensornetwork operating system is feasible and preferable.

Index Terms— Sensor Networks, Sensor Network Oper-ating Systems, Performance Evaluation, TinyOS, MANTIS.

I. I NTRODUCTION

Wireless sensor networks consist of battery poweredsensor nodes used to gather information about a moni-tored physical phenomenon. To ensure long periods ofunattended network operation, the energy consumption ofthe sensor nodes must be very low. The operating systemused for the sensor nodes influences energy consumptionon two levels. First, the design of the operating systemdefines the minimum resource requirements such as CPUspeed and memory capacity that the sensor hardware mustprovide. Second, the operating system design influencesthe usage pattern of the CPU and thus defines how oftenenergy-efficient sleep periods can be activated. As thesensor node must provide a service, the operating systemdesign can not be focused solely on energy efficiency.Sensor readings and incoming messages must be handledpromptly by the sensor node. The responsiveness of thenode must be sufficient to handle events and requestsaccording to their deadlines.

Application responsiveness is influenced by the oper-ating system design in two ways. First, the responsive-ness depends on the task scheduling capabilities of theoperating system. Second, the design of the operating

system influences the degree to which task scheduling isperformed by the system or by the systems programmer.Obviously, the last point is subjective as it depends on thequality and knowledge of the programmers. Nevertheless,the issue has strong impact in real world applicationdesigns.

Currently, operating systems for sensor nodes followeither one of two different design concepts. Operatingsystems following the first design approach are calledevent driven. Each operating system process is triggeredin response to an event (e.g. a timer, an interrupt indicat-ing new sensor readings or an incoming radio packet).The tasks associated with the event are executed andthereafter the node is sent to an energy-efficient sleep stateor the next event is processed. As events are processedsequentially, expensive context switching between tasks isnot necessary. An example of such an operating systemis TinyOS [1]. The second approach follows the classicmulti-threadedoperating system design. The operatingsystem multiplexes execution time between the differenttasks, implemented as threads. While switching from onethread to another, the current context has to be saved andthe new context must be restored. This consumes costlyresources in the constrained sensor node. An example ofsuch an operating system for sensor nodes is MANTIS[2].

Currently it is assumed that an event driven operatingsystem is more suitable for sensor networks because lessresources are needed resulting in a more energy-efficientsystem. However, the exact figures are unknown andtherefore determined and presented in this paper. On theother hand it is claimed that a multi-threaded operatingsystem has superior event processing capabilities. Again,an in depth analysis is currently missing and is there-fore shown in the paper. For comparison purposes, theevent-based systemTinyOSand the multi-threaded systemMANTISexecuting the same sensor network applicationson theDSYS25[3] sensor platform are used. The contri-butions of the paper can be summarized as follows:

1) Comparison of the memory requirements of theevent-based TinyOS and multi-threaded MANTISoperating system.

JOURNAL OF NETWORKS, VOL. 3, NO. 3, MARCH 2008 57

© 2008 ACADEMY PUBLISHER

2) Comparison of the event processing capabilities ofthe event-based TinyOS and multi-threaded MAN-TIS operating system.

3) Comparison of the energy consumption of theevent-based TinyOS and multi-threaded MANTISoperating system.

The results presented in the paper can be used to decidewhich type of operating system should be used for aspecific sensor network application. The results also showthat for a number of application areas a thread-basedsensor network operating system is actually feasible andeven preferable.

The rest of the paper is organized as follows. Thenext section describes related work in the research areaof operating systems for sensor networks. Section IIIgives an overview of the operating systems TinyOS andMANTIS. Section IV describes the sensor platform, appli-cation scenarios and their implementation as used for thecomparative experiments. Sections V,VII and VI presentthe experimental comparison of the operating systems.Section VIII concludes the paper.

II. RELATED WORK

Historically, there has been much debate on whetheran event-based or multi-threaded architecture is moreefficient. To decide which operating system architectureshould be used, the specific application environment ofinterest has to be considered as each application envi-ronment dictates very specific constraints. Unfortunately,results obtained for one application environment are nor-mally not directly applicable to another environment.Thus, a comprehensive study covering a multitude ofapplication scenarios is required.

Preliminary results of our research presented in thispaper were published in [4]. This paper describes theperformed experiments in much greater detail. In addi-tion, the results and their implications are analyzed anddiscussed.

Besides our preliminary work, no published researchexists that presents a comprehensive comparison of event-based and multi-thread sensor network operating systemstaking event processing, energy consumption and memoryusage into account. The lack of such a study is the mainmotivation for the work presented in this paper. Existingwork targets only a subset of aspects investigated in thispaper. For example papers analyzing or describing onespecific operating system (e.g. [5], [6], [7], [8], [9]), orpublications comparing only one aspect (e.g. memoryusage in [6]). As each single existing analysis is basedon different assumptions and experimental setups, it isnot possible to extract an objective comparison.

As previously mentioned, the choice of operating sys-tem design has an impact on the way an application isprogrammed. In [10] the event-based system CONTIKI[7]uses a programming concept called “proto-threads” whichallows the programmer to develop a program using amulti-threaded programming syntax. It is argued thatan event-based system is more power efficient but that

programming concurrent (sensor network) applicationswith threads as opposed to event handlers is easier forthe programmer. The proto-threads in CONTIKI allowa combination of both benefits. However, an objectiveperformance comparison of an event-based and multi-threaded system is not provided.

In [11] it is argued that TinyOS, in contrast to MAN-TIS, has a problem with multiplexing long running tasksand short running tasks. However, an in-depth analysis- an analytical evaluation or by measurement - of thetask handling problem is not given. It is argued that bothoperating systems should be combined to overcome theexisting problems. As a solution, the complete TinyOSoperating system is executed as a thread in MANTIS. Iftasks in TinyOS need to be able to preempt other tasks,they can be executed in a separate MANTIS thread. Verysimilar earlier work [12] uses the AvrX [13] as a multi-threaded extension of the TinyOS scheduler. In this work,the event processing capabilities of the basic TinyOS andthe modified AvrX extended TinyOS are investigated (avery similar experimental setup to the setup described inSection IV is used for evaluation). However, an analysis ofother parameters such as power consumption and memoryusage is omitted. The differences of a generic thread-based system are also not investigated. In summary, theresearch focus of [11] and [12], is to integrate multi-threaded features in TinyOS.

In [14] TinyOS is compared with eCos, an embeddedmulti-threaded operating system. eCos is not specifi-cally a sensor network operating system as many ofthe core operating features are designed for more com-plex embedded processors. The paper does evaluate bothmemory, processing performance and power efficiency.For processing performance the number of clock cyclesrequired to process kernel functions are summed up, butactual application performance is not investigated. Thepower performance is compared theoretically based onthe memory requirements of each operating system, butno investigation into operating system power-managementis analyzed in this paper.

To our knowledge, this is the first work that providesspecific data to determine the effectiveness of both anevent-based and a multi-threaded operating system in arange of application scenarios.

III. SENSORNODE OPERATING SYSTEMS

Two different sensor network operating system typesare currently considered:event drivenandmulti-threaded.To compare both operating system concepts, a well knownand widely used implementation of each was selected,namelyTinyOSand MANTIS. The following paragraphsdescribe the basic functionality of each operating system,especially regarding power consumption and event pro-cessing which are the parameters of interest in our study.It has to be noted that the terms event, task and thread areused differently within the literature describing TinyOSand MANTIS. Thus, definitions are given below to avoidconfusion:

58 JOURNAL OF NETWORKS, VOL. 3, NO. 3, MARCH 2008


• Event:An environmental occurrence.• Interrupt: An interrupt triggered by an event.• Event-Handler:A system function, invoked in re-

sponse to an interrupt.• TinyOS Task:A deferred procedure, usually triggered

by an event-handler.• MANTIS Thread:A portion of the MANTIS OS

that can run independently of, and concurrentlywith, other portions of the OS, typically invoked inresponse to an interrupt.

A. TinyOS

The TinyOS [1] operating system was one of thefirst operating systems specifically designed for wirelesssensor networks. The inventors felt that an event drivenlight-weight kernel was the best solution to handling a“large number of concurrent flows and juggle numerousout-standing events”.

The operating system and specialized applications arewritten in the programming language nesC and are orga-nized in self-contained components. A simplified view ofthis component structure is shown in Fig. 1. Componentsconsist of interfaces in the form ofcommandand eventfunctions. Components are assembled together, connect-ing interfaces used by components to interfaces providedby others, forming a customized sensor application. Theresulting component architecture facilitates event-basedprocessing by implementing event-handlers and TinyOStasks. TinyOS tasks are deferred function calls and areplaced in a simple FIFO task-queue for execution (seeFig. 1, line 8). TinyOS tasks are taken sequentially fromthe queue and are run to completion. Once running,the TinyOS task can not be interrupted (preempted) byanother TinyOS task. Event-handlers are triggered inresponse to a hardware interrupt and are able to preemptthe execution of a currently running TinyOS task (seeFig. 1, line 5). Event-handlers perform the minimumamount of processing to service the event. Further nontime-critical processing is performed within a TinyOStask that is created by the event handler. After all TinyOStasks in the task queue are executed, the TinyOS systementers a sleep state to conserve energy (see Fig. 1, line 10).The sleep state is terminated if an interrupt occurs.

Functionality in TinyOS is distributed among manycomponents. Each function (e.g. sensing or packet for-warding) is normally segmented into a series of sub-steps, (task, command and event hander functions). Thus,functionality can execute concurrently by multiplexingthese atomic sub-steps. When used correctly, this ap-proach leads to an efficient system structure. On theother hand it is difficult for the programmer to achievea proper division of functionality. The programmer mustbe familiar with the internal affairs of all low levelcomponents in the system. Additionally, it can be difficultfor some programmers to handle functionality in suchsub-steps.

1: component_A2: task do(){...}3: command X(){...}4: event Y(){...}

5: int_A6: ...7: post_task(A)

8: TOSH_run_task()9: while(TOSH_run_next_task())

10: TOSH_sleep()

Fig. 1. TinyOS structure

B. MANTIS

The MANTIS operating system [2] is a light-weightmulti-threaded operating system capable of multi-taskingprocesses on energy constrained distributed sensor net-works. The system provides a classical multi threadedsystem in the context of wireless sensor networks.

Each task the operating system must support can beimplemented - using standard C - as a separate MANTISthread. A simplified view of this thread structure is shownin Fig. 2. A new thread is initialized and thread processingis started (line 1). Processing might be halted using thefunctionmos_semaphore_waitwhen a thread has to waitfor a resource to become available (line 3). An interrupthandler (line 4) using the functionmos_semaphore_post(line 5) is used to signal the waiting thread that theresource is now available and thread processing is re-sumed. While a thread is waiting on a resource to becomeavailable, other threads might be activated or, if no otherprocessing is required, a power saving mode is entered.Power saving is handled by a thread called idle-taskwhich is scheduled when no other threads are active.Thread scheduling is performed within the kernel functiondispatch_threadshown in Fig. 2, line 6. This functionsearches a data structure calledreadyQfor the highest pri-oritized thread and activates it. When thedispatch_threadfunction is called, the current active thread is suspendedcalling PUSH_THREAD_STACK(line 7) which savesCPU register information. The highest priority threadis then selected from thereadyQ (line 8) and its reg-ister values are restored by thePOP_THREAD_STACKfunction (line 10). Before thedispatch_threadfunctionis called, thereadyQ structure is updated. Threads thatare currently sleeping or that are waiting on a semaphore(resource) are excluded from thereadyQ. The schedulingthrough thedispatch_threadfunction can be initiated bytwo different means.Dispatch_threadis called when asemaphore operation is called (e.g. to let the currentthread wait on a resource).Dispatch_threadis also calledperiodically by a time slice timer to ensure processing ofall threads according to their priority.

Like all multi-threaded operating systems, MANTISwas developed with a complement of built-in mem-ory protection techniques such as binary and countingsemaphores to manage and coordinate threads. MANTISthreads are implemented in the MANTIS kernel witha static thread table, which stored priority and state



1: thread_A2: while(running)3: ...;mos_semaphore_wait(A1);...

4: int_A5: ...;mos_semaphore_post(A1);...

6:dispatch_thread()7: PUSH_THREAD_STACK()8: CURRENT_THREAD = readyQ.getThread()9: CURRENT_THREAD.state=RUNNING

10: POP_THREAD_STACK()

Fig. 2. MANTIS structure

information for each MANTIS thread and a pointer tothe thread stack. Typically only 12 MANTIS threads canbe either queued or active at any one time and willconsume 120 bytes of SRAM [2]. The heap space ofeach thread is managed by the kernel memory manager.The memory manager assigns thread space according to abest-fit policy. Thread heaps will, by default, be assignedin the lowest possible memory address spaces, ensuringthat a thread heap space doesn’t collide with the processorheap.

By defining processes as threads, algorithms can berepresented in a more intuitive sequential fashion asprocesses don’t need to be segmented into specific stateswhen waiting for a system event to occur. Compared toTinyOS, functionality is multiplexed by the OS, not by theprogrammer. Thus, MANTIS has the potential of beingeasier for programmers to use.

IV. EVALUATION SETUP

For the evaluation of event processing capabilities,power consumption patterns and memory usage ofTinyOS and MANTIS, both operating systems wereported to the same sensor platform, the DSYS25 [3].Additionally, measurement facilities are integrated in bothoperating systems which allow us to observe the requiredparameters without altering the system behavior. To ac-tually perform the comparative evaluation, an abstractapplication scenario is defined and implemented on thesensor nodes. This abstract application scenario corre-sponds to a broad spectrum of real-world sensor networkdeployments.

In the following paragraphs, the selected abstract ap-plication scenario is motivated and described. Thereafter,the DSYS25 sensor-node platform, the ported operatingsystems and the measurement hooks are explained. Thisevaluation setup is used for the experiments described inthe remaining sections of the paper.

A. Application Scenarios

To evaluate operating system performance, an applica-tion context must be defined. Subsequently, the operatingsystem performance of a sensor node supporting the givenscenario can be investigated. Obviously, to obtain usefulresults, it is important that the investigated applicationscenario corresponds to real-world deployment and usagescenarios of wireless sensor networks.

In many cases, a sensor network is used to collectperiodically obtained measurement data at a central point(sink or base-station) for further analysis. The sensornodes in such a network perform two major tasks. Sensornodes perform the sensing task and they are used toforward the gathered data to the sink. If the sink is notin direct radio range of a node, other nodes closer tothe sink are used to forward data. The execution timeof the sensing task will depend on the nature of thephysical phenomenon monitored and the complexity ofthe algorithm used to analyze it. Therefore, the positionof the node in such a network and the complexity ofthe sensing task define the operating system load of thesensor node. The complexity of the sensing task is variedin the experiments and hence the application scenario isconsidered abstract, as it can be compared with manydifferent real-world deployment scenarios.

The complexity of the sensing operation depends onthe phenomenon monitored, the sensor device used andthe data pre-processing required. As a result, the operatingsystem can be stressed very differently. If, for example, anATMEGA128 CPU with a processing speed of4Mhz isconsidered (a currently popular choice for sensor nodes),a simple temperature sensing task (processed through theAnalogue to Digital Converter) can be performed in lessthan 1ms [15]1. In this case only a16bit value has tobe transferred from the sensing device to the CPU. Ifthe same device is used in conjunction with a camera,image processing might take some time before a decisionis made. Depending on camera resolution and imageprocessing performed, a sensing task can easily take morethan100ms [16]. Other application examples documentedin the literature are situated in between these values. Forexample if a sensor value needs to be cryptographicallysecured before transmission, the sensing task is prolongedby 5ms [17]. Thus, combined with a simple sensor, asensing task can be around10ms. Note that a long sensingtask can be split-up into several sub-tasks. However, inpractice this is often not possible due to two factors.First, many data-processing algorithms are difficult orimpossible to be split into separate tasks. For examplesome image processing algorithms cannot be divided, asexplained in [16]. Second, it is difficult from a program-ming (or programmers) perspective to handle and managesuch divided entities. The experimental evaluation spansthe task sizes described (1ms...100ms). Thus, the abstractapplication scenario corresponds to a wide range of realworld application scenarios.

The following paragraphs give an exact specification ofthe abstract application scenario used, which is definedby its topology, traffic pattern and sensing pattern. Theapplication scenario is then implemented on the DSYS25sensor platform for evaluation.

1This calculation is based on the amount of processing time necessaryto process an analog sensor reading taken from the Atmega ADC. TheADC is typically used to process signals from analog sensorssuch as atemperature sensor.



n=3

n=1

n=2

Fig. 3. Binary Tree

Topology: The sensor network is used to forwardsensor data towards a single base-station in the network. Itis assumed that a tree topology is formed in the network.To simplify the evaluation process, a binary tree shownin Fig. 3 is assumed. Depending on the positionn in thetree, a sensor nodesi might process varying amounts ofpackets. Leaf nodes put less demand on the processor.Nodes closer to the root are more involved in packetforwarding and these nodes have to multiplex packetforwarding operations with their sensing operations. Theposition in the tree has therefore - besides other param-eters - a significant impact on the event processing andenergy consumption properties of the node.

Sensing Pattern:A homogeneous activity in the sensorfield is assumed for the abstract application scenario.Each sensor gathers data with a fixed frequencyfs. Thus,every ts = 1/fs a sensing task of the durationls has tobe processed. As mentioned, the durationls is variablebetween ls = 4000 and ls = 400000 clock cyclesdepending on the type of sensing task under consideration(Which corresponds to1ms/100ms on a4MHz CPU).

Traffic Pattern: Depending on the positionn of a nodesi in the tree, varying amounts of forwarding tasks have tobe performed. It is assumed that no time synchronizationamong the sensors in the network exist. Thus, even ifeach sensor produces data with a fixed frequency, dataforwarding tasks are not created at fixed points in time.The arrival rateλn of packets at a node at tree-leveln ismodeled as a Poisson process. As the packet forwardingactivity is related to the sensing activity in the field,λn

is given by:

λn = (2n− 1) · fs (1)

This equation is a simplification; queuing effects andlosses are neglected, but nevertheless provides an accuratemethod to scale the processing performance requirementsof a sensor network application. It is assumed that theduration (complexity)lp of the packet forwarding task, islp = 4000 clock cycles. This is the effort necessary toread a packet from the transceiver, perform routing andre-send the packet over the transceiver. This is a commonprocessing time and was obtained analyzing the DSYS25sensor nodes using the Nordic radio [18].

Summary: The abstract scenario described above isused in Section V, Section VI and Section VII to compareevent processing capabilities, energy consumption andmemory usage of the two operating systems. For theevaluation, a single DSYS25 sensor node is fitted withthe operating systems under investigation and the packetforwarding tasks and sensing tasks are generated suchthat the nodes activity corresponds to a place in the treetopology. The parameters defining the abstract application

TABLE I

EVALUATION SETUP

Task Duration Clock cycles Frequency

Forwarding Fixed:

1ms

Fixed:

lp = 4000

Variable, Poisson:

λn = (2n−1) ·f

n ∈ 1, ..., 8Sensing Variable:

100ms,75ms,50ms,10ms,5ms,1ms

Variable:

ls = 400000,ls = 300000,ls = 200000,ls = 40000

ls = 20000,ls = 4000

Fixed, Periodic:

fs = 1 1

s

Fig. 4. DSYS25 Sensor Hardware

are set for all following experiments according to thevalues or ranges listed in Table I (Set for the4MHzCPU used in the DSYS25 platform).

B. Evaluation Platform

The DSYS25 [3] is a sensor platform developed aspart of the D-Systems project at University College Corkand the Tyndall Institute. The hardware platform is anATMega128L micro-controller based Lego-like 25mm x25mm stackable system. Its modular nature lends itself tothe development of numerous layers for use in differentapplication scenarios. Layers can be combined in an inno-vative plug and play fashion and include communication,processing, sensing and power. The sensor hardware isdepicted in Fig. 4.

The chosen Nordic nRF2401 transceiver performs com-munication tasks such as address and CRC computation,freeing the micro-controller from these activities. Thus themicro-controller can be either used for processing othertasks or can be sent to an energy saving sleep mode.

C. OS Ports and Measurement Facilities

The goal of the study is a comparison of operatingsystem concepts. To provide a fair comparison our exper-iment hinges on a number of important setup/experimentalparameters.

Firstly, conceptual differences rather than functionaldifferences between the two operating systems must bemeasured. For example the networking stack used inTinyOS and MANTIS provide different functionality (e.gMAC protocols, duty cycles ....). Thus, a direct com-parison of the operating systems might be caused byfunctionality unrelated to the operating system. To avoid



this problem, the operating systems are reduced to theirbare minimum and simple components emulating thebehavior of the aforementioned abstract application areimplemented.

Secondly, in measuring the performance of each appli-cation it is imperative that the performance of the abstractapplication is not hindered. The experiment revolvesaround a setup in which tasks are dispatched up untilthe end time of the experiment. Performance is evaluatedas to how timely each task is completed with respectto this clock. However in measuring clock cycles wetake away processing resources from the tasks and wouldtherefore provide imprecise measurements. This overheadis carefully calculated with an additional hardware timerand subtracted from the results.

TinyOS: In case of TinyOS, the abstract application isimplemented using nesC, the programming language usedin TinyOS. The packet processing task and the sensingtasks are initiated by an interrupt. For the experiment,the interrupt is not generated by the transceiver or sensorhardware, instead, the interrupt is generated by a timer.The timer intervals are configured by the parametersgiven for the abstract application (λn, fs, see Table I) .Within the interrupt routine for the transceiver, a TinyOStask for packet forwarding is created and queued inthe task list. The size of the packet forwarding tasklpis set to 4000 clock cycles (implemented as assemblerNOOP operations, to ensure cycle accurate timings). Inthe interrupt routine of the sensor, the sensing task iscreated. The sensing task has the size (complexity)lsdefined by the application scenario.

The TinyOS is modified such that essential parameterscan be measured during the experiment. The task creationtime in the interrupt routine is recorded. When a taskfinishes, the task duration from creation to completion isknown. Additionally, idle times of the CPU are recorded.The time from the completion of the last task to theoccurrence of a new event is measured. The records ofthese two parameters - task execution time and systemidle time - enables us to analyze event processing andpower consumption capabilities of the operating system.

MANTIS: The thread-based architecture of the MAN-TIS operating system requires a syntactically differentimplementation of the abstract application. The sensingtask and the packet forwarding task are implemented asMANTIS threads. Again, the interrupts initiating packetprocessing and sensing are generated by a timer, notthe hardware. The timer intervals are configured by theparameters given for the abstract application (λn, fs, seeTable I). Following an interrupt, the appropriate interruptroutine is called. Within the interrupt routine, the neces-sary thread for processing is activated. MANTIS allowsfor a prioritization of threads. The packet processingthread is configured to have a higher priority than thesensing thread. The size of the packet forwarding threadlp is set to 4000 clock cycles The sensing thread has thesize (complexity)ls defined by the application scenario.When a thread completes execution, it is set to sleep and

TABLE II

MEMORY USAGE

Operating System Programmable FlashMemory (kB)

Required RAM (B)

TinyOS 9 283MANTIS 13.1 287

waits to be woken again by the appropriate interrupt.The time from waking a thread until its completion is

measured during the experiments. Thus, an analysis of theevent-processing capabilities of MANTIS is possible. Ifno thread is active, MANTIS activates an idle thread. Theidle thread is used to implement the power managementcapabilities of the operating system. The idle thread de-cides which power saving mode has to be activated. Powersaving is terminated when an interrupt occurs. The timefrom activating the idle thread until an interrupt occursis measured. Thus, the power management capabilities ofMANTIS can be investigated.

The following Sections V, VI, VII detail how eachperformance result was measured and the findings andrelevance of these results.

V. M EMORY USAGE

The memory footprint of the operating system has to beas small as possible. Additional memory increases the costof the sensor nodes and the more memory is integrated inthe hardware, the more energy is consumed by the system.In practice, not each reduction of memory requirementsleads to a cheaper and more energy-efficient system. Asensor built out of standard components normally uses achip combining CPU and memory. Whether the systemuses the available memory or not has little impact on thepower consumption of the chip. Thus, improvement canonly be expected if memory (and CPU) requirements canbe reduced to a point where a simpler CPU/memory chipis available.

A. Measuring memory

In order to determine operating system memory usage,we use the GNU project binary utility avr-size [19]. Avr-size is a flash image reader that outputs the program size,and the initialized and uninitialized memory size.

B. TinyOS

The TinyOS operating system core consists of an ab-solute minimum of functionality. Simple algorithms suchas the FIFO queuing algorithm are used to implementcore TinyOS features such as the task scheduler, in orderto maintain compatibility with very limited processors. Astructure is provided to multiplex many timed events witha single hardware timer (see Section III). All additionalfunctionality is provided by the application code in theform of a component-based architecture. The TinyOS coreelements alone do not form a useful system. Thus, it isnecessary to analyze the memory usage of the TinyOSoperation system combined with specific application code.



Operating system and application code are compiled intoa single binary file which is executed on the sensornode. Depending on the application used, the executablerequires an extremely small memory space. The abstractapplication scenario presented in Section IV is used forthe evaluation.

Memory requirements of the OS/Application are fur-ther reduced by a specialized custom compiler (nesC)provided with the TinyOS framework. The nesC compilerexploits the component-based architecture to include onlycomponents required by the application’s wiring schemain the compiled program image. The nesC compilercan further deduce and remove any unused componentfunctions within the application configuration [20]. ATinyOS program image therefore contains no programcode unrelated to the target application.

The necessary memory space for TinyOS providing theabstract application functionality is shown in Table II. Theprogrammable flash memory, represents the amount ofspace to store the application code. The RAM field repre-sents the amount of statically compiled memory/variablesrequired by the application at runtime. Both applicationswill require more RAM to assign memory space for localvariables which are dynamically allocated at run-time2.

C. MANTIS

MANTIS OS provides more core functionality. The op-erating system core must provide functionality to handlemultiple threads. Furthermore, preemption and contextswitching must be implemented within the operatingsystem core. This includes semaphores, timer structuresand memory management. Semaphores must be imple-mented to ensure that critical sections of code cannot beused by multiple threads at the same time. A memorymanager must be set up to dynamically assign S-RAMaddress space during thread initialization. Three differenthardware timers are required to implement time-slicing,general purpose timers and a sleep timer. As such thecomplexity of the MANTIS kernel requires considerableoverhead in code size, memory requirements and proces-sor complexity. An application is implemented as a set ofMANTIS threads. Kernel and application are compiledinto one binary file executed on the CPU. To be able tocompare the memory footprint of MANTIS with TinyOS,the abstract application scenario is used for evaluation.

The MANTIS OS framework does not provide anytools to optimize memory size at compile time. Thus, it isleft to the programmer to decide which operating systemelements should be included in the binary. In practice, it isvery difficult to determine which elements are necessaryand a tool - comparable to nesC in TinyOS - would bevery useful.

The necessary memory space for MANTIS to providethe abstract application functionality is shown in Table II.

2By dynamic memory, we refer to memory assigned by a stack, notthe more common term of heap memory allocation.

D. Discussion and Findings

The initial build of each operating system highlightsthe huge memory savings gained by using TinyOS incombination with the nesC compiler. By compiling onlythe necessary functionality the nesC compiler can buildextremely optimized binary images. A direct memorycomparison between the operating systems is not fair if nooptimization is performed for MANTIS. Thus, the MAN-TIS operating system was optimized manually for theexperiment by removing all non-essential functionality.The results are shown in Table II. With this optimiza-tion, the MANTIS operating system takes nearly a thirdextra programmable memory space the TinyOS operatingsystem. Both operating systems statically allocate almostequal amounts of RAM, however both applications willrequire more memory to cater for the stack (local/nonstatic memory). Furthermore the MANTIS scheduler dy-namically allocates a memory pool to store the stack andprocessor registers for each thread. If there is insufficientmemory to store both the threads and stack, then duringthe course of execution, memory will be corrupted andthe application will fail.

The experimental results show that an optimizationmethod, comparable to the nesC features in TinyOS,would be desirable for the MANTIS operating system.If a standard Atmel 128 processor is used to realize sucha system, the applications realized with either operatingsystem can be accommodated easily in the memory3. Ifa more constrained platform is selected, such as the PIC16 micro processor [21], a TinyOS implementation mightbe the only option to realize such a system.

VI. EVENT PROCESSING

Many monitoring tasks in sensor networks require aresponsive network reaction as sensing data has to bereported in a timely fashion. If the operating systemof a sensor node is not capable of quickly respondingto events, such time-critical applications are difficult tobuild. Furthermore, it is desirable to have sensor nodesthat react to events in a deterministic and constant way.Sensor nodes with a predictable and constant performancecan be used as building blocks for sensor network appli-cations that require more deterministic network behavior.

This section investigates the event processing capabili-ties of the TinyOS and MANTIS operating system whilesupporting the abstract application scenario (see SectionIV-A). The average processing time required to handle thepacket forwarding task in the abstract application scenariois investigated. Additionally, the standard deviation of thisprocessing time is investigated to determine the stabilityof processing times.

A. TinyOS

The simple TinyOS scheduler schedules tasks to runatomically with a FIFO queuing algorithm. Tasks will

3The Atmega 128 is capable of supporting 128kB of programmableflash memory, and 4096B of Static Random Access Memory(SRAM)[15].



execute atomically with respect to one another, and canonly be preempted by an asynchronous event (an eventspawned from a hardware interrupt). The scheduling al-gorithm is very simple, and it can therefore schedule taskswith a minimum processing overhead. By schedulingtasks to run atomically, TinyOS precludes any potentialdeadlock errors or any potential inter-task race conditions,greatly simplifying the task of programming the appli-cation. However, atomic/non-preemptive task schedulescan result in non-ideal situations. A long TinyOS task(e.g. a sensing task) is occupying the CPU. Periodicinterrupts occur that signal the arrival of packets. Thepackets are read from the radio in the interrupt routine(interrupting the long TinyOS task) and a TinyOS taskfor the packet processing is created to be processed afterthe long TinyOS task. Packet processing is deferred ina non-predictable way and deadlines regarding packetprocessing can be missed. Thus, if an application requiresmore precise control of the task execution schedule, theexisting TinyOS scheduler is not suitable.

The task-blocking problem can be tackled with a num-ber of different solutions. The functionality of the high-priority tasks can always be programmed into an interrupthandler. However code executed in an interrupt handlerwill block all other activity and should therefore be usedsparingly. A more viable solution would be to segmentthe long sensing task into a series of sub-tasks, such thathigher priority TinyOS tasks will not be blocked for thefull duration of the long task execution time. However, asdescribed in Section IV-A the segmentation of processingfunctions is often not possible or simply not done by theprogrammer.

B. MANTIS

In the multi-threaded MANTIS operating system, allprocesses are defined as individual MANTIS threadswhich can be preempted at any time during the courseof execution. MANTIS thread preemption provides asignificant scheduling advantage, as all higher prioritytasks can be executed on demand. Thus, it is easierto ensure that deadlines of high priority tasks are met.However, MANTIS thread preemption facilitates extraprocessing cost, as the MANTIS scheduler must swapactive/idle threads in addition to managing a memorypool to store the state of inactive threads. These extracosts add to the overall processing time needed to realizethe task functionality. In the MANTIS operating systemthe processor will consume approximately 1200 clockcycles from the time the interrupt occurs until the time thetasks starts executing. The significance of this overheaddepends on the length of each process and the number ofcontext switches.

A long sensing task, implemented as low priorityMANTIS thread can be interrupted for packet process-ing. Packets arrive at the radio triggering interrupts. Aninterrupt causes a high priority MANTIS thread to beactivated for packet forwarding. As context switch time

and packet forwarding have deterministic time bounds,deadlines regarding packet processing can be met.

C. Measuring Event Processing Capabilities

To ascertain the operating system responsiveness, ex-periments with the network model and experiment setupshown in Section IV-A are carried out. Two tasks, asensing task and a packet processing task are executedconcurrently on the same processor. In this model, the(lengthily) sensing task and the packet-processing taskcompete for CPU resources on the sensor node. It isassumed that the packet-processing task within the nodeshas priority so that deadlines regarding packet forwardingcan be met. Thus, in the MANTIS implementation, thepacket processing task has a higher priority than the sens-ing task. In the TinyOS implementation, no prioritizationis implemented as this feature is not provided by theoperating system.

Task Execution Time:To characterize processing per-formance of the operating system, the average task execu-tion time Et of the packet forwarding task, is measured.During the experiment,J number of packet processingtimesej are recorded. To do so, the task start timeestart

and the task completion timeestop are measured and thepacket processing time is recorded ase = estop − estart.In case TinyOS is used,estop is the time when the packetprocessing task is completed and is removed from the taskqueue. When MANTIS is used,estop is the time when thepacket processing thread finishes and is sent into the waitstate. In both cases, the start timeestart is recorded in theinterrupt routine when the packet processing is initiated.The average task execution timeEt is calculated at theend of the experiment as:

Et =

∑

ej

J(2)

In order to investigate how deterministic the packetprocessing time is, the standard deviation of the averageexecution time is also calculated.

D. Evaluation

In the experiment, the average task execution timeEt

is determined for TinyOS and MANTIS supporting theabstract application scenario. The average task executiontime Et is shown in Fig. 5.

Where MANTIS is used, it can be observed that theaverage packet processing time is independent of thesensing task execution time. Furthermore,Et is alsovery independent from the positionn of the node inthe tree. Only under heavy load, the average processingtime slightly increases. This is due to the fact that underheavy load packet forwarding tasks have to be queued(see Fig. 5 a)).

Where TinyOS is used, the average processing timefor the packet forwarding taskEt depends on the lengthof the sensingls of the sensing task. In addition, underheavy load the queuing effects of the packet forwarding



0

2

4

6

8

10

12

1 2 3 4 5 6 7 8

Avera

ge P

rocessin

g T

ime E

t (m

s)

Tree Position [n]

MANTIS Et, ls=1 msMANTIS Et, ls=5 ms

MANTIS Et, ls=10 msMANTIS Et, ls=25 msMANTIS Et, ls=50 msMANTIS Et, ls=75 ms

MANTIS Et, ls=100 ms

0

2

4

6

8

10

12

1 2 3 4 5 6 7 8

Avera

ge P

rocessin

g T

ime E

t (m

s)

Tree Position [n]

TinyOS Et, ls=1 msTinyOS Et, ls=5 ms

TinyOS Et, ls=10 msTinyOS Et, ls=25 msTinyOS Et, ls=50 msTinyOS Et, ls=75 ms

TinyOS Et, ls=100 ms

a) MANTIS b) TinyOS

Fig. 5. Average packet processing timeEt , scheduled with a sensing task of execution timels.

tasks also contribute somewhat to the average processingtime (see Fig. 5 b)).

The thread prioritization capability of MANTIS isclearly visible in the experiment results. Packet processingtimes are independent of the concurrently executed andlower priority sensing task. In TinyOS, sensing and packetforwarding task delays are coupled, and the influence ofthe sensing activity on the packet forwarding activity isclearly visible.

In the case of small sensing tasks withls ≤ 5ms,TinyOS outperforms MANTIS. MANTIS has to performa context switch (either from the idle task or a runningsensing task) and this overhead adds to the averageexecution timeEt.

Fig. 6 shows the standard deviation of the packetforwarding task execution timeEt for a sensing task oflengthls = 75ms. In the case where MANTIS is used, thestandard deviation in the task processing time is very low.The standard deviation is noticed only for a high systemload. This is caused by packets queuing and waiting tobe processed. If TinyOS is used, a huge variation inthe processing times is observed. This variation increasesconsiderably with the size of the sensing task. This is dueto the fact that a packet processing task in TinyOS hasto wait for a sensing task to finish. However, for smallsensing tasks, as shown in Fig. 6, the standard deviationof Et is very small.

E. Discussion and Findings

It is evident that MANTIS provides better stabilityand predictability of packet processing time than TinyOS.Additionally, MANTIS decouples the packet processingtime from the sensing time delay.

However, if small sensing tasks are used, TinyOS isable to provide a similar stability in the packet processingtimes. In fact, in the case of small processing times,TinyOS is capable of processing incoming traffic fasteras there is no operating system overhead in the form ofcontext switches.

If a sensor network with a controlled and predictablenetwork performance has to be implemented, a deter-ministic packet forwarding behavior of sensor nodes is

required. For such an application, the MANTIS operatingsystem would generally be more suitable, especially ifprocessing intensive sensing tasks have to be supportedconcurrently. However, in the particular case of smallsensing tasks, TinyOS is the better choice. The samestability in packet processing time can be achieved whilepacket processing requires less time. Thus, a higherthroughput can be achieved.

Our findings regarding event processing of event-basedand multi-threaded sensor network operating systems canbe summarized as follows:

• A multi-threaded system is preferred if long (sens-ing) tasks have to be supported concurrently with thepacket forwarding tasks.

• An event-based system is preferred if short (sensing)tasks have to be supported concurrently with thepacket forwarding tasks.

While these conclusions are not surprising, they neverthe-less provide quantitative results on which to support thechoice of operating system.

VII. E NERGY CONSUMPTION

The lifetime of a sensor network is related to theenergy consumption of the sensor nodes. Therefore, thetask of reducing a sensor node’s power consumption is ofparamount importance. Operating systems for sensor net-works achieve low power consumption rates by exploitingprocessor idle times. Available idle time can be used toput the CPU in an energy-efficient power saving mode.Depending on the processor used, different power savingmodes might be available. The different power savingmodes vary in the time and energy necessary to enterand leave the mode and the power spent in the particularmode. Thus, to determine the energy efficiency of sensornetwork operating systems, it is necessary to investigatethe available idle times during system operation.

This Section investigates the available idle time inTinyOS and MANTIS while supporting the abstract appli-cation scenario (see Section IV-A). First, the ratio betweenidle and active time is determined as this number dictateshow much energy can possibly be saved. Second, the



1

1.5

2

2.5

3

3.5

4

1 2 3 4 5 6 7 8

Avera

ge P

rocessin

g T

ime E

t (m

s)

Tree Position [n]

MANTIS Et, ls=1 ms

1

1.5

2

2.5

3

3.5

4

1 2 3 4 5 6 7 8

Ave

rag

e P

roce

ssin

g T

ime

Et (m

s)

Tree Position [n]

TinyOS Et, ls=1 ms

0

2

4

6

8

10

12

14

1 2 3 4 5 6 7 8

Avera

ge P

rocessin

g T

ime E

t (m

s)

Tree Position [n]

MANTIS Et, ls=75 ms

0

2

4

6

8

10

12

14

1 2 3 4 5 6 7 8

Ave

rag

e P

roce

ssin

g T

ime

Et (m

s)

Tree Position [n]

TinyOS Et, ls=75 ms

Fig. 6. Standard deviation of packet processing timeEt, scheduled with a sensing task of execution timels = 1ms and ls = 75ms.

distribution of idle time periods is analyzed as the lengthand variation of available idle periods determines whichpower saving mode can be used by the processor.

A. TinyOS

TinyOS executes all pending tasks sequentially (SeeSection III). A processing task is only interrupted byhardware interrupts which in turn process interrupt ser-vice routines. Thus, the TinyOS operating system spendsalmost all processing activity in the execution of theapplication functionality. Little processing effort is spenton operating system related functionality.

As soon as the task queue is emptied and no eventshave to be processed, the TOSH_sleep() routine is called.In this routine it can be decided how the CPU shouldspend the available idle time. More specifically, it canbe decided which power saving mode will be used. Hereit is theoretically possible to use an algorithm to predicthow long the idle period will be and to select the powersaving mode accordingly. Such an algorithm might takeapplication layer knowledge into consideration. However,the current implementation of TinyOS only considers thesimplest available power saving mode and optimizationoptions are not exploited.

B. MANTIS

The drawback of a multi-threaded operating system,is that a considerable part of the CPU processing time isneeded for the organization of the system itself. A contextswitch between different threads requires the operatingsystem to save the context information. This operatingsystem overhead can not be spent for power efficientsleep times. The question is how much overhead has tobe allocated for the operating system itself.

If no MANTIS threads are scheduled for executionand no events have to be processed, the system ex-ecutes a so-called idle-task. The function of the idle-task is to determine which power saving mode shouldbe activated. For this decision, the idle-task uses threadstate information. If all threads are waiting for a serviceto respond, i.e the radio to return an acknowledgmentmessage, then it is in a THREAD_STATE_IDLE and theidle-task will activate a CPU idle mode as soon as allthreads finish processing. If however, all threads are ina THREAD_STATE_SLEEP then there are no threadswaiting on a service to complete, and the CPU will entera sleep mode once all threads finish processing. Thissimple policy is used as it is assumed that a thread in

THREAD_STATE_IDLE will become active very soonand no long idle period can be expected. The idle modecan be activated without a transition phase but is notvery energy-efficient. The sleep mode has a transitionphase before the actual power saving starts. This simplepolicy helps to optimize power consumption and dependsvery much on the type and specification of the CPUused. However, a better policy might be available if, forexample, application layer knowledge is used to predictidle times.

C. Measuring Power Efficiency

To measure the power efficiency of each operatingsystem, we use the abstract application scenario in SectionIV-A, to evaluate each operating system under varyingdegrees of duress. Both operating system applicationsmust process a sensing task and a number of packetforwarding tasks. The frequency of packet forwardingtasks increases progressively as the position of the sensornode reaches the root of the network topology, i.e. thenumber of child nodes increase.

In evaluating power efficiency, the aforementioned op-erating system specific power management policies areignored (In fact, only MANTIS currently implementssuch a policy). This study investigates the available idletimes for power management purposes, not the differentpower management policies available. An efficient powermanagement policy must be tailored to the particularCPU used and can take application layer information intoaccount. However, the results presented can be used todesign appropriate power management strategies.

Idle time: The first parameter measured is the per-centage idle time. The percentage idle time indicateshow power efficient the system can be. The longer theidle time the more energy-efficient the system. In theexperiment, the abstract application scenario is executedby the sensor node running TinyOS or MANTIS. Theduration of the experimentT and the durationik of kidle time periods during the experiment is recorded.i isdefined asi = istop − istart . In case of TinyOS,istart

is the point in time when to TOSH_sleep() is called. Incase of MANTIS,istart is defined as the point in timewhen the idle-task begins execution.istop is for TinyOSand MANTIS the point in time when the system resumesoperation by processing an interrupt. All idle periodsikare summarized and the percentage idle time,It, thepercent of experiment time, in which the processor is idle,which is calculated as follows:



It =

∑

ikT

· 100 (3)

The percentage idle time is compared with the the-oretical maximal percentage idle time,Imax

k . Imaxk is

calculated by taking only application processing of theabstract application scenario into account. Thus,Imax

k

represents the percent of running time the processor wouldbe idle for an ideal operating system which would have nooperating system processing overhead.Imax

k depends onthe task sizesls and lp of sensing and packet forwardingtask, the frequency of the sensing taskfs, the CPU speedscpu and the positionn of the node in the abstract appli-cation scenario.Imax

k is calculated using Equation (1):

Imaxk =

(

1 −fs

scpu

· (ls + lp · (2n− 1))

)

· 100 (4)

The operating system overheadIok is calculated using

Equation (3) and Equation (4):

Iok = Imax

k − It (5)

Idle Periods: The second parameter of interest is theaverage length of the idle periodsIp. This parametershows which sleep modes a CPU could use. Long contin-uous idle periods allow for deep sleep modes. Again, itis necessary to record the durationik of all K idle timeperiods during the experiment. The average length of theidle periodIp is calculated as:

Ip =

∑

ikk

(6)

The standard deviation of the idle period length is alsocalculated to determine the stability of the idle periodlengths.

D. Evaluation

In the first experiment, the percentage idle timeIt

is determined for TinyOS and MANTIS supporting theabstract application scenario. The idle timeIt is shownin Fig. 7.

The time spent in idle mode drops for both operatingsystems exponentially with the increasing node positionin the tree described by the parametern. This behavioris expected as the number of packet tasks increasesaccordingly (See Equation (1)). Less obvious is the factthat the available idle time drops faster in MANTIS thanin TinyOS. The fast drop in idle time is caused by thecontext switches in the MANTIS operating system. Themore packet forwarding tasks are created, the more likelyit is that a sensing task is currently running when a packetinterrupt occurs. Subsequently, a context switch to thehigher prioritized forwarding task is needed.

When MANTIS is used, the length of the sensing taskhas a significant impact on the idle time. If TinyOS isused, the length of the sensing task does not influenceidle time that strongly. Again, the difference is down tothe necessary context switches in MANTIS. The longer

the sensing task, the more likely it is that a sensing taskis running when a packet arrives.

As expected, MANTIS proves to be less energy-efficient than TinyOS. However, in the case of low systemactivity both systems have roughly the same energyefficiency. For example for a leaf node withn = 1 anda sensing task with the size ofls = 1ms TinyOS is only0.09% more energy-efficient than MANTIS. In the worstcase, for a node at positionn = 8 and a sensing taskwith the size ofls = 1ms TinyOS is 7.6% more energy-efficient.

If TinyOS is used, the length of the sensing task haslittle impact on the idle time measured. This is due to thelow operating system overhead introduced by TinyOS. Aspreviously mentioned, TinyOS spends nearly all availableCPU time for application processing, not for operatingsystem related tasks.

The theoretical maximum available idle timeImaxk and

the measured idle time for TinyOS and MANTIS with along sensing task ofls = 75ms is shown in Fig. 8 a).In Fig. 8 b), the resulting operating system overheadIo

k

is shown. As we can see, MANTIS has an increasingoverhead with the increasing activity of the system. TheTinyOS operating system overhead on the other hand isnot very dependent on the load of the system.

Fig. 8 shows the average length of the idle periodsIp

and the standard deviation ofIp for a short sensing taskwith the lengthls = 1ms. MANTIS has slightly shorteridle periods as it has a higher operating system overhead.Despite this difference, both operating systems achievevery similar sleeping periods. Also, the standard deviationin the idle period length is not very different. Thus, bothoperating systems can make use of available power modesin the same way. As shown, the thread-based MANTISoperating system does not fragment unnecessarily theavailable idle times. This effect however might be presentin other thread-based operating system where a periodiccontext switch to a kernel-thread is performed.

E. Discussion and Findings

It can be confirmed that the multi-threaded MANTISis not as power efficient as the event-based TinyOSoperating system. However, the difference between bothoperating systems under specific conditions is not verybig. If the system is not loaded (leaf node withn = 1and a sensing task with the size ofls = 1ms) a differenceof only 0.09% in idle time is measured. Still, even if thesystem is in a heavy load situation (leaf node withn = 8and a sensing task with the size ofls = 75ms) only a7.05% difference in the idle time is encountered. Thus, ifan application is implemented where sensors are inactivefor long periods and suddenly an event is detected thatleads to an activity increase in the sensor field MANTIS iscomparable to TinyOS in power efficiency. Additionally,as detailed in Section VI, MANTIS would be able tohandle such bursty activity in a more deterministic way.

The experiments also show that the idle-times are notmore fragmented in MANTIS than in TinyOS. Thus, the



40

50

60

70

80

90

100

1 2 3 4 5 6 7 8

Perc

enta

ge Idle

Tim

e I

t (%

of T

)

Tree Position [n]

MANTIS Ik, ls=1 msMANTIS Ik, ls=5 ms

MANTIS Ik, ls=10 msMANTIS Ik, ls=25 msMANTIS Ik, ls=50 msMANTIS Ik, ls=75 ms

MANTIS Ik, ls=100 ms 40

50

60

70

80

90

100

1 2 3 4 5 6 7 8

Perc

enta

ge Idle

Tim

e I

t (%

of T

)

Tree Position [n]

TinyOS Ik, ls=1 msTinyOS Ik, ls=5 ms

TinyOS Ik, ls=10 msTinyOS Ik, ls=25 msTinyOS Ik, ls=50 msTinyOS Ik, ls=75 ms

TinyOS Ik, ls=100 ms

a) MANTIS b) TinyOS

Fig. 7. Percentage idle timeIk for both operating systems.

50

60

70

80

90

100

1 2 3 4 5 6 7 8

Perc

enta

ge Idle

Tim

e I

t (%

of T

)

Tree Position [n]

MANTIS It, ls=75 ms

TinyOS Ik, ls=75 ms

Ikmax

ls=75 ms

0

10

20

30

1 2 3 4 5 6 7 8

opera

ting s

yste

m o

verh

ead I

ot (m

s)

Tree Position [n]

MANTIS Io

k, ls=75 ms

TinyOS Io

k, ls=75 ms

1

10

100

1000

1 2 3 4 5 6 7 8

Avera

ge Idle

Period (

ms)

Tree Position [n]

MANTIS ik, ls=1 ms

1

10

100

1000

1 2 3 4 5 6 7 8

Ave

rag

e I

dle

Pe

rio

d (

ms)

Tree Position [n]

TinyOS ik, ls=1 ms

a) Imaxk andIk b) Io

k c) MANTIS ik d) TinyOS ik

Fig. 8. a) Percentage idle timeIk and the theoretical maximum idle timeImax

k, b) Operating system overhead, for both TinyOS and MANTIS

with ls = 75ms , c+d) Variation of Idle periods,ik.

common argument that a multi-threaded operating systemleads to high fragmentation of sleep times is provenwrong in the case of the MANTIS operating system. Bothoperating system types are able to exploit the same energysaving modes provided by the hardware.

Our findings regarding power efficiency of event-basedand multi-threaded sensor network operating systems canbe summarized as follows:

• The processing overhead resulting from a more com-plex threaded scheduler reduces the amount of timethat a sensor node can sleep. The reduction dependsheavily on the system activity.

• A multi-threaded scheduler has little impact on howfragmented a sleep schedule is. All energy savingmodes provided by hardware can be accessed.

VIII. C ONCLUSION

Most sensor network application scenarios used todayare built using event-driven operating systems. The vastmajority of these deployments use the event driven (andwell established) operating system TinyOS. However,alternative operating system concepts such as the clas-sical thread-based system exist. A well known exampleof a multi-threaded operating system is MANTIS. Webelieve a study, as presented in this paper, is needed todecide objectively which operating system type shouldbe used for a particular application scenario. This de-cision is currently made on mainly subjective groundsfor event driven systems and TinyOS. We believe thispaper presents the first objective comparison betweenboth main operating system concepts, taking the important

performance parameters memory usage, event processingand energy usage into account. Based on the presentedstudy we conclude that an event driven operating systemis often, but certainly not always the best choice. Classicthread-based operating systems are of use for many sensornetwork application scenarios.

A. Results

Memory Usage:If memory efficiency is a primary goalof the target application then TinyOS would be the betteroperating system. In the experiment, the application sce-nario was compiled to a binary image 4kB less in TinyOSthan MANTIS. While these memory requirements willhave no impact on processors such as the Atmega 128, ifthe application is to be compiled for a more cost effectivebut less capable processor such as the PIC16 [22] thenonly TinyOS could be used. While both operating systemsrun a very light-weight scheduler, the TinyOS scheduleris smaller in code size as it does not provide a threadswitching capability.

Event Processing:The processing time of packet for-warding tasks in the abstract application scenario dependsmainly on the complexity of the sensing task, not onnetwork activity. This dependency can be summarized as:

• If the sensing task is small (e.g.ls = 1ms, n = 1),MANTIS has an17.15% longer average packet for-warding task execution time than TinyOS. MANTISand TinyOS have variation in the task execution timein the same order (Variation MANTIS0µs, VariationTinyOS 454µs).



• If the sensing task is large (e.g.ls = 75ms, n = 1)TinyOS has an204% longer average packet forward-ing task execution time than MANTIS. TinyOS has amuch higher variation in the task execution time thanTinyOS (Variation MANTIS34µs, Variation TinyOS58ms).

As a consequence, TinyOS is the preferred operatingsystem in the case where no long sensing task has to besupported (Or in case the sensing task can be divided insmall sub-tasks). In case long sensing tasks are executed,MANTIS would be the preferred system to achieve adeterministic and fast packet processing. Ultimately, theapplication requirements for the message forwarding per-formance of the network dictate which operating systemcan be used.

TinyOS has a high variation in the packet forwardingtimes as a pending packet task has to wait for a sensingtask to finish. Especially when long sensing tasks arepresent, this delay is visible in the variation of theforwarding times.

Energy Usage:The energy usage of nodes in the ab-stract application scenario depends mainly on the networkactivity, not on the complexity of the sensing task. Thisdependency can be summarized as:

• If there is low network activity (n = 1, lS =1ms), TinyOS has an0.09% longer idle time thanMANTIS. Thus, TinyOS is more power efficient.The fragment size of idle periods is of the sameorder for both operating systems (Idle Time Lengthfor TinyOS is499ms and Idle time for MANTIS is490ms).

• If there is low network activity (n = 8, ls = 1ms),TinyOS has an7% longer idle time than MANTIS.Thus, TinyOS is more power efficient. The fragmentsize of idle periods is in the same order for bothoperating systems (Idle Time Length for TinyOS is3.8ms and Idle Time for MANTIS is3.9ms).

As a consequence, TinyOS is the preferred in all operatingconditions as it is more power efficient. The MANTISoperating system is not far behind in terms of powerefficiency, especially if a low network activity is present.The more responsive MANTIS is not as power efficientas the necessary context switches reduce the available idletime. Especially when a large amount of network trafficis present, the context-switch overhead is prominent.

Summary: The difference in memory usage of thetwo operating systems is a (nearly) static parameter anddoes not depend on the particular application scenariosupported. In most cases, this small difference wouldnot be the deciding factor in choosing one of the op-erating systems. If a deterministic behavior regardingpacket forwarding times is required, AND a complicatedsensing task is carried out at the same time, MANTISwould be the better choice. This has to be paid withan additional energy consumption, but in some cases adeterministic network behavior would be preferred over alow energy consumption. If a TinyOS application requiresa long sensing task, the poor packet processing times

resulting from the non-preemptive scheduler, could leadto a congested network. This in turn would consumeprobably more energy retransmitting lost packets.

B. Application Scenarios

It was shown in Sections V, VI, VII that neither operat-ing system would be optimal for all application scenariosas both operating systems target different performanceobjectives. In TinyOS for example, a key design metric isfocused on compiling small application images, howeverthe light-weight multi-threaded kernel in MANTIS isdesigned to provide predictable performance for moreprocessing intensive applications.

To measure the operating system performance for arange of applications scenarios, the experiment evaluationused a range of sensing tasks sizes. These tasks timeswere chosen to model a spectrum of sensor networkapplications. Examples of such task times can be foundin literature, for example encryption algorithms such asTinySec can take up to1ms to encrypt a byte of data[17]. A structural monitoring application presented in [23]uses a wavelet decomposition algorithm that takes12msto compress vibration data readings, before transmittingdata over the network. It can be concluded from ourexperiment that TinyOS would provide a more efficientscheduler for processing these applications. Such smalltask times achieve a better average response time and alsomore predictable response time in TinyOS due to the lowprocessing overhead of the scheduler.

Image processing algorithms used in target trackingapplications would be represented by the larger sensingtasks in the application scenario. Image processing al-gorithms are very processing intensive and difficult torealize on a sensor network. In [16] an object detectionalgorithm is used that can take up to240ms on a 128x128px image,60.8ms on a 64 x 46 image and16ms ona 32 x 32 pixel image. Scheduling a processing taskof this magnitude in a non-preemptive scheduler suchas in TinyOS can provide very ineffective schedules.We see from Section VI-D that high priority tasks suchas a packet forwarding task can take a further5ms toprocess when scheduled with a75ms Sensing task. Anobvious solution is to try and segment the sensing task,so that the high-priority task does not get delayed bythe full duration of the sensing task. However in [16],the authors maintain that “Image processing operationsare typically long-running and not suitable for sequen-tial decomposition”. In that paper, the authors realizedtheir application on the TinyOS operating system with adedicated ASP (application specific processor). The ASPprocessed all image algorithms to alleviate the CPU ofall long sensing tasks and facilitate responsive processingof high-priority tasks. Adding an extra processor for allimage processing sensor networks, considerably increasesthe cost of the sensor node platform. It might thereforehave been more feasible to implement such algorithmswith a multi-threaded operating system such as MANTIS.



C. Further Study

Both operating systems leave room for further study.Our work described in [24] shows how preemptive

scheduling capabilities can be added to TinyOS in order totackle the performance problems described in Section VI.The established event driven processing concepts canbe retained while adding preemption through contextswitching. Established TinyOS programming conventionscan be used which ensures that existing application codecan be re-used. Preemption features can be integratedseamlessly in existing TinyOS infrastructures.

Our work described in [25] shows how overheadsdue to context switching can be significantly reducedwithin the MANTIS operating system. Thus, the problemsdescribed Section VII can be diminished. The overheadcan be reduced to such an extent that in usual sensornetwork application scenarios the modified MANTIS hasa similar overall performance to TinyOS.

REFERENCES

[1] J. Hill, R. Szewczyk, A. Woo, S. Hollar, D. Culler, and K. Pister,“System architecture directions for networked sensors,” in ACMSIGOPS Operating Systems Review, vol. 34, pp. 93–104, Decem-ber 2000.

[2] H. Abrach, S. Bhatti, J. Carlson, H. Dai, J. Rose, A. Sheth,B. Shucker, and R. Han, “MANTIS: System support for mul-timodal networks of in-situ sensors,” in2nd ACM InternationalWorkshop on Wireless Sensor Networks and Applications, pp. 50–59, September 2003.

[3] A. Barroso, J. Benson, T. Murphy, U. Roedig, C. Sreenan, J. Bar-ton, S. Bellis, B. O’Flynn, and K. Delaney, “Demo abstract:The DSYS25 sensor platform,” in2nd international conferenceon Embedded networked sensor systems, pp. 314–314, November2004.

[4] C. Duffy, U. Roedig, J. Herbert, and C. J. Sreenan, “An Exper-imental Comparison of Event Driven and Multi-Threaded Sen-sor Node Operating Systems,” in3rd IEEE International Work-shop on Sensor Networks and Systems for Pervasive Computing(PERSENS2007), White Plains, USA, Mar. 2007.

[5] S. Bhatti, J. Carlson, H. Dai, J. Deng, J. Rose, A. Sheth,B. Shucker, C. Gruenwald, A. Torgenson, and R. Han., “MANTISOS: An embedded multithreaded operating system for wirelessmicro sensor platforms,”ACM kluwer Mobile Networks & Applica-tions Journal, special Issue on Wireless Sensor Networks, August2005.

[6] C. Han, R. Kumar, R. Shea, E. Kohler, and M. Srivastava, “Adynamic operating system for sensor nodes,” in3rd Interna-tional Conference on Mobile Systems, Applications, and Services,pp. 117–124, June 2005.

[7] A. Dunkels, B. Gronvall, and T. Voigt, “Contiki - a lightweightand flexible operating system for tiny networked sensors,” in29th Annual IEEE International Conference on Local ComputerNetworks, pp. 455–462, November 2004.

[8] A. Eswaran, A. Rowe, and R. Rajkumar, “nano-RK: An energy-aware resource-centric operating system for sensor networks,” in26th IEEE Real-Time Systems Symposium, pp. 265–265, Decem-ber 2005.

[9] S. Dulman, T. Hofmeijer, and P. Havinga., “AmbientRT - real time,data centric system software for wireless sensor networks,” in 1st

international Conference on Intelligent Sensors, Sensor Networksand Information Processing, pp. 61–66, December 2004.

[10] A. Dunkels, O. Schmidt, and T. Voigt, “Using protothreads forsensor node programming,” inWorkshop on Real-World WirelessSensor Networks, June 2005.

[11] E. Trumpler and R. Han., “A systematic framework for evolvingTinyOS,” in IEEE Workshop on Embedded Networked Sensors,pp. 61–65, May 2006.

[12] J. Regehr, A. Reid, K. Webb, M. Parker, and J. Lepreau, “Evolvingreal-time systems using hierarchical scheduling and concurrencyanalysis,” in 24th IEEE Internation Real-Time Systems Sympo-sium, pp. 25–36, December 2003.

[13] L. Barello, “Avrx real time kernel.” http://barello.net/avrx/, 2006.[14] S. Li, R. Sutton, and J. Rabaey, “Low power operating systems

for heterogeneous wireless communication systems,” inWorkshopon Compilers and Operating Systems for Low Power, pp. 1–16,September 2001.

[15] Atmel Corporation,Atmega128 Datasheet, rev n ed., March 2006.[16] M. Rahimi, R. Baer, O. I. Iroezi, J. C. Garcia, J. Warrior, D. Estrin,

and M. Srivastava., “Cyclops: In situ image sensing and interpre-tation in wireless sensor networks,” inIn proc. 3rd internationalconference on Embedded Networked Sensor Systems,, pp. 192–204, November 2005.

[17] C. Karlof, N. Sastry, and D. Wagner, “TinySec: A link layersecurity architecture for wireless sensor networks,” in2nd ACMConference on Embedded Networked Sensor Systems, pp. 262–175,November 2004.

[18] Nordic Semiconductor,Datasheet NRF2401, rev 1.1 ed., June2004.

[19] “The gnu project, http://www.gnu.org/,” 2006.[20] D. Gay, P. Levis, R. von Behren, M. Welsh, E. Brewer, and

D. Culler, “The nesC language: A holistic approach to networkedembedded systems,” inACM SIGPLAN Conference on Program-ming Language Design and Implementation, pp. 1–11, June 2003.

[21] Microchip,PIC16F877 Datasheet. Microchip, f ed., October 2003.[22] C. Lynch and F. O’Reilly, “PIC-based TinyOS implementation,” in

2nd European Workshop on Wireless Sensor Networks, pp. 378–385, 31st Jan.-2nd Feb. 2005.

[23] N. Xu, S. Rangwala, K. Chintalapudi, D. Ganesan, A. Broad,R. Govindan, and D. Estrin, “A wireless sensor network for struc-tural monitoring,” in ACM Conference on Embedded NetworkedSensor Systems, pp. 13–24, November 2004.

[24] C. Duffy, U. Roedig, J. Herbert, and C. J. Sreenan, “Adding Pre-emption to TinyOS,” inProceedings of the The Fourth Workshopon Embedded Networked Sensors (EmNets2007), Cork, Ireland,June 2007.

[25] C. Duffy, U. Roedig, J. Herbert, and C. J. Sreenan, “Improvingthe energy efficiency of the MANTIS kernel,” inProceedings ofthe 4th IEEE European Workshop on Wireless Sensor Networks(EWSN2007), Delft, Netherlands, Jan. 2007.

Cormac Duffy received his B.Sc degree in Computer Science fromthe Dublin Institute of Technology, Ireland and M.Sc DegreefromUniversity College Cork. He had completed his Ph.D. at UniversityCollege Cork on the topic of Lightweight Scheduling for WirelessSensor Networks at the time of his sudden tragic death in August 2007.

Utz Roedig is a Lecturer in the Department of Communication Systemsand the Department of Computing at Lancaster University. Hepursuesresearch in the area of wireless sensor networks. A specific aspect of thisresearch is the data transport performance of wireless sensor networks.Utz received his Ph.D. in Computer Science at Darmstadt University ofTechnology in 2002.

John Herbert received a B.Sc. (Physics, Mathematics) and M.Sc.(Physics) from University College Cork, Ireland and a Ph.D.(ComputerScience) from Cambridge University, England. He has held positionsas Research Associate at Cambridge University, International Fellowat DEC Systems Research Centre, California and Computer Scientistat SRI International, Cambridge. At present he is a Senior Lecturer inComputer Science at University College Cork.

Cormac Sreenanjoined University College Cork as a Professor in theDepartment of Computer Science in August 1999, and was Head ofDepartment from 2000-2004. Prior to that he was a Principal TechnicalStaff Member with AT&T Labs Research (in Florham Park, NJ, USA),and also worked at Bell Labs Research (in Murray Hill, NJ, USA). Hereceived his Ph.D. at University of Cambridge in 1992.



A Comprehensive Experimental Comparison of Event Driven ...

Documents

Transcript of A Comprehensive Experimental Comparison of Event Driven ...