Bachelor Informatica

Benchmarking the real-time Linuxextension Xenomai in an em-bedded environment

Rik van der Kooij6064965

August 12, 2012

Supervisor(s): drs. A. van Inge and drs. T.R. Walstra

Signed:

Informatica—

Universiteit

vanAmst

erdam

2

Abstract

Real-time extensions are available for Linux in the form of patches. When using these real-timeextensions, knowledge of the performance is required to know if it will meet your requirements.It is more important to understand the behavior of the system and understand how the systemhandles different situations. This paper will be a report on performance measurements of theLinux real-time extension Xenomai in an embedded environment.

This is similar to research done the University of Aveiro where performance of multiplereal-time operating systems was measured. Another similar study was done by Rep InvariantSystems, Inc. This study looked at the interrupt handling time of Linux and Linux with real-timeextensions.

The benchmarks show that the maximum response time to external interrupts of Xenomai isat least twice as fast a standard Linux kernel. Xenomai also handles multiple incoming interruptsmore predictable.

2

Contents

1 Introduction 5

2 Buildroot and real-time benchmarks 72.1 Creating real-time Linux kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Building embedded Linux systems with Buildroot . . . . . . . . . . . . . . . . . . 82.3 Benchmarking real-time Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Characteristics of the platforms 113.1 Characteristics of the NanosG20 . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1.1 Hardware specific . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.1.2 Kernel modules for NanosG20 . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2 XEM6001 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.2.1 FrontPanel configuration protocol . . . . . . . . . . . . . . . . . . . . . . 123.2.2 Implemented configuration protocol . . . . . . . . . . . . . . . . . . . . . 12

3.3 Boards setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4 Benchmarks results 154.1 Measurement of the response time . . . . . . . . . . . . . . . . . . . . . . . . . . 154.2 Multiple Interrupts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.3 Communication benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5 Conclusion 215.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

A NanosG20 kernel module makefile 25

B OpenOK communication code for XEM6001 27B.1 OpenOK.h . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27B.2 OpenOK.c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

C Response time benchmark code 37C.1 Normal and Debian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37C.2 Xenomai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

D Communication benchmark code 39D.1 Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39D.2 Xenomai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40D.3 XEM6001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3

4

CHAPTER 1

Introduction

Embedded systems can be found in all kinds of places like automobiles, refrigerators and planes.The design of an embedded system depends on the needs of the system. For example an embed-ded system in an automobile or refrigerator, both measuring temperature, could use differenthardware due to the difference in measurement frequency.

A subset of embedded systems are real-time embedded systems. In real-time systems taskscan have deadlines. The goal of a real-time system is to let every real-time task meet its deadline.Deadlines come in different flavours: hard, firm and soft [7]. Deadlines are considered hard whenfailing to meet one of the deadlines can have catastrophic results. An example is a pacemakerwhere failure means death of a patient. Missing soft deadlines will only decrease the performanceof the systems. Example of this is a music player. If audio is not played at the correct time itmay sound less pleasant, but the audio player is able to continue playing. Firm deadlines area combination of hard and soft deadlines. A firm real-time system will not fail on one misseddeadline, but is able to fail on a couple of missed deadlines.

Linux as general-purpose operating system is not suitable for real-time tasks. The Linuxscheduler is optimized for average performance and tries to give every process a fair share ofcomputing time [8]. For real-time tasks precise timing and predictable performance is moreimportant than average performance. Another problem is the non-preemptive Linux kernel.A kernel is the center of an operating system. It is the layer between the hardware and thesoftware 1. A non-preemptive kernel is able to hold the systems up for an unknown amount oftime. Normally this will not be unending long, but real-time system itself could include kernelmodules which will be able to shut down the system, from other more important activity, for alonger amount of time.

There are two basic approaches to modifying the standard Linux kernel into a real-timekernel 2. The first uses a microkernel which runs the unchanged Linux kernel as a task. Themicrokernel takes control over the system for real-time processes and is responsible for schedulingreal-time tasks, interrupt handling and scheduling Linux. With these changes it is possible tolet a real-time task run on the CPU without the Linux kernel being able to interrupt the task.As the Linux kernel is run as a task it is preemptive at any time. With the second approach,changes are made the Linux kernel in order to be real-time. This include changes as real-timescheduling and adding preemption to the kernel.

In 1996 a real-time Linux called RTLinux was developed by a group at the New MexicoInstitute of Mining and Technology in Soccorro. Early in the development it was clear thatchanging the kernel itself would be an enormous amount of work [8]. They therefore used themicrokernel approach which added some code, but needed only small changes to the existingLinux kernel code. Changes had to be made to the Linux cli() routine. This routine wouldinstead of disabling interrupts set a flag. Using this flag, the microkernel can upon catching anon-real-time interrupt decide whether to pass it to the Linux Kernel or wait for a reset of the

1Linux simple guide. http://www.worldusers.net/index.php/Documents/linux-simple-guide.html2Simon. Introduction to Linux for real-time control. Technical report, National Institute of Standards and

Technology, 2002

5

http://www.worldusers.net/index.php/Documents/linux-simple-guide.html

flag. The real-time tasks were non-preemptive and FIFO scheduled. Since then a lot of researchhas been done in preemptive real-time task scheduling with different priorities and it is in fact anNP-complete problem 3. In order to reduce paging delays, RTLinux will never page out real-timetasks.

Around the same time members of the Department of Aerospace Engineering, Politecnico diMilano envisioned a real-time hardware abstraction layer (HAL) onto which a real-time appli-cations interface (RTAI) could be mounted [6]. The Linux kernel was at that time not matureenough for this concept. It was only after the team had seen the design of RTLinux and changescame to the kernel with newer versions that RTAI was made. RTAI ended up very similar toRTLinux. The main difference is that RTLinux makes most of its changes to the kernel sourcefiles itself while RTAI limits the changes in kernel code by adding a hardware abstraction layer.The HAL was implemented with only 70 lines of code which makes it easy to maintain with newkernel versions. Since RTAI used the same HAL technique as used by RTLinux it ran into patentdiscussions. This resulted in abandoning HAL and the use of ADEOS. Adaptive Domains En-vironment for Operating Systems is in fact still a hardware abstraction layer between hardwareand the running operating system. The difference is that ADEOS is able to run several kernelstogether.

Xenomai is another real-time extension for Linux which uses ADEOS. RTAI and Xenomaihave been fused together between 2001 and 2005. This has made them very similar. ‘The majordifferences derive from the goals the projects aim for, and from their respective implementation.While RTAI is focused on lowest technically feasible latencies, Xenomai also considers cleanextensibility (RTOS skins), portability, and maintainability as very important goals.’ 4.

In later versions of the Linux kernel changes have been made to make the kernel more real-time. The changes include an O(1) scheduler, being able to disable swapping, fixed priority forprocesses and most important a more preemptive kernel. The kernel has been made preemptiveby protecting critical sections with spin locks instead of disabling interrupts. As of Linux kernelversion 2.6 with the these changes it can be stated that the kernel is a soft real-time operatingsystem [1].

Real-time systems are often used as control system. Many real-time tasks in control systemsare periodic. An example is a periodic sensor reading which the system responds to. Otherreal-time tasks can be non periodic and start whenever an event occurs. In this paper we willbe benchmarking the Linux real-time extension Xenomai within an embedded system. Theembedded system will consist of an FPGA (Opal Kelly XEM6001) and a Linux minicomputer(NanosG20). The XEM6001 will be used as a digital signal processor. Sensor data which needfurther analysis are sent to the NanosG20. As modern FPGA include multiple processors sensorreadings can be done completely parallel. The communication between the boards will on theother hand be done sequential because the NanosG20 has a single processor core. The mainquestion of this paper will be answering is:

What do the benchmarks say about the performance of Xenomai?

In chapter 2 we will look at ways to build real-time Linux kernels for the NanosG20 anduseful benchmarks for real-time systems.Chapter 3 will be used to look at the hardware characteristics of the NanosG20 for real-timesystems and we create an user space C program to configure the XEM6001.In chapter 4 we will be performing benchmarks and discuss the results.In chapter 5 we discuss the results and possible future work.

3N. Audsley and A. Burns. Real-time System Scheduling. University of York, 19904Difference between Xenomai and RTAI by the Xenomai wiki page http://www.xenomai.org/index.php/FAQs

6

http://www.xenomai.org/index.php/FAQs

CHAPTER 2

Buildroot and real-time benchmarks

2.1 Creating real-time Linux kernels

Creating a real-time Linux kernel is similar to creating a standard Linux kernel. Creating astandard Linux kernel starts with downloading kernel source from for instance the Linux KernelArchives1 and unpacking kernel sources with the Linux/Unix ‘tar -xvjr /path/to/linuxsources’command. After that the kernel can be configured with the Makefile. The command ‘makemenuconfig’ starts a user-interface where kernel options can be selected. Configuration includesselecting the hardware type, device drivers, file system support and more. Compiling the kerneland kernel modules with ‘make’ is what comes next. We end up with a kernel image and loadablekernel modules.

A soft real-time Linux kernel is created by simply setting the CONFIG PREEMPT configu-ration flag. The process of creating a hard real-time Linux kernel is for each real-time extensiondifferent, but it usually consists of patching the kernel and configuring real-time kernel options.Creating a hard real-time kernel with Xenomai is more than just patching. Xenomai uses ashell script to change the kernel code and includes user-space components. The shell scriptis run with the command ‘scripts/prepare-kernel.sh --linux=/path/to/linux/sources[--adeos=/path/to/adeos/patch] [--arch=]. It should be used right afterunpacking the source code. Xenomai includes user space components these components shouldalso be compiled using ‘make’.

For the NanosG20 custom Linux source code is available from the Ledato, the companywhich created the board, website 2. These kernel sources include default configuration for theNanosG20. We can use these settings by running the ‘make ARCH=arm nanosg20 defconfig’command before menuconfig.

A kernel created as described above is compiled for the hardware of the machine compilingthe kernel. The kernel compilation for embedded boards is usually done on another machine,a host. A cross compiler will be needed in order to compile the kernel for the hardware of theNanosG20. The Linux and Xenomai support the use of cross compilers. This is done by addingextra arguments when using make. For the ARM NanosG20 board this changes the make andshells script commands explained above into: ‘make ARCH=arm menuconfig’, ‘make ARCH=armCROSS COMPILE=arm-angstrom-linux-gnueabi-’ and ‘scripts/prepare-kernel.sh--linux=/path/to/linux/sources [--adeos=/path/to/adeos/patch] [--arch=ARM]’. TheXenomai user space components have a shell script for cross compilation configuration. For theNanosG20 the configuration command is ./configureCC=/usr/local/angstrom/arm/bin/arm-angstrom-linux-gnueabi-gcc --host=arm-linux-gnueabi

--enable-arm-eabi --enable-arm-mach=at91sam9 --disable-arm-tsc. The default instal-lation directory for the Xenomai user components is /usr/xenomai. The directory can be changedby adding DESDIR to the make command: ‘make DESTDIR=/new/destination/dir’.

1The Linux Kernel Archives, http://www.kernel.org2Custom Linux kernel (version 2.6.35.7) source code can be downloaded from:http://www.ledato.de/

7

http://www.kernel.orghttp://www.ledato.de/

2.2 Building embedded Linux systems with Buildroot

We have seen how we can create a kernel for the NanosG20 board, but there are still openquestions. Where do we get a cross compiler? How do we create the Linux file system structure?Does every user space application and library need custom compilation? We use Buildroot toget these things done. ‘Buildroot is a set of Makefiles and patches that makes it easy to generatea complete embedded Linux system. Buildroot can generate any or all of a cross-compilationtoolchain, a root file system, a kernel image and a bootloader image.’ 3.

Buildroot can be configured by using the command ‘make menuconfig’. Configuration ofBuildroot covers hardware, build process and software-related options4. Buildroot will downloadall necessary files, compiles a kernel with the cross compiler it creates, creates a file systemstructure and compile selected user space packages just by running ‘make’.

Figure 2.1: Configuration interface for Buildroot after using ‘make menuconfig’

We use Buildroot to create Linux systems for the NanosG20. Using Buildroot we create anon real-time Linux system. We will go through the Buildroot configuration for the NanosG20.As can be seen in figure 2.1, we use target architecture arm, variant arm926t and target ABI(EABI). Linux 2.6.35.x kernel headers are used for the toolchain. At the system configuration wechange the ‘to run a getty (login prompt) on’ option from ttyS0 to ttyS1. At package selection weselect the network application OpenSSH and text editor Vim. We change the kernel options touse our custom Linux source code, for Linux kernel configuration we select the nanosg20 defconfigfile and choose the zImage as kernel binary format. We leave build options, host utilities, filesystem images and bootloaders untouched.

The root file system Buildroot created are stored on a SD card for the NanosG20. Weadd a boot directory to the Linux file system and copy the zImage to this directory. We alsocopy two boot files from the default Debian file system5 to this directory, /boot/boot.sh and/boot/cmdline.

We could repeat all these steps to create a real-time Xenomai kernel. To save time we use thefile system we just created. We change the kernel image to a Xenomai kernel and store the user

3Buildroot website, http://buildroot.uclibc.org/4Roll Your Own Embedded Linux System with Buildroot, Linux Journals http://www.linuxjournal.com/

article/107955The NanosG20 comes default with a Debian version. This can be downloaded from http://www.ledato.

deaswell

8

http://buildroot.uclibc.org/http://www.linuxjournal.com/article/10795http://www.linuxjournal.com/article/10795http://www.ledato.de as wellhttp://www.ledato.de as well

components on the new Xenomai system. The Xenomai kernel image was created as describedat section ‘Creating real-time Linux kernels’. We now have one real-time Linux system and onenon real-time Linux system. In future references to Xenomai stands for the real-time system andnormal stands for the non real-time system.

2.3 Benchmarking real-time Linux

There are three categories of tests for a real-time kernel [5]: functionality, throughput and la-tency. When a real-time Linux kernel is expected to have the same behavior for all the APIs as astandard kernel, this can be tested. There are multiple standard functionality tests available forLinux systems. Throughput benchmarks can also be used for real-time systems. The throughputof a real-time kernel can be lower due to the changes in the kernel where predictability had ahigher priority over performance. For instance real-time Linux using a microkernel will intro-duce extra latency for non-real-time interrupts because they will now have to go through themicrokernel as well. The latency benchmarks is the most important for real-time systems sincemost real-time tasks are responding to an event.

We want to know the rate at which sensor data can be exchanged between the NanosG20 andXEM6001. The protocol we will create in our last benchmark uses external interrupts to let theNanosG20 know new data can be read. Before we create this protocol we can ask the question:What is the response time to external interrupts for the NanosG20 kernels? The first benchmarkwill be answering this question by measuring the response time. Our second benchmark willtest the behavior of the kernels when multiple interrupts arrive at the same time. In our lastbenchmark we will try to communicate between the two boards and measure the rate of datatransfer.

More detailed information about the benchmarks will be given in chapter 4.

9

10

CHAPTER 3

Characteristics of the platforms

3.1 Characteristics of the NanosG20

3.1.1 Hardware specific

We will look briefly at the characteristics of the NanosG20 that are important for a real-timesystem.

The board has 32KB of data cache, 32KB of instruction cache and 128MB of SDRAM. TheLinux kernel version used for this board (2.6.35.7) has a size of 2.0MB. This means that itcannot fit inside the cache, but it fits inside the SDRAM. The same size is observed with customcompiled (real-time) kernels. The Debian file system, which comes standard with the board, hasa size of almost 150MB. It can thereby not fit inside the SDRAM, but a custom compiled Linuxhas a smaller file system and uses only 85MB. The result is a system which will experience ahigh amount of cache misses even when running in kernel space and will only have to go to theflash memory once.

As stated before interrupts are an important part of an embedded real-time system. Theboard has an Advanced Interrupt Controller (AIC) which is able to handle 32 interrupts. Theinterrupts can either be internal or external. All the boards PIO pins can also be used forinterrupts. It is important to note that the GPIO pins on the NanosG20 can only be configuredto trigger on both the rising and falling edges.

3.1.2 Kernel modules for NanosG20

The benchmarks in the next chapter will consist of kernel modules for the NanosG20. ‘Modulesare pieces of code that can be loaded and unloaded into the kernel upon demand. They extendthe functionality of the kernel without the need to reboot the system.’ 1. As kernel modulesare run in kernel space they do not use the main() function and user space libraries are notavailable. This makes module programming different from user space programs. The Makefileused to create kernel modules for the benchmarks is included in appendix A. The Makefilespecifies the compiler for the system to be the arm-linux- compilers Buildroot created for us.Compiling a kernel module gives us a KO file. These can be inserted in the kernel using insmodmodule name.ko and removed with rmmod module name.

3.2 XEM6001 Configuration

Opal Kelly, the company which created the XEM6001, provides a program called FrontPanelfor interaction with the XEM6001. FrontPanel is able to run on Windows and Apple platforms.There is no support for Linux platforms. In our environment the FPGA will have to be configured

1The Linux Kernel Module Programming Guide 2.6, a great book on programming kernel modules for Linuxversion 2.6. http://tldp.org/LDP/lkmpg/2.6/html/lkmpg.html

11

http://tldp.org/LDP/lkmpg/2.6/html/lkmpg.html

from a Linux environment, the NanosG20 board. We will have to reverse engineer the protocolused by FrontPanel in order to create a configuration program that can be used on the NanosG20.

We are not the first with this problem. On the online source code repository SourceForgeJennifer Holt placed a C program named OpenOK to configure an OpalKelly FPGA in a Linuxenvironment 2. The code was made for the OpelKelly FPGAs XEM3005 and XEM3010. OpenOKdoes not work for our XEM6001. We will reuse the OpenOK code to create a configurationprogram for the XEM6001.

3.2.1 FrontPanel configuration protocol

Using a program to log USB messages, we can see what type of packages and data is sent to theFPGA during configuration. We will use this to reproduce the protocol.

Analyzing the output we can see that FrontPanel starts with getting information from thedevice by sending standard USB GET DESCRIPTOR and GET STATUS messages. FrontPanelcontinues by reading information every 110 milliseconds with control transfer messages. TheFPGA responds with the byte 0x02. We do not know what it means, but since it is always sendingthis package before the configuration, a good guess would be that the FPGA is telling FrontPanelthat it is ready to be configured. As we start configuration, FrontPanel reads information fromthe device by using a control transfer packet. FrontPanel then sends a control transfer package.The information it reads and answer it sends remain the same when using different configurationfiles. FrontPanel continues by sending the configuration file using bulk or interrupt transfer.FrontPanel uses a last control transfer to check whether or not the configuration worked.

3.2.2 Implemented configuration protocol

We did not have to follow the full protocol we have seen FrontPanel use for our configurationprogram. The final code, which can be seen in appendix B, uses almost all the OpenOK code.The code does not use the setup messages before configuration that OpalKelly used. When wetry to create those setup messages we run into a problem with the answer message. USB controltransfer messages use a setup packet. The second byte of the setup packet is used to let thedevice know what the request is. The USB log did not show the values of this byte. Trying everynumber, 0 to 255, show us that all the numbers from 150 to 250 seem to arrive correctly. Sincewe do not know which request to use and the configuration works without these messages we leftit out of our protocol.

The configuration is split into three functions. The function get OK devices creates a list ofall connected OpalKelly devices. This is done by looping through all the connected devices andchecking the Vendor ID of the device. We select the OpalKelly device we want to use from the listand ‘open’ the device using the OK open dev function. This is done using two Libusb functionsand sending a control transfer message. The last function OK configure FPGA’ configures theFPGA by sending the configuration file using bulk transfer.

3.3 Boards setup

The setup of the boards can be seen in figure 3.2. We use the serial port (2) and the pro-gram minicom to log in to the Linux system. The USB connection between the FPGA (4) andNanosG20 (5) is used to configure the FPGA with a configuration file. Our Linux kernels arestored on different SD cards and is used by the boards micro SD card slot (7). We use the inter-net connection (8) for easy file exchange between the NanosG20 and a desktop computer usingOpenSSH. The benchmarks will use GPIO pins (9) for external interrupts on the NanosG20.The wired connection between the two boards (3) will be used to exchange sensor data.

We created a communication protocol which the boards will use. The protocol will use fourof the five wires since one of them is used as electrical ground. The four wires are used as a bitof data. Both boards can read or set the values of these ‘bits’. Each board uses two lines foroutput to the other board: one for data and one to let the other board know data is ready. For

2Link to the OpenOK repository: http://sourceforge.net/projects/openok/

12

http://sourceforge.net/projects/openok/

Figure 3.1: The setup of the embedded environment with NanosG20 (above) and XEM6001(under).

the NanosG20 the ‘is ready line’ is handled as interrupt. The XEM6001 will poll this line fromthe NanosG20 every cycle to check whether data from the NanosG20 is ready.

Using the protocol the boards should send bits of data back and forth. A timing diagram ofthe protocol can be seen in figure 3.1. Time 0 is the start situation with all the lines at zero. Attime is 1 the NanosG20 wants to send a bit of data. It sets the data bit and the ‘is ready’ highfor a short time. At time 3 the XEM6001 reads the data and starts sending data back: data

13

lines goes to 1, interrupt line is set high and low right after. At time 5 the NanosG20 handlesthe incoming interrupt and begins with sending a bit to the XEM6001. This should be able togo on until the desired amount of bits have been send. We will see in our benchmark that theboards will not be able to handle the protocol with our implementation.

Figure 3.2: Timing diagram for communication protocol between NanosG20 and XEM6001.W: write edge line from NanosG20. Dn: data line from NanosG20. I: interrupt line from

XEM6001. Dx: data line from XEM6001.

14

CHAPTER 4

Benchmarks results

4.1 Measurement of the response time

This benchmark is a measurement of the response time on periodic external interrupts of theNanosG20. The setup for the benchmark includes a pulse generator and an oscilloscope. Thesetup can be seen in figure 4.1. The pulse generator will be used to generate periodic externalinterrupts on a GPIO pin of the NanosG20. The NanosG20 interrupt handler will toggle anoutput GPIO pin likewise. The output of the pulse generator and the output GPIO pin of theNanosG20 are given to an oscilloscope. From the display of the oscilloscope we can read theresponse time.

Figure 4.1: Setup of response time benchmark

Kernel modules will be used to configure the GPIO pins as interrupts and output. We createda kernel module for the normal and Debian kernel and a kernel module for the Xenomai kernel.The kernel modules can be seen in appendix C. The difference is the use of the real-time drivermodel for the Xenomai kernel.

We measure the minimal response time and the total jitter. The total jitter is the timedifference between the minimal response time and the maximal response time.

We will benchmark the upper limit of the response time by benchmarking multiple timeswith different amount of workload tasks running on the system. Each benchmarks will run for

15

0 100 200 300 400 500Number of workload processes (#)

0

2

4

6

8

10

12

14

Min

imal re

sponse

tim

e(µ

s)

Minimal response time on benchmark

debiannormalxenomai

Figure 4.2: Average response time of resonse time benchmark.Time in microsecond (10−6).

Workload is ‘dd if=/dev/zero of=/dev/null’

0 100 200 300 400 500Number of workload processes (#)

0

10

20

30

40

50

60

70

80

Tota

l jit

ter

(µs)

Total jitter on benchmark

debiannormalxenomai

Figure 4.3: Jitter of response time benchmark.Time in microsecond (10−6).

Workload is ‘dd if=/dev/zero of=/dev/null’

10 minutes. Figures 4.2 and 4.3 show the results.

The two graphs clearly show that the Xenomai kernel has the best performance. Not onlyis the response time and the jitter significantly lower, the jitter itself is more consistent. Thismeans a more predictable response time, which is what one would like to see in a real-time system.

16

While the average response time of the Debian and the normal kernel are almost identical, thetotal jitter of the Debian makes it the kernel with the worst performance.

4.2 Multiple Interrupts

Some real-time systems must be capable of handling multiple events. In our last benchmark weonly had one interrupt at a time. The next benchmark will consist of testing the behaviour ofthe kernels on multiple interrupts.

The setup of the benchmark is shown in figure 4.4. The setup is similar to the last benchmark,but now we use the pulse generator to trigger two interrupts on GPIO pins. We use the samekernel modules from the last benchmark, but know use two of them to handle both interrupts.We create a copy of the kernel modules and change the GPIO pins for our second module. Weinstert the two kernel modules and are able to use the two output pins to look at the handlingof the interrupts.

The benchmarks were preformed using a differend osciloscope. The new osciloscope had moreoptions and was able to calculate the mean, minimum, maximum and mean of the response times.The benchmark used 15000 samples to create reliable values to one decimal.

Figure 4.4: Setup for multiple interrupts benchmark

Table 4.1: Normal kernel multiple interrupt benchmark.Workload 0 and 200. Time in microseconds (10−6s)

0 workload mean minimum maximum standard deviationpin A 5.4 2.8 13.2 .203pin B 6.7 4.1 15.6 .209

200 workload mean minimum maximum standard deviationpin A 3.5 .002 44.8 2.5pin B 4.9 .002 44.4 2.8

17

Table 4.2: Xenomai kernel multiple interrupt benchmark.Workload 0 and 200. Time in microseconds (10−6s)

0 workload mean minimum maximum standard deviationpin A 3.3 3.2 8.9 .156pin B 7.7 7.4 14.7 .292

200 workload mean minimum maximum standard deviationpin A 3.3 3.2 29.5 3.1pin B 9.0 7.6 41.5 4.4

Looking at the results in the tables 4.1 and 4.2 we can see that the kernels handle interruptsdifferently. The Xenomai kernel handles one interrupt at a time. This can be seen at workload0 where the standard deviation of .292s is almost the double of .156s. The normal kernel favorsinterrupt pin A over B, but seems to handle interrupts in another way since the maximum valueand standard deviation are less apart. It could be that some of the interrupts on pin B preemptrunning interrupts of pin A. This is possible since we had the configuration for a preemptivekernel. The mean and minimum response time of both pins seem to decrease with a workload of200 processes for the normal kernel. This is not consistent with previous benchmark where theresponse time climbed above 13s. It is not clear why the normal kernel has this behavior.

4.3 Communication benchmark

We use a kernel module for the NanosG20 to implement the communication protocol describedin chapter 3. The XEM6001 uses a configuration file created with the hardware descriptionlanguage Verilog. The code can be seen in appendix D. Our first implementation failed andfigure 4.5 is a timing diagram showing what goes wrong.

Figure 4.5: Timing diagram for communication protocol between NanosG20 and XEM6001.W: write edge line from NanosG20. Dn: data line from NanosG20. I: interrupt line from

XEM6001. Dx: data line from XEM6001.

The problem arises when the XEM6001 starts sending a bit to the NanosG20. After thetwo lines are set the board checks the ‘is ready’ line again from the NanosG20. The NanosG20has not set this line low yet and the XEM6001 sends another bit at time 3. This results in anoverload of multiple interrupts send to the NanosG20.

In our second implementation we let the XEM6001 only send new data when it has seen thewrite edge go down. This way the NanosG20 can take as much time as needed to set the data

18

line low without the XEM6001 sending more interrupts. We still had a problem in the XEM6001code. The interrupt line is set high and low the cycle afterwards. Since the GPIO pins on theNanosG20 can only be triggered on both the rising and falling edges this sends two interrupts tothe NanosG20.

We tested the second implementation on the normal and Xenomai kernel. Due to the highnumber of interrupts the kernels had to process they would not be able to do anything else. Wehad a counter keep track of the number of handled interrupts. Using a stopwatch we checkedhow many interrupts were handled after 30 seconds. For the normal kernel this was somewherein between 10 and 50. The Xenomai kernel handled on the other hand 10514194 interrupts. TheXenomai kernel was able to handle this interrupts because it only preempts real-time tasks withreal-time task having higher priority. It was handling the interrupts in FIFO order. It was doingthis at the amazing speed of only 3µs per interrupt. Interrupts on the normal kernel interruptswere able to preempt each other since they were implemented as kernel threads.

19

20

CHAPTER 5

Conclusion

In this paper, we have seen Xenomai performing better to our benchmarks. Xenomai was ableto respond twice as fast to external interrupts than the standard Linux kernel. Xenomai handledmultiple interrupts with the same priority in a first come first serve order. This lowered themaximum response time of the interrupt handled first and gave a more predictable responsetime. Even when too many interrupts were given to handle, Xenomai was able to handle a lotof them.

The way Xenomai handles interrupts is more predictable. This is the exact result we wouldlike to see for a real-time system.

5.1 Future work

Future research can be done by creating a working communication protocol between the twoboards. A new protocol could make use of the measured response times of the NanosG20. It isthen possible to see how the system reacts to sensor data being send sequential. Questions ashow long sensor data will need buffering inside the FPGA can be answered.

It is also possible to go another way and benchmark other Linux real-time extensions. Real-time extensions as RTAI and real-time linux patches are examples.

21

22

Bibliography

[1] A. Barbalace, A. Luchetta, G. Manduchi, M. Moro, A. Soppelsa, and C. Taliercio. Perfor-mance comparison of vxworks, linux, rtai, and xenomai in a hard real-time application. IEEETransactions on Nuclear Science, 55(1):435–439, Feb. 2008.

[2] Dr. Jeremy H. Brown and Brad Martin. How fast is fast enough? choosing between Xenomaiand Linux for real-time applications. Rep Invariant Systems, 2010.

[3] Cecilia Ekelin and Jan Jonsson. Real-time system constraints: Where do they come from andwhere do they go? In Proc. of the Intl Workshop on Real-Time Constraints, pages 53–57,1999.

[4] Tomas Kalibera, Pavel Parizek, Ghaith Haddad, Gary T. Leavens, and Jan Vitek. Challengebenchmarks for verification of real-time programs. SIGPLAN Notices, 44(11):7–8, 2009.

[5] Sripathi Kodi and Chirag H. Jog. Testing real-time linux. what to test and how. presentedat the Linux Kongress 2008, Germany, 2008.

[6] P. Mantegazza, E. Bianchi, L. Dozio, S. Papacharalambous, S. Hughes, and D. Beal. Rtai:Real-time application interface. Linux Journal, 72, April 2000.

[7] Kang G. Shin and Parameswaran Ramanathan. Real-time computing: A new discipline ofcomputer science and engineering. PROCEEDINGS- IEEE, 82:6, 1994.

[8] Victor Yodaiken and Michael Barabanov. A real-time linux. Linux Journal, 34, 1997.

23

24

APPENDIX A

NanosG20 kernel module makefile

1 ARCH=arm2 CROSS COMPILE=/path/ to / b u i l d r o o t / output / host / usr / bin /arm−l inux−3 KDIR := /path/ to / l inux −2.6.35.7− nanosg204 PWD := $ ( s h e l l pwd)56 obj−m += module name . o78 a l l :9 $ (MAKE) −C $ (KDIR) SUBDIRS=$ (PWD) modules ARCH=$ (ARCH)/\

10 CROSS COMPILE=$ (CROSS COMPILE)1112 c l ean :13 $ (MAKE) −C $ (KDIR) SUBDIRS=$ (PWD) c l ean

25

26

APPENDIX B

OpenOK communication code forXEM6001

B.1 OpenOK.h

1 /∗2 ∗ openok . h3 ∗4 ∗ Created on : Oct 15 , 20095 ∗ Author : Jenn i f e r Hol t6 ∗ Open Source Opal Ke l l y i n t e r f a c e l i b r a r y78 Copyright ( c ) 2009 Jenn i f e r Hol t9

10 Permission i s hereby granted , f r e e o f charge , to any person ob t a in ing a copy11 o f t h i s so f tware and a s s o c i a t e d documentation f i l e s ( the ” Sof tware ”) , to dea l12 in the Sof tware wi thout r e s t r i c t i o n , i n c l u d i n g wi thout l im i t a t i o n the r i g h t s13 to use , copy , modify , merge , pub l i s h , d i s t r i b u t e , s u b l i c en s e , and/or s e l l14 cop i e s o f the Software , and to permit persons to whom the Sof tware i s15 f u rn i s hed to do so , s u b j e c t to the f o l l ow i n g cond i t i on s :1617 The above copy r i g h t no t i c e and t h i s permiss ion no t i c e s h a l l be inc luded in18 a l l cop i e s or s u b s t a n t i a l po r t i on s o f the Sof tware .1920 THE SOFTWARE IS PROVIDED ”AS IS ” , WITHOUT WARRANTY OF ANY KIND, EXPRESS OR21 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,22 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE23 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER24 LIABILITY , WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,25 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN26 THE SOFTWARE.2728 ∗/2930 #ifndef OPENOK H31 #define OPENOK H323334 #endif /∗ OPENOK H ∗/35

27

36 #ifndef USB H37 #include 38 #define USB H39 #endif /∗ usb . h ∗/40 /∗ ∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗ d e f i n e s ∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗ ∗/41 #define USB DIR HOST 0x804243 /∗ ∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗ s t r u c t u r e s ∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗ ∗/44 struct OKdevice{45 struct usb dev i c e ∗OK dev ; // usb dev i c e i d e n t i f i e r , use wi th usb open (OK dev)46 char OK idVendor [ 2 5 5 ] ; // usb vendor id47 char OK idProduct [ 2 5 5 ] ; // usb product id48 char OK ser ia l [ 2 5 5 ] ; // dev i c e s e r i a l number49 char OK man [ 2 5 5 ] ; //manufacturer s t r i n g50 char OK prod [ 2 5 5 ] ; // product s t r i n g51 } ;5253 /∗ ∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗∗ f unc t i on s ∗∗∗∗∗∗∗∗∗∗∗∗∗ ∗/5455 int get OK devices ( struct OKdevice d e v i c e l i s t [ ] ) ;56 usb dev handle ∗ OK open ser ia l (char s e r i a l [ ] ) ;57 usb dev handle ∗ OK open dev ( struct usb dev i c e ∗OK dev ) ;58 int OK close ( usb dev handle ∗ dev i c e ) ;59 int OK configure FPGA ( usb dev handle ∗ OK dev , const char∗ Filename ) ;60 int OK update wirein ( usb dev handle ∗ OK dev , char∗ data ) ;61 int OK get wireout ( usb dev handle ∗ OK dev , char∗ data ) ;62 int O K s e t t r i g g e r i n ( usb dev handle ∗ OK dev , int ep , char∗ data ) ;63 int OK getTrigger out ( usb dev handle ∗ OK dev , char∗ data ) ;

B.2 OpenOK.c

1 /∗2 ∗ openok . c3 ∗4 ∗ Created on : Oct 15 , 20095 ∗ Author : Jenn i f e r Hol t6 ∗ Open Opal Ke l l y i n t e r f a c e l i b r a r y78 Copyright ( c ) 2009 Jenn i f e r Hol t9

10 ∗ ve r s i on 0 . 3 : august 1 , 201211 ∗ Author : Rik van der Kooij , Taco Walstra12 ∗ Small changes to suppor t XEM6000 s e r i e s1314 Permission i s hereby granted , f r e e o f charge , to any person ob t a in ing a copy15 o f t h i s so f tware and a s s o c i a t e d documentation f i l e s ( the ” Sof tware ”) , to dea l16 in the Sof tware wi thout r e s t r i c t i o n , i n c l u d i n g wi thout l im i t a t i o n the r i g h t s17 to use , copy , modify , merge , pub l i s h , d i s t r i b u t e , s u b l i c en s e , and/or s e l l18 cop i e s o f the Software , and to permit persons to whom the Sof tware i s19 f u rn i s hed to do so , s u b j e c t to the f o l l ow i n g cond i t i on s :2021 The above copy r i g h t no t i c e and t h i s permiss ion no t i c e s h a l l be inc luded in22 a l l cop i e s or s u b s t a n t i a l po r t i on s o f the Sof tware .2324 THE SOFTWARE IS PROVIDED ”AS IS ” , WITHOUT WARRANTY OF ANY KIND, EXPRESS OR25 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

28

26 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE27 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER28 LIABILITY , WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,29 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN30 THE SOFTWARE.31 ∗/3233 #include 34 #include 35 #include ”openok . h”36 #include ”usb . h”3738 /∗39 ∗ USE DEFINES DURING MAKE TO COMPILE FOR THE CORRECT BOARD TYPE! !40 ∗ ∗/414243 int get OK devices ( struct OKdevice d e v i c e l i s t [ ] ) {44 /∗ c a l l u s b i n i t ( ) b e f o r e c a l l i n g t h i s f unc t i on ∗/45 struct usb bus ∗bus ;46 struct usb dev i c e ∗dev ;47 int i ;48 i =0; //used to keep t rack o f d e v i c e s4950 u s b f i n d b u s s e s ( ) ;51 u s b f i n d d e v i c e s ( ) ;5253 for ( bus = usb busse s ; bus ; bus = bus−>next ) {5455 for ( dev = bus−>d e v i c e s ; dev ; dev = dev−>next ) {5657 usb dev handle ∗udev ;58 int r e t ;59 char s t r i n g [ 2 5 6 ] ;6061 i f ( dev−>d e s c r i p t o r . idVendor==0x151F ){ // check i f d ev i c e i s made by62 // opa l k e l l y63 d e v i c e l i s t [ i ] . OK dev = dev ; // save dev i c e64 s p r i n t f ( d e v i c e l i s t [ i ] . OK idVendor , ”%04X” ,65 dev−>d e s c r i p t o r . idVendor ) ; // save vendor id6667 s p r i n t f ( d e v i c e l i s t [ i ] . OK idProduct , ”%04X” ,68 dev−>d e s c r i p t o r . idProduct ) ; // save product id69 udev = usb open ( dev ) ; //open dev i c e70 i f ( udev ) {71 i f ( dev−>d e s c r i p t o r . iManufacturer ) {72 r e t = u s b g e t s t r i n g s i m p l e ( udev ,73 dev−>d e s c r i p t o r . iManufacturer ,74 s t r i ng , s izeof ( s t r i n g ) ) ;75 i f ( r e t > 0)76 s p r i n t f ( d e v i c e l i s t [ i ] . OK man, ”%s ” , s t r i n g ) ;77 else78 s p r i n t f ( d e v i c e l i s t [ i ] . OK man,79 ”Unable to f e t c h manufacturer s t r i n g ” ) ;80 }81

29

82 i f ( dev−>d e s c r i p t o r . iProduct ) {83 r e t = u s b g e t s t r i n g s i m p l e ( udev ,84 dev−>d e s c r i p t o r . iProduct ,85 s t r i ng , s izeof ( s t r i n g ) ) ;86 i f ( r e t > 0)87 s p r i n t f ( d e v i c e l i s t [ i ] . OK prod , ”%s ” , s t r i n g ) ;88 else89 s p r i n t f ( d e v i c e l i s t [ i ] . OK prod ,90 ”Unable to f e t c h product s t r i n g ” ) ;91 }9293 i f ( dev−>d e s c r i p t o r . iSer ia lNumber ) {94 r e t = u s b g e t s t r i n g s i m p l e ( udev ,95 dev−>d e s c r i p t o r . iSerialNumber ,96 s t r i ng , s izeof ( s t r i n g ) ) ;97 i f ( r e t > 0)98 s p r i n t f ( d e v i c e l i s t [ i ] . OK ser ia l , ”%s ” , s t r i n g ) ;99 else

100 s p r i n t f ( d e v i c e l i s t [ i ] . OK ser ia l ,101 ”Unable to f e t c h s e r i a l number s t r i n g ” ) ;102 }103104 u s b c l o s e ( udev ) ;105 i ++; // increment dev i c e index106 }107 }108 }109 }110 return i ;111 }112113 usb dev handle ∗ OK open ser ia l (char s e r i a l [ ] ) {114 usb dev handle ∗udev ;115116 return udev ;117 }118119 usb dev handle ∗ OK open dev ( struct usb dev i c e ∗OK dev) {120 usb dev handle ∗ udev ;121 char c [ 1 ] ;122 int i ;123124 udev=usb open (OK dev ) ; //open dev i c e125 i f ( udev ) {126 //need to send se tup packe t on endpoint 0x00127 // packe t i s 0x00 09 01 00 00 00 00 00128 // l i b u s b prov ide s a convenient f unc t i on u s b s e t c o n f i g u r a t i o n f o r t h i s129 i = u s b s e t c o n f i g u r a t i o n ( udev , 1 ) ;130 i f ( i == 0) {131 i = u s b c l a i m i n t e r f a c e ( udev , 0 ) ;132 i f ( i == 0){ // claim i n t e r f a c e 0133 //need to send USB CONTROL packet , 0 xc0 : b9 00 00 00 00 01 00134 // t h i s i s cop ied from a capture o f t r a f f i c when us ing the135 // o f f i c i a l i n t e r f a c e l i b r a r y136 //don ’ t know what t h i s r e que s t i s , but i t r e tu rns a by t e o f 0x80137 //when c a l l e d by r e a l l i b r a r y

30

138 // r e q u e s t t y p e=0xc0 = USB DIR HOST & USB TYPE VENDOR139 i = usb contro l msg ( udev , 0xc0 , 0xb9 , 0 , 0 , c , 1 , 5 0 0 ) ;140 i f ( i == 1){ // shou ld have read 1 by t e141 i f ( c [ 0 ] != (char )0 x80 ) { // dev i c e did not re turn 0x00142 p r i n t f ( ” Device d i t not re turn 0x00” ) ;143 u s b c l o s e ( udev ) ; // c l o s e dev i c e144 udev = NULL; // re turn NULL145 }146 }147 else { // usb con t ro l mes sage f a i l e d148 p r i n t f ( ” Control message f a i l e d \n” ) ;149 u s b c l o s e ( udev ) ; // c l o s e dev i c e150 udev = NULL; // re turn NULL151 }152153 }154 else {155 p r i n t f ( ” claming i n t e r f a c e f a i l %d\n” , i ) ;156 u s b c l o s e ( udev ) ; // claim didn ’ t work157 udev = NULL; // re turn NULL158 }159 }160 else { // problem with s e t t i n g con f i gura t i on , c l o s e dev i c e and re turn NULL161 u s b c l o s e ( udev ) ;162 udev = NULL;163 }164 }165 return udev ;166 }167168 int OK close ( usb dev handle ∗ OK dev) {169 return u s b c l o s e (OK dev ) ;170 }171172 #ifde f XEM3001173 /∗Use t h i s f unc t i on f o r a l l the Opal Ke l l y o l d e r boards (non 6000 s e r i e s ) ∗/174 int OK configure FPGA ( usb dev handle ∗ OK dev , const char∗ Filename ) {175 // FPGA con f i g u r a t i on i s sen t as a bu l k t r an s f e r , wi th a con t r o l t r a n s f e r176 // preced ing to t e l l t he dev i c e to expec t c on f i g u r a t i on data . the con t r o l177 // t r an s f e r has no data by tes ,178 // and has se tup packe t 0x40 b2 00 00 00 00 00 00179 // the bu l k t r an s f e r i s the b i t s t r eam from a Xi l i nx . b i t f i l e , s t a r t i n g wi th180 // the sync word FF FF FF FF AA 99 55 66 however the b i t s t r eam i s sen t wi th181 // each by t e padded to 16 b i t s on the r i gh t , so 0xFF becomes 0xFF00 in the182 // data a c t u a l l y sen t over USB. in the r e a l l i b r a r y , a con f i gu r e c a l l i s183 // ended wi th another con t r o l t r ans f e r , t h i s appears to be a check to see i f184 // con f i g u r a t i on was s u c c e s s f u l the t r an s f e r has 1 data by t e and the se tup185 // packe t i s 0xc0 b2 00 00 00 00 01 00. a by t e o f 0x01 i s re turned on186 // success , and I have seen a by t e o f 0x02 re turned when i t didn ’ t work , but187 // I don ’ t know what the 0x02 means , or i f t h e r e are o ther re turn va l u e s188 // i n d i c a t i n g o ther e r ro r s189190 char c [ 9 ] ;191 char∗ b u f f e r ;192 int i , s i z e ;193 long pos , p o s i t i o n ;

31

194 FILE ∗ pFi l e ;195 // con f i gu r e s the fpga on an xem with the b i t s t r eam in Filename196197 // f i r s t g e t and parse the b i t s t r eam from the f i l e198199 pF i l e = fopen ( Filename , ” r ” ) ;200 i f ( pF i l e ) {201 // ge t l e n g t h o f f i l e202 pos = f t e l l ( pF i l e ) ;203 f s e e k ( pFi le , 0 , SEEK END) ;204 s i z e = f t e l l ( pF i l e ) ;205 f s e e k ( pFi le , pos , SEEK SET ) ;206207 // a l l o c a t e memory b u f f e r f o r b i t s t r eam ( s i z e o f f i l e ∗2 bytes , s ince the208 // b i t s t r eam i s sen t padded to 16 b i t s )209 b u f f e r = ( char∗) mal loc ( s i z e ∗2 ) ;210 p o s i t i o n =0; // put b u f f e r p o s i t i o n at 0211 i f ( b u f f e r ) {212 //need to throw out e v e r y t h in g u n t i l we reach213 // the FF FF FF FF AA 99 55 66 sync word214 // f i r s t read in 7 by t e s215 for ( i =1; i

250 // by t e in the upper b i t s , i e 0 x f f −> 0 x f f 0 0 . t h i s was taken251 // care o f in the l oad ing and pars ing o f the f i l e252 i=usb bu lk wr i t e (OK dev , 0x02 , bu f f e r , po s i t i on −2, 2000 ) ;253 i f ( i==pos i t i on −2) {254 //do another con t r o l t r a n s f e r to check s t a t u s255 i=usb contro l msg (OK dev , 0xc0 , 0xb2 , 0 , 0 , c , 1 , 7 0 0 ) ;256 i f ( i ==1) {257 i =0;258 i f ( c [ 0 ] ! = 0 x01 ){ // check f o r s u c c e s s f u l s t a t u s259 i =0;260 } else {261 p r i n t f ( ” Status not c o r r e c t ( returned : %x )\n” ,262 c [ 0 ] ) ;263 }264 }// end i f c on t r o l check265 }// end i f bu l k wr i t e succe s s266 }// end i f c on t r o l se tup267 }// end i f sync word not found268 f r e e ( b u f f e r ) ; // d e a l l o c a t e memory f o r b i t s t r eam bu f f e r269 }// end i f a l l o c a t e b u f f e r f o r b i t s t r eam270 }// end i f f i l e open271 return 0 ;272 }273 #endif274275 #ifde f XEM6001276 int OK configure FPGA ( usb dev handle ∗ OK dev , const char∗ Filename ) {277 char c [ 1 7 ] ;278 char∗ b u f f e r ;279 int i , s i z e ;280 long pos , p o s i t i o n ;281 FILE ∗ pFi l e ;282 // con f i gu r e s the fpga on an xem with the b i t s t r eam in Filename283284 // f i r s t g e t and parse the b i t s t r eam from the f i l e285 pF i l e = fopen ( Filename , ” r ” ) ;286 i f ( pF i l e )287 {288 // ge t l e n g t h o f f i l e289 pos = f t e l l ( pF i l e ) ;290 f s e e k ( pFi le , 0 , SEEK END) ;291 s i z e = f t e l l ( pF i l e ) ;292 f s e e k ( pFi le , pos , SEEK SET ) ;293294 // a l l o c a t e memory b u f f e r f o r b i t s t r eam . The b i t s t r eam i s as s i n g l e b y t e s295 b u f f e r = ( char∗) mal loc ( s i z e ) ;296 p o s i t i o n = 0 ; // put b u f f e r p o s i t i o n at 0297298 i f ( b u f f e r )299 {300 // f i r s t read in 16 by t e s301 for ( i = 1 ; i < 16 ; i++)302 c [ i ] = f g e t c ( pF i l e ) ;303 c [ 1 6 ] = 0x00 ; // terminate s t r i n g304 do {305 for ( i = 0 ; i < 15 ; i++)

33

306 c [ i ] = c [ i +1] ; // s h i f t chars in b u f f e r307 c [ 1 5 ] = f g e t c ( pF i l e ) ; // read in new char308 i f ( ! strcmp ( c ,309 ”\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF\0” ) )310 break ;311 } while ( ! f e o f ( pF i l e ) ) ;312313 i f ( ! f e o f ( pF i l e ) ){ // i f we reached the end o f the f i l e , the sync314 // word wasn ’ t found315 // put sync word at s t a r t o f bu f f e r , appending 0x00 to each by t e316 for ( i =0; i < 8 ; i++) {317 b u f f e r [ p o s i t i o n ++] = c [ i ] ;318 }319 // read in b i t s t r eam320 while ( ! f e o f ( pF i l e ) ) {321 b u f f e r [ p o s i t i o n++]=f g e t c ( pF i l e ) ;322 }323 // f i r s t t h e r e i s a con t r o l t r a n s f e r to s e t up the opera t ion324 //0x40 :0 xb2 :0 x000000000000325 i f ( i >= 0){ // check f o r error326 i = usb bu lk wr i t e (OK dev , 0x02 , bu f f e r , po s i t i on −2, 2000 ) ;327 i f ( i == p o s i t i o n − 2) {328 //do another con t r o l t r a n s f e r to check s t a t u s329 i = usb contro l msg (OK dev , 0xc0 , 0xb2 , 0 , 0 , c , 1 , 7 0 0 ) ;330 i f ( i == 1) {331 i = 0 ;332 i f ( c [ 0 ] != 0x01 ){ // check f o r s u c c e s s f u l s t a t u s333 i = 0 ;334 } else {335 p r i n t f ( ” s t a t u s e r r o r : %x\n” , c [ 0 ] ) ;336 }337 }// end i f c on t r o l check338 }// end i f bu l k wr i t e succe s s339 }// end i f c on t r o l se tup340 }// end i f sync word not found341 f r e e ( b u f f e r ) ; // d e a l l o c a t e memory f o r b i t s t r eam bu f f e r342 }// end i f a l l o c a t e b u f f e r f o r b i t s t r eam343 }// end i f f i l e open344 return 0 ;345 }346 #endif347348 int OK update wirein ( usb dev handle ∗ OK dev , char∗ data ) {349 // wire in ’ s are sen t as a con t r o l t r a n s f e r . se tup packe t i s350 //0x40 b5 00 00 00 00 40 00. t h e r e are 64 data b y t e s r ep r e s en t i n g the va l u e s351 //on each o f the 32 p o s s i b l e w ire in ’ s . w i re in endpoint 0x00 i s sen t as the352 // f i r s t pa i r o f b y t e s LSB:MSB, and the r e s t f o l l ow in order u n t i l the l a s t353 // p o s s i b l e wire in endpoint o f 0x1F , which t ak e s the l a s t two by t e s in the354 // data355 //356 // the r e a l l i b r a r y b u f f e r s a l l the wire in va l u e s and a l l ows you to update a357 // s i n g l e endpoint in the bu f f e r , and sub s e quen t l y update a l l endpointe wi th358 //a sepe ra t e c a l l . I have not implemented t ha t f o r s imp l i c i t y , so you have359 // to keep t rack o f a l l w i re in va l u e s in your code and c a l l t h i s f unc t i on360 // wi th a l l the data .361 int i = usb contro l msg (OK dev , 0x40 , 0xb5 , 0 , 0 , data , 64 , 7 0 0 ) ;

34

362363 return i ;364 }365366 int OK get wireout ( usb dev handle ∗ OK dev , char∗ data ) {367 // wireout ’ s are sen t as a con t r o l t r a n s f e r . se tup packe t i s368 //0xc0 b5 20 00 00 00 40 00. the 0x0020 va lue in wValue was somewhat369 // unexpected , s ince the r e i s no corresponding one in the wire in c a l l . t h e r e370 // are 64 data b y t e s r ep r e s en t i n g the va l u e s on each o f the 32 p o s s i b l e371 // wireout ’ s . wireout endpoint 0x20 i s sen t as the f i r s t pa i r o f b y t e s372 //LSB:MSB, and the r e s t f o l l ow in order u n t i l the l a s t p o s s i b l e wireout373 // endpoint o f 0x3F , which t a k e s the l a s t two by t e s in the data374 //375 // the r e a l wireout f unc t i on a l l ows you to s p e c i f y a p a r t i c u l a r endpoint you376 //want data f o r . my func t i on f i l l s the b u f f e r po in ted to by data wi th a l l 64377 // by t e s and you have to e x t r a c t the one you want y o u r s e l f . endpoint 0x20 i s378 // the f i r s t two by t e s LSB:MSB and the r e s t f o l l ow in order u n t i l endpoint379 //0x3F which i s the l a s t two by t e s .380 int i = usb contro l msg (OK dev , 0xc0 , 0xb5 , 0x0020 , 0 , data , 64 , 7 0 0 ) ;381382 return i ;383 }384385 int O K s e t t r i g g e r i n ( usb dev handle ∗ OK dev , int ep , char∗ data ) {386 // t r i g g e r ins are sen t as a con t r o l t r a n s f e r . the format appears i s387 //0x40 0xb5 0x [ endpoint ] 0x00 0x01 0x00 0x02 0x00 which corresponds to388 // usb con t ro l msg (OK dev , 0x40 , 0xb5 , endpoint , 1 , data , 2 , 700)389 // the 2 data by t e s r ep re s en t the 16 p o s s i b l e t r i g g e r s a s s o c i a t e d wi th the390 // endpoint . The o r i g i n a l l i b r a r y only a l l ow s f o r s e t t i n g 1 b i t a t a time , so391 // f o r each t ransac t i on , the data b y t e s would have only 1 b i t s e t . I don ’ t392 //know what the r e s u l t o f s e t t i n g more than on b i t would be . A reasonab l e393 // guess would be t ha t mu l t i p l e t r i g g e r s would be s e t . so f o r compat i b l e394 // operat ion , on ly have one s e t b i t in the 2 by t e s po in ted to by data395 int i = usb contro l msg (OK dev , 0x40 , 0xb5 , 0x0001 , 1 , data , 2 , 7 0 0 ) ;396397 return i ;398 }399400 int OK getTrigger out ( usb dev handle ∗ OK dev , char∗ data ) {401 // t r i g g e r outs are sen t e x a c t l y the same as wire outs , but wi th a d i f f e r e n t402 // se tup by t e : 0xc0 0xb5 0x60 0x00 0x10 0x00 0x40 0x00403 int i = usb contro l msg (OK dev , 0xc0 , 0xb5 , 0x0060 , 1 , data , 64 , 7 0 0 ) ;404405 return i ;406 }

35

36

APPENDIX C

Response time benchmark code

C.1 Normal and Debian

1 #include 2 #include 3 #include 4 #include 56 #include 78 int g p i o i n = AT91 PIN PB17 ;9 int gp io out = AT91 PIN PB1 ;

1011 stat ic i r q r e t u r n t i r q h a n d l e r ( int i rq , void ∗ dev id )12 {13 int output va lue = g p i o g e t v a l u e ( g p i o i n ) ;14 g p i o s e t v a l u e ( gpio out , output va lue ) ;1516 return IRQ HANDLED;17 }1819 int i n i t in i t modu l e ( )20 {21 a t 9 1 s e t g p i o i n p u t ( gp io in , 0 ) ;22 a t 9 1 s e t g p i o o u t p u t ( gpio out , 0 ) ;2324 return r e q u e s t i r q ( gp io in , /∗ i r q number ∗/25 i rq hand l e r , /∗ i r q hand ler f unc t i on ∗/26 0 , /∗ f l a g s ∗/27 ” q p i o i r q ” , /∗ name ∗/28 0 ) ; /∗ po in t e r f o r i r q hand ler ∗/2930 return 0 ;31 }3233 void cleanup module ( )34 {35 f r e e i r q ( gp io in , 0 ) ;36 }37

37

38 MODULE LICENSE( ”GPL” ) ;

C.2 Xenomai

1 #include 2 #include 3 #include 45 #include 6 #include 78 #include 9 #include

1011 r t dm i rq t i r q h a n d l e ;12 int g p i o i n = AT91 PIN PB17 ;13 int gp io out = AT91 PIN PB1 ;1415 int i r q h a n d l e r ( r t dm i rq t ∗ i r q h a n d l e )16 {17 int output va lue = g p i o g e t v a l u e ( g p i o i n ) ;18 g p i o s e t v a l u e ( gpio out , output va lue ) ;1920 return RTDM IRQ HANDLED;21 }2223 int i n i t in i t modu l e ( )24 {25 a t 9 1 s e t g p i o i n p u t ( gp io in , 0 ) ;26 a t 9 1 s e t g p i o o u t p u t ( gpio out , 0 ) ;2728 r t d m i r q r e q u e s t (& i rq hand l e , /∗ i r q h and l e r ∗/29 gp io in , /∗ i r q number ∗/30 i rq hand l e r , /∗ i r q hand ler f unc t i on ∗/31 0 , /∗ f l a g s ∗/32 ” gpio i r q ” , /∗ name ∗/33 0 ) ; /∗ po in t e r f o r i r q h and l e r ∗/3435 r tdm i rq enab l e (& i r q h a n d l e ) ;36 return 0 ;37 }3839 void cleanup module ( )40 {41 r t d m i r q d i s a b l e (& i r q h a n d l e ) ;42 r t d m i r q f r e e (& i r q h a n d l e ) ;43 }4445 MODULE LICENSE( ”GPL” ) ;

38

APPENDIX D

Communication benchmark code

D.1 Normal

1 #include 2 #include 3 #include 4 #include 56 #include 78 int D0 = AT91 PIN PA0 ; /∗ wr i t e edge from nanosg20 proces sor ∗/9 int D1 = AT91 PIN PA1 ; /∗ da t a l i n e from nanosg20 proces sor ∗/

10 int D2 = AT91 PIN PA2 ; /∗ i n t e r r u p t l i n e to nanosg20 ∗/11 int D3 = AT91 PIN PA3 ; /∗ da t a l i n e to nanosg20 ∗/1213 stat ic i r q r e t u r n t i r q h a n d l e r ( int i rq , void ∗ dev id )14 {15 int data = g p i o g e t v a l u e (D3 ) ;1617 g p i o s e t v a l u e (D1 , ! data ) ;18 g p i o s e t v a l u e (D0 , 1 ) ;19 g p i o s e t v a l u e (D0 , 0 ) ;2021 return IRQ HANDLED;22 }2324 int i n i t in i t modu l e ( )25 {26 a t 9 1 s e t g p i o i n p u t (D2 , 0 ) ;27 a t 9 1 s e t g p i o i n p u t (D3 , 0 ) ;28 a t 9 1 s e t g p i o o u t p u t (D0 , 0 ) ;29 a t 9 1 s e t g p i o o u t p u t (D1 , 0 ) ;3031 return r e q u e s t i r q (D2 , /∗ i r q number ∗/32 i rq hand l e r , /∗ i r q hand ler f unc t i on ∗/33 0 , /∗ f l a g s ∗/34 ” q p i o i r q ” , /∗ name ∗/35 0 ) ; /∗ po in t e r f o r i r q hand ler ∗/36 }37

39

38 void cleanup module ( )39 {40 f r e e i r q (D2 , 0 ) ;41 }4243 MODULE LICENSE( ”GPL” ) ;

D.2 Xenomai

1 #include 2 #include 3 #include 4 #include 5 #include 67 #include 8 #include 9

10 r t dm i rq t i r q h a n d l e ;11 int D0 = AT91 PIN PA0 ; /∗ wr i t e edge from nanosg20 proces sor ∗/12 int D1 = AT91 PIN PA1 ; /∗ da t a l i n e from nanosg20 proces sor ∗/13 int D2 = AT91 PIN PA2 ; /∗ i n t e r r u p t l i n e to nanosg20 ∗/14 int D3 = AT91 PIN PA3 ; /∗ da t a l i n e to nanosg20 ∗/1516 int i r q h a n d l e r ( r t dm i rq t ∗ i r g h a n d l e )17 {18 int data = g p i o g e t v a l u e (D3 ) ;1920 g p i o s e t v a l u e (D1 , ! data ) ;21 g p i o s e t v a l u e (D0 , 1 ) ;22 g p i o s e t v a l u e (D0 , 0 ) ;2324 return RTDM IRQ HANDLED;25 }2627 int i n i t in i t modu l e ( )28 {29 a t 9 1 s e t g p i o i n p u t (D2 , 0 ) ;30 a t 9 1 s e t g p i o i n p u t (D3 , 0 ) ;31 a t 9 1 s e t g p i o o u t p u t (D0 , 0 ) ;32 a t 9 1 s e t g p i o o u t p u t (D1 , 0 ) ;3334 return r t d m i r q r e q u e s t (& i rq hand l e , /∗ i r q h and l e r ∗/35 D2 , /∗ i r q number ∗/36 i rq hand l e r , /∗ i r q hand ler f unc t i on ∗/37 0 , /∗ f l a g s ∗/38 ”xen i r q ” , /∗ name ∗/39 0 ) ; /∗ po in t e r f o r i r q h and l e r ∗/40 }4142 void cleanup module ( )43 {44 f r e e i r q (D2 , 0 ) ;4546 r t d m i r q d i s a b l e (& i r q h a n d l e ) ;47 r t d m i r q f r e e (& i r q h a n d l e ) ;

40

48 }4950 MODULE LICENSE( ”GPL” ) ;

D.3 XEM6001

1 // S t r ipped from i n i t i a t i o n par t s o f the code .2 // s t a t e machine hand l ing the pio p ins3 // D0 = wr i t e edge from nanosg20 proces sor4 // D1 = da t a l i n e from nanosg20 proces sor5 // D2 = in t e r r u p t l i n e to nanosg206 // D3 = da t a l i n e to nanosg207 // S0=> wai t f o r wr i t e pin from nanosg20 . I f 1 move to S18 // S1=> Read D1 pin o f nanosg20 put D3 to the same va lue as D1. Put D2 high9 // and move to S2

10 // S2=> put D2 low and move back to S011 always @( s t a t e or r e s e t 1 )12 begin13 i f ( r e s e t 1 )14 d3

51 endcase52 end53 endmodule

42

IntroductionBuildroot and real-time benchmarksCreating real-time Linux kernelsBuilding embedded Linux systems with BuildrootBenchmarking real-time Linux

Characteristics of the platformsCharacteristics of the NanosG20Hardware specificKernel modules for NanosG20

XEM6001 ConfigurationFrontPanel configuration protocolImplemented configuration protocol

Boards setup

Benchmarks resultsMeasurement of the response timeMultiple InterruptsCommunication benchmark

ConclusionFuture work

NanosG20 kernel module makefileOpenOK communication code for XEM6001OpenOK.hOpenOK.c

Response time benchmark codeNormal and DebianXenomai

Communication benchmark codeNormalXenomaiXEM6001