Implementations in OSE and Linux

download Implementations in OSE and Linux

of 56

Transcript of Implementations in OSE and Linux

  • Evaluation of Two SCTP Implementations in OSE and Linux

    S O P H I A L I

    Master of Science Thesis Stockholm, Sweden 2006

  • Evaluation of Two SCTP Implementations in OSE and Linux

    S O P H I A L I

    Masters Thesis in Computer Science (20 credits) at the School of Computer Science and Engineering Royal Institute of Technology year 2006 Supervisor at CSC was Olof Hagsand Examiner was Stefan Arnborg TRITA-CSC-E 2006:056 ISRN-KTH/CSC/E--06/056--SE ISSN-1653-5715 Royal Institute of Technology School of Computer Science and Communication KTH CSC SE-100 44 Stockholm, Sweden URL: www.csc.kth.se

  • Abstract

    This thesis aims to compare the performance of Stream Control Transmission Protocol (SCTP) in the two different operating systems, OSE and Linux. The CPU load of the performances was measured.

    SCTP traffic was sent between two regional processors in the OSE environment. The same traffic scenarios were tested between another two regional processors in the Linux environment. The same algorithm was designed to calculate the CPU load of SCTP in OSE and Linux. The measurement results were then compared and analyzed.

    The final conclusion is that the SCTP performance in OSE is better than that in Linux with small bit rates while worse than that in Linux with big bit rates.

  • Utvrdering av tv SCTP-implementationer i OSE och Linux

    Sammanfattning

    Uppgiften i detta exjobb r att jmfra CPU-belastning av SCTP(Stream Control transmission Protocol) i tv operativsystem, OSE och Linux.

    SCTP trafik skickas mellan tv regionala processorer i OSE. Samma trafikscenario testas ocks mellan tv processorer i Linux. En algoritm designas s att CPU-belastning av SCTP i OSE och Linux kan berknas. Mtningsresultat jmfrs sedan och analyseras.

    Slutresultatet r att SCTP i OSE har mindre CPU-belastning n i Linux med lgre bitrate och strre CPU-belastning n i Linux med hgre bitrate.

  • Acknowledgement

    This is a master thesis in Computer Science in Nada, Royal Institute of Technology (Kungliga Tekniska Hgskolan). The supervisor in Nada is Olof Hagsand and the examinator is Professor Stefan Arnborg. This thesis work is implemented at Ericsson AB in lvsj and the supervisor in Ericsson AB is Anders Magnusson and Stefan Mattson.

    I would like to express my gratitude to Anders Magnusson for his excellent guidance. His advice has been of vital importance during my work.

    I also wish to thank my partner Qi Zhang in IMIT, KTH for his companionship and uncountable fruitful discussions and all the colleagues at the UZ/DK in Ericsson AB for their companionship, with special thanks to Stefan Mattson, Richard Tham and Per Holmgren for their kind help during this work.

    Last but not least, I wish to thank my family My husband, Min Qiu and my two kids, Katarina and Henrik, for their loves and supports.

  • Content

    1. Introduction .......................................................................................1 1.1 Overview of the thesis ......................................................................1 1.2 Problem .......................................................................................2 1.3 Assigner .......................................................................................2 1.4 Assignment and objectives ...............................................................2 1.5 Tasks .......................................................................................3 1.6 Limitations .......................................................................................3 2. SCTP .......................................................................................4 2.1 Introduction of SCTP .......................................................................4 2.2 Message Format................................................................................5 2.3 Multihoming .....................................................................................6 2.4 Multistreaming .................................................................................7 2.5 Conclusion of SCTP and TCP..........................................................8 3. Operating System ....................................................................................9 3.1 General .......................................................................................9 3.2 Processes .......................................................................................9 3.3 Scheduling .....................................................................................10 3.4 Memory Management.....................................................................10 3.5 Inter-process Communication (IPC)...............................................11 3.6 Process Synchronization.................................................................12 3.7 Profiler .....................................................................................12 4. IP Stack .....................................................................................15 4.1 TIP stack .....................................................................................15 4.2 Linux IP stack.................................................................................17 5. Test Application ....................................................................................19 5.1 General .....................................................................................19 5.2 Traffic sending format ....................................................................20 5.3 The CPU load of test applications ..................................................21 6. Hardware Environment .........................................................................22 7. CPU Load Measurement Algorithms ....................................................24 8. Measurement Results ............................................................................25 8.1 Measurement results of SCTP in OSE and Linux ..........................25 8.2 Discussions about the measurement results ...................................27 8.3 Conclusion .....................................................................................27 9. Analysis .....................................................................................28 9.1 Buffer Copying ...............................................................................28 9.2 Context Switch ...............................................................................28 9.3 Chunk bundling for SCTP ..............................................................31 9.4 Other Aspects .................................................................................32 10. Final Conclusion .................................................................................33 11. Future studies .....................................................................................33 REFERENCES .....................................................................................34 APPENDIX A Terms and Abbreviations................................................36 APPENDIX B Test Procedure in OSE....................................................38 APPENDIX C Implementation of Memory profiler in OSE ..................41 APPENDIX D SCTP Measurement Data for Linux ...............................42 APPENDIX E TCP Measurement Data in Linux ...................................46

  • 1 Introduction

    This report is a master thesis report in Computer Science at Nada, the Royal Institute of Technology. It is carried out at Ericsson AB in lvsj. The supervisor at Ericsson is Anders Magnusson and Stefan Mattson and the supervisor at Nada is Olof Hagsand. The detailed information of literature background and how the work is implemented is provided in the report. After reading this report, the reader will be able to get a general idea of the performance of SCTP in OSE and Linux.

    1.1 Overview of the thesis

    The whole report is divided into 11 sections. .

    Section 1 is the introduction part. It contains the disposition of the report and the general information about the assigner and this thesis work.

    An introduction of the Stream Control Transmission Protocol (SCTP) is provided in section 2. The basic and outstanding features of SCTP, such as message format, multihoming and multistreaming are discussed by the way of comparing SCTP with TCP.

    In section 3, the two systems that are used in this thesis work, OSE Delta and Montavista Linux 3.1, are introduced and compared from the aspects of processes, scheduling, memory management, inter-process communication, process synchronization and profilers.

    The basic structures of the two IP stacks, TIP stack and Linux IP stack(Linux kernel implemented TCP/IP stack and OpenSS7) are described in detail in section 4.

    In section 5, the general information of the two test applications, TEIP in OSE and Iperf in Linux, are described. The traffic generating format is the same in TEIP and Iperf and presented in this section.

    The hardware environment for OSE and Linux is very similar so that the results are comparable. In section 6 the hardware environment is presented with the help of figures.

    In section 7 the algorithm to calculate the CPU load is explained in detail.

    The measurement results and some discussions of the results are shown in section 8.

    1

  • Section 9 aims to analyze buffer copying, context switch, chunk bundling and fast path etc. in OSE and Linux. These factors usually have big influence on the CPU load.

    Section 10 aims to draw a conclusion of the whole project.

    Section 11 aims at outlining possible future work.

    1.2 Problem

    OSE Delta is an operating system kernel produced by ENEA Systems. Ericsson has a large number of applications that are built to run on OSE Delta. However, there are alternatives, such as Linux, to build their own tailor- made applications, especially within the IP domain. Linux has a mature and standardized IP stack with a lot of open source code available. Ericsson AB therefore wants to analyze the open source of Linux versus the real time characteristics of OSE Delta. This thesis work, Evaluation of the SCTP performance in OSE and Linux is a major part of that analysis.

    1.3 Assigner

    The assigner of this master thesis is Ericsson AB and the contact person is Anders Magnusson in section UZ/DK in lvsj. UZ/DK is responsible for protocol handling. They use the IP stack from Ericsson Telebit in Denmark and produce their own adaptation layer to import the IP stack to GARP (the Regional Processor).

    1.4 Assignment and objectives

    The scope of this thesis work is to measure and compare the CPU load of the SCTP/IP stack on GARP (the Regional Processor) for both OSE and LINUX. On the two platforms, the hardware is identical while the software differs much. Not only the SCTP/IP stack, the OS and the test application are different, but also the system design of the stack. There are also differences in buffer handling and copying of data. Note that these differences are to a large extent not something that is implied by the OS itself but to the actual implementation.

    2

  • The assignment consists of two parts. The first part is to send SCTP and TCP traffic between two GARPs (Regional Processors) running OSE and then send the same traffic between two GARPs running Linux. The CPU load of the IP stacks is measured. TCP is not really interesting for this project. However, TCP is already a mature protocol in OSE while SCTP is just a prototype. Ericsson wants to estimate the comparison result for a mature SCTP in OSE and SCTP in Linux. It is believed that the comparison result of SCTP should be the same as that of TCP since SCTP and TCP are two similar protocols. So the results from TCP tests are good inputs to the estimation of comparison results between the future mature SCTP in OSE and that in Linux. The project is trying to make the traffic scenario as similar as possible for both OSE and Linux so that the results are comparable.

    The second part is to analyze the results from all the tests. The main goal of this part is to find why the CPU load is different for the SCTP in OSE and in Linux.

    1.5 Tasks

    1. Design Test Application to generate SCTP and TCP traffic. 2. Run TCP and SCTP traffic. 3. Design an algorithm to measure the CPU load of SCTP in OSE and

    Linux. 4. Prototype and ideas might have surfaced during the tests. 5. Analyze and present the results.

    1.6 Limitations

    This thesis was carried out by Qi Zhang at KTH and me. Qi is responsible for Linux part and I am responsible for OSE part. As a result, this report mainly focuses on OSE.

    Due to the business securitys reason, the measurement data for OSE are not attached in this report.

    3

  • 2 SCTP

    The stream control transmission protocol (SCTP) [13, 14] is a new standard for general-purpose transport proposed by the Internet Engineering Task Force. SCTP addresses application and security gaps left open by its predecessors, TCP and UDP. This section will provide an overview of SCTP with focusing on the outstanding features by the way of comparing SCTP and TCP.

    2.1 Introduction of SCTP

    With the exponential growth of the Internet, IP technology has established itself as a cornerstone for modern digital communication. Until now, there have been two general purpose transport protocols widely used for applications over IP networks User Datagram Protocol (UDP) and Transmission Control Protocol (TCP). Each provides a set of services that cater certain classes of applications. However, the service provided by TCP and UDP are disjoint and together do not satisfy ideally the needs of all network applications. SCTP is designed to bridge the gap between UDP and TCP, and addresses shortcomings of both. SCTP was originally developed to carry telephony signalling messages over IP networks for telecommunications and e-commerce systems. With continued work, SCTP evolved into a general purpose transport protocol. Today it is a proposed Internet Engineering Task Force standard (RFC 2960) [14, 15].

    Like TCP, SCTP provides a reliable, full-duplex connection and mechanisms to control network congestion. Unlike both TCP and UDP, however, SCTP offers new delivery options that are particularly desirable for telephony signaling and multimedia applications. Table 1 compares SCTPs services and features with those of TCP and UDP.

    4

  • Table 1: Comparison of SCTP services and features with those of TCP and UDP [21]

    An SCTP connection, called an association, provides novel services such as multihoming, which allows the end points of a single association to have multiple IP addresses, and multistreaming, which allows for independent delivery among data streams. SCTP also features a four way handshake to establish an association, which makes it resistant to blind denial-of-service attack and thus improves overall protocol security.

    2.2 Message Format

    TCP provides a byte-stream data delivery service, whereas SCTP provides a message-oriented data delivery service. SCTP packets are structured to provide a message-oriented service, and allow flexible message bundling. Figure 1 illustrates a generalization of the SCTP packet format.

    5

  • Figure 1: SCTP packet format. The common header is followed by one or more concatenated chunks containing either control or data information [13].

    The packets always begin with an SCTP common header. The common header is a minimal structure that provides three basic functions:

    1. Source and destination ports. Together with the IP addresses in the IP header, the port numbers identify the association to which an SCTP packet belongs. 2. Verification tag. Vtags ensure that the packet belongs to the current incarnation of an association. 3. Checksum. This computed value maintains the entire packets data integrity.

    The remainder of an SCTP packet consists of one or more concatenated building blocks called chunks that contain either control or data information. This format differs from TCP and UDP packets, which include control information in the header and offer only a single optional data field. SCTP control chunks transfer information needed for association functionality, while data chunks carry application layer data. SCTP is extensible, allowing new control chunk types to be defined in the future. Each chunk has a chunk header that identifies its length, type, and any special flags the type needs. SCTP has the flexibility to concatenate different chunk types into a single data packet. The only restriction is on packet size, which cannot exceed the destination paths maximum transmission unit (MTU) size.

    2.3 Multihoming

    6

  • SCTP has the functionality of multihoming. A multihomed host is accessible through multiple IP addresses. If one of its IP addresses fails, the host can still receive data through an alternative interface. Currently, SCTP uses multihoming only for redundancy but not for load balancing. Each end point chooses a single primary destination address for sending all new data chunks during normal transmission. Continued failure to reach the primary address ultimately results in failure detection, at which point the end point transmits all chunks to an alternate destination until the primary destination becomes reachable again. SCTP keeps track of each destination addresss reachability through two mechanisms: ACKs of data chunks and heartbeat chunkscontrol chunks that periodically probe a destinations status. All chunks which require a response can be used to determine a destinations reachability. Usually, DATA chunks represent the majority of such chunks, but DATA chunks are generally sent to only one destination. Regardless of the chunk sent, if an ack is received, the sender can conclude that the address the chunk was sent to is reachable. If an ack is not received, however, the sender cannot conclude immediately that the destination is unreachable. Instead, the sender credits that destination address with a loss. If a significant number of consecutive chunks are lost to the same destination, the sender concludes that that destination is unreachable, and an alternate destination IP address is chosen dynamically [15]. Because SCTP/IP stack in OSE is just a prototype without the functionality of multihoming by now, this function is out of the range of this thesis work.

    2.4 Multistreaming

    One of the unique features that SCTP brings to the transport layer is multistreaming. An SCTP stream is a unidirectional logical data flow within an SCTP association. Streams are specified during association setup and exist for the life of the association. Each stream is allocated independent send and receive buffers. In Figure 3, Host A and B have a multistreamed association, where during association setup, Host A requested three streams to Host B (numbered 0-2), and Host B did not request multiple streams to Host A, therefore Host B maintains only one stream to Host A(numbered 0).

    7

  • Figure 3. SCTP multistreamed association. Streams are unidirectional logical data flows that the SCTP end points negotiate during association setup [13]. Within streams, data order and reliability are preserved with the use of Stream Sequence Numbers (SSNs) for each DATA chunk. However, between streams, only partial data order will be preserved to address head-of-line blocking. In TCP, when a sender transmits multiple TCP segments and the first of such segments is lost, the remainder of the segments must wait in the receiver's queue until the first segment gets retransmitted and arrive correctly. This blockage will delay the delivery of data to the application, which in signalling and some multimedia applications is unacceptable. In SCTP, however, if data on Stream 1 is lost, only Stream 1 is blocked at the receiver while awaiting retransmissions. The data on the remaining streams is deliverable to the application. An endpoint's multiple streams are logically independent from an endpoint's multiple interfaces. Data from any stream may potentially travel over any path. Also, streams themselves do not have sending restrictions. Instead, they conform to the restrictions placed on the association as a whole, such as the receiver's advertised window and the destinations' congestion window.

    2.5 Conclusion of SCTP and TCP

    SCTP is designed to bridge the gap between TCP and UDP and addresses some shortcomings of both. In this thesis work, it is expected that SCTPs performance is similar with TCPs performance but with improvement.

    8

  • 3 Operating System

    OSE Delta and MontaVista Linux 3.1 are the two operating systems that are used in this thesis work. OSE Delta is a real time operating system kernel which is provided by ENEA OSE Systems AB, Sweden. MontaVista Linux 3.1 (based on GNU Linux 2.4.20 kernel) is a pre-emptive Real Time OS kernel provided by MontaVista Software Inc [23]. In this section, these two systems are introduced and compared from the aspects of processes, scheduling, memory management, inter-process communication and process synchronization.

    3.1 General

    OSE OSE Delta [11] is a proprietary RTOS designed to satisfy the specific needs of the tele- and data-communication industries. OSE Systems are Wireless System, Network Infrastructure and Safety Critical systems [24]. Linux Linux is an UNIX-like general-purpose operating system. It is available in source code and some distributions are available free of charge. Linux is designed for high throughput and high average performance. Linux is considered to be a very reliable general purpose OS with good support for communication protocols.

    3.2 Processes

    OSE There are different types of processes. Interrupt processes are activated in response to a hardware interrupt or a software event. Timer Interrupt processes behave in the same way except that the system timer activates them. All interrupt processes run from start to end unless interrupted by an interrupt process with higher priority. A prioritised process is the most common process. It runs as long as no interrupts are received, or until it blocks on a resource, or until a prioritised process with higher priority is ready to run. Background processes are run in a time sharing mode (round-robin) at a priority level below the prioritised processes, i.e. they run only when no other processes are ready. Linux A Linux system allocates CPU time through the use of processes and threads. A process is independent from its creator with its own memory area and its own process identification (PID). A new thread of execution has its own stack (local variables) but shares global variables and file descriptors with its creator. However, the overhead of creating a new thread is less than that of creating a new process. Further, switching between threads is more efficient

    9

  • than switching between processes. Both the processes and the threads come in two types, normal and real-time. The difference between them is in how they are scheduled.

    3.3 Scheduling

    OSE OSE supports pre-emptive priority scheduling [18]. Processes can be scheduled in a priority-based, a timer-based and a time sharing way. Regardless of its priority, a process may be pre-empted at any time. After that the kernel may either return control to the pre-empted process or give control to a new process. Linux Linux has three different scheduling policies, one for normal applications(SCHED_OTHER) and two for real-time applications (SCHED_FIFO and SCHED_RR). SCHED_OTHER uses a time sharing credit based algorithm for fair pre-emptive scheduling among multiple processes/threads. The more credits a process has, the more likely it is to run. The process credits, or dynamic priority, are based on the processs historic running time. The other scheduling algorithms are designed for real-time applications where static priorities are more important than fairness. Processes/threads scheduled with SCHED_FIFO always pre-empt any currently running SCHED_OTHER process. If two real-time processes are ready to run at the same time, the scheduler chooses the one with the highest static priority.

    3.4 Memory Management

    OSE In an OSE system there is always a system pool of memory[11]. The system pool is crucial to the kernel and it is located in kernel memory. All processes can allocate their memory from the system pool, with one major disadvantage: If the system pool is corrupted the whole system will crash. A better way that is available is to group processes into blocks and to dedicate local pools to each block. If this local pool gets corrupted, it will only affect the processes in that block. Another advantage is that many system calls also work on entire blocks, i.e. you may kill or start all processes in a block with one single system call. It is also possible to send signals to a block and that the block acts as a router and transfers the signal to the right process inside the block. One or several blocks can be placed in a single segment. However, the only way to get full security is to isolate each segment with the MMU.

    10

  • Linux Linux has a virtual memory mechanism that allows demand paging and process swapping. According to much of the literature, virtual memory should not be used in a real-time system due to the lack of determinism. However, in Linux both the process swapping and demand paging can be turned off with the swapoff and mlock system calls. The memory management will still have some of the benefits of using a virtual memory mechanism, such as the protection and lack of external fragmentation, but not the downside when it comes to real-time performance.

    3.5 Inter-process Communication (IPC)

    OSE The way to pass data between processes in an OSE system is by signals. In an OSE system a signal is a message that is sent directly from one process to another. A message contains an ID, and the addresses of the sender and the receiver, as well as data. Once a signal is sent, the sending process can no longer access it, i.e. ownership of a signal is never shared. The receiving process may specify what signal it wants to receive at any particular moment. The process can wait or poll the signal queue for a specific signal. The kernel manages the ownership of signals. The known ownership makes it possible for a system debugger to keep track of the communication between processes, e.g. it makes it possible to set a break point on a specific signal during system debugging. Normally when transferring signals, the sending process only sends a pointer to a signal buffer. The receiving process uses this pointer to access the signal buffer. This has the advantage of making the system fast, but there is also the danger of the receiving process destroying the pool of the sending process. However, if a signal is sent between processes that are located in different segments the user can choose between having only the pointer transferred or having the signal buffer copied. Signals will die with the owning process and be cleaned up by the OS itself. Linux Linux supports pipes, FIFOs, message queues and shared memory as ways of passing data between processes or threads. Pipes are unidirectional byte streams that connect the standard output from one process to the standard input of another. Under Linux, pipes appear as just another type of in-node to the virtual file system. To synchronise the reader and the writer, each pipe has a pair of wait queues. Signals can also be used for synchronisation. Pipes can only be used between related processes, i.e. processes that have been started from a common ancestor process.

    11

  • For communication between unrelated processes Linux supports named pipes, known as FIFOs because they operate on a First In, First Out basis. Unlike pipes, FIFOs are not temporary objects, they are entities in the file system and are created using the mkfifo command. Linux itself will not delete a FIFO if the communicating processes dies. Message queues allow one or more processes to write messages that can be read by one or more reading processes. The lengths of these queues are limited so some protection against deadlocks is needed. Linux itself will not delete a message queue if one of the communicating processes dies and it is difficult for the application to find out that the queue should be deleted. Shared memory is a fast way to communicate large or small amount of data. It allows two unrelated processes to access the same logical memory. However, it must be used with semaphores or some other synchronisation mechanism since it can not provide any synchronisation by itself.

    3.6 Process Synchronization

    OSE The recommended way to synchronise two processes is by signals (see above). Linux The standard Linux mechanism for informing a process that an event has occurred is a signal. The most important difference between an OSE signal and a Linux signal is that the Linux signal cannot carry information. Only the kernel and the superuser can send signals to any other process. Normal processes can only send signals to processes with the same user-id and group-id or to processes in the same process group.

    3.7 Profiler

    Profilers are the tools which are used to measure statistics.

    OSE

    There are two profilers in OSE which were used in this project, process profiler and memory profiler.

    The process profiler is used to measure statistics for different processes. It is a "sample-based" profiler.

    12

  • The basic theory of the process profile is shown in Figure 5. A time interval T is selected in order to analyze the CPU load of processes. T is divided into N equal time intervals. After each time interval tn (at t0, t1, t2...tn), a timer interrupt is invoked, and the ISR (Interrupt Service Routine) finds out the current running process and increments the counter that is associated with this process. At the end of the interval, all processes have one counter, each indicating how many times it was running when an interrupt occurred. From this data, the CPU load can be estimated. One problem is if a process (C) is running during a time interval less than N and then stops running due to some reasons (e.g. finished or pre-empted), and another process (A) starts and continues on running until the time slice expired. Then, only the counter of (A) will be incremented but not (C). Therefore this is not a perfect way to measure the CPU load of the processes. But it is easy to implement and therefore adopted widely. In fact, the precision of the measurement can be increased by minimizing the time interval N. That is, for the same T, the bigger N is, the higher precision it has. On the other hand, when N is increasing, the cost of the timer interrupt will also increase.

    Figure 5 Sample based profiling

    The memory profiler in OSE divides the memory into a number of equally sized slices. A counter for a slice is incremented each time an interrupt occurs and the Program Counter points to an area within the slice. By looking at the counters for each slice, it is possible to determine which part of the code that runs most frequently.

    13

  • The Memory Profiler is implemented as a timer interrupt process, reading the register SRR0 every 2 milliseconds. In the tests every second system tick is used by another timer interrupt process. This makes it possible to collect usable data on the remaining ticks. This has the effect that the useable date collected by this profiler is collected every 4 milliseconds (every second system tick). The address of the tick process will, hence, take half of the hits of the profiler.

    8Mbyte memory is used to store the hit counters. Each counter is 32 bits, so the Profiler can divide the memory into max 2M (2097152) slices.

    Both the process profiler ant the memory profiler were used in the experiments to get the pure CPU load of the TIP stack.

    Linux No Linux profilers are used in this project.

    14

  • 4 IP Stack

    The IP stack used for OSE is the TIP stack produced by Ericsson Telebit in Denmark. The TCP/IP stack used in Linux is the kernel implemented TCP/IP stack for Linux kernel 2.4.20. The SCTP/IP stack used for Linux is the Open SS7 SCTP which is an open-source project[22].

    In this section, the basic structures and functions of the two IP stacks, TIP stack and Linux IP stack(Linux kernel implemented TCP/IP stack and OpenSS7) are presented in detail.

    4.1 TIP stack

    The TIP stack in OSE supports raw IP, UDP, TCP and SCTP. It is mainly constructed of INETL and INETR, see Figure 6. INETL is a unit that contains the interface library. It contains of header files, OSE signal definitions and a socket library to be linked into the application. The services provided by INETL can be divided into two parts:

    Stack configuration and management Traffic and Socket configuration

    INETR is the IP stack for GARP. INETR consists of TIP Core and RP adaptation layer (glue). The TIP core is the generic (OS and HW independent) IP stack. The TIP core includes a number of modules (RAW, UDP, TCP and so on). TIP core also include a scheduler which is used to prioritize the different threads (IP datagrams, configuration requests...). Because the TIP core is HW and OS independent, it needs a supporting glue layer for everything related to the HW and OS. The glue layer provides not only IP stack with access to OS functions, like timer, memory and so on, but also configuration interface towards applications and OS monitor commands, such as ifconfig, route, and ping. Additionally, the glue layer helps to attach interface drivers to the IP stack.

    An application interfaces the stack by socket (function) calls to INETL (for sending and receiving IP datagrams) and by OSE signals to the TIP stack. The socket calls are forwarded from INETL to INETR by means of OSE signals.

    15

  • Application int main()

    { int sd = socket(); }

    Process

    RAZOR Library

    INETL

    Socket GLUE

    OSI GLUE

    TIP CORE

    OM GLUE

    Socket Layer INETR

    SCTP TCP UDP

    IP

    Lower Layer

    Ethernet Distributor (ED)

    OSE Delta

    OS Kernel

    Figure 6. This picture shows the structure of TIP stack

    16

  • Note that TIP SCTP is still only a prototype by now. There are many functions which are not implemented yet, such as fragmentation, fast path and retransmission, etc.[1,20].

    4.2 Linux IP stack

    4.2.1 Linux native TCP/IP Stack

    The TCP/IP Stack used for Linux is the kernel implemented TCP/IP stack for Linux kernel 2.4.20. See Figure 7.

    `

    Socket Layer

    OpenSS7 SCTP

    TCP UDP

    IP

    Lower Layer

    Application int main()

    { int sd = socket(); }

    Process

    OS Kernel

    Other Linux Kernel components

    Figure 7. OpenSS7 SCTP and Linux TCP/IP Stack

    17

  • 4.2.2 OpenSS7 SCTP

    OpenSS7 SCTP is distributed as a Linux kernel patch for various version of the Linux kernel, because MontaVista Linux 3.1 is based on GNU Linux kernel 2.4.20. In the experiments in this thesis work, kernel-2.4.20-28.9-sctp-0.2.19.patch was used.

    In order to use the OpenSS7 SCTP, the following steps were made:

    1 Apply OpenSS7 SCTP patch to the source code of MontaVista Linux

    Kernel. 2 Re-compile the kernel into the kernel image using MontaVista kernel

    development tools. 3 Load the image to the GARP via PBOOT and run the kernel.

    18

  • 5 Test Application

    TEIP is the test application in OSE. Because TEIP is specially designed and only suitable for OSE, Iperf was chosen as the test application for Linux. To make the results comparable, modifications was made in Iperf in this project so that Iperf used the same algorithm with TEIP to generate exactly same traffic scenarios.

    This section consists of two parts. In the first part the general information about TEIP and Iperf are described. Then follows a description of the traffic generating algorithm which was used in both TEIP and Iperf .

    5.1 General

    5.1.1 TEIP

    TEIP[9] is the test application used in OSE. It generates different kinds of traffic and measures the efficiency of the IP stack. TEIP supports RAW, UDP and TCP. As part of this thesis work, TEIP was modified to support SCTP.

    To be able to send and receive IP traffic, a user has to start up TEIP Traffic Processes (TTP). These processes can be configured to open many sockets and handle different protocols. A user can initiate sending of the traffic. statistics are stored in the TTP and can be fetched with a command.

    TEIP can generate acknowledgement-based traffic and time-based traffic. In acknowledgement-based traffic, a new burst of packets are sent when the receiver has acknowledged that it has received the last burst. In time-based traffic, a new burst of packets are sent when a specified time passes. All the test scenarios in this project are time-based traffic scenarios.

    Each started traffic scenario will be measured in two ways. First, the time it takes to execute the traffic scenario will be measured. Then the idle load will be measured. This will be done with a process running on priority 31 called TEIP_IDLE_COUNT. This process will automatically start counting loops when the first TTP-process is created. The number of loops passed in a traffic scenario and the time it takes will then be stored in the socket structure and can be fetched with a command.

    5.1.2 Iperf

    Iperf is the test application in Linux. Iperf[7] is an open source network bandwidth measurement tool for TCP and UDP. OpenSS7 has modified Iperf to support OpenSS7 SCTP.

    19

  • Iperf were modified in this project so that it uses the same algorithm as TEIP to generate the traffic and measure the CPU load of IP stacks.

    5.2 Traffic sending format

    The message sending format is shown in figure 8:

    Packet 2 Delay Packet N

    Traffic Time

    Burst Timeout

    DelayTime

    Figure 8. Traffic sending format.

    The following terms are used in Figure 8:

    Traffic time: the exact time that all packets in one burst are sent.

    Burst Timeout: the predefined time for sending one burst.

    DelayTime: the time the sender needs to wait for sending the next burst. It is calculated by the bit rate, the burst size and the packet size according to the following formulas:

    Burst timeout = Packet Size * Burst Size * 8 / Bit Rate

    DelayTime = Burst Timeout Traffic Time

    The traffics generated in both TEIP and Iperf are based on bursts. The sender sends the packets in one burst without any delay. As shown in Figure 8, packet 1, packet 2, , packet N belong to one burst and are sent without any delay. After all the packets in one burst are sent, the traffic sender will stop sending for DelayTime and then begin to send the next burst.

    20

  • 5.3 The CPU load of test applications

    TEIP The process profiler can be used directly to fetch the CPU load of TEIP. See the detailed information of the process profiler in section 3.7. Iperf There are no suitable profilers found in Linux to get the CPU load of Iperf directly. Therefore the CPU load of Iperf is calculated in the following ways: There are many functions in Iperf, such as getSystemTime etc., which are non-related to the traffic. So two versions of Iperf were used to extract the CPU load of Iperf. One is the normal version and the other is the modified version where all the non-related operations run twice. The CPU load of IP stack is calculated for the two versions respectively (see section 7 for detailed information of calculation the CPU load of IP stack). The difference is then the CPU load of Iperf. According to the test results, the CPU load of Iperf is always less than 1% and will therefore be ignored.

    21

  • 6 Hardware Environment

    The hardware environment for both OSE and Linux is identical, including the test target (GARP) and the local network equipments (Ethernet interface, switch, Ethernet cables, etc.). The only difference is the configuration part, as it is shown in the figures 9(a) and 9(b). However, this configuration difference will not influence the test results.See figure 9(a) and 9(b)

    Figure 9(a). Hardware environment for OSE

    22

  • Figure 9(b). Hardware environment for Linux

    23

  • 7 CPU Load Measurement Algorithms

    The algorithm to calculate the CPU load is explained in this section.

    The idle process is created in both OSE and Linux as a process which has a lower priority level and does nothing but counts loops. This process always stays in a ready state and will run if there is no other process with higher priority running.

    The measurement algorithm is designed in the following way:

    1. Measure how many loops the idle process can run per second in a silent system, S. A silent system refers to a system without traffic. Reading the value of the counter in the idle process at time T1 and T2 results in values N1 and N2, respectively.

    S [loop/second] =( N2 - N1 ) / ( T2 - T1 ).

    2. Measure how many loops the idle process can run during a time when there is traffic, L. The traffic starts at the time T3 and finishes at the time T4, and the value of the counter of the idle process is N3 at T3 and N4 at T4

    L [loop] = N4-N3.

    3. The total capacity of the silent system during the time when there is traffic, C: C=S*(T4-T3).

    4. The CPU load of the Idle process during the time when there is traffic, N: N=L/C

    5. The CPU load of the traffic, Ytraffic: Ytraffic = 1-N/C.

    Ytraffic includes not only the CPU load of IP stack, but also the CPU load of the test application. However, the impact of the test application (Iperf) in Linux is so small that it is neglectable (see section 5.3). So the pure CPU load of IP stack in Linux, Ylinux: Ylinux = Ytraffic

    6. In OSE, the CPU load of the test application (TEIP) is not neglectable and should be subtracted from Ytraffic. The CPU load of TEIP, Yteip, can be fetched directly from the process profiler. However, Yteip includes part of the CPU load of IP stack because TEIP calls functions from INETL which also belongs to the TIP stack. In other word, part of the CPU load of the TIP stack, Ylib, has also been subtracted from Ytraffic. So Ylib should be added back to Ytraffic. Ylib can be fetched directly by the help of the memory profiler. Therefore, the pure CPU load of the IP stack in OSE , Yose:

    Yose =Ytraffic-Yteip+Ylib

    24

  • 8 Measurement Results

    This section consists of three parts. In the first part the measurement results of SCTP in OSE and Linux are presented with the help of several figures. Then follows a discussion about some problems that lie in the measurement results. The last part of this section is the conclusion.

    The CPU load for SCTP and TCP are tested and measured for both sender and receiver sides in both the OSE environment and the Linux environment. Burst sizes 1, 5 and 20 are chosen for the tests. Packet sizes range from 25 to 1600 bytes (in Linux tests 10 bytes are also tested), and bit rates range from 1 Mbit per second to 60 Mbit per second (in some tests higher bit rates are also tested). However, due to the limitation of SCTP in OSE, 1400 bytes is as the biggest packet size. The measurement data of SCTP and TCP in Linux is given in Appendix D respective Appendix E.

    Note:

    z For business reasons, the measurement data for SCTP and TCP in OSE is not shown in this report.

    z For business reasons, all the figures in this section are only sketches to show the tendency of the comparison results of SCTP or TCP in OSE and Linux. The figures aim to give an overview of the comparison results. The data in the figures deviate from the real ones.

    8.1 Measurement results of SCTP in OSE and Linux

    Figures 10 and 11 shows the performance of SCTP in OSE and Linux for both sender and receiver sides.

    25

  • Bitrate (Mbit/s)

    CP

    U lo

    ad(%

    )

    SCTP senderpacket size = 100 byte OSEpacket size = 100 byte LINUX

    Figure 10. The CPU load of SCTP sender

    Bitrate (Mbit/s)

    CP

    U lo

    ad(%

    )

    SCTP receiverpacket size=100 byte OSEpacket size=100 byte LINUX

    Figure 11. The CPU load of SCTP receiver

    26

  • The above figures for SCTP shows that the CPU load in OSE increases much more quickly than that in Linux.

    8.2 Discussions about the measurement results

    Linux

    According to the algorithm of the CPU load measurement, the bias of the CPU load should be zero when the bit rate is close to zero. It is true in OSE. But in Linux, the bias of the CPU load is around 10% instead of zero.

    It is believed that this is caused by the polling mechanism. In the Linux IP stack, the lower layer checks the upper layer if there is packet needed to be sent (Also, the upper layer checks the lower layer whether there is packet needed to be received) in the polling way. The polling method is efficient for high bit-rate traffic (more frequently packets checking), but not for low bit-rate.

    OSE

    The above comparison results are not that valuable because SCTP in OSE is just a prototype. It is doubted whether the comparison result will be the same if the SCTP is a complete protocol in OSE. So the comparison for TCP in OSE and Linux are also made because TCP in OSE is already a complete protocol. It is believed that differences between TCP and SCTP performance in OSE are not that big regarding the CPU load.

    According to the test results, the comparison results of TCP in OSE and Linux are very similar to figure 10 and figure 11 (The detailed figures about TCP performances are not shown here).

    8.3 Conclusion

    The following conclusions can be drawn: 1. When the bite rate increases, the CPU load of SCTP in OSE increases much

    faster than that in Linux.

    2. When the bit rate is low, SCTP in OSE is better than that in Linux because CPU load in OSE is smaller than that in Linux. When the bit rate is high, SCTP in Linux is better than that in OSE. The cross point of the bit rate depends on the packet size. The bigger the packet size is, the higher the cross point of the bit rate is.

    3. According to the test results, the CPU load of the traffic with small packets changes rapidly as the bit rate changes compared with big packets. This is not shown in the figures above, but it is true according to the measurement data.

    27

  • 9 Analysis

    There are many differences in the software environments between OSE and Linux which result in different behaviors. But it is difficult to analyze and list all the differences of the OSs, stacks and test applications. Therefore only some key aspects are highlighted and analyzed in this section.

    9.1 Buffer Copying

    OSE

    For SCTP and TCP, the protocol stack makes two copies of packets: in the sender side, INETR copies each packet received from INETL into the TIP internal memory. When the packet has been processed by the stack, it is copied into a signal buffer before it is passed on to the driver. In the receiver side, packets are also copied twice in a similar way.

    LINUX:

    The packet is managed with kernel data structure sk_buff, which is a dynamic and flexible structure, and the payload and various headers are added into sk_buff at different time without copying, but when the user data (payload) is copied into sk_buff, a user-to-kernel copy is performed.

    There are two execution modes in Linux: user mode and kernel mode. The user mode provides a context for all user processes. The kernel mode provides a context for the kernel threads (bottom half, task let, softirq, etc) and all kernel functions. The user mode and the kernel mode have different memory spaces. If the task in the kernel space uses the data in the user space, the data must be copied from the user space to the kernel space first.

    The Linux IP stack lies in the Linux kernel. All the stack related data structures and functions are in the kernel space. Therefore, the user space payload must be copied to kernel space sk_buff before the stack can use the payload.

    9.2 Context Switch

    9.2.1 Overview of context switch in OSE and Linux

    Context switching is the switching of CPU from one process to another.

    OSE

    Signal is the tool chosen in OSE on GARP to implement context switch. TEIP, INETR (TIP stack on GARP) and the idle process are the processes handling traffic sending in OSE. TEIP and INETR are running on priority 16 while the idle process is running on priority 31. All three processes are regarded as user processes from the OS kernels point of view. See figure 12.

    28

  • OSE kernel

    TEIP TIP Stack

    Context switch

    Context switch

    Process

    OS Kernel

    Idle Counter

    Figure 12. Context switching in OSE

    Linux

    Iperf is running on behalf of the IP stack and the IP stack (kernel) provides the services to the Iperf (user process).

    There are two modes in Linux: the user mode and the kernel mode (as mentioned in section 9.1). The switch between the two modes is called mode switch instead of context switch. The mode switch is made by the help of system calls. See Figure 13.

    29

  • MontaVista Linux kernel

    Iperf Process

    OS Kernel

    OpenSS7 SCTP and Linux TCP/IP Stack

    System call

    Mode switch

    Idle Counter

    Context Switch

    Figure 11. Context switch in Linux

    9.2.2 The latency of one context switch in OSE and Linux.

    OSE:

    A process, P, with the priority 16 is created to do the context switch with TEIP. The process P sends a signal to TEIP. TEIP receives the signal and sends back another signal to P. In more detail, the sender process P allocates a signal buffer, attaches a U32 number to the signal buffer, and then sends the signal to the receiver process (TEIP). TEIP receives the signal, frees the signal buffer and then works as a sender process (do all the work a sender process needs to do) to send back a signal to the process P. And then the process P will send signal back again to TEIP. The start time and end time are saved in the program so that the total time of context switching can be calculated.

    A new shell command tipContextSwitchTest is added to implement the test. During the test time the processes do nothing than context switch. 10,000 context switches happens without any delay. And the test result is:

    The test time: 252681 us

    The Number of Context Switch: 10,000

    30

  • The mean latency of one Context Switch [test time/the number of Context Switch]: 25 us

    Note that this result is not the pure cost of a context switch. It also includes the cost of allocating signal buffers and freeing signal buffer.

    LINUX:

    LMBench is an open source tool to measure various system latencies (including context switch latency) for UNIX and Linux. It was chosen to measures the latencies of context switching in Linux in the project.

    The latency of a context switch for MontaVista Linux (2.4.20) on GARP is: 1.10 -1.12 us.

    9.3 Chunk bundling for SCTP

    The performance of SCTP bundling for Linux and OSE is tested for three scenarios, which is shown in Table 5 and 6. In the table, the following terms are used:

    Np = Number of packets sent from Application to IP stack.

    Np2 = Number of packets actually sent on Ethernet.

    Average bundling rate = Np/Np2.

    Bit rate

    (Mbit/s)

    Burst size

    Packet size

    (Byte)

    Np Np2 Average bundling rate

    30 20 100 2400000 165000 14.5

    26 5 400 600000 165000 3.6

    20 1 50 2400000 82000 29.3

    Table 5. SCTP bundling performance for Linux

    31

  • Bit rate

    (Mbit/s)

    Burst size

    Packet size

    (Byte)

    Np Np2 Average bundling rate

    30 20 100 2400000 320000 7.5

    26 5 400 600000 396000 2.4

    20 1 50 2400000 200000 11.8

    Table 6. SCTP bundling performance for OSE

    For OSE, there is a bundling timeout in TIP SCTP which is defined as 200ms. TIP SCTP will try to bundle more chunks in a SCTP packet within 200ms as long as not exceeding the MTU.

    For Linux, OpenSS7 SCTP bundles the data chunks when not exceeding MTU and the send queue of socket layer is not empty.

    According to the above tables, Linux bundles more chunks for each SCTP packet (about twice) than OSE does.

    9.4 Other Aspects

    As is mentioned above, there are many differences regarding the software environments for OSE and Linux. It is difficult to find out and analyze all the differences that result in the different behaviors.

    Other aspects that may also influence the results include different implementations of TEIP and Iperf, TIP and OpenSS7, Linux IP stack as well as OS dependent differences like task management and scheduling, memory management, Ethernet driver and so on.

    32

  • 10 Final Conclusion

    The goal of this thesis was to perform a practical comparison of SCTP performance in OSE and Linux. The two test applications, TEIP and Iperf were used in OSE and Linux, respectively. Since TEIP and Iperf use the same algorithm to generate traffics, the measurement results are comparable.

    According to the measurement results, the CPU load of SCTP in OSE increases much faster than in Linux. When the bit rate is low, SCTP in OSE performs better than in Linux. When the bit rate is high, SCTP in Linux performs better than in OSE. The cross point of the bit rate depends on packet size. The bigger the packet size is, the higher the cross point of the bit rate is.

    Further, the traffic scenarios are based on bursts in both OSE and Linux. However, according to the test results, the performance of SCTP has no relation with the burst size. That is, even if the burst size is different, the CPU load of SCTP is the same if the same packet size and bit rate are chosen in the traffic scenarios.

    11 Future studies

    The SCTP implementation in OSE is still a prototype. Functions such as retransmissions, have not been implemented yet. So there is still a lot of work left to make SCTP a complete protocol stack. Further, more smart techniques need to be designed in the implementation of the SCTP protocol to get better performance.

    33

  • REFERENCES

    1. Henrik Karlsson, (2003), General Build Support, Ericsson No. 198 17- CAA 139 1318 Uen

    2. William LeFebvre, Unix Top, http:// www.groupsys.com/top/about.shtml

    3. Michael Biebl, Debian Package: lksctp-tools(1.0.2-1), http://packages.debian.org/testing/net/lksctp-tools

    4. SS7 over IP Signaling Transport & SCTP http://www.iec.org/online/tutorials/ss7_over/index.html

    5. Testing tools http://lksctp.sourceforge.net/testing.html

    6. Fredrick Beste, SS7 Introduction, Ericsson No., UAB/UKY/I-02:108 Uen

    7. Mark Gates, Ajay Tirumaala, Iperf User Docs, http://dast.nlanr.net/Projects/Iperf/iperfdocs_1.7.0.ht

    8. Solaris Performance Monitoring & Tuning iostat, vmstat & netstat http://www.adminschoice.com/docs/iostat_vmstat_netstat.htm

    9. Anders Magnusson, Block TEIP, Ericsson-Internal information

    10. Stefan Mattson, INETR Performance Improvements in MSC R12, Ericsson No. 095/159 41 FCPW 101 97/F Uen

    11. A complete Real-time Operating System Environment for Embedded System Applications, Ericsson No., UAB/D-00:193 Uen

    12. Basic concept of real-time operating systems, http://linuxdevices.com/articles/AT4627965573.ht

    13. Armando L. Caro Jr., SCTP: A New Internet Transport Layer Protocol

    14. RFC 3286 An Introduction to the Stream Control Transmission Protocol(SCTP), http://www.faqs.org/rfcs/rfc3286.html

    15. RFC 2960 Stream Control Transmission Protocol, http://rfc.net/rfc2960.html

    16. Miguel Rio, A Map of the Networking Code in Linux Kernel 2.4.20, http://www.datatag.org

    34

  • 17. Hans Feldt, PBOOT MANAGER HOWTO , Ericsson No., 1/1551 CNZ 212 268 Uen

    18. Jrgen Hansson, Dynamic Real-time Scheduling for OSEdelta, Technical Report No: HS-IDA-TR-94-007

    19. H.Jensen, TIP SCTP Feasibility/Prototype, Ericsson No. LMD/RI-05:006 Uen

    20. Stefan Mattson, BLOCK INET, Ericsson No., 1551-CNZ 224 06 Uen

    21. R.Steward and Q. Xie. Stream Control Transmission Protocol (SCTP): A Reference Guide. Addison Wesley, New York, NY, 2001

    35

  • APPENDIX A Terms and Abbreviations

    ACK Data Acknowledgement

    ECN Explicit Congestion Notification

    GARP Regional Processor

    GPEXR Loadable Operating system on GARP

    ICMP Internet Control Messages Protocol,

    INET The loadable IP stack. Consists of INET

    INETL INET library

    INETR The regional program in INET

    IP Internet Protocol

    ISR Interrupt Service Routine

    MMS Memory Management System

    MTU Maximum Transmission Unit

    OpenSS7 An Open Source Organization provide SS7 protocol stack

    OSE An Real Time OS kernel from ENEA

    PROC A virtual file system for Unix/Linux

    RP Regional Processor

    SCTP Stream Control Transmission Protocol

    SLAB A memory allocation method that first introduced in Solaris OS in version 2.4

    SCTP Stream Control Transmission Protocol

    SSN Stream Sequence Number

    RTOS Real Time Operating System

    TCP Transmission Control Protocol

    TIP Telebit IP stack

    36

  • TEIP Test application from EAB/UZ/DK for TIP

    TTP TEIP Traffic Process

    TCP Transmission Control Protocol

    UDP User Datagram Protocol

    37

  • APPENDIX B Test Procedure in OSE

    The traffics are tested by using gpexr_CXC146091_R1C01 as OS, INETR_CXC146091_R1C01 as TIP stack and modified TEIP_R1A06 as test application.

    Test: Sending SCTP packet from RP17 to RP15

    Environment: INET running on GARP

    Purpose of Test: Get the CPU load of TIP stack

    Preset variables: The following ttp variables is set on RP15( the receiver)

    tipttpconfig -1 sctp_mode server

    tipttpconfig -1 measurement no_check

    tipttpcreate 192.168.202.152 sync sctp 1

    Calibrate the idle load algorithm:

    tipidleloadtest 5000

    The following ttp variables is set on RP17( the sender):

    tipttpconfig -1 sctp_mode client

    tipttpconfig -1 connect yes

    tipttpconfig -1 connect_ip 192.168.202.152

    tipttpconfig -1 connect_port 1

    Set packet size to be 400:

    tiptrafficmix -1 400

    tipttpcreate 192.168.202.172 sync sctp 2

    tipidleloadtest 5000

    The following command will send 120000 bursts of scpt packets from socket 0 on the local ip address to socket 0 on port 1 on 192.168.202.152 with burst size 5:

    38

  • tiptraffictime 0 0 192.168.202.152 5 120000 X 1

    The following command can be used on both sender and receiver side to show the cpu load for TIP stack including impact of TEIP:

    tipttpprint m

    sender: cpuload: 40.53%

    receiver: cpuload: 52.63%

    The following command can be used on both sender and receiver sides to get the cpu load of TEIP:

    Connect

    set profile proc INETR teip_idle_count teip_sync0 eth2

    enable profile

    display load

    Mean value of CPU load for TEIP(sender): 7%

    Mean value of CPU load for TEIP(receiver): 17%

    The following commands can be used to get the impact of LIB in TEIP:

    Before sending traffic, predefine the memory:

    profile_start 0x21000000 0x21066c00 60 4

    After the the traffic sending , use the following command to see the hits on different memory slice:

    Get the total hits for TEIP including TEIP_IDLE_COUNT.

    profile_print top 0x21000000 0x21066c00

    Total hits of TEIP(sender): 11174

    Total hits of TEIP(receiver): 11420

    Get the total hits for TEIP_IDLE_COUNT:

    profile_print acc 0x2100f8b0 0x2100f954

    Total hits of TEIP_IDLE_COUNT(receiver): 10779

    39

  • Total hits of TEIP_IDLE_COUNT(receiver): 10293

    So the actual hits of TTP (sender) is: 12423-12207=395

    So the actual hits of TTP(receiver) is: 11420-10293=1127

    By the following commands can we get the total hits for INETL:

    profile_print acc 0x2101c2b4 0x2101d227

    profile_print acc 0x2101d228 0x2101ee39

    profile_print acc 0x2101ee40 0x2102134b

    profile_print acc 0x2102134c 0x21021c28

    the total hits of INETL (sender) is : 182

    the total hits of INETL(receiver) is: 102

    the impact of LIB in TEIP(sender): 182/395 =46%

    the impact of LIB in TEIP(receiver): 102/1127=9%

    the Pure CPU load of TIP stack (Sender): 40.53%-(1-46%)*7%=36.75%

    the pure CPU load of TIP stack (receiver): 52.63%-(1- 9%)*17%=37.16%

    40

  • APPENDIX C Implementation of Memory profiler in OSE

    In our tests Memory profile is used to calculate the percentage of Lib(INETL) in TEIP. It is implemented in the following way: 1. With the help of the file TEIPR.abs.map, the detailed information about

    the memory address of each function in TEIP can be fetched. So the memory address of INETL in TEIP and TEIP_IDLE_COUNT (the idle process)can be fetched directly.

    2. Run the command display -l to get the memory address of the process TEIP.

    3. Exactly after sending traffic, run the following command:

    Profile_start 0x21000000 0x2126ec00 60 4

    This command divides the memory address range of TEIP (0x21000000 - 0x2126ec00) into memory slices of 16 byte(2^4=16) and this memory profile will run 60 seconds. Note that the memory profile is predefined to run in 60 seconds because all the traffics is designed to run in 60 seconds. By this command, how many hits before, in and after this address range can be fetched. Thereby can the total hits during the traffic time be fetched, represented as H; Note that if the total hits of the profile is 30000 , the actual hits for the traffic should take half of the hits of the profiles. That is 15000.

    4. After finishing sending traffic, run the command profile_print top

    0x21000000 0x2126ec00. By the help of this command, total hits of TEIP can be fetched, written as Ht ; Note that even if the idle process is a separate process, it is built in TEIP. So, to calculate the hits for the process TEIP, the hits in the memory range of TEIP_IDLE_COUNT(the idle process) need to be substracted. If Hidle is used to represent the hits of TEIP_IDLE_COUNT and Hteip is used to represent the hits of the process TEIP, then Hteip=Ht-Hidle And the CPU load of TEIP should be: teip=Hteip/H;

    5. Run the command profile_print acc 0x2101c2b4 0x21021c28.

    0x2101c2b4- 0x21021c28 is the memory range of INETL. By this command, the total hits in INETL can be fetched, written as Hlib. SO the procentage of INETL in TEIP can be calculated: Plib = Hlib/Hteip

    41

  • APPENDIX D SCTP Measurement Data for Linux

    In this appendix the SCTP measurement data for Linux is presented from both the sender and receiver sides.

    SCTP SENDER

    BURST SIZE = 1

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 15.90 13.97 13.42 13.51 13.04 13.32 13.29 13.26

    10 43.07 27.53 21.87 19.74 18.09 17.75 17.44 16.95

    20 70.75 41.19 30.93 26.44 23.76 22.73 22.13 21.53

    30 94.56 58.54 40.31 33.41 29.82 27.77 26.74 26.33

    50 - 77.12 54.46 44.93 39.54 35.83 34.36 33.95

    60 - 94.55 74.27 54.99 47.58 42.14 40.52 39.43

    BURST SIZE = 5

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 15.36 14.08 13.31 13.11 13.36 12.96 12.92 13.23

    10 37.69 25.24 21.02 18.71 18.05 17.24 17.00 17.22

    20 60.90 38.02 29.08 25.04 23.43 22.12 21.65 21.74

    30 83.07 49.56 37.74 31.56 28.75 27.11 26.42 26.39

    50 - 69.10 53.38 44.10 38.45 35.29 34.41 34.65

    60 - 86.94 66.34 51.37 44.20 40.18 40.65 40.86

    42

  • BURST SIZE = 10

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 14.91 14.05 13.29 13.45 13.34 12.94 13.24 13.31

    10 36.66 24.90 20.85 18.96 18.06 17.35 17.30 17.23

    20 60.16 37.32 28.95 25.13 23.39 22.04 21.94 21.75

    30 81.59 49.59 36.76 31.30 28.79 27.30 26.66 26.39

    50 - 69.38 53.11 43.92 38.63 35.35 35.17 35.57

    60 - 85.32 64.84 51.83 44.49 40.74 41.54 42.09

    BURST SIZE = 20

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 15.24 13.68 13.62 13.09 13.34 13.26 12.87 13.17

    10 36.37 24.56 20.76 18.60 18.04 17.54 16.99 17.21

    20 60.69 36.31 28.84 24.82 23.35 22.05 21.60 21.78

    30 80.55 48.09 36.49 31.06 28.74 26.99 26.31 26.38

    50 - 71.52 52.89 43.49 38.50 35.94 36.09 35.54

    60 - 86.07 68.83 51.71 44.50 41.96 42.60 44.89

    43

  • BURST SIZE = 80

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 15.18 13.99 13.60 13.05 13.29 13.20 13.19 13.17

    10 35.90 24.43 20.62 18.57 17.69 17.49 17.37 17.01

    20 58.80 36.31 28.42 24.70 22.94 22.34 21.92 21.58

    30 83.60 47.17 36.72 30.89 28.40 27.20 26.67 26.35

    50 - 70.30 52.60 43.64 39.55 37.11 36.12 35.48

    60 - 92.87 67.08 52.72 47.24 47.08 45.50 44.76

    SCTP Receiver

    BURST SIZE = 1

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 14.52 14.32 12.32 12.51 13.34 13.12 13.49 13.00

    10 43.43 28.48 20.43 18.02 18.23 17.29 17.99 16.46

    20 69.35 41.18 31.02 26.02 24.71 22.20 22.18 21.28

    30 93.50 57.58 41.22 33.91 29.82 27.43 26.38 26.33

    50 - 75.79 55.82 44.10 39.54 35.32 34.90 33.31

    60 - 94.38 71.90 54.29 47.58 42.93 40.30 39.23

    44

  • BURST SIZE = 5

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 15.54 14.10 13.63 13.37 13.27 13.17 13.16 13.15

    10 39.13 25.29 21.00 17.56 17.38 16.73 16.40 16.22

    20 65.66 37.80 29.49 24.23 21.95 20.67 19.95 19.65

    30 89.45 50.07 37.50 29.31 25.71 24.60 23.32 23.29

    50 - 68.34 53.60 39.69 33.81 31.94 29.70 29.94

    60 - 87.30 66.01 47.44 38.09 36.88 34.04 33.56

    BURST SIZE = 20

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 15.32 14.32 13.63 13.37 13.27 13.11 13.16 13.15

    10 38.23 25.21 21.00 17.43 17.38 16.72 16.40 16.12

    20 65.39 35.32 29.43 24.01 21.32 20.01 19.76 19.05

    30 87.43 50.04 37.50 29.31 25.41 24.90 23.32 23.31

    50 - 68.39 53.60 39.69 33.80 31.94 29.43 28.12

    60 - 87.98 65.34 46.02 36.09 36.43 33.89 33.01

    45

  • APPENDIX E TCP Measurement Data in Linux

    In this appendix the SCTP measurement data for Linux is presented from both the sender and receiver sides.

    TCP Sender

    BURST SIZE = 1

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 15.33 14.23 13.27 13.37 13.23 13.17 13.14 13.11

    10 37.70 24.55 19.05 16.79 15.99 15.38 15.07 14.88

    20 62.77 35.69 26.13 20.88 19.00 17.82 17.23 16.88

    30 94.58 50.04 33.00 25.44 21.94 20.17 19.22 18.72

    50 - 67.41 44.70 33.72 28.21 24.43 23.22 22.26

    60 - 94.58 58.21 40.84 30.74 28.01 26.29 25.51

    BURST SIZE = 5

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 14.96 13.55 13.53 12.94 13.20 12.79 13.09 13.02

    10 31.64 21.37 17.89 16.18 15.59 14.88 14.97 14.81

    20 51.56 29.88 23.25 19.86 18.36 17.10 16.98 16.84

    30 70.36 38.96 28.42 23.32 20.96 19.54 19.14 18.80

    50 - 56.63 38.58 30.45 26.64 23.96 22.79 22.28

    60 - 72.17 49.46 36.18 29.62 27.18 25.93 24.83

    46

  • BURST SIZE = 10

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 14.88 13.52 13.31 12.94 12.83 13.11 12.67 12.96

    10 30.75 21.18 17.64 16.12 15.19 15.18 14.64 14.79

    20 50.05 29.20 22.88 19.45 17.90 17.39 16.67 16.82

    30 69.16 37.78 28.02 23.11 20.66 19.78 19.08 18.83

    50 - 55.24 37.50 30.14 26.40 23.55 22.90 22.83

    60 - 71.35 44.50 33.99 29.25 26.90 25.40 25.32

    BURST SIZE = 20

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 14.40 13.47 13.50 12.95 13.15 13.04 12.98 12.95

    10 30.08 20.81 17.89 16.06 15.53 14.83 14.92 14.41

    20 47.54 29.04 21.64 19.39 18.16 17.08 17.01 16.82

    30 66.02 37.00 27.82 23.19 20.55 19.40 18.77 18.84

    50 - 52.64 37.33 30.07 25.60 24.00 23.07 23.06

    60 - 69.42 44.08 32.87 29.47 27.52 25.81 25.47

    47

  • BURST SIZE = 80

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 14.72 13.75 13.39 13.18 13.04 12.98 12.95 12.91

    10 30.01 21.21 17.58 16.40 15.16 15.12 14.89 14.50

    20 46.98 28.87 22.27 19.63 18.16 17.37 16.64 16.56

    30 65.10 36.85 27.38 23.02 20.76 19.66 18.87 18.63

    50 - 52.12 37.01 29.59 25.96 24.50 23.25 23.05

    60 - 68.65 46.86 33.29 29.93 29.10 28.00 27.37

    TCP Receiver

    BURST SIZE = 1

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 15.65 14.14 13.63 13.37 13.26 13.21 12.10 13.12

    10 38.09 24.34 19.79 17.50 16.37 15.77 15.42 14.13

    20 61.44 35.63 26.53 22.04 19.77 18.52 17.86 17.41

    30 94.59 49.62 33.91 26.91 23.85 21.95 20.47 19.72

    50 - 65.69 46.11 34.93 29.89 25.15 23.99 22.98

    60 - 94.49 60.05 42.45 33.13 29.50 28.16 26.64

    48

  • BURST SIZE = 5

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 14.46 13.32 13.39 12.92 13.77 12.32 13.09 13.20

    10 32.83 21.37 18.44 16.20 15.63 14.88 14.43 14.39

    20 51.32 29.02 23.25 19.86 18.36 17.48 16.00 16.84

    30 71.10 39.30 28.42 23.32 20.75 19.74 19.14 18.80

    50 - 56.03 38.58 30.45 26.59 23.36 22.30 22.43

    60 - 72.57 49.69 36.76 29.60 26.42 25.33 23.28

    BURST SIZE = 20

    Bit Rate Packet Size(Bytes)

    10 25 50 100 200 400 800 1600

    1 14.12 13.37 13.32 12.92 12.73 13.23 12.09 12.57

    10 30.04 21.84 18.93 16.19 15.19 15.83 14.64 14.80

    20 51.09 29.95 22.06 19.40 17.80 17.39 16.94 16.45

    30 69.39 38.39 28.02 23.12 20.66 18.72 18.34 18.75

    50 - 55.00 37.34 30.03 26.40 23.59 22.90 22.94

    60 - 72.78 44.43 34.89 29.26 25.99 24.82 25.01

    49

  • TRITA-CSC-E 2006:056 ISRN-KTH/CSC/E--06/056--SE

    ISSN-1653-5715

    www.kth.se

    report_kth_part1.pdfreport_kth_part2.pdf 1 Introduction1.1 Overview of the thesis1.2 Problem1.3 Assigner1.4 Assignment and objectives1.5 Tasks1.6 Limitations

    2 SCTP2.1 Introduction of SCTP2.2 Message Format2.3 Multihoming2.4 Multistreaming2.5 Conclusion of SCTP and TCP

    3 Operating System3.1 General3.2 Processes3.3 Scheduling3.4 Memory Management3.5 Inter-process Communication (IPC)3.6 Process Synchronization3.7 Profiler

    4 IP Stack4.1 TIP stack4.2 Linux IP stack4.2.1 Linux native TCP/IP Stack4.2.2 OpenSS7 SCTP

    5 Test Application 5.1 General 5.1.1 TEIP5.1.2 Iperf

    5.2 Traffic sending format5.3 The CPU load of test applications

    6 Hardware Environment7 CPU Load Measurement Algorithms8 Measurement Results 8.1 Measurement results of SCTP in OSE and Linux8.2 Discussions about the measurement results8.3 Conclusion

    9 Analysis9.1 Buffer Copying9.2 Context Switch9.2.1 Overview of context switch in OSE and Linux 9.2.2 The latency of one context switch in OSE and Linux.

    9.3 Chunk bundling for SCTP9.4 Other Aspects

    10 Final Conclusion11 Future studiesREFERENCES1. Henrik Karlsson, (2003), General Build Support, Ericsson No. 198 17- CAA 139 1318 Uen2. William LeFebvre, Unix Top, http:// www.groupsys.com/top/about.shtml3. Michael Biebl, Debian Package: lksctp-tools(1.0.2-1), http://packages.debian.org/testing/net/lksctp-tools4. SS7 over IP Signaling Transport & SCTP http://www.iec.org/online/tutorials/ss7_over/index.html5. Testing tools http://lksctp.sourceforge.net/testing.html6. Fredrick Beste, SS7 Introduction, Ericsson No., UAB/UKY/I-02:108 Uen7. Mark Gates, Ajay Tirumaala, Iperf User Docs, http://dast.nlanr.net/Projects/Iperf/iperfdocs_1.7.0.ht8. Solaris Performance Monitoring & Tuning iostat, vmstat & netstat http://www.adminschoice.com/docs/iostat_vmstat_netstat.htm9. Anders Magnusson, Block TEIP, Ericsson-Internal information10. Stefan Mattson, INETR Performance Improvements in MSC R12, Ericsson No. 095/159 41 FCPW 101 97/F Uen11. A complete Real-time Operating System Environment for Embedded System Applications, Ericsson No., UAB/D-00:193 Uen12. Basic concept of real-time operating systems, http://linuxdevices.com/articles/AT4627965573.ht13. Armando L. Caro Jr., SCTP: A New Internet Transport Layer Protocol14. RFC 3286 An Introduction to the Stream Control Transmission Protocol(SCTP), http://www.faqs.org/rfcs/rfc3286.html15. RFC 2960 Stream Control Transmission Protocol, http://rfc.net/rfc2960.html 16. Miguel Rio, A Map of the Networking Code in Linux Kernel 2.4.20, http://www.datatag.org17. Hans Feldt, PBOOT MANAGER HOWTO , Ericsson No., 1/1551 CNZ 212268 Uen18. Jrgen Hansson, Dynamic Real-time Scheduling for OSEdelta, Technical Report No: HS-IDA-TR-94-00719. H.Jensen, TIP SCTP Feasibility/Prototype, Ericsson No. LMD/RI-05:006 Uen20. Stefan Mattson, BLOCK INET, Ericsson No., 1551-CNZ 224 06 Uen21. R.Steward and Q. Xie. Stream Control Transmission Protocol (SCTP): A Reference Guide. Addison Wesley, New York, NY, 2001

    APPENDIX A Terms and Abbreviations APPENDIX B Test Procedure in OSE APPENDIX C Implementation of Memory profiler in OSE APPENDIX D SCTP Measurement Data for Linux APPENDIX E TCP Measurement Data in Linux