Technical Sciences Brassersplein 2 2612 CT Delft P.O. Box 5050 2600 GB Delft The Netherlands

www.tno.nl

T +31 88 866 70 00 F +31 88 866 70 57 infodesk@tno.nl

TNO report

Enabling Technology Program Adaptive Multi Sensor Networks (ETP AMSN)

Date 13 March 2014

Author(s) Huib Pasman, Joris Sijs, Peter Laloli

Copy no No. of copies Number of pages 46 (incl. appendices) Number of appendices

0

Sponsor Peter Werkhoven Project name Adaptive Multi Sensor Networks Project number

All rights reserved. No part of this publication may be reproduced and/or published by print, photoprint, microfilm or any other means without the previous written consent of TNO.

In case this report was drafted on instructions, the rights and obligations of contracting parties are subject to either the General Terms and Conditions for commissions to TNO, or the relevant agreement concluded between the contracting parties. Submitting the report for inspection to parties who have a direct interest is permitted.

© 2013 TNO

TNO report | | 1.0 2 / 46

Management summary

Titel : Enabling Technology Program Adaptive Multi Sensor Networks (ETP AMSN)

Auteur(s) : Peter Laloli Datum : 2 February 2014 Adaptive Multi Sensor Networks (AMSN) is TNOs Enabling Technology Program (ETP) directed towards knowledge and technology concerning new generations of sensor networks. The primary goal of AMSN is to build new knowledge eligible to define universal engineering methods and tools. These methods and tools will be used to develop a new generation of sensor network systems capable of on demand delivery of information to meet the needs of many users (multi-use) at present as well as in the future. This generation of sensor networks will be capable to adapt their performance in real-time to the new emerging information needs of the users. In 2013 a total of 16 projects have been executed around four lines of crucial system properties. These properties: time critical performance, adaptive, multi-use, and scalable play a role in all application areas. Following the first two years, 2013 proved to be an inspiring and successful year for the AMSN program. The scientific output nearly doubled in 2013 compared to 2012 bringing the AMSN program on par with international benchmarks. The program extended the IP position of TNO, e.g. by adding 4 patents. In addition, AMSN has enlarged its network with new universities, research organisations and companies, while strengthening existing relations. In the public domain the potential value of AMSN technology was shown by means of two AMSN demonstration projects. Technological breakthroughs such as guided wave crack sensors, new methods for acoustic emission, and state of the art crack acceleration and retardation models have been shown to society in the Structural Integrity Monitoring demonstration project on the van Brienenoord bridge and subsequent press coverage. The largest societal impact was reached by the cooperative driving project. Newly developed technologies such as distributed traffic control system, real-time state-of-the-art wireless communications, control mechanisms to handle (merging) manoeuvres in mixed traffic, lateral vehicle controller and fail safety mechanisms enabled the successful demonstration on the A10 highway to a large audience, including the minister of infrastructure. This demonstration was covered extensively in Dutch media (TV, radio, newspapers and internet). Parallel to the technology research planned in the last year of the program, a stronger focus will be put on effective transfer of the knowledge to enable successful application of AMSN outcome in society and industry. The results so far have shown that the achieved breakthroughs will contribute to solutions and applications essential for a better society. In 2014 AMSN must find partners in industry who are willing to commission (shared) research programs with TNO. In parallel we will build consortia to participate in the Horizon 2020 program to strengthen the development of more fundamental knowledge and technology.

TNO report | | 1.0 3 / 46

Contents

Management summary ..................................................................................................... 2

1 Introduction .......................................................................................................... 4

2 Program Execution 2013 ..................................................................................... 6

2.1 AMSN core projects of 2013 .................................................................................. 6

2.2 Other projects ......................................................................................................... 7

2.3 Cooperations by AMSN projects ............................................................................ 8

2.4 AMSNs connections ............................................................................................... 9

3 Program Results 2013........................................................................................ 11

3.1 Goals for 2013 ...................................................................................................... 11

3.2 Achieved results on program level ....................................................................... 11

3.3 Highlights: Technological breakthroughs ............................................................. 13

3.4 Highlights: Societal impact ................................................................................... 16

3.5 Results per project ............................................................................................... 17

3.6 Patents ................................................................................................................. 35

3.7 ISN conference 2013............................................................................................ 35

3.8 Publications .......................................................................................................... 36

4 Management Opinion ......................................................................................... 42

4.1 Highlights .............................................................................................................. 42

4.2 Strategic Advisory Board mid-term review ........................................................... 42

4.3 Improvements to be made in 2014 ....................................................................... 44

4.4 Conclusions .......................................................................................................... 44

5 Signature ............................................................................................................. 46

TNO report | | 1.0 4 / 46

1 Introduction

The six ‘Enabling Technology Programs’ (ETPs) target generic knowledge and technologies which are expected to contribute to the leading issues in the seven TNO thematic areas. To ensure development of unique knowledge and technology each ETP pro-actively connects to relevant leading centres of academic and commercial research. The research questions are driven from insights in leading developments in the relevant international R&D networks. Adaptive Multi Sensor Networks (AMSN) is the ETP directed towards knowledge and technology concerning new generations of sensor networks. It is expected that the number of sensors will continue to grow exponentially. Billions of sensors will be embedded in all parts of the living environment. As part of the public infrastructure as well as part of every individual’s home, vehicles and devices providing data for all sorts of applications to support our daily life e.g. energy supply, security, traffic control, etc. Internet technology enables the interconnection of sensors in networked systems to provide various users access to the data from almost any place, any time. The AMSN research roadmap is based on the vision that technology is rapidly moving towards full scalable networks of intelligent sensors, consisting of very simple passive sensors (e.g. temperature) as well as very complex active sensors (e.g. phased array radar). The intelligent sensors individually are capable of delivering data in response to human and machine requests one-on-one. As a part of the network they contribute to complex information needs in cooperation with other distributed sensor systems. The primary goal of AMSN is to build new knowledge eligible to define universal engineering methods and tools. These methods and tools will be used to develop a new generation of sensor network systems capable of on demand delivery of information to meet the needs of many users (multi-use) at present as well as in the future. For instance in AMSN, we aim at enabling multi-layer greenhouses, thus producing factors more on the same area (Theme Built Environment), which can only be done when the climate can be controlled per layer. Obviously this requires state of the art sensor networks. Another example is the work on Structural Integrity Monitoring (Themes Built Environment, Industrial Innovation and Energy), where sensor networks are designed and tested that can monitor the state of steel constructions (bridges, pipelines as well as off shore constructions) This generation of sensor networks will be capable to adapt their performance in real-time to the new emerging information needs of the users. From the insights AMSN gains, methodologies and tools are derived to conceive future large scale complex sensor networks delivering the right information at the right time in the right place to a variety of users, humans as well as machines. Four lines of crucial system properties have been identified to develop the core line of thought for the AMSN breakthroughs. These properties play a role, to a greater or lesser extent, in all application areas: • time critical performance • multi-use • adaptive • large scale, scalable

TNO report | | 1.0 5 / 46

The outcome of the research is enhanced with practical experiments in “live” settings to investigate specific aspects of the application of sensor networks. These experiments demonstrate the value of developed AMSN technology. From the demonstration stage the research outcome can be transferred to implementation projects. This report presents the key results achieved in 2013 the third year of the AMSN program as well as a selection of highlights. In accordance with the request of the ministry of Economic Affairs, TNO will publish titles of the involved projects and results obtained so far on its website in due course. The purpose of this report is to give to interested readers an overview of the activities in ETP AMSN.

TNO report | | 1.0 6 / 46

2 Program Execution 2013

The core of the AMSN program consisted of 9 projects in 2013. These are depicted in the figure below, followed by a short description. In case of the demonstration projects it is also indicated which of the TNO themes are actively involved in guiding the project.

2.1 AMSN core projects of 2013

Fundamental Research projects • System Design: Project name: Dynamic AMSN Architectures (DYNAA) Description: Develop system engineering method plus tooling for robust and adaptive sensor systems. • Self Organisation: Project name: Self Organising X (SOX) Description: studying run time adaptivity of the system as a whole, including dynamic resources, aiming at maximising of overall system effectiveness • Data Analysis: Project name: Big Data analysis of Video and other Sensors (Big Davids) Description: innovative applications of affordable large scale (near) real-time big-data analysis technology of video and other sensor streams in various domains • Control: Project name: InControl Description: applying novel control solutions in networked systems across various application domains, e.g. 3D climate control

TNO report | | 1.0 7 / 46

Demonstration projects: • Time critical performance: Project name: Supported Cooperative Adaptive Driving (SCAD) Description: realize a cooperative application facilitating smart merging of vehicles onto the highway. Theme: Transport & Mobility Project name: Intelligent Energy Network Aggregator Technology (ieNAT) Description: build a common base framework, existing of a set of state estimation algorithms , topology processing and data handling and expand on one use case (sensor placement). Theme: Energy • Adaptive:/ Multi-use: Project name: Open Sensor lab: Gorillas in the Cloud (GITC) sensor lab Description: develop a multi-use sensor network ecosystem that is open to other developers and suitable for end-users with very different requirements Theme: Information Society Project name: Sensordata Search: Google for Sensors (Goose) Description: capability to search semantically for any relevant information within “all” sensor streams, in near real time, in the entire internet of sensors. Theme: Defence, Safety and Security and Information Society • Adaptive:/ Scalable: Project name: Structural Integrity Monitoring (SIM) Description: Demonstrate the monitoring of fatigue in steel on real life infrastructure (bridge). Thus demonstrating scalability by adding advanced automatic reconfiguration capability to the sensor network and to demonstrate scalability in dealing with high data volumes and high processing requirements. Theme: Built Environment, Industrial Innovation and Energy

2.2 Other projects

Besides these 9 core AMSN projects, the program also participates in several projects that are part of EU or national programs. These are listed below. AMSN in National/EU projects (FP7, ARTEMIS, etc.) • Protectrail

provide a viable integrated set of railway security solution • EMC2

enabling European manufacturing industries to overachieve Europe 2020 program targets through development of a breakthrough paradigm for cost-effective, highly productive, energy-efficient and sustainable production systems

• Flexigas Develops components for the biogas chain in order to produce, transport and use in the most efficient manner.

TNO report | | 1.0 8 / 46

• Dike Data Service Center (DDSC) Anomaly detection R&D for future use in the Dike Data Service Center.

• DEMANES The main goal of DEMANES is to provide component-based methods, framework and tools for development of runtime adaptive systems, making them capable of reacting to changes in themselves, in their environment (battery state, availability and throughput of the network connection, availability of external services, etc.) and in user needs (requirements).

Finally, the AMSN program has 3 activities that are harder to depict in the project portfolio. These are the program management, links to academia and short feasibility studies. Outreach & Management • Program Management and Outreach: program management, contact and setting up

new collaborations with external parties • Knowledge Team: PhD student projects and link to academia via part time

professorships and STW projects. The PhD students as well as professors involved are also part of the project teams of the 9 core projects, ensuring optimal interaction.

• Sparks: validation of a limited number of new high gain, high risk ideas (e.g. autonomous ships, sensor localisation). To investigate the potential of the idea, and determine whether a new project is needed.

2.3 Cooperations by AMSN projects

Innovation requires cooperation, bringing together several experts and problem owners. Within the AMSN program the demonstration projects benefit from the results of the fundamental projects. Whereas the fundamental projects receive guidance on their research from the applications. Besides cooperation between projects within the AMSN program, several projects are linked to projects in other ETPs, EU or national programs.

2.3.1 Within the AMSN program DYNAA - Big Davids - SIM By using the DYNAA tooling DYNAA delivers sensor network designs for SIM (both laboratory and demonstration set-up at Brienenoord bridge), Big Davids develops anomaly detection and visualization on data resulting from monitoring. SCAD - InControl - SOX InControl develops roadside control of cooperative vehicles to let un equipped cars merge into a platoon based on camera observation SOX develops methods to communicate only when necessary in order to prevent bandwidth scarcity. The SCAD project implements both technologies into the cooperative driving demonstration. GITC sensor lab - Goose - Big Davids Big Davids develops annotation methods for use on video data from the GITC sensor lab project. These annotation methods are also used in the Goose project to facilitate semantic searching.

TNO report | | 1.0 9 / 46

2.3.2 With other ETPs SIM – Faimos (ETP models) - BLSD (VP infrastructure) The SIM project created both a test set up in the laboratory as well as a real life set-up underneath the Brienenoord bridge. The data that this monitoring generates is used to fine tune the models developed in the ETP Models project Faimos. Using the monitoring data and model outcomes the BLSD project (ETP models and VP Infrastructure of Theme Built Environment) creates decision support tooling. The SIM project tunes its experiments to the requirements of the other two projects.

2.3.3 With external/EU projects DYNAA + SOX - Demanes DYNAA and SOX work closely together with the Demanes (Artemis) project. Demanes uses the results of both project to complete its tool chain. Demanes will also add functionality to the tools created by the DYNAA project, so that these become even more valuable. Gorillas in the Cloud (GITC) - iCore The GITC field lab builds further on the results of the iCore FP7 project. The GITC field lab has become the first live implementation of the technology developed in iCore, for multi use of (virtualized, platform independent) hardware and software resources, aiming at the later wider spread controlled and non-conflicting use of the GITC fieldlab hardware and software infrastructure, observation data and meta data. Findings from the GITC implementation are fed back to the iCore project. InControl - EMC2 InControl results on Model Predictive Control for use in climate control for greenhouses are used in EMC2, (which is about creating green factories). The InControl result has been used successfully in a demonstration of Emc2 in a German factory. Big Davids - DDSC The Dike Data Service Center project exchanges anomaly detection techniques with the Big Davids project. Extending the range of data sets that can be processed (e.g. short cyclic, long cyclic, trend)

2.4 AMSNs connections

2.4.1 Universities The contacts between AMSN and universities are warm, and with some of them there are strong ties (i.e. MSc projects, PhD positions, professorships) with others it is related to a specific project. • University of Amsterdam, joint PhD position, part-time professor • University of Twente, joint PhD position, part-time professor • University of Eindhoven, several Master students, joint PhD positions • University of Delft, several Master students, joint PhD positions • University of Nijmegen, joint PhD positions, part time professor • Strathclyde (UK), joint PhD position • Karlsruhe Institute of Technology (D), part time assistant professor

TNO report | | 1.0 10 / 46

2.4.2 Research organisations The list of research organisations that AMSN is in contact with to explore possible projects is far larger than the list below, in this list only the ones we actively cooperate with are mentioned. • Deltares (NL, TO2 institute) • Marin (NL, TO2 institute) • NLR (NL, TO2 institute) • STW (Technology Foundation, PhD positions as well as related projects) • Fraunhofer Gesellschaft (D) • Max Planck Institute (D) • DARPA (USA)

TNO report | | 1.0 11 / 46

3 Program Results 2013

This chapter focusses on the results achieved in 2013, first on program level, secondly per project more detailed results are given. For both the demonstrator projects as the more fundamental projects.

3.1 Goals for 2013

the AMSN program focused on the next steps in the AMSN roadmap • Multi-use (one user uses several sensor networks simultaneously in order to satisfy

his information needs) • Time Critical (enlarging the system, while remaining or even improving the level of

time critical performance) • Adaptivity (adapting to changes in the tasks/goals of the system) • Scalability (moving to scalable data processing, in order to turn data into information). Upon advice of the Strategic Advisory Board Adaptive is to be seen as an overall property of which Multi use and Scalability are a part.

3.2 Achieved results on program level

The high level goals of the AMSN program are to make breakthroughs on four desired system properties that are essential in order to broaden the application possibilities for sensor networks. These four properties contribute largely to going from ‘design-time’ to ‘run-time’ optimalisation, and shifting from data-driven to adaptive demand driven sensor networks. These four crucial system properties have guided the research agenda for the ETP AMSN. The research challenges addressing these key system properties are expected to lead to technical breakthroughs that unlock the next generation of applications of sensor systems promising innovative applications to support societal and economical advances.

TNO report | | 1.0 12 / 46

The four system properties can be mapped on four technology areas which are embedded in the ETP AMSN program:

System Design, Self Organisation, Data Analysis and Control Design. The fundamental knowledge will be brought to reality in algorithms, methods and tools. For each of these four technology areas, there will be a dedicated project in ETP AMSN, on top of these four projects ETP AMSN build demonstrations, that give a hint of application the application potential. The performance of sensor networks is highly related to the application, whereas it is a huge challenge to have a timely system response within 1 second in one application (e.g. cooperative driving) , in other applications timely means minutes (e.g. climate control) or milliseconds (e.g. mobile communication). Therefore, the four system properties are discussed in relation to the chosen demonstration application.

3.2.1 Time critical performance: The corresponding demonstration project is SCAD. The goal for 2012 was to implement a system capable of going below 0.9 seconds time headway (the gap between two vehicles expressed in time) , while still being save. The total reaction time of the demonstration system (cooperative driving cars in longitudinal movement) as a whole has been reduced further to 0.25s while driving at the speed limit in the Netherlands (130 km/h). In practice, while testing on public roads, a distance between the cars corresponding to 0.5s has been used. In 2013 the total system was expanded drastically, by adding lateral movement (steering) and communication between the vehicles and the infrastructure (e.g. cameras, radar). The goal for 2013 was to keep the time critical performance at the 2012 level even though the system was expanded, which was achieved.

3.2.2 Adaptive In 2012 the goal was to reach “graceful degradation” as well as “easy expansion”, meaning the sensor network can cope with changes in the sensor network (e.g. new sensors, or sensors breaking), but also with changes in the available information (e.g. its reliability, timeliness) Adaptive on system level has been reached in several ways in 2012. Self-configuration of sensor networks has been implemented to accommodate the addition or removal of new sensors, so the sensor network can grow to several times its original size efficiently and without human intervention. And moreover, self-organization has been achieved in distributed sensor networks i.e. the system decides itself to change means or frequency of communication and/or accuracy of the algorithm. The demonstration project adhering to this property Bright Light (adaptive Lighting) was handed over to the TNO Shared Research Program “Snellius”, and no separate demonstration project remained for this property. No action was taken to find another demonstration since the Strategic Advisory board found multi use and scalability to be sub-properties of adaptive. Nevertheless, the fundamental projects of AMSN resulted in achieving the goal of 2013: adapting to changing tasks of the system: This was done by developing distributed solutions for decision making. This resulted in the capability of dealing with various & varying user (driver) needs, changing goals, variable communication resources, processing power and sensing capability.

TNO report | | 1.0 13 / 46

3.2.3 Multi use: The goal related to this aspect was to demonstrate that several complex sensor networks can be used by a user. Two projects attempted to demonstrate this in 2013: the Goose project and the Gorillas in the Cloud sensor lab Gorillas in the Cloud (GITC) sensor lab In 2012 the goal was to demonstrate that 1 complex sensor network can be used by at least 3 completely different user groups. This was materialized by the development of a multi-use sensor network in the Apenheul. Due to technical challenges we were not successful in extending the multi-use sensor network in the Apenheul with different sensors. The focus shifted to achieving robust detection and tracking with cameras only before adding additional types of sensors. Goose The Goose project successfully demonstrated the capability to enable trained users to search in high volumes of sensor data from different video sensors. To truly make this available to any user and any sensor in real time, substantial additional efforts are needed. However, this constitutes a significant first step.

3.2.4 Scalability The goal set on this aspect was to be able to design a network for a laboratory set-up, and then expand the design to a real life situation without any changes to the design. The demonstration for this aspect is the structural integrity monitoring of large steel structures (i.e. bridges), so the design needs to be expandable to more than 1000 m2. Self-configuration and self organisation contribute to scalability of the sensor network, in addition tooling has been developed to evaluate the efficiency of the network design. So on the network level scalability can be achieved with the results of 2012. In 2013 this was be put to the test in practice since TNO moved from a laboratory set-up of 8 m2 to an actual bridge, being the van Brienenoord of several 10.000 m2. Terabytes of data per day were retrieved. This brings us to the other big step for 2013 was scalable data processing, to retrieve the data from sensor networks and transform it into information. Several data-analysis technologies have been developed (e.g. sensor clustering, anomaly detection), all aiming to reduce the amount of data and extract information. The technologies developed in 2013 have proven to be successful, the challenge that remains open for 2014 is to analyse in real time and to find generic methods.

3.3 Highlights: Technological breakthroughs

All projects have been working on challenging technology quests. Each of which delivered new insights, knowledge and sometimes practical implementations of advancements in the AMSN research areas. From all projects a selection of the 2013 results has been made is presented here to illustrate the significance of the AMSN program outcome. First the technology is mentioned (where patents were filed a more general description is given), followed by the application and societal relevance.

3.3.1 Time critical performance: Cooperative Driving: Supported Cooperative Adaptive Driving (SCAD)

• Cooperative driving (system property: Time Critical, Adaptive) The research delivered unique methods to integrate in-car and roadside sensor data while maintaining the required level of safety. To reach this breakthrough the SCAD project had to develop and combine the following knowledge and technology:

TNO report | | 1.0 14 / 46

1. Distributed traffic control system, focusing on on-ramp merging as a use case,

employing real-time state-of-the-art wireless communications between vehicles (V2V) and between vehicles and infrastructure (I2V).

2. Control mechanisms to handle (merging) manoeuvres in mixed traffic, using roadside infrastructure to extend the horizon of intelligent vehicles. Instructed by the road side, the platoon vehicles automatically increase inter vehicle distance and decrease velocity to allow non-equipped vehicles to merge.

3. Lateral vehicle controller, thus realizing a fully automated vehicle, capable of automatically following the preceding vehicle, or, alternatively, of automatic lane keeping.

4. Robust fail safety mechanism that now smoothly adapts the desired time headway, which depends on the actual quality of the local dynamic map, based on the Multiple-Model Adaptive Control with Mixing method. As a consequence, the adaptation happens smooth and comfortable, while safety is guaranteed. In addition, a graceful degradation mechanism for lateral control has been developed, which automatically switches between vehicle following and lane keeping, depending on the quality of the lane detection, the presence of a preceding vehicle, and the quality of the preceding vehicle detection.

This frontier technology enables real-time multi sensor integration for cooperative driving systems to optimize utilization of public roads and to enhance driving comfort. The SCAD project team successfully demonstrated the novel Cooperative Adaptive Cruise Control (CACC) concepts in a live driving event including advanced merging scenarios. • Observed pattern based calibration (system property: Adaptive) A novel method has been developed which is able to create an extrinsic and continuous calibration for a camera with a) unknown position, b) unknown orientation and c) unknown zoom factor, without any human interaction, by using the patterns of the observed entities. This enables a very easy and fast set up of large networks of public cameras for intelligent monitoring (e.g. of traffic). This auto-calibration system was successfully implemented for a PTZ dome camera at the A270 test site. When an operator changes the viewing direction, the orientation and/or the zoom level of the camera, the system automatically performs a new calibration. After the calibration is completed, the camera can be added in the global vehicle tracking system, to provide world-coordinate positions and speeds of observed vehicles in real-time.

3.3.2 Control: inControl • Computation time breakthrough (system property: Time-critical) Novel implementation techniques were developed for a seamless integration of Model Based Control and CFD-calculus (Computer Fluid Dynamics), enabling real-time control of CFD-processes; The concept has been successfully tested in a practical large indoor set-up, leading to two patent applications in 2013. It significantly improves accuracy and latency in climate control in large buildings (e.g. greenhouses or factories)

TNO report | | 1.0 15 / 46

• Platoon-wide control solution (system property: Scalable) The InControl project contributed to demonstration of cooperative driving (SCAD project above) since it developed and implemented a Model Based Controller for controlling the platoon during a merging manoeuvre. Both personal comfort and traffic throughput were improved by this new control solution, which acts on the platoon as a whole rather than its individual vehicles.

3.3.3 Scalable: Structural Integrity Monitoring (SIM) • Novel sensor for measuring growth of crack size in steel (system property: - ) Based on guided wave technology, i.e. a sensor principle which measures the integrity by sending and receiving sound waves through steel, TNO has developed the world’s first crack size sensor capable of detecting cracks up to 50% of the thickness of the steel structure with a length over 5 cm. This unique sensor technology has been filed for a patent application. Although this does not contribute to the crucial system properties, it is an essential development for the purpose of the demonstration. The sensor technology is a breakthrough for intelligent monitoring systems and opens up new possibilities for novel risk based maintenance scheduling, enabling owners of steel structures (bridges, pipelines, windmills etc.) to completely monitor the operational condition of their assets. • Quasi Beam Forming (QBF) method for Acoustic Emission (system property:- ) The new Quasi Beam Forming (QBF) method leads to a doubling of the range, and more importantly enables to distinguish between the waves resulting from crack activity and those resulting from traffic and expansion joint. This novel method has been filed as a fourth patent application in the AMSN program in 2013. • Combining models and sensornetworks Together with ETP models and VP Infrastructure we have a demonstration on the van Brienenoord bridge. The demonstrator features a scalable sensor network designed with the help of DYNAA tooling. This network includes the Acoustic Emission sensors and 3 other sensor types i.e. strain, vibration, temperature, resulting in real-time detection of crack growth in a real bridge deck. This is then combined with the state of the art crack growth and probabilistic models developed in ETP Models. This combination is a breakthrough in steel bridge monitoring. Allowing condition based maintenance of steel assets enabled by accurate monitoring of crack growth in the bridge deck and location of the crack.

3.3.4 Self Organisation: Self Organising X (SOX) • Runtime reconfiguration tool (system property: Adaptive) An interactive Matlab-environment was developed for tracking individual objects in a network of 9 subsystems. The user is able to re-assign local algorithms running at the different individual subsystems, while the system as a while remained operationally stable. This is fundamental technology that could find its way in to various applications (mobility, smart energy grids) in the long run.

TNO report | | 1.0 16 / 46

3.4 Highlights: Societal impact

In 2013 AMSN projects reached the level where the public could see the impact. Two AMSN demonstration projects were covered in the national press. At prime time (8 o`clock NOS news, RTL news etc.) on November 12th an item on Cooperative Driving was shown in which the minister of Infrastructure and Environment Mrs. M.H. Schulz van Haegen joined one of our TNO colleagues for a test drive in a CACC enabled car on a public road in Amsterdam. This was during The “Innovatie Estafette”. Mrs. M.H. Schulz van Haegen sat in one of the vehicles and could experience automated driving.

In 2013 Rijkswaterstaat agreed to having a test facility on the largest steel bridge in the Netherlands. Together with ETP models and VP Infrastructure a demonstration on the van Brienenoord bridge has been created. During the year TNO monitored the condition of this bridge. A press release featuring this experiment to monitor the condition of the van Brienenoord Steel Bridge was picked up by several newspapers amongst others de Telegraaf, and AD and was addressed in radio news items. The ISN conference also demonstrated the international interest in AMSN technology. The program was highly international, both in terms of speakers as well as visitors. The number of visitors doubled in respect to 2012.

TNO report | | 1.0 17 / 46

3.5 Results per project

In 0 all AMSN projects are listed. Per project the project description and project goals are followed by the results in knowledge and technology development in 2013. This will show that the results can be used for a wide range of applications. Fundamental Research projects

3.5.1 System Design: Dynamic AMSN Architectures (DYNAA) Objectives: The following figure gives an overview of the DynAA framework and different DynAA tool(s):

x y

KPI valuesmodel

Optimizer

model

construction

DynAA

simulation

KPI evaluation

The DynAA Model Construction and DynAA Simulation tools were already developed during the first phase of the project. The main target for 2013 was to transform the existing DynAA design evaluation (simulation) tool to an automatic system design optimizer tool – research, design and implement the remaining components of the optimization loop. DynAA aims to develop a system evaluation and design method for robust and adaptive sensor systems. The method should be implemented in a software tool, which allows system designer to quantify the emerging key performance indicators of the various design alternatives and for selecting design scenarios to carry out automatic design optimizations (e.g. optimal topology choice, protocol parameterization, optimal task allocation). The KPI evaluation is indicated in the figure above, these are often generic indicators for distributed sensor systems, like battery life, processor load, network load, but possibly also case specific performance indicators as defined and modelled by the system designer. In 2013, the focus was on parametric design optimization. This means that the designer provides a parameterised set of architectures, and the design tool should produce the optimal parameter settings, combined with the achieved values of the different emerging system characteristics. The secondary goal was to improve / extend the previously developed network models (the Model Construction tool in the above figure). An important element in distributed system performance, and especially an important source of performance variations, is the performance of the network connecting the individual nodes. In 2012 we have only modelled the individual links of the network. The goal for 2013 was to extend our models to include 1) the MAC layer, i.e. effects of the shared usage of the medium, 2) the

TNO report | | 1.0 18 / 46

network layer, and especially the effects of using multiple wireless hops, and dynamic routing effects. Focus in the research was on the protocols implemented in commonly used sensor network nodes. The main challenge was to arrive at sufficiently low-complexity models to allow efficient simulation and optimization in DynAA Knowledge and technology development In the course of 2013 a large step forward has been made in extending our knowledge in the following areas: • Automatic design space exploration and design optimization methodologies of the

adaptive reconfigurable networked embedded systems. A new architecture of the DynAA optimization framework was researched, designed, implemented and evaluated. In this architecture, the DynAA consists of several tools as presented in the figure above. The designer has to provide a system model description and specifications on the optimization problem (which concerns the parameter space of the model, functions that constraint the parameters and the objective function that is to be optimized). At each loop iteration, DynAA is fed with models (experiments) that are constructed based on the model description, but with different parameterizations, according to the settings provided by the optimizer tool. The DynAA simulation in turn provides the target Key Performance Indicators (KPIs), which are fed back in the optimizer. Based on the results, the optimizer proposes a new model parameterization and advances the space exploration.

• Implementation strategy for the DynAA communication models. A new communication model, covering the physical, MAC and network layers, was researched, designed, implemented and evaluated. The resulting model addressed the following essential goals: It preserved the fidelity of chosen communication mechanisms, maintained the dynamics of individual communicating nodes in the DynAA context, and simultaneously managed to keep the computation complexity low.

Plans 2014 The DynAA project will be continued in 2014 and will follow the originally defined goals and strategy. Specifically, the research and development targets for 2014 are twofold:

1. Extend the optimization capabilities of the tool completed in 2013. This accounts for joint system architecture and communication network optimization.

2. Increase the usability, accessibility and maturity of the tool, so that it can be used after the project / program end.

3.5.2 Self Organisation: Self Organising X (SOX) Objectives 2013 Self-organization and self-optimization play fundamental role in large-scale monitoring and control applications. Without these properties the resulting systems are either over-dimensioned, expensive and power hungry or they remain fragile (and thus unsafe or expensive to operate). The terms are used to denote a desired system property also known as (autonomous) reconfiguration: the system is not entirely specified a priori to its deployment but some design aspects are organized during operation by the system itself. The main purpose of such a property is to increase the efficiency of system resources for meeting the desired system goals. In addition, reconfiguration will improve robustness of the system with respect to operational system changes (internal) as well as environmental changes (external). For example, to enable a system where adding and removing subsystem components can be done without re-programming the existing set-up (scalable), to support systems that consist of various mobile subsystems

TNO report | | 1.0 19 / 46

observing particular areas (dynamic sensor management), to cope with environmental changes that effect the communication resource (changing network capacities), to allow a wide variety of system goals that can be adapted during operation depending on current needs and the monitored situation (multi-purpose). SOX covers distributed solutions suitable for autonomous reconfiguration of large-scale networked systems, see also Figure 1. Such networked systems are often affected by (unforeseen) operational events that should be accounted in runtime, so to maintain a certain level of system performance. Furthermore, these networks are desired to have scalable and ad-hoc properties, which implies that the core-functionality is subject to the current communication and computational resources. Hence, these resources will have a dynamic availability.

Knowledge and technology development WP1: Distributed solutions for decision making (core & shell) After extensive literature search we defined a terminology that could be used across different research domains. We developed a generic model for self-managing systems and formulated a mathematical framework according to which the cost effectiveness of the (cognitive) system can be quantified. In this we adopted the dec-POMDP terminology as most suitable for the managing (shell) capabilities of the cognitive functions. It is the intention to present a paper on this at Fusion 2014. Besides DEMANES and ACCUS also defence related projects like NECSAVE, STARS, the MPEC initiative and future programs on reduced manning will benefit from this. WP2: Distributed solutions for optimization (core & shell) The project contributed to our understanding of the optimization challenges in runtime reconfiguration, provided insight into the state of the art of this field and hands-on experience on implementation of resource aware optimization algorithms. The project is a “stepping stone” to addressing real-life optimization problems. The DEMANES and ACCUS (EU Artemis project) directly benefits from the results of SOX. Via these projects the results will be visible for commercial parties and implementations for the pilots will be carried out. WP3: Implementation of reconfigurable state-estimators A Graphical User Interface (GUI) was made to demonstrate the benefits of reconfiguration software when operational events are present in the system. The GUI addresses an object tracking case with a system consisting of 9 nodes. Both the user and/or the reconfiguration software can make changes in some aspect of the node: which estimation algorithm is used, what information is exchanged with neighbours, the local sampling frequency, etc. The GUI should give the user an understanding of the SOX-idea. WP4: Dynamic behavior (interdependent shells) We have obtained new knowledge on dynamic behaviour of interdependent (self-organizing) subsystems. In particular, insight in conditions for overall system stability and

TNO report | | 1.0 20 / 46

parameters that influence settling time for system reconfiguration. This helps us e.g. to make (in the design phase for a particular case) an appropriate trade-off between centralized and distributed control, and provides requirements for the solutions for decision making and optimization (WP1/WP2). Plans 2014 Building further on the previous results of the SOX project, the SOX project in 2014 is concerned with methods for designing runtime adaptive systems. This means that not every aspect is decided upon prior to the deployment of the system but that some aspect are left open so that the system itself makes a decision suitable to the situation at hand. In order to make such decisions, an architecture has been developed where design models are used as an exploration space by a combination of decision making and optimization algorithms. A literature study on these two latter algorithms was performed in Sox 2013 and assessed in some basic case studies of runtime adaptive systems. These implementations are based on heuristic solutions and expert rules, which means that a sound development of solutions making runtime design decisions is still open. There are three reasons for developing such sound solutions when comparing them to the current heuristic ones: • The performance of the ‘reasonar’ will be improved; • The predictability of the ‘reasonar’ will be improved; • The set of considered run-time actions will be extended These aspects are desired, either for a designer or for a customer/user, when deploying runtime adaptive systems in the real world.

3.5.3 Data Analysis: Big Data analysis of Video and other Sensors (Big Davids) Objectives Goal of this project is to become one of the important players in the world of innovative applications of affordable large scale (near) real-time big-data analysis technology of video and other sensor streams in various domains.

Knowledge and technology development Current state The current state in sensor analysis is that most algorithms are implemented as offline batch processes. More complex analysis are sometimes highly optimized for multi cores and/or GPUs. No scalable cloud based solutions exist which can host many algorithms, analysing many sensor streams. Big Davids in 2013 In 2013 work is done on several building blocks for data analysis: • Adaptive Autonomous Anomaly Detection: Matlab-based anomaly detection algorithm

and visualisation for advanced crack detection in steel structures using strain gauge sensor data (TRL 5 in 2013)

• SensorClustering: Clustering technology based on dendograms to get insight in multisensor behaviour. Extended with a continuous algorithm which is implemented as a changeability sensor to indicate cluster-changes over time, and a prototype implementation to visualise these cluster changes.(TRL 4 in 2013)

• Realtime Sensordata Classification: development of an efficient and automated classification and structuring tool which uses affective labelling by crowd-sourcing as input (TRL 4 in 2013)

TNO report | | 1.0 21 / 46

• Distributed big data processing: Proof of principle based on Storm to process in realtime video and 1-d sensor streams. (TRL 4 in 2013)

• Big data visualisation: Component for generating space-efficient pixel-based visualisations of (raw) big sensor data (TRL 5 in 2013).

Plans 2014 In the next picture the work packages and their relations are depicted:

3.5.4 Control: InControl Objectives Distributed Model Predictive Control of highly complex dynamics: • Can evolutionary optimisation algorithms be distributed? These are algorithms such

as ant colony optimization and genetic optimization, that can cope with non-linearity and discontinuity and therefore can be used to optimize complex dynamic systems. This will be applied in the context of a greenhouse.

• Can we move from Computational Fluid Dynamics (CFD) simulation to a CFD optimisation? CFD offers the possibility tp accurately calculate many systems (thermodynamics, construction, sound, etc…) by calculating coupled differential equations in iterations to a convergence.

Model Predictive Control in an uncertain environment (time critical MPC): • Can we adjust the aggressiveness of the controller based on the uncertainty in the

system? Whenever a controller is uncertain of the system it is controlling, it is wise to be more conservative. If a controller uses a state-estimator, we obtain information on the uncertainty in the system. Can the state estimator contribute to adding adaptivity in the controller? This will be applied in the context of cooperative driving.

TNO report | | 1.0 22 / 46

Knowledge and technology development Feasibility study for Lattice-Boltzmann based control concluded, which resulted in taking a process model with Lattice-Boltzmann from literature. For this model a MPC-strategy was developed that controls the process. A MPC strategy was developed for the control of a simulated car that determines the aggressiveness level online, depending on the accuracy of the sensor information. Also for this controller it was determined when the closed-loop behavior is stable. We made a Matlab model that describes the dynamic behaviour of a platoon of CACC (Cooperative Adaptive Cruise Control)-cars, including communication delay between cars and between car and roadside. Implementation of the control algorithm and communication was realised. A method was developed for controlling the indoor climate based on the Partial Differential Equations (PDE) that describe the dynamic behaviour of the climate. A state estimator was designed that incorporates the Navier-Stokes equations (PDE) explicitly. Plans 2014 Research questions for real-time climate monitoring & control Optimization of very complex dynamics: • Can we expand the optimization algorithm that was developed in 2013 to a truly

holistic controller? The algorithm in 2013 is able to quickly and precisely optimize thermal sources to attain thermal set points, while taking flow into consideration. It can simultaneously control fans to attain flow velocity set points (in a certain direction), while taking into account thermal buoyancy. However, it is still impossible to use e.g. fans to help attain thermal set points.

• Can we make use of and/or compare with recent developments in optimization of coupled partial differential equations, known in a small academic society of mathematicians? A number of mathematically complex papers seem to provide a means of holistic control. We need to 1) understand and be able to apply these techniques to attain holistic control if our own methods cannot be extended and 2) compare results for publication and patent application in case we do succeed in expanding our own method.

Monitoring of very complex dynamics • Can we assimilate a CFD model with measurements in a linearly scalable fashion?

We have to face the fact that CFD models are poorly scalable, though we do not have an alternative. Hence, any further computations of algorithms using CFD models, such as estimation and control, should not further exasperate that fact. In particular, a promising estimation algorithm was devised in 2013, showing good results, yet scaling very poorly.

Modeling of very complex dynamics • Can we couple our algorithms with the core of the open source CFD software suite

OpenFoam? With such a coupling, we gain an powerful means of modeling, opening up myriad domains (all kinds of fluid dynamics, electro-magnetism, chemical reactions, acoustics) for us to monitor and control.

Research questions for event-based networked control with packet dropout Event based sampling for packet dropout: • Can we develop a new event sampling strategy that will assist the controller to cope

with packet dropout? The sensor exchanges new measurement samples at the

TNO report | | 1.0 23 / 46

instance of a predefined event. In case the receiver (estimator/controller) does not receive a new measurement value it can either mean two things: 1. There was no event and thus no new measurement sample was to be exchanged or 2. There was an event but the packet containing the measurement value was lost. The idea is to add contextual variables to the exchanged measurement value so that the receiver will have a better understanding on whether packages were dropped

Estimation and control solutions for event based sampling • Can we develop a stable feedback control loop suffering from packet dropout at the

sensor with an event sampling strategy? Stable feedback control loops can be designed for any event sampling strategy when they assume that not receiving new measurement samples still implies valuable information on the sensor value. Yet, this assumption is only valid when packages are not lost. Therefore, estimation and control solutions suitable for event sampling strategies should be extended so to cope with package loss

Event based CACC • Can the current CACC technology safely be extended with an event sampling

strategy? The existing CACC technology developed by TNO Automotive assumes that vehicles communicate with each other at 10 Hz. While this requirement is still feasible for the first CACC vehicles driving on our highways and in cities, it is already known that serious problems arise when our highway and cities are covered by many CACC vehicles. Inevitably, the enormous amount of data exchange through the air will cause high probabilities of package dropout resulting in a degradation of the CACC performance, throughput and worse, safety on the highway. The developed event sampling strategy is able to relax this requirement, reducing the probability of package dropout and thus keeping performance and throughput at desirable levels. However, this does require event based estimation & control solutions that can cope with packet dropout in a safe manner. The goal is to develop such solutions in the previous and to implement this solution here

Exploratory research questions Relating to real-time climate monitoring & control • Can Lattice-Boltzmann be a suitable alternative for CFD and can we develop better

monitoring and control algorithms based on it? The Lattice-Boltzmann method is an upcoming technique for solving problems in fluid dynamics (and other domains). It has certain appealing properties, the most relevant of which being an innate suitability for parallelization and distribution, hence solving the scalable issues currently faced with CFD-based algorithms. Depending on the results yet to come in 2013, continuation along this path might proceed.

• Can we use quadrotors as mobile sensors to measure temperature and air flow velocity? Having flying sensors to measure temperature is probably achievable. It might also be possible to relate the actuation needed to keep a quadrotor stationary at some location, to the airflow that the quadrotor is experiencing there as a disturbance to counteract. This might be a very innovative way of measuring low velocity air flow, a problem for which currently no cost-effective solution exists. In cooperation with TU Delft, which is building an in-house test site for quadrotors.

Relating to estimation and control • Can the emergent behavior of complex systems betterbe predicted when using bond-

graph models? The models used nowadays in estimation and control theory are causal, i.e., they are defined with an input and an output representing the idea that cause and effect can be separated. Yet, other representations are at the edge of emerging into this research area [4]. These representations, called ‘a-causal bond

TNO report | | 1.0 24 / 46

graphs’ and/or ‘behaviors’, describe the relations between different entities. Such a modeling approach could be a promising alternative for modeling complex systems consisting of many interactive subsystems, since cause and effect in these type of systems is difficult to identify due to the many interaction.

Demonstration projects:

3.5.5 Time critical performance: Cooperative Driving: Supported Cooperative Adaptive Driving (SCAD)

Technological Goal: Creating situation awareness in rapidly changing circumstances, focusing on time critical performance. This means sharing relevant information in real-time between cooperative systems and real-time fusion of available information based on requirements of the control-unit

Societal Goal: Implementing cooperative driving will result in better and safer traffic flow both in urban areas and on motorways. The effects are: reduction of traffic jams, reduction of the number of casualties in traffic, and lowering CO2 emissions. Objective The objective of SCAD is to realize a cooperative application facilitating smart merging of vehicles onto the highway. Based on the availability of roadside sensors or the availability of cooperative vehicles, smart merging application has different realizations (varying from informing via variable message signs (VMS), to fully autonomous merging). Using earlier results (being ETSI Geonet wireless communications implementation, in-vehicle object tracking algorithms, and longitudinal and lateral vehicle control algorithms and scalable RSU architecture including a standalone reference system) it is now time to integrate the results in order to arrive at adaptive, robust realization of smart merging, which demonstrates the huge potential of cooperative ITS systems in a deployable manner. The following concrete research questions need to be investigated:

1. How to improve the accuracy with which other objects (traffic participants) are currently being tracked? More specifically, the topic of data association needs to be investigated in order to fuse inaccurate GPS information, wirelessly

TNO report | | 1.0 25 / 46

communicated information and on-board sensor information (amongst which radar and camera).

2. How to efficiently use digital maps as a beyond-line-of-sight “sensor” for situation awareness and what are the minimum requirements for such a system in terms of attributes? More specifically, the application of digital maps to create a dynamic region of interest (DROI) filter and the integration with the object tracking/sensor fusion needs to be investigated.

3. How to design controllers for specific use cases, focusing on full automation (automatic vehicle following) that fulfill the control objectives and, at the same time, allow for graceful degradation?

4. Which IT architecture assures an open, extendible realization of roadside part of SCAD?

5. How can different sensor inputs be fused into a roadside world map, taking into consideration the limitations of the individual sensor inputs in accuracy, timeliness and quality?

6. How can a best-effort dynamic map be realized, that is able to deal with sparse sensor distributions?

7. What data sharing facilities are required for communication between RISs? 8. How to best represent world knowledge at different levels for RIS applications

(e.g. microscopic, traffic flow level, meso and macro levels)? 9. Which system architecture needs to be applied in case of an integrated roadside-

vehicle ITS system, taking into account requirements regarding flexibility, fault-tolerance, reliability, efficiency, and effectiveness?

Knowledge and technology development

Use cases and functional architecture The use case and

architecture development do not intend to generate new knowledge as such, but enable a common understanding and consistent, future provide framework for the development of the individual components into a robust, integrated cooperative system. The most important contribution of the architecture is a common language and common understanding of the problem and solutions. The use case development has provided insight in different solutions in solving the merging problems of cooperative and non-cooperative solutions: what solutions are realistic, and in what situation can they be applied. The combination of people with a more practical and more theoretical background, people from with a background in ICT, in sensor processing, and in control, and people from a vehicle and roadside perspective has proven very valuable to find the best solutions. Digital maps Knowledge and experience on reliable matching is gained. Lane level positioning is desired for identifying correct advises, and to determine whether the vehicle is currently driving on the merging lane, the right most lane or the left lane. Camera data with lane information can be applied to enable lane level positioning.

On-board System

Adaptive DataExchange

Robust & AdaptiveObject Tracking

Robust & Fail SafeControl Design

TNO report | | 1.0 26 / 46

Selecting whether detected objects (by radar/camera/wifi) are in the region of interest is important for the CACC algorithm. A dynamic region of interest can be constructed from map data, however the accuracy of both map data and the vehicle position and heading is not of sufficient quality to reliable perform the selection at longer distances from the vehicle. Commercially available map data is of better quality and will be applied in future research. In vehicle object tracking For the host tracking a more sophisticated method is used to estimate the host position. In the process of applying an extended Kalman filter, more insight was gained in the inaccuracies and delays of the (U-blox) GPS. Furthermore, knowledge on the accuracies of the other sensors is developed and a flexible logic-based approach is applied to the estimation of the states. In-vehicle trajectory control

1. Knowledge and experience has been gained on how to actively control “nodes” in a communication-based networked control system, operating under strict constraints with respect to real-time behavior. This knowledge can be relatively easy transferred to traffic control at large industrial plants such as the automatic-guided vehicle system on the European Container terminal, or to other transport modes (trains, metro trains, ships, busses).

2. Knowledge developed regarding the application of soft-real-time road side information to support real-time safety-critical maneuvers such as gap creation and merging.

3. In general, the step has been made from “low-level” vehicle control (CACC, platooning) to the higher maneuvering level, thus entering the domain of agent-based interaction protocols.

4. The possibility for fully automated vehicle-following at high velocity has been actively explored, which will be very useful in related projects such as Virtual Tow Bar.

Automatic road side sensor calibration We found that it is feasible to automatically and robustly create an extrinsic calibration for an unknown camera within 30 seconds to 3 minutes, using only the patterns of the moving vehicles, without any human aid. To achieve this, first a new object tracker was created that does not require world-coordinates but works solely in image-coordinates using the overlap between predicted and found bounding boxes of vehicle detections in the consecutive camera frames. Secondly, an vehicle-pattern auto-correlation algorithm was created that finds corresponding vehicle passing patterns in the ‘to-be-calibrated’ camera and the other cameras. Finally, an detection matching algorithm was created and applied to obtain corresponding image-coordinates in the ‘to-be-calibrated’ camera and world coordinates of other cameras, which are needed for the final extrinsic calibration. All algorithms were implemented in a real-time scenario at the A270 test site. Adaptive information generation The algorithms to extract traffic flow parameters from the VBM (Video Based Monitoring) trajectories have been improved. VBM only includes (on purpose) very limited amount of domain specific knowledge to allow it to be used as advanced sensor system. As an example, no assumptions are made about (maximum) speeds of vehicles are the (dis)

TNO report | | 1.0 27 / 46

appearance of vehicles. The DYNAMAP has been extended to handle this imperfections by applying advance vehicle and traffic models, allow to handle temporary missed vehicles (e.g. Due to errors in a single camera). In this way, reliable traffic flow data can be calculated. A vehicle emulator has been developed that mimics the communication behaviour of cooperative vehicles for any subset of vehicles on the test site. For SCAD the vehicle emulator is only used to emulate non cooperative vehicles on the merging lane. In other contexts, e.g. In (commercial) experiments for single OEMs, the vehicle emulator is a valuable asset to test the performance co-operative systems for larger penetration rates without the need of large quantities of cooperative vehicles. Of course, emulation only allows non-cooperative vehicles to behave as cooperative vehicles with respect to sensing, not to actuation as the emulated communication is only one way. Algorithms have been developed in the InControl project and are described there. From the implementation efforts within SCAD it has become clear that the step from simulation to operational system for such complex cooperative control algorithms is far from easy. The effect of an UBR (Utility Based Reasonar) in a concrete case has been investigated. It became clear that in the use case at hand, an implementation would not yet be beneficial and has therefore not been executed. A still open question is how to combine an UBR (on the information layer) with mechanisms on the communication layer as currently foreseen in the ETSI ITS standardisation. Plans 2014 2014 will fully focus on the following topics: • Object state estimation: Improvement of the object state estimation by sharing not

only ego positions of vehicles, but instead of this entire world maps. Information of the roadside is actively used for this. As a result, the object state becomes more complete, and allows more advanced automated maneuvers.

• Combined longitudinal and lateral control. Instead of car following, it will be possible to follow entire trajectories autonomously.

• Higher level hierarchical control, focusing on configuration of platoons, as well as managing different sub-stages of complex maneuvers (e.g. coordinating the sub-stages platooning, splitting, gap-creating, merging).

• Efficient information exchange, reducing the required communication bandwidth at the communication level as well as the information level.

3.5.6 Smart Grids: Intelligent Energy Network Aggregator Technology (ieNAT) Many monitoring systems (e.g. SCADA including state estimation algorithms) are on the market. But few applications actually use the information generated by the existing monitoring systems. The ieNAT project defined several use cases , that could benefit from using current information. • Overloading & Congestion • Phase Imbalance Detection • Sensor Placement Optimization • Fault Localization • Demand Side Management All of these use cases require a common base framework, existing of a set of state estimation algorithms , topology processing and data handling. The objective of ieNAT 2013 was to build this framework and expand on one of these use cases. As a use case sensor placement was chosen.

TNO report | | 1.0 28 / 46

Results 2013 Sensor-placement tooling was developed. This is a technological innovation that can be applied outside the electricity domain as well. E.g. optimal placement of routers on a communications network or determining optimal locations of cameras on a road network . The state estimation algorithms can also be applied to other networks that flow, i.e. gas and oil distribution, or water. The Flexigas project is the likely candidate to benefit from it first.

Plans 2014 The project will not continue. The results of 2013 can be used by other projects.

3.5.7 Multi use: Open Sensorlab: Gorillas in the Cloud (GITC) sensor lab Technological Goal: To develop a multi-use sensor network ecosystem that is open to other developers and suitable for end-users with very different requirements (industry, consumers, education and research) . Societal goal: The GITC sensor lab acts as a testbed for various applications, e.g. patient monitoring (hence the interest of parties active in healthcare) The GITC sensor lab increases entertainment value for zoo visitors by real time gorilla localisation and visualisation to the visitors. Finally it is also used to demonstrate the attractiveness/potential of ICT in order to attract more students to ICT studies.

Results 2013 New knowledge has been developed in the field of reliable visual change detection and tracking of objects (gorillas) in complex observation environments. Complicating factors were (and are) combinations of dynamically changing illumination (light level, shadows, coloring), different structured and changing backgrounds (due to vegetation, wind, sun, moving shadows, seasonal coloring), low contrast situations of the moving object with respect to the background, and lack of object movement complicating the detection.

TNO report | | 1.0 29 / 46

Several detection and filtering methods have been investigated and implemented aiming at minimizing the number of false positives (false detections) and minimizing the number of false negatives (e.g. missed detections). New knowledge has also been developed in the field of multi use of (virtualized, platform independent) hardware and software resources, aiming at the later wider spread controlled and non-conflicting use of the GITC field lab hardware and software infrastructure, observation data and meta data. This knowledge builds further on and will be shared with the iCore FP7 project (related to internet of things). This knowledge will be elaborated upon and actually implemented in the 2014 AMSN GITC project, enabling the use of the GITC field lab by different users, developers, researchers in a controlled way. Plan 2014 In 2012, the basis of the GITC sensor lab in the Apenheul was realized. On top of the systems contributed by the partners (cameras , broadband network , servers and workstations ), TNO implemented processing and architecture as a reference platform . In 2013, TNO developed further on this base and further enhanced features such as detection and tracking with special attention to the reliability and robustness of the gorilla detections to avoid false detections. The objectives for 2014 are

1. To transfer the control of the operational GITC sensor lab at the end of 2014 to Apenheul and partners so that it is self- maintaining and growing . This requires

a. It can carry out reliable and robust detections among all relevant conditions

b. Ease of operation and maintainability by third parties, including achieving targeted documentation, technical transfer, workshops

2. Extend the possibilities of the GITC sensor lab as a research platform for TNO, Apenheul and others for multiple types of uses and users:

a. multi -party / multi use : add management functionality to the field lab in order to allow expansion with new sensors and software / hardware.

b. Face recognition of monkeys (in cooperation with Fraunhofer) can be added to the perception chain as new data extraction function and also serves as an example for other parties to add functionality to the field lab.

3. Testing what is an effective approach for (semi - automatic) annotation of video data that is easier to search for immediate or later use. Together with AMSN Big Davids project This is also of importance in the development of new methods of interpretation , such as for the detection, tracking, classification , identification , behavioral analysis , anomaly detection.

4. Dissemination : Continue making the GITC field lab known in both industry and research institutions as a research platform , in order to establish that (national and international) research is attracted to the field of sensor networks.

3.5.8 Multi use Sensordata Search: Google for Sensors (GOOSE) Technological Goal: The GOOSE (GOOgle for SEnsors) concept has the ambition to provide the capability to search semantically for any relevant information within “all” (including imaging) sensor streams ,in near real time, in the entire internet of sensors. Similar to the capability provided by presently available search engines which enable the retrieval of information on “all” pages on the internet.

TNO report | | 1.0 30 / 46

Societal Goal: Goose will provide the military to consider all sensor sources. This will happen in the intelligence process, where it now is limited in analysis capacity as well as during mission execution where situational awareness of a larger areas will be made available. For example, the system will also made available to the platoon level what is happening after the next corner in the road. Besides this GOOSE connects well to the roadmap Big Data Evolution in which data interpretation and integrity of big data are important research items. Objectives The GOOSE project was to make an initial demonstration system, allowing semantic search on videos. The planned work for 2013 consisted of the following Essential features or primitives, Abstraction model or interpretation layer, Data transformations and primitives search and GOOSE architecture To benchmark TNO’s position on this technology it was decided to do a submission for the MED (Multimedia Event Detection) Task in the TRECVID international benchmark held by the US body NIST (National Institute of Standards and Technology) Knowledge and technology development A TRECVID-MED system has been realised, which allows to recognize videos based on their content as representing a certain event. The system has two different modes: in the first the system learns on example videos and retrieves videos showing similar events, and in the second mode the system has no training data but only a textual event description. In that case the system parses the textual description, matches these to concepts, and uses the evidence found for these concepts to determined event evidence. Major new knowledge and technology acquainted in this process are:

- Knowledge on low-level features: new features are LBP (Local Binary Patterns, a dense texture description); CC-D STIP (a motion CorreCted Dense version of the Spatial Temporal Interest Point motion-in-video feature); MFCC (Mel-Frequency Cepstrum Coefficients, a description of the sound heard) which are used together with previously known SIFT, Opponent-SIFT and STIP features to get a complementary feature set;

- Knowledge to process large video data sets on cluster computing, needed to process the TRECVID data set of total 6000 hours in the limited time available;

- Knowledge how to utilize external cloud services such as Amazon for processing; - Insight in technology to generate and use BoW (Bag of Words) descriptions for

reducing large video data sets into smaller sets which allow fast search for specific events

- Knowledge on spatial tiling and VLAD (Vector of Locally Aggregated Descriptors) to improve retrieval performance on BoW-style video representations

- Knowledge on text interpretation to known concepts, including semantic matching using ontologies

- Insight into classification performance using only text descriptions and pre-computed concept classifiers.

The improvement of the TNO technology position has been large. Yet, an earlier start of the TRECVID work would have let to a more cost effective use of resources available. The insights gained in this process are very relevant for external stakeholders, who in their day-to-day work are limited by analysis capability. The knowledge of current possibilities and performance of processing components can help them to integrate automatic analysis components into their workflow.

TNO report | | 1.0 31 / 46

Plans 2014 For the GOOSE 2014 project the desired situation is to be able to improve the accuracy of the search capability by closing the semantic gap. This Semantic Gap is wide, and for 2014 we propose to work on three pillars which should provide support to bridge this semantic gap without too large stretches: (1) improve semantic analysis by matching ontologies; (2) improve concept detection in sensors and (3) better concept definition using user feedback

3.5.9 Scalable: Structural Integrity Monitoring (SIM) Technological Goal: System adaptivity & scalability; To develop a scalable system that monitors both the conditions that cause degradation and the degradation itself. This requires that the system is capable of localizing degradation areas, ‘hot spots’, where additional sensors can be placed to monitor the extent of the damage (sizing). This requires flexibility and adaptivity to permit up-scaling and sensor integration of the sensor network. Ease of deployment and robustness; Ease of deployment means, that sensors can be easily installed at accessible locations, without special surface preparation. The sensor should provide information about a certain area around the sensor, avoiding installation at a location difficult to access. The concept requires for random (temporarily) placement of the sensors. Societal Goal:

Structural Integrity Monitoring (SIM) systems should provide up-to-date information about the past and current condition of the infrastructure, as well as a reliable prediction of the remaining structural life.

Permanently installed monitoring systems can provide up-to date information at any desired moment without interfering with the operation of the asset . Because permanent sensors can be remotely monitored, the number of expensive inspections on site can be decreased. With an effective use of the monitoring data, users can optimally plan maintenance and prolong the life span. This will limit maintenance costs, failure probability or costly repairs and increases availability.

TNO report | | 1.0 32 / 46

Objectives The goal for 2013 will be addressing the scalability such that the size and complexity of an actual SIM system can be handled. The following objectives are therefore identified: The first objective is to facilitate scalability by adding advanced automatic reconfiguration capability to the sensor network. The second objective is to demonstrate scalability by dealing with high data volumes and high processing requirements. The third objective is to evaluate the sensor network functionality in a real-life environment. Knowledge development is concentrated on obtaining scalability in a sensor network by incorporating automatic reconfiguration. Special attention is given to obtaining optimal (distributed) architectures for demanding applications such as the specific SIM measurement techniques which incorporate high demands on data volume, processing and synchronization. The knowledge development is closely related to development of the DYNAA architecture evaluation toolset and Big Davids. Results 2013

The overall project objective to monitor small and large cracks in large steel structure to ensure safety at all times, prevent unforeseen down-time and minimize maintenance costs is realized by means of a real time & innovative sensor system that:

1) Predicts remaining life time by load measurement (Strain) 2) Is able to detect large cracks or fractures (Vibration) 3) Is able to detect & locate small cracks (Acoustic Emission) 4) Is able to quantify the size of a small crack (guided wave)

1) Prediction of remaining life time; We have developed:

• enhanced model that incorporates crack acceleration and retardation as a result of load sequence effects in combination with real-time crack measurement data

• Probabilistic tools to extrapolate local monitoring results over the entire structure 2) Detect large cracks or fractures (vibration); We created novel algorithms that can be

applied in a layout with wireless sensor nodes that are easy to install and avoid the costs of installing long cables

3) Detect small cracks (Acoustic Emission); We have developed • the quasi-beamforming (QBF) method to accurately localize the crack covering a

TNO report | | 1.0 33 / 46

large area, and estimate the crack growth by decomposing the AE signals. • novel algorithms for the QBF analysis that can be implemented in a wireless

layout with easy-to-install hardware which can cover a large area, radius is currently 4 m (50 m2); 2013; to be enlarged to 6 m (110 m2); 2014

4) Quantify the size of a small crack. We have developed a Proof of Principle of new sensor technology (guided wave); No other technique can monitor the exact size of a small crack covering a relatively large area (10 m2)

Plans 2014 The Structure Integrity Monitoring AMSN demonstrator project has a planned duration of 4 years. It started in 2011 and planned to finish in 2014 by means of successful lab trail tests (SIM breadboard) and demonstration in a real life environment (BLSD project). In order to achieve this goal, several tasks have to be performed which are schematically presented as a flow logic below.

TNO report | | 1.0 34 / 46

In 2011, the unique TNO approach to Structural Integrity Monitoring has been defined. The use of an advanced sensor network architecture is however heavily related to the measurement techniques that are available now and will become available in the future. Therefore assessment of the available sensor- and modelling techniques was necessary. Parallel to the technical work packages attention was given to the business cases related to the application fields: bridges, wind turbines at sea and piping.

In 2012 a lab scale demonstrator (breadboard) was realised which served for testing and validation of several sensor- and modelling techniques. The breadboard also functioned as a means of demonstrating the possibilities of distributed processing in a sensor network architecture to facilitate scalability. In parallel with the breadboard related work, preoperational work has been performed for realizing a future field demonstrator.

TNO report | | 1.0 35 / 46

In 2013 the lab scale demonstrator (breadboard) was used for further testing and validation of sensor- and modelling techniques in combination with variable amplitude loading. In parallel a measurement and validation program has been setup on the van Brienenoord bridge in Rotterdam in the framework of the BLSD project. Besides the technical aspects, considerable effort has also been put in the business case for a steel bridge and sharing the outcome (and its input parameters) with potential users.

In 2014 the project will be organised along the measurement- and modelling techniques as schematically presented above. The main techniques are: remaining life time prediction based on TOFD measurements (inspection data) and the guided wave crack monitor technology, distributed vibration monitoring, acoustic emission monitoring and data presentation. Implementation of these techniques will make use of scalable wireless implementations and adaptive sensor networks. Predominantly tested on the breadboard but also on the van Brienenoord bridge.

3.6 Patents

4 patents/premier depots resulted from AMSN in 2011-2012: 1. Sample Rate Changeability (patent) 2. Safety Checker (premier depot) 3. Smartphone Time of Arrival precision positioning system (premier depot) 4. Automatic video based action classification (premier depot)

In 2013 4 new patent applications have been submitted: 1. Premier depot January 2014 “Navier-Stokes based optimization” 2. Internal depot PLT 2013123 Decentralized and Quantitative Acoustic Emission for

Crack Monitoring 3. Internal depot PLT 2011148 Baseline generation for crack detection 4. Internal depot December 2013, “Navier-Stokes based estimation”

3.7 ISN conference 2013

The results of the AMSN projects have been successfully presented on the fourth annual ISN conference on November 12th , 2013 at the High Tech Campus in Eindhoven. Following the 2012 edition of the event at the Apenheul, this year’s objective was to present the results of AMSN research to a much wider audience reaching beyond the sensor community. The events was promoted internationally to attract potential sensor networks users and researchers either as presenter or as visitor of the conference. To achieve a substantial growth of the footprint of the ISN conference and more important to ensure a sustainable life after 2014 the concept and organization has been outsourced to a commercial event publisher. This led to the fruitful cooperation with Jakajima BV which now has adopted the ISN Conference concept and will continue the series to drive . The continued cooperation with STW NanoNextNL research program has again proven to be valuable in assembling an inspiring presentation track on advanced sensor technologies. As a result a multi-tier program could be presented including international key note speakers and workshops in the field of nano technology and sensors, applications of sensor networks and AMSN systems technology. Many delegates from industry, universities as well as government organizations actively participated the event. With the 2013 edition of the ISN Conference the AMSN research outcome has been bookmarked for further use in a broad range of applications.

TNO report | | 1.0 36 / 46

The plenary session was moderated by Pieter Hermans, Matchmaker for innovators, Jakajima. The Key note of the conference was presented by Prof. dr. Robert Meijer, Sr. Strategist, TNO and University of Amsterdam. He presented his views on the impact of Intelligent Sensor Networks on society. In a forum discussion with leading representatives from academics and industry the essential aspects of ISN systems was addressed. The forum consisted of Olivier Verscheure, Senior Research Manager – Big Data Analytics & Systems, IBM Research Ireland, Prof. Kay Römer, Graz University of Technology, Prof. Dr.-Ing. Uwe Hanebeck, Karlsruhe Institute of Technology (KIT). During the conference a demo market place was set up accessible for the delegates featuring demonstrations from companies and several AMSN projects. In the afternoon a total of eight breakout sessions were organized covering application areas for ISN systems featuring topics like Lighting, Mobility, Science, Agriculture, Infrastructure, Tracking & Tracing and Big Data. A workshop facilitated by Syntens focused on doing business in the Internet of Things. The conference was visited by an all-time high number of delegates. In 2014 the 5th ISN conference is planned on November 4th in Eindhoven.

3.8 Publications

For a program of the size of AMSN between 24 and 40 peer reviewed publications is considered on par in international benchmarks. In 2012, there were 17 accepted publications, in 2013 these numbers almost doubled to 32 publications. Thus AMSN has improved its scientific output, and is performing in correspondence with its benchmarks.

Besides the scientific peer-reviewed papers, the activities in the projects of AMSN have resulted in several other types of publications in 2013. The numbers are indicated in the figure below, followed by an overview per project of all publications.

TNO report | | 1.0 37 / 46

3.8.1 System design: DYNAA: Peer-reviewed, accepted, published full papers: • Relja Djapic, Yohan Toh, Job Oostveen, “Radio channel characterization of a metal

bridge segment at 868MHz and 2.4GHz”, ACM MSWim 2013. • Julio de Oliveira Filho, Zoltan Papp, Relja Djapic and Job Oostveen, “Model-based

design of self-adapting networked signal processing systems”, IEE SASO 2013. (partially SOX related/financed)

Both papers were personally presented during the corresponding international conferences in September and November 2013. Poster, demo stand and presentations • ISN Conference in Eindhoven 12th of November 2013.

3.8.2 Self Organisation: SOX: Peer-reviewed, accepted/published full papers • Harm van Seijen, Shimon Whiteson, Leon Kester, "Efficient Abstraction Selection in

Reinforcement Learning", Computational Intelligence, 2013 Peer-reviewed, accepted, published and presented full conference papers: • A Multi-Objective Approach to Evolving Platooning Strategies in Intelligent

Transportation Systems, Willem van Willigen, Evert Haasdijk, Leon Kester, GECCO 2013, Amsterdam

• Ditzel, M., L. Kester, S. van den Broek and M. van Rijn (2013). Cross-layer Utility-based System Optimization. 16th International Conference on Information Fusion. Istanbul, Turkey

• J. Sijs, B. Noack and U.D. Hanebeck, Event-based state estimation with negative information, In the Proc. of the 16th International Conference on Information Fusion (Fusion ’13), Istanbul, Turkey, 2013.

• C van Leeuwen, J Sijs, Z Papp, A reconfiguration framework for self-organizing distributed state estimators, 16th International Conference on Information Fusion. Istanbul, Turkey

• In preparation: P. Kempker, J.L. van den Berg, L. Kester, K. Kok, The Two-time Scale PowerMatcher Algorithm for Balancing Demand and Supply in Smart Energy Grids. In preparation, December 2013.

TNO report | | 1.0 38 / 46

Presentations by project members • 2 presentations on Fusion 2013 (see papers) • 1 Presentation on GECCO 2013 (see papers)

3.8.3 Data Analysis: Big DAvids submitted full papers • Trichias, K, e.a, Structural Health Monitoring using Anomaly Detection

− Accepted abstract for SPIE Smart Structures/NDE 2014 − To be presented as a full paper at SPIE Smart Structures/NDE, March, 9-13 2014

in San Diego − This paper is a collaboration of Big Davids-project and SIM-project

invited / keynote presentations by program members on topics of the program • ISN 2013 presentation Matthijs Vonder “Large scale sensor data logistics for Smart

Dairy Farming“ − Presented at the ISN 2013 conference 12-11-2013 in Eindhoven - http://www.isnconference.com/assets/Matthijs-Vonder-1-ISN-Large-scale-sensor-data-logistics-in-

SDF-TNO-ISN-Conference-v10.pdf

accepted/published short papers, workshop papers, and posters • Veen, J.S van der, Deployment Strategies for Distributed Applications on Cloud

Computing Infrastructures, IEEE CloudCom 2013 − Accepted paper for poster presentation − Presented 2-12-2013 at CloudCom 2013 in Bristol.

Demonstrations

• Poster presentation and demonstration: Bram van der Waaij: “Getting insight in live sensordata, our clustering approach” − Presented at the ISN 2013 conference 12-11-2013 in Eindhoven

• Poster presentation and demonstration: Erik Boertjes: “Raw sensor data visualisation” − Presented at the ISN 2013 conference 12-11-2013 in Eindhoven

• Poster presentation and demonstration: Kostas Trichias “Autonomous Adaptive Anomaly Detection on Sensor Data” − Presented at the ISN 2013 conference 12-11-2013 in Eindhoven − Co-demonstration with the SIM-project

3.8.4 Control: InControl Peer-reviewed, accepted, published full papers ▬ J. Sijs, U.D. Hanebeck and B. Noack, An Empirical Method to Fuse Partially

Overlapping State Vectors for Distributed State Estimation, In the Proc. of the European Control Conf. 2013, Zurich, Switzerland, 2013

Submitted full papers ▬ “ MPC implementation to a platoon of CACC cars” Joris Sijs, Tijs Donkers, Elham

Semsar Kazerooni; ▬ “Event based state estimation and control strategies”, Joris Sijs, Maurice Heemels and

Mircea Lazar, Int. Journal on Nonlinear Analysis: Hybrid Systems. Presentations ▬ Presentation at the ISN Conference, 12 November 2013, Paul Booij; ▬ Presentations about state estimation and MPC control were given at:

o Wageningen University; o Marin Wageningen;

TNO report | | 1.0 39 / 46

o I-demodagen 2013 in the Demokwekerij Honselersdijk.

3.8.5 Cooperative driving: SCAD Peer-reviewed, accepted, published full papers − J. Ploeg, N. van de Wouw, and H. Nijmeijer, “Lp String Stability of CaSCADed

Systems: Application to Vehicle Platooning,” IEEE Transactions on Control Systems Technology. Accepted.

− J. Ploeg, D.P. Shukla, N. van de Wouw, and H. Nijmeijer, “Controller Synthesis for String Stability of Vehicle Platoons,” IEEE Transactions on Intelligent Transportation Systems. Accepted.

Submitted full papers − S. Öncü, J. Ploeg, N, van de Wouw, and H. Nijmeijer, “Cooperative Adaptive Cruise

Control: Network-aware Analysis of String Stability,” IEEE Transactions on Intelligent Transportation Systems. Submitted.

Peer-reviewed, accepted, published and presented full conference papers − E. van Nunen, J. Ploeg, A. Morales Medina, and H. Nijmeijer, “Fault Tolerancy in

Cooperative Adaptive Cruise Control,” in Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems, The Hague, The Netherlands, October 6–9, 2013, pp. 1184–1189.

− J. Ploeg, E. Semsar-Kazerooni, G. Lijster, N. van de Wouw, and H. Nijmeijer, “Graceful Degradation of CACC Performance Subject to Unreliable Wireless Communication,” in Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems, The Hague, The Netherlands, October 6–9, 2013, pp. 1210–1216.

− E. Semsar-Kazerooni, and J. Ploeg, “Performance Analysis of a Cooperative Adaptive Cruise Controller Subject to Dynamic Time Headway,” in Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems, The Hague, The Netherlands, October 6–9, 2013, pp. 1190–1195.

− Willem van Willigen, Evert Haasdijk, Leon Kester, Fast, Comfortable or Economical: Evolving Platooning Strategies with Many Objectives, ITSC2013

− ITS World Tokyo: “DynaMap: a Dynamic Map for Road side ITS Stations.” Bart Netten, Leon Kester, Harry Wedemeijer, Igor Passchier, Bart Driessen. TS129, 20th ITS World Congress, Tokyo, 2013

Press articles, interviews, and other types of publicity − RTL 19:30 news 12th of November, − NOS 20:00 news 12th of November, − Jeugdjournaal 12th of November, − RTL 7 12th of November − Hart van Nederland 12th of November − http://www.volkskrant.nl/vk/nl/2824/Politiek/article/detail/3543435/2013/11/12/Minister

-Schultz-test-zelfrijdende-auto.dhtml − Nu.nl

http://www.nu.nl/politiek/3626633/schultz-verwacht-binnen-twintig-jaar-zelfrijdende-autos.html

− Telegraaf http://www.telegraaf.nl/digitaal/article22050448.ece

− Euronews http://www.euronews.com/2013/11/11/fast-and-curious-smart-cars-to-reduce-the-dangers-of-driving/

TNO report | | 1.0 40 / 46

3.8.6 Smart grids ieNAT none

3.8.7 Sensordata Search: Goose Peer-reviewed, accepted, published papers − Klamer Schutte, Freek Bomhof, Gertjan Burghouts, Jurriaan van Diggelen, Peter

Hiemstra, Jaap van 't Hof, Wessel Kraaij, Huib Pasman, Arthur Smith, Corne Versloot, Joost de Wit , “GOOSE: semantic search on internet connected sensors”, Proc. SPIE 8758, (2013).

Submitted full papers − Henri Bouma, George Azzopardi, Martijn Spitters, Joost de Wit, Corné Versloot,

Remco van der Zon, Pieter Eendebak, Jan Baan, Johan-Martijn ten Hove, Adam van Eekeren, Frank ter Haar, Richard den Hollander, Jasper van Huis, Maaike de Boer, Gert van Antwerpen, Jeroen Broekhuijsen, Laura Daniele, Paul Brandt, John Schavemaker, Wessel Kraaij, Klamer Schutte, “TNO at TRECVID 2013: Multimedia Event Detection and Instance Search”, TRECVIC (2013) (to be published)

− Over, P., Awad, G., Michel, M., Fiscus, J., Sanders, G., Kraaij, W., Smeaton, A., Queenot, G., “TRECVID 2013 - An overview of the goals, tasks, data, evaluation mechanisms and metrics,” TRECVID, (2013). (to be published)

− Maaike de Boer, Klamer Schutte and Wessel Kraaij, “Event Classification using Concepts”, ICT-OPEN, 27&28 November 2013. (to be published)

3.8.8 Open Sensorlab: GITC Sensor lab invited / keynote presentations by program members on topics of the program − the National Informatics Congress (http://stichting.snic.nl/snic/the-foundation) , − the i3B platform ('ICT for Brain, Body & Behavior' (i3B) (www.i3B.org)) − the SIA RAAK yearly congress (http://www.innovatie-alliantie.nl/english.html)

3.8.9 Structural Integrity Monitoring: SIM Peer-reviewed, accepted, published full papers − Pahlavan, L. e.a., Multidisciplinary Health Monitoring of a steel bridge deck structure,

9th International Workshop on Structural Health Monitoring, Palo Alto, USA, September 2013

− Pijpers, R. e.a, Structural Health Monitoring for fatigue life prediction of orthotropic bridge decks, 3rd Orthotropic bridge conference, Sacramento, USA, June 2013

− Basten, T. e.a., Structural health monitoring with a wireless vibration sensor network, International Conference on Noise and Vibration Engineering, Leuven, Belgium, September 2012

− Meulenhoff, P. e.a., Autonomous Automated Anomaly Detection (A3D) and the application in the SIM experiments. 2013

− Trichias, K., e.a.“Structural Health Monitoring using Anomaly Detection”, submitted abstract for SPIE, 2013

Demonstrations

– ISN event 2013 – Infraquest symposium 2013

Press articles, interviews, and other types of publicity

– http://www.rijnmond.nl/nieuws/06-11-2013/sensoren-strijd-tegen-files-van-brienenoordbrug

TNO report | | 1.0 41 / 46

– http://www.telegraaf.nl/binnenland/22033786/__Brug_vraagt_zelf_om_onderhoud__.html

– http://www.omroepzeeland.nl/inhoud/sensoren-strijd-tegen-files-van-brienenoordbrug

3.8.10 Flexigas Peer-reviewed, accepted, published full papers − Pierie / Broekhuijsen / Vonder – Conference paper for Biomass Conference & Exhibit:

A tool for analysing, researching and modeling energy efficiency, sustainability and flexibility of biogas chains operating as load balancer within decentralized (smart) energy systems.

− Rajagopal / Wijbrandi / Pierie – Conference Paper for World Renewable Energy Technology Congres: Flexible Biogas Chain Simulator

accepted/published short papers, workshop papers, and posters − Pierie / Broekhuijsen / Vonder – Conference poster for Biomass Conference &

Exhibit: Modeling decentralized energy systems

3.8.11 Knowledge team (the PhD positions in AMSN a.o.) Peer-reviewed, accepted, published full papers

• P. Brandt, V. Bui, H. Liu, J. Lukkien, en T. Basten, “Adding semantics to software architectures - The ‘4+2’ View Model”, ETAPS 2014, pp. 1-15.

• P. Brandt en L. Daniele, “Pragmatism versus formalism: the relation between Linked Open Data, semantics and ontologies”, in Pilot Linked Open Data Nederland, nr. 10, GeoNovum, pp. 275-282.

• M. de Boer, P. van Maanen and G. Vreeswijk, “Supporting Intelligence Analysts with a Trust-based Question-Answering System”, In 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT), Atlanta USA, 17-20 Nov 2013, pp 183-186

• Willem van Willigen, Leon Kester, Ellen van Nunen and Evert Haasdijk. Safety in the Face of Uncertainty: Critical Headway Estimation in Cooperative Adaptive Cruise Control. Accepted for the International Journal of ITS Research, Springer-Verlag, Berlin/Heidelberg.

• G.A. Klunder, E. Jonkers, and Z.T. Woldeab. ‚The potential of Connected Cruise Control in the Netherlands’. IEEE ITSC 2013, Scheveningen.

• Gerdien Klunder, Henk Taale, Serge Hoogendoorn. ‘The Impact of Loop Detector Distance and Floating Car Data Penetration Rate on Queue Tail Warning’. MT-ITS Dresden, 2013

• Van Leeuwen, C., Halma, A., & Schutte, K. (2013). Anomalous Human Behavior Detection : An adaptive approach. In the Proc. of SPIE DSS (Vol. 31).

• Willem van Willigen, Evert Haasdijk and Leon Kester (2013). “Evolving Intelligent Vehicle Control using Multi-Objective NEAT”. In Proceedings of the 2013 IEEE Symposium on Computational Intelligence in Vehicles and Transportation Systems, at the 2013 IEEE Symposium Series on Computational Intelligence, SSCI, 2013, Singapore, April 16–19, 2013, Pages 9–15, IEEE Press, Piscataway, NJ.

TNO report | | 1.0 42 / 46

4 Management Opinion

4.1 Highlights

In its third year the outcome of the AMSN projects show the potential impact on the grand challenges of society. Two AMSN projects were covered in the national press. At prime time in the 8 o`clock news on November 12th an item on Cooperative Driving was shown in which the minister of Infrastructure and Environment joined one of our TNO colleagues for a test drive in a CACC enabled car on a public road in Amsterdam A press release featuring the SIM project experiment to monitor the condition of the van Brienenoord Bridge was picked up by several newspapers amongst others de Telegraaf, and het Algemeen Dagblad and was also addressed in radio news items. TNO was given the opportunity to present the AMSN research on the annual National Informatica Congres. More than 300 IT students followed the discussion on the AMSN technology challenges and the projects. The results of the AMSN projects have been successfully presented on the fourth annual ISN conference on November 12th , 2013 at the High Tech Campus in Eindhoven. Following the 2012 edition of the event at the Apenheul, this year’s objective was to present the results of AMSN research to a much wider audience reaching beyond the sensor community. The events was promoted internationally to attract potential sensor networks users and researchers either as presenter or as visitor of the conference. To achieve a substantial growth of the footprint of the ISN conference and more important to ensure a sustainable life after 2014 the concept and organization has been outsourced to a commercial event publisher. This led to the fruitful cooperation with Jakajima BV which now has adopted the ISN Conference concept and will continue the series after the end of the AMSN program. The continued cooperation with STW NanoNextNL research program has again proven to be valuable by adding presentations on advanced sensor technologies. As a result a multi-tier program could be presented including international key note speakers and workshops in the field of nano technology and sensors, applications of sensor networks and systems technology. An all-time high number of delegates from industry, universities as well as government organizations actively participated the event. With the 2013 edition of the ISN Conference the AMSN research outcome has been bookmarked for further use in a broad range of applications.

4.2 Strategic Advisory Board mid-term review

In May 2013 the AMSN Strategic Advisory Board (SAB) performed a one day mid-term review based on the results achieved in the first two years of the program (2011-2012). In addition the SAB reviewed the plans for the remaining years (2013-2014). In a round table session the SAB reflected on the outlook of the AMSN technology. The SAB conclusions and recommendations provided valuable feedback to the AMSN program. The SAB concluded that the basis of the AMSN research program is sound and that there is great potential for a various application areas. The self-evaluation of the AMSN

TNO report | | 1.0 43 / 46

program and the review by the board were aligned, for the present (6/7 on the scale of the knowledge position audit) as well as the future potential (8)1.

The most important improvements that need to be made are:

1. Communication: The board found the communication and presentation of the program insufficient. Breakthroughs should be described in specific terms.

2. Collaboration: Internal collaboration should be improved.

3. Number of projects: The number of projects in the program is large and should be reduced.

Actions that AMSN will take to incorporate these improvements into the program:

1. The AMSN team will improve communication of the results. Clearly stating the breakthroughs/fundamental concepts that are developed in the program, the synergies between projects and motivating what TNO’s unique value is. If needed, coaching/training will be provided.

2. The collaboration/synergy will be improved by a focus on synchronizing activities in the plans for 2014. In the meantime in 2013 additional work items were added on the interfaces between different projects. This will be done on top of existing collaborations and the current activities that are already in place to enhance cooperation.

3. Reducing the number of projects was discussed in the AMSN Steering committee. Substantial reductions have already been made in the beginning of 2013 (nearly 50% reduction in number of projects, from a total of 29 projects in 2012 to 16 projects in 2013). Since it is not possible to reduce the number of projects any further without harming the interests of one or more TNO themes. The steering committee felt that, as soon as collaboration between projects is strengthened, there will be no need to reduce the number of projects below the current number.

The improvements adopted from the Strategic Advisory Board review have already been applied during the remainder of 2013 and will be continued in 2014.

In the opinion of the SAB The AMSN scientific and technological output can be improved. One important indicator is the number of publications and patents resulting from the projects. For a program of the size of AMSN between 24 and 40 publications is considered on par in international benchmarks. In 2012, there were 17 accepted publications and 2 patents, in 2013 these numbers almost doubled to 32 publications and 4 patents. Thus AMSN has improved its scientific output, and is performing in correspondence with its benchmarks. In terms of professorships and PhDs the AMSN program is well represented in academia. Three part time professors are involved. A key-member of the AMSN-program was asked for the position of assistant Professor at the University of Karlsruhe, due to contributions in the area of monitoring and control. Two PhDs have already successfully finished during the course of the program, another three PhD candidates have submitted their theses, and six more PhD candidates are currently doing their research within the program.

1Meaning of the marks used in the Knowledge Position Audit: (8) Strong/dominant: ETP operates internationally. One of the leading programs. (7) Strong: ETP operates internationally. Is not the leading program, but able to set new directions on its own. (6) Favorable/strong: ETP operates predominantly national and is nationally leading. Able to sustain technological competitiveness with peers in other countries

TNO report | | 1.0 44 / 46

In addition a qualitative indication for the level of science and technology of AMSN is the number of EU/Artemis projects in which TNO participates with experts from the AMSN team. Through the AMSN program TNO contributes in 3 recent Artemis projects on topics related to the research areas and where TNO is the science & technology coordinator. Also AMSN scientists are invited in prestigious positions e.g. to participate in the Dagstuhl seminar, or to contribute to EU programs as a EU senior strategist.

4.3 Improvements to be made in 2014

In 2014 the project portfolio of AMSN will be similar to 2013. Since 2014 is the final year, it is essential to shift the focus towards finalizing and demonstrating the results of the program. Thus the fundamental project work is reduced while more effort will be allocated to both applied project work as well as dissemination. Instead of the larger number of relatively small projects which was the situation in 2012, in 2013 larger coherent projects were run. This was mainly done to simplify interaction between projects by reducing the number of interfaces, and it paid off in terms of improved synergy. In 2014 the remaining projects will be linked even stronger than before. Special attention will be given to the few projects that did not benefit fully from each other in 2013 (i.e. Big Davids, Goose and GITC Sensor lab). In 2013 there was quite some interest from external parties (in the scientific community as well as among companies) in the work AMSN is doing. Materializing this interest with industry partners has proven to take more time than anticipated, whereas setting up new cooperation with universities and research institutes was successful on multiple occasions. In 2014 AMSN should be able to commit a few parties in strategic research together with all relevant themes and their business developers. This will lead to building more and stronger relations with research organizations, university and industry that will last beyond the duration of the AMSN program.

4.4 Conclusions

Following the first two years 2013 proved to be an inspiring and successful year for the AMSN program featuring encounters with frontier applications demonstrating the potential of the research outcome both in the industrial as well in the societal domain. The scientific output nearly doubled in 2013 compared to 2012 bringing the AMSN program on par with international science and technology. The program extended the IP position of TNO, including 4 patents, to strengthen TNO’s position in future markets. In the public domain the potential value of AMSN technology was disseminated through TV and Radio channels, through newspaper featured articles on two of the AMSN demonstration projects. Technological breakthroughs such as guided wave crack sensors, new methods for acoustic emission, and state of the art crack acceleration and retardation models have been shown to society in the demonstration project on the van Brienenoord bridge and subsequent press coverage The largest societal impact in 2013 was reached by the Cooperative Driving project. Newly developed technologies such as distributed traffic control system, real-time state-

TNO report | | 1.0 45 / 46

of-the-art wireless communications, control mechanisms to handle (merging) manoeuvres in mixed traffic, lateral vehicle controller and fail safety mechanisms enabled the successful demonstration on the public highway A10 to a large audience, including national press and the minister of infrastructure. This demonstration was covered extensively in all national media (TV, radio, newspapers and internet). Parallel to the technology research planned in the last year of the program, a stronger focus will be put on effective transfer of the knowledge to enable successful application of the AMSN outcome in society and industry. The results so far have shown that the achieved breakthroughs have the potential to open up new solutions and applications essential for a better society. In 2014 AMSN must find partners in industry who are willing to commission (shared) research programs with TNO based on the demonstration projects. In parallel we will build consortia with partners to participate in the European Horizon 2020 program to strengthen the development of more fundamental knowledge and technology.

TNO report | | 1.0 46 / 46

5 Signature

Delft, 13 March 2014 Peter Werkhoven Peter Laloli Managing Director Technical Sciences Author