MaDaM the web-based Measurement Data Management for Big … · jBEAM: an open ASAM-CEA Framework...

MaDaM – the web-based Measurement Data Management

for Big Data

by Dr. Bernhard Sünder, Managing Director, AMS GmbH

17. June 2015 MaDaM – Measurement Data Management for Big Data

AMS GmbH located in Chemnitz (Saxonia)

founded 1993 by Dr. B. Sünder

since 1998 our vision is:

Using Internet Technologies for distributed Work-Flows

in Measurement-Data Postprocessing


Architecture

MDF4

MaDaM

by

jBEAM - We

b Se

rvice

jBEAM

Server

Datenbank

pure HTML-5

jBEAM

Desktop

jBEAM

Client

Optimized traffic with

iPad

PC

Windows, Linux, Mac, iOS, Android

Importer jBEAM

MDF4

xxx


2014: introduced MaDaM™

Using the best technologies for distributed working-flow scenarios

• Server-based Measurement Data Management (MDM-) System with web interface

• Flexible and fast search system

• PDF-report creation and interactive analysis

• Access from desktop and smart devices


– MaDaM™

DIAdem MDF3 MDF4

ATFX

NetCDF MATLAB many more

MaDaM

by jBEAM - W

eb

Service

jBEAM

Server

Lucene

pure HTML-5

Optimized traffic with

Lucene: Indexing Technology No relational database necessary Just a file system storage

HTML5: User interface, available in nearly any browser

PDF: Reports as PDF-files readable in any PDF-reader

jBEAM: Client post processing software for interactive analysis, connected via EnCom

MaDaM General and business logic for organizing test data


Importing of Data

Dedicated importer components

• Specialized for certain data formats, e.g. MDF v3 & v4, ISO13499 MME, Diadem™, Excel, …

• Running on client side

• Based on JavaFX

• With modern user guidance (wizard)

• Guides the editing process and force consistency


Sample-Importer: Passive Safety (Crash)


DIAdem MDF3 MDF4

ATFX

NetCDF MATLAB many others

MaDaM

by

jBEAM - We

b Se

rvice

jBEAM

Server

Lucene

pure HTML-5

jBEAM

Client

iPad PC

Windows, Linux, Mac, iOS, Android

Indexing Technology

No relational database necessary

Just a file system storage

File System


Apache Lucene • Objective: Powerful search functionality

with high performance

• Data-based index

• Project of the Apache Foundation

• Used by: Google, Apple, IBM, LinkedIn, Twitter, Eclipse Foundation, …

• OpenSource


Relational Database

• Problem: Model unnecessarily complex

• Problem: Adaptation to customer needs and new formats difficult

• Problem: Queries are complex & complicated

• Problem: Additional license costs for high-performance database solutions (Oracle, Microsoft SQL or others)


Search for Tests: Web-Frontend

• Web-Frontend based on modern web technologies: – HTML5

– JavaScript

– AJAX

• Requires only a browser on client side

• Device independent


Device Independent • Desktop

– Windows – Linux – Mac OS X – Solaris




• Smart device – iOS




• Smart device – iOS – Android – Windows 8


Search for Measurements

• Full text search

Required time: 118 ms, show 100 of 1000 result(s)

Full text search:



• Combination of search items

Full text search:

Required time: 21 ms, show 100 of 1000 result(s)



• Facet search

• „Show all China NCAP tests which were performed by test lab LaboTron“



• Any combination of facet and full text search

• Search for: • Date of measurement

• Test engineer, Laboratory

• Many more

• Search in measurements • Statistical data of channels are extracted during import



• Search for statistical quantities: Min, Max, …



• High-performance – 1000 measurements in < 20ms

– Scales independent from measurement size

• Flexible and powerful search syntax – Full text search

– Search in channels

Data found, what now?


Reports

• PDF Reports in 3 simple steps

1. Search / Choose

2. Choose template

3. Click button


Reports (on demand)

• Reports are generated on demand on the server using jBEAM

as a web service

• Only a PDF-viewer is required on the client

• PCs or smart devices

MaDaM

by

jBEAM - We

b Se

rvice

jBEAM

Server

Lucene

pure HTML-5

File System


Interactive Analysis

• jBEAM also works as an app on client PC

• No pre-installation on the client: Using Java WebStart technology, jBEAM is deployed over the web on-the-fly

jBEAM

Client

PC

optimized data-transfer with

by

jBEAM - We

b Se

rvice

jBEAM

Server

Lucene

pure HTML-5

MaDaM


Interactive Analysis: jBEAM

jBEAM

DIAdem

Excel

MDF

Crash

NetCDF

ASCII

ATFX

jBEAM Example: Crash Evaluation Based on the MME Standard page 1 of 37

Head Accelerations

Dummy front left, Hybrid III mid-sized adult male 50%

Test

Laboratory No.: 98/7707

Customer No.: M5533

Type: Vehicle into Vehicle frontal 0°

Date: 1998-11-06

Test Object

Name: FIESTA (HTE)

Velocity: 14,5 m/s

Mass: 1.151 kg

Time

s0,00 0,05 0,10 0,15 0,20 0,25

grav

-10

0

10

20

30

40

50

60

70

Acc

eler

atio

n

X

Y

Z

Res

Channels

AXL TETE HVG CFC1000

AXT TETE HVG CFC1000

AXV TETE HVG CFC1000

Resulting acceleration CFC1000

Results

Calculation range: 0 - 250 ms

Max.res.ac: 60,01 grav (76,3

ms)HIC: 534,4 (57,6 ms - 96,8 ms)

HIC 36: 529,0 (60,7 ms - 96,6 ms)

HIC 15: 348,8 (70,6 ms - 85,6 ms)

3 ms cumulative: 58,09 grav

3 ms continuous: 58,29 grav


Optimized traffic using

DIAdem MDF3 MDF4

MME

NetCDF MATLAB many more

jBEAM

Client

iPad PC

jBEAM

Client jBEAM

Client

jBEAM

Client

MaDaM

by

jBEAM - We

b Se

rvice

jBEAM

Server

Lucene

pure HTML-5


Enterprise Communication

jBEAM

Data

jBEAM

Server Client


Enterprise Communication

jBEAM

Data

jBEAM

Server Client What do we have? 1 GB

What was requested? 2 MB

• Only channels actually used are transmitted

• Only value ranges actually needed are transmitted


Enterprise Communication • Only required channels are transmitted • The channel data transmission is optimized

– In less than ½ second, the user sees the complete graphic of a channel with more than 10 Mio. values in jBEAM on client side

– the user can gradually zoom in even to individual values of the signal

In summary: • minimizes network traffic by intelligent

and selective data transfer.


jBEAM: an open ASAM-CEA Framework

Online Measurement

& Control

Test Bench SCADA Operator GUI

ATFX

DIAdem

MDF

UFF

NetCDF

Offline File Import

Automation System

Component & Flow

Management

>100 Calculations

Services

Input Graphic Layout Report

Unit Data Item

EnCom

…

…

FFT Formula User

CEA-Calc …

3rd party MDM

other jBEAM

Data Base


jBEAM: > 50 Data formats are supported ASAM-ODS: ATF and ATFX

Diadem: DAT, TDM & TDMS

MDF: version 3, incl. resampling version 4, ASAM compliant

NetCDF

ASCII: comma, tab, blanks

Excel: binary and CVS

CAN log

Famos

EDAS

ETAS: DCM

LabVIEW

ISO13499: MME, ISO

DBase 3

Dasylab

Google: KML

GPS-exchange: GPX

Garmin Database

AVL: iFile

Q-DAS

Racelogic-Vbox

RPC-III

UFF: 15, 58 and 151

TEAC

TecPlot


jBEAM: > Curves with >20,000,000 values

All > 20,000,000 values in one diagram without delay

Tracking window, Values at cursor,

reference/delta Axis, others …


jBEAM: > Georeferenced Data


jBEAM: Geo & Video & Num. Data

Graph content: 1. Google Maps as background 2. GPS-Track with speed as color 3. Sprite „Car“

Video of Test

Linegraph: Speed vs. Time


jBEAM: Orderanalysis


jBEAM: Sound – FFT, Ampl., Player

Sonogram

RMS-signal display

Audio-player

Synchronized


jBEAM: Testdata in 3D-Space


Summary

Measurement Data Management with MaDaM™:

• Modern web technologies (HTML 5) for simple workflows under PC and smart devices

• Fast and powerful search function

• No license costs for third party database products

• Server based NCAP-report generation using jBEAM

• Client based interactive analysis with jBEAM

• Optimized data transmission with

Bahnhofstraße 6 1760 Opdyke Court German Centre, Unit 719A 09111 Chemnitz Auburn Hills, MI 48326 88 Keyuan Road, Pudong Germany USA Shanghai 201203 / PR China

Tel.: +49 (371) 918 668-0 Tel.: +1 (248) 270-7779 Tel.: +86 (21) 289 866 19 Fax.: +49 (371) 918 668-99 Fax: +1 (248) 393-0340 Fax: +86 (21) 289 865 11 E-Mail: [email protected] E-Mail: [email protected] E-Mail: [email protected] Web: www.AMSonline.de Web: www.AMSonline.eu Web: www.AMSonline.cn

Gesellschaft für angewandte Mess- und Systemtechnik mbH

mailto:[email protected]



http://www.amsonline.de/

http://www.amsonline.eu/

http://www.amsonline.cn/

MaDaM the web-based Measurement Data Management for Big … · jBEAM: an open ASAM-CEA Framework...

Documents

Transcript of MaDaM the web-based Measurement Data Management for Big … · jBEAM: an open ASAM-CEA Framework...