Boost AGL Application Framework performance by 10+

Stéphane DesneuxCTO at IoT.bzh<sdx@iot.bzh>

July 15 – 16, 2020Virtual EventVideo available at:

https://vimeo.com/438457064

July 20Boost AGL AppFW performance by 10+ 2

➢ 28 engineers mostly dedicated to AGL➢ Our goal is to deploy AGL in cars, boats...

● https://iot.bzh/en/● http://github.com/iotbzh

LORIENT

About us

July 20Boost AGL AppFW performance by 10+ 3

Why performance matters ? Case study...

● A boat equipped with 4 drone radars for “boat dock assist”● Serial transmission at 115200 bps per radar● Data are sent by each radar at 50Hz = 200 messages per

second to be emitted by a low-level binding

SP70CRadar The

boat !

July 20Boost AGL AppFW performance by 10+ 4

A JSON event in the real life...

Typical message to emit at 50Hz for each radar:{ "sensorVersion": { "dataType": 1, "result": 1, "master": 2, "second": 3, "step": 10 }, "sensorStatus": { "actl_mode": 3, "rollcount": 1, "cfgStatus": 1 }, "targetStatus": { "noOfTarget": 2, "rollcount": 1 },...

... "targetInfo": [ { "index": 1, "rcs": 4.5, "range": 6.3524, "azimuth": 45.369, "vrel": 0.235, "rollcount": 1, "SNR": -3 }, { "index": 2, "rcs": 9.64, "range": 2.324, "azimuth": 25.23,...

... "vrel": 0.351, "rollcount": 1, "SNR": -5 }, { "index": 3, "rcs": 3.2, "range": 15.987, "azimuth": 65.391, "vrel": 0.357, "rollcount": 1, "SNR": -1 } ]}

July 20Boost AGL AppFW performance by 10+ 5

Resources usage profiling

● Profiling on the low level binding shows that 80% of time is used for creating JSON-C objects

~25 000 events => 1M malloc()

July 20Boost AGL AppFW performance by 10+ 6

Benchmarking serialization formats

● IoT.bzh started investigation on serialization formats:– JSON vs CBOR vs PROTOBUF vs XDR

– Multiple JSON libraries (JSON-C, FASTJSON, STRINGJSON)

– JSON and CBOR tested with keys/values objects (MAP) and lists of values (ARRAY)

– Multiple architectures: x86_64 and aarch64 (Renesas H3)● Code is publicly available on git.iot.bzh:

https://git.iot.bzh/redpesk-labs/benchmark-serialization● Measurements:

– Time to serialize/deserialize 1 million messages

– CPU usage to serialize/deserialize sensor data at 50kHz (period=20µs)

July 20Boost AGL AppFW performance by 10+ 7

Benchmark: encode/decode time on H3

S11N Technology Time to create/encode/decode 1 000 000 events

Total time (ms) S11N Overhead (ms)7321 1813 55084569 1812 27578119 1812 63075337 1810 35279054 1813 72416154 1807 43476181 1811 43703857 1815 2042

PROTOBUF 3652 1808 1844XDR 2173 1836 337

Time to fill C struct (ms)

JSON-C : MAPJSON-C : ARRAYFASTJSON : MAPFASTJSON : ARRAYSTRINGJSON : MAPSTRINGJSON : ARRAYCBOR : MAPCBOR : ARRAY

July 20Boost AGL AppFW performance by 10+ 8

Benchmark: encode/decode CPU% on H3

S11N Technology

S11N Overhead (%)4.57 2.86 1.713.24 2.53 0.715.56 1.11 4.453.51 2.46 1.054.88 1.05 3.833.61 2.23 1.384.08 2.08 23.62 2.64 0.98

PROTOBUF 2.81 1.58 1.23XDR 2.17 1.64 0.53

CPU PERCENTAGE (100 000 events, 1 every 20 µs = 50 kHz)

Total CPU %CPU % to fill C struct ( %)

JSON-C : MAPJSON-C : ARRAYFASTJSON : MAPFASTJSON : ARRAYSTRINGJSON : MAPSTRINGJSON : ARRAYCBOR : MAPCBOR : ARRAY

July 20Boost AGL AppFW performance by 10+ 9

Benchmark analysis

● JSON libraries comparison: JSON-C is quite good● CBOR is usually faster than JSON but mileage varies

– Gain depends on object representation (map vs. array)

– Gain varies with hardware platform (x86_64 vs. aarch64)

– Gain is not that impressive● Static representations (Protobuf, XDR) are better but still not fast enough:

– Time divided by 16 (XDR) and 3 (Protobuf) vs. JSON-C

– Overhead divided by 3,2 (XDR) and 1,4 (Protobuf) vs. JSON-C

July 20Boost AGL AppFW performance by 10+ 10

Benchmarking outcome

● Changing the serialization format doesn’t solve the issue!● Even if gain in CPU, throughput or latency is significant, it

remains far from our needs● Serialization using static structures and code (XDR or

Protobuf) provides better results, but generates too many constraints and performance gain remains insufficient

● Solution: skip serialization when possible!

July 20Boost AGL AppFW performance by 10+ 11

Needs for a dual object representation

● AGL Framework uses a dual representation for objects:– “internal” representation = C structures

– “external” representation = JSON strings● Bindings mainly work on internal representation● External representation is used when exchanging data with

the “rest of the world”● V3 Bindings spend too much time to switch between both

representations.

July 20Boost AGL AppFW performance by 10+ 12

Optimize with Lazy Serialization

● Keep internal representation as long as possible:– Between threads inside the same process

– Between processes on the same host● Convert to/from external representation only when needed:

– Dynamic data models

– Messages between ECUs

– Messages outside of vehicle (V2Cloud, V2X)

– Monitoring/Debugging

July 20Boost AGL AppFW performance by 10+ 13

Revisit Transport Mechanisms

● New transport mechanisms allow to move objects from one binding to another while keeping the internal representation.

● In the same binder: no object copy (use reference counts)● On the same SoC: use zero-copy, shared memory or other

optimized mechanisms to share objects between processes● For the rest of the world: convert to external representation

and fall back to existing transports (Unix Sockets, WebSockets …)

July 20Boost AGL AppFW performance by 10+ 14

New Binder API V4 – type definition● An object type is defined by a structure:

● Usual types are pre-defined: integers, floats, strings ...● A predefined ‘JSON’ type brings compatibility with APIv3:

– internal representation is a json-c object– external representation is a JSON string

struct { const char *name; void *handle; size_t size; bool (*update_external)(afb_object_t *o, void *in); bool (*update_internal)(afb_object_t *o, void *ex); bool (*delete)(afb_object *o);} afb_type_t;

July 20Boost AGL AppFW performance by 10+ 15

New Binder API V4 – type registration

● As many types as needed can be defined by a binding and registered:

const afb_type_t type_hello4 = { .name = "hello4", .handle = NULL, .size = sizeof(hello4), .update_internal = func_from_STRING_to_hello4, .update_external = func_from_hello4_to_STRING, .delete = func_delete_hello4};

static int my_binding_preinit() { ... afb_register_type(type_hello4); ...}

typedef struct { char * key; int val;} hello4;

Type definition Type registration in binding

July 20Boost AGL AppFW performance by 10+ 16

New Binder API V4 – object retrieval

● Objects can be retrieved directly from requests and conversions are called automatically when needed:

void myverb (afb_req_t req) {int rc;afb_object_t *o;

// get first objectrc = afb_req_parameter(req,0,&o);if (rc && afb_object_type(o) == type_hello4) {

// get internal representationhello4 *h=(hello4 *)afb_object_pointer(o);

// do something with object ... } ...}

Lazy conversion

July 20Boost AGL AppFW performance by 10+ 17

Comparison with binder API V3

● AGL binder with API V3:– Uses only JSON-C objects for verbs requests, responses and

events payload

– Objects models don’t have to be “known” by the binder● AGL binder with API V4:

– Objects models allowing lazy serialization must be registered

– The binder calls conversion routines from/to external representation when needed (and tries to avoid to call them!)

July 20Boost AGL AppFW performance by 10+ 18

Generating a binding from C/C++ code● A generator parses the C/C++ code, analyzes types and

functions prototypes then generates the binding code

PUBLIC int fact(int n) { if (n <= 1) return 1; else return n*fact(n-1);}

#include <libafb-runtime.h>

void verb_fact (afb_req_t req) {int rc;int param_0;int result;

rc=afb_runtime_getarg(req,0,&param_0,AFB_TYPE_INT);if (rc==0) {

result = fact(param_0);afb_runtime_response(req,0,&result,AFB_TYPE_INT);

return } afb_runtime_error(req,-1);}

const struct afb_binding_v4 afbBindingV4 = { .api = “MyMaths”, .verbs = { .verb=”fact”, .callback=verb_fact, ... } ...};

Generator

Source code

Binding code (generated)

July 20Boost AGL AppFW performance by 10+ 19

Conclusion● Our first experiments show that “lazy serialization” provides significant performance

gains:– 50 to 100 between bindings running in the same binder

– 10 to 20 between bindings running in two binders on the same SoC

– Equivalent performance for remote binders● Binding code is more efficient but also more complex

– C/C++ generator helps to generate 90% of the code, covering marshaling/unmarshaling automatically

– Less constraints on libraries to integrate them as they can get any complex type as a parameter

– Types and references counts handling is done automatically by the binder● Compatibility with APIv3 is possible

– V3/V4 exchanges: using JSON strings is always available as a fallback

– V3/V3 exchanges: using a builtin type with json-c for internal object representation brings compatibility

– V3 Bindings are compatible at source code level (only a rebuild is necessary)

July 20Boost AGL AppFW performance by 10+ 20

Roadmap

● September 20: APIv4 availability as beta, targeting CES demos

● Q1 21: Full integration into redpesk AGL

July 20Boost AGL AppFW performance by 10+ 21

Links

● IoT.bzh:– Website: https://iot.bzh/

– Publications: https://iot.bzh/en/publications

– Github: https://github.com/iotbzh● redpesk :

– Website: https://redpesk.bzh/

– Documentation: https://docs.redpesk.bzh/

July 20Boost AGL AppFW performance by 10+ 22

Q&A

Lorient Harbor, South Brittany, France

Thi

s p

ictu

re is

an

orig

inal

pic

ture

ta

ken

by J

ack

Mam

ele

t in

200

6. I

t is

un

der

the

GN

U F

ree

Doc

ume

ntat

ion

Lic

ense

an

d th

e C

reat

ive

Com

mo

ns A

ttrib

utio

n.