Benchmarking Software Implementations of 1st Round ......Benchmarking Software Implementations of...
Transcript of Benchmarking Software Implementations of 1st Round ......Benchmarking Software Implementations of...
Benchmarking Software Implementations of 1stRound Candidates of the NIST LWC Project on MCUs
Sebastian Renner, Enrico Pozzobon and Jürgen Mottok
Laboratory for Safe and Secure Systems, OTH Regensburg
November 3, 2019
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 1 / 19
I GoalsI FrameworkI Test SetupI Test CasesI PlatformsI ResultsI Conclusion & Future Work
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 2 / 19
Table of Contents
I Obtain performance figures for all NIST LWC candidatesI Provide results on popular embedded platformsI Test as many cipher variants as possible (per platform)I Fair comparison, i.e. no change of ref. implementationsI High degree of automationI Easy extensibility
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 3 / 19
Goals
I Written in C, Python and BashI Common test procedure for all platformsI MCU-specific template (platformIO/STM32CubeMX)
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 4 / 19
Framework I
I Fully automated compilation and test routineI Measurements are taken directly on the DUTI Use of NIST test vectors and API
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 5 / 19
Framework II
Test vectors sent and
verified overstdout/stdin
SubmittedAlgorithmsSubmitted
AlgorithmsSubmittedAlgorithms
MCU-specific template
C/C++/Arduinoproject structure
MCU
adapter.py
UART
JTAG / SWD
CompiledBuilds
test.pycompile_all.py
test_all.shGenerates
Compiles
All combi-nations are compiled
LogicAnalyzer
Git
Measurements from probes
TestVectors
Measurements
Logs
Runs
Figure: Core components and data flow of the test framework
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 6 / 19
Framework III
I SEGGER J-LinkI Saleae logic analyzerI GPIO pull up/down for signaling
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 7 / 19
Test Setup I
Figure: Benchmark Test Setup
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 8 / 19
Test Setup II
I Average over 1089 generated NIST test vector en- anddecryptions
I Expected PT and CT are compared to the resultsI nocrypt memcpy “algorithm“I AES-GCM implementation stripped out of mbedTLS for
reference
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 9 / 19
Test Cases: Avg. Encryption Speed
I nocrypt template size as baselineI Compilation with optimization for speed (where possible)I Bash script to determine code size
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 10 / 19
Test Cases: ROM Footprint
I Conducted only on the STM32 F746ZGI RAM gets filled with a known patternI Memory dumped after execution of algorithm(s)I Consecutive untouched memory segments are seen as
“unused memory“I nocrypt is used as a reference
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 11 / 19
Test Cases: SRAM Usage
I Arduino Uno R3: 8 bit ATmega328P MCU, 16 MHz clock, 32 KBflash
I STM32F1 “bluepill“: 32 bit ARM Cortex-M3 core, 72 MHz clock,64 KB flash
I STM32 NUCLEO-F746ZG: 32 bit ARM Cortex-M4, 216 MHzclock, 1MB flash
I Espressif ESP32 WROOM: 32 bit Xtensa LX6 MCU, 240 MHzclock, 4 MB flash
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 12 / 19
Tested Platforms
nocryp
t.ref
sneiken1
28.opt
sneiken1
92.opt
sneiken2
56.opt
tinyjam
bu12
8.op
t
tinyjam
bu19
2.op
t
clx1
28.opt
sneiken1
28.ref
clx1
92q.op
t
clx1
28q.op
t
sneiken1
92.ref
cipher
0.00000
0.00002
0.00004
0.00006
0.00008
averag
een
cryp
tiontim
e[s]
Figure: Ten fastest implementations on the STM32 F746ZG
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 13 / 19
Results: Speed
tinyjam
bu12
8.ref
tinyjam
bu25
6.ref
clx128
.ref
tinyjam
bu19
2.ref
clx128
h.ref
clx128
q.ref
clx192
h.ref
clx192
q.ref
clx256
h.ref
clx256
q.ref
cipher
0
200
400
600
800
1000
ROMusagecomparedto
nocrypt[b]
Figure: Ten smallest implementations on the STM32 F746ZG
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 14 / 19
Results: ROM
knot12
8v1.opt
ascon1
28av12
.bi32_arm
ascon1
28v12.bi32
_arm
tinyjam
bu12
8.opt
tinyjam
bu12
8.ref
ascon1
28av12
.bi32_lowreg
ascon1
28av12
.bi32
clx128
.opt
ascon1
28v12.bi32
clx128
.ref
cipher
0
20
40
60
80
100
RAMusagecomparedto
nocrypt[b]
Figure: Ten least RAM-intensive implementations on the STM32 F746ZG
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 15 / 19
Results: RAM
Figure: Test results for top ciphers per test case on the STM32 F746ZG
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 16 / 19
Results: Comparison
I Use of plain reference implementationsI Some cipher variants have not been tested (e.g. optimized
implementations)I The level of protection against SCA/FI was not taken into
account
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 17 / 19
Conclusion I
I Development of an easily extensible LWC performanceevaluation tool for embedded devices
I Successful test of 200+ cipher variants (enc/dec test case)I AES-GCM is neither best nor worst in all test cases
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 18 / 19
Conclusion II
I Extension of evaluation toolI Determine why some tests failedI SCA/FI attacks & countermeasures on e.g. fastest candidates
Renner, Pozzobon and Mottok NIST LWC Benchmarking November 3, 2019 19 / 19
Future Work