LCE13: LAVA Multi-Node Testing
-
Upload
linaro -
Category
Technology
-
view
173 -
download
2
description
Transcript of LCE13: LAVA Multi-Node Testing
LAVA Multi-node Testing
Antonio Terceiro, Senthil Kumaran, Neil Williams, James Tunnicliffe & Fu Wei
LCE13 - Dublin, July 2013
Support for testing use cases across multiple devices
DISCLAIMER: none of the stuff you are about to see is in production yet.
→�You get an Unicode cat, though
● Scope and use cases● Proposed API● Demo (???)● Status and Future Plans● Feedback - open discussion
Session Outline
Scope and use cases
● test jobs involving multiple devices● devices may be of same type, or not - it
doesn't matter● devices get roles assigned to them● each device can be flashed with a different
test image
Scope (1)
● test writers can write any test they want.● LAVA provides information/synchronization
primitives● each device runs tests as it would normally
run, plus they can synchronize with each other using the API
● in the test results are aggregated into a single results bundle.
Scope (2)
● Requests 2 devices● Role 1: server● Role 2: client● Server:
○ install/start server application● Client:
○ install client application○ hits server, does benchmarks
Example 1: client/server interaction
● Requests 6 devices. Roles:○ database-server x 1○ web-backend x 2○ load-balancer x 1○ client x 2
● wiring up the application together is test writer's job
● provisioning/orchestration tools can be used since tests are shell scripts○ salt○ puppet○ chef○ etc
Example 2: multi-tier application
● the test suite for toolchain component X can be cleanly split in say 3 components, a, b, c
● Request 3 devices with roles, a, b, c● Node with role ${label} will download pre-
built binaries and run the test suite component labelled with ${label}
Example 3: parallel build/test run
● Physical setup:○ Phone connected through USB to x86 host○ Host configured to pass through the USB device to a
KVM device● Request one phone device and one KVM
device● Deploy latest Android build to phone● Boot Ubuntu build to KVM● Test that you can do ADB over USB
operations from Ubuntu to the phone
Example 4: ADB connectivity
● The LAVA team will provide the infrastructure and support
● Other teams can come up with whatever tests they need○ that are physically possible ;-)
● use cases that require specialized hardware setups require lab work and will be handled in a case-by-case basis.
Example n ...
Proposed API
device_group: [
{
"device_type": "highbank",
"role": "server",
"count": 1,
},
{
"device_type": "panda",
"role": "server",
"count": 1,
},
]
JOB submission (1)
device_group: [
{
"device_type": "galaxy-nexus",
"role": "phone",
"count": 1, "tags": "usb-pair-01"
},
{
"device_type": "kvm",
"role": "x86-box",
"count": 1, "tags": "usb-pair-01"
},
]
JOB submission (1.1)
Requesting a specific hardware setup
actions: [{
"command": "deploy_linaro_image","parameters": {
"image": "url://to/images/server.img"},"role": "server"
},[...]
]
JOB submission (2)
only applies to the devices
with the "server" role
actions: [// ...{
"command": "lava_test_shell","parameters": {
"testdef_urls": ["url://to/testdef.yaml"
]}
},// ...
]
JOB submission (3)
THIS DID
NOT CHA
NGE
# testdef.yamlmetadata: name: lava-test-shell-simple description: "description of this testdef" version: 1.0 format: "Lava-Test-Shell Test Definition 1.0"
run: steps: - lava-test-case test1 --result pass - lava-test-case test2 --result fail - lava-test-case test3 --shell ./test-script - [...]
LAVA-Test-Shell Review (1)
# testdef.yamlmetadata: name: lava-test-shell-simple description: "description of this testdef" version: 1.0 format: "Lava-Test-Shell Test Definition 1.0"
run: steps: - ./run-my-test-suite
LAVA-Test-Shell Review (2)
Set of programs in $PATH that are available for your LAVA-Test-Shell tests to call
lava-test-caselava-test-case-attachlava-test-run-attachlava-os-buildlava-installed-packageslava-install-packages
The LAVA Test Shell API
An extension of the existing LAVA Test Shell API that adds:
● Commands that provide contextual information
● Commands for communication and synchronization
The LAVA multi-node API
lava-role
Prints the role of the current device
# testdef.yamlrun:
steps:- ./run-$(lava-role).sh
lava-group
Prints the hostnames and roles in the group
$ lava-grouppanda01 clienthighbank01 backendhighbank02 backendhighbank03 load-balancer
lava-sync
Global synchronization across the group
# testdef.yamlrun:
steps:- ./setup-$(lava-role).sh- lava-sync setup-done- ./run-$(lava-role).sh
lava-send <msg-id> [k1=v1 k2=v2 ...]
Sends a message to the group
lava-wait <msg-id>
Waits for <msg-id> from some other node
lava-wait-all <msg-id>
Waits for <msg-id> from all other nodes
Other building blocks
Demo
Status and Future Plans
● Implementation in advanced stage● Started initial testing● Should be available in production at 2013.
XX
Status
● Exclusive lock on groups of devices○ "this 10 devices share a network bus. Even if I am
only using 5, I don't want the other 5 to be used simultaneously because that may affect the results of my networking benchmarks
● [...]
Future work
Feedback(Open discussion)
● Tell us about your testing use cases● Does lava-send <msg-id> k1=v1 k2=v2
look useful?● "What if I need a specific hardware
configuration for my tests?"● ...
Seed questions