B2G Phone-Automation Lab

Goals, Challenges, Accomplishments

July 31, 2014

Stephen Donnersdonner@mozilla.com

Goals● greater capacity:

○ keep up with b2g-inbound builds for UI + perf (esp. the latter), as much as possible

○ multi-node mapping/support● high reliability● easier device

troubleshooting/administration

From this...wait for it!

1st Buildout

To this...getting there; closer there

2nd Buildout

And here we areBam!

And still going...

Challenges (1)● balancing buildout with current-rig’s maintenance● near-constant task (re)prioritization● real carriers/networks (AT&T/T-Mobile), real problems:

○ spam calls/voicemail○ Amber alerts○ low signal/data-connection resets

■ slow throughput■ DNS-lookup failures

Challenges (2)● multi-node ADB mapping

○ solution had to work with both UI and perf tests○ also required:

■ Jenkins-config changes (job/system)■ physical reconfiguration

● active SIMs vs. inactive SIMs○ reduce interference/cost/maintenance cost

● SecOps/NetOps requirements○ new office, new problems (new policies)

■ port-by-port, host-by-host firewall pass-through○ VPN/Jenkins (CI) access granted on a user-by-user basis

Challenges (3)● RF (Radio Frequency)

○ Wi-Fi■ ateam vs. Mozilla Mobile

● capacity + speed + reach○ FM tuner interference

■ requires cabling magic● ...and perhaps a low-power, local FM transmitter?

○ more devices, more interference?● physical space/configuration

○ shelving space○ power-strip capacity

Challenges (4)● remote teams requested live Air Mozilla

streams of the phones○ prototype works, but doesn’t immediately scale

bandwidth (# streams, codecs/encoders)■ informs # of encoder boxes/# of channels

○ live (remote) demo● “keeping the light on”

○ camera test failures due to low light■ ...but intermittent

Currently*, we have:● ~23 active Flame nodes (phones)

○ 12 dedicated to UI / 11 perf○ 26 potentially

● attached to 13 active Mac Minis

* number of active nodes fluctuates due to investigational/maintenance work, etc.

What’s Left?● remainder of phones (20) to come online in

the next few weeks● MozPool for managing devices, including:

○ taking devices offline○ power measurement through ammeter○ remote-reboot capability through power harness

● Puppetization for managing node configurations

Comments? Questions?● Mailing list: mozwebqa@mozilla.org● Team page: https://quality.mozilla.

org/teams/web-qa/● IRC: #mozwebqa on irc.mozilla.org