MozillaUnder the Hood

Presented byMarcia Knous

Team Mozilla QA

Brief History of Mozilla

Mozilla is...

A global effort to promote choice and innovation on the Internet

An open source project

Software for the people, by the people

NY Times two page adDecember 16, 2004

What fuels us?

Community‣ 2008 Mozilla Summit

We build on top of each other

A Good Day at Mozilla :)

The tree is green...

...So what do we test?

Everything!

• Firefox Release builds

• Including Firefox Localizations

• Firefox Nightly Builds

• Firefox Mobile (Fennec)

• Firefox web properties

• Thunderbird

What do we focus on?

•Basic functionality

•Standards Compliance

•Performance

How do we do this?•Automated Tests

•Manual Tests

The Mozilla Platform

Automated Tests

Unit Tests

xpcshell

Compiled code

XPC Shell‣Anything available to the XPCOM layer (through scriptable interfaces) can be tested with xpcshell.‣Limitations: Can’t easily open windows, test chrome, work with HTML parser or the DOM‣

Graphical & Interactive Tests

Mochitest

Chrome Tests

Reftest

Crash Tests

Content Mochitest

Highlights

Runs tests in a full browser environment with chrome privileges

Can simulate user input (keyboard/mouse)

Good for testing things such as password manager and popup blocking

Mochichrome

Highlights

Elevated privileges allow access to XPCOM components and services

Uses simple JS function calls to aid in correctness

Useful for testing UI components such as:

Download Manager/Addons Manager

Browser Chrome Tests

Tests are written as JS files

Uses simple JS function calls to aid in correctness

Useful for testing Browser UI features

Crash Tests

Based on the reftest framework

The test works by loading the file and completing without crashing or asserting

Reftests➡Layout Engine Visual Tests➡Results are reported as Pass, Fail or Unexpected

Desired Result of the test

Taloshttp://graphs.mozilla.org

Leak Tests

Topsite Testing

Other Tests➡Fuzz Tests➡Accessibility➡JS Tests

Mozmill

Functional UI testing for Mozilla Applications.

Manual Tests

Litmus

‣Open 24/7 for your testing enjoyment

‣Test cases written by Mozilla QA & community

Litmus Full Functional Tests

Litmus Basic Functional Tests

Litmus Smoketests

Manual Debugging ToolsThe Error Console is your friend!

Even with all these tests...

There is no magic bullet

The bug that forced FF 3 RC2

There is humor in testing software...

Keynote software errorCommon sight while preparing this presentation, but

at least I can send feedback...

Who tests?Fosdem, 2009

The Community

Nightly Testers - ~ 10K

Betatesters - ~ 3 Million

Web Developers

Mozilla QA Team - 18

The QA CompanionUseful add on that allows the community to test

An Example of Code Coverage

How do we get feedback?

Community Members file bugs and help triage

The Mozilla “kitchen”

Bugzilla

Hendrix

Feedback link is postioned prominently on Release Notes and first run page

Blocker bug for Firefox 3 was found through this feedback mechanism

First run page before we changed the version to Firefox 3.5...But nonetheless a good example of a the direct feedback links (Bugzilla, Hendrix) on first run page

Twitter

Tweets regarding issues with Firefox are reported and monitored

BreakpadCrash reporting tool

MozillaZine

Firefox Forum Discussions

At one time was the premier Mozilla news site

Newsgroups - moz.dev.quality

QMO - The QA PortalCentral repository for project info and documents

reporter.mozilla.org

Linked directly from the Help menu

Allows you to report a broken website right from the UI

So who reviews all this feedback?

Sometimes tough to separate the “noise” from the real issues

We give back...

Songbird

Blackbird

Instantbird

Seamonkey

KompoZer

Nokia’s Internet Tablet

Wyzo

Miro

Flock

Celtx

Komodo

Projects/Technologies

Projects

Firefox

Camino

Bugzilla

Fennec

Lightning and Sunbird

Seamonkey

Thunderbird

Technologies

Gecko

XULRunner

Mozilla Labs

http://labs.mozilla.com/projects/

The end...questions?

Resources/Background material

The History of Mozilla: https://library.mozilla.org/The_History_of_Mozilla

QMO: http://quality.mozilla.org/

Litmus: https://litmus.mozilla.org/

Mozmill: http://code.google.com/p/mozmill/

Developer info: https://developer.mozilla.org/

Mozilla Labs: http://labs.mozilla.com/

Tinderbox for Firefox 3.5: http://tinderbox.mozilla.org/showbuilds.cgi?tree=Firefox3.5

Resources/Background material

Get the Firefox Beta: http://www.mozilla.com/en-US/firefox/all-beta.html

QA Companion: https://addons.mozilla.org/en-US/firefox/addon/5428

Hendrix: http://hendrix.mozilla.org/

Mozilla support: http://support.mozilla.com/en-US/kb/

Mozilla Digital Memory Bank: http://chnm.gmu.edu/mozilla-digital-memory-bank/

Marcia’s Firefox Robot Wars Video: http://www.youtube.com/watch?v=OnZClPMoYEY

Jaguar Inline 6 Car Engine Photo courtesy of dave_7. Licensed under a Creative Commons license - http://www.flickr.com/photos/daveseven/250993044/

Bullet photo courtesy of JinglyJon. Licensed under a Creative Commons license : http://www.flickr.com/photos/jinglyjon/180600454/

Photo credits