[XCode] Automating UI Testing
-
Upload
phineas-huang -
Category
Design
-
view
202 -
download
1
Transcript of [XCode] Automating UI Testing
XCODE: AUTO TESTPhineas
ABOUT AUTO TEST
ABOUT AUTO-TEST
Why?
If we're testing internals we should probably test what user could see.
UI is the first thing users that see, so it is more important than how it does whatever.
Hard to forget to test something.
Much expensive, time, labor, money, boring, etc.
■ Logic tests. These tests check the correct functionality of a unit of code by itself (not in an app). With logic tests you can put together specific test cases to exercise a unit of code. You can also use logic tests to perform stress-testing of your code to ensure that it behaves correctly in extreme situations that are unlikely in a running app. These tests help you produce robust code that works correctly when used in ways that you did not anticipate.
■ Application tests. These tests check units of code in the context of your app. You can use application tests to ensure that the connections of your user-interface controls (outlets and actions) remain in place, and that your controls and controller objects work correctly with your object model as you work on your app. You can also use these tests to perform hardware testing, such as getting the location of the device on which your app is running.
UNIT TEST
TEST SCRIPT
INITIAL UNIT TEST
1. Static function will be executed once
2. General function always run at each test
XCTESTCASE
■ Fundamental Test XCTAssert(expression, format...)
■ Boolean Tests XCTAssertTrue(expression, format...) XCTAssertFalse(expression, format...)
■ Equality Tests XCTAssertEqual(expression1, expression2, format...) XCTAssertNotEqual(expression1, expression2, format...) XCTAssertEqualWithAccuracy(expression1, expression2, accuracy, format...)
XCTAssertNotEqualWithAccuracy(expression1, expression2, accuracy, format...)
■ Nil Tests XCTAssertNil(expression, format...) XCTAssertNotNil(expression, format...)
■ Unconditional Failure XCTFail(format...)
XCTESTCASE
PERFORMATION TESTING
■What are the absolute performance characteristics of
this code? Is the procedure bound by computation or
memory? What is the limiting behavior across different
sample sizes?
■What are the relative performance characteristics of this
code, as compared to its alternatives? Which is faster,
methodA or methodB?
CFTimeInterval startTime = CACurrentMediaTime(); {
. . . . . . . } CFTimeInterval endTime = CACurrentMediaTime(); NSLog(@"Total Runtime: %g s", endTime - startTime);
uint64_t t = dispatch_benchmark(iterations, ^{ @autoreleasepool { ……… } }); NSLog(@"[[NSMutableArray array] addObject:] Avg. Runtime: %llu ns", t);
measureBlock() { ……… }
■A performance test takes a block of code that you want to evaluate and runs it ten times, collecting the average execution time and the standard deviation for the runs. These statistics combine to create a baseline for comparison, a means to evaluate success or failure. To implement performance measuring tests, you write methods using new API from XCTest in Xcode 6 and later.
Test Suite 'Selected tests' started at 2014-12-11 01:21:27 +0000 Test Suite 'testerTests.xctest' started at 2014-12-11 01:21:27 +0000 Test Suite 'testerTests' started at 2014-12-11 01:21:27 +0000 Test Case '-[testerTests testPerformanceExample]' started. 2014-12-11 09:21:27.328 tester[724:13120] test start /Users/sunxiaoshan/Desktop/Report/XCodeTest/tester/tester/testerTests/testerTests.m:164: Test Case '-[testerTests testPerformanceExample]' measured [Time, seconds] average: 0.003, relative standard deviation: 3.742%, values: [0.002862, 0.002867, 0.002876, 0.003214, 0.003048, 0.002881, 0.002917, 0.002864, 0.002859, 0.002884], performanceMetricID:com.apple.XCTPerformanceMetric_WallClockTime, baselineName: "Local Baseline", baselineAverage: 1.810, maxPercentRegression: 10.000%, maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.100, maxStandardDeviation: 0.100 2014-12-11 09:21:27.705 tester[724:13120] test end Test Case '-[testerTests testPerformanceExample]' passed (0.378 seconds). Test Suite 'testerTests' passed at 2014-12-11 01:21:27 +0000. Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.379) seconds Test Suite 'testerTests.xctest' passed at 2014-12-11 01:21:27 +0000. Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.379) seconds Test Suite 'Selected tests' passed at 2014-12-11 01:21:27 +0000. Executed 1 test, with 0 failures (0 unexpected) in 0.378 (0.381) seconds
XCTESTEXPECTATION
XCTestExpectation *documentOpenExpectation = [self expectationWithDescription:@"document open”];
[documentOpenExpectation fulfill];
[self waitForExpectationsWithTimeout:15 handler:^(NSError *error) { if (error != nil) { XCTFail(@"timeout error: %@", error); } }];
UI AUTOMATION TEST
How to do it
XCode tool: Automation
Script Language : JavaScript
How to get object
window.buttons()[0]
window.buttons()[“login”]
HOW TO START
HOW TO START (I)
Create a new trace document in Instruments using the Automation trace template.
HOW TO START (II)
Click Add > Create.
HOW TO START (III)
In the Detail pane Navigation bar, select Script to enter the code for your script.
VIEW STRUCTURE
ELEMENT HIERARCHY (I)
UIATarget.localTarget()
frontMostApp()
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
ELEMENT HIERARCHY (II)
UIATarget.localTarget()
frontMostApp()
mainWindow()
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
ELEMENT HIERARCHY (III)
UIATarget.localTarget()
frontMostApp()
mainWindow()
tableViews()[0]
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
ELEMENT HIERARCHY (IV)
UIATarget.localTarget()
frontMostApp()
mainWindow()
tableViews()[0]
cells()[6]
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
ELEMENT HIERARCHY (V)
UIATarget.localTarget()
frontMostApp()
mainWindow()
tableViews()[0]
cells()[6]
elements()["Name"]
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
ELEMENT HIERARCHY (VI)
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
INTERACTION
INTERACTION
element.tap()
touchAndHold()
doubleTap()
scrollToVisible()
captureScreenWithName()
JENKINS
It’s a tool that can manager projects easily and write the custom script quickly by yourself.
JENKINS PLUGIN
NEW PROJECT
XCTOOL COMMAND LINE
REFERENCE
REFERENCE
https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
https://developer.apple.com/library/ios/documentation/DeveloperTools/Reference/UIAutomationRef/_index.html#//apple_ref/doc/uid/TP40009771
http://blog.manbolo.com/2012/04/08/ios-automated-tests-with-uiautomation
http://justbm.blogspot.tw/2011/12/my-ios-ui-automation-testing.html
http://www.slideshare.net/DavidReidy/automated-ui-testing
http://nshipster.com/xctestcase/
https://developer.apple.com/Library/ios/documentation/DeveloperTools/Conceptual/testing_with_xcode/testing_3_writing_test_classes/testing_3_writing_test_classes.html
https://developer.apple.com/legacy/library/documentation/DeveloperTools/Conceptual/UnitTesting/01-Unit-Test_Overview/overview.html
https://developer.apple.com/library/IOs/documentation/DeveloperTools/Conceptual/testing_with_xcode/testing_4_running_tests/testing_4_running_tests.html
http://jenkins-ci.org/