Functional Test Cases for Windows...

Functional Test Cases for Windows Applications Windows 8.x, Windows Phone 8.x, and Windows Universal (Desktop and

Mobile device families – Windows 10)

July 2015

Contents Introduction ........................................................................................................................................................................ 3

1. Launch-related Test Cases ......................................................................................................................................... 3

1.1 Low memory devices ............................................................................................................................................. 3

1.2 Installation on all supported device families ................................................................................................... 4

1.3 Unique Tiles ............................................................................................................................................................. 5

1.4 Offline launch .......................................................................................................................................................... 5

1.5 Splash screen ........................................................................................................................................................... 5

2. Log In Test Cases .......................................................................................................................................................... 6

2.1 Offline log in ............................................................................................................................................................ 6

2.2 Invalid credentials .................................................................................................................................................. 6

2.3 All log in methods are functional ...................................................................................................................... 7

2.4 Switching away after logging in ........................................................................................................................ 7

3. Application Use Test Cases ........................................................................................................................................ 8

3.1 App Switching ......................................................................................................................................................... 8

3.2 Software Input Pad implementation ................................................................................................................ 8

3.3 Touch targets .......................................................................................................................................................... 9

3.4 Visual feedback ...................................................................................................................................................... 9

3.5 Gestures ................................................................................................................................................................. 10

3.6 Secondary tiles ..................................................................................................................................................... 10

3.7 Command bar with contextual commands ................................................................................................... 11

3.8 Application is stable ............................................................................................................................................. 11

3.9 Application performance is responsive ......................................................................................................... 12

3.10 Toasts or notifications are actionable .......................................................................................................... 12

3.11 Offline mid-use scenarios................................................................................................................................. 13

3.12 Saved status preservation after exiting ........................................................................................................ 13

3.13 Portrait and landscape mode scenarios ...................................................................................................... 14

3.14 Core testing scenarios ...................................................................................................................................... 14

3.15 In-App purchases............................................................................................................................................... 15

3.16 Geofencing .......................................................................................................................................................... 15

3.17 Cross-Platform adaptive layout ..................................................................................................................... 15

3.18 Device-Specific features ................................................................................................................................... 16

3.19 App Resume ........................................................................................................................................................ 16

3.20 Network request cancellation (in-app) ........................................................................................................ 17

4. Phone-Only Test Cases ............................................................................................................................................ 17

4.1 Application resumes successfully from tombstoned state ....................................................................... 17

4.2 Light and dark themes ....................................................................................................................................... 18

4.3 Back button for mobile ...................................................................................................................................... 18

5. Windows-Only Test Cases ....................................................................................................................................... 19

5.1 Split screen views and display resolutions ..................................................................................................... 19

5.2 Application resumes from a suspended state ............................................................................................. 20

5.3 Back button for desktop .................................................................................................................................... 20

5.4 Desktop and tablet mode switching .............................................................................................................. 21

5.5 Multiple input methods ..................................................................................................................................... 21

6. Social Networking Integration Test Cases ........................................................................................................... 22

6.1 Social networking interactions .......................................................................................................................... 22

6.2 Share targets ......................................................................................................................................................... 22

7. Game Specific Test Cases ......................................................................................................................................... 23

7.1 Game leaderboard ............................................................................................................................................... 23

7.2 Game achievements and scores .....................................................................................................................24

7.3 Game multiplayer support ................................................................................................................................24

7.4 Game DVR .............................................................................................................................................................24

7.5 Roaming Game Saves ........................................................................................................................................ 25

8. Media Specific Test Cases ........................................................................................................................................ 25

8.1 Background audio playback .............................................................................................................................. 25

8.2 Background audio controls .............................................................................................................................. 26

8.3 Video pauses playback when the application is in the background ..................................................... 26

Media applications that display video should pause playback when the application is not in the

foreground. .................................................................................................................................................................. 26

9. JMA Feature Verification .......................................................................................................................................... 26

10. WACK Test Findings ................................................................................................................................................. 27

11. Competitive Parity Test Cases ................................................................................................................................ 27

11.1 Feature parity – Competitive Platforms ........................................................................................................ 27

11.2 Feature parity - Website ................................................................................................................................... 27

11.3 Performance parity............................................................................................................................................. 27

12. Essential and Showcase Features ......................................................................................................................... 28

12.1 Essential features ................................................................................................................................................ 28

12.2 Showcase features ............................................................................................................................................. 28

Introduction

Application functionality is the foundation of user satisfaction. The tests included here are designed to

test the most fundamental functional scenarios in Windows Phone 8.x, Windows 8.x, and Universal

Windows (Mobile and Desktop device families) Store applications and games. These tests make no

assumptions regarding an application’s user interface and verify that an application’s behavior is

consistent with user expectations and the Windows design guidelines. The reader is encouraged to

incorporate these test cases into their software development projects to help ensure high user

satisfaction – and high user ratings!

1. Launch-related Test Cases

1.1 Low memory devices

The application should install and function properly on low memory devices.

Expected result:

The application should function properly on low memory devices.

Note: A ‘Low memory’ phone is defined as a 512MB Windows Phone 8.0, 8.1 or 10 device.

1.2 Installation on all supported device families

The application should install and launch properly on all device families supported in the application’s

manifest file.

With Windows 10 developers can write a single application that can be installed across a variety of

device families including Mobile, Desktop, and XBOX. By default, Windows 10 applications target all

device families. In the application manifest, this is called out as the ‘Universal’ Target Device Family.

Such applications will be tested against all device form factors where this application may be

deployed. Alternatively, developers can limit the device families for which an application can be

deployed. For example, if ‘Desktop’ is specified and the Target Device Family, that application can

only be installed on PC devices. When published, this application will only be available in the Store

running on PCs – it will not be available in the Phone store.

Test Steps:

1. Attempt to install and launch the application on all device families supported in the

application’s manifest file.

Expected result:

The application should install and launch properly on all device families supported in the application’s

manifest file.

Target Device Family Hardware

Desktop PCs and Tablets running Desktop OS SKU.

Mobile Phone and Tablets running Mobile OS SKU.

Team Surface Hub running Team OS SKU.

Universal All

More information:

For more information see DeviceTargetFamily.

https://msdn.microsoft.com/en-us/library/windows/apps/dn986903.aspx

1.3 Unique Tiles

Using a unique tile can draw more people to an application. The tile should subtly reinforce your

brand with your users.

Test Steps:

1. Locate the application in the device’s list of applications.

2. Pin the application to the Start screen.

3. Adjust the tile size.

Expected result:

Tiles should be unique to the application.

Tiles should not display the default ⊠, a Unity logo, or be blank.

More information:

For more information see Guidelines for tiles and badges.

1.4 Offline launch

The application should be stable and not crash or close unexpectedly if the device is offline. When the

application is offline, the application should warn the user for any requested action that cannot be

completed.

Test Steps:

1. Turn airplane mode on.

2. Launch the application.

3. Perform an action that will cause a connection to be attempted.

Expected result:

The application will detect that the network is not available and alert the user.

Application should not crash and provide a relevant error message to user. Application should

not display the exception stack to the user.

The application can also implement an offline (cached) mode without alerting the user.

1.5 Splash screen

Use the splash screen to smooth the transition between when people launch your application and

when it's ready for use. The splash screen should subtly reinforce your brand with your users.

Test Steps:

http://msdn.microsoft.com/en-us/library/windows/apps/hh465403

1. Launch application and collect transition times for splash screen to initial screen of the

application.

Expected result:

Splash screen should reflect the brand of the application. The splash screen should not display

the default icon ⊠ or hourglass or blank splash screen.

Smooth transition between splash screen and the initial screen. A blank screen between

splash screen and initial screen should be no more than 2 seconds.

2. Log In Test Cases

The following test cases apply only to applications that support log in.

2.1 Offline log in

When the application is offline and the user attempts to log in, the application should warn the user

of the offline status.

Test Steps:



3. Attempt to log in with valid credentials.

4. Verify that an appropriate error message appears.

5. Turn airplane mode off.

6. Return to the application.

7. Log in with valid credentials.

Expected result:

When attempting to log in without a network connection, an appropriate error message

should appear warning the user of the offline state.

When the network connection has been resumed, log in should work appropriately.

2.2 Invalid credentials

The application should warn the user appropriately when invalid credentials are entered.

Test Steps:


2. Attempt to log in with invalid credentials.

3. Verify that an appropriate error message appears.

4. Log in with valid credentials, changing the casing in the username field.

Expected result:

When attempting to log in with invalid credentials, an appropriate error message should

appear warning the user that the credentials are invalid.

The username field should not be case-sensitive.

When correct credentials are entered after a failed attempt, log in should work appropriately.

2.3 All log in methods are functional

All available log in options should be fully functional and appropriately implemented.

Test Steps:


2. Log in using all available options with valid credentials.

Expected result:

Application logs in when valid credentials are entered.

Social networking log in and/or sign up is processed using the services’ login page(s) and not

using application pages for this purpose.

2.4 Switching away after logging in

The application should load content successfully after switching away during the loading process.

Test Steps:


2. Log in with valid credentials.

3. While the application is loading, switch away from the application.

4. Wait one second and return to the application.

Expected result:

Content should load successfully after returning to the application.

3. Application Use Test Cases

3.1 App Switching

When switching away from the application, network requests are cancelled. The application should

detect this situation and re-issue network requests when application is reactivated.

Test Steps:

1. Launch application.

2. Navigate through the application.

3. During loading (when loading indicator is present), switch away from the application.

4. Wait one second and return to the application.

Expected result:

Functionality completes successfully and data is downloaded without error or exception.

More information:

For more information on Fast App Switching in Windows Phone applications, see Get Ready for Fast

Application Switching in Windows Phone and App activation and deactivation for Windows Phone 8

For Windows applications, see Managing app lifecycle so your apps feel "always alive"

3.2 Software Input Pad implementation

The Software Input Pad (SIP) should be contextually-aware of the types of input allowed per input

field and it needs to be dismissed when input field is not selected on an application or when the

application is deactivated.

Test Steps:

1. Switch the device to Tablet mode.


3. Find an input textbox that allows alphanumeric characters.

4. Verify when this control has focus (is tapped with touch), the full keyboard and symbols are

displayed.

5. Find an input textbox that allows only numbers.

6. Verify when the control has focus (is tapped with touch), only the keypad SIP is displayed.

7. Move focus out of the input textbox (tap outside of the textbox with touch).

Expected result:

The SIP is dismissed when input controls lose focus.

http://channel9.msdn.com/events/MIX/MIX11/DVC09

http://channel9.msdn.com/events/MIX/MIX11/DVC09

https://msdn.microsoft.com/library/windows/apps/ff817008(v=vs.105).aspx

http://blogs.msdn.com/b/windowsappdev/archive/2012/04/30/10292044.aspx

Users can see what they are typing while they are typing it.

3.3 Touch targets

Touch targets should not be smaller than 7 mm or 26 pixels square.

Test Steps:



3. Identify actionable UI elements on the application.

Expected result:

Actionable UI elements in the application should be clearly visible and easily touchable by finger

without interfering with other neighboring touchable UI elements.

More information:

For more information on the Windows touch language, see Touch interaction design

3.4 Visual feedback

Visual feedback helps users recognize whether their interactions with your application are detected,

interpreted, and handled as they intended.

Test Steps:


2. Explore and navigate the application. Tap on each actionable UI element.

Expected result:

There is visual feedback for actionable or interactive UI elements.

There is no visual feedback for non-actionable or non-interactive UI elements.

Note: Acceptable visual feedback can be a slight change in color and/or movement of the

button when selected by touch or mouse (not when hovering over with a mouse.)

More information:

For more information on visual feedback, see Guidelines for visual feedback

Touch interaction guidance: Touch interactions for Windows

Responding to user interactions: Responding to user interaction


https://msdn.microsoft.com/en-us/library/windows/apps/hh465342.aspx

http://msdn.microsoft.com/en-us/library/windows/apps/hh465415.aspx


3.5 Gestures

Primary actions should be enabled by directly interacting with content. For example, tap to open,

swipe to select, pinch to zoom, and drag to pan/paginate. If a panorama or pivot control contains

other controls which use the same left / right flick gestures, this may result in a poor user experience

due to gesture overlaps. UI elements, controls or other parts of an application should not interfere

with Windows edge gesture.

Test Steps:


2. Explore and navigate the pages and functionality of application with touch only.

3. Inspect the application’s panorama/pivot control for scrollable map, horizontal scroll viewer,

toggle switches, sliders, or other controls that use left / right flick (or pan) gestures.

4. Navigate to each page and test the edge gestures for commanding surfaces (Title bar, Task

bar, Action center, Task View). (For Windows 8 apps: app bar, navigation bar, charms bar,

recent app.)

Expected results:

The application should fully support touch input. For example, the user should be able to

navigate or interact with commands using touch input only.

Touch gestures should only be used for actions that match the Windows touch language.

Pivot and/or panorama controls do not have controls with competing gestures and can be

used reliably.

If the hub, panorama or pivot reacts by scrolling then the child control should not trigger.

Any UI elements, controls, or other parts of the application should not interfere with Windows

edge gestures.

More Information:

Gesture interaction guidance: Gestures, manipulations, and interactions

3.6 Secondary tiles

Secondary tiles should enable users to pin context specific content on the Start screen.

Test Steps:


2. Explore the application and identify functionality (if any) where the user has the option to ‘Pin

to Start’.

https://msdn.microsoft.com/en-us/library/windows/apps/xaml/JJ883700(v=win.10).aspx

3. Select ‘Pin to Start’ to create a secondary tile on the Start screen.

4. Launch the application by tapping the primary tile of the application on the Start screen or

tapping the application name from the application list page.

5. Navigate to a second or third level page in the application.

6. Tap the Start button to return to the Start screen.

7. On the Start screen tap the secondary tile of the application.

8. Exit the application.

9. Launch the application from the secondary tile.

Expected result:

Launching the application from the secondary tile results in displaying content related to the

secondary tile.

On pressing the Back button (if applicable), user exits the application or navigates to the main

page.

More information:

For more information on tiles refer below link: Secondary tiles overview (Windows Runtime apps)

3.7 Command bar with contextual commands

Command bar commands need to be functional as well as predictable and organized so users can

easily find them.

Test Steps:


2. Explore and navigate the pages and functionality of application.

3. Select items with contextual commands.

Expected result:

All command bar buttons are functional, relevant, and contextual.

3.8 Application is stable

The application should be stable and not crash or hang. The application should not have functional

bugs that prevent completion of core scenarios.

Test Steps:




Expected result:

Application should not crash or hang.

3.9 Application performance is responsive

The application must appear responsive to the user at all times. Visual feedback is needed to indicate

the application is not frozen. Load times may be longer on slow connections resulting in the

application appearing to be frozen if no visual feedback is provided.

Test Steps:



3. Invoke functionality that requires time to load. E.g. network request to load data.

Expected result:

The application displays a loading or progress indicator for tasks that may take time to load,

and the content loads in a reasonable amount of time.

When the content has fully loaded, the loading indicator should no longer be present.

Application should be responsive and fluid.

Transition animations should be smooth.

The application should not display a blank screen, blank tiles, or empty white rectangles.

More information:

Guidelines for progress controls: Guidelines for progress controls

3.10 Toasts or notifications are actionable

Notifications should be used for time-sensitive or personally relevant notifications. Notifications are an

invite back into the application when it is not in the foreground. Tapping the notification should bring

the application to the foreground and in context to the notification.

Test Steps:


2. Invoke functionality to create a toast or notification. E.g. push notification or scheduled

reminder.

3. Switch away from the application and/or close the application.

4. Wait for the notification.

5. Select the notification.

Expected result:


When selecting the toast or notification, the application should be brought to the foreground

in the context of the toast or notification.

Toasts or notifications should not be used to show error messages or minor changes.

Toasts should not be shown when the app is visible on the screen (in-app toasts).

More information:

For more information see Guidelines for toast notifications and Guidelines for toasts and badges

3.11 Offline mid-use scenarios

When the system is in an offline scenario (no cell coverage, no WiFi, airplane mode, etc.), the

application should alert the user appropriately.

Test Steps:





5. Perform an action that will cause a connection to be attempted.

6. Verify that the application notifies the user of the offline state.

7. Turn airplane mode off.


Expected result:

The application will detect that the network is not available and alert the user.

Application should not crash and provide a relevant error message to user. Application should

not display the exception stack to the user.

The application can also implement an offline (cached) mode without alerting the user.

When resuming to the application in an online state after being previously using in an offline

state, the application should detect that the connection has been restored and work properly.

More information:

For more information, see Offline Sync for Mobile Services and Build offline apps with Azure Mobile

Services

3.12 Saved status preservation after exiting

Application progress and saved states should be maintained after exiting the application.


http://msdn.microsoft.com/en-US/library/windows/apps/hh465403

http://azure.microsoft.com/blog/2014/07/30/offline-sync-for-mobile-services/

http://channel9.msdn.com/Series/Windows-Azure-Mobile-Services/Build-offline-apps-Azure-Mobile-Services

http://channel9.msdn.com/Series/Windows-Azure-Mobile-Services/Build-offline-apps-Azure-Mobile-Services

Test Steps:


2. Explore and navigate the pages and functionality of application, saving whenever possible (for

games, clear 2~3 levels/stages/checkpoints).



Expected result:

The application should retained saved states after re-launch.

3.13 Portrait and landscape mode scenarios

The application should adapt properly to portrait or landscape view if supported.

Test Steps:


2. Rotate the device to portrait or landscape view.

3. If the application supports both portrait and landscape view (layout changes), navigate to

each page in the application.

Expected result:

All application pages render properly and adapt to the views supported. Ex: no text

truncation, all buttons are accessible, etc.

Applications should not display a logo or error message when the view is changed.

3.14 Core testing scenarios

All core scenarios are completed within the application.

Test Steps:


2. Navigate through the entire application.

Expected result:

All core scenarios are accessible from within the application, not from an external web

browser.

Note: It is acceptable for a privacy policy or a login page that redirects back to the application

to open in a web browser.

Note: It is acceptable for the application to contain an in-app web browser.

3.15 In-App purchases

In-App purchases should display the corresponding Store UI.

Test Steps:


2. Locate an in-app purchase.

3. Cancel the purchase.

Expected result:

The item for sale in the application should correspond to the one found in the Store UI.

Cancelling a purchase should not charge the user or reward them with the item.

After June 29, 2015, third party solutions for selling digital content consumed in the

application are no longer permitted. Microsoft IAP must be used to sell digital items that are

consumed or used within the application. The use of an external web browser or in-app

webview is not acceptable.

3.16 Geofencing

When geofenced sections of the application are accessed outside of the targeted area, the

application should display an appropriate error message.

Test Steps:


2. Locate or create the application scenario dependent on geofencing.

3. Perform an action that invokes the geofenced feature.

Expected result:

There should be an indication that the application is being used outside of the required region.

3.17 Cross-Platform adaptive layout

The application layout should adapt properly on all supported devices.

Test Steps:

1. Launch the application on all supported devices.

2. Navigate through the application and inspect the layout.

Expected result:

All application pages render properly and adapt across all supported devices. Ex: no text truncation,

all application buttons are accessible, etc.

More information:

For more information see Introduction to Universal Windows Platform (UWP) applications for

designers and TargetDeviceFamily (Windows 10 Insider Preview).

3.18 Device-Specific features

The application should adopt device-specific features properly on all supported devices.

Test Steps:

1. Launch the application on all supported devices.

2. Navigate through the application using device-specific features (ex: the device back button on

Windows Phone).

Expected result:

Device-specific features function properly on all supported devices.

More information:

For more information see Introduction to Universal Windows Platform (UWP) applications for

designers and TargetDeviceFamily (Windows 10 Insider Preview).

3.19 App Resume

App Resume allows the previous suspended instance of the application to be resumed when user re-

launches the application from the Start screen.

Test Steps:

1. Launch the application by tapping the primary tile of the application on the Start screen or

tapping the application name from the application list page.

2. Navigate to a second or third level page in the application.

3. Tap the Start button to return to the Start screen.

4. On Start screen, tap the primary Tile again to resume the application.

Expected result:







The application resumes to the same page as in step 2 and pressing the back button takes the user

through the correct sequence of pages.

Or

The application resumes at the main page (without the splash screen and loading indicator) and

pressing the Back key exits the application. Pressing the Back key should NOT navigate to another

page.

More information:

For more information see: App lifecycle - Windows app development

3.20 Network request cancellation (in-app)

Lengthy network requests may be canceled by the user at any time through in-app navigation. When

this occurs, users should be able to navigate through the application without error or exception.

Test Steps:

1. Launch the application (in Tablet mode for desktop applications).


3. During loading (when loading indicator is present), select the back button to return to the

previous page.

4. Navigate to another page, letting the data load completely.

Expected result:

Functionality completes successfully and data is downloaded without error or exception.

4. Phone-Only Test Cases

The following test cases apply only to Windows Phone applications.

4.1 Application resumes successfully from tombstoned state

Application should resume successfully from tombstoned state.

Test Steps:

1. Using a low-memory device if possible, launch a different application (ex: Internet Explorer).


2. Press the Start button.

3. Launch the application under test.

4. Perform some steps to get the application into a state with data present.

5. Run memory consumption application(s) to use all of the phone's memory (on 512mb devices

this is generally 5-6 games). This will cause application under test to tombstone.

6. Press the back button until the application under test is brought back.

Expected result:

The application should resume to the same page and state that it was in before entering

tombstoned state. There should be no data loss.

Application should not crash on resuming and user should be able to continue using the

application as normal.

More information:

For more information on tombstoning, see How to preserve and restore page state for Windows

Phone 8, App lifecycle - Windows app development, and ApplicationExecutionState enumeration.

4.2 Light and dark themes

Applications should adapt to a phone set to both light and dark themes.

Test Steps:

1. Set the device to the dark theme.



3. Set the device to the light theme.


Expected result:

The application displays properly when a device is set to the light or dark theme (ex: no light text

against white backgrounds, etc.)

4.3 Back button for mobile

Applications should properly navigate back through pages when using the back key.

https://msdn.microsoft.com/en-us/library/windows/apps/ff967548(v=vs.105).aspx

https://msdn.microsoft.com/en-us/library/windows/apps/ff967548(v=vs.105).aspx


https://msdn.microsoft.com/en-us/library/windows/apps/windows.applicationmodel.activation.applicationexecutionstate.aspx

Test Steps:


2. Navigate to a second or third page within the application.

3. Press the back key until reaching the Start menu.

Expected result:

The application should not bring up the Start screen or exit the application unless back key is

pressed while on the main menu.

The application should navigate back through the previously selected pages in the correct

order and eventually arrive at the Start menu.

5. Windows-Only Test Cases

The following test cases apply only to Windows Client applications.

5.1 Split screen views and display resolutions

The application should support a variety of screen sizes and split view(s).

Test Steps:

1. Set display resolution to minimum recommended resolution of 800x600 (For Windows 8.x set

the resolution to 1024x768).

2. Ensure that the device is in Desktop mode.


4. Select the window button in the title bar to resize the application screen.

5. On each page in the application, adjust the horizontal and vertical size of the application.

Inspect the layout and application functionality.


7. Switch the device to Tablet mode.


9. Split the screen by swiping down from the top of the screen.

10. On each page in the application, adjust the horizontal size of the application. Inspect the

layout and application functionality.


12. Set display resolution to 1920x1080.

13. Repeat steps 2-11.

Expected result:

All application pages render properly and adapt at minimum resolution and split and resized

view(s). Ex: no text truncation, all command and app bar buttons are accessible, etc.

Consumer applications should not display a logo or error message (ex: "this app does not

support split screen view") in split screen view.

When snapping an application the currently viewable position and state of the application

should still be visible when the application is in split screen view and full screen view.

Navigation between pages is still functional when in split screen view.

More information:

For more information see: Defining layouts and views and Guidelines for snapped and fill views

5.2 Application resumes from a suspended state

Applications should support the application lifecycle and resume successfully from a suspended state.

Test Steps:


2. Navigate to a page other than the default landing page for the application. Change the state

of the page. E.g. fill in a text box.

3. Switch to the desktop.

4. Open Task Manager (enable View | Status values | Show suspended state) and verify

application is suspended.

5. Switch back to the application.

Expected result:

Application should be in the same state as when the application was suspended.

More information:

For more information see Application lifecycle overview

5.3 Back button for desktop

Windows 10 applications should properly navigate back through pages when using the Windows 10

system back button.




Test Steps:

1. Set the device to Tablet mode.


3. Navigate to the second or third page within the application.

4. Select the system back button located on the taskbar until reaching the Start menu.

Expected result:

The application should not bring up the Start menu unless it is a Windows 8 application or if

the system back button is pressed while on the main menu.

The application should navigate back through the previously selected pages in the correct

order and eventually arrive at the Start menu.

5.4 Desktop and tablet mode switching

Applications should seamlessly support switching between Desktop and Tablet modes.

Test Steps:

1. Set the device to Desktop mode.


3. Navigate through the application and check the functionality.

4. Switch to Tablet mode.


6. Navigate through the application and check the functionality.

7. Switch to Desktop mode.


9. Navigate through the application and check the functionality

Expected result:

The application should not crash when switching between desktop and tablet mode.

The application should adapt to the change in screen format and handle the differing

functionality (Software Input Pad, split view, etc.)

5.5 Multiple input methods

The application should properly handle multiple input methods.

Test Steps:


2. Navigate through the application using a mouse. Checking boxes, right clicking on items, and

dragging if possible.

3. Navigate through the application using touch (and Xbox controller and Pen if applicable).

Attempt to perform all of the same functionality as the mouse (ex: right-clicking highlighted

text displays an additional menu).

Expected result:

Application fully supports navigation and selection using a mouse and touch.

Listbox items should not change state if there are no actions associated with marking or

‘ticking’ the item.

More information:

For more information on Touch: Gestures, manipulations, and interactions Keyboard: Responding to

keyboard interactions Mouse: Responding to mouse interactions

6. Social Networking Integration Test Cases

The following test case applies only to applications that have Social Network Service integration.

6.1 Social networking interactions

Social network content and status should function properly.

Test Steps:


2. Navigate through application where social networking tied features are provided (also check

the “Share” option located in the command bar or Share Charm if applicable.)

3. Verify that actions taken in the application are properly reflected on the social network.

Expected result:

The contents and status provided by the social network function properly.

6.2 Share targets

Applications that are a share target should be able to have content shared to them from another

application.





Test Steps:


2. Launch another application that is a share source.

3. Share to the application under test.

4. Verify that the actions taken in the application are properly reflected on the selected share

target.

Expected result:

The content shared using another application is properly shared to the application.

7. Game Specific Test Cases

Game applications may have special user scenarios where additional attention or different

interpretation of requirement is needed. The following test cases ensures applications meet general

quality bars commonly expected from game applications.

7.1 Game leaderboard

Implementation of a leaderboard is not required. If a leaderboard is implemented, ensure the

leaderboard is updated for single and multiplayer scenarios.

Test Steps:


2. Play the game long enough to post to the leaderboard for both single and/or multiplayer

sessions if offered on the application.

3. Look at all the leaderboards in the game.

Expected result:

With an active connection to the game’s backend service, the application should post to the

leaderboard showing both the gamer info such as gamer name and the associated score that

is the same as the one from the game session played.

Without an active connection to the game’s backend service, the application should not show

stability issues when not posting to a leaderboard and should inform users about the

connection issue to refresh leaderboard when users access leaderboards.

7.2 Game achievements and scores

Implementation of achievements and score are not required. If implemented, posted achievements

and scores should be recorded correctly.

Test Steps:


2. Play the game and earn as many achievements and high scores as possible.

Expected result:

The achievements and high scores are posted on the application and the application’s online

service accurately.

The achievements and high scores are earned while the device has no network connection

are saved accurately on the application and the application’s online service when connection

becomes available.

7.3 Game multiplayer support

Implementation of multiplayer support is not required. If implemented, multiplayer game functionality

should gracefully handle any player leaving the game.

Test Steps:


2. Select multiplayer game mode and start the game playing.

Expected result:

The game is played in a multiplayer mode with valid user.

The application gracefully handles the case where other user(s) playing the game left the

multiplayer game session.

The application gracefully handles the case where the user lost network connection during

multiplayer game session.

7.4 Game DVR

All games should properly support Game DVR.

Test Steps:

1. Launch the application in desktop mode.

2. Press Windows Key + G and record video while playing through a level.

Expected result:

If the game has been published to the Store, users should not be prompted to confirm that

the application is a game when starting Game DVR.

The video records successfully and the application does not freeze or crash during recording.

7.5 Roaming Game Saves

UWA games that can be used across multiple device families should support roaming game saves.

Test Steps:

1. Launch the application on the first device family (ex: desktop).

2. Complete a level and/or earn an achievement.

3. Launch the application on the second device family (ex: mobile), ensuring that this device is

logged into the same Microsoft account as the device in step 1.

4. Complete another level and/or earn an achievement.

5. Launch the application on the first device family (ex: desktop).

Expected result:

The game should retain achievements and level completions across multiple device families.

8. Media Specific Test Cases

The following test cases only apply to applications that support background audio or play video.

8.1 Background audio playback

Media applications that play audio (music, radio, podcasts, etc.) should continue audio playback when

the application is not in the foreground.

Test Steps:


2. Start audio playback.

3. Switch away from the application.

Expected result:

Audio continues to play after switching away from the application.

8.2 Background audio controls

Media applications that play audio (music, radio, podcasts, etc.) should support the system transport

control when the application is not in the foreground.

Test Steps:


2. Start playback.


4. Use hardware hot keys for volume to activate the system transport control.

5. Switch tracks or pause using the system transport control.

Expected result:

Audio playback should be controlled using the system transport control.

8.3 Video pauses playback when the application is in the background

Media applications that display video should pause playback when the application is not in the

foreground.

Test Steps:

1. Launch the application (in Tablet mode for desktop).

2. Start video playback.



Expected result:

Video playback should be paused when switching away from the application.

More information:

For more information, see Application lifecycle (Windows Runtime apps) [Activated and suspending

events]

9. JMA Feature Verification



10. WACK Test Findings

11. Competitive Parity Test Cases

11.1 Feature parity – Competitive Platforms

The application should have feature parity with competitive platforms (Android and iOS if applicable).

Test Steps:

1. Launch application on Windows, Android, and iOS devices (if applicable).

2. Check the application on Windows for feature gaps when compared to Android or iOS.

Expected result:

Non-OS specific features that are found in the Android or iOS versions of the application are also

found in the Windows version.

11.2 Feature parity - Website

The application should have feature parity with its respective website.

Test Steps:

1. Launch application on a Windows device and open the website the application is based on.

2. Check the application on Windows for feature gaps when compared to the website.

Expected result:

Features that are found on the website are also found on the Windows application.

11.3 Performance parity

The application should perform similarly to competitive platforms using comparably capable devices

(Android and iOS if applicable).

Test Steps:


2. Record the start up time for the initial launch.

3. Record the data loading time between a page or level.

4. Close the application.


6. Record the start-up time for subsequent launches.

7. Observe the graphical performance and overall application responsiveness.

8. Repeat steps 1-7 on competitive platforms (Android or iOS).

Expected result:

Applications should start up within 5 seconds. Games within 10. If this is not the case then the

start up time should be no more than the time found on competitive platforms.

Load times between pages should be no more than the time found on competitive platforms.

The graphics and responsiveness of the application is comparable across platforms.

12. Essential and Showcase Features

12.1 Essential features

At least one Windows 10 essential feature must be implemented.

Universal Windows app that supports multiple form factor families

Adaptive app, responsive layout & Continuum

Live tiles

Notification & Action Center

Game DVR

Roaming Game Saves

12.2 Showcase features

At least one Windows 10 showcase feature must be implemented.

Cortana

Drag and Drop

Continuum campanion experiences

Ink

Enterprise Data protection

Contact Card

Next Generation Credentials

Xbox Live Services

Cross-play with Phone

Direct X 12

Virtual Currency Cross-buy

Xbox Gamepad

Functional Test Cases for Windows...

Documents

Transcript of Functional Test Cases for Windows...