Office Protocol Tools and TestingJinghui Zhang

Protocol Implementation Scenarios

Client Implementation Application connecting to a Microsoft Server

Server Implementation Client connecting to a Microsoft Server

Scenarios

Tools

MailSim

Protocol Test Suites

Message Analyzer

Message Analyzer Parsers

OXML SDK

Fiddler Inspectors

Many more!

Man-in-the-MiddleApplication manipulating traffic over the wire

Test Driven Quality Assurance

▪ Ambiguity Review▪ Simulation of the implementation Process

1 Open Specification Review

▪ Normative language ▪ Requirement Extraction

2 Requirement Analysis

▪ Requirements become Test Cases ▪ Groups of Test Cases become Scenarios

3 Test Suite Development

…, this method MUST return a SOAP fault with the specified error code.

SUT Control Adapter

Test Cases

TC TC TC

TC TC TC

System Under Test

Test Suite Design

Adapters Protocol Adapter• Pluggable• Different transports

• Scenario Based • Derived from Open Specification

Protocol Behavior Configuration

• Generate/Receive protocol messages

• Less likely to be replaced• C#

• SUT configure/query• Likely to be replaced (product

specific)• PowerShell, C#

Test Suite Example: Versioning

• Generates Protocol Traffic “synthetic client”• Source code provided• Customizable against 3rd party implementations • Automatable

Office Test Suite Components

Test Cases

To drive test steps and

validate requirements.• Traditional test cases

• MBT (model-based

testing) test cases

Protocol Adapter

To create/manage

connection between

client and server, and

generate/consume

protocol messages

SUT Control Adapter

To control SUT at

runtime

• PowerShell Script

Adapter

• Managed Source

Code Adapter (C#)

• Interactive Adapter

Office Test Suite Environment Topology

Test Suite Network Topology

Machine name

Exchange Test Suite Environment

SharePoint Test Suite Environment

SUT * • Microsoft Exchange Server 2016/2013/2010/2007

• Microsoft SharePoint Server 2016/2013/2010/2007

CLIENT • Microsoft Visual Studio 2013 Ultimate

• Protocol Test Framework

• Microsoft Visual Studio 2013 Ultimate

• Protocol Test Framework

Optional • Spec Explorer 2010• Microsoft Message Analyzer or

Fiddler

• Microsoft Message Analyzer or Fiddler

CLIENT

IPv4 - 192.168.1.2

IPv6 - 2008::65

contoso.com

SUT

IPv4 - 192.168.1.1

IPv6 - 2008::C9

*3rd party implementation

Running the Test Suite

Using Visual Studio

• Open Visual Studio Test

Suite Solution and

rebuild the solution

• Open the Test Explorer

• Select test cases, right

click and select the

option to Run/Debug

them

Using the included

batch files

• All tests in all

protocols, all tests

within one protocol,

or a single test case

Using command line

• Use mstest.exe

https://github.com/OfficeDev/Interop-TestSuites

• MS-LISTSWS• MS-VERSS• MS-SITESS• MS-DWSS• MS-ADMINS• MS-MEETS• MS-OUTSPS• MS-WDVMODUU• MS-WEBSS• MS-WWSP• MS-AUTHWS• MS-SHDACCWS• MS-CPSWS• MS-WSSREST• MS-OFFICIALFILE• MS-COPYS• MS-VIEWSS

• MS-FSSHTTP• MS-FSSHTTPB• MS-FSSHTTPD• MS-WOPI

• MS-OXWSATT• MS-OXWSBTRF• MS-OXWSCONT• MS-OXWSCORE• MS-OXWSFOLD• MS-OXWSMSG• MS-OXWSMTGS• MS-OXWSSYNC• MS-OXWSTASK

• MS-ASAIRS

• MS-ASCAL

• MS-ASWBXML

• MS-ASCNTC

• MS-ASCON

• MS-ASDOC

• MS-ASEMAIL

• MS-ASHTTP

• MS-ASNOTE

• MS-ASPROV

• MS-ASRM

• MS-ASTASK

• MS-ASCMD

• MS-OXCFOLD• MS-OXCFXICS• MS-OXCMAPIHTTP• MS-OXCMSG• MS-OXCNOTIF• MS-OXCPERM• MS-OXCPRPT• MS-OXCROPS• MS-OXCRPC• MS-OXCSTOR• MS-OXCTABL• MS-OXNSPI• MS-OXORULE

Office Protocol Test Suite Environment - Test Matrix

Role Computer Name

Public IP Internal IP Domain User Password for Domain User

Active Directory DC01 **.**.***.** 192.168.0.*

Contoso\Username ********

Exchange 2016 ServerSUT01 **.**.***.** 192.168.0.*

Contoso\Username ********

SharePoint 2016 Server SUT02 **.**.***.** 192.168.0.* Contoso\Username ********

Client ENDPOINT01 **.**.***.** 192.168.0.* Contoso\Username ********

Office Protocol Test Suite Environment - Overview

Process of using Office protocol test suites

How to use Office Test Suite Environment this week

1. Ask for Azure VM environment • Talk to Tarun Chopra Tarun.Chopra@microsoft.com

2. Remote desktop connect to VMs

3. Try it out!

Thank you!Questions?

(dochelp@microsoft.com)