QTI: IMS Question & Test Interoperability Specification

10/06/2011

1

QTI: IMS QTI: IMS QuestionQuestion & Test & Test Interoperability SpecificationInteroperability Specification

Laura FarinettiLaura FarinettiDipartimento di Automatica e InformaticaPolitecnico di Torino

[email protected]

1

IMS Question & Test Interoperability IMS Question & Test Interoperability (QTI) specification (QTI) specification

Describes a data model for the representation of question (assessmentItem) and test (assessmentTest) data and their corresponding results reportsEnables the exchange of this item, test and results data between authoring tools item results data between authoring tools, item banks, test constructional tools, learning systems and assessment delivery systems

2

10/06/2011

2


U XML tUses XML syntaxPedagogy neutralSubject neutralAble to display formatted text, pictures, sound files video clips interactive sound files, video clips, interactive applications or applets

3


Supports several types of userSupports several types of userresponse … ◦ item selection, text input, numeric input,

xy-position selection and group selection

… that can be combined with several diff i h i different input techniques ◦ radio button, check box, text entry box, mouse

xy position dragging or clicking, slider bar, …

4

10/06/2011

3

QTI elementsQTI elements

Tests are called “assessments”An assessment has one or more questionsAn assessment has one or more questionsQuestions plus their associated data are called “items” ◦ Data needed to deliver the question, such as

layout rendering information, the score if it is correct, the feedback to be given, …

5

QTI specificationsQTI specifications

AssessmentItem◦ Explains the internal structure of an individual ◦ Explains the internal structure of an individual

item (i.e. a question)◦ Composed of

several sectionsAssessment

Section

Section

6

Item

Item

10/06/2011

4


AssessmentTest◦ A test is a group of assessmentItems with an ◦ A test is a group of assessmentItems with an

associated set of rules that determine which of the items the user sees, in what order, and in what way the candidate interacts with them◦ The rules describe the valid paths through the

test, when responses are submitted for test, when responses are submitted for response processing and when (if at all) feedback is to be given

7


A test is a logical aggregation of several QTI itemsQTI itemsMoreover, a test contains instructions on◦ how to create groups of items within a test◦ shuffle the sequence of items presented to the

user◦ specify when the candidate may submit

answers for response processing◦ provide feedback to the candidate at

different points within a test 8

10/06/2011

5


A test is a logical aggregation of several QTI itemsQTI itemsMoreover, a test contains instructions on◦ how to create groups of items within a test◦ shuffle the sequence of items presented to the

user◦ specify when the candidate may submit

answers for response processing◦ provide feedback to the candidate at

different points within a test 9

Question “anatomy”Question “anatomy”Statement

Stylesheet

Attributes

Choices

Logic for marking (response processing)

Variable declaration

10

processing)

Example 1

10/06/2011

6

Question “anatomy”Question “anatomy”

SAME logic for marking (response processing)

Choices

11

Example 2


Feedback

12

Example 3

10/06/2011

7


Data generated automaticallyaccording to a logic (template

i )

If it takes 3 women 40 minutes to dig a hole, how long would it take 6 women ?

minutes.2020

2 men 303 men

processing)

13

Example 4

QTI elementsQTI elements

Sections are used to group a series of questions within an assessmentquestions within an assessment◦ E.g. different section for each subtopic, to

know the score obtained foreach section as well as overthe assessment as a whole

At the end of the assessment

Assessment

Section

Section

At the end of the assessmenta result report is generated

14

Item

Item

10/06/2011

8

Item types Item types

Multiple choice SlideTrue falseMultiple responseImage hot spotFill in the blankS l t t t

Drag object, drag targetOrder objectsMatch itemsConnect pointsSelect text

15

Connect points

AssessmentAssessmentItemItem: structure: structure

AssessmentItem<AssessmentItem>

...

Attributes

Item Variable Declaration

Template Processing

Stylesheet

</AssessmentItem>

16

ItemBody

Response Processing

ModalFeedback

10/06/2011

9

AttributeAttribute

Characteristic of an itemComposed ofComposed of◦ A name predefined by the QTI specification

(e.g. “Title”)◦ A corresponding value (e.g. “Test on set

associative cache”) supplied by the item author

Attribute values are read and written by the systems which process QTI itemsNo additional attribute can be created

17

AttributeAttribute

Set of attributes predefined by the specificationspecification◦ IDENTIFIER - Uniquely identifies item

(mandatory)◦ TITLE - A short descriptive title for the item

(mandatory)LABEL A h t l b l f th it ( ti l)◦ LABEL - A short label for the item (optional)◦ LANGUAGE - QTI supports different

languages (optional)

18

10/06/2011

10

AttributeAttribute◦ ADAPTIVE - Is this item adaptive? (mandatory);

adaptive means that it changes appearance or p g ppscoring (or both) in response to each of the user attempts; otherwise it is a non-adaptive item (the default)◦ TIMEDEPENDENT - Will time taken to respond

to the question affect user score? (mandatory)◦ TOOLNAME - Name of the tool which created

the item (optional)◦ TOOLVERSION - Version of the tool which

created the item (optional)19

Attributes Attributes -- exampleexample

20

Example 1

10/06/2011

11

Attributes Attributes -- exampleexample

<assessmentItemxmlns="http://www.imsglobal.org/xsd/imsqti_v2p1"xmlns:xsi="http://www.w3.org/2001/XMLSchema instance"xsi:schemaLocation="http://www.imsglobal.org/xsd/

imsqti_v2p1 imsqti_v2p1.xsd"

identifier="choice"title="Unattended Luggage"

21

adaptive="false"timeDependent="false"

>

Item Variable DeclarationItem Variable Declaration

An item variable is an internal variable declared inside a question declared inside a question ◦ used to hold information temporarily during the

run time processing of an itemThree types of item variable◦ response variables: hold answers submitted by

the candidate◦ outcome variables: hold scores attained by the

candidate◦ template variables: used to generate cloned

items22

10/06/2011

12

Response Declaration SectionResponse Declaration Section

Declares one response variable inside the iitem

<responseDeclarationidentifier="RESPONSE"cardinality="single"baseType="identifier">< tR >

Name of the response variable

Can hold oneidentifier

23

<correctResponse><value>ChoiceA</value>

</correctResponse></responseDeclaration> Correct response

for this variable

Response Declaration SectionResponse Declaration Section

ASSIGNEDd d

ResponseVariable

during ItemBodyInteractions

24

DECLAREDin ResponseDeclaration

REFERENCEDinside Response

Processing

10/06/2011

13

Outcome Declaration SectionOutcome Declaration Section

Declares one outcome variable inside the item

<outcomeDeclarationidentifier="SCORE"cardinality="single"baseType="integer"><defaultValue>

Name of the outcome variable

Can hold oneinteger value

25

<value>0</value></defaultValue>

</outcomeDeclaration> Default value for this variable

OutcomeOutcome Declaration SectionDeclaration Section

ASSIGNED

OutcomeVariable

during ResponseProcessing

26

DECLAREDin OutcomeDeclaration

REFERENCEDinside Modal &

IntegratedFeedback

10/06/2011

14

Template Declaration SectionTemplate Declaration Section

Declares one template variable inside the pitem

<templateDeclarationidentifier="PEOPLE"cardinality="single"baseType="string"mathVariable="false"

Name of the template variable

Can holda singlestring value

27

paramVariable="false"/>

string value

TemplateTemplate Declaration SectionDeclaration Section

ASSIGNED

TemplateVariable

DECLAREDin TemplateDeclaration

during TemplateProcessing

REFERENCEDinside

ItemBody

28

Declaration ItemBody

REFERENCEDinside Response

Processing

10/06/2011

15

Template processingTemplate processing

Template processing generates cloned itemsA template processing section specifies a sequence of template rules which are evaluated by a delivery engine◦ Template rules generate values for one or more

template variablestemplate variables◦ These values are used inside the ItemBody

section to populate values in the item text

29

Template Template variablesvariables

APEOPLE

If it takes 3 women 40 minutes to dig a hole, how long would it take 6 women?

i t20

AMIN

30

Example 4

minutes.20

B

RESPONSE (Response variable)

PEOPLE

Item 53

10/06/2011

16


<responseDeclaration identifier="RESPONSE" cardinality "single" baseType "float" /> O cardinality="single" baseType="float" />

<templateDeclaration identifier="PEOPLE" cardinality="single" baseType="string" mathVariable="false" paramVariable="false" />

<templateDeclaration identifier="A" cardinality="single" baseType="integer" mathVariable="false" paramVariable="false" />

<templateDeclaration identifier="B" cardinality="single" baseType="integer" mathVariable="false" paramVariable="false" />

Four template variables

One response variable

31

mathVariable false paramVariable false /> <templateDeclaration identifier="MIN"

cardinality="single" baseType="integer" mathVariable="false" paramVariable="false" />


<templateProcessing><setTemplateValue identifier "PEOPLE">

Assign template variable PEOPLE the value “men ” or <setTemplateValue identifier="PEOPLE">

<random><multiple><baseValue baseType="string">men</baseValue><baseValue baseType="string">women</baseValue><baseValue baseType="string">children</baseValue>

</multiple></random>

</setTemplateValue><setTemplateValue identifier="A">

value men or “women” or “children”

32

<setTemplateValue identifier A ><randomInteger min="2" max="4" />

</setTemplateValue>... Assign template variable A the value 2 or

3 or 4

10/06/2011

17


<templateCondition><templateIf>

If A equals 2 thenAssign template variable Bthe value 4 or 6 or 8 or 10 or 12

<match><variable identifier="A" /><baseValue baseType="integer">2</baseValue>

</match><setTemplateValue identifier="B"><randomInteger min="4" max="12" step="2" />

</setTemplateValue></templateIf><templateElseIf>

t h

the value 4 or 6 or 8 or 10 or 12.

33

<match><variable identifier="A" /><baseValue baseType="integer">3</baseValue>

</match>... Else If A equals 3 then …

Template processingTemplate processing<setTemplateValue identifier="B"><random><multiple><baseValue baseType="integer">6</baseValue><baseValue baseType="integer">12</baseValue>

… Assign template B the value 6 or 12

<baseValue baseType= integer >12</baseValue></multiple>

</random></setTemplateValue>

</templateElseIf><templateElse><setTemplateValue identifier="B"><random><multiple><baseValue baseType="integer">8</baseValue>

34

<baseValue baseType integer >8</baseValue><baseValue baseType="integer">12</baseValue>

</multiple> </random>

</setTemplateValue></templateElse>

</templateCondition>...

ElseAssign template B the value 8 or 12

10/06/2011

18


<setTemplateValue identifier="MIN">Assign template variable MIN the value (120 / A)p

<integerDivide><baseValue baseType="integer">120</baseValue><variable identifier="A" />

</integerDivide></setTemplateValue><setCorrectResponse identifier="RESPONSE"><integerDivide><baseValue baseType="integer">120</baseValue><variable identifier="B" />

MIN the value (120 / A)

35

</integerDivide></setCorrectResponse>

</templateProcessing> Assign response variable RESPONSE the value (120 / B)

Template processingTemplate processing<itemBody> If it takes<printedVariable identifier="A" /><printedVariable identifier="PEOPLE" /><printedVariable identifier="MIN" />minutes to dig a hole, how long would it take

<printedVariable identifier="B" /><printedVariable identifier="PEOPLE" />to dig a similar hole?



36

Answer:<textEntryInteraction responseIdentifier="RESPONSE" expectedLength="8" /> minutes.

</itemBody>

10/06/2011

19


The ItemBody section uses these variables to generate a single cloned itemto generate a single cloned item◦ Uses the printedVariable element to insert the

value of a template value at a particular point in the question text

PEOPLE = “women”

37

A = 3B = 6 (note: because A=3, B can only take the value 6 or the value 12)MIN = 40RESPONSE = 20

If it takes 3 women 40 minutes to dig a hole, how long would it take 6 women?

Item BodyItem Body

Contains the text, graphics, media objects and interactions that describe the itemand interactions that describe the itemcontent and information about how it is structuredSpecifies◦ The item as seen on screen ◦ The interaction(s) between the user and the

item

38

10/06/2011

20

InteractionInteraction

Allow the candidate to interact with theitem item ◦ Through an interaction, the user selects or

constructs a response ◦ The user responses are stored in the

responseVariables E h i t ti i i t d ith ( t l t) ◦ Each interaction is associated with (at least) one response variable

39

Internal structureInternal structure

ItemBodyblockInteraction

choiceInteractionmatchInteraction

blockStatic< >

…

40

<div>

customInteractionpositionObjectStage

10/06/2011

21

Internal structureInternal structureblockInteraction blocks◦ the most commonly used block type: action types such

lti l h i t hi d lid bas multiple choice, matching gaps and slider bars(11 types)

blockStatic blocks◦ do not support interactions, but specify visual

components such as text, pictures and tablescustomInteraction blocks◦ allow to extend the specification and build new

interaction typespositionObjectStage blocks◦ blocks support images onto which the user can place

other visual objects41

choiceInteractionchoiceInteraction

42

Example 1

10/06/2011

22


<itemBody>Look at the text in the picture.

Two blockStaticblocks

<img src="images/sign.png"alt="NEVER LEAVE LUGGAGE UNATTENDED" />

<choiceInteraction responseIdentifier="RESPONSE" shuffle="false" maxChoices="1"><prompt>What does it say?</prompt><simpleChoice identifier="ChoiceA">You must stay with

your luggage at all times.</simpleChoice><simpleChoice identifier="ChoiceB">Do not let someone

else look after your luggage.</simpleChoice><simpleChoice identifier "ChoiceC">Remember your

43

<simpleChoice identifier="ChoiceC">Remember your luggage when you leave.</simpleChoice>

</choiceInteraction></itemBody>

One choiceInteraction block

If user selects the second option, the response variable RESPONSE is set to ChoiceB


<responseDeclaration identifier="RESPONSE" cardinality="single" baseType="identifier">

<correctResponse><value>ChoiceA</value>

</correctResponse></responseDeclaration> The correct answer of the question appears

earlier in the item, where the response variable RESPONSE is declared, incorporating the correctResponse element

44

10/06/2011

23

matchInteractionmatchInteraction

45

Example 5

matchInteractionmatchInteraction<responseDeclaration identifier="RESPONSE"

cardinality="multiple" baseType="directedPair"><correctResponse>

l C R / l

Declaration of the item variables

<value>C R</value><value>D M</value><value>L M</value><value>P T</value>

</correctResponse><mapping defaultValue="0"><mapEntry mapKey="C R" mappedValue="1" /><mapEntry mapKey="D M" mappedValue="0.5" /><mapEntry mapKey="L M" mappedValue="0.5" /><mapEntry mapKey="P T" mappedValue="1" />

RESPONSE stores one or more pairs of variables

46

<mapEntry mapKey= P T mappedValue= 1 /></mapping>

</responseDeclaration><outcomeDeclaration identifier="SCORE"

cardinality="single" baseType="float" />

The mapping element assigns a value to any correct combination (e.g. “C R”) entered by the user

10/06/2011

24

matchInteractionmatchInteraction<itemBody><matchInteraction responseIdentifier="RESPONSE"

shuffle="true" maxAssociations="4">shuffle true maxAssociations 4 ><prompt>Match the following characters to the

Shakespeare play they appeared in:</prompt><simpleMatchSet><simpleAssociableChoice identifier="C"

matchMax="1">Capulet</simpleAssociableChoice><simpleAssociableChoice identifier="D"

matchMax="1">Demetrius</simpleAssociableChoice><simpleAssociableChoice identifier="L"

matchMax="1">Lysander</simpleAssociableChoice>

47

y p<simpleAssociableChoice identifier="P"

matchMax="1">Prospero</simpleAssociableChoice></simpleMatchSet>...

matchInteractionmatchInteraction

...<simpleMatchSet><simpleMatchSet><simpleAssociableChoice identifier="M"

matchMax="4">A Midsummer-Night's Dream</simpleAssociableChoice>

<simpleAssociableChoice identifier="R" matchMax="4">Romeo and Juliet</simpleAssociableChoice>

<simpleAssociableChoice identifier="T" matchMax="4">The Tempest</simpleAssociableChoice>

</simpleMatchSet>

48

</matchInteraction></itemBody> The matchMax attributes places an upper

limit of how many associations can be specified against a particular value in a list

10/06/2011

25

Response ProcessingResponse Processing

Allows an item author to embed marking instructions inside an iteminstructions inside an item◦ These instructions are expressed in a high-level

language, not tied to any particular delivery tool or scoring engine

Similarities between response processingd t l t iand template processing

◦ Both execute logic written using QTI rules and update item variables

49

Response Processing sectionResponse Processing section

50

Example 6

10/06/2011

26


<responseDeclaration identifier="RESPONSE" cardinality "ordered" baseType "identifier">


cardinality="ordered" baseType="identifier"><correctResponse><value>DriverC</value><value>DriverA</value><value>DriverB</value>

</correctResponse></responseDeclaration>

<outcomeDeclaration identifier="SCORE" cardinality="single" baseType="integer" />

Ordered list of identifiers

51

cardinality single baseType integer />


<itemBody><orderInteraction responseIdentifier "RESPONSE"

ItemBody section

<orderInteraction responseIdentifier="RESPONSE" shuffle="true">

<prompt>The following F1 drivers finished on the podium in the first ever Grand Prix of Bahrain. Can you rearrange them into the correct finishing order?</prompt>

<simpleChoice identifier="DriverA">Rubens Barrichello</simpleChoice>

<simpleChoice identifier="DriverB">Jenson Button</simpleChoice>

52

Button</simpleChoice><simpleChoice identifier="DriverC">Michael

Schumacher</simpleChoice></orderInteraction>

</itemBody>

10/06/2011

27


<responseProcessing><responseCondition>

Processing section

<responseCondition><responseIf><match><variable identifier="RESPONSE" /><correct identifier="RESPONSE" />

</match><setOutcomeValue identifier="SCORE"><baseValue baseType="integer">2</baseValue>

</setOutcomeValue></responseIf>

53

</responseIf>... If current value of RESPONSE

equals the “correct value ” of RESPONSE then Set SCORE to 2

ResponseResponse Processing Processing sectionsection...<responseElseIf><match><variable identifier="RESPONSE" />

Processing section

<ordered><baseValue

baseType="identifier">DriverC</baseValue><baseValue

baseType="identifier">DriverB</baseValue><baseValue

baseType="identifier">DriverA</baseValue></ordered>

</match>

54

<setOutcomeValue identifier="SCORE"><baseValue baseType="integer">1</baseValue>

</setOutcomeValue></responseElseIf>...

Else if current value of RESPONSE equals the sequence “DriverC,DriverB,DriverA” thenSet SCORE to 1 (i.e. give partial credit for nearly-correct response)

10/06/2011

28


...<responseElse>

Processing section

<responseElse><setOutcomeValue identifier="SCORE"><baseValue baseType="integer">0</baseValue>

</setOutcomeValue></responseElse>

</responseCondition></responseProcessing> Else

Set SCORE to 0

55

Response Processing TemplatesResponse Processing Templates

Offer an alternative to hard wiring logic inside an iteminside an itemTemplates are prewritten segments of response processing which the item authorcan reference inside a responseProcessing element

56

10/06/2011

29


57

Example 1 -template


<responseProcessing template="http://www.imsglobal.org/ti / ti 2 0/ t l t / t h t l" /question/qti_v2p0/rptemplates/match_correct.xml" />

<?xml version="1.0" encoding="UTF-8" ?><responseProcessing

xmlns="http://www.imsglobal.org/xsd/imsqti_v2p0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.imsglobal.org/xsd/imsqti_v2p0 imsqti_v2p0.xsd">

58

... Template

10/06/2011

30

Response Processing TemplatesResponse Processing Templates...<responseCondition><responseIf><match><variable identifier="RESPONSE" /><correct identifier="RESPONSE" />

</match><setOutcomeValue identifier="SCORE"><baseValue baseType="integer">1</baseValue>

</setOutcomeValue></responseIf><responseElse>

tO t V l id tifi "SCORE"

59

<setOutcomeValue identifier="SCORE"><baseValue baseType="integer">0</baseValue>

</setOutcomeValue></responseElse>

</responseCondition></responseProcessing>

FeedbackFeedback

The item author can specify feedback at different points during user interaction with different points during user interaction with an itemThree types of feedback Modal Feedback: presented to the user immediately after response processing

60

10/06/2011

31

FeedbackFeedback

Integrated feedback: integrated into theitem itemBodyitem itemBody◦ users are free to update their responses while

viewing integrated feedback◦ not dependent on having submitted an answer

for response processing

T F db k d h Test Feedback: presented to the userconditionally based on the value of test Outcomes

61

Modal and integrated feedbackModal and integrated feedback

Integrated feedback

62

Example 3

Modal feedback

10/06/2011

32


63


<responseDeclaration identifier="RESPONSE"


espo se ec a at o de t e S O Scardinality="single" baseType="identifier">

<correctResponse><value>MGH001C</value>

</correctResponse></responseDeclaration><outcomeDeclaration identifier="SCORE"

cardinality="single" baseType="float" /><outcomeDeclaration identifier="FEEDBACK"

cardinality="single" baseType="identifier" />

64

10/06/2011

33

Integrated feedbackIntegrated feedback<itemBody><choiceInteraction responseIdentifier="RESPONSE"

shuffle="false" maxChoices="1">t Wh i th P id t f M i ? / t

ItemBody section

<prompt>Who is the President of Mexico?</prompt><simpleChoice identifier="MGH001A">George W Bush<feedbackInline outcomeIdentifier="FEEDBACK"

identifier="MGH001A" showHide="show">No, he is the President of the USA.

</feedbackInline></simpleChoice><simpleChoice identifier="MGH001B"> Tony Blair<feedbackInline outcomeIdentifier="FEEDBACK"

identifier="MGH001B" showHide="show">No he is

65

identifier= MGH001B showHide= show >No, he is the Prime Minister of England.

</feedbackInline></simpleChoice>...

IntegratedIntegrated feedbackfeedback...<simpleChoice identifier="MGH001C"> Vicente Fox<feedbackInline outcomeIdentifier="FEEDBACK"

identifier="MGH001C" showHide="show">Yes.

ItemBody section

</feedbackInline></simpleChoice><simpleChoice identifier="MGH001D">Ariel Sharon<feedbackInline outcomeIdentifier="FEEDBACK"

identifier="MGH001D" showHide="show">No, he is the Prime Minister of Israel.

</feedbackInline></simpleChoice>

</choiceInteraction>

Compare current value of outcome variable called “outcomeIdentifier” with value of “id ifi ” D h i l h?

66

</itemBody> “identifier”. Do their values match?

If showHide is set to “show” AND the values in step 1 match THEN

show the feedback messageElse

hide the feedback message

10/06/2011

34

ModalModal feedbackfeedback<responseProcessing><responseCondition><responseIf><match><variable identifier "RESPONSE" />

Response Processing section

<variable identifier="RESPONSE" /><correct identifier="RESPONSE" />

</match><setOutcomeValue identifier="SCORE"><baseValue baseType="float">1</baseValue>

</setOutcomeValue></responseIf><responseElse><setOutcomeValue identifier="SCORE"><baseValue baseType="float">0</baseValue>

67

<baseValue baseType float >0</baseValue></setOutcomeValue>

</responseElse></responseCondition><setOutcomeValue identifier="FEEDBACK"><variable identifier="RESPONSE" />

</setOutcomeValue></responseProcessing>

Modal feedbackModal feedback

<modalFeedback outcomeIdentifier="FEEDBACK"

Modal Feedback section

oda eedbac outco e de t e Cidentifier="MGH001C" showHide="show">Yes, that is correct.

</modalFeedback>

<modalFeedback outcomeIdentifier="FEEDBACK" identifier="MGH001C” showHide="hide">No, the correct answer is Vicente Fox.

</modalFeedback>

68

10/06/2011

35

StylesheetStylesheetAn item may use one or more external stylesheetsto enhance its appearance

<stylesheet href="shared/orkney.css" type="text/css" /><it B d ><itemBody><div class="rightpane"><object data="shared/orkney.html" type="text/html" />

</div><div class="leftpane">Read the text about the Orkney Islands and then

decide ifthe following sentence is correct or incorrect.

<choiceInteraction responseIdentifier="RESPONSE“shuffle="false" maxChoices="1">

69

shuffle= false maxChoices= 1 ><prompt>You can fly direct from London to

Kirkwall.</prompt><simpleChoice identifier="T">Correct</simpleChoice><simpleChoice identifier="F">Incorrect</simpleChoice>

</choiceInteraction></div>

</itemBody>

AssessmentAssessmentTestTest: structure: structure

A T

<AssessmentTest>...

</A tT t>AssessmentTest

Attributes

Outcome Declaration

TimeLimits

</AssessmentTest>

70

TestPart

OutcomeProcessing

TestFeedback

10/06/2011

36

AttributeAttribute

Attributes of the whole assessmentTest elementelement◦ IDENTIFIER - Uniquely identifies the test

(mandatory)◦ TITLE - Short title of the test (mandatory)◦ TOOLNAME - Name of the tool which created

th t t ( ti l)the test (optional)◦ TOOLVERSION - Version of the tool which

created the test (optional)

71

OutcomeDeclarationOutcomeDeclaration

An assessmentTest element can contain zero one or more outcome declaration zero, one or more outcome declaration elementsAn outcome declaration declares one outcome variable whose scope is the entire test◦ Stores a value relevant to the candidate’s

performance at the test level

72

10/06/2011

37


Other outcome variables are used at the item levelitem level◦ Item outcome variables generally determine

the value of test outcome variables◦ E.g. in a test composed of ten items, the value

of a test outcome variable may be calculated as the sum of the values of ten item outcome as the sum of the values of ten item outcome variables

73

OverallOverall approachapproach toto OutcomeDeclarationOutcomeDeclaration

Declare test outcome variables in the outcome declarations section of assessmentTestdeclarations section of assessmentTestFor each item response submitted by the candidate◦ First execute the item’s response processing (if

declared inside assessmentItem): this will update item’s outcome variablesitem s outcome variables◦ Then execute test’s outcome processing (if

declared inside assessmentTest): this will update test’s outcome variables

End for74

10/06/2011

38

OutcomeDeclarationOutcomeDeclaration<?xml version="1.0" encoding="UTF-8" ?><assessmentTest

xmlns="http://www.imsglobal.org/xsd/imsqti v2p1"

7 outcome variables

p g g q _ pxmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.imsglobal.org/xsd/imsqti_v2p1 imsqti_v2p1.xsd"identifier="RTEST-25"title="Basic Statistics as Outcomes">

<outcomeDeclaration identifier="NCORRECTA" cardinality="single" baseType="integer" />

<outcomeDeclaration identifier="NCORRECTB" cardinality="single" baseType="integer" />

i i i i

75

<outcomeDeclaration identifier="NINCORRECT" cardinality="single" baseType="integer" />

<outcomeDeclaration identifier="NRESPONSED" cardinality="single" baseType="integer" />

...


...<outcomeDeclaration identifier="NPRESENTED"<outcomeDeclaration identifier NPRESENTED

cardinality="single" baseType="integer" /><outcomeDeclaration identifier="NSELECTED"

cardinality="single" baseType="integer" /><outcomeDeclaration identifier="PERCENT_CORRECT"

cardinality="single" baseType="float" /><testPart navigationMode="nonlinear“

submissionMode="simultaneous"><outcomeProcessing>

</assessmentTest>

76

Sections explained later

10/06/2011

39

TestPartTestPart

Top-level sections that compose anassessmentTestassessmentTestA testPart contains one or more internal subsections calledassessmentSections◦ Allow the test author

assessmentTest

testPart

to disaggregate whatwould otherwise be alinear set of items

77

assessmentSection

assessmentItemRef

TestPartTestPart

An assessmentTest contains one or more testPartstestPartsA testPart contains one or more assessmentSectionsAn assessmentSection may reference one or more assessmentItems, via the assessmentItemRef element and/or contain one or more nested assessmentSections

78

10/06/2011

40

ExampleExample

A QTI test does not physically contain full items It “points to” items by specifying theirexternal filenames

79

Functionality associated withFunctionality associated withItemItem GroupsGroups

Item Groupsare managedvia attributes and elements

80

10/06/2011

41

TestPart TestPart -- attributes and elementsattributes and elementsidentifier (mandatory attribute )◦ Uniquely distinguishes an individual testPartUniquely distinguishes an individual testPart

within one assessmentTestnavigationMode (mandatory attribute )◦ determines the sequence in which the

candidate must answer items in the testPart◦ “linear”: the candidate must answer each item

one after the other and may not navigate back to a previously answered item◦ “nonlinear”: the candidate may answer the

items in any sequence

81

TestPart TestPart -- attributes and elementsattributes and elements

submissionMode (mandatory attribute )◦ determines when the candidate is permitted to ◦ determines when the candidate is permitted to

submit item responses◦ “individual”: the candidate submits the

response for one item before answering any other item in the testPart◦ “simultaneous”: responses for all the items in ◦ simultaneous : responses for all the items in

this testPart are submitted for response processing in one batch at the end of the testPart

82

10/06/2011

42


precondition (optional element)◦ evaluated just before a candidate enters a ◦ evaluated just before a candidate enters a

testPart◦ if a precondition expression evaluates to false,

then the associated testPart will not be presented to the candidate◦ more than one precondition can be associated ◦ more than one precondition can be associated

to a testPart◦ the precondition element is also defined in

sectionPart

83


branchRule (optional element)◦ determines “where next?” after a testPart has ◦ determines where next? after a testPart has

been completed by the candidate◦ if a branchRule evaluates to true, either the

candidate will be directed to another nominated testPart or the entire test will be terminated immediatelyterminated immediately◦ more than one branchRule can be associated

with a testPart

84

10/06/2011

43


itemSessionControl (optional element)◦ specifies a list of attributes which affect each ◦ specifies a list of attributes which affect each

item session in the testPart◦ attributes:maxAttempts, showFeedback,

allowReview, showSolution, allowComment, allowSkipping, validateResponses

timeLimits (optional element)timeLimits (optional element)◦ specifies a minimum and/or a maximum time

period for the candidate to answer all the items in the testPart

85


testFeedback (optional element)◦ specifies test (not item) level feedback which ◦ specifies test (not item) level feedback, which

can either be displayed during or at the end of a testPart◦ more than one testFeedback element can be

associated with a testPart

Note all the features apply to just one Note: all the features apply to just one testPart◦ different testParts within the same

AssessmentTest can have different features

86

10/06/2011

44

SectionPart SectionPart -- attributes and elementsattributes and elements

Contains attributes and elements which are common to both assessmentSection and common to both assessmentSection and assessmentItemRef ◦ sectionPart stores “a number of common

parameters shared by both types of child element”

id tifi ( d t tt ib t )identifier (mandatory attribute)◦ uniquely distinguishes the section or the item

reference within the overall test

87


required (optional attribute)◦ if set to true means this child element mustif set to true means this child element must

appear in the final selectionfixed (optional attribute)◦ relates to item shuffling within a section◦ if set to true, this element must remain in its

original position when the final shuffledg psequence is determined

preCondition (optional element)◦ determines if this item or section will be

skipped88

10/06/2011

45


branchRule (optional element)◦ contains rule(s) which identify under certain ◦ contains rule(s) which identify, under certain

circumstances, an alternative item or section

itemSessionControl (optional element)◦ contains parameters to control the allowable

states for each item session

timeLimits (optional element)◦ determines the period of time during which the

candidate may answer this item or this section

89

AssessmentSection AssessmentSection -- attributes and attributes and elementselements

title (mandatory attribute)◦ stores a string title which may or may not be◦ stores a string title,which may or may not be

displayed to the candidate

visible (mandatory attribute)◦ if set to true, the candidate is aware that this is

a distinct section within the overall testPart◦ if set to false, the candidate is not aware of

this section’s existence per se, and all items inside it will appear to belong to an outer section

90

10/06/2011

46


keepTogether (optional attribute) applies to invisible sections only to invisible sections only ◦ if set to false, the items in this section may be

mixed with items from a parent section◦ if set to true, the items in this section must

remain en bloc

l i ( i l l )selection (optional element)◦ determines how many child elements may be

chosen by a delivery engine from an assessmentSection

91


ordering (optional element)◦ determines the sequence in which the chosen◦ determines the sequence in which the chosen

child elements will be presented to the candidate◦ if the Boolean attribute shuffle is set to true,

the delivery engine will randomize the sequence in which the items are presentedsequence in which the items are presented

92

10/06/2011

47


rubricBlock (optional element)◦ determines the introductory text displayed for ◦ determines the introductory text displayed for

this assessmentSection. ◦ the audience of this text is determined by the

attribute view, which can take one or more values from a predefined set (e.g. candidate, author, tutor, … )author, tutor, … )

93

AssessmentItemRef AssessmentItemRef -- attributes and attributes and elementselements

href (mandatory attribute)◦ includes an item in the test◦ includes an item in the test◦ the attribute value should be the URI of the

external file containing the item

category (optional attribute)◦ associates a user-defined category (e.g.

“biology”) with an item◦ these categories can be detected and filtered

within outcome processing

94

10/06/2011

48

AssessmentItemRef AssessmentItemRef -- attributes and attributes and elementselements

variableMapping (optional element)◦ renames within the scope of a test a variable ◦ renames, within the scope of a test, a variable

declared inside an item

weight (optional element)◦ assigns a weighting for this item specifically in

the context of this test

templateDefault (optional element◦ overrides the default value of a template

variable declared by the item based on an expression evaluated at testlevel

95

TestPart TestPart -- exampleexample<?xml version="1.0" encoding="UTF-8" ?><assessmentTest

xmlns="http://www.imsglobal.org/xsd/imsqti_v2p1"_xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.imsglobal.org/xsd/imsqti_v2p1 imsqti_v2p1.xsd"identifier="RTEST-13"title="Steve's Test">

<testPart navigationMode="linear"

b i i M d "i di id l"

96

submissionMode="individual"><itemSessionControl showFeedback="true" maxAttempts="1"

showSolution="false" /><assessmentSection identifier="sectionA"

title="Section A" visible="true">...

10/06/2011

49

TestPartTestPart -- exampleexample...<rubricBlock view="candidate">Instructions for Section A

</rubricBlock><assessmentItemRef identifier="item034"<assessmentItemRef identifier= item034

href="item034.xml"><itemSessionControl maxAttempts="0" />

</assessmentItemRef><assessmentItemRef identifier="item160"

href="item160.xml"><preCondition>

97

<preCondition><match><variable identifier="item034.RESPONSE" /><baseValue baseType="identifier">Y</baseValue>

</match></preCondition>

</assessmentItemRef>...

TestPart TestPart -- exampleexample...<assessmentItemRef identifier="item063"

href="item063.xml">href item063.xml<preCondition><match><variable identifier="item034.RESPONSE" /><baseValue baseType="identifier">N</baseValue>

</match></preCondition>

98

</assessmentItemRef></assessmentSection>...

10/06/2011

50

TestPart TestPart -- exampleexample...<assessmentSection identifier="sectionB“

title="Section B" visible="true"><itemSessionControl maxAttempts="2" /><rubricBlock view="candidate">Instructions for Section B

</rubricBlock><assessmentItemRef identifier="item347"

href="item347.xml">

99

<branchRule target="item656"><match><variable identifier="item347.RESPONSE" /><baseValue baseType="identifier">N</baseValue>

</match></branchRule>...

TestPart TestPart -- exampleexample...test ><branchRule target="EXIT_SECTION"><not><match><variable identifier="item347.RESPONSE" /><baseValue baseType="identifier">Y</baseValue>

</match></not>

</branchRule></assessmentItemRef>

100

/<assessmentItemRef identifier="item653"

href="item653.xml" /><assessmentItemRef identifier="item656"

href="item656.xml" /></assessmentSection>

</testPart></assessmentTest>

10/06/2011

51

TimeLimitsTimeLimits

Option to specify the time period available to the candidate for answering questionsto the candidate for answering questionsThe time constraints can be applied to◦ A single item◦ A group of items◦ The whole

test

101


For each, the test author may specify atimeLimits element which contains two timeLimits element, which contains two attributes◦ minTime (in number of seconds, integer value)◦ maxTime (in number of seconds, integer value)◦ Either or both of these attributes may be

fspecified

102

10/06/2011

52

TimeLimitsTimeLimitsTimeLimits element of assessmentTest◦ The “total time” covers not only the sum of The total time covers not only the sum of

times taken to answer each item, but also any extra time the candidate spends in navigating the screen to move amongst questions

A delivery engine can keep track of the time taken to complete a test◦ This value is held in a predefined QTI response

variable called “duration”◦ This variable value can be referenced inside

outcome processing logic

103


<?xml version="1.0" encoding="UTF-8" ?><assessmentTest

xmlns="http://www.imsglobal.org/xsd/imsqti_v2p1"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.imsglobal.org/xsd/imsqti_v2p1 imsqti_v2p1.xsd"

identifier="RTEST-09"title="Steve's Test">

...

10/06/2011

53

TimeLimitsTimeLimits...<testPart navigationMode="linear"

submissionMode="individual">it S i C t l h F db k "t "<itemSessionControl showFeedback="true"

maxAttempts="1" showSolution="false" /><timeLimits maxTime="3600" /><assessmentSection identifier="sectionA"

title="Section A" visible="true"><rubricBlock view="candidate">Instructions for Section A


href="item034 xml">

105

href= item034.xml ><itemSessionControl maxAttempts="0" /><timeLimits minTime="180" maxTime="600" />

</assessmentItemRef>...

TimeLimitsTimeLimits...<assessmentItemRef identifier="item160"


href="item063.xml" /></assessmentSection><assessmentSection identifier="sectionB"

title="Section B" visible="true"><itemSessionControl maxAttempts="2" /><timeLimits maxTime="7200" /><rubricBlock view="candidate">Instructions for Section B


h f "it 434 l" />

106




</testPart></assessmentTest>

10/06/2011

54

OutcomeProcessingOutcomeProcessing

Specifies logic which assigns values to one or more test outcome variablesor more test outcome variablesOutcome processing takes place each time the candidate submits the responses◦ for an item, when in individual submission mode◦ or a group of items, when in simultaneous

submission mode

107

OutcomeProcessingOutcomeProcessing

The values assigned to outcome variables can be used tocan be used to◦ display test-level (not item-level) feedback to

the candidate◦ branch to another section or item within the

assessmentTest

108

10/06/2011

55

OutcomeProcessing OutcomeProcessing -- exampleexample

<outcomeDeclaration identifier="NCORRECTA" cardinality="single" baseType="integer" />

<outcomeDeclaration identifier="NCORRECTB" cardinality="single" baseType="integer" />

<outcomeDeclaration identifier="NINCORRECT" cardinality="single" baseType="integer" />

<outcomeDeclaration identifier="NRESPONSED" cardinality="single" baseType="integer" />

<outcomeDeclaration identifier="NPRESENTED" cardinality="single" baseType="integer" />

<outcomeDeclaration identifier="NSELECTED" cardinality "single" baseType "integer" />

109

cardinality="single" baseType="integer" /><outcomeDeclaration identifier="PERCENT_CORRECT"

cardinality="single" baseType="float" />...

OutcomeProcessing OutcomeProcessing -- exampleexample...<testPart navigationMode="nonlinear"

submissionMode="simultaneous"> Execute once<assessmentSection identifier="sectionA"

title="Section A" visible="true"><rubricBlock view="candidate">Instructions for Section A

</rubricBlock><assessmentItemRef identifier="item034“


href="item160.xml" />i i i i 063

Execute once

110

<assessmentItemRef identifier="item063" href="item063.xml" />

</assessmentSection>...

10/06/2011

56

OutcomeProcessing OutcomeProcessing -- exampleexample...<assessmentSection identifier="sectionB"

title="Section B" visible="true"><selection select="2" /><rubricBlock view="candidate">Instructions for Section B




h f "it 365 l" /

111



</testPart>...

OutcomeProcessing OutcomeProcessing -- exampleexample...<outcomeProcessing><setOutcomeValue identifier="NCORRECTA"><numberCorrect sectionIdentifier="sectionA" />

</setOutcomeValue><setOutcomeValue identifier="NCORRECTB"><numberCorrect sectionIdentifier="sectionB" />

</setOutcomeValue><setOutcomeValue identifier="NINCORRECT"><numberIncorrect />

</setOutcomeValue><setOutcomeValue identifier="NRESPONSED"><numberResponded />

Predefined QTI functions

112

<numberResponded /></setOutcomeValue><setOutcomeValue identifier="NPRESENTED"><numberPresented />

</setOutcomeValue>...

10/06/2011

57

OutcomeProcessing OutcomeProcessing -- exampleexample

...P d fi d QTI <setOutcomeValue identifier="NSELECTED">

<numberSelected /></setOutcomeValue><setOutcomeValue identifier="PERCENT_CORRECT"><product><baseValue baseType="float">100.0</baseValue><divide><numberCorrect /><numberSelected />

</di ide>

Predefined QTI functions

100 * (numberCorrect / numberSelected)

113

</divide></product>

</setOutcomeValue></outcomeProcessing>

numberSelected)

TestFeedBackTestFeedBack

Components at the assessmentTest levelaccess (mandatory attribute)access (mandatory attribute)◦ “during”: test feedback is shown immediately

after response processing◦ “atEnd”: test feedback appears only once,

after submission of a response to the last item tt t d i th t tattempted in the test

outcomeIdentifier (mandatory attribute)◦ refers to a previously declared outcome

variable114

10/06/2011

58


showHide (mandatory attribute)◦ “show” or “hide”◦ show or hide

identifier (mandatory attribute) ◦ used in conjuction with the outcomeIdentifier

and showHide ◦ if the value of this attribute is included in the

value of outcomeIdentifier, and the showHide attribute is set to true, then feedback will be shown◦ otherwise feedback will not be shown

115


title (optional attribute)◦ can be displayed to the candidate◦ can be displayed to the candidate

flowStatic (zero or more elements)◦ contain the actual test feedback to be

rendered to the candidate◦ fixed text or, more likely, information based on

responses submitted so far

116

10/06/2011

59

More examplesMore examples

Multiple choice questionMultiple choice question

118

Example 7

10/06/2011

60

AssociationAssociation questionquestion

119

Example 8

Text entry questionText entry question

120

Example 9

10/06/2011

61

Hot text questionHot text question

121

Example 10

Hot spot Hot spot questionquestion

122

Example 11

Item 38

10/06/2011

62

SelectSelect pointpoint questionquestion

123

Example 12

Item 50

QTI: IMS Question & Test Interoperability Specification

Documents

Transcript of QTI: IMS Question & Test Interoperability Specification