Anchoring and Adjustment in SQL Query Reuse

A Little Help can be a Bad Thing:

Gove Allen November 10, 2006 University of Houston

Gove AllenTulane University

Jeffrey ParsonsMemorial University of

Newfoundland

Code Reuse in Information Systems

• Code reuse (Cox 1990)

• Study of IS reuse (Frakes & Terry 1996)

• Query Reuse

• SQL is: A standard, portable across development environments, portable across DBMS, easily learned

Anchoring and Adjustment

• Tversky and Kahanman 1973

• Heuristic used to reduce cognitive burden

• Bias arises due to insufficient adjustment

• Example from Judgment

• Example from Problem Solving

Anchoring and Adjustment in Query Reuse

• A query is a candidate for reuse if it satisfies a similar information request

• Anchoring is promoted by the fact query executes and produces results (plausibility)

• Adjustment bias:

Surface level vs. Deep structure• Also consider role of domain familiarity

Hypotheses

• More errors H1

• Shorter time H2

• Higher confidence H3

• Worse prediction H4

When users reuse queries:

• Less reuse H5 • Better adjustment H6

Domain familiarity will lead to:

• Adjust surface anchors better than deep ones H7

In general, users will:

Domain Familiarity

• Difficult to study empirically

• Virtually impossible to assign subjects to different levels of domain familiarity

• Parallel database schemata

• Within subject assignment

• Implications for problem-solving in general

Gen

e O

nto

log

y D

om

ain

University

Un

iver

sity

Do

mai

n

Gen

e O

nto

log

y D

om

ain

Parallel Questions

How many times have professors employed students from Arizona (state='AZ')?

How many sponsors are the primary sponsors for courses that can be taken for credits ranging from 1 to 3 (credit_range='1-3')?

How many definitions are there for obsolete terms (is_obsolete=1)?

How many species are the primary species for gene products with a symbol of ADH2 (symbol='ADH2')?

Experimental Instrument

• Implemented as website

• Allows for presentation of example query

• Collects confidence information

• Allows dynamic query evaluation

• Automatic scoring

Question without Anchor

Question with Anchor

Subjects

• 157 Subjects

• 6 universities in United States

• Introduction to database management

Results for Anchoring

H1 More Errors P < .0001

Our sample: Half as likely to produce correct result

H2 Shorter Time P < .0001

Our sample: Less than half the time (40%)

H3 Higher Confidence P = 0.6981

Our sample: Only 0.6% difference

H4 Worse Prediction P < .0001

Our sample: More than 50% more overconfident

Results for Domain Familiarity

H5 Less Reuse P = 0.8875

Our sample: Only 0.5% difference

H6 Better Adjustment P = 0.7757

H6a Surface Anchors P < .0001

Our sample: 25% more likely for low familiarity

H6b Deep Anchors P = . 0148

Our sample: 7% more likely for high familiarity

Counter Significance

Surface vs. Deep Anchors

H7 Users will adjust surface anchors better than they will deep

anchorsP < .0001

Our sample: Subject almost three time as likely to correctly adjust surface-level anchors than deep-structure anchors

Implications• Given an opportunity, query writers will

reuse existing queries• Reusing queries can lead to higher

error rate• Reusing queries can lead to more

overconfidence• Adjustment bias is not entirely

mitigated by domain familiarity• Query writers focus on surface-level

changes when modifying queries

Questions and

Discussion