Click Models for Web Search - Lecture 2 · Click Models for Web Search Lecture 2 Aleksandr...

Click probabilities Applications Parameter estimation

Click Models for Web SearchLecture 2

Aleksandr Chuklin§,¶ Ilya Markov§ Maarten de Rijke§

[email protected] [email protected] [email protected]

§University of Amsterdam¶Google Research Europe

AC–IM–MdR Click Models for Web Search 1


Course overview

Basic Click Models

Parameter Estimation Evaluation

Data and ToolsResultsApplications

Advanced Models

Recent Studies

Future Research



Lecture 1

Basic Click Models



Advanced Models

Recent Studies

Future Research



Lecture 1 recap

CTR models: counting clicks

Position-based model (PBM): examination and attractiveness

Cascade model (CM): previous examinations and clicks matter

Dynamic Bayesian network model (DBN): satisfactoriness

User browsing model (UBM): rank of previous click



Lecture 2

Basic Click Models



Advanced Models

Recent Studies

Future Research



Probability theory

Partitioned probability: A = A1 ∪ A2, A1 ∩ A2 = ∅

P(A) = P(A1,A2) = P(A1) + P(A2)

Bayes’ rule

P(A | B) · P(B) = P(B | A) · P(A)

B causes A: B → A

P(B) = P(B | A) · P(A)



Lecture outline

1 Click probabilities

2 Applications

3 Parameter estimation



Click probabilities

Full probability – probabilitythat a user clickson a document at rank r

P(Cr = 1)

Conditional probability –probability that a user clickson a document at rank rgiven previous clicks

P(Cr = 1 | C1, . . . ,Cr−1)



Dependency between examination and clicks

document u

Eu

Cu

Au

↵uq�ru



Full click probability

P(Cr = 1) = +P(Cr = 1 | Er = 1) · P(Er = 1)

P(Cr = 1 | Er = 0) · P(Er = 0)

= P(Aur = 1) · P(Er = 1) + 0

= αurqεr



Cascade models: dependency between examinations

document urdocument ur�1

Er�1

Cr�1

Ar�1

Er

Cr

Ar

......

↵ur�1q ↵urq



Full click probability: Dynamic Bayesian network model

Dynamic Bayesian network model: satisfactoriness

document urdocument ur�1

Er�1

Cr�1

Ar�1

Er

Cr

Ar

......

↵ur�1q ↵urq

Sr�1 Sr

�

�ur�1q �urq

P(Cr+1 = 1) = αur+1qεr ·

(+P(Er+1 = 1 | Er = 1,Cr = 1) · P(Cr = 1 | Er = 1)

P(Er+1 = 1 | Er = 1,Cr = 0) · P(Cr = 0 | Er = 1)

)

P(Cr+1 = 1) = αur+1qεr ·

(+

(1− σurq)γ · αurq

γ · (1− αurq)

)AC–IM–MdR Click Models for Web Search 14


Click probabilities summary

Full probability

P(Cr+1 = 1) =

αur+1qεr ·

(+P(Er+1 = 1 | Er = 1,Cr = 1) · P(Cr = 1 | Er = 1)

P(Er+1 = 1 | Er = 1,Cr = 0) · P(Cr = 0 | Er = 1)

)

Conditional probability

P(Cr+1 = 1 | C1, . . . ,Cr )

= αur+1q ·

+

P(Er+1 = 1 | Er = 1,Cr = 1) · c(s)r

P(Er+1 = 1 | Er = 1,Cr = 0) · εr (1− αurq)

1− αurqεr· (1− c(s)

r )



What do click models give us?

General:

Understanding of user behavior

Specific:

Conditional click probabilities

Full click probabilities

Attractiveness and satisfactoriness for query-document pairs



Lecture outline


2 ApplicationsUser interaction analysisSimulating usersModel-based metricsApproximating document relevance

3 Parameter estimation



Lecture outline




User interaction analysis

Random click model (global CTR): ρ = 0.122

Rank-based CTR:ρ1 = 0.429, ρ2 = 0.190, ρ3 = 0.136, . . . , ρ10 = 0.048

Position-based model:γ1 = 0.998, γ2 = 0.939, γ3 = 0.759, . . . , γ10 = 0.260

Dynamic Bayesian network model: γ = 0.9997

Click models are trained on the first 10K sessions of the WSCD 2012 dataset.



Lecture outline




Simulating users

Algorithm Simulating user clicks

Input: click model M, query session sOutput: vector of simulated clicks (c1, . . . , cn)

1: for r ← 1 to |s| do2: Pr ← PM(Cr = 1 | C1 = c1, . . . ,Cr−1 = cr−1)︸︷︷︸

conditional click probability

3: Generate cr from Bernoulli(Pr )4: end for



Lecture outline




Model-based metrics

Utility-based metrics

uMetric =n∑

r=1

P(Cr = 1)·Ur

Effort-based metrics

eMetric =n∑

r=1

P(Sr = 1) ·Fr



Expected reciprocal rank

RR =1

r, where Sr = 1

ERR =∑r

1

r· P(Sr = 1)



Dynamic Bayesian network model (DBN)

P(Ar = 1) = αurq

P(E1 = 1) = 1

P(Er = 1 | Sr−1 = 1) = 0

P(Er = 1 | Sr−1 = 0) = γ

P(Sr = 1 | Cr = 0) = 0

P(Sr = 1 | Cr = 1) = σurq

P(Sr = 1) =?



DBN: Satisfaction

P(Sr = 1) = P(Sr = 1 | Cr = 1) · P(Cr = 1)

= σurq · P(Cr = 1)

= σurq · αurq · P(Er = 1)

= σurq · αurq ·r−1∏i=1

(γ · (1− σuiq · αuiq)

)= Rurq ·

r−1∏i=1

(γ · (1− Ruiq)

)



Expected reciprocal rank

ERR =∑r

1

r· P(Sr = 1)

=∑r

1

r· Rurq ·

r−1∏i=1

(γ · (1− Ruiq)

)



Model-based metrics

Model-based metric

Click model Utility-based Effort-based

DBN uSDBN ERRDBN EBU rrDBNUBM uUBM –



Lecture outline




Approximating document relevance

αu1q σu1q

αu2q σu2q

αu3q σu3q

αu4q σu4q

αu5q σu5q

PBM, UBM DBN



Approximating document relevance

Clicks are affected by rank =⇒ do not represent documentrelevance directly

Attractiveness and satisfactoriness do not depend on rank =⇒can be used as indicators of document relevance

They are used by search engines as retrieval features

Documents can simply be ranked by αuq, σuq, or αuqσuq



Applications summary

Click model’s output Application

Understanding of user behavior User interaction analysisConditional click probabilities User simulationFull click probabilities Model-based metricsParameter values Ranking



Lecture outline


2 Applications

3 Parameter estimationMaximum likelihood estimationExpectation maximizationExpectation maximization examplesAlternative estimation methods



Lecture outline




MLE for random click model

P(Cu = 1) = ρ

L =∏s∈S

∏u∈s

ρc(s)u (1− ρ)1−c(s)

u

︸︷︷︸likelihood of Bernoulli random variable

LL =∑s∈S

∑u∈s

(c

(s)u log(ρ) + (1− c

(s)u ) log(1− ρ)

)

ρ =

∑s∈S

∑u∈s c

(s)u∑

s∈S |s|=

# clicks

# shown docs



Lecture outline




Expectation maximization

1 Set parameters to some initial values2 Repeat until convergence

E-step: derive the expectation of the likelihood functionM-step: maximize this expectation



EM terminology

θc – parameter of a click model

Xc – random variablecorresponding to θc

P(Xc) – parents of Xc

Examples:

P(C ) = {A,B}P(A) = ∅

A

C

B

E

DA

C

B

E

DA

C

B

E

D



EM objective

Find the value of parameter θcthat optimizes log-likelihood LL of the model

given observed query sessions S



E-step

LL =∑s∈S

log

(∑X

P(

X,C(s) | Ψ))

X – all random variables

Ψ – all parameters

C(s) – clicks in a query session s

Q =∑s∈S

EX|C(s),Ψ

[logP

(X,C(s) | Ψ

)]



E-step (grouping)

Q(θc) =∑s∈S

EX|C(s),Ψ

[logP

(X,C(s) | Ψ

)]=∑s∈S

EX|C(s),Ψ

[logP

(X (s)c ,P(X (s)

c ) = p)

+ Z]

=∑s∈S

EX|C(s),Ψ

[∑ci∈s

(I(X (s)ci = 1,P

)log(θc) +

I(X (s)ci = 0,P

)log(1− θc)

)+ Z

]

=∑s∈S

∑ci∈s

(P(X (s)ci = 1,P | C(s),Ψ

)log(θc) +

P(X (s)ci = 0,P | C(s),Ψ

)log(1− θc)

)+ Z



M-step

∂Q(θc)

∂θc=∑s∈S

∑ci∈s

(P(Y

(s)ci = 1)

θc− P(Y

(s)ci = 0)

1− θc

)= 0

θ(t+1)c =

∑s∈S

∑ci∈s P(Y

(s)ci = 1)∑

s∈S∑

ci∈s∑x=1

x=0 P(Y(s)ci = x)

=

∑s∈S

∑ci∈s P

(X

(s)ci = 1,P(X

(s)ci ) = p | C(s),Ψ

)∑

s∈S∑

ci∈s P(P(X

(s)ci ) = p | C(s),Ψ

)

Probabilities are computed using parameter values θ(t)c

calculated on previous iteration t



EM summary

Q(θc ) =∑s∈S

EX|C(s),Ψ

[log P

(X, C(s) | Ψ

)]

=∑s∈S

EX|C(s),Ψ

[ ∑ci∈s

(I(X (s)ci

= 1,P(X (s)ci

) = p)

log(θc ) + I(X (s)ci

= 0,P(X (s)ci

) = p)

log(1− θc )

)+ Z

]

=∑s∈S

∑ci∈s

(P(X (s)ci

= 1,P(X (s)ci

) = p | C(s),Ψ)

log(θc ) + P(X (s)ci

= 0,P(X (s)ci

) = p | C(s),Ψ)

log(1− θc )

)+ Z

∂Q(θc )

∂θc=∑s∈S

∑ci∈s

(P(X

(s)ci

= 1,P(X(s)ci

) = p | C(s),Ψ)

θc−

P(X

(s)ci

= 0,P(X(s)ci

) = p | C(s),Ψ)

1− θc

)= 0

θ(t+1)c =

∑s∈S

∑ci∈s P

(X

(s)ci

= 1,P(X(s)ci

) = p | C(s),Ψ)

∑s∈S

∑ci∈s

∑x=1x=0 P

(X

(s)ci

= x,P(X(s)ci

) = p | C(s),Ψ)

=

∑s∈S

∑ci∈s P

(X

(s)ci

= 1,P(X(s)ci

) = p | C(s),Ψ)

∑s∈S

∑ci∈s P

(P(X

(s)ci

) = p | C(s),Ψ)



Lecture outline




EM for User Browsing Model

document ur

Er

Cr

Ar

...

↵urq

�rr0

P(Au = 1) = αuq

P(Er = 1 | Cr ′ = 1,Cr ′+1 = 0, . . . ,Cr−1 = 0) = γrr ′

P(Au) = ∅P(Er ) = {C1, . . . ,Cr−1}




α(t+1)uq =

1

|Suq|∑s∈Suq

(c

(s)u + (1− c

(s)u )

(1− γ(t)rr ′ )α

(t)uq

1− γ(t)rr ′ α

(t)uq

)



EM for User Browsing Model: Examination

P(Er = 1 | Cr ′ = 1,Cr ′+1 = 0, . . . ,Cr−1 = 0) = γrr ′

P(Er ) = {C1, . . . ,Cr−1}p = [c1, . . . , cr ′−1, 1, 0, . . . , 0]

Srr ′ = {s : cr ′ = 1, cr ′+1 = 0, . . . , cr−1 = 0}

P(Er = x ,P(Er ) = p | C) = P(Er = x | C)

P(P(Er ) = p | C) = 1




γ(t+1)rr ′ =

1


(c

(s)u + (1− c

(s)u )

γ(t)rr ′ (1− α(t)

uq )

1− γ(t)rr ′ α

(t)uq

)



EM for User Browsing Model

α(t+1)uq =

1

|Suq|∑s∈Suq

(c

(s)u + (1− c

(s)u )

(1− γ(t)rr ′ )α

(t)uq

1− γ(t)rr ′ α

(t)uq

)

γ(t+1)rr ′ =

1


(c

(s)u + (1− c

(s)u )

γ(t)rr ′ (1− α(t)

uq )

1− γ(t)rr ′ α

(t)uq

)



Parameter estimation summary

Maximum likelihood estimation

Parameters estimated directly from dataParameters do not depend on each otherSingle pass over a click logVery efficient but not very effective

Expectation maximization

Parameters depend on each otherIterative estimationEffective but not efficient



Lecture outline




Alternative estimation methods

Bayesian inference

Probit link

Matrix factorization



Course overview

Basic Click Models



Advanced Models

Recent Studies

Future Research



Next lecture

Basic Click Models



Advanced Models

Recent Studies

Future Research



Acknowledgments

All content represents the opinion of the authors which is not necessarily shared orendorsed by their respective employers and/or sponsors.


Click Models for Web Search - Lecture 2 · Click Models for Web Search Lecture 2 Aleksandr...

Documents

Transcript of Click Models for Web Search - Lecture 2 · Click Models for Web Search Lecture 2 Aleksandr...