The Twenty-ninth Annual Conference on Neural Information...
1
Max-Margin Majority Voting for Learning from Crowds Tian Tian and Jun Zhu from Department of Computer Science and Technology,Tsinghua University. The Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS) Palais des Congrè sde Montréal, Montréal CANADA Apricot Peach Peach Peach Apricot Apricot Apricot Peach Objective Aggregating noisy crowdsourcing labels to find Ground Truths. Majority Voting (MV) Items: ∈ Workers: ∈ Worker labels: ∈ Each item have a ground truth: ∈ [] ：{ ，∀} MV: find the most frequent labels Limitations: 1. Workers are equal, lack of discrimination ability 2. Do not consider worker confusability Constraint Formulation of MV Expansion Expression Def: , ∈ {0,1} , element is ( = ) Constraint Formulation MV is equivalent to find satisfying the constraints: (1 -1 -1 -1) : (1 0 0 0) (0 1 1 1) ,1 : , −1 : Dawid-Skene Model (DS) Define and estimate worker confusion matrices. is the confusion matrix of worker = = = , ∀ CrowdSVM Consider Majority Voting and confusability in a single model. Max Margin Majority Voting (M 3 V) Incorporate max-margin principle to estimate Here we using soft-margin for robustness. (3 1) : (0 1) (0 0) ,1 : ,2 : (1 0) ,3 : Solving by iteratively updating and . Solver for : is the solution of the dual problem: Solver for : Experimental Results Convergence: Generative vs. Discriminative: Conclusion: 1.Max-margin principle can enhance majority voting. 2.Both generative and discriminative component benefits from the other. Contact Info Tian Tian (田天) PhD Student @ THU Email: [email protected] Welcome for discussing! Weighted MV We introduce worker weights ∈ℝ , then the constraint formulation changed into The discriminative function for infer ground truth: We test the aggregation error rate on several popular datasets.
Transcript of The Twenty-ninth Annual Conference on Neural Information...
Max-Margin Majority Voting for Learning from CrowdsTian Tian and Jun Zhu
from Department of Computer Science and Technology, Tsinghua University.
The Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS) Palais des Congrès de Montréal, Montréal CANADA
Apricot Peach Peach Peach
Apricot Apricot Apricot Peach
Objective
Aggregating noisy crowdsourcing labels to find Ground Truths.
Majority Voting (MV)Items: 𝑖 ∈ 𝑀 Workers: 𝑗 ∈ 𝑁 Worker labels: 𝑥𝑖𝑗∈ 𝐷
Each item have a ground truth: 𝑦𝑖 ∈ [𝐷] 𝒙𝑖：{𝑥𝑖𝑗，∀𝑗}
MV: find the most frequent labels
Limitations:1. Workers are equal, lack of discrimination ability
2. Do not consider worker confusability
Constraint Formulation of MVExpansion Expression
Def: 𝒈 𝒙𝑖 , 𝑑 ∈ {0,1}𝑁 , element 𝑗 is 𝕀(𝑥𝑖𝑗 = 𝑑)
Constraint Formulation
MV is equivalent to find 𝒚 satisfying the constraints:
(1 -1 -1 -1)𝒙𝑖:(1 0 0 0)
(0 1 1 1)
𝒈 𝒙𝑖 , 1 :
𝒈 𝒙𝑖 , −1 :
Dawid-Skene Model (DS)
Define and estimate worker confusion matrices.𝜙𝑗 is the confusion matrix of worker
𝜙𝑗𝑘𝑑 = 𝑝 𝑥𝑖𝑗 = 𝑑 𝑦𝑖 = 𝑘 , ∀𝑖
CrowdSVM
Consider Majority Voting and confusability in a single model.
Max Margin Majority Voting (M3V)
Incorporate max-margin principle to estimate 𝜼
Here we using soft-margin for robustness.
(3 1)𝒙𝑖:
(0 1)
(0 0)
𝒈 𝒙𝑖 , 1 :
𝒈 𝒙𝑖 , 2 :
(1 0)𝒈 𝒙𝑖 , 3 :
Solving by iteratively updating 𝜼 and 𝒚.
Solver for 𝜼:
𝝎 is the solution of the dual problem:
Solver for 𝒚:
Experimental Results
Convergence:
Generative vs. Discriminative:
Conclusion:
1.Max-margin principle can enhance majority voting.
2.Both generative and discriminative component benefits
from the other.
Contact Info
Tian Tian (田天) PhD Student @ THU
Email: [email protected]
Welcome for discussing!
Weighted MVWe introduce worker weights 𝜼 ∈ ℝ𝑁, then the constraint
formulation changed into
The discriminative function for infer ground truth:
We test the aggregation error rate on several popular
datasets.
