Quality-Aware Collaborative Quality-Aware Collaborative Question Answering: Methods Question Answering: Methods

and Evaluationand Evaluation

Maggy Anastasia Suryanto, Ee-Peng Lim, Aixin Sun, and Roger H. L. Chiang.

In Proceedings of the Second ACM International Conference on Web Search and Data Mining

(Barcelona, Spain, February 9-12, 2009).

Prepared and Presented by Baichuan LiApril 19, 2023

OutlineOutlineIntroductionQuality-Aware FrameworkExpertise Based MethodsExperimentsConclusion

23年4月19日 Paper Presentation2/21

IntroductionIntroductionCommunity-Based Question-

Answering (CQA) Services

23年4月19日 Paper Presentation3/21

Diverse Answer QualitiesDiverse Answer Qualities

23年4月19日 Paper Presentation4/21

good

poor

fair

ObjectiveObjectiveAutomatically find good answers

for a user given questions from a community QA portal◦answer features◦user expertise of answers

23年4月19日5/21 Paper Presentation

Quality-Aware FrameworkQuality-Aware Framework

23年4月19日 Paper Presentation6/21

Expertise Based MethodsExpertise Based Methods

23年4月19日 Paper Presentation7/21

Relevance score Quality score

Expertise Based MethodsExpertise Based Methods

23年4月19日 Paper Presentation8/21

Question Independent Question Independent ExpertiseExpertise

23年4月19日 Paper Presentation9/21

EXHITS uses qscore_exhits(a) as the quality score of an answer a given in below equation:

authority

hub

Question Dependent Question Dependent ExpertiseExpertise

23年4月19日 Paper Presentation10/21

Question Dependent Question Dependent ExpertiseExpertise

23年4月19日 Paper Presentation11/21

EX_QD

EX_QD’

Answer Relevance ModelsAnswer Relevance ModelsAnswer ranking by Yahoo!

Answers Query likelihood retrieval model

23年4月19日12/21 Paper Presentation

all answers and questions in the dataset

ExperimentsExperiments

23年4月19日 Paper Presentation13/21

Methods Compared◦ BasicYA

BasicYA(subject + content) BasicYA(subject + content + best answers)

◦ BasicQL Adopts query likelihood retrieval model to score

the relevance of an answer

◦ NT (classification based on non-textual answer features) maximum entropy approach 9 features

DatasetDataset

23年4月19日 Paper Presentation14/21

EvaluationEvaluation

23年4月19日 Paper Presentation15/21

The top 20 of the ranked answers of each methods were manually judged in terms of their relevance and quality.

The following evaluation metrics are used to evaluate the accuracy of the methods:

ResultsResults

23年4月19日 Paper Presentation16/21

ResultsResults

23年4月19日 Paper Presentation17/21

ResultsResults

23年4月19日 Paper Presentation18/21

ConclusionConclusion Introduce a quality-aware QA framework that

considers both answer relevance and quality in selecting answers to be returned.

Develop several QA methods (namely, EXHITS, EXHITS QD, EX QD and EX QD') that consider answerer expertise to determine answer quality.

Conducted extensive experiments and these experiments showed that quality-aware methods can improve both quality and overall performance. Among them, the methods EX QD and EX QD' using question dependent answerer expertise have the best performance.

23年4月19日 Paper Presentation19/21

IdeasIdeas

23年4月19日 Paper Presentation20/21

Q&AQ&A

23年4月19日 Paper Presentation21/21