QA Lab-4: QALab-PoliInfo QA Lab so far QA Lab is aimed at complex real-world question answering (QA)...

of 18 /18
QA Lab-4: QALab-PoliInfo https://poliinfo.github.io/ YASUTOMO KIMURA *5 *6, HIDEYUKI SHIBUKI*1, KOTARO SAKAMOTO*1, *2, MADOKA ISHIOROSHI*2, TERUKO MITAMURA*3, NORIKO KANDO*2, *4, TATSUNORI MORI*1, *1: YOKOHAMA NATIONAL UNIVERSITY, *2: NATIONAL INSTITUTE OF INFORMATICS, *3: CARNEGIE MELLON UNIVERSITY, *4: THE GRADUATE UNIVERSITY FOR ADVANCED STUDIES (SOKENDAI), *5: OTARU UNIVERSITY OF COMMERCE, *6: RIKEN AIP

Embed Size (px)

Transcript of QA Lab-4: QALab-PoliInfo QA Lab so far QA Lab is aimed at complex real-world question answering (QA)...

NTCIR-14-QALab-PoliInfo-Ver20180319QA Lab-4: QALab-PoliInfo https://poliinfo.github.io/ YASU T O M O K I M U R A * 5 * 6 , H I D E Y U K I S H I B U K I * 1 , K O T A R O S A K A M O T O * 1 , * 2 ,
M A D O K A I S H I O R O S H I * 2 , T E R U K O M I T A M U R A * 3 , N O R I K O K A N D O * 2 , * 4 , T A T S U N O R I M O R I * 1 ,
* 1 : Y O K O H A M A N A T I O N A L U N I V E R S I T Y , * 2 : N A T I O N A L I N S T I T U T E O F I N F O R M A T I C S , * 3 : C A R N E G I E M E L L O N U N I V E R S I T Y ,
* 4 : T H E G R A D U A T E U N I V E R S I T Y F O R A D V A N C E D S T U D I E S ( S O K E N D A I ) , * 5 : O T A R U U N I V E R S I T Y O F C O MM E R C E , * 6 : R I K E N A I P
Various technologies are required!
QA Lab-PoliInfo in NTCIR-14 QA Lab-PoliInfo is QA for political information using Japanese regional assembly minutes, to show summaries on the opinions of assembly members, and the reasons and conditions for such opinions. The importance of fact checking owing to the negative impact of fake news l International Fact-Checking Day, April 2 from 2017 http://factcheckingday.com/ However, fact-checking is difficult for general Web search engines l because of the ‘filter bubble’ developed by Eli Pariser For fact checking, l we should confirm the primary sources such as the assembly minutes according to critical thinking
Gïù*äâÍYšŒet¼yý»dÛ£å È_š¹l ç
¹¹²²ç±¶½SçM^:7 ð§[email protected]@[§6 @^"çyE« úç0ßA±Òš¹ çµ*çnܹ wsT§¶T§¶eçn¼ AÔýHå#ê#í ^!'[email protected]ç ¹å¹åÝåëe¾l
ær"[email protected]Ãr"nF®I§®!'ÔÅ·¼Âr"ë§ ©?¼ ?å_š ·¤
T<§®[email protected]±[email protected]@K+¹Æ÷A±NÈð¹ §/
ò§ü6<[email protected]îuóškç­Ñà¯Õ AÙÁ SºAl"
[email protected]¹#Ú.šk {c}õ[email protected]¹BäÏa¾¯uškÏ Äuì×ú´ lóƒ[email protected]ÚäϨ {ÑàB×uškð §ÈLìn!Ùá¼ `Pmÿ$S7uPe)u7/¤QGïÿ$Fù ^·È ç­>[email protected]ÜÃRÎAºì;KøOìçFjnS î èx4SnSAD(¾e
Ç»Ðò§uXeVÜã¸*yX^:îý¸025Üã`u¹x4S ùñ Íæ x4S]XÌ ¾S¢ uvîý;" [email protected]= [email protected]¾SÊ
QÌvxT¦ûDxoT,Bª²ŒDcvxþ{ôDçÆvx|t ûDxïúX 8Ó½ w½[email protected]öP~¡b8h¤KgøO ÈM¹¿K8¤èw ï½ëe_š & œìuDíÉu=nìø&l_š¹ 7î%;T?Þ¥d¥&9´°Zq &9´Àú ec¹¹ ò§ù_šI:ù;uJ²U¬¬¦ÈT§~: &°³l7_šÃ :Ì ÷¶ü/
»TpÖ-T-²èr"pp²p²¥ëdÛË UCA{ _šeÖë37\Mšë" lŠØÍ
It is difficult to understand the contents, including the opinions of the assembly member at a glance.
This is a single speech given by an assembly member that is a request to the governor!
Transcript of a speech. However, the speech is very long.
New information access technologies to support user’s understanding are expected.
Support for user’s understanding For confirmation of primary information source
l When a citation is given, we need to identify the corresponding texts in primary sources
l If the texts is too long, we need to summarize them
For critical thinking
l We need to get the whole view of opinions
5
→ Segmentation task
→ Summarization task
→ Classification task
Task description Segmentation Task l Given Japanese regional assembly minutes and a brief citation l Extract a text corresponding to the citation from the minutes
Summarization Task l Given a text including an assembly member’s opinion l Make a summary which guarantees to keep the opinion
Classification Task l Given a text including political keyword l Classifiy the description whether merit or demerit
Question 2362
Segmentation Task
0 1
Segmentation Task
0 1
Indicate the range of the content.
Question 40

Classification task Input : An utterance which includes political keyword
Output : Classification concerning the description whether merit or demerit
1. Merit:
2. Demerit: 108
Keywords “Tsukiji market”
Related work (Shared task) Shared tasks on the credibility of political information
1. Fake News Challenge Stage 1 (FNC-I) 2. CLEF-2018 Fact checking Lab 3. NTCIR-14 QALab-PoliInfo Task
Difference from related work FNC-1 Fact checking NTCIR QALab-PoliInfo
Dataset News article Political debate Assembly minute and News letter
Task Classification
Check-worthiness • Binary Classification
Language English English and Arabic Japanese
Data and Resource We provided the Japanese Regional Assembly Minutes Corpus. l JSON format data of the Tokyo metropolitan assembly for 4 years l Data fields
Participants can use any resources (and need to report)
13
Identifier Prefecture name Volume Number Year Month Day Period Title Speaker expression Speaker ID Speaker name Speaker position Speech URL HTML file
Data characteristics (1) Dialog including questions and answers (2) Beliefs and attitudes of the assembly member (3) Mental spaces for other assemblymen (4) Contexts, including reasons (5) Several topics in the political documents (6) Colloquial Japanese including dialect and slang
14
Evaluation We will discuss the appropriate representation, evaluation metrics and methodologies with the participants
The discussions will be held through round table meetings, mailing lists, and other means.
15
Scope This task will contribute to the development as follows :
l QA technologies, l information extraction, l semantic representation, l context understanding, l information credibility, l automated summarization, l dialog system, l and others
16
Important Dates Feb 20, 2018: QALab-PoliInfo Kickoff meeting in NII (room 1901, 1902)
Mar 20, 2018: NTCIR-14 Kickoff event in NII
Apr 19, 2018: 1st round table meeting in NII (room 1901, 1902)
Jun 2018: Dataset Release
Jul 2018: Dry Run
Nov 2018: Formal Run
Feb 1, 2018: Task overview paper release (draft)
Mar 15, 2019: Submission due of participant papers
Jun 2019: NTCIR-14 Conference & EVIA 2019 in NII, Tokyo
Contact us qalab-admin l [email protected]
Web site l https://poliinfo.github.io/