RumourEval
The more people build rumour analysis technology, the further we advance. In 2017, a shared evaluation will be held, comparing systems that do rumour analysis. This is RumourEval. Anybody can enter – it’s an open exercise. RumourEval is part of SemEval 2017, a long-running semantic evaluation event, and the task was proposed by PHEME members.
The format is that there’s a lot of data available for training and development, all open access. Then, during January 2017, new content is provided related to unseen rumours, and participants use their systems to judge these.
Two sub-tasks are to be attempted: first, to label whether a document supports, denies, queries or comments on a claim – or, for short, SDQC. This is vital to dissecting a rumour. The second subtask is, given some content about a claim, to give the veracity of the claim – how true it is. This is hard. There are two variants: one, closed, where systems only have some documents related to the claim; and the other, open, where a snapshot of Wikipedia and a Newsdiff are given as fixed world-knowledge.
The official page with technical details and a link to download data is here – http://alt.qcri.org/semeval2017/task8/ – come join us!
