What Philippine fake news sounds like, according to a classifier
- Article
- Media
People are bad at telling fake news from legitimate news, and that weakness is exploited to shape what the public believes. Aaron Carl T. Fernandez and Madhavi Devaraj set out to measure the difference in writing style between the two in the Philippine context and to test how well that difference works as input to a machine learning model.
Approach
The researchers built their own dataset, the Philippine Fake News Corpus. It holds 14,802 credible news samples from The Philippine Daily Inquirer, Manila Bulletin, and The Manila Times, and 7,656 fake news samples from sites such as Adobo Chronicles, GR Pundit, Get Real Philippines, VerifiedPH, and Pinoy Trending Altervista, all dated between January 1, 2016 and October 30, 2018.
From each article they computed 76 linguistic features, grouped into eight categories: readability scores, linguistic dimensions, summative cues, affective cues, informality cues, cognitive cues, punctuation cues, and time-orientation cues. Three models were trained on those features: Gaussian Naive Bayes, logistic regression, and linear support vector machines.
Findings
Legitimate and fake news differed significantly in word count, sentence count, verb tense, and readability. Fake news has wordier headlines and fewer sentences in the body. It also reads slightly angrier than legitimate news and focuses more on the present.
Logistic regression performed best of the three models. It classified credible and non-credible news at 87 percent precision from the headline alone and 88 percent from the content. The most informative features were the headline syllable count and the headline word count, followed by words per sentence in the headline and in the body.
What comes next
The study shows that a classifier can discriminate fake from credible news from textual features alone. The authors recommend adding more linguistic features, such as grammar and spelling scores, to improve the model and to surface more differences between the two kinds of writing. The corpus is public on GitHub.
Source: Fernandez, A. C. T., and Devaraj, M. (2019). Computing the Linguistic-Based Cues of Fake News in the Philippines Towards its Detection. Proceedings of the 9th International Conference on Web Intelligence, Mining and Semantics. doi:10.1145/3326467.3326490. Dataset: Philippine Fake News Corpus.