Detecting bias in Philippine political news with machine learning
- Article
- Media
For many people the internet is the main source of news, and much of what circulates there is slanted or unreliable. A 2019 conference paper by T. A. Quijote, A. D. Zamoras, and A. Ceniza of the Department of Computer and Information Sciences at the University of San Carlos in Cebu City asks whether a program can tell if a political news article, and by extension the outlet that published it, is biased. The work was presented at the International Conference on Information Technology and Digital Applications in Manila in November 2018.
The data
The authors collected political articles from five Philippine news sites, chosen from Alexa’s rankings of the most visited: Inquirer, Philstar, Manila Bulletin, The Manila Times, and Journal Online. The articles were cleaned up in the usual ways for text processing, such as removing punctuation and common filler words, before scoring.
Scoring sentiment
Each word was looked up in SentiWordNet, a lexicon that assigns every English word a positive, negative, and neutral score. Adding these up gives an article a sentiment score. The premise is that a straight news report should sit close to neutral, and that an article leaning strongly positive or negative about its subject is showing its hand.
Deciding bias
The sentiment scores then went into an inverse reinforcement model. Ordinary reinforcement learning learns a policy from a known reward. Inverse reinforcement learning does the opposite: it observes behavior and infers the reward that would explain it. Here the observed behavior is the pattern of sentiment across an outlet’s articles, and the inferred reward is a bias value. The authors mapped that value to four levels: under 0.1 is allowable bias, 0.1 to 0.2 is relatively considerable bias, 0.2 to 0.25 is biased, and above 0.25 is extremely biased.
Results
On their test set the system reached an accuracy of 0.89, a precision of 1.0, a recall of 0.60, and an F-measure of 0.75. In plain terms, when it called an article biased it was always right, but it missed about four in ten of the biased articles. Applied to the five outlets, the model rated all of them as unbiased except The Manila Times.
What to take from it
A sentiment lexicon is a blunt instrument. It cannot tell whether negative words describe the reporter’s opinion or the events being reported, and it knows nothing about framing, selection, or omission, which are the more common forms of bias in news. The study is best read as an early attempt to put a number on something people argue about, using tools that were available and cheap. It is also one of the few published attempts to do this on Philippine media specifically.