Can Facebook and Twitter posts stand in for fire reports? A Cebu City test

Gathering and reporting fire incidents across the Philippines is slow and costly for an archipelago. People, on the other hand, post about fires as they happen, and those posts stay searchable long afterward. Jan Joshua Velasco, Daniel Hans Tan, Christine Peña, and Ken Gorro of Cebu Technological University asked whether the text of social media posts could serve as an alternative source of fire incident data, using machine learning to sort the relevant posts from the rest.

Approach

The researchers collected Facebook and Twitter posts containing the keywords “fire” and “sunog” from March to August 2017, filtered them to Cebu City, and preprocessed the text to remove unnecessary words. Term frequency-inverse document frequency (TF-IDF) extracted the features for the model. Two support vector machine (SVM) classifiers were trained, one for each keyword, to label a post as related or unrelated to a fire incident. Both were evaluated with 10-fold cross-validation on accuracy, precision, recall, and F1 score.

The team then obtained the number of fire incidents officially reported by the Bureau of Fire Protection (BFP), grouped by alarm level (0, 1, 2, 3, Alpha, Bravo, General), and compared it against the number of fires reported on social media. They also computed the correlation between the number of posts and the alarm raised.

Findings

A total of 474 posts were collected. The “fire” classifier on Facebook posts did best, at 82.50 percent accuracy and 64.81 percent precision; the “sunog” classifier on Twitter posts was lowest, at 70.89 percent accuracy. Recall and F1 scores ranged from 52 to 63 percent, with some folds scoring low because precision and recall need at least two labels present in a fold.

The comparison with BFP records is the more useful result. Every high-severity fire the BFP recorded (Alpha, Bravo, and General alarms) also appeared on social media. Low-severity fires (alarm levels 0 and 1) often did not, despite 120 such incidents in the BFP data. The number of social media posts correlated positively (72.8 percent) with the official reports and with fire severity: more posts about a fire is a sign that the fire is serious.

What they recommend

Social media posts can be an alternative source for fire incident reports, provided the fire is severe enough (Alpha alarm and above). Future work should find how many posts correspond to a given alarm level, use larger or equal-sized folds to stabilize recall and F1, and collect over a longer period.

Source: Velasco, J. J., Tan, D. H., Peña, C., and Gorro, K. (2017). Social Media Platforms: Alternative Source of Fire Data in Cebu City Using SVM and Correlation Analysis. ResearchGate

Originally shared on Facebook.