Slate.com writes about our upcoming study on Internet Research extending our findings presented in "Information Credibility on Twitter" [pdf].
Social media hoaxes: Could machine-learning algorithms help debunk Twitter rumors before they spread?
...
In a new paper, to be published in the journal Internet Research next month, the authors of the Chile earthquake study—Carlos Castillo, Marcelo Mendoza, and Barbara Poblete—test out their algorithm on fresh data sets and find that it works pretty well. According to Meier, their machine-learning classifier had an AUC, or “area under the curve,” of 0.86. That means that, when presented with a random false tweet and a random true tweet, it would assess the true tweet as more credible 86 percent of the time. (An AUC of 1 is perfect; an AUC of 0.5 is no better than random chance.)
My guess is that a knowledgeable and experienced human Twitter user could do better than that under most circumstances. And of course, if a given algorithm became widespread, committed trolls like the Hurricane Sandy villain @ComfortablySmug could find ways to game it. Still, an algorithm has the potential to work much faster than a human, and as it improves, it could evolve into an invaluable "first opinion" for flagging news items on Twitter that might not be true.
...
Source: Slate.com
... A big problem with theories floated on social media is that information can go viral simply because it is popular, whether or not it is true. Patrick Meier of the Qatar Computing Research Institute (QCRI) in Doha is building Verily, a system that allows users to submit verification requests for information they are interested in. Each request prompts a crowd of online workers to set off into their networks to figure it out. The system gathers evidence for and against the claim, though it won't pass judgement.
