The Stack Archive

Science makes first study of know-it-all internet commenters

Mon 5 Sep 2016

Researchers from Stanford and Microsoft have created a scientific methodology to measure the frequency and output of intransigent commenters in social networks – those whose contributions are written as statements of fact, rather than contributions to a discourse.

In Identifying Dogmatism in Social Media: Signals and Models, Human Computer Interaction researcher Ethan Fast and Technical Fellow at Microsoft Eric Horvitz trained their analysis system using only 5000 annotated posts from Reddit communities but were able to apply the derived principles to millions of other comments on the popular social site.

The research, which used crowdworkers on Amazon Mechanical Turk, rated users’ contributions on a scale of 1-5, effectively ranging from ‘let’s talk’ to ‘flame on!’.

Predictably, the nature of the discussion influences the likely level of dogma that commenters engage in. The Cringepics subreddit, wherein users mock screenshots from unfortunate romantic communications, produces very little uncertainty and a great deal of dogmatic language, as does the DebateAChristian subreddit.

The lowest level of didactic comment is found in hobby groups concerned with pursuits such as home PC builds, brewing and photography.

Fast and Horvitz set themselves a number of knowledge tasks, among which was to find out whether dogmatic commenters are uniformly stubborn across a range of subjects; to this end they generated a body of 10 million analysed posts from 2007-15 across 1000 users:

The commenting communities where dogmatic users are most likely to continue a policy of retrenchment in their established beliefs

The commenting communities where dogmatic users are most likely to continue a policy of retrenchment in their established beliefs

Many of the subjects are likely to provoke dogmatic comment; even where the topic is radically different, the level of controversy is elevated.

However, the researchers did find evidence that some of the most stubborn commenters display their convictions across less likely subjects, and seem to be (as their antagonists often suspect) just plain ornery:

‘For example, among the users who are dogmatic on politics, they are also disproportionately dogmatic on unrelated subreddits such as science, technology, IAmA, and AskReddit.’

The paper finds that dogmatic commenters post frequently and in favourite communities, but since they are “not as inclined to engage with discussion, once it has begun”, also seem to most value placing a first or at least early comment.

The paper also studies the language and vocabulary of the absolutely-certain internet commenter, finding that references to ‘they’ and ‘you’ are far likelier to prevail in a dogmatic post. Examples given include ‘You are a moron’ and ‘They are keeping us down’.

The scientists also sampled 600,000 one-on-one conversational Reddit exchanges to see if an antagonist’s dogmatism becomes infectious and forces a more rational correspondent to abandon discourse in favour of retrenchment. It does, with the researchers establishing that ‘engagement with a dogmatic comment tends to make a user more dogmatic themselves’.

Fast and Horvitz conclude, some would say optimistically, that the results of their computational model could help users engage in more pro-social behaviour in online communities.


internet news research
Send us a correction about this article Send us a news tip