12,000 People Have Something To Say


First published on our blog

When we told you we care about user research… well, we meant it.

In order to find out how commenters behave and what they want, we decided to ask them. Across 20 different news orgs from around the U.S., in fact, and more than 12,000 people responded in one of the largest surveys ever carried out on commenters across news sites.

We surveyed commenters at Alaska Dispatch News, AL.com, The Arizona Republic, The Atlantic, Civil Beat, The Dallas Morning News, Deseret News, Fort Worth Star-Telegram, KXAN, PBS NewsHour, Philly.com, The Seattle Times, Southern California Public Radio/KPCC, The State Journal-Register, The Texas Tribune, TribLIVE, Twincities.com / St. Paul Pioneer Press, Voice of San Diego, The Washington Post, and Willamette Week.

The study was conducted for us by The Engaging News Project at the University of Texas-Austin. (Read our interview with its director, Dr Natalie Stroud.) As part of our mission to help newsrooms with audience engagement, each participating organization received a detailed report about their own onsite communities (which nobody else, including us, got to see.)

We’ve learned so much from the results. Among the highlights:

<li>More than 70 percent of respondents said they’d like journalists to respond to questions about facts in comment spaces. </li>
<li>More than 60 percent said they’d like to see expert opinions in comment threads. </li>
<li>Nearly half of respondents said journalists should highlight selected comments.</li>

Opinions among regular commenters on whether the sites they comment on are civil vary widely: between 14 percent and 78 percent of respondents per site rated the comments there as very or somewhat civil.

Some respondents were confused about how to report comments to moderators: only 62 percent said that their news site had a means to report offensive comments, even though a reporting function was available on all of the sites surveyed.

The survey served up more evidence that anonymity plays less of a role in uncivil discourse than some might think: in response to a question about the site’s civility, there was no difference between the results for sites that used Facebook comments, which requires real(istic) names, and systems that allow pseudonyms, such as Disqus, Civil, or Livefyre. Forty-one percent of users on sites that use Facebook comments rated comments as civil; on other sites, the result was 40 percent.

All of this is influencing the feature roadmap for our comment system, Talk, which we’re getting close to releasing in beta form with a handful of testing partners. The first alpha release of Talk happened at the end of December.

We’re adding features and functionality to Talk into the summer, as sites deploy it and we see real commenters start to kick the tires. So much more on that very, very soon. If you’d like to learn more in the meantime, get in touch.

You can read the full survey results here.
Read more at The Engaging News Project website here.

Adovcatus Commenti

Thank you for running and sharing this study.

This is a good problem for a few reasons:

  • It’s more straightforward with less effort involved to make an existing feature more usable than having to build the feature from nothing.

  • As these comment flagging / reporting features do exist, product people and publishers can reasonably think they’ve worked to address problems of incivility. Since they feel they’ve worked to address it, they can feel ongoing incivility is an intractable issue.

  • With 38% of respondents not realizing those flagging filtering tools even existed, rethinking those existing features is clearly worthwhile.

The three points @andrew_coral highlights are outstanding. The third -

– coupled with the rethinking the UX of flagging / reporting make a powerful 1-2 punch for increasing civility and signal:noise in conversations:

  1. Highlighting thoughtful comments promotes a virtuous cycle of thoughtfulness.
  2. Reporting / flagging / burying incivility minimizes incentive for that kind of participation.

From the straightforwardness PoV, The Dallas Morning News and Philly.com, e.g., could follow up with some of their survey respondents who wished for a reporting feature, point out that the feature exists, and explicitly request those respondents’ help in using the reporting feature on a particularly problematic discussion.

This is 0 additional development, design, and UX effort. It’s targeting a pool of pre-qualified, known-engaged community members. It’s an opportunity to collect detailed feedback – via explicit actions taken and feedback given – while addressing a real need with specifically-troublesome discussions.


Thanks Steve - I completely agree, and it’s definitely helpful for us as we design and build Talk. There are some good wins that can come from making such features more prominent - as well as increasing the incentives to using them.