Designing our TALK product


#1

This is a thread where we’ll discuss how comment modules are designed, and what you’d like to see in a comment box.


Let’s Begin at the Beginning
#2

Repost from Let’s Begin at the Beginning

original writer please reply – button


#3

Yes and no. I think that currently some posts are directed to the original writer. But because they never receive an answer, most comments are directed to other commenters and comment readers. The amount of comments directed to the original writer would increase though, as soon as the writer would be known for engaging in the comment section.

But even if every commenter would expect a respond to her/his comment, that would just be the opinion of this single commenter. The important thing is, if other commenters and comment readers judge the comment as worthy of a reply by the original writer and upvote this comment by clicking on the originial-writer-please-reply button. Maybe my comparison to the flag button gave you the wrong impression: Each click on the button should increase a publicly displayed counter in the comment box, and the comments should be sortable by this metric (it should even be the default sorting imo).

I don’t think that abusive comments would get much upvoting by the community. But if they do, the community has already turned toxic and needs disaster management.


Based on this metric, a newspaper with limited resources – but willingly to engage in the comment section – could make up rules for responding. For example:

Reply to all comments that reach a specific threshold, like 60.

or

Reply to the top three comments, if they meet a specific threshold, like 70.

or

Reply to the top comment indepth (a couple rounds of back-and-forth with the commenter)


#4

These are fascinating ideas @Christoph ! Thank you for sharing your input. We’re very interested in creating safe, inclusive spaces where people want to comment.

From my understanding, you’re suggesting readers have the option to upvote comments by marking a comment as one journalists should respond to. While I like the idea of a community deciding which comments are important to pay attention to. my number one concern with this idea is harassment. I worry that every single person will think their response not only deserves, but demands a response.

How would a journalist see this notification? What happens when someone marks every comment on a story as one a journalist should read? How do the notifications change as a comments thread grows and changes? For example, does a comment marked as deserving of a reply at 11am outweigh one marked as deserving a reply at 11pm later that day? Would all commenters see the 11am comment marked as deserving of a reply if it’s now 11pm and the comment has gotten lost in the shuffle? How should journalists and community managers deal with community members who consistently mark their abusive comments as ones journalists should reply to?

Every feature has nuances worthy of consideration and discussion. I’d love to hear your suggestions for how we can design a system which encourages civil, informed voices to be heard while simultaneously discouraging abuse.


#5

Hi @Egrdina,

thank you for replying. Glad you generally like the idea.

I think the metric should be implemented together with rules for responding, as I suggested above. The newsroom should define a threshold of upvotes and a defined amount of responses for each article. So for comments meeting those conditions, the commenter should indeed expect a response. If the rules are defined and the comment does not meet the conditions, the commenter should not expect to get a reply.

  1. The sorting of the comments by “most please-reply” shows the journalist the top comments
  2. If rules are defined, the journalist could get a notification, when the defined threshold has been exceeded.

Excellent questions! I have not thought about this. Lets look at an example:

The newsroom has defined a threshold of 40 upvotes and the obligation to respond to the top 3 comments (if the threshold is met). The response to the top comment has to be in-depth (obligation to talk back in forth with the OP for four rounds).

The article about founding a debt buyer company to buy and cancel people’s debt goes online at 11am. At 2pm the first comment passes the threshold of 40 upvotes. At 4pm three comments have passed the threshold. The journalist gets a notification. She looks at the comments, but does not want to respond to the comment with the third most votes. So she waits until noon the next day. Top 1 has stayed, but top 2 and top 3 are different comments now. She is content with the comments and writes her responses. Three days later none of the replied comments are among the top 3. They got lost in the shuffle :frowning:

Hm, I think you a right: time is an important factor. So I guess the newsroom should define a time, when the conditions are evaluated. 24h comes to mind. So the rule would be: the journalist has the obligation to respond to the top 3 comments that have passed the threshold, exactly 24h after the article has been published.

And… the comments that have qualified for responding are staying forever at the top (of the default sorting). Maybe with a different background color :slight_smile:

  1. I think that abusive comments wont reach the top ranks or pass the threshold. It would require a couple of trolls coordinating to accomplish that.
  2. But better be safe. Maybe a maximum of this kind of upvotes: A limit of three upvotes per person per article!

What do you think about the refinement? Could I dispel your concerns? Did I raise new ones?


#6

@Egrdina FYI The original-writer-please-reply metric is similar to the voting system I suggested here:


#7

Thank you for your explanations @Christoph

What I think your example fails to realize is that every community, blog, vertical, author, etc. has a different threshold for conversation and engagement. Discussions vary across sites given a multitude of factors some of which include: who the author is, whether a story has gone viral, community norms, typical number of comments, etc.

It’s impossible for a newsroom to create one metric which every story must meet. Not only is this an unrealistic standard for journalists, but it also places a high amount of pressure on individual stories. The internet is incredibly unpredictable. Sometimes, the best stories don’t have the most comments. Sometimes, they do. It is very difficult to determine when a story will go one way or the other.

Even if a newsroom were to create metrics for every single part of its site, this is a massive undertaking for a Community Manager. What happens when community norms are no longer the same? Or a journalist leaves their newsroom and takes a community with them to another publisher’s site? How do our rules change then? We’ve heard from a variety of newsrooms some of which have very little time and resources. I would hope The Coral Project is able to deliver a suite of tools which accommodates newsrooms and audience engagement teams of all sizes.

Just because we don’t expect something does not mean that it won’t happen. As a designer, considering the worst-case scenario before it happens, and designing a potential solution is a crucial part of my job.

I believe signaling to journalists which comments deserve replies is an important part of Talk. I’m not convinced that this idea is the best way to do it.


#8

Hi @Egrdina,

thank you for sharing your thoughts on the matter. I’d love to hear more about your ideas about signaling which comments deserve replies. But let me first respond to you criticism.

I agree. The level of engagement is different in every article. Typically the science sections has less then world politics, but it depends on many things as you explained. But different levels of engagement are not an argument against defining an overall threshold for responding.

Imagine the threshold more as a safety measure for the original writer. If the the newsroom would only implement the rule: “original writer has always respond to the top three comments”, this rule could be easily exploited by abusive commenters in articles with very low engagement. They could then write low quality comments and feel being entitled for a response. A threshold is necessary to make sure that a certain amount of readers are interested in a response. It is a matter of legitimacy. I mean, lots of newspapers are shutting down their comment section and they are justifying it by comparing the proportion of the commenters to the silent readers. Rescuing the comment section requires therefore means to legitimate the commenters.

So let the staff of the newsroom discuss on how many votes are necessary to make a comment worthy of a response. If they come to the conclusion that comments are never worthy, we are at status quo. It can only get better, right? :smiley:

Why do you think that the threshold would become a measure of quality for the article? Newspapers are already measuring visits, likes and amount of comments, reading time and many more. This metric is directed at the comment. So hopefully the metric puts some pressure on the commenters for writing better comments :wink:

I do not understand your point here. Could you explain a little further? Why should the rules change, when a journalist leaves? Maybe she was a strong advocate of the approach and now the majorities are different. If the newsroom decides to change the rules, they should communicate it to their readers.

Especially newsrooms with little resources should use the “please-reply” metric, because it scales. A writer who is outnumbers 100 to 1 by the commenters of her article, only has to read and reply to one or two comments and only if the threshold is met. That cannot be too much to ask for (otherwise the newspaper should think about shutting down their comment section).

Repost: But better be safe. Maybe a maximum of this kind of upvotes: A limit of three upvotes per person per article!

I hope you see things differently now, but tell me about your ideas.