Strategy for comments about the US election


#21

If we combine this limited responses ? So if we are in state 2 you get 3 comments per hour that are put directly into pre-mod. Comments under 200 characters that repeat words are automatically banned.

Maybe moderators can can stage a vote where people can choose three commenters they think added the most and only they get to continue for a day .

Also built in sundown clauses on certain comment threads , so gaming of the system isn’t as fun. It’s premodded AND set in three hour bursts so it’s just not worth making a trash fire of ?


#22

Hi Lilah!

I’ll speak to the “major technology solutions that will magically avail themselves pre-election”. We’ve built one! Sydette was even nice enough to cover us on the Coral blog.

We’re using artificial intelligence to create meaning from thousands of user interactions automatically, with dramatically less burden on moderators, and we shine on Op-Ed (we’re less useful on real-estate blogs that have narrative based comments).

In pol.is, users submit comments as usual - we send them out in real-time for others to vote on. Based on the voting patterns across all users, our algorithms then discover, and visualize, opinion groups. Each person, as they vote, moves towards others that voted like they did in the visualization.

You can see it in action here (try agreeing and disagreeing, and clicking on the opinion groups, to see what brought each together): https://preprod.pol.is/2arcefpshi

This works for highly contentious issues with thousands of stakeholders and produces surprising insights and consensus. The tech, used en masse with thousands of citizens by a national government abroad, recently broke a 6 year legislative deadlock.

On average it sees 100x more people engage with voting than with commenting. It engages a new segment of lurkers on the site (ie., 200 commenters --> 2000 voters). You’ll also see ‘real people’ connect Twitter, which draws others in.

Since you’re working with a major brand, however, I would also mention that FT could use our API to create a completely custom experience (or use our widget until that point if it’s desirable) with a text summary of the group output an analysis rather than an explorable visualization.

It’s hosted SaaS - so you just have to share a link or embed a script tag to try it (zero integration pain - minutes to set up).

If you’d like a walk through about how it can help, you can schedule a demo time that works for you here: https://calendly.com/colinmegill/30min and email me at colin at pol.is (I’ll respond with a hangout link).

Good luck this election season :slight_smile:


#23

This is really great advice, Nassir, thanks. I do think readers are getting used to our presence (commenters often now quote the guidelines to each other) and that has helped. And I like the idea of doing this in close collaboration with the writers.

Yeah, LiveFyre has finally made collapsed threading mainstream, but it currently only works with the default indentation. So because we only indent once (whether something is a reply to a parent or a reply to a reply, if that makes sense) it’s not working for us yet. I’m hoping when we can implement that it should help - allowing people to get in the weeds on an argument without forcing everyone else to have to read it.

I noticed in this piece that you guys will put people in temporary ‘time outs’. How do you keep track of that - and does it work?


#24

I’m glad the collapsing is gradually getting there. I really think that’s a good idea and would love to see it tried.

When we did the Digiday piece, I was running a big user management system in Trello. When a commenter seemed like trouble, I created a card and tracked their behavior with screenshots and notes, and of course since Trello is a project management tool I was able to simply schedule them into time outs. I was so proud of that system; it was sophisticated and well thought out, and I really felt that I had the entire site under control when I was using it. For about a year, every argument or insult on the site was occurring either because I allowed it or because I was asleep. That is an icky boast to make in a way, but there it is. Word would sometimes come down that there was concern over problem X or Y, and I would just clean it up because I knew exactly who was causing the trouble. Really only Israel/Palestine never came under control. I’ve managed to stop the obvious racists on both sides but never achieved what we did at Huffington Post, where I think we had the best discussions on the topic anywhere outside of Israel itself. It’s just too much effort, with only me and my small third party team from ICUC.

It’s a long story about how I came to abandon the Trello system, but really I just succeeded out of it. With only a shadowban to use, a “time out” really just tests to see what comments look like if a particular user is removed from active participation. Like a mad scientist, I was intentionally seeking to form a core user group that more than less worked together and which I could handle, slyly shuffling out any dissenters and silently quashing dissent, like a wee Stasi of one. And it worked. After that, the effort to run this weird manually organized kanban tracking system was more than it was worth. By about February or March of 2015, I’d reached a point where there were three groups of commenters:

  1. The core group who I was willing to put effort into and didn’t want to ban even temporarily for fear of losing them. Sometimes if they were having a bad jag I would shadowban them and then spend day and night manually approving or deleting their comments so they wouldn’t notice they were banned. More often I just tried to get into touch with them and work through it. These are powerusers mostly, so there are a lot of comments but it’s a small group so I can handle it.

  2. Grey area commenters: those with about 300-500 comments, activity once or twice a week or maybe a short burst once a month, or who behave badly for some topics only, or people who come in mad but it’s a controversial subject, that sort of thing. If these commenters break bad I’ll just open up their profile, have a quick glance, ban them and stick it into a big list using the Chrome extension OneTab. Then I’ll review them after a day or two to see if the bad behavior has persisted across more than one article or day. If it looks good, I’ll approve any shadowed comments and unban, otherwise I’ll just leave them on ban. Sometimes I’ll give them a second review, or even a few over time if they are putting effort in and showing some improvement. I track behavior over time with tags, and the better I get to know a user the less likely they are to go on ban. The number of comments in this group is pretty small so it’s easier to manage them casually like this.

  3. New users. I’ll ban a newer commenter pretty easily and forget about it. Honestly I was thinking about it the other day and I think I give users with fewer than 100 comments something like 5-10 seconds of thought before deciding to ban them permanently. I screw up of course but think it’s an acceptable rate. The newer the user, the more likely it is that they are going to be banned on the basis of a single glance at the first two or three lines of a single comment. Again, kind of an icky admission to make, but it is the only way to stay ahead.

This more casual system fell apart in the heat of the primary elections and I lost control of the site again, but that’s another story. I made a few attempts to revive the Trello tracking system with bookmarklets and webhook automation and such. Under the influence of Jeffrey Lin (formerly of Riot Games) I tried to set up a proper automated customer service platform that would work around the problem of the shadowban by automatically emailing out notifications and unbanning users when they responded in a certain fashion. Very quickly, I realized what I needed was a developer. Salon had none to spare so I’ve been taking evening coding classes! In fact, I’ll soon be leaving Salon to concentrate on the final round of classes. A proper user management system is the missing element of moderation, I think. I’ve told Livefyre how bad their system is many times, although it has improved a little since I began in 2014, to their credit.


#25

Boy have we heard that more than a few times. Thanks for such a terrific response, Nassir. What languages are you learning in the coding classes? Are you thinking about how you’d apply it to a community platform?


#26

Thanks Andrew. I’m doing a full stack Javascript and front end web development course, but after that I don’t know what I’m doing. Of course I would like to try again, and I am telling myself that whatever happens in this election, the next one is likely to be just as much of a mess or worse. So I’ve got four years to prepare and get in place, or two years if maybe I want to dry run at the midterms. It’s a goal to aim for.

I think almost all the success I got out of Livefyre came from using it in ways the designers didn’t anticipate or intend, but the big gap was that I didn’t know how to do anything with their APIs. I really think I could have just built my own user management system, even knowing as little as I do now. Also, we’re just styling Livefyre like it comes out of the box, which isn’t the best. But noodling around with CSS tweaks in response to confused and conflicted requests from users wasn’t something I wanted to inflict on Salon’s product team. Heroic as they are, it’s hardly a priority. Also I wanted to build out some of the more social aspects of a community, give people the ability to build an identity, body of work and following rather than repeat the same thing over and over again with each new iteration of a topic. We have user profile pages, for example, but they do very little besides showing a users comments. Even so, my powerusers use them to navigate the site and practically panic when something goes wrong with them. It’s a missed opportunity. This is simple stuff that I could have worked on if I’d known how. A developer can get way more done than any moderator, even if they aren’t building something from scratch but just playing with what’s available. It took me too long to realize this.