The Witchfinder encounters a supremely poorly coded, ‘comedy’ SJW tool for evaluating levels of Reddit ‘bigotry’. The tool, Free Reddit Check, is made by a developer at Akismet manufacturers Automattic and even labels the phrase, ‘sargon of akkad’ as ‘potentially offensive’.

The Akismet official blog promotes their employee’s ‘hack’, saying “[…] being obsessed with content analysis, community moderation, and keeping the web’s underbelly in check, we can’t help but think it’s a nifty idea […]”.

Earlier today I was setting up Twitter preview cards for this site when I noticed a blog by Akismet (archive here). Akismet is the name of a commercial anti-spam tool. It can be used for excluding spam from WordPress blogs. I had evaluated it before but found it unsatisfactory.

One of their employees had participated in ‘Cultivated Wit’s Comedy Hack Day in Los Angeles’ and created a ‘Free Reddit Checker’ tool (archive here). The tool crawls a given user’s posting history and evaluates them with a percentage of how ‘Terrible’ they are. Although it is said to be a personal site and billed as a comedy, the Akismet blog promotes it, saying –

“[…] While perfectly suited for a hack day which pairs developers and comedians, there is certainly usefulness in determining the respectability of a potential online acquaintance. Or just knowing who to ignore. And being obsessed with content analysis, community moderation, and keeping the web’s underbelly in check, we can’t help but think it’s a nifty idea […]”

Put another way … there is a serious ‘underbelly’ to this ‘joke’ site. We are talking about ‘comedy’ ideological purity here.

It is clear that the developer, Dan Walmsley, was using the same skills he uses in his workplace to identify spam in his site to purportedly identify bigots. Unfortunately for him, he pretty much sucks. The tool has … design flaws.

Below is my result. Also below is the result from a random Ghazian. Without searching on the sample post text, can you tell which is which?

This user posts in KiA and GamerGhazi. According to the Free Reddit Checker, they hate women!

This user posts in KiA and GamerGhazi. According to the Free Reddit Checker, they hate women!

The tool ranks users based on key words or phrases and posting history. Unfortunately it is unable to distinguish context. The second user is a random Ghazian. I will not name them. Unfortunately, they used the term ‘gamergate’ whilst condemning us. Merely saying, ‘sargon of akkad’ is apparently also flagged as hate speech. They also visited KiA to condemn us. As a result of this, they were identified as a person who ‘probably hates’ ‘women’.

My account is the one above. Ironically, the tool has actually flagged fewer instances of ‘potentially offensive’ phrases for me. The ones it has flagged are wrong. I have written various posts condemning hate speech – the sample post it quotes is my criticism of Nick Denton’s Gawker.com commissioning a twitter script to tweet passages from Mein Kampf. A comment condemning a GamerGate opponent for using the words, ‘nigger’ and ‘faggot’ also flagged my account as using potentially racist language.

Both myself and the Ghazian used the word, ‘gamergate’ so we were both flagged as haters. It is like flagging someone as a Nazi for saying they are ‘anti-Nazi’. Tough break for the Anti-Nazi League. Hell, I was flagged for saying, ‘anti-GG’. Under this even Zoe Quinn and Anita Sarkeesian are evil misogynists. The program would take one look at Sarkeesian’s recent presentations and start a klaxon ringing!

I contacted the author of the tool, Dan Walmsley (@danwalmsley), and he gave me this unhelpful comment –

“[…] I deeply appreciate you confirming that my algorithm, while rudimentary, is also surprisingly accurate a lot of the time.

On a more serious note, the issue of whether certain subreddits are primarily offensive or not is clearly up for debate, with well-meaning people on both sides. The fact that we took a particular side does not in any way diminish the fact that there _are_ two sides, and that there are people who consider anyone who posts to KiA to be terrible people (FWIW, I’m not one of them, since I’ve barely even been aware of Reddit until last weekend and I have no real experience with folks on either side – though I did go to the Rally to Restore Fear/Sanity and loved it, and had a soft spot for the Reddit logo ever since).

We could pick any arbitrary set of subreddits and say they’re terrible or not and there would be good people on both sides, so I’d argue it’s not that we’re being unfair, but more that we picked the side that you’re not on.

I also could dig in on whether your paraphrasing of derogatory terms is offensive or not, but you’re the almost-a-lawyer here, not me. I’d say if I was queer or a person of color I’d be shocked to see you using them without even attempting to recognise the power of those words to shock and offend.

Look, I’m sure you’re not actually 100% Terrible, maybe even nice a lot of the time, and I haven’t done anything to link this automated analysis of your Reddit account to your real identity (nor could I). The best suggestion I have is that it only took me a weekend to build this site, maybe you could build one with a different algorithm, host it at betterfreeredditcheck.com and see which one people use?

Regards,

Dan

p.s. I tried fixing the bug which adds quoted text to the score, but you were still 100% Terrible.”

… whilst a spokesperson for his employer’s PR took the more pragmatic approach of disassociating themselves, (which might work better if they removed their blog link to the site) saying –

“Thanks for reaching out. This is not an Automattic site however we did pass on your request to those responsible for it.”

The quality of Walmsley’s tool for detecting #GamerGate supporters left me … a little doubtful of his algorithmic skills. Akismet’s blog post specifically referred to the similarity of purpose between their product and Walmsley’s ‘comedy’, ‘hack’. I did a casual Google search to see if anyone had poor experiences with Akismet. Unsurprisingly, some people felt that ‘Akismet really sucks’. Others felt that ‘Akismet sucks’, whilst more users asked, ‘Does Akismet Suck Now?’ (archive here [1], here [2] and here [3]).

Your author is being a little flippant and has admittedly deliberately sought critical sites. Clearly, however, there are questions to be asked about the quality of Akismet developer work. There are also questions to be asked about the judgement of Automattic management in permitting their brand to promote inflammatory rhetoric.

Shortly after my initial media inquiry, the ‘Free Reddit Check’ site changed its behaviour, branding me ‘awesome’ and ceasing to return results for any other username I tried –

Better. After I raised a media inquiry with Akismet the Free Reddit Check site was changed to return me as 100% Awesome and no longer seemed to work with any other username.

Walmsley chose to brand a subreddit with 50,000 members a ‘hate’ sub and now he has been called out he is making light of it. The only ‘comedy’ I see here is the standard of his code.

Based on the accuracy level of ‘Free Reddit Check’, as a consumer, I would fear to install it! I am stuck with a ‘comedy’ image of alternate Akismet products. Imagine a web nanny program written by Walmsley or Akismet that trapped children in the .xxx domain?

Many KiA users are technical professionals in a position to influence buying decisions and many have WordPress blogs. At the same time the recent SPJ Airplay has demonstrated that #GamerGate is gaining acceptance as a consumer movement. Why insult thousands of demographic-core customers?

If you feel that KotakuInAction is not a hate site, if you feel that ‘sargon of akkad’ is not offensive speech, or if you just feel that Akismet should focus on spam detection (and lets face it, it really needs to focus) then why not let Dan (@danwalmsley) and his boss, Automattic CEO Matt Mullenwegg (@photomatt) know?

The official WordPress Akismet plugin has over a million installs … but only about 400 reviews on the WordPress site. At present, 374 people have given Akismet 5 stars. That is less that 1% of the membership of /r/KotakuInAction/ and less than half a percent of Sargon’s subscribers. If any readers have tried Akismet recently, why not leave a polite, reasoned review or rating?