by Darius Kazemi, Jun 23, 2015
So I'm sitting here at the excellent Open Source Bridge conference, and I was reminded of an old project I started on and let fall by the wayside.
I made this Twitter bot called @TwoHeadlines: it takes the subject of two headlines and swaps them. It's very funny and very popular.
But it does one thing I hate. It sometimes tells jokes that are transphobic. There are a lot of ways to tell a transphobic joke, but in the context of Two Headlines it boils down to something like "[Male celebrity] looks stunning in her red carpet dress." This "man in a dress" trope is sadly very common and all it does is denigrate people who don't deserve it. @TwoHeadlines doesn't tell this kind of joke often (maybe once every couple weeks, 1 out of every 200 jokes) but it does happen. I've long wanted a way to programmatically detect these jokes and block them before the bot makes them. (For more on the ethics of bot humor, see Leonard Richardson's excellent Bots Should Punch Up.)
Fortunately, the super diverse and awesome crowd at OSBridge today inspired me to get off my ass and do something about it. Today I implemented my system for transphobic joke detection, and I deployed it to the production instance of @TwoHeadlines. Now I'll tell you a little bit about how it works.
Gender detection is super tricky and fraught, especially if you're trying to serve people ads or otherwise interfere with their lives. Generally I would recommend avoiding gender detection entirely. But in this case I'm not looking to categorize people. I'm looking to looking to detect certain linguistic constructions in an effort to not ruin someone's day.
Fortunately, Open Gender Tracker by Irene Ros and Adam Hyland provides pretty decent probabilistic gender results for names based on real-world data. It understands that some names are gender-ambiguous, and quantifies this intelligently. It understands US and UK names, separately. It is a really cool project you should read more about, but in a nutshell it'll tell you "I'm 99% sure that Mike is a male name" and "I'm not really sure whether Casey is male or female".
A few months back I created gender-probability, which is an npm package based on that data, and is what I'm using for this code.
It took me a little while to wrap my head around this, but in the context of Two Headlines, a transphobic joke is one where I'm taking a male-gendered subject and swapping it with a female-gendered subject, or vice versa. This is not, of course, every transphobic joke that the bot could possibly tell, but it captures a great deal of them, and is relatively easy to detect.
I added the following code to my bot:
var genderProb = require('gender-probability');
genderProb.load();
function isTransphobic(topic, newTopic) {
var isTransphobic = false;
var name = topic.split(/\s/)[0];
var swapped = newTopic.split(/\s/)[0];
var prob = genderProb.get(name);
var probSwapped = genderProb.get(swapped);
if (!prob.error && !probSwapped.error) {
isTransphobic = (
(prob.data.prob.gender === 'Male' && probSwapped.data.prob.gender === 'Female') ||
(prob.data.prob.gender === 'Female' && probSwapped.data.prob.gender === 'Male')
);
}
return isTransphobic;
}
This code is simplistic, maybe even to a fault. It takes the first word of the original topic, and the first word of the replacement topic, and assumes that each represents a "first name". Then it asks gender-probability what the probable genders are. If gender-probability is sure that one is male and the other is female, then I assume the joke is transphobic.
This usually catches the base case of "[Male celebrity] looks gorgeous in her dress."
The algorithm does create false positives at times; that is, there are some jokes that it thinks are transphobic that are not actually transphobic. For example, I had one joke where it refused to swap "Star Wars" for "Sean Combs" because Star is a woman's name and Sean is a man's name. I'm willing to accept this: in my particular design, for this particular application, false positives aren't necessarily a bad thing.
I'm fine with throwing away ten maybe-good jokes if it means that one definitely-bad joke gets caught.
I didn't come up with a general solution for removing transphobic content on the internet. This solution only makes sense in the context of @TwoHeadlines. But I think it's a start. I was able to identify a behavior in my bot that I didn't like and come up with a way to stop that behavior. Maybe you can do it for the things you create, too. It might be easier than you think.
Lastly I just wanted to apologize to the whole world, and especially trans people, for these shitty jokes that @TwoHeadlines tweeted at their expense. I hope it doesn't happen again, and if it does, I hope I'm quicker to squash it than I was with these jokes. I have long maintained that I don't want my bots saying things I wouldn't say, and have even written code to try and programmatically enforce this. But in this instance I didn't hold my bot up to my own standards. As always, if you notice any of my bots doing something shitty, please reach out to me on Twitter or through my contact form.
So, again: I'm sorry. I hope to do better in the future.