There was a post on some news site last week with an article by Louis Bedigian quoting an analyst (Arvind Bhatia) claiming that “Zynga loses $150 on every new paying customer.” I read it, thought to myself, “That’s absurd linkbait,” and then assumed that nobody would take the bait. I was wrong: it got picked up everywhere. Sigh.

This morning, Andrew VandenBossche alerted me to an article by Dylan Collins, quoting industry CEO Torsten Reil, that responds, no, you idiots! Your methodology is wrong! “Zynga is probably MAKING $30 on every paying user!” So… here’s what I think: nobody knows what the fuck is going on. (For those of you wondering why I’m writing about this, before I did HTML5 stuff full time, I spent 6 years as a data analyst for game studios, both MMO and Facebook games.)

Surface analysis

Collins/Reil are absolutely right to call the original analysis oversimplified. It was based on a model that completely failed to account for attrition — they’re correct when they state that Zynga certainly gained far more than 400k paying users for their marketing money.

Unfortunately, Collins/Reil pick a number out of thin air (20% attrition rate) which results in a rough estimate where Zynga spends $120 per paying user, and makes $150 per paying user, resulting in a net profit of $30 per paying user. I say unfortunately because if the number is 10%, then by Reil’s metric they’re losing $21 on every paying user. If it’s 30% they’re earning $57 per acquired paying user. It all hinges on their attrition rate, which we don’t know! Some games see 10%. Some games see 90%. 20% seems like a roughly correct ballpark for a mix of successful and unsuccessful games, but honestly we have no idea what it is because we’re on the outside looking in. But the truly weird thing to consider is: Zynga doesn’t know what their attrition number is either.

Models and black boxes

All numbers like this are built on models that analysts put together, and models are built on assumptions. Simple example: when we talk about attrition, what phenomenon do we refer to? Typically we mean “the moment when someone is no longer a player of the game.” Yet in the context of a social game, how do you define that? Facebook users don’t typically uninstall an app — they usually just stop using it. So you have to pick an arbitrary cutoff point. Does someone fall into an “attrition” bucket after 1 week of inactivity? 2 weeks? A month? Remember, this number is arbitrary, so you can adjust that number all you like (within reason, you’re not going to pick 100 years) until you come up with an attrition percentage that meets your criteria. Whether those criteria are “seems more realistic” or “would appease our shareholders” is another question!

But regardless, this attrition percentage then affects all of your other calculations. Now, ideally you want to remain internally consistent once you pick this number, but a dirty secret is that even if you maintain perfect internal consistency in always using “2 weeks of inactivity” as your cutoff for attrition, there will always be dozens of other fiddly and less directly consequential definitions that you can tweak. And the thing is, on some level you have to tweak these numbers! Otherwise you might find yourself stuck with a model that doesn’t reflect what looks like the reality of your game.

To put it another way: the internal game studio analyst’s job is to assemble a black box known as the concept of “attrition” — to the CEOs and CFOs and shareholders and external analysts and pundits at home, this concept seems pretty straightforward: it’s the people who leave your game. End of story. The black box behaves and does its job, reporting a number between 0% and 100%, and presumably you panic if the number is closer to 100%.

But the internal studio analyst needs to assemble this concept from a variety of sources. They might ask the game designers what they see as a “normal” or “natural” amount of time away from the game — if a game is designed to be played during the work week, then you shouldn’t sweat it when someone isn’t playing over the weekends or on Christmas. They might look at historical data for the game and notice that 80% of players who are inactive for 12 days never come back. And 90% of players inactive for 15 days never come back. So maybe we pick 90% and say 15 days is our cutoff. But of course we’re looking at historical data for the current game, which is different today than it was back then, so it’s not a perfect analogy! So maybe we want to rely on data from the last 30 days, when the game was most similar — but now our definition of “never come back” really means “people who were inactive for 15 of the last 30 days and haven’t been back.” But of course, those people “haven’t been back” for a maximum of 15 days since we’re looking at a 30 day window. So now that our historical data is more representative of the current state of the game, our very definition of “never” comes into question!

An infinite regress of assumptions

“ An internal game studio data analyst does in fact work in a vacuum, and will get fired for sharing with outside analysts. This means that the chances that our assumptions are off-base are pretty good.

In summary: games are very complex systems, and the numbers that get thrown around in the media are built on black-box-style assumptions. These black boxes can always be broken down into components, and those components into subcomponents, forever and ever into an infinite regress. If this seems mind-bogglingly weird, well: it is. On some level you need to stop digging into the infinite and come up with assumptions about the way the game works that become the foundation for your models. There’s nothing wrong about that in principle: science does this all the time, and manages to come up with some great models to describe the world. But there’s a huge difference between science and analyzing the metrics for social games. Scientists do not work in a vacuum within their universities or corporations. Scientists do not work with “proprietary data” and they do not run the risk of getting fired for sharing their results and even their methodologies with other scientists. An internal game studio data analyst does in fact work in a vacuum, and will get fired for sharing with outside analysts. This means that the chances that our assumptions are off-base are pretty good. And it means that the numbers that different companies throw around can’t even be compared. “Average revenue per user,” which sounds straightforward, can be based on entirely different foundational assumptions at different companies and on different games.

This whole mess is one of the main reasons I stopped being a data analyst for games. I did not feel comfortable coming up with assumptions that weren’t, on some level, complete bullshit. Now, the level on which these assumptions operated was often very low-level, fiddly stuff. But it was an art, not a science. Which, again, nothing wrong with that — except that the black boxes that I generated were being treated as science rather than as art.

In the end, for the purposes of arguments about how much money a company is making, the only numbers that matter are: how much money is coming into the company each month? How much money is leaving the company each month? Everything else should be viewed with utmost suspicion.

Graham Harman’s Prince of Networks is the most exciting book of philosophy I’ve read in a long, long time. Partly I find it exciting due to its prose style. I’m fascinated by the Whitmanesque enumeration of objects/actors/actants in the book, so I wrote a script to parse the original text for things that are probably lists of objects, and then I did a little manual data cleanup on the resulting output.

Once I had the data, it was easy to create a page that gives you a random object.

I’m not really sure why I made it, but hey, it only took a couple hours.

I gave a lecture yesterday at Worcester Polytechnic Institute, my alma mater. The class I spoke to was IMGD 2001, Philosophy and Ethics of Computer Games. A little bit of background on that: during my senior year in college, WPI was still putting together its undergraduate video game development major. I took on a thesis project for Prof. John Sanbonmatsu where I essentially wrote a grab-bag paper on video games and critical theory. As John educated me in critical theory, I educated John in video games. The end result was not just my paper, but a framework which John applied to creating the IMGD 2001 class.

I bring up that background to highlight what a joy it is to speak to this particular class. I try to be a guest speaker every time it’s run.

Someone mentioned to me that I should probably try and post what I lectured about on my blog. So this is my attempt to cover some of my points. It was a pretty fragmented lecture, and I was responding in part to some of their reading (Marcuse on aggression in technological societies).

My Lecture

I framed my lecture by showing two videos: real footage of AC-130 combat from Afghanistan, and the AC-130 footage from Call of Duty: Modern Warfare. I didn’t really comment on the footage except to remind them that in the real footage they are watching human beings die on camera. It is difficult to interpret the real footage as anything but a video game. Again, this was really just to set a mood and perhaps shock them into paying attention.

I wanted to show them that professional game developers do think about the philosophical issues that they cover in John’s class. My first stop was Clint Hocking’s blog. I told them to read his lecture transcripts, particularly the ones about intentional play, improvisational play, and the exploration of the self through systems.

I then went into a discussion of Far Cry 2 versus Halo (or most other FPS games). The basic argument I covered was that FC2 promotes improvisational play, whereas Halo promotes the play of dominance. Because of its game systems, in Halo you can find a dominant strategy and become a godlike slayer of baddies.  Halo is a game where you dominate the system. FC2 is a game where the system dominates you. Plans often do not work out as you expect. The game is often brutally difficult, but gives you a wide array of tools with which to mitigate that difficulty.  But the important thing is that the system dominates the player, and the best you can do is survive long enough to meet the end conditions of the game and “win.” Some gamers take FC2′s stance towards improvisation without domination as literally offensive: people claim that the game’s endlessly respawning enemy encampments never let you feel like you’ve “cleaned out” an area. You’re never safe. Ever.

I think this is fundamentally a good thing for a game system to present a player. By giving us a system that is stacked against us, where we need to be clever given our limited resources to survive, I feel like FC2 is doing a service by training our brains to perhaps be more receptive to those kinds of situations in the real world.

A feeling of progression is perhaps a way of forgetting about death for a while.

This led into a discussion of Ben Abraham’s FC2 permanent death experiment, and then the concept of permadeath in general, and discussing reasons why gamers might be as opposed to permadeath in games as they are. It was particularly interesting in light of the Marcuse essay they’d been assigned. Playing games and getting a feeling of progression is perhaps a way of forgetting about death for a while — the institution of permadeath flies in the face of forgetting the inevitable.

Progression-as-soma led to examining Facebook games. Farmville and Mafia Wars were brought up as prime examples of “spreadsheet games” that are about a feeling of progress towards who-knows-what… and not much more than that. Which then prompted me to bring up Jonathan Blow’s 2007 MIGS keynote, which is centered around this assertion/question:

When millions of people buy our game, we are pumping a (mental) substance into the (mental) environment.

This is a public mental health issue.

We have the power to shape humanity. How will we use it?

Other things were discussed in the 90 minutes I had to speak. The last half hour was Q&A. I spoke a little bit about what it’s like to work for a big game company, and the amount of creative freedom you have (or more accurately, do not have). I talked a little bit about Auntie Pixelante’s dissatisfaction with the Guildhall at SMU. I also discussed the indie game scene.

“Good games are made by interesting people.”

Probably the best question came from a student in the back, who said:

No offense to the professor, but in my other classes it seems like they’re encouraging us to make video games, and they talk about how video games are good things. This class is the opposite. It’s depressing, it’s almost like you don’t want us to make games at all. What’s the point of taking a class like this if what I want to do is make games?

My response came surprisingly easily:

If you look at the people who are at the vanguard of the game industry, the people who are pushing the form and helping video games become better than they ever have been, most of those people are reflective about the process of making games. The best game developers are the ones  who think about this stuff. So the answer is simple: if you want to make the best games in the world, you need to actively think about exactly the sorts of issues that are brought up in this class.

After class, I met with some of the IMGD faculty, one of whom is (I say with no small amount of pride) Brian Moriarty. I mentioned this last question/answer exchange to him and his response was: “Well, yes. Good games are made by interesting people.” I couldn’t have said it better myself.

So Gamasutra just posted a summary of a talk Chaim gave at the Dutch Festival of Games, called “Magic Crayons: Spore and Beyond.” This is actually a reprise and probably an update of a talk he gave at GDC 2007. Most of what is in there is great, but I always get bothered by sentiments like this one:

In the end, Gingold says that he believes computers can ease “this anxiety and alienation that we have from doing one thing.” Even though people become experts at their trade, “we can design houses, human beings, pinball sets,” he said.

See, I don’t want to make games that ease alienation from labor. That just makes it easier for people to accept alienation. I want to make games that end alienation from labor. Games that inspire people to learn new things, to make positive changes in their lives, to learn about themselves in the process of interacting with systems.

Of course, I have no idea how to do any of this. But hey, the first step is admitting there’s a problem.

I gave a guest lecture a few weeks ago at WPI (where I got my degree). My friend and former professor John Sanbonmatsu teaches a course on critical theory and video games, and he invited me to be a guest lecturer–I suspect mostly to mollify student’s suspicions that no real world game developer could actually give a crap about all this Adorno and Horkheimer stuff. Well, I certainly do.

Anyway, here are the notes that made up the first part of the lecture I gave. It’s all over the place, but I think it’s worth putting up here.

—

One of the unique qualities of capitalism is that it’s about the domination of use value for exchange value. That is, instead of valuing the intrinsic qualities of a thing in itself, we instead care about its market value. This simple statement actually encapsulates two of the biggest complaints that your hear from within the game industry.

One complaint is basically that art games won’t get funded. That is, if you want to make a really interesting game that stands on its own as a work of art, you either have to do it yourself and nobody will know about it, or you get a publisher and venture capital and you spend millions of dollars and end up with a derivative piece of crap that’s nothing like your original vision. This is because the publishers, the capitalists who control the means of production, care about profit more than the overall increase of human happiness that could be caused by making a thoughtful, moving game. It’s actually pretty soul-crushing when you claim that something you’re working on is art, but what you’re really producing is a cheap commodity. At least the folks toiling away in the stapler assembly factory don’t have any delusions that they’re making people’s lives better.

Another big complaint is that game journalism sucks. Some people want interesting analyses of video games. Let’s look at games in a critical way, tease out their influences. Yet when you read a review of a game like Team Fortress 2 (which I love, by the way), everybody’s talking about graphics and art and does it get an 8.5 of out 10 or a 9 out of 10 and whether the medic/heavy combo is a dominant strategy. Almost nobody notices, to cite a pithy example, that TF2 and all games like it are actually a modern recreation of Valhalla, the Norse warrior heaven where dead soldiers fight endless battles and are resurrected upon death to continue their ceaseless fighting. (Which is an observation I stole entirely from Ernest Adams, in his semi-famous lecture “The Philosophical Roots of Computer Game Design”.)

Most game journalists defend their writing by saying that all they’re doing is helping consumers make an informed decision about whether they should be spending their money on X, Y, or Z game. But who’s to say that’s any more important than the Valhalla assessment? I mean, when I am playing Team Fortress 2 I get lost in the strategy and the adrenal viscerality of the game, I forget that I’m just logging on to a server and fighting a bunch of guys for no particular reason whatsoever. And then I have to think: is this truly making me a happier, better person? And after some reflection, I come to the conclusion that the only reason that I play this game is that my best friend Darren also plays, and I enjoy playing with him. I rarely get to see him in person, but we can spend one or two nights a week playing together and talking over voice chat, and it really does strengthen my friendship with him. And that’s why I play Team Fortress 2: the friendship. (Cue rainbows and puppies.) And I wouldn’t have known that if I hadn’t actually taken a step back and thought about the game for a few minutes.

So I’ll go back to my earlier question: who’s to say what’s more important, consumer reporting or qualitative analysis? And the answer’s simple, again: the capitalists who control means of production. They’re the ones who have the final say, they’re the ones who run the magazines and the TV networks and so forth, and so people are only going to be exposed to the shitty consumer reports. Yes, there’s been some democratization with the internet, and you can go to certain websites and get good game criticism, but it’s still an uphill battle because 99% of the population has been indoctrinated to believe that being informed on HOW to spend your money is more important than being informed on how to THINK about what you’re spending it ON. Your average gamer is literally insulted when you ask him (or possibly her) to think instead of to consume.

And there’s another problem. As gamers, we think of ourselves as consumers first and foremost. We fetishize commodities, we stand in line 20 hours for a Nintendo Wii, and our self-worth is reflected in our Xbox Live Achievement scores. And that’s dangerous. I was just watching a 2004 lecture called “The Tragedy of Suburbia” by an architectural critic named James Howard Kunstler, and he said something that really resonated with me: people need to stop thinking of themselves as consumers, and start thinking of themselves as citizens. And not “citizens” in the sense of the word where you’re an obedient servant of the government. Citizens have responsibility to their fellow humans. You need to look at everything you do and ask yourself how you’re increasing the general welfare. And as long as gamer culture is focused on consumerism, that won’t happen.

There’s a great quote from Adorno’s essay on “free time,” where he’s lamenting the idea of having hobbies. To paraphrase, he says that every time someone asks him if he has any hobbies, he has to say no. This isn’t because he’s a workaholic who can’t enjoy himself, but rather that a hobby is merely something to pass the time, and he doesn’t lower himself to passing time. He reads and makes music and listens to music with his full attention, and to call these things “hobbies” would be to degrade them. And the gamers that I really admire approach video games the same way: they are extremely mindful of their play. They really truly think about it.

And that’s all I can ask of you. Be mindful of your play.