Joe Ludwig gave a talk at ION 2008 called “Pirates of the Burning Sea: A Post Partum.” These are the notes I took on the talk, any mistakes and misinterpretations are my own. My comments are in brackets.
Pirates of the Burning Sea (PotBS): Standard $50+$15/mo subscription model. 4 nations, 5 careers, 3 sword fighting schools, player driven economy, tactical ship combat as star feature.
Project started in October 2002, 4 people, 6-12 month schedule. Planning to do something pretty small, get it out there, small core user base, gradually grow it.
Launched Jan 22 2008, 70+ pepole, more than 5 years of development.
How did that happen?
Under a year left on project for the whole time they were developing. We’d make some progress, grow the scope, repeat. We’d fool ourselves into thinking it’s 6 months instead of 12, then revise the estimate back to 12. We never decided to make a big game.
Lesson 1: know how big your game is when you start. Don’t start small and grow the scope. Problem with scope creep: implement features that are scaled down to meet the unrealistic schedule, then years later you rip them out and put in the feature you should have had. Never the right time for long-term investment. Why make tools if shiipping “soon”? Never work on anything requiring more than a month of programmer time (although actually you spend 6 months extending it month to month). Also never a good time to hire a new programmer because it takes 3 months to do hiring, then another 3 to train up. It’s also never a good time to fire anybody!
Upside of scope creep: if we started with a big project we never would have even tried in the first place. It’s boiling frog development, based on the possibly apocryphal story that if you put a frog in boiling water it will jump out, but if you put it in warm water and heat it until it’s boiling, it won’t jump out to save itself.
Lesson 2: figure out what game you’re making as quickly as possible. The relationship with publisher usually drives this, but PotBS didn’t have a publisher really until the last 8 months. Self-funded, so no tight vision required.
In fact, there were four visions: lead designer, executive producer, content lead, lead programmer.
Lead designer: deep combat sim at the very core, realistic historic setting. [Designer was in the audience: "I thought it was cool at the time."] Cannon fired (range table for realistic ballistics) -> did I hit the ship? -> which hull section (12 sections)? -> did I hit crew? -> did I hit superstructure? -> if not, did it hit the crew on the other side of the ship? -> did it hit the armor on the other side of the ship? REPEAT THESE CALCS FOR SPLINTERS THAT COME OFF THE HULL!
Executive Producer: an MMO but with good graphics. Started in 2002, so it was going up against UO, EQ, DAoC. Didn’t recognize at the time that good graphics meant big budget. Wanted naval traditions. And a Starfleet Command style combat system.
Content Lead: rich single-player content. Storytelling emphasis. Scenarios respond to player actions and tell interesting stories.
Lead Programmer [Joe Ludwig, the speaker]: 6 month project! Minimal feature set that will grow after launch. Keep costs down. Developed reputation as the “NO” guy. Not necessarily a good thing either, even after it was clear the game wasn’t small, he still kept pushing for small scale solutions and made bad investments.
They lived with four visions for two years!
Meanwhile they worked on a pitch for a scifi license game, realized that the scifi pitch was a better game than the pirate one. When scifi game fell through, they ported the design back to the pirate game. They created the overview spec, a 50 page document (probably a little short for such a big overview).
Result of overview spec: historical setting is kept. Kept realism. Not super hardcore but moreso than other MMOs. Starfleet Command style combat made it through. No ability to tune combat with the very complex ballistics. Kept naval traditions. Kept single player style content. Ditched the simulation, not just in ship combat but also ecnonmy. Threw out the 6 months deadline, realized it was a big game all of a sudden. [Oddly, they kept pretty much everything from all the visions except the sim.]
Time for a development reboot. For the first time they could think long-term and could afford to have the game not playable while doing infrastructure work.
They had a todo list for the first time since the scope was there. Eliminated entire categories of things we weren’t doing: player-to-player combat, player-driven economy. (although they made it into the final game)
Still didn’t have a single vision owner. Switch to new combat made the design team weak within the company politically. Vision was incomplete because it was pretty high level and also pretty short. Also had no art vision whatsoever. Had hired an artist but not an art director. Still paying the price for this: good graphics was the goal, so system specs were higher than they should’ve been. Needlessly complex ship models. Naval tradition is good for history buffs but they had to deemphasize piracy to a certain extent. Single player content focus caused lack of focus on group play.
Lesson 3: don’t make scheduling harder than it has to be.
Impossible to really schedule the game due to unrealistic scope and planning, but they tried some stuff.
Tracking flat task lists in Excel, and a summary, one milestone at a time. Worked well when there were 6-8 people on team, but didn’t support dependencies at all. You just had to remember stuff. Didn’t scale over 15 people.
Critical Chain, a Methodology with lots of books and conferences and classes. Tracking every dependency between all tasks anywhere in your system. 2-3 month milestones, and used it from 20-50 people. But it had high overhead. Had someone spending a week of every month on the schedule. Unpredictable results! The purpose of CC is that it tells you on any day who the important person to allocate resources to. But sometimes 8 or 10 people were most critical according to CC! On a 40 person team that’s crazy. Also you could have two people scheduled for the same task and one would be critical and one wouldn’t. Hard to understand. Kept using CC well past the point where it was clearly hurting us.
Task lists in email. It’s an agile thing, one month sprints. Little-a agile. Responds well to change, is lightweight. Doesn’t track dependencies. The big thing as a part of this switch though was that we shortened the length of sprints from 2-3 months to 1 month. It’s way easier to keep track of 1 month sprints and change direction as needed. Decentralized the scheduling. 8-40 hour tasks. Requires QA time to happen after the sprint is over. 3 week chunk ofwork, one week integration, 2 weeks on test server. Works well with 60-70 people.
Lesson 4: don’t save polish for the end.
Don’t fool yourself that beta is polish period. Launched pirates with thousands of known bugs. Testers were very good at finding bugs, but the programmers had no time to fix bugs. In last 6 months of development, only critical bugs ever got fixed [wow, crazy].
Next time I’d like to give people time to polish as they’re working on things. Allow people to have enough time to finish a task for real. Prior to this programmers would leave edge cases in a state where they wouldn’t crash but there’d be annoying problems. Not super common, but with thousands of players uncommon bugs come up a lot. Mission designers had same problem, always more content to create.
Lesson 5: don’t abandon quality for quantity of content.
At some point we decided we needed 1000 missions for launch. We did it, we launched with it. But we repeated lots of content and the missions were pretty cookie cutter. It’s been criticized by players.
Lesson 6: deal with client performance from the start.
What we did was build for a min spec that was getting less and less cutting edge as the project grew older. So that was okay, but next time around I want to make the game run on all the laptops in this room, even the tiny little Thinkpads. You really have to define the art budgets to make the game look great on the min spec. Artists should test as they go and have the poly budgets set.
Lesson 7: make sure everyone is on the same side.
We had a problem during development where the mission designers and the testers had a bizarre war going on. Mission designers would make a mission but not polish it, give it to testers, they’d find the bugs, then keep entering trivial bugs. There was a message where the doctor asks you to get some rhubarb. As he said it “the local soil didn’t work very well for rhubarb.” Tester looks at port and says that the port has a resource “fertile soil”, files a bug. It goes back to content, content bounces it back wontfix. Tester gets mad, flings it back, eventually management had to step in. Two departments developed an antagonistic relationship. On the QA side it was really just one problem tester.
Lesson 8: don’t be afraid to fire people who aren’t working out.
Fired 5 people in 5 years. People who fight against the team aren’t worth keeping. Sometimes they make the people they work with uncomfortable and unproductive. Just not worth keeping.
Lesson 9: company culture is important.
People like working at Fying Labs. People get teased, lots of photoshopping, afternoon coffee train. We have an open door policy that’s taken pretty seriously. Anyone can talk to anyone all the way up to the CEO about anything. How’s the game doing, are we going to finish on time, what about benefits, etc. Another thing that’s helped us out is we have a CEO who’s genuinely enthusiastic about what we’re working on. I mentioned that we were close to shipping a number of times, and at any one of those times at another company we might have started crunching. That’s not something that we’ve done. We avoid it liuke the plague. Maybe a day or two at the end of a sprint, rarely the entire team, and rarely the same person from sprint to sprint. We never schedule people for 7 days a week or expect people to come out on the weekend. So we’ve had really low turnover. 5 firings, lost 3 more, out of 70 people.
Lesson 10: your closed beta should not be as long as ours.
Our closed beta was two years. Didn’t get too much actionable feedback because the game was too much in flux, and burned out key community members. Distracted the team from implementing core systems to deal with feedback anyway even thought the feedback wasn’t very useful. Also, the problem with the feedback was we never got a large enough beta population for most of the beta because we knew we were burning out the community: we didn’t want to lose EVERYONE!
Lesson 11: hire some experienced people for key positions.
A little experience goes a long way. Almost no MMO launch experience. One animator, content director, community director had launch experience. No programmers, no designers, no operations [but Joe said after the talk that the launch was pretty good. they did have large site experience but not MMO experience], no QA. In a lot of cases didn’t have game experience or even WORK experience. Art director is a teacher at the Art Institute, so art department staffed by the best student interns who convert to contractors to full time. great pipeline, but every single person through the pipeline was entry level. Median experience level of art department was very low. Content department almost the same way, but even the experienced folks were not experienced in content, had converted rom other departemnts.
They’re experienced now!
Lesson 12: build more tools. [yeah, he didn't say any more than that!!]
Lesson 13: hire a great art director.
We hired ours halfway through, you should start with one.
Lesson 14: players love character customization.
We have 17 customization slots, a dozen pieces per slot, two colors per piece, millions of combinations, most of it available at character creation. Players spent almost an hour in character creation. Part of the reason that works is they divorced stats from appearance, so nobody has to look ugly to be more effective. On the other hand, it makes status stuff harder, you can’t look at someone and gauge their level. But specific clothing options are unlocked by mission content to make up for that. Including peglegs!
lesson 15: unique systems are both good and bad
Ship combat is very well polished and tuned. It really works, is best system in the game. I’d hold it up to the best system in any game. People love it, they talk about it, they strategize. It places emphasis on player skill, which is big with PVPers. On the other hand it’s nothing like any other MMO combat ever. Which makes it hard to teach. People don’t understand it: it’s a little too weird. It’s slower placed than many games’ combat systems. It also requires more attention than most mmo combat almost paradoxically: way more variables to keep track of. Especially at the high end game.
lesson 16: don’t try to cram an entire new combat system in in a year.
Avatar combat not nearly as polished as ship combat. Content and system built in parallel and they don’t work so well. Post-launch content for the system is great because you understand the system. No iteration time on the system, though.
lesson 17: love your community and they will love you back.
We’ve got a very active community relations program. Everybody writes devlogs, everybody posts to the forums. When we make a mistake, tune something poorly, we own up to it, and we tell people what we’re doing to fix it. We listen to the players, we don’t give them veto power over design decisions but we do listen to them, and we really made relations with the community everybody’s job.
The result is that even negative reviews of the game would talk about how responsive Flying Labs is to the community. This also means that players who are often frustrated tend to give us the benefit of the doubt. They will often respond to something they don’t like with “I don’t like it because ____” as opposed to “you all suck, die in a fire”. This makes our forums comparatively civil. We still have flame wars but they’re better in general. People are nicer to each other and to us. Our community director is a big part of that.
[Audience Q&A started here]
Q: How did you make the funding without a publisher?
a: We funded ourselves by being lucky enough to be self-funded.
Q: What were your marketing challenges?
a: One of the big ones was the combat style. Some reviewers get it, some reviewers don’t. Somewhat difficult learning curve at first. People just don’t totally get the game.
q: Automated testing?
a: Early on we had a programmer whose job was to build an automated test system.
q: What’s the frequency of post-launch content updates given the sprint system?
a: 4-5 weeks per update.
q: You mentioned that your design team was politically weakened after the combat system change. How was that eventually resolved?
a: It was resolved, don’t want to get to specific. We made some personnel changes.
q: Do you feel the game was ready at launch?
a: It was in pretty good shape. Could always be better. A couple months past launch would have been great, but without launching it would not have become great. Needed the live feedback. would have liked to do a larger scale closed beta before launch.
Comments on this entry are closed.