Back in September, my friend Brendan and I made a game-like website about Sarah Palin (speaking for myself, I was trying to blow off some angst). We called it PalinSpeak.
I think the "game" and our process failed on a number of levels. But it was a learning experience, so I thought it would be worthwhile to reflect on the production process. Here are some lessons.
1. The pace of the news cycle is insane.
We conceived the idea on a Friday night, and finished Monday evening. Normally, this would be an impressive turnaround time. But in terms of the news cycle, we were eons late. The original Couric interviews aired Wednesday/Thursday and so by Friday night, pundits everywhere were talking about the interview. Saturday night, SNL even parodied the interview in their opening sketch. That is to say, the Couric-Palin interview satire was already well-trodden ground by Sunday.
But at least we could be the first interactive satire on the topic, right? Not so much. Another NLP quote generator (interviewpalin.com, seemingly defunct now) launched early Monday.
Is it even important to be first? Yes. By the time we published, the issue was already passé. McCain and Palin had even done a follow-up interview with Couric. The economy was crashing, and new issues had largely replaced last week's issues.
Our original goal was to release on Monday morning, and even those 12 hours would have made a big difference because our game comes across as very similar to interviewpalin.com. Novelty is a huge factor in generating buzz and attracting users.
The breakneck pace of the news cycle raises some serious concerns. Providing "the first" commentary is difficult, especially when tackling such hot news. Writing and video-editing is hard enough, but coding and balancing simulation dynamics is even more challenging on such a short cycle.
One approach is to use existing game templates, but then the newsgame lacks novelty and seems shallow (content and form become divorced). Another answer is that games might not be an appropriate medium for these kind of short-cycle events. Rather, games might be more suited to overarching narratives and other subjects with longer time-frames.
2. The line between "real" material and fictional material is very thin, but important.
In one sense, our source data was very real. The algorithm was trained on real interview/speech transcripts, and all words and phrases come from those sources. But when recombined at a certain level, is it fair to advertise that our game "uses" real Palin material? At what granularity does recombination become new creation?
Compare PalinSpeak.com to, say, Molleindustria's McDonald's game. Is PalinSpeak.com somehow more realistic because its model runs on "actual data"? Certainly, some of the phrases are real, and the fun is in seeing the similarities between Palin's own words and the output of a rudimentary computer algorithm. But when recombined into gibberish, does the authenticity of the source material carry any weight?
This middle ground between fiction and reality led us to confront some tough questions. Did we have an ethical responsibility to only include real quotes? Some friends encouraged us to intentionally blend fiction and reality by incorporating Tina Fey's satire into the training model. We resisted, because such a "pollution" of fiction might call into question the core strength of the simulation - the stark reality of the individual phrases and quote fragments. But the counter-argument is, the game already puts sentences (even if not words) into Palin's mouth, so in fact it might be more ethical to run with that and make it even clearer that the game is fun fiction and nothing more.
So the core issue boils down to this: is such a game merely fun parody, or is there something more "serious" about using the real words? Or rather: is the game a laugh with a message attached to it, or a message with a laugh attached to it?
It's important to agree on these "vision" questions before the production process. For example, Brendan wanted to put ads on the page, but I insisted that such a move would jeopardize the integrity (or at least the aesthetic). Yet perhaps it's dishonest for me to even believe that the simulation imparts some sort of serious message, in which case ads might usefully signal a type of levity or entertainment.
The problem is, it was tough to even understand our own goals before seeing how the final product played. In a short news cycle where the pressure is to release quickly, rather than iterate and re-design in service to clear ideals, it seems impossible to honestly make anything but entertainment.
In other words, the core journalistic tenant of "citizens first" (or in the case of advocacy games, "message first") needs to guide all decisions; otherwise the game should probably just be viewed for what it is - a game. Not that there's anything wrong with that, but we should be honest with ourselves.
3. Design and implementation cannot be separated from issues of principle.
There were so many small and unexpected decisions that we had to make along the way.
For example: we gathered the training data from several different interviews, so sometimes Palin addressed "Katie" (Couric) and sometimes she addressed "Charlie" (Gibson). Was it dishonest of us to replace all "Charlie" references with "Katie"? We ended up deciding that we were de-contextualizing the phrases anyway, so that preserving the right name was not a big concern.
These kinds of questions do come up all the time. As such, a newsgame production team cannot be "external" to the newsroom culture, hired in a kind of mercenary fashion. This is not a new realization. In their The Elements of Journalism, Kovach and Rosenstiel argue that even support teams like the advertising bureau should be centrally motivated by journalistic reasoning, rather than separated from the journalists.
4. Complex models work against transparency.
We do our best to explain the core algorithm on our About page. But the vast majority of players will never fully understand how we assembled Palin's responses. Thus, one could call the game dishonest because it might trick people into believing that Palin said more than she actually did.
Is our only option, then, to make the simulation completely over-the-top (e.g. adding fictional quotes, as mentioned above), to prevent people reading more into the simulation than they should?
I think the answer is that PalinSpeak is ultimately less about Palin's specific words and concepts, and more about her general incoherency. Whether or not we successfully convey this point, I don't know. But this realization did affect the written framing of the game (see below).
5. Research should dictate design, not visa versa.
We originally conceived the game as a commentary on Palin's, well, "lack of intelligence." But in making the game, we did a lot of research and were forced to confront some of our own assumptions.
For example, I couldn't help but note that some of Palin's older interviews - which generally focus on local Alaska issues - are significantly more coherent:
Thus, the message of the game should not be that Palin is some sort of idiot. Rather, the point is that she seems wholly unprepared (to put it mildly) to talk about larger domestic issues and foreign policy concerns.
But even that more specific insight can't reasonably be inferred out of a silly NLP algorithm. What I ended up realizing is that our game, from a more journalistic standpoint, was really about the absurdity of Palin's responses to Couric. Nothing less, nothing more. To infer anymore would be to overly-politicize.
I'd like to tell you that this realization impacted the design of the game, but honestly I'm not so sure. Our thinking only fully crystallized when we tried to frame the game in words. On our About page, we ended up writing:
"This web site is a parody. We don't think Sarah Palin is 'stupid.' Some older interviews on local Alaskan issues demonstrate a more coherent Palin. But these new interviews are simply ludicrous."
So, my own political opinions aside, my research suggested a more nuanced reality. If I were to act as a journalist, I would have to at least acknowledge that research.
All of this suggests two takeaways. First, for a game to be journalistic, research should dictate the design, rather than the other way around. Or, at least, if new research is uncovered that leads you to challenge your assumptions, you have to be ready and willing to scrap your work for a re-design. This is why the pace of the news cycle is so dangerous, especially if you're a game designer.
Second, the newsgame production process, though typically focused on the user, might very well be as valuable (if not more so) for the people making it. In other words, we might learn a lot more about the news in making games, rather than playing them.




This is fun. Thanks for making it available.
It might be interesting to make this into a reverse Turing test. Present human judges with two paragraphs, one Markov and one from the training base. If people do not get it right X% of the time we can declare her intelligence mechanical.
What is a good value for X? Repeat the entire exercise using someone else as a base, say Barack Obama.
I am sure this proposal is too much work for too little payoff, but we can dream, can't we?