It’s been a while since the last discussion. The format will be the same as always; I just drop you into our chat room. Today we are talking about the pull of 1up’s Neverwinter Nights 2 review. If you aren’t sure what I’m talking about, don’t worry, Horatio wasn’t, either. For those of you who would prefer to read opinions in a more traditional format, check out Craig’s editorial on this issue.
Horatio: Before we start, I quickly read the article, but can someone do like a three sentence explanation of what happened?
Christian: Basically, Matt Peckham wrote a scathing review of NWN, 5/10. People thought it wasn’t a fair review, and 1up’s editorial staff pulled it. Later it was explained that their editorial process was a bit different for this one. It might not have been read by as many people as normal.
Horatio: Also, unfair in the sense that like, he just doesn’t like the genre?
Jay: It happened to not be well written so a lot of people argue that’s why it was pulled. But these are all points of contention. He did cite Icewind Dale and Torment as good games.
Christian: Anyway, the editor still didn’t like the review, told Matt to redo it a few times, and posted a final draft he still didn’t like much. And later decided he couldn’t in good faith keep it up.
Jay: The first question: Is pulling something because it’s poorly written acceptable? (ignoring the possibility it was pulled for other reasons)
Craig: Pulling something for the sole reason that it’s poorly written is perfectly acceptable, but the editors shouldn’t even have let it get that far. It’s an editor’s job to make sure the article is top notch before it goes live, not afterwards.
Horatio: In theory I agree with what you say, and I know, Jay said we should ignore other factors. On the other hand, if you let people pull articles because they’re “poorly written,” it seems much harder to trust a site that that’s the reason they’re actually pulling an article. As opposed to inadvertently pissing off their fan base or advertisers etc. etc…
Jay: I think it’s absurd to pull something for poor quality and makes it look worse than not pulling it. But then the beauty of the internet is that you can edit things subtly. Print mags can’t change stuff, but we can. If it’s nothing of material (which is important) and just grammar fixes, just go in during the night like a ninja and fix it. Don’t retract it.
Craig: There’s another problem with pulling things in such a way – it’s opening the floodgates for the rest of the site. Now the community has seen their prayers answered so to speak, they’re going to request this every time they see something they don’t like. That review wasn’t that badly written, just mediocre.
Christian: On one hand, the explanation was quite detailed. I appreciate that a lot; its not something mysterious like a lot of pulled reviews. On the other hand, if it was that bad, it should have been caught. That’s just sloppy on the editor’s part. Plus, 1up is already on shaky ground with some people. Examples: accusations of their DOA4 guide being ripped from a fansite. Dan Hsu’s call out of game journalists without giving names. They seem to make huge blunders, make a fancy excuse, and wait for the fans to forgive them (after all, those editors are like rock stars to them). I just don’t trust them at this point.
Jay: Does anyone want to defend the pulling? Perhaps with the argument that the guy didn’t seem like the guy who should review that game?
Craig: It isn’t defending as such, but I think it’s in a games journalist’s forte to be versatile. If he likes the genre, then that can sometimes lead to bias reviews. Likewise, if he hates the genre, that can lead to bias. A reviewer should be able to just see each game individually.
Pat: I think you want some subjectivity to reviews. If a reviewer is a fan of a genre then he probably has more in common with his readers (who will be interested in the genre of the game he is reviewing) than someone completely out of his depth. If the reviewer makes his prejudices known (as this guy did, to some degree) all the better. I find a review most useful if it is biased, but I know what the reviewers biases are.
Chris: If a reviewer has a bias toward a particular subgenre (as in this case), then that will fail to appeal to a lot of readers. This particular guy liked some RPGs but was apparently really jaded with the D&D system, which makes it seem like he would not be the best choice.
Pat: You are correct. In a lot of ways it’s a tradeoff of expertise for subjectivity. I don’t see a solution to that besides letting robots review games.
Craig: Well, once again it’s down to the editorial team to sniff out who is the best for each review, but I still stand by the fact that a professional reviewer should be able to review anything.
Horatio: I think what this comes down to is that it was a poorly written article. What they should have done is let him rewrite it in a more compelling way so that he could get his point across. In some ways I think he did the best to admit his bias– he offered a separate score for people who are fans of the genre.
Jay: I think this comes down to what we expect from reviewers. If people want to know if a movie is good, they check a random movie review. If people want to know what film experts thought they check Ebert’s review. This may not be entirely correct, but my point is that some critics are well known and are supposed to express their opinions and not the opinion of an objective reviewer explaining the movie/game for people.
Craig: People tend to get up in arms when reviewers don’t conform to the norm of the rest of the scores.
Christian: I’m going to defend Peckham’s stance here. Regardless of how it was written, his view is solid; its all great from a numbers and stats standpoint but he didn’t enjoy the story, the gameplay, the content we look for in a game. And to him, judging it as “a toolset” is bullshit, because he was looking for a game. I think this fiasco brings up just how scary the “hivemind” is in gaming culture. People complained that the review didn’t match their expectations, that he should have realized it’s most important as a great toolset. And 1up fans claimed they were glad the site was catering to the people who wanted the game. That scares the hell out of me. I don’t want a site catering to anyone. I want to know what they really think.
Jay: Do you think it was a clear cut of catering to fans?
Chris: It certainly looks that way.
Pat: Probably. Although it was somewhat poorly written, it wasn’t pull-worthy. If this were real journalism they would have been crucified for retracting that. 1up will give that game a seven when all is said and done.
Craig: Absolutely, this is the editorial team bowing out under pressure. There have been three reviews similar to this at Rewiredmind.com in the last couple of weeks like these and they’re still there.
Horatio: Random interjection, but I think that’s one of the benefits of most gaming magazines is they always panel review games. So you can have different viewpoints/subjectivities, and choose a reviewer you know represents a similar stance on games to you.
Christian: I think the reasons people are decrying Peckham are as asinine as anything he might have said. But look who won. Also, if the new review is significantly cheerier, and gets a much higher score will you guys be more inclined to suspect foul play?
Chris: Maybe it all comes down to whether an editor will defend a writer or cave in to the site’s fans. Definitely.
Jay: I don’t understand how they can possibly justify doing anything but reposting the same review after cleaning it up a little writing wise.
Craig: There’s another factor at play here too that we’ve neglected – the all seeing PR people. Do you think that that has affected 1 UP’s decision to pull too?
Jay: Yes. Advertising. Atari must have gone nuts.
Horatio: So I think they should repost the review (but better written). But it might also be that they should have someone who is a fan of the genre review the game. Because there are definitely niche genres (think puzzle games) that should be avg. reviewed so non-fans of the genre know to buy them, but also a review so people who love the genre know what to do about the game.
Pat: That’s a good point but also that’s what Metacritic is for.
Horatio: Well I agree, but also a normal game website should recognize that there are many criteria to review a game and if one review can’t encompass them all because of inherent bias, then multiple reviews may be necessary.
Jay: I don’t necessarily agree on the multiple review thing. It takes a lot to have two reviews of everything.
Craig: That also removes the whole purpose of a review.
Jay: I think we need to expect more from readers. They need to understand that people have different opinions and learn to excise important information from reviews and weigh it in their own minds
Pat: That’s like expecting more from voters. Good luck.
Horatio: Well multiple reviews aren’t a requirement, I’d say. But if a site wants to give a good review to all its readers then it either needs omniscient reviewers who can take multiple stances, which is often possible or it needs multiple reviewers.
Craig: Or editors willing to back their opinions.
Jay: Ha. Before we finish does anyone have anything final to add?
Craig: Bottom line: this isn’t about what is expected of a reviewer; it’s about what people expect a review to be.
Pat: These places make their decisions when they hire a reviewer. They choose the person because they respect their ability to critique games. As such they should edit (as that’s their job) but stand behind the review.
Christian: Peckham is a freelancer. Just wanted to clarify that.
Pat: I’ll admit I did not know that, but the point’s the same. They paid him to review a game, they didn’t pay him to give the game a good review fans would be happy with.
Christian: Not to sound melodramatic, but I think this is a huge loss for indie sites, for true fans, for a lot of us. 1up has proven to me in the past that they don’t always care about the amateurs, the New games Journalists and they know that their fans see them as gods in the industry. They’ll cater to those fans, and at the same time, their clout as editors will sway the minds and opinions of a lot of readers. Read any list of 1up blog comments and look at how damn homogenous it is. It’s scary. And in turn, 1up can do whatever the fuck they’ll please, and the fans will gobble it up. That makes it hard to get a fair shake in a review, and for guys like us to get any sort of strong voice
Craig: A scary thought and a scary position of power.
Jay: OK, last call. Anyone? Anyone?
Craig: BOYCOTT 1UP!