This may be old news to many of you, especially since they even tell you on their About page, but Metacritic weighs reviews. Meaning they count a review more or less depending on how big the site the review came from is. This seems a little shady, but they at least don’t hide the fact. Here’s what tipped me off to some sort of mathematical incongruity.
Something struck me about the math. Mostly it was how (100 + 90 + 90 + 80) /4 = 90 and not 89. If CNET owns Gamespot and CNET owns Metacritic and Metacritic favors some sites over others, and CNET has competitors…well, isn’t this a possible conflict of interests?
This actually sort of makes sense to me. It’s the same way google ranks sites; the assumption is that a site that is linked to by more people and bigger websites is going to be more useful to more people. Of course, that means it’s that much harder to start up a website. Even if it’s good, it takes a long time to get the word out. I’m guessing they’re just making the same assumption, that a site that has a bigger following will tend to have more reliable reviews. As far as conflict of interest goes… that’s pretty much the plague of a lot of review sites. You want to be accurate, but you don’t want to piss off any companies that might have a vested interest in how glowing your review is. It’s just how honest reviewers and editors want to be with themselves.
The first layer of problems is that reviews may not all be on the up and up. Now the people compiling them own some of the reviews they’re compiling and are direct competitors with other reviewers. I may be overreacting, but in most places just the possible conflict of interest would be significant. What am I saying, Bush assigns people from oil companies to protect the environment. No one cares about conflicts of interest.
Yeah, it’s a big problem… but I’m afraid it’s what’s liable to happen when a company reaches the top. Money becomes more important than principles. I don’t think it’s right, but people incorporate & do shady things sometimes. :|
When I tried (unsuccessfully) to get GameLemon to be included on Metacritic’s list of reviewers a while back, the issue their editor told me he had is that we didn’t produce ENOUGH reviews PER MONTH. I found this to be a strange argument, to be honest, but now it sort of makes sense as a cover up for the REAL reason he didn’t want us – we’d just tip his weighted scale right over with all of our mega-coolness and all :)))