Browser benchmarks: When did they get so stupid?

So, the claim that IE is faster than Firefox, Safari, or Chrome, is ridiculous at many levels (MS claims IE faster than other browsers), and Microsoft was appropriately ridiculed for it.

But so is the idea that you have a test that demonstrates that the new Safari builds are "above 15 times better performance than Internet Explorer 7 in the same system".

Seriously, that's just stupid., and renders the index meaningless.

Why not just multiple the index scores by 10? Then you can claim Safari is 150X faster than IE7.

Without scaling the index into a range that meaningfully communicates (or at least correlates) to user experience (which things like FPS and even 3DMark did for video cards), it renders the testing both invalid and irrelevant.

Comscore v. Hulu: garbage in---garbage out?

Interesting. The New York Times is reporting that Hulu is disputing audience count with Nielsen, stating "While Nielsen reported 8.9 million visitors to Hulu in March, another measurement firm,comScore, counted 42 million. "

Wow.

Slightly embarassing, but I think that the Times is confusing "Unique Visitors" (how many unique cookies are counted by a site - a reasonable proxy for people visiting the site) with "Unique Viewers" (a syndicated video player concept - how many unique cookies were counted by the syndicated player; a reasonable proxy for the number of viewers who were served video by the site).

In layman's terms, the first number would represent, in our example, the number of people who visited Hulu.com (unique visitors), while the second (unique viewers) would represent how many people watched a Hulu sourced video, whether on Hulu, a third party site (like Fancast), or embedded elsewhere (like on somebody's blog).

A visit to alexa or compete shows the number of "unique visitors" to be comparable to what Nielsen reported (a fact others have noted). And guess what? Even Comscore doesn't put Hulu in the top 50 for April 2009 - which means even Comscore suggests that the number of unique visistors to Hulu is less than 19M (if someone has the actual number, I'd appreciate it).

So... move along... nothing to see here...