May 20, 2009

Browser benchmarks: When did they get so stupid?

So, the claim that IE is faster than Firefox, Safari, or Chrome, is ridiculous at many levels (MS claims IE faster than other browsers), and Microsoft was appropriately ridiculed for it.

But so is the idea that you have a test that demonstrates that the new Safari builds are "above 15 times better performance than Internet Explorer 7 in the same system".

Seriously, that's just stupid., and renders the index meaningless.

Why not just multiple the index scores by 10? Then you can claim Safari is 150X faster than IE7.

Without scaling the index into a range that meaningfully communicates (or at least correlates) to user experience (which things like FPS and even 3DMark did for video cards), it renders the testing both invalid and irrelevant.

1 comment:

Sree Kotay said...

I think you're right - part of the problem is finding the right combination of technical merit and credibility to make a test stick is tough.

Like - SunSpider is great for JS implementors, but not at all meaningful for consumers - so these things get warped.