Browser benchmarks: When did they get so stupid?
So, the claim that IE is faster than Firefox, Safari, or Chrome, is ridiculous at many levels (MS claims IE faster than other browsers), and Microsoft was appropriately ridiculed for it.
But so is the idea that you have a test that demonstrates that the new Safari builds are "above 15 times better performance than Internet Explorer 7 in the same system".
Seriously, that's just stupid., and renders the index meaningless.
Why not just multiple the index scores by 10? Then you can claim Safari is 150X faster than IE7.
Without scaling the index into a range that meaningfully communicates (or at least correlates) to user experience (which things like FPS and even 3DMark did for video cards), it renders the testing both invalid and irrelevant.
posted by Sree Kotay at 12:53 PM
26 Comments:
Ok yes it's ridiculous to use somebody's hacked .onLoad()...it lets everybody cheat in order to win a benchmark.
Also using VirtualPC 2007 is simply wrong. This is seriously the worst virtualization software in the world. I've noticed it run 20x slower at some things than the native hardware. Installing Windows 7 on my laptop using VPC took TWELVE HOURS.
All that said, a test suite for this sort of thing shouldn't be impossible to construct. Firing keyboard events and waiting until the screen stops redrawing? You can do that in a few lines of code.
Regarding the construction of a benchmark: the mix of Javascript and number of sockets a browser allows (pipelining, 8 vs. 2 connections, etc.) will win some tests, often by a landslide.
Some browsers will do better with multi-core, some on single. But single-task "load Google.com" should be a predictable test that everyone can understand.
I think you're right - part of the problem is finding the right combination of technical merit and credibility to make a test stick is tough.
Like - SunSpider is great for JS implementors, but not at all meaningful for consumers - so these things get warped.
I'm sure I'll get ridiculed and harassed for it but I have no problem believing IE8 is faster than the other (current) browsers. Besides going up to 6 parallel connections like everyone else they made the downloading of javascript a non-blocking process. All of the others will not download anything else in parallel while they download and evaluate the js.
They are only getting a short head start since everyone else is copying the technique but for the vast majority of sites this is way more impactful than a fast js engine.
There have been over 25,000 unique urls tested on my web performance site (http://www.webpagetest.org) and it is VERY rare to see a site that spends any noticable time executing js. A ton of sites pull in several different js files though (all in the head, blocking even the HTML from rendering). A fast js engine is important for rich apps like gmail and google docs but not browsing.
Mbt sale is worldwide. discount Mbt shoes can be got, Mbt walking shoes is only $87.
Mbt Shoes Sale is on promotion now,welcome to order Mbt sport 2 shoes, men's Mbt walking shoes & Mbt black lami shoes at a discount price.
ghd straightners catch people's eyes.people expect to see new items of ghd hair straighteners, while the classic items are still hot, like ghd purple.
2010 latest style Seven Jeans.A variety of styles of True Religion Jeans offered here.High praised from the cusotmers.Official online store for Seven for mankind,free shipping and fast delivery
The best Coach Wallet and Coach Handbags
supplier on the Internet here. We are proud to offer our customers high quality and low price Coach Outlet.About 70% discount now.
Post a Comment
Links to this post:
Create a Link
<< Home