Wednesday, May 20, 2009

Browser benchmarks: When did they get so stupid?

So, the claim that IE is faster than Firefox, Safari, or Chrome, is ridiculous at many levels (MS claims IE faster than other browsers), and Microsoft was appropriately ridiculed for it.

But so is the idea that you have a test that demonstrates that the new Safari builds are "above 15 times better performance than Internet Explorer 7 in the same system".

Seriously, that's just stupid., and renders the index meaningless.

Why not just multiple the index scores by 10? Then you can claim Safari is 150X faster than IE7.

Without scaling the index into a range that meaningfully communicates (or at least correlates) to user experience (which things like FPS and even 3DMark did for video cards), it renders the testing both invalid and irrelevant.

Labels: , ,

24 Comments:

Blogger Michael said...

Ok yes it's ridiculous to use somebody's hacked .onLoad()...it lets everybody cheat in order to win a benchmark.

Also using VirtualPC 2007 is simply wrong. This is seriously the worst virtualization software in the world. I've noticed it run 20x slower at some things than the native hardware. Installing Windows 7 on my laptop using VPC took TWELVE HOURS.

All that said, a test suite for this sort of thing shouldn't be impossible to construct. Firing keyboard events and waiting until the screen stops redrawing? You can do that in a few lines of code.

Regarding the construction of a benchmark: the mix of Javascript and number of sockets a browser allows (pipelining, 8 vs. 2 connections, etc.) will win some tests, often by a landslide.

Some browsers will do better with multi-core, some on single. But single-task "load Google.com" should be a predictable test that everyone can understand.

May 20, 2009 8:33 PM  
Blogger Sree Kotay said...

I think you're right - part of the problem is finding the right combination of technical merit and credibility to make a test stick is tough.

Like - SunSpider is great for JS implementors, but not at all meaningful for consumers - so these things get warped.

May 25, 2009 11:10 PM  
Blogger aiqing said...

Do you know the runescape power leveling, in the game you need the runescape gold. it can help you increase your level. My friends always asked me how to buy runescape accounts, i do not know he spend how much money to buy the runescape money, when i see him in order to play the game and search which the place can buy it. i also happy with him.
======================

May 26, 2009 3:31 AM  
Blogger Patrick Meenan said...

I'm sure I'll get ridiculed and harassed for it but I have no problem believing IE8 is faster than the other (current) browsers. Besides going up to 6 parallel connections like everyone else they made the downloading of javascript a non-blocking process. All of the others will not download anything else in parallel while they download and evaluate the js.

They are only getting a short head start since everyone else is copying the technique but for the vast majority of sites this is way more impactful than a fast js engine.

There have been over 25,000 unique urls tested on my web performance site (http://www.webpagetest.org) and it is VERY rare to see a site that spends any noticable time executing js. A ton of sites pull in several different js files though (all in the head, blocking even the HTML from rendering). A fast js engine is important for rich apps like gmail and google docs but not browsing.

May 28, 2009 9:44 PM  
Blogger qwe said...

We pay cheap rs gold when we feel you will appreciate the updates. We feel that these changes and some runescape money are worth telling you about. runescape gold of RS Online Game, we have made several changes to Fun Orb. If you wish to buy runescape to explore, you should subscribe as a Fun Orb member. Shattered Plans use rs gold to allow up to six players to battle for supremacy.

June 21, 2009 12:57 AM  
Blogger aiqing said...

cheap jordan shoes
For further information about the Warhammer CD-key and Warhammer Timecard, Please keep an eye on
air jordan shoes.
For more information please keep an eye on michael jordan shoes
air jordans
air jordon

July 05, 2009 9:11 PM  
Blogger modernsharewaredown said...

Breitling Watches
Cartier Watches
Omega Watches
Tag Heuer Watches
Panerai Watches
IWC Watches
Patek Philippe Watches
Chopard Watches
Gucci Watches
Bell & Ross Watches

August 11, 2009 5:59 AM  
Blogger michal said...

See UGG boots uk store large range of UGG boots online. We have mens and womens boots in all styles. Plus you UGG boots in many types and sizes
Classic Cardy UGG Boots
Classic Leopard UGG Boots
ugg mini boots
UGG Classic Short
UGG Classic Tall
Metallic UGG Boots
ugg nightfall boots
Paisley UGG Boots
Sundance UGG Boots
Links of London

August 11, 2009 6:13 AM  
Blogger ed-hardy funny said...

ed hardy

buy ed hardy

cheap ed hardy

ed hardy

ed hardy hoody

ed hardy men hoody

ed hardy women hoody

ed hardy long Sleeve

ed hardy men long sleeve tee

ed hardy women long sleeve tee

ed hardy tanks

ed hardy tank top

ed hardy twist specialty tank

ed hardy jeans

ed hardy men Jeans

ed hardy women jeans

ed hardy bottoms

ed hardy women platinum drawstring pants

ed hardy women stretch miniskirt

ed hardy swimwear

ed hardy bikini

ed hardy women's one piece swim suit

ed hardy underwear

ed hardy men underwear

ed hardy lingerie

ed hardy belts

ed hardy sunglasses

ugg boots
ed hardy

August 18, 2009 3:11 AM  
Blogger caiboy said...

Looking for ugg boots? you can buy ugg boot online. Many choices, like ugg cardy, ugg classic cardy,ugg classic tall,ugg classic cardy boots, etc. You can certainly find your ideal one there. The ugg boots online mall wins many customs for its unique ugg boots uk and cardy ugg boots.You can also find cheap ugg cardy boots and reading glasses there.

September 08, 2009 10:20 PM  
Blogger caiyilinlovelove said...

A man can fail many times,Infants Erin Ugg Boots but he isn't a failure until he begins to blame somebody else. Classic Mini Ugg Boots dan Ugg Sandals

September 10, 2009 3:35 AM  
Blogger Tiến Trung said...

風俗
デリヘル
風俗
デリヘル 風俗
エロゲー
ソープランド
吉原 ソープランド
出会い
出会い
デリヘル
すすきの ソープランド
新宿 ソープランド
千葉 ソープランド
埼玉 ソープランド
神奈川 ソープランド
吉原 ソープランド
sod
バイブ
風俗求人 高収入
都内 キャバクラ 全額日払い
無料動画
大阪 風俗
大阪 風俗
神戸 風俗
裏DVD
裏DVD
風俗 求人
風俗 求人
デリヘル 新宿
デリヘル 東京
千代田区 デリヘル
台東区 デリヘル
墨田区 デリヘル

September 12, 2009 1:11 PM  
Blogger Tiến Trung said...

サイト制作
アートメイク
エクステ
アパレル
アロマセラピー
ダイエット クエン酸
ダイエット サプリ
レストラン ウェディング
ランコム
エスティローダー
ルナソル
神社
建設機械レンタル
ワイン
シャンパン
ダイエット 方法
ダイエット 運動
競馬 ゲーム
競馬
SPUR
シュプール
コーディネート
防災グッズ
ペット用品
SEO
格安 SEO
SEO 成果報酬

September 12, 2009 1:11 PM  
Blogger Tiến Trung said...

和歌山 デリヘル
神戸 デリヘル
姫路 デリヘル
大阪 デリヘル
京都 デリヘル
大阪 デリヘル
神戸 デリヘル
和歌山 デリヘル
奈良 デリヘル
京都 デリヘル
神戸 デリヘル
姫路 デリヘル
奈良 デリヘル
京都 デリヘル
大阪 デリヘル
大阪 デリヘル

September 12, 2009 1:12 PM  
Blogger Tiến Trung said...

和歌山 デリヘル
神戸 デリヘル
姫路 デリヘル
大阪 デリヘル
京都 デリヘル
奈良 デリヘル
大阪 デリヘル
神戸 デリヘル
和歌山 デリヘル
奈良 デリヘル
京都 デリヘル
神戸 デリヘル
姫路 デリヘル
和歌山 デリヘル
奈良 デリヘル
京都 デリヘル
大阪 デリヘル
南大阪 デリヘル
大阪 デリヘル

September 12, 2009 1:12 PM  
Blogger 58488 said...

帯広 デリヘル風俗情報

帯広 デリヘル風俗情報モバイル


帯広 デリヘル gold(ゴールド)

帯広 デリヘル gold(ゴールド)モバイル


帯広 デリヘル風俗情報ブログ


札幌焼肉遥月

September 21, 2009 7:50 AM  
Blogger caiyilinlovelove said...

Whether you call them "sheepskin boots" or "ugg boots,"Ugg Sandals they are incredibly warm, and soft. These are made for winter conditions and they have become major hits with Infants Erin Ugg Boots trendsetters and celebrities alike over the past couple of years.

October 26, 2009 3:18 AM  
Blogger ed-hardy funny said...

gucci shoes
ugg boots
gucci boots
gucci sandals
gucci wedge shoes

bailey button ugg boots
classic cardy ugg boots
classic mini ugg boots
classic short ugg boots
classic tall ugg boots
infant erin ugg boots
nightfall ugg boots
short metallic ugg boots
sundance II ugg boots
tall metallic ugg boots
ugg tasmina braid sandals
ugg tasmina sandals
ultra short ugg boots
ultra tall ugg boots

October 27, 2009 11:36 PM  
Blogger hanyi said...

winter ugg boots

cheap ugg boots

Air Max
UGG Classic Cardy Boots

UGG Classic Tall Boots

November 09, 2009 4:14 AM  
Blogger worlda bush said...

Mbt sale is worldwide. discount Mbt shoes can be got, Mbt walking shoes is only $87.

December 09, 2009 6:27 AM  
Blogger ed-hardy funny said...

ugg classic
classic tall ugg boots
short metallic ugg boots
tall metallic ugg boots
ugg classic argyle knit

ugg sandals
ugg amelie suede sandals
ugg coquette sandals
ugg dakota sandals

ugg morocco sandals
ugg tasmina braid sandals
ugg tasmina sandals

ugg ultra boots
ultra short ugg boots

kids ugg boots
ugg sundance grab bags
classic cardy ugg boots
classic mini ugg boots
classic short ugg boots
ugg classic crochet boots
ugg gypsy sandals
ugg halendi sandals
ultra tall ugg boots

December 14, 2009 4:31 AM  
Blogger ed-hardy funny said...

ugg classic
classic tall ugg boots
short metallic ugg boots
tall metallic ugg boots
ugg classic argyle knit

ugg sandals
ugg amelie suede sandals
ugg coquette sandals
ugg dakota sandals

ugg morocco sandals
ugg tasmina braid sandals
ugg tasmina sandals

ugg ultra boots
ultra short ugg boots

kids ugg boots
ugg sundance grab bags
classic cardy ugg boots
classic mini ugg boots
classic short ugg boots
ugg classic crochet boots
ugg gypsy sandals
ugg halendi sandals
ultra tall ugg boots

December 14, 2009 4:31 AM  
Blogger worlda bush said...

Mbt Shoes Sale is on promotion now,welcome to order Mbt sport 2 shoes, men's Mbt walking shoes & Mbt black lami shoes at a discount price.

December 29, 2009 9:55 PM  
Blogger worlda bush said...

ghd straightners catch people's eyes.people expect to see new items of ghd hair straighteners, while the classic items are still hot, like ghd purple.

December 31, 2009 4:17 AM  

Post a Comment

Links to this post:

Create a Link

<< Home