backtop

Print E-mail del.icio.us 39 comment(s) - last by Smilin.. on Sep 14 at 12:55 PM
The conclusion of our three part series looking to expose each next gen browser's greatest strengths and weaknesses

In our exploration of next generation browsers we first examined the user interface, installation details, time to install, and application launch times.  We next turned our attention to CPU and memory usages, as well as comparing and contrasting the security of the various browser offerings.  Topics we haven't looked at yet include rendering comparisons, synthetic benchmarks, plug-ins, and standards support, as well as our conclusions.  We planned just one more piece, but that's a lot of ground to cover, so we've broken it up into two more pieces. 

In this first piece, we'll look at synthetic benchmarks and plug-in support.  In the next (and final) segment, we'll examine rendering performance, standards support (including performance in the Acid3 benchmark), and our conclusions on the state of the browser war and who we believe the current winner(s) is/are.

6.  Synthetic Benchmarks:


The first synthetic benchmark we ran was Celtic Kane's Javascript test.  The homepage for the test showed Safari 4 to be the leading web browser.  Our own testing indicated that it has been passed by Google's Chrome.  Safari came in second, Opera in third, Firefox in fourth, and Internet Explorer came in at a distant fifth.


The next synthetic benchmark run was the popular Sunspider Javascript test.  This test again showed Chrome 4 beating Safari 4.  This time, though, Firefox was a runner up.  Opera performed unexpectedly poorly in this test, though Opera 10 managed a bit better performance.  And Internet Explorer 8 performed the worst of all, taking nearly eight times as long as Chrome 4 to complete this test.  Combined with Celtic Kane, these tests indicate the Safari's Squirrelfish engine, Chrome's V8 to be the clear JS leaders and the JS performance of Trident, IE 8's engine, to be dismal. 


Our next synthetic benchmark was Peacekeeper, a Futuremark test suite which we recently profiled.  Our testing with the suite indicated Chrome to be in the lead, followed closely by Safari and then Firefox.  Opera 10 managed middling results.  Meanwhile, Internet Explorer was the slowest, managing a mere sixth of the score of Chrome.  The suite looks at a number of aspects, including JS performance, CSS performance, rendering, and more, so it is a good general indicator of speed.


The last synthetic benchmark we used was How-To-Create UK's CSS test.  The test loads approximately 2,500 DIVs and times how fast the load takes.  Unfortunately, WebKit browsers (Safari and Chrome) aren't supported due to how they measure time, so we could only get results for Firefox, IE 8, and Opera.  For these browsers,  Opera 10 barely led Firefox 3.6a1, while Internet Explorer 8 yet again lagged in performance.

Conclusions to be drawn from the synthtetic tests -- Chrome is the fastest browser with Safari close behind.  Opera and Firefox are just slightly behind.  And Internet Explorer 8 is the slowest browser.  Even on content heavy sites, though, this performance difference is not as great as these tests might seem to indicate.  In fact, it may only account for a couple of extra seconds of load time.  Still, it could become an annoyance on content-heavy pages like Facebook.

7.  Plug-ins

A common misconception is that Firefox is the only browser that has plug-ins, add-ons, extensions, or otherwise named optional components.  Plug-ins/add-ons/extensions are, in fact, a vital part of modern browsers.  There are a diverse variety of formats include ActiveX, NPAPI, Java, Google Gears, RSS, and Atom. 

Mozilla Firefox does arguably lead in this field, having the most enthusiastic developer community for plug-ins, and the most useful plug-ins.  Firefox 3.6a1 does not provide is support for classic Java plugins and ActiveX plug-ins. 

Looking at the other browsers, Opera (which does not support Gears, ActiveX), Chrome (also no ActiveX), and Safari (no ActiveX, Gears partial only) all offer decent plug-in support, but their developer communities are still in a fledgling state.  Internet Explorer 8, on the other hand, offers a lot of plug-ins -- in fact, plug-ins are essential to improving the browser's standards support.  Unfortunately IE 8 does not support the NPAPI extension language commonly used for Mozilla.  IE 8 plug-ins are generally more for utility rather than aesthetics.

Plug-ins are most accessible to Firefox users, as Firefox's plug-in system is friendly enough for even beginning users.  Nonetheless, if other factors make you pick another browser, it's a good idea to check out what kind of plug-ins are available for it, as there will surely be some useful ones.

Note: All benchmarks were performed in 32-bit Vista on a Sony VAIO laptop with 3 GB of RAM, a T8100 Intel Processor (2.1 GHz), and a NVIDIA 8400 GT mobile graphics chip. The number of processes was kept consistent and at a minimum to reflect stock performance.

Comments     Threshold


This article is over a month old, voting and posting comments is disabled
If only
By bhieb on 9/10/2009 9:52:25 AM , Rating: 5
Not that I don't want to see this comparison. But if only there were some sort of sister company to DT that specialized in benchmarks.

I don't know .... maybe random pieces thrown into the middle of dozens of other articles over the course of a couple weeks, seems like a better idea /sarcasm.




RE: If only
By SilthDraeth on 9/10/2009 10:07:28 AM , Rating: 5
You mean... sort of like Anandtech?

I have Anandtech bookmarked, and if there is some interesting looking headline at DT, I click over to it.


RE: If only
By bhieb on 9/10/2009 11:06:56 AM , Rating: 3
Yes that was the joke. Why is this here and not on AT? It seems out of place. DT usually features articles about hardware reviews, usually linking to Tom's, AT, ... but it does not do their own (at least not to my recollection).


RE: If only
By ClownPuncher on 9/10/2009 1:02:44 PM , Rating: 4
Anand and crew are probably mired in P55 board tests right now.


RE: If only
By DLeRium on 9/10/2009 7:19:03 PM , Rating: 3
quote:
Anand and crew are probably mired in P55 board tests right now.


Have you seen the level of coverage on AT recently? We were promised a roundup of X58 mobos and we got a roundup of like 4. Tom's had to do like 3 - 4 SERIES of roundups because there's JUST SO MANY of them. We were promised an OCing guide to explain things like Uncore and all that crap on the i7 that I still don't get because I don't have an i7 yet, but that never came. We were promised a DDR3 roundup that never came.

It seems the only thing that they're solid on is video card reviews, but we haven't seen a huge release since the 4870/4850 really and at least AT is keeping on top of the minor half gen steps too.

Meanwhile Anand himself is running iPhone reviews like a CPU review when it should be left to the more knowledgeable mobile crowd out there who can ACTUALLY comment on Nokia's N-series and HTC's Hero because they truly know Symbian and Android in and out instead of looking at it as a MHz/FLOPS perspective that a computer user might.

Honestly I don't know what they're trying to do. More and more articles are about product previews and not REVIEWS. Honestly, I hope the i5/P55 coverage is solid. I hope...


RE: If only
By Bladen on 9/11/2009 7:00:47 AM , Rating: 2
We never did get an SSD article with RAID in it did we?


RE: If only
By fatedtodie on 9/10/09, Rating: -1
RE: If only
By bhieb on 9/10/2009 11:04:19 AM , Rating: 4
OMG even with the /sarcasm you still missed the point.

I was wondering why this was not featured in Anandtech instead of as a series of articles on DT, spread out over several days. It would be better served in one comprehensive article on AT than DT.

DT = News
AT = Benchmarks/Reviews.

I did not speak to the validity or bias at all.

Some ppl need it spelled out I guess.


RE: If only
By inperfectdarkness on 9/10/2009 8:36:02 PM , Rating: 2
i just ran acid 3 5 minutes ago.

ie = 12

ff = 92

given how slow ie is in the rest of these benchmarkts...isn't that just icing on the cake?


RE: If only
By baddog121390 on 9/10/2009 8:51:09 PM , Rating: 2
Opera = 100

Yep, Opera is definitely the best choice here.


RE: If only
By InsaneScientist on 9/12/2009 4:11:58 PM , Rating: 2
The thing that I don't get is that Opera 10 is supposed to be one of the fastest browsers out there... certainly faster than Firefox.
On my system however, it's noticeably slower than Firefox (with 16 add-ons - and one of them is greasemonkey, so then you've got a bunch of scripts to deal with).

Its initial load time is a little faster than FF, but once I'm in it's noticeably slower - and since I tend to leave browsers open for days at a time, the initial start time isn't much of an issue.

I just don't get it...


Ignoring
By Smilin on 9/10/2009 10:12:19 AM , Rating: 2
Your last set of benchmarks had a very obvious flaw and you have done nothing to address it.

I therefore place no faith in any of your testing.




RE: Ignoring
By Roffles on 9/10/2009 10:38:28 AM , Rating: 2
The biggest flaw of all is the notebook he's using. Understandably, I get entirely different numbers for some of these benches on my Q9550 desktop system. But the differences aren't as extreme....telling me these numbers are exposing the hardware limitations of his computer rather than the software limitations of the browsers.


RE: Ignoring
By foolsgambit11 on 9/10/2009 12:55:56 PM , Rating: 2
While I agree that there is little value in single-platform benchmarks, I disagree with your analysis of your benchmarks versus his. Unlike in hardware benchmarking, where you don't want any other hardware to bottleneck the product being tested, in software benchmarking it can be very valuable to see how the software handles resource starved situations. I'm not saying that Jason's single benchmarks on an average laptop accomplish that, by any means.

I don't think that these numbers are exposing the hardware limitations of his computer. They are exposing how each browser responds to his specific hardware limitations. In some ways, that's more valuable for software benchmarking than testing on a high-performance rig, since it gives the average Joe an idea of the performance he can expect.


RE: Ignoring
By Targon on 9/10/2009 11:19:28 PM , Rating: 2
The problem here is that speed(or lack of it) can come from multiple sources, so looking a bit deeper at WHERE the delays come from is important.

If the browser tends to use a lot of hard drive space for temporary files, or is slow to check for and load cached images, the speed of the hard drive will cause reduced performance. This is why the memory footprint needs to be looked at along side performance vs system memory to see WHERE the real speed of the browser comes from. A browser that is designed to keep as little in RAM as possible MIGHT use the hard drive more then.


RE: Ignoring
By gstrickler on 9/10/2009 1:16:27 PM , Rating: 2
The differences in your results aren't as extreme in absolute terms or in percentage terms? A faster machine will have much small absolute differences, but the percentages should be similar. On a "fast enough" machine, the differences would be below the level of perception, but that doesn't mean there aren't important difference between these browsers.

Most users don't have quad core machines, in fact, since laptop/notebook machines now outsell desktops, the machine Jason used is a much better representation of what a typical user will have.


RE: Ignoring
By fatedtodie on 9/10/09, Rating: 0
RE: Ignoring
By Smilin on 9/11/2009 2:26:02 AM , Rating: 2
Thanks. Now we just need an intelligent post to follow yours and Karma will again be normalized.

Why don't you go read the last article, troll?


RE: Ignoring
By Nekrik on 9/10/2009 1:51:07 PM , Rating: 3
It was frustrating that they did not address the memory usage issue in the last article (in comments or by way of an update), but to then post additional results in a follow up article using the same defunct configuration is embarrassing to both Jason and the site.

This exemplifies a lack of understanding and professionalism, or the skewed data serves their underlying intentions thus they refuse to correct it.


RE: Ignoring
By fatedtodie on 9/10/09, Rating: -1
RE: Ignoring
By Smilin on 9/11/2009 2:28:19 AM , Rating: 3
Everyone backed up their claims. We already had this debate. His data very much appears flawed and he needs to double check it.

Are you saying that his data is right?


RE: Ignoring
By Smilin on 9/11/2009 2:52:40 AM , Rating: 2
Actually...before I get more flawed logic spewed at me, here...memory use after 25min.

FF 3.5: 196,924 KB (started at ~156meg)
IE 8: 200,632 KB (started at ~185meg)

Windows 7 RTM.
Intel i7 950, 6gig DDR3

This is a far cry from 526megs that jason's previous data shows.


RE: Ignoring
By Alexstarfire on 9/11/2009 3:25:25 PM , Rating: 2
Win 7 != 32-bit Vista

Perhaps you should read before making useless comparisons to his test data. I'd be interested to know if you have any plug-in/add-ons installed for FF. I only have a couple and mine ends up lower than that. Curious to know how much RAM they can actually take up by themselves.


RE: Ignoring
By Smilin on 9/14/2009 12:55:48 PM , Rating: 1
Maybe you should just STFU.

Seriously all you do is bitch. If this is bothering you so much why don't you get off your lazy ass and go do some benchmarking yourself. I'm tired of reading your trolling.

The IE8 memory numbers in the original article are FLAWED. Before I was just concerned there might be a problem. Now I know.


rendering engine vs javascript engine
By puffpio on 9/10/2009 11:20:04 AM , Rating: 2
quote:
these tests indicate the Webkit engine (Safari, Chrome) to be the clear JS leader


Webkit is the RENDERING engine for Safari and Chrome. They each have their own JAVASCRIPT engines...

Safari's is called SquirrelFish
Chrome's is V8
Firefox's is TraceMonkey

dunno what IE's is called




RE: rendering engine vs javascript engine
By slyadams on 9/10/2009 12:07:10 PM , Rating: 2
Correct. Its somewhat worrying that the author is writing benchmarks and drawing conclusions when he clearly is clueless about how a browser is put together.


By Sazar on 9/10/2009 12:28:03 PM , Rating: 2
I am pretty sure the author is not writing the benchmarks :)


By gstrickler on 9/10/2009 1:07:57 PM , Rating: 3
quote:
dunno what IE's is called
Slow! I propose "SlugIE".


Plug-ins
By rpsgc on 9/10/2009 9:36:43 AM , Rating: 2
I think you should point out that Opera doesn't need a lot of extensions/widgets because it already has a lot of features built-in, unlike Firefox (mouse gestures, adblocking, etc)




RE: Plug-ins
By invidious on 9/10/2009 10:03:46 AM , Rating: 2
This kind of stuff was elluded to in the first installment of this series but it mainly covered setup features. Ad blocking was the only convinience feature covered.

I do not know why Mick would do a browser run down and not compare convinience features at all, hell I would focus primarily on them. They should have their own installment.


RE: Plug-ins
By jragosta on 9/10/2009 11:43:58 AM , Rating: 1
"elluded to"????

Maybe you should take an English course before posting?


RE: Plug-ins
By anuraaga on 9/11/2009 3:25:47 AM , Rating: 2
A better explanation of what plugins actually are is probably in order...

I think it's generally excepted that plugins are a way for the user to customize the browser. While I personally feel it's important to distinguish between plugins and extensions (as popularized by Firefox) due to their separate implementations and as such, separate goals, it'd be fair to group them together. I don't see why RSS and Atom are being labeled as plugins though - they're syndication methods for transmitting news. If clicking on an RSS button to have the feed show up in your bookmark bar/RSS window is enough to being a plugin format, then all browsers support a similarly awesome plugin format, the "bookmark" which allows you to customize your browser so you can jump to certain web pages with one click.

In addition, Java and Gears are plugins, not plugin formats. Web developers use them to customize their websites, users don't use them to customize their browser. This is most easily evidenced by the fact that users don't ever install a Java applet or a Gears...javascript code segment. I don't think labeling them as plugins and putting them in the same category as NPAPI and ActiveX really makes sense here.

Finally, NPAPI is a plugin language supported by all non-IE browsers, not the extension language commonly used by Mozilla (confusing if we group plugins and extensions together unfortunately). The latter is a combination of XUL for interface and Javascript for logic, and it's this combination of easy to use web-like technologies that has resulted in the large Mozilla extension community and will likely result in a similar community for Chrome as it is also using web technologies (HTML rather than XUL which is how Firefox should have been in the first place imho) for its experimental extension system. The former is generally (maybe always?) used to provide web developers with added resources when making websites, can be used with the <object> tag and are run in sandboxed native environments.


Opera 10 Turbo = Blurry Images?
By EJ257 on 9/11/2009 9:58:26 AM , Rating: 2
Does anyone else notice a difference in quality of images in Opera 10 with Turbo Mode enabled? They seem to be blurry when I browse with turbo but those same images will look fine when I turn turbo off and reload the page. I first noticed it while viewing test result graphs on here and on Anandtech. The text doesn't seem to be affected.




By Alexstarfire on 9/11/2009 3:19:13 PM , Rating: 2
What did you think Turbo Mode did? It doesn't just magically download stuff faster. It's the same deal with all the programs years ago that "made dial-up 5x faster."


RE: Opera 10 Turbo = Blurry Images?
By Belard on 9/11/2009 6:35:16 PM , Rating: 2
Its a image compression system. Useful for those still on dial up (or only have dial up access).

Turbo mode also has Auto mode.. but I'd say, turn it OFF and remove it from the tool bar.

Or... take off the Status bar, and turn ON the View bar, set it to the bottom. its more useful anyway.


Maxthon
By Klober on 9/10/2009 10:51:43 AM , Rating: 2
I wish more places would look at Maxthon. While it is based on IE it adds it's own things to the mix. For example, in Maxthon 3 (which currently is in beta testing) there is a Turbo mode which uses Webkit instead of Trident for improved JS performance. Also, it has many features built-in such as mouse gestures, Ad Hunter, and several others. Just my 2 cents. :)




Please....
By damianrobertjones on 9/10/2009 12:03:41 PM , Rating: 2
Fix the results for the memory yesterday which is obviously in-correct.

So, which is the best browser? Is there going to be a conclussion called part 4? Part 1,2 and 3 would have been better as one article. Please also use multiple machines if you can.




RE: Please....
By fatedtodie on 9/10/09, Rating: 0
Thanks for the info
By coachingjoy on 9/11/2009 10:07:37 PM , Rating: 2
Thank you for the time consuming work.
Might try the FF or Opera offerings.




"Vista runs on Atom ... It's just no one uses it". -- Intel CEO Paul Otellini

DailyTech Poll
Which web browser do you use on your primary personal machine? 






44 Comments









botimage
Copyright 2009 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki