backtop


Print 88 comment(s) - last by testerguy.. on Mar 15 at 10:46 AM

Benchmarks or GTFO!

Yesterday when Apple unveiled the new iPad, the crew from Cupertino took some time to brag about its new A5X processor in comparison to NVIDIA’s Tegra 3. Apple certainly isn't widely known for offering up benchmarks on its own, so we'll likely have to wait until iPads land in the hands of reviewers and geeks around the web.
 
Apple used the iPad unveiling to boast that the A5X chip inside the new iPad is two times faster than A5, and four times more powerful in graphics performance than the Tegra 3.
 
 
NVIDIA isn't buying those claims without proof. The graphics company wants to know how Apple came by that number. Ken Brown, a spokesman for NVIDIA, stated, "[It was] certainly flattering " for Apple to compare its newest chip to their part.
 
Brown continued, “We don’t have the benchmark information. We have to understand what the application was that was used. Was it one or a variety of applications? What drivers were used? There are so many issues to get into with benchmark.”
 
Anyone that follows tech knows benchmarks are often handpicked to favor one particular brand over another when it comes to claims such as these. So it should be interesting to see if the new iPad’s performance lives up to the claims.

Source: ZDNet



Comments     Threshold


This article is over a month old, voting and posting comments is disabled

RE: Apple Picked the Wrong @!**@fight
By therealnickdanger on 3/8/2012 11:02:58 AM , Rating: 1
Exactly, the iPad 2 is already 10-200% faster than Tegra 3. What's so unbelievable about the A5X being 4X faster? NVIDIA never could handle their pride very well.

http://www.anandtech.com/show/5163/asus-eee-pad-tr...

For the record, I do not own or plan to own any Apple products.


RE: Apple Picked the Wrong @!**@fight
By theapparition on 3/8/2012 12:17:39 PM , Rating: 5
This is why the kiddies shouldn't interpret benchmark results.

The iPad2 is indeed faster than the Tegra 3 in the Transformer Prime for GPU operations. For CPU operations, the iPad2 is slower in every benchmark.

Oh, why did you leave that out? Typical cherry picking results to satisfy your argument.

Then there is the iPad 3/HD. The quad core SGX540MP4 is certainly going to be a screamer, but then again, it also has to render and push 4X the number of pixels. Overall, until we see it benchmarked, I postulate graphics performance will be close to the existing iPad2, with maybe a slight advantage. Certainly not at anything approaching twice the speed of the iPad2 though, despite Apple's claims.


RE: Apple Picked the Wrong @!**@fight
By Guspaz on 3/8/12, Rating: -1
RE: Apple Picked the Wrong @!**@fight
By harshbarj on 3/8/2012 1:05:00 PM , Rating: 4
What benchmarks? There has yet to be a benchmark on the new ipad and there will not be until after the 16th. If you're going ONLY by apples claim you have a LOT to learn. Never trust the claims of performance from a company.


RE: Apple Picked the Wrong @!**@fight
By Guspaz on 3/8/12, Rating: -1
RE: Apple Picked the Wrong @!**@fight
By harshbarj on 3/8/2012 1:34:04 PM , Rating: 3
That's a rather bold statement given we don't know what kind of bottlenecks the system might have. What if they did not double the graphic ram bandwidth, what if the cpu can't feed the quad core gpu fast enough.

Now I'm not saying it's not 4x faster, just we can't even guess till it's released.


By Uncle on 3/8/2012 1:31:49 PM , Rating: 4
"Never trust the claims of performance from a company."

Especially when apple and its affiliates are taking in pre-orders.


RE: Apple Picked the Wrong @!**@fight
By theapparition on 3/8/2012 3:45:50 PM , Rating: 4
You have some nerve to claim I'm "talking out of my ass."

I only stated facts that the graphics power on the A5 are superior, yet the CPU power is inferior compared to the quad core Tegra 3. The A5X adds two new cores (SGX540MP4 vs SGX540MP2), but also has 4X the number of pixels to render due to the higher resolution.

I was also quite clear that I was "guessing" the final benchmark numbers.

However, my reply was to the OP, and to many others who consistently point at one benchmark and then claim product A is faster than product B....when there's other benchmarks that state the opposite. At no time did I ever claim Apple stated anything.

Your reading more into my reply is certainly your fault at comprehension, not mine.


RE: Apple Picked the Wrong @!**@fight
By testerguy on 3/9/2012 8:04:59 AM , Rating: 1
Well lets be clear what you actually claimed.

First, we evidenced the claim Apple made about superior GPU performance by citing GPU performance being faster even on the iPad 2. While we didn't specifically refer to 'GPU' every time, I would assume that could logically be inferred given the context and the subject of this article.

You then stated 'Typical cherry picking results to satisfy your argument.'

THAT is your failing. The discussion is about GPU, not CPU. You did NOT simply point out 'facts' as you claim, you accused our absolutely relevant comments of being 'cherry picked' - which was clearly a nonsense.

Finally - the difference between the CPU performance is very much negligible when compared to the gap in GPU performance, so overall even if you misinterpreted what was being said, you were still wrong. Indeed, to believe that just because the CPU is very marginally slower in the A5 (and you don't know what clock speed its running at in the new iPad), that means that the device isn't faster overall, is exactly the 'cherry picking' you accused us 'kiddies' of doing.


RE: Apple Picked the Wrong @!**@fight
By theapparition on 3/9/2012 9:21:13 AM , Rating: 2
Nowhere did anyone specify anything about the GPU. While clear in your head, your argument about being in context holds no merit.

The convenient argument is that the iPad2 is faster than the Prime. That is only true for certain graphics operations. Other tasks can be completed faster on the Prime, such as web browsing.

The general statement that the iPad2 is twice as fast, hence the iPad3 will be 4 times as fast is false, unless you add the qualifier that you are talking about GPU operations. That was my issue. And people who claim anything else are cherry picking the results.

Stating that I didn't point out facts is non-sense. There is nothing non-factual about my original post. Go challenge any point. A single one. You may not have liked how I phrased it, but the content was factual. Claiming that I falsified something just goes to show how desperate you are to make your point.

And your final comment is such a stretch that it's not worth debating. Try again. Nothing I said is incorrect, and the only part that is up for debate is where I'm making a prediction.


By testerguy on 3/9/2012 11:41:18 AM , Rating: 2
quote:
Nowhere did anyone specify anything about the GPU


The comment everyone was replying to did. As did the article. As did Apple when they made the claim. The claim being discussed which referred to GPU only .

quote:
While clear in your head, your argument about being in context holds no merit.


So all of the above escaped you? You are not aware of context? You do not understand that CPU benchmarks are irrelevant to this article?

quote:
The convenient argument is that the iPad2 is faster than the Prime. That is only true for certain graphics operations. Other tasks can be completed faster on the Prime, such as web browsing.


It's not a 'convenient argument', it's the accepted reality, as evidenced by the conclusion reached by numerous impartial tech sites such as Anandtech. They concluded that the graphics performance is often 2x faster on the iPad 2 than the Tegra 3. Web browsing benchmarks are just as much a measure of browser technology as they are hardware, and any difference in speed of web browsing is barely distinguishable to an end user. Double the graphics performance in games, however, and that is noticable.

quote:
The general statement that the iPad2 is twice as fast, hence the iPad3 will be 4 times as fast is false, unless you add the qualifier that you are talking about GPU operations.


Again, the whole claim made by Apple referred to the GPU, and the GPU only. So your condition was already satisfied before this article was even published.

quote:
Stating that I didn't point out facts is non-sense. There is nothing non-factual about my original post. Go challenge any point. A single one. You may not have liked how I phrased it, but the content was factual. Claiming that I falsified something just goes to show how desperate you are to make your point.


I can't believe I have to re-explain to you that your failing was in calling us 'kiddies', implying we had misinterpreted any benchmarks, and stating that we were cherry picking the results. All three have proven to be your own failure to understand context and relevance.

quote:
And your final comment is such a stretch that it's not worth debating. Try again. Nothing I said is incorrect, and the only part that is up for debate is where I'm making a prediction.


It's not up for debate for reasons of obviousness. A 10% CPU deficit (which may or may not be present in the A5X - since as I already told you, it may be clocked higher in the new iPad) is barely noticeable for an end user. a 100% GPU deficit most definitely is. Again, refer to any impartial and expert tech site like Anandtech for confirmation of these figures. It's also logically obvious, to pretty much anyone.

http://images.anandtech.com/graphs/graph5163/42762...
http://images.anandtech.com/graphs/graph5163/42763...
http://images.anandtech.com/graphs/graph5163/42748...
http://images.anandtech.com/graphs/graph5163/42749...


RE: Apple Picked the Wrong @!**@fight
By testerguy on 3/9/12, Rating: -1
RE: Apple Picked the Wrong @!**@fight
By theapparition on 3/9/2012 9:40:52 AM , Rating: 2
quote:
Oh and also, the number of pixels is absolutely irrelevant when comparing GPU's in terms of benchmarks, so that argument of yours is also a nonsense.

Huh?

It's only non-relevant if the iPad3 will run at lower resolution, which it wont.

This is similar to how the iPhone4 GPU benchmarked lower than the iPhone3GS. This was due to the screen resolution doubling (sound familiar?) while using the same exact GPU.

So while the MP4 is certainly faster, the only thing that matters is real world use. And you have to take that higher resolution into account. You don't test at lower resolution to try to compare, you test exactly how the devices are meant to be used.

And in the end, with the iPad3, you'll get a screen that is much higher resolution, looks better, and still has a graphics advantage over currently released Android tablets. I just think it's not going to be as rosy as Apple claims.

quote:
Otherwise, you would end up with ridiculous conclusions like a far superior and faster GPU in every case being labelled as inferior, and we wouldn't want such delusion, would we?

Your conclusion is beyond ridiculous. Yes, you absolutely want that conclusion. You completely miss the point.

Screen resolution doubled (4x more pixels). If the new improved GPU system (including memory, speed, etc) can't push all those pixels fast enough, the user experience is going to be compromised (eg, laggy operation). It doesn't matter if it's twice as fast at the same lower resolution, it only matters if it can handle the new higher resolution and provide the same or better experience. So in a nutshell, we absolutely want to see if the new GPU can't keep up. Users don't care about theoretical performance at lower resolution, they only care if their shiny new toy works better than their old one.

Get with the program.


RE: Apple Picked the Wrong @!**@fight
By testerguy on 3/9/2012 11:49:13 AM , Rating: 2
quote:
It's only non-relevant if the iPad3 will run at lower resolution, which it wont.


You don't understand how GPU's are benchmarked, then.

quote:
This is similar to how the iPhone4 GPU benchmarked lower than the iPhone3GS. This was due to the screen resolution doubling (sound familiar?) while using the same exact GPU.


No.

http://images.anandtech.com/graphs/graph4971/41964...
http://images.anandtech.com/graphs/graph4971/41965...
http://images.anandtech.com/graphs/graph4971/41966...

quote:
So while the MP4 is certainly faster , the only thing that matters is real world use. And you have to take that higher resolution into account. You don't test at lower resolution to try to compare, you test exactly how the devices are meant to be used.


This is not true in the context of Apples claim. For their claim to be true, the GPU simply has to be 4x faster in the same scenario , which means the same resolution. The bold part in your sentence is the part Apple claimed. The rest is a rant about how resolution impacts FPS - that doesn't change the GPU performance. Furthermore, performance arguably has to include resolution, since as I've said in another post, 100 FPS at a resolution of 10 x 10 isn't good performance. To confuse FPS with graphics performance is your next failing.

quote:
And in the end, with the iPad3, you'll get a screen that is much higher resolution, looks better, and still has a graphics advantage over currently released Android tablets. I just think it's not going to be as rosy as Apple claims.


I think what Apple claimed and what you heard are two different things.

quote:
Screen resolution doubled (4x more pixels). If the new improved GPU system (including memory, speed, etc) can't push all those pixels fast enough, the user experience is going to be compromised (eg, laggy operation). It doesn't matter if it's twice as fast at the same lower resolution, it only matters if it can handle the new higher resolution and provide the same or better experience. So in a nutshell, we absolutely want to see if the new GPU can't keep up. Users don't care about theoretical performance at lower resolution, they only care if their shiny new toy works better than their old one.


All of the above is analysing the performance of the iPad as a whole. It has nothing to do with benchmarking the GPU, it's a measure of user experience. Those are not the same thing

Whether the GPU is connected to a 50 mile monitor with 200 trillion pixels, or a 10x10 display, it is still the same GPU, and it still has the same performance relative to other GPU's.

And then you say 'Get with the program.'? Talk about clueless.


RE: Apple Picked the Wrong @!**@fight
By testerguy on 3/9/2012 12:00:20 PM , Rating: 2
I've just seen you compared the iPhone 3GS to iPhone 4 (rather than 4 vs 4S) and said they benchmarked differently.

They have the same GPU - it benchmarks exactly the same.

The links do still serve a purpose in proving that benchmarks are at a constant resolution.


RE: Apple Picked the Wrong @!**@fight
By theapparition on 3/9/2012 1:42:14 PM , Rating: 2
Glad you finally realized that.

Here's the phone benchmark on Anandtech. Since you seem hung up on only caring about individual component performance, try taking a look at how the phone performs.

http://www.anandtech.com/show/4243/dual-core-snapd...

GLBench-Egypt
iPhone3GS: 14.5
iPhone 4: 5.9

GLBench-Pro
iPhone3GS: 24.5
iPhone 4: 16.2

From Anandtech's own commentary:
quote:
Keep in mind that with GLBenchmark 2.0 we still cannot run at any resolution than native – in this case 800x480 (WVGA) – and the same applies for other devices in the suite, they're all at respective native resolutions.

The reason for the iPhone 4 lagging iPhone 3GS is display resolution, which unfortunately right now we can't test at anything other than native.<\quote>


By testerguy on 3/14/2012 4:54:18 PM , Rating: 2
The comment I made about comparisons of iPhone 4 to 4S still prove my point every bit as much. All of them, and benchmarks against other phone GPU's, are all at the same resolutions, not native resolutions.

I can't believe you posted that quote from Anandtech, which actually further proves my point.

'Unfortunately right now' they can't test on 'anything other than native'. That's an APOLOGY. Because, unlike you, Anand knows how proper benchmarks work.

Here's another quote from him:

quote:
they're all at respective native resolutions. GLBenchmark 3.0 will fix this somewhat with the ability to render into an off-screen buffer of arbitrary size.


Can you read that? Keyword - FIX. In all subsequent reviews where there has been an option to keep the resolution constant, he has.

As for why it's wrong, I will approach in my reply to your other failed argument on this topic.


RE: Apple Picked the Wrong @!**@fight
By theapparition on 3/9/2012 1:28:29 PM , Rating: 2
quote:
You don't understand how GPU's are benchmarked, then.

I understand completely. You don't understand that people don't care about benchmarks on equal footing, only how a device performs in it's configured state. Only a complete moron would believe a 4core part of the same series would perform worse than it's two core counterpart on equal settings. I've never said anything to the contrary. The only thing that matters is if the new part works better at the higher resolution.

What part of that don't you get?

I can't possibly believe you're arguing this. It's pointless, give it up.

Your comment below about testing an engine on equal ground shows your complete lack of knowledge on the real world. The whole package matters, not trying to interpret the sum of it's components.

I can see you in a product design meeting right now.

You: "Yes Mr. Product Manager, the overall system is slower, it's quite laggy. But, but.....the GPU is faster. It's twice as fast as the old product tested on the same settings."

PM: "But we don't have the same settings, do we? Didn't you design to the human factors requirements definition?"

You: "But I can product a benchmark that shows it's twice as fast at the old settings"

PM: "And that leaves us with an unusable product"

You: "I don't care, it's twice as fast"

Boss: "Clear out your desk. Security!"

And for the record, I don't trust either Apple's or Nvidia's claims, which have been wildly inflated in the past. No amount of your doublespeak will justify those claims.


By testerguy on 3/14/2012 5:07:32 PM , Rating: 2
quote:
I understand completely.


No, you don't. You seem to think that GPU benchmarks are a 'human factors' metric. Guess what, they aren't. They are what they are described as - a way of comparing the relative capabilities of GPU's. No more, no less.

quote:
You don't understand that people don't care about benchmarks on equal footing


My point doesn't depend on what people 'care' about. It depends on the reality of how benchmarks work. Again, you're making a fundamental mistake in not realising what GPU benchmarks are. Your point is essentially that you don't care much about GPU benchmarks. That doesn't mean that the way benchmarks are carried out changes. Native resolutions is another, different, and equally flawed metric, in other ways. It is not the same thing to test in native resolutions as it is to benchmark a GPU when keeping all other variables constant.

quote:
The only thing that matters is if the new part works better at the higher resolution.


Again, you refer to what 'matters' - instead of what constitutes a GPU benchmark. You are, again, confusing your flawed belief that GPU benchmarks don't matter with meaning that the fundamental method of calculating benchmarks is different.

quote:
Your comment below about testing an engine on equal ground shows your complete lack of knowledge on the real world. The whole package matters, not trying to interpret the sum of it's components.


Actually, your ignorance to what I'm saying shows a complete lack of ability to read. When you benchmark GPU's, you want to do precisely that, benchmark a GPU. Benchmarking a DEVICE , as opposed to a GPU , at it's native resolution, is a VERY DIFFERENT METRIC. It is also just as flawed. As in the other examples I've given, is 100 FPS native resolution of 10 x 10 better than 60 fps at 2000 x 1000? Of course not. Clearly, then, native FPS isn't the only relevant metric. More relevantly, in this case, Apples claim specifically refers to the GPU, and not to 'native performance'. So you're wrong on multiple levels.

Here's a further reason why you're wrong. Just like the iPhone 4 could, the new iPad can actually render games at the resolution of the iPad 2, and scale it up. So it's 'native' resolution can actually change, depending on what the game designer builds. This gives the developers the choice of either 4x the resolution, or 4x the performance, or some mix of the two. What the GPU benchmarks tell us, is that at any given graphics quality, the new iPad GPU will be up to 4x faster. This is clearly backed up by the benchmarks which show the iPad 2 as significantly faster than Tegra 3 already, and the logical deduction that the new GPU is 2x faster. Again, something we learn through GPU BENCHMARKS. If we took your flawed method we would conclude that the GPU in the iPhone 4 is worse than the iPhone 3GS. Clearly a nonsense.

It's beyond belief that you are in complete denial about the established and accepted method of benchmarking GPU's, you're absolutely delusional. Your whole argument about the experience of the device as a whole is nothing to do with GPU benchmarks, and that you confuse the two is the whole foundation of your failure. Your failed 'conversation' also fails to factor in the fact that you can't simply compare speed, even when looking at the device experience, because the higher resolution can make it worth sacrificing FPS in certain situations.


By really on 3/9/2012 1:08:52 PM , Rating: 2
You don't know what your talking about if you think resolution has nothing to do with performance or benchmarks. Go checkout any reputable GPU Benchmark and they always show results from different resolutions. Why? Because the better the resolution the harder the GPU has to work to render the same information.


By Uncle on 3/8/2012 1:27:08 PM , Rating: 2
Forgot to mention the increase in size to accommodate the larger battery to increase the processor speed for the video rendering. Apple does admit tho that they have reached the tablets shrinkage capability in a round about way.


RE: Apple Picked the Wrong @!**@fight
By Kurz on 3/8/2012 12:53:35 PM , Rating: 3
Just because you put in a new engine doesn't mean it'll be faster. Example if you put in a new engine into a truck to tow something large, you will not see the performance increase at least on the highway since you are going the same speed as before.

In engineering its always about designing for the intended use. They are pushing alot more pixels for that new screen of theirs.

Regardless of all, there is no point arguing. We need the Ipad in our hands before we can figure out its performance.


RE: Apple Picked the Wrong @!**@fight
By testerguy on 3/9/2012 8:18:32 AM , Rating: 1
The performance claims of the GPU performance assume a constant resolution.

Just like if you were benchmarking engines, you would compare it in two equally heavy chassis. You wouldn't say a truck engine was less powerful just because it's in a truck.

They are pushing more pixels, which is why they require the extra performance, but that has nothing to do with the general capability of the GPU.


RE: Apple Picked the Wrong @!**@fight
By theapparition on 3/9/2012 1:14:30 PM , Rating: 2
So you're totally lost.

If you care about performance, the 0-60 and 1/4 mile times mean nothing. The suspension settings, the tires, and engine controls. The end user experience means nothing.

All that matters is that you take every engine out of every vehicle, put it into a neutral chassis, and test there. That way you can find the true performance of each engine.

Good job, you go right ahead and start trying that. Because the performance of the completed vehicle means nothing, just how each part works.

I guess I need to add sarcasm tags, or that might escape you.
<\sarcasm>


By testerguy on 3/14/2012 5:11:33 PM , Rating: 2
quote:
So you're totally lost. If you care about performance, the 0-60 and 1/4 mile times mean nothing. The suspension settings, the tires, and engine controls. The end user experience means nothing. All that matters is that you take every engine out of every vehicle, put it into a neutral chassis, and test there. That way you can find the true performance of each engine. Good job, you go right ahead and start trying that. Because the performance of the completed vehicle means nothing, just how each part works. I guess I need to add sarcasm tags, or that might escape you. <\sarcasm>


It's so ironic that you say this sarcastically when it's the only time you've ever used logic.

To educate you once again - we are not measuring the speed of the car. We are measuring the POWER of the ENGINE. It's a GPU BENCHMARK, not a TABLET benchmark. Learn the difference, before you continue to embarrass yourself.

If you wanted to know which engine was more powerful, you WOULD put it in a neutral chassis. That is EXACTLY the point of benchmarks, and it's why the same resolution is used by every established tech site where it is available.

Your idiocy is astounding.


"Nowadays, security guys break the Mac every single day. Every single day, they come out with a total exploit, your machine can be taken over totally. I dare anybody to do that once a month on the Windows machine." -- Bill Gates














botimage
Copyright 2014 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki