backtop


Print 34 comment(s) - last by Monkey's Uncle.. on Aug 6 at 10:51 AM

Samsung uses program to underclock GPU when not performing benchmarks

The Galaxy S IV from Samsung Electronics Comp., Ltd. (KSC:005930) has generally been getting beat in reviews by HTC Corp.'s (TPE:2498) One flagship phone, although Samsung has a modest sales lead.

Beyond3D poster @AndreiF made an interesting accusation about the Samsung handset in late June, alleging that for some benchmarks the phone was purposely boosting to its maximum GPU clock speed --533 MHz -- while operating at a slower 480 MHz for all other benchmarks and games:
Was the accusation true?

It appears so.  Last week, AnandTech's Brian Klug and Anand Shimpi dug into the bizarre scheme by first monitoring using the debugger "/sys/module/pvrsrvkm/parameters/sgx_gpu_clk" -- a file that contains the clock speed.  They confirmed the behavior the previous poster had written on.

Samsung GS4 benchmark clocked Samsung Galaxy S IV, standard clock
Upclocked for a benchmark (left) versus normal clock for an unrecognized benchmark (right).
[Image Source: AnandTech]

The hack ran even deeper, AnandTech discovered, while monitoring the CPU clock speed with the System Monitor app.  While in the benchmarks of choice the CPU was chugging away at 1.2 GHz -- the speed of the four onboard Cortex-A15 cores.  But in other graphics intensive applications outside the select benchmarks it switched over to the four power-sipping Cortex-A7 cores clocked at 500 MHz.

Ultimately it was found that the Samsung phone was bumping the GPU and CPU clock in many of the top benchmarks -- GLBenchmark, AnTuTu, Linpack, Benchmark Pi, and Quadrant.  Moreover, AnandTech found the behavior was not exclusive to the "octacore" international version; the U.S.'s Qualcomm, Inc. (QCOMSnapdragon 600 equipped model was doing the same thing.

(The Unreal Engine and GFXBench benchmarks do not appear to be effected.)

The responsible app was a Samsung-exclusive program named TwDVFSApp.apk.  When viewed in a hex editor the app was found to have several "BENCH_BOOSTER..." strings, indicating clearly this manipulation was purposeful.
Samsung Galaxy S4 wide
AnandTech concluded:

You should be careful about comparing Exynos 5 Octa based Galaxy S 4s using any of the affected benchmarks to other devices and drawing conclusions based on that.... We’ve said for years now that the mobile revolution has/will mirror the PC industry, and thus it’s no surprise to see optimizations like this employed. Just because we’ve seen things like this happen in the past however doesn’t mean they should happen now.

What Samsung needs to do going forward is either open up these settings for all users/applications (e.g. offer a configurable setting that fixes the CPU governor in a high performance mode, and unlocks the 532MHz GPU frequency) or remove the optimization altogether. The risk of doing nothing is that we end up in an arms race between all of the SoC and device makers where non-insignificant amounts of time and engineering effort is spent on gaming the benchmarks rather than improving user experience. Optimizing for user experience is all that’s necessary, good benchmarks benefit indirectly - those that don’t will eventually become irrelevant.

It should be noted that PC graphics card makers such as Advanced Micro Devices, Inc. (AMD) and NVIDIA Corp. (NVDA) are believed to include special tweaks and adjustments in their GPU firmware to perform better in certain top games.  This is also somewhat misleading as a less high-profile title may not see the same strong performance as a hit.  However, what Samsung is doing is arguably even worse for the consumer as there's no gains to the consumer from these benchmark inflating tweaks.

Sources: AnandTech, Beyond3D



Comments     Threshold


This article is over a month old, voting and posting comments is disabled

Don't forget
By spaced_ on 8/5/2013 10:17:18 PM , Rating: 2
- Yes, it's cheating and misleading. It's a white-list of benchmark programs. NO OTHER PROGRAM IS ON THAT WHITELIST OTHER THAN BENCHMARKING PROGRAMS.
- Yes, other companies (ATi, Nvidia, etc) have artificially raised benchmark scores in the past to try to fudge reviews to appear to be faster than their competitors.
- Yes, other companies would and will stoop this low in future.
- Yes, this behaviour should be reported in the media. Not just because of iTards complaining. Remember, there are three types of people in this world, not just two. iTards, Samsung tards and normal people.

Try to remember:

- No, Samsung are not high & mighty.
- No, Samsung should not be held in high regard for this.
- No, this is not 'ok' behaviour. If profiling is going on and has reason to, it needs to be transparent.




RE: Don't forget
By flyingpants1 on 8/6/2013 2:17:30 AM , Rating: 2
Correct


RE: Don't forget
By bug77 on 8/6/2013 8:20:00 AM , Rating: 3
quote:
- Yes, it's cheating and misleading. It's a white-list of benchmark programs. NO OTHER PROGRAM IS ON THAT WHITELIST OTHER THAN BENCHMARKING PROGRAMS


Except that: "a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player , and certain benchmarking apps"
Straight from Anandtech.

It's true the the permission to up the power envelope is only available only to Samsung preloaded applications, but if it were freely available, you'd be complaining that a third party malicious application has just fried your phone.
Your claim is erroneous anyway.


RE: Don't forget
By spaced_ on 8/6/2013 9:38:42 AM , Rating: 2
Yes, it appears there's a scant few legitimate apps in their profile after anand's 2nd follow-up article shed some further light.

However, given the benchmark programs in the profile are 3D/gaming related, why is there no other apps except for S Browser, Gallery, Camera, Video Player?

In any case, re-read that second article to the end. The quote you are referring to is straight from Samsung .

It's extremely hard to believe Samsung included benchmarking programs in this profile for any other reason than to inflate numbers, thus defeating the purpose of the benchmarks in the first place.

But you're free to believe everything they say. To me actions speak louder.


RE: Don't forget
By bug77 on 8/6/2013 9:55:35 AM , Rating: 2
I was just pointing out that what the OP claimed in all caps is actually wrong. Who says Samsung won't include third party apps to that list if they can certify them first? There are many ways in which application profiling can be legit.

Yes, running benchmarks at a higher frequency and games at a lower one defeats the purpose, but it is also off topic.


Not cool.
By retrospooty on 8/5/2013 12:00:54 PM , Rating: 2
Samsung needs to stop fudging #'s and seriously reduce the bloat that slows their phones down. It's absolutely ridiculous the amount of apps running at all times on a stock Samsung ROM.




RE: Not cool.
By Monkey's Uncle on 8/6/2013 10:48:29 AM , Rating: 2
Agreed. I replaced the Samsung stock ROM on my phone with a lightly modified Android 4.3 Google Edition image. What a difference in memory/battery usage and performance!

Samsung puts way too much bloat into their standard ROMs.


Just enjoy the all drug olympics...
By SublimeSimplicity on 8/5/2013 12:06:20 PM , Rating: 2
I've worked in the 3D graphics and embedded devices almost my entire career. Everyone codes drivers to benchmarks. Whether they key off of the app (most do) or simply use the benchmark as the test app for their development, everyone is boosting their numbers.




By Monkey's Uncle on 8/6/2013 10:51:00 AM , Rating: 2
True. I remember 3Dfx doing it with the VooDoo graphics coprocessors (showing my gray hairs here).

Selective tuning to suit conditions is common practice and not just in computers.


"Cheating?"
By Samus on 8/5/13, Rating: -1
RE: "Cheating?"
By retrospooty on 8/5/2013 12:20:23 PM , Rating: 4
It's misleading because it doesn't run at that speed in games that require alot of GPU power. It's trickery no matter how you look at it.


RE: "Cheating?"
By bug77 on 8/5/2013 1:03:03 PM , Rating: 1
Samsung says (and I tend to believe them) apps that constantly require a lot of power don't get the speed bump, because that would ruin the battery life. This is how application profiles debuted on the PC, too. IIRC, it was ATI that hard coded quake3.exe in their driver.

The only downside I see here, is that benchmarks are not indicative of the general performance anymore.

Either way, I'm pretty sure everyone will follow and in a couple of years we'll see application profiles for mobiles everywhere.


RE: "Cheating?"
By retrospooty on 8/5/2013 1:11:31 PM , Rating: 5
"apps that constantly require a lot of power don't get the speed bump, because that would ruin the battery life. "

That is a BS way of saying the chip doesn't run well at this speed so we dont use it unless benchmarking.

Come on, call a terd a terd here.


RE: "Cheating?"
By ritualm on 8/5/2013 2:07:39 PM , Rating: 1
It's not just battery life though. Most smartphones are passively cooled. Sure, you can run everything at the rated 533MHz GPU clockspeed, however... would you want to? It's not like a laptop where at least you can place it ontop of an active cooling pad, it's meant to be cradled by your hands, hug your face, or sit in a clothes pocket.

If the entire phone gets too hot to handle, I'd be a lot more worried about bodily damage than I'd worry about battery life, let alone benchmarking at that point.


RE: "Cheating?"
By retrospooty on 8/5/2013 2:15:43 PM , Rating: 2
Yup... that is why I say "runs well". There is stability, heat and battery life to consider. The fact is, it just doesnt run well at 533mhz, so to bump it up in benchmarks to try and bend opinions is pure BS.


RE: "Cheating?"
By hughlle on 8/5/2013 2:24:07 PM , Rating: 3
It is akin to selling a computer stating it runs at 5ghz because you managed to get a screenshot of cpu-z. So long as you can open cpu-z, then you can declare it a 5ghz cpu speed, we'll just ignore what happens when you try and do anything at that speed.

The issue goes for a lot of devices though. They claim they have a 1.7ghz processor or whatnot, yet 20 minutes into a game, thermal throttling kicks in. If it cannot run in a stable manner 24/7 under suitable environmental conditions, then it should not be sold as running at that speed. and this is just talking about heat, let alone battery. Throttling should be to save power when it's not being used, not because it can't use the power it has.


RE: "Cheating?"
By bug77 on 8/5/2013 5:18:34 PM , Rating: 2
quote:
It is akin to selling a computer stating it runs at 5ghz because you managed to get a screenshot of cpu-z.


I believe AMD recently released their FX-9590 CPU as a "5GHz" part.


RE: "Cheating?"
By othercents on 8/5/2013 2:26:04 PM , Rating: 2
Another issue is that the HTC One doesn't run into the same heat and battery life issue that the S4 has. Granted the device does get hot when playing games, but not too much to handle.

Overall the end user has paid for the same performance from their device in a benchmark and in a game. If heat and battery life is an issue it should be a user choice to run at higher performance and not something the manufacturer randomly degrades.


RE: "Cheating?"
By Monkey's Uncle on 8/5/2013 2:57:47 PM , Rating: 2
I refer to this as benchmark burning. It is nothing new and a common practice among computer enthusiasts.

For instance I will overclock my computer to 5Ghz for burn through some benchies but for everyday use I drop the clock it down to a much cooler & more power efficient 4.3Ghz. I even have profiles set up to put my system into burner mode if I am going to generate some CPUz banners. I most certainly wouldn't leave my system overclocked to 5Ghz for everyday use.

Nor would I expect anyone using one of these phones to overclock the heck out of their phone's GPU for anything except benchmarks.

My thoughts are that if the competitors don't like this, they can do the same as Samsung and compete apples to apples.

Personally I couldn't give a rat's arse if my phone's GPU is faster or slower than its competitors. As long as it runs to spec and has decent battery life, I'm happy a s pig in poop!


RE: "Cheating?"
By retrospooty on 8/5/2013 3:35:58 PM , Rating: 2
" I will overclock my computer to 5Ghz for burn through some benchies but for everyday use I drop the clock it down to a much cooler & more power efficient 4.3Ghz."

Yes, and its perfectly fine to do that, but if you were to sell that system and say it was a 5ghz machine, you would then be full of it. Agreed though, it is fairly common.


RE: "Cheating?"
By Monkey's Uncle on 8/6/2013 10:45:12 AM , Rating: 2
In order to sell my system at 5Ghz I would have to run a suite of stability tests to prove it is stable at that speed (it does by the way -- it just eats huge amounts of power at that speed & gets hot ;D )

I agree that it would be highly uncool to advertise their GPU speeds as 533 MHz if they are only bumping it there only for specific benchmarks. But are they actually doing that? What speed do they advertise they are running their GPU at? I would say they are just simply using on-the-fly tuning to get the best possible performance out of their benchmarks.

Most technically-oriented folks understand that benchmarks will never reflect real-world performance and any results you see needs to be swallowed with a pound of salt. These same folks also know that device vendors, regardless of who they are, are going to do everything they can to make their products look attractive - and this includes "On The Fly" tuning of their hardware based on the benchmarks being run.

Any vendor that cares about benchmarks does this. Samsung got caught doing it. But they are in really good company - so has Intel, AMD, Nvidia and Bugatti (Veyron).

It is not honest, I agree, but it is something that really happens out there. Folks such as myself and many of the DT crowd assume this takes place and really don't put a lot of faith in benchmarks.


RE: "Cheating?"
By Reclaimer77 on 8/5/13, Rating: 0
RE: "Cheating?"
By aicom on 8/5/2013 4:56:54 PM , Rating: 4
The difference (which the article doesn't cover) is that the clock speed bump isn't the entire story. They are actually raising the thermal limits of the SoC and overclocking the bus when one of the "boosted" benchmarks runs. This kind of performance boost is only allowed for the apps that are on Samsung's cherrypicked list and this list consists EXCLUSIVELY of benchmarks (that's what puts them over the line for me).

This contradicts a "fair" boosting system like Intel's Turbo Boost, AMD's Turbo Core, and NVIDIA's GPU Boost. These boosting systems work exclusively on the actual load and power consumption rather than a special list of blessed apps. I suspect that Samsung is trying to behave in a similar way, but can't due to issues with their DVFS implementation.

Their coherent bus between the A7 and A15 clusters is broken, requiring a cache flush when switching. I suspect this is why they manually forced the app onto the A15 cluster. Similar to their GPU clock boosting, they're providing behavior to benchmarking apps that normal apps would not benefit from and preventing benchmarks from noticing the failures of their SoC.

In an ideal world, Samsung could just program their DVFS implementation such that it throttles gracefully when the thermal limit is reached (mimicking other "boosting" implementations), but I understand that the real world sometimes gets in the way. They would need to spend many days/months of validation to ensure that it works properly in all cases, and it probably just wasn't worth it. They figured that it was better to use a whitelist than a blacklist. I just wish they had whitelisted things other than just benchmarks.


RE: "Cheating?"
By Reclaimer77 on 8/5/13, Rating: 0
RE: "Cheating?"
By drycrust3 on 8/5/2013 6:27:09 PM , Rating: 2
quote:
The purpose of a benchmark is to show maximum theoretical performance. And that's just what Samsung showed.

No, a benchmark is meant to give a pseudo-realistic means of comparison between different smartphones (or whatever), so users can compare one with another to determine what is superior and what is inferior in everyday use.
For example, we have cars and other vehicles do particular tests for fuel economy as a benchmark. I can drive my bus so it is using absolutely no diesel, 0 litres per 100 km, but I can only do this while coasting down hill. Would it be true advertise this bus, saying "this bus uses 0 litres per 100 km"? No, it is an inaccurate statement because the bus has to climb up the hill before it can coast down it.
The same applies here, Samsung have set up their phones so that when known benchmark applications are run the phone runs in a manner that it doesn't normally run in, it uses its processors so they use more power, and one other poster suggested it uses its memory much more efficiently as well, than most applications on it have access to, thus it gives an unrealistic result.
The interesting thing about this is maybe Samsung could have patented this technology (that's assuming someone else hasn't already done so), thus limiting the ability of competitors to use it; however, since they've been caught using this technology, it's now "in the wild" and they may not be able to do that (or they'll get a letter from the patent owner's lawyers).


RE: "Cheating?"
By Reclaimer77 on 8/5/2013 7:26:37 PM , Rating: 2
quote:
The interesting thing about this is maybe Samsung could have patented this technology


All they did was optimize the OS for certain software. Samsung isn't Apple. They wouldn't claim they invented software optimizations and patent them lol.


RE: "Cheating?"
By flyingpants1 on 8/5/2013 8:33:49 PM , Rating: 2
I'm surprised none of you seem to understand this or why it's even a discussion. It was discussed at length in the latest Anandtech podcast by Anand and Brian. Yes, Samsung is cheating. Please get over it.


RE: "Cheating?"
By ritualm on 8/5/2013 9:23:22 PM , Rating: 2
The iSheep section of DT won't let it go until every other company goes out of business because it's not Apple of Cupertino, CA.

Imagine the Republican National Convention, but replace the usual politics circlejerk with all things Apple. It's hilarious.


RE: "Cheating?"
By MozeeToby on 8/5/2013 12:23:38 PM , Rating: 2
It depends if the API to request full speed is free and open or not. If Samsung apps are the only ones that can reach full clock speed then the posted clock speed is misleading in my opinion. Even if it requires the developer to go out of their way to access it just for this one phone (or family of phones) it's still pretty obnoxious, you shouldn't have to target specific hardware when writing your apps.


RE: "Cheating?"
By aicom on 8/5/2013 12:33:13 PM , Rating: 2
The API (broadcast intent) is not accessible unless you're Samsung-signed, so it's not accessible by 3rd party apps at all. Benchmarks are being explicitly recognized by APK name in the DVFS service, which even raises the thermal limit.


RE: "Cheating?"
By Nortel on 8/5/13, Rating: 0
Peaking
By Da W on 8/5/13, Rating: -1
RE: Peaking
By chris2618 on 8/5/2013 1:49:51 PM , Rating: 3
Its the beginning of the end for smartphones growth. The market is getting close to saturated and especially with all the 24 month contracts people are waiting longer and longer between new phones.

I remember at one point people would buy out contracts near the end to get the latest phones but from my experience that doesn't seem to be happening anymore.

The other things i noticed is the number of people with a 4s and GS3 compared to 5 and GS4. Alot more seem to have stuck with the phone from a year ago than upgrade.


RE: Peaking
By hughlle on 8/5/2013 2:08:07 PM , Rating: 2
It's also not helped by the fact that a lot of people are having their wallets squeezed these days. I was in town today looking at all the new phones in the phone shop, and while i thought that the black version of the One was sexy as hell, it was also £35 a month for 24 months, or £500. I can buy a car for that amount, i can insure my present car for 3 years for that amount. When i first bought a smartphone, my htc desire, i was happy to have a £20 a month 24 month contract, because it offered me things my sony couldn't dream of, such as a good screen, good web access, google maps, etc etc etc, it was revolutionary in comparisson to what i had, wheras now, i can either keep my desire on a £10 a month 30 day contract, or i could spend a LOT of money and have a new phone, that is bigger, faster, but doesn't actually do anything new for me. Other than speed and screen size, it doesn't actually bring anything to the table to make me need it over my old smartphone.

Feature phone --> smartphone = groundbreaking new feature set
smartphone --> fancy new smartphone = unnecessary luxury purchase

It is going to take more than eye scrolling and a better camera to convince me to spend £500 of my money.


"When an individual makes a copy of a song for himself, I suppose we can say he stole a song." -- Sony BMG attorney Jennifer Pariser














botimage
Copyright 2014 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki