Develop Bugs

[Q] Benchmark questions!

First off I'm fairly aware of the fact that benchmarks are not accurate representations of the day to day real life usefulness of the handset.
That said, I used both linpack and quadrant standard edition for the first time tonight while testing another kernel with my current rom (which is cm7, ggingerbread-6).
At the conclusion of my testing it was very obvious that one kernel completely outclassed the other in a benchmarking situation, however something else became apparent that leads to this post.
If I follow and believe everyone else's benchmark scores, even those posted an hour earlier in the same kernel thread, then I might have the slowest Evo on planet earth.
I see other users of the same rom and kernal posting scores which are never below 1500 in quadrant, I saw one instance of 1300 but nonetheless, even overclocking to 1075 I can barely break 1100 and usually fall just below that. Sadly enough on the "slower" of the 2 kernals I was barely surpassing 900.
Now on the linpack side of things I don't have any comparative scores to judge against, but ill post what I received anyhow for information's sake. On the "faster" of the two kernels (the one that came prebuilt into the rom) I was getting between 33-34, on the new kernel I was testing I was getting between 19 and 22, these are all "mflops" of course, whatever that may be.
Someone give me some information or advice here! Do I just happen to have a slow evolution, or are others either exaggerating or using some trick/mod/tweak I'm royalty unaware of??
Thanks in advance!

some people brag, some people cheat, most have low scores, few have high, there isn't a very good baseline and the benchmark programs dont scale very well at all, I have run 1800 scores and I have run 600 scores, guess what. both roms were smooth and you wouldn't have been able to tell a difference, what does that mean? do we believe the benchmark programs? are they spitting a random number at us? who knows! dont believe them, be satisfied with how your evo is running and if it's not running very well then try a different kernel or rom, keep trying new ones until your satisfied, only then will some benchmark program output not mean a thing

Most of my Quadrant benchmarks with aftermarket ROMS+kernels have been in the 1100-1400 range, using VaelPak and various kernels to get most of the better scores there. The highest I've had was CM7RC1 with the SnapTurbo kernel, got an 1821. It was unusable, though.
I've come to the conclusion that the benchmarks aren't as important as battery life, especially with the Evo.

Biggest reason for the huge difference in numbers? Different versions of the app. The dev changed how it rates phones.
Sent from my PC36100 using XDA App

Thanks!
Explained. I knew they were totally not concurrent with the outward performance and usability of the device, and for what its worth while I've only ever flashed a total of 3 roms, this one is perfect for me and I seem to be one of the rare few with no problems whatsoever, everything works exactly as I would expect it to. So yes, l never feared my device was suddenly slower now that I knew the all knowing superultrabenchmark number.

Quadrant: Worse than you thought

As we all know quadrant is no reliable measure for speed. At least I knew this for a while now and it was repeated and quoted many times.
This article tells anybody with a functioning brain (that is used of course) that quadrant means pretty much nothing.
I can't help to run it from time to time anyway
So I sat on the to... in my room in front of my computer with my phone. I9000 with supersonic ROM and the remount script from adrenaline shot 7. I sat there and said to myself "how hight can you score in quadrant LOL"
I started quadrant up and ran the benchmark: 2309
Then I opened the task manager-> Exit all & Clear memory
Then via long press homebutton back to quadrant to run the benchmark again score: 2453
But since I am a programmer and can imagine all kinds of optimizations and caching I pressed the back button and just ran it again just after it finished
Score: 2675
How the hell could anyone call that a benchmark?^^
just to be sure could anyone confirm that behavior? And does anyone know of a mor reliable alternative? I'd like to collect that knowledge in this thread.
TL;DR: quadrant sucks, you know anything better or want to flame away: do it here
Those are not the actual numbers from my first experiment, I repeated the scenario just now and took the numbers from those runs.
Additional runs scored 2775, 2907 and 2820, that's just silly

I think this behaviour is well known and has to do with JIT optimizations or something like that

allotrios said:
I think this behaviour is well known and has to do with JIT optimizations or something like that
Click to expand...
Click to collapse
The reason is irrelevant. The fact it doesn't provide a reliable benchmark is.

no benchmark is precise if you don't use it as intended. Quadrant produces a reliable comparative benchmark when used as designed: run it five times, remove the lowest and highest scores and average the remaining 3 -- that is your benchmark. You may not like it, but that is how it is designed to be used.
Now if you want to be pedantic, you could reasonably test again, by running quadrant 5 times, removing the outliers and average your 3 remaining scores. Repeat 10 times and then tell me how your average scores do or do not vary: they will in fact be within a narrow range, your actual benchmark.
Alternatively, tell us which benchmark produces the same score each run, as that appears to be the sum total of your objection to quadrant.
There are other benchmarks, such as Caffiene Mark, AnTuTu and NenaMark, but they are all apps just as Quadrant is and all require several runs and averaging to produce a comparable benchmark.
Moreover, the primary use of any benchmark is to compare firmware (kernel and rom) builds on the same phone to see relative performance gain and drop.

A benchmark is supposed to give way of comparing the capabilities of a given device. This means that a device with a high average score implies a better device than a lower score.
But the Quadrant score does nothing of this sort! In a competition with a friend I achieved an average Quadrant score of about 4300, with a peak of 4462. According to Quadrant my device is a lot better than the OP! Which is just not true.
Quadrant is unreliable as a benchmark, no matter how it is "designed to be used".
Sent from my GT-I9000 using XDA App

whaave said:
But the Quadrant score does nothing of this sort! In a competition with a friend I achieved an average Quadrant score of about 4300, with a peak of 4462. According to Quadrant my device is a lot better than the OP! Which is just not true.
Click to expand...
Click to collapse
You're doing it wrong.
lgsshedden said:
Moreover, the primary use of any benchmark is to compare firmware (kernel and rom) builds on the same phone to see relative performance gain and drop.
Click to expand...
Click to collapse

Quadrant scores are useless. I've used custom roms with scores of 2500+ but they aren't as smooth as stock roms, which only have scores of 1600-1800.

Antutu is indeed quite reliable imho. My results never fluctuate more than +-5% on the same config. That's an acceptable range, considering I don't set cpu governor to performance before running my tests.
Sent from my GT-I9000 using XDA App

upichie said:
You're doing it wrong.
Click to expand...
Click to collapse
w00t?
Quadrant does not reflect performance, and therefore can not be used as a comparison parameter.

It can't be much worse than I thought.
My phone with 2.1 and 'lag fix' scored 2200 and lagged so bad I wanted to throw it against a wall multiple times a day.
With stock 2.3 quadrant can be ~1000 but the phone runs much smoother.
Other than the obvious file systems I/O 'cheats' that resulted in the above, there is also the frame rate cap that makes the GPU tests useless as well.

if your trying to measure height with a scale , u wont get your answer .
The only benchmark tool that ever reflected how the phone felt in my hands , in real life usage is linpack .
changing OC / kernel is mainly the only thing that will affect linpack if your trying to use it to compare roms ill efer you to my first statement .
In order to have a good feel of a rom / set up on the phone , use some apps that will use lots of ressources , for example TW4 launcher , go in there scroll a lot open gallery (if you have many pics) scroll thru them and repeat ... Any benchmark tools will basically tell you the 'ability of your device ' ( comparing 2 different models like an inspire and an sgs2 for example will be accurate )

ZioGTS said:
Antutu is indeed quite reliable imho. My results never fluctuate more than +-5% on the same config. That's an acceptable range, considering I don't set cpu governor to performance before running my tests.
Click to expand...
Click to collapse
I recently tried Passmark Mobile. Still a beta version, but I like it. Test results reflect real performance improvement and degradation pretty closely, particularly for what concerns I/O and memory speed.

[KERNELS][ICS][I9000] The ICS Kernel Benchmarking Project -Update: Devil

Goal of this little project is to dispel myths and hearsay and trying to assess the elusive performance of custom kernels for our beloved SGS I9000.
So far this has proven quite challenging as there is no single good benchmark on Android (yet):
a lot of people have been misled by ridiculous Quadrant scores: ridiculous because, with some small tweaks which do not affect real performance in any way, shape or form, it is possible to boost the Quadrant score by factor 3x.
You're free to believe that your SGS I9000 which scores 3000+ on Quadrant is faster than a SGS II, but then please leave this thread and move on.
some kernels may seem smooth with some games, and get high scores on some synthetic benchmark, yet the UI appears "laggy" and stutters a lot in comparison to other kernels which score lower on the same benchmark
some popular benchmarks give results with unacceptably low reproducibility, i.e. if you run them multiple times without changing a thing on your system, you get scores varying by 50% of more, in a completely random fashion
most popular benchmarks do not measure or take into account multitasking and CPU contention with other applications, yet on a typical usage one has background tasks such as the media scanner or synchronization which kick in often and unpredictably
So this will be mostly a work in progress, i'm testing several benchmarks and several kernels in multiple combinations, trying to analyze which benchmarks offer certain criteria which make them useful, namely:
Reproducibility of results: running the same tests multiple times, should result in a very small variance of the final score
Performance separation: benchmarks which are too "synthetic" and show only a dependency on clock speed are not useful to discriminate "fast" kernels from "slow" kernels
Performance representation: we all know when a kernel "looks" or "feels" fast or smooth. If a benchmarks shows you that a "laggy" kernel scores higher than a fast and responsive one, it's likely that the benchmark is not well designed
I'll work more on this thread explaining my (current) choice of tests and what they're good for.
But for now i'll just post a link to the summary table, and give a brief recommendation concerning popular ICS kernels; recommendation which i'll explain in the coming days.
Base ROM:
Slim ICS 2.8
(because is fast, smooth and has the least background stuff of all ICS ROMs which i tested)
Test Conditions:
Whenever possible, i tried to overclock the kernels to 1.2GHz which most / all phones should have no trouble achieving.
In case of Semaphore i had to use the bus / live overclock but it wasn't fully stable at 1.2GHz on my phone so i ran most of the tests at 1.14GHz.
Tested Kernels:
Stock Teamhacksung V17
Devil 1.1.6b BFS
Devil 1.1.6b CFS
Icy Glitch V14 b
Semaphore ICS 0.9.5b
Recommendation:
Devil 1.1.6b CFS, Icy Glitch V14b (with SmartassV2 and FIOPS), and Midnight ICS (with a tweaked Conservative) are trading blows for the fastest kernel.
At the time of testing, Midnight is slightly worse in terms of overclocking though, apparently due to different voltages, also it doesn't allow overclocking beyond 1.2GHz.
But what's interesting is that it achieves great performance while using a tweaked conservative governor.
Devil 1.1.6b BFS is good but obviously inferior to its CFS brother.
Semaphore has the lowest cache and memory latency in the multithreaded test, it also has impressive sd card read speed and in general appears super responsive, but it's a bit worse in 3D gaming and especially it lacks "true" overclocking, "live overclocking" changes the bus clock and is way more unstable, in fact on my phone i couldn't run it stable at 1.2GHz.
All kernels are significantly faster than the stock teamhacksung's kernel, so you have no excuses not to upgrade to one of the popular custom kernels!
ICS 4.0.4
Started testing Android ICS 4.0.4 kernels on Slim ICS 3.2.
All tested kernels are "huge mem" versions with 380+MB of available RAM, without breaking video playback or 720p recording.
Summary:
the stock kernel from Teamhacksung is now a very respectable performer, unless you plan to overclock probably you don't need to install one of the other kernels
Semaphore, Midnight and Devil are all very fast and smooth
Results table:
https://docs.google.com/spreadsheet/ccc?key=0AuBUEB4dGFSSdHIyN2VIeWU4QnhLOFpJejFPWDh5S1E

Res 1
One request for the kernel developers:
could you please post me what are your preferred / recommended settings in terms of Governor and I/O Scheduler?
Only one configuration per kernel please, as running these tests is rather time consuming.

Test Settings
So for anybody who wants to follow the same methodology as I used to test kernels, please pay attention that in some tests i didn't use stock settings, to try to improve the reproducibility of the results.
Before all test, i put the phone in flight mode, and disable all synch services.
Antutu: DB I/O and SD Write and Read have poor reproducibility. So i run these tests separately 5 times, and take the best scores.
RealPi: the number of iterations is increased by factor 10x i.e.: 100000 digits
MPAC: lots of customization here. Also be careful as it's not very stable and some settings will make it crash.
All tests: 8 threads (or 8 producer / consumer pairs)
CPU: 10000000 iterations, use case: integer (i'm considering to add logical too)
Memory: stock apart for nr of threads. Repeat the test 5 times and get best numbers
Cache: 40 iterations

Res 3
With this should be enough.

Judging from those results, CFS Devil looks really promising.

Semaphore live oc stability issues happen only on Slim ICS indeed. On ICSSGS I have perfect stability at 1.2 ghz. And performance is just great, paired with very good battery life.
GT-i9000 / ICSSGS 4.2 / Semaphore 0.9.0

A quick question: did you lock the max freq to eliminate the "governor" variable?
Because each kernel could have governor's tweaks that the other don't.
Based on what you posted here, the differences between Glitch and Devil is practically none.
I tested both and didn't feel any tangible difference, in the end, it comes down to the unique features of each kernel.

Overclocking bus vs adding an extra step aren't even slightly comparable. Maybe do tests not overclocked?
Also there is a new glitch build with 100% working bln.
Sent from my GT-I9000 using xda premium

+1 for tests without overclock. Majority of us, users do not overclock. Maybe a seperate test for overclocking could be nice , but comparisons should be done with stock speeds imho.
Thanks for the time and effort. We needed this.

Overclocking bus Vs adding an extra step isn't an apple to apple comparison, I agree.
However my goal was to use each kernel in the best possible way, and if some kernels have the possibility to use higher multipliers / extra frequency steps, that is an advantage for the user, compared to the kernels who only offer live overclock.
Don't get me wrong, i love Semaphore and i've been using it for a long time.
And i have no doubt that some users can get it stable with live overclock to 1.2GHz.
But that is the ceiling, while with other kernels even my phone can reach stable overclocks of 1.5GHz, and that is something to consider.
I chose as the basis for my tests an overclock of 1.2GHz because it's something which practically everybody can use, without massive battery drain, overheat or shortening the life of the device.
I'll try to add measurements at stock speeds for those who don't like to overclock.

cba1986 said:
A quick question: did you lock the max freq to eliminate the "governor" variable?
Because each kernel could have governor's tweaks that the other don't.
Click to expand...
Click to collapse
I didn't want to take the governor variable out.
Because, as you said, each kernel could use (and often does) governor tweaks which make the kernel "special" or different from the others, and that has to be taken into account in evaluating them.
Because nobody will use the phone locked at the maximum frequency.
So for me the governor and its tweaks is part of the user experience of a certain kernel, and a distinctive factor.
At the end, all kernels are coming from almost the same sources, so it's the little things which make the difference.

phzi said:
Also there is a new glitch build with 100% working bln.
Sent from my GT-I9000 using xda premium
Click to expand...
Click to collapse
That's great!
This test i run is not the "be all end all", it was just a recommendation at the time of writing.

Pipperox said:
Overclocking bus Vs adding an extra step isn't an apple to apple comparison, I agree.
However my goal was to use each kernel in the best possible way, and if some kernels have the possibility to use higher multipliers / extra frequency steps, that is an advantage for the user, compared to the kernels who only offer live overclock.
Click to expand...
Click to collapse
Agreed but, then again, benchmarks should be done at original CPU clock IMHO.
Otherwise, results are distorted.

HiKsFiles said:
Agreed but, then again, benchmarks should be done at original CPU clock IMHO.
Otherwise, results are distorted.
Click to expand...
Click to collapse
agreed. Especially since stock team hacksung seems to be clocked at 1GHz
what's the point of the comparison? Really?
As expected, there is no noticeable difference between all 1.2 GHz kernels.
It's not as if there was a real difference between them anyway.

zorxd said:
agreed. Especially since stock team hacksung seems to be clocked at 1GHz
what's the point of the comparison? Really?
As expected, there is no noticeable difference between all 1.2 GHz kernels.
It's not as if there was a real difference between them anyway.
Click to expand...
Click to collapse
That's not quite true.
If you look closer, you'll see that Devil CFS has quite a distinct advantage over all others in 3D tests.
The point of the comparison between stock hacksung @1.0GHz and the others, who can overclock, is to show what kind of benefit you get from switching to kernels which are overclock friendly.
Especially considering that you can't assume that a 20% clock speed increase will bring a 20% performance speedup across the board.
At last, i'd say that you may have "expected" that the kernels tested at 1.2 GHz don't have such a difference in performance.
But expectations have to be verified.
I tried to answer the questions:
On Devil's kernel, is BFS really better than CFS?
The "popular belief" is that BFS is faster than CFS.
According to my tests, CFS results faster instead.
Another question may be, what kernel gives you the best gaming performance.
If you pay attention to the An3D Bench XL, you'll see that Semaphore 0.9.5b, even overclocked a 1.2GHz, is significantly slower than Devil.

If i recall correctly Semaphore Author claimed that some kernel developers overclock GPU, and he didn't. Idk anything about it, but i recall something about it.
Is it possible to overclock only GPU, without overclocking CPU??

zipgenius said:
so you should benchmark them without setting anything. Average users don't overclock and don't change governor or scheduler: they flash the new kernel and stop.
Click to expand...
Click to collapse
I completely agree on benchmarking every kernel at the same frequency (stock 1Ghz max) but I think there are two different options for further benchmarking:
1) Benchmark kernels configured as similar as possible regarding CPU governor, IO scheduler, readahead -> comparable results for all kernels.
2) Benchmark kernels with default settings (only makes sense if all compared kernels are optimized for similar purpose like performance, does not make sense if a kernel does *not* focus on max. performance and uses e.g. Conservative CPU governor as default setting.
@Pipperox: Would it be possible to check my Mindnight-ICS dev version with your benchmark suite? I'd be really interested in the results as you use the same setup for all kernels (1.2Ghz would not be a problem, does not use LiveOC but old school 1/1.128/1.2Ghz OC).

Interesting thread... I never used devil's CFS version, always BFS. Will try CFS out now.
@Mialwe Where can we get your ics kernel?

mialwe said:
I completely agree on benchmarking every kernel at the same frequency (stock 1Ghz max) but I think there are two different options for further benchmarking:
1) Benchmark kernels configured as similar as possible regarding CPU governor, IO scheduler, readahead -> comparable results for all kernels.
2) Benchmark kernels with default settings (only makes sense if all compared kernels are optimized for similar purpose like performance, does not make sense if a kernel does *not* focus on max. performance and uses e.g. Conservative CPU governor as default setting.
@Pipperox: Would it be possible to check my Mindnight-ICS dev version with your benchmark suite? I'd be really interested in the results as you use the same setup for all kernels (1.2Ghz would not be a problem, does not use LiveOC but old school 1/1.128/1.2Ghz OC).
Click to expand...
Click to collapse
Sorry guys, i understand your logic but i do not fully agree with it.
I'm not comparing overclocked kernels with heavy tweaking of voltages and special settings with which they only work.
I did the "poor man"'s overclock, setting to 1.2GHz using NSTools, a setting where 95% of phones should have no problem working.
I think that if some kernels offer you this possibility while others do not, it is fair to use this "advantage" that they have over the other kernels.
Because a lot of users will have the possibility to do the same as i do, without esoteric knowledge and with just a couple of clicks in the menus.
That being said, "due to popular demand" i will also try to retest those kernels at 1.0GHz as soon as i get a bit of time.
BUT in my recommendations, i will also consider the overclocking capabilities.
@mialwe: sure, i'll give a run to your kernel as well!

mialwe said:
@Pipperox: Would it be possible to check my Mindnight-ICS dev version with your benchmark suite? I'd be really interested in the results as you use the same setup for all kernels (1.2Ghz would not be a problem, does not use LiveOC but old school 1/1.128/1.2Ghz OC).
Click to expand...
Click to collapse
Dude, sorry but i don't seem to find your ICS kernel anywhere.. can you provide a link?

Low benchmark scores after flashing faux007u...

On the stock kernal with the ViperX rom.
I got 12500.
Now with faux kernal I get like 9000.
Why?
I have a MUCH lower CPU score in benchmarks...

Geez you already asked this, why another identical thread? Benchmark means nothing, case closed.
Sent from my HTC One X using Tapatalk 2

yeah what is everyones obsession with benchmarks its just a number..

Benchmarks just a number? Tell that to people who drive race cars.
0-60 in 3 seconds is still better performance than 0-60 in 3.5 seconds. If both cars have the same engine, same gearing and weigh the same, then something else must determine why one is slower than the other.
Same way with our phones. Benchmark tests, especially AnTuTu, do offer insights into the performance of our machines. So, instead of giving the snarky answer that 'benchmarks don't matter', just say you have no idea or don't post at all.

yes maybe numbers are more applicable to cars. but this is a mobile phone forum not a car forum.....
the benchmarks mean sod all. the actual experience of using the phone is what matters.
benchmarks can vary from the slightest differences in environment, so yeah it is just a number, and will change for almost anything..

lawrence750 said:
yes maybe numbers are more applicable to cars. but this is a mobile phone forum not a car forum.....
the benchmarks mean sod all. the actual experience of using the phone is what matters.
benchmarks can vary from the slightest differences in environment. they mean absolutely sod all.
Click to expand...
Click to collapse
Really? A Mobile Phone Forum? How did I miss that part??
Sure experience matters, but to a lot of people, so do numbers. And 3000 points would make a difference, both numerically and perceptually.

I think this thread should be closed and open a new one talking about 10b5.
Enviado desde mi HTC One X usando Tapatalk 2

Baldilocks said:
Sure experience matters, but to a lot of people, so do numbers. And 3000 points would make a difference, both numerically and perceptually.
Click to expand...
Click to collapse
well seeing as the benchmarks can change literally from one minute to the next, and the fact they aren't that consistent, that essentially proves that the numbers don't mean as much as everyone makes out.
And therefore instead of basing how good your phone is operating on benchmark scores, just judge it on the actual performance of flicking through screens, loading apps , playing games etc etc - if there isn't a problem there, which i bet in the OPs case, there isn't any general performance issues, just a lower score. if there isn't any actual performance issues, then what the hell is the problem?

Hold on everyone. Small fluctuations in benchmark scores are nothing but a consistent drop of 25% is more meaningful, especially if all that changed was the kernel.
Why does everyone on here seem to just repeat what they read last week only with attitude added and regardless of factors such as severity.
To the OP... might be an idea to go back to stock and run several benchmarks to get a good overall picture. Note all the different scores for different tests (CPU, GPU, memory etc). Flash this kernel again and run several benchmarks. See which tests are affected and report your findings to Faux.

lawrence750 said:
well seeing as the benchmarks can change literally from one minute to the next, and the fact they aren't that consistent, that essentially proves that the numbers don't mean as much as everyone makes out.
And therefore instead of basing how good your phone is operating on benchmark scores, just judge it on the actual performance of flicking through screens, loading apps , playing games etc etc - if there isn't a problem there, which i bet in the OPs case, there isn't any general performance issues, just a lower score. if there isn't any actual performance issues, then what the hell is the problem?
Click to expand...
Click to collapse
Numbers shouldn't be inconsistent. My scores fall within 0.5-1% of each other every time I run them.
Certain ROMs can have a major impact on performance. Just like going to 4.0.4 and Sense 4.1 made a huge positive performance difference. Guess what, you could actually see that difference with 'useless' benchmark tests too.

Op,
Did you cry? Did your heart feel broken? I've heard benchmark scores are everything, they define smartphones these days.

ben-fisher-bro said:
On the stock kernal with the ViperX rom.
I got 12500.
Now with faux kernal I get like 9000.
Why?
I have a MUCH lower CPU score in benchmarks...
Click to expand...
Click to collapse
I found a fix for you! Just read this post here! http://forum.xda-developers.com/announcement.php?a=81
Sent from my HTC One X using xda app-developers app

Google edition roms lower benchmarks?

when I install a google edition or cyanogenmod rom or any rom thats not touchwiz I am getting lower scores just wondering why?

Lower's better. And all the TW junk in the background is taken out, leaving room for more processes, so of course it scores better.
Sent from my GT-I9505G using XDA Premium 4 mobile app

Tak23 said:
when I install a google edition or cyanogenmod rom or any rom thats not touchwiz I am getting lower scores just wondering why?
Click to expand...
Click to collapse
But does it feel faster? Because that's what really matters...

lordcheeto03 said:
But does it feel faster? Because that's what really matters...
Click to expand...
Click to collapse
This plus eleventy bagillion.
Benchmarks are totally arbitrary and meaningless.
TouchWiz ROMs scores great on benchmark tests because TW is specifically designed to perform well on such tests by putting the whole device into overdrive when the test is run. It doesn't mean that your day to day usage is better.

Tak23 said:
when I install a google edition or cyanogenmod rom or any rom thats not touchwiz I am getting lower scores just wondering why?
Click to expand...
Click to collapse
That's a very subjective statement. "Lower is better" really depends on the apo being used. Some, eg. Quadrant, a higher score is better. Linpack is the same way.
As said before, benchmark apps don't tell you the real performance of a device. Though they can give you a baseline or at least a general idea of different aspects. I wouldn't jump on the "benchmarks are useless" bandwagon as they do serve a purpose.
Sent from my SGH-M919 using xda app-developers app

KCRic said:
That's a very subjective statement. "Lower is better" really depends on the apo being used. Some, eg. Quadrant, a higher score is better. Linpack is the same way.
As said before, benchmark apps don't tell you the real performance of a device. Though they can give you a baseline or at least a general idea of different aspects. I wouldn't jump on the "benchmarks are useless" bandwagon as they do serve a purpose.
Sent from my SGH-M919 using xda app-developers app
Click to expand...
Click to collapse
Well they are useless when ROMs are set up specifically to perform well on benchmark tests. That gives aboslutely no real world value.
The only time there is any validity to a banchmark comparison is when you are trying to compare 2 things that are set up almost exactly the same with only a single differing variable.
Like if you want to see if a phone is bad, you'd need to run a benchmark on the target phone and another on a different phone of the same model with the exact same settings. Or if you want to see in a mod if the cause of reduced performance, you would need to run a benchmark without the mod, then immediately after the mod, and repeat it several times to ensure you get repeatable results. Etc etc etc...
You can't just abandoned the scientific method and compare a benchmark of a TouchWiz rom with a benchmark of an AOSP rom with god only knows how many differences in the set up (kernel, clock speed, gpu settings, voltage, etc etc etc) and have it be a valid means of determining which is the better rom in any way. You can't even set it up properly so that all the settings match and the only difference is the ROM because there is so much incompatibility betweent the two that factors into it. Especially when the TouchWiz rom is set up with methods of inflating it's scores on benchmark tests and the AOSP ROM isn't.

I agree. Scientific method comes first in determining the performance of a ROM. Actually I would say real world and use experience is the best method. Given today's Hugh end phones, our for example, getting the most out of a benchmark or anything else serves little purpose. I can disable 2, even 3 core and still have a smooth experience with most daily a activities. Even underclocking the CPU & GPU still retains all the speed and usability.
I think most tech literate people use benchmarks just to see what the phone can do. Also, as a quick though fairly inaccurate method of determining if you're ROM and it's mods/scripts are efficient.
To answer the OP's question though - since AOSP is HW accelerated, has been for a while now, it should be "faster" and yield better benchmarks. Less running in the background. The most things are integrated as opposed to added on to the base ROM and code. That's a very generic and butchered explanation.
Sent from my SGH-M919 using xda app-developers app