Develop Bugs

Benchmarks

Just done small comparison between 2.1 and 2.2 using linpack. On 2.1 it comes out with an average os 2.2 Mflops while on 2.2 it comes out with about 3.7 Mflops. What's everyone else getting?

I got the same scores

i heard the linpack doesnt give accurate results...i may be wrong.
have you all been able to use the Neocore(Qualcomm benchmark app) or Open GL benchmark with this new ext2 builds? all of those wrked perfectly on the older system sqsh. now i cant get them to run on this new builds with ext2. also alot of live wallpapers from b4 not wrking. anyone with any insight into this? ill do a linpack on mine device, which is basically overclocked to the max, to post what scores i get.

Fresh on Frozen Eclair 537mhz 10 mflops - 1125mhz 21 mflops

Just a general question. Fresh, dc, stock, or baked with even overclock highest mflops i got were 9 mflops. So I added Frozen Eclair JIT to Fresh and was able to get 21 mflops @ 1125mhz. I have balanced my phone now for battery life and performance. I am at 537mhz which is giving me 10mflops and 55 fps on netarchy kernel 3.7.1. My question, Is my phone faster @ 537mhz with jit than stock @ 998mhz? I believe that the higher the mflop the faster the phone right?

mflops, is millions of floating point operations. therefore more is better.

This is a VERY interesting question. I kind of doubt that it will receive a good answer, though. I don't know what the answer is.
I would assume that processor speed is king. But if a lower processor speed gets you higher Mflops than, say, my phone than I guess that Mflops are king. I don't know why that would be that way, though.

SWEET I'll take 10mflops @ 537mhz because everything runs flawlessly internet, streaming video, and phone. When I need to game I can turn up the processor to 998 or 1125 and get 17 to 21 mflops. Thanks. My daily driver for now will be 537mhz. More battery life and more power than 2.1 android.

pongoface said:
SWEET I'll take 10mflops @ 537mhz because everything runs flawlessly internet, streaming video, and phone. When I need to game I can turn up the processor to 998 or 1125 and get 17 to 21 mflops. Thanks. My daily driver for now will be 537mhz. More battery life and more power than 2.1 android.
Click to expand...
Click to collapse
**EDIT**Nevermind............i need to get rid of Sense ui

I had a hard time to getting rid of my raining animations too lol
Sent from my PC36100 using XDA App

Quadrant: Worse than you thought

As we all know quadrant is no reliable measure for speed. At least I knew this for a while now and it was repeated and quoted many times.
This article tells anybody with a functioning brain (that is used of course) that quadrant means pretty much nothing.
I can't help to run it from time to time anyway
So I sat on the to... in my room in front of my computer with my phone. I9000 with supersonic ROM and the remount script from adrenaline shot 7. I sat there and said to myself "how hight can you score in quadrant LOL"
I started quadrant up and ran the benchmark: 2309
Then I opened the task manager-> Exit all & Clear memory
Then via long press homebutton back to quadrant to run the benchmark again score: 2453
But since I am a programmer and can imagine all kinds of optimizations and caching I pressed the back button and just ran it again just after it finished
Score: 2675
How the hell could anyone call that a benchmark?^^
just to be sure could anyone confirm that behavior? And does anyone know of a mor reliable alternative? I'd like to collect that knowledge in this thread.
TL;DR: quadrant sucks, you know anything better or want to flame away: do it here
Those are not the actual numbers from my first experiment, I repeated the scenario just now and took the numbers from those runs.
Additional runs scored 2775, 2907 and 2820, that's just silly

I think this behaviour is well known and has to do with JIT optimizations or something like that

allotrios said:
I think this behaviour is well known and has to do with JIT optimizations or something like that
Click to expand...
Click to collapse
The reason is irrelevant. The fact it doesn't provide a reliable benchmark is.

no benchmark is precise if you don't use it as intended. Quadrant produces a reliable comparative benchmark when used as designed: run it five times, remove the lowest and highest scores and average the remaining 3 -- that is your benchmark. You may not like it, but that is how it is designed to be used.
Now if you want to be pedantic, you could reasonably test again, by running quadrant 5 times, removing the outliers and average your 3 remaining scores. Repeat 10 times and then tell me how your average scores do or do not vary: they will in fact be within a narrow range, your actual benchmark.
Alternatively, tell us which benchmark produces the same score each run, as that appears to be the sum total of your objection to quadrant.
There are other benchmarks, such as Caffiene Mark, AnTuTu and NenaMark, but they are all apps just as Quadrant is and all require several runs and averaging to produce a comparable benchmark.
Moreover, the primary use of any benchmark is to compare firmware (kernel and rom) builds on the same phone to see relative performance gain and drop.

A benchmark is supposed to give way of comparing the capabilities of a given device. This means that a device with a high average score implies a better device than a lower score.
But the Quadrant score does nothing of this sort! In a competition with a friend I achieved an average Quadrant score of about 4300, with a peak of 4462. According to Quadrant my device is a lot better than the OP! Which is just not true.
Quadrant is unreliable as a benchmark, no matter how it is "designed to be used".
Sent from my GT-I9000 using XDA App

whaave said:
But the Quadrant score does nothing of this sort! In a competition with a friend I achieved an average Quadrant score of about 4300, with a peak of 4462. According to Quadrant my device is a lot better than the OP! Which is just not true.
Click to expand...
Click to collapse
You're doing it wrong.
lgsshedden said:
Moreover, the primary use of any benchmark is to compare firmware (kernel and rom) builds on the same phone to see relative performance gain and drop.
Click to expand...
Click to collapse

Quadrant scores are useless. I've used custom roms with scores of 2500+ but they aren't as smooth as stock roms, which only have scores of 1600-1800.

Antutu is indeed quite reliable imho. My results never fluctuate more than +-5% on the same config. That's an acceptable range, considering I don't set cpu governor to performance before running my tests.
Sent from my GT-I9000 using XDA App

upichie said:
You're doing it wrong.
Click to expand...
Click to collapse
w00t?
Quadrant does not reflect performance, and therefore can not be used as a comparison parameter.

It can't be much worse than I thought.
My phone with 2.1 and 'lag fix' scored 2200 and lagged so bad I wanted to throw it against a wall multiple times a day.
With stock 2.3 quadrant can be ~1000 but the phone runs much smoother.
Other than the obvious file systems I/O 'cheats' that resulted in the above, there is also the frame rate cap that makes the GPU tests useless as well.

if your trying to measure height with a scale , u wont get your answer .
The only benchmark tool that ever reflected how the phone felt in my hands , in real life usage is linpack .
changing OC / kernel is mainly the only thing that will affect linpack if your trying to use it to compare roms ill efer you to my first statement .
In order to have a good feel of a rom / set up on the phone , use some apps that will use lots of ressources , for example TW4 launcher , go in there scroll a lot open gallery (if you have many pics) scroll thru them and repeat ... Any benchmark tools will basically tell you the 'ability of your device ' ( comparing 2 different models like an inspire and an sgs2 for example will be accurate )

ZioGTS said:
Antutu is indeed quite reliable imho. My results never fluctuate more than +-5% on the same config. That's an acceptable range, considering I don't set cpu governor to performance before running my tests.
Click to expand...
Click to collapse
I recently tried Passmark Mobile. Still a beta version, but I like it. Test results reflect real performance improvement and degradation pretty closely, particularly for what concerns I/O and memory speed.

benchmark confusion

Hi!
i just got my dhd, and i am a bit confused about the bench scores.
firstly, i'm tunning rcmix3d runny 3.3, with defoult kernel. what confuses me is that i get a normal result in quadrant (around 1900), i only get around 9-ish in linpack, where i should get around 40 if i understand correctly. phone feels quite smooth...
could somebody please explane what i dont get here

t0mas_ said:
Hi!
i just got my dhd, and i am a bit confused about the bench scores.
firstly, i'm tunning rcmix3d runny 3.3, with defoult kernel. what confuses me is that i get a normal result in quadrant (around 1900), i only get around 9-ish in linpack, where i should get around 40 if i understand correctly. phone feels quite smooth...
could somebody please explane what i dont get here
Click to expand...
Click to collapse
Even I am getting the same results.
Changing cpu speed using setCPU has no effect on Linpack.
Maybe its something to do with Linpack i suppose.

Most benchmarks have absolutely nothing to do with phone performance. Especially quadrant and Linpack.
Linpack tests CPU Speed, which is pretty pointless since there are few tasks that actually require CPU (everything should still feel smooth @ 768 MHz).
And quadrant (Especially standard) gets quite alot of its points from IO on the Desire HD, which is actually pretty pointless. The 3D Performance is around the same on every ROM.
I would suggest getting a 3D benchmark like Nenamark/etc if you actually want to test performance, since 3D gaming is the only thing that would actually require it.

thanks for info, i figured that everything works quite fast so why bother with some benchmarks
but since i like to understend stuff, i feel better now

Benchmark Scores For Non-Stock ROM's

Ive used several ROM's in the past on my Desire so far CM7.1 has had the highest benchmark for me at 1600 to 1700 i thought that to be pretty high but can people post what their highest benchmark is on the Desire, i want to know if 2000 is possible.

Which benchmark tool do you use.
On Antutu I consistently get over 2400.

Why do you care for benchmark scores anyway? The only important thing should be if a rom feels smooth and fast enough for oneself. This personal feeling cannot be shown by a number (no matter how high or low it is).

I was meaning Quadrant not antutu.
Mattdroid I was just wondering because on cm7 I got 1700 but on runnymede aio I get 400 usually which is really slow

With miui Rom and amarulz d2x script I got a 1970 on quadrant but with normal a2sd script on the same Rom quadrant score was 1200 although made no diff to smoothness of Rom
Sent from my HTC Desire using Tapatalk

kiwiflasher said:
With miui Rom and amarulz d2x script I got a 1970 on quadrant but with normal a2sd script on the same Rom quadrant score was 1200 although made no diff to smoothness of Rom
Sent from my HTC Desire using Tapatalk
Click to expand...
Click to collapse
I can say quadrant score is not really reliable. It's all depends on the ROM setup. It is always at a high score for a data2sd, data2wherever, adx2sdx, int2sd and lower at a2sd and app2sd. It can be lower or higher with some tweaks in int.d.
If I ever need a benchmark, I would go for antutu, with whatever setup it always give more or less the same score for the same ROM.

In my opinion, Linpack will show if the rom is fast or slow. I tried Antutu and Quadrant but the scores do not match how I feel for the roms speed.