Develop Bugs

Quadrant: Worse than you thought

As we all know quadrant is no reliable measure for speed. At least I knew this for a while now and it was repeated and quoted many times.
This article tells anybody with a functioning brain (that is used of course) that quadrant means pretty much nothing.
I can't help to run it from time to time anyway
So I sat on the to... in my room in front of my computer with my phone. I9000 with supersonic ROM and the remount script from adrenaline shot 7. I sat there and said to myself "how hight can you score in quadrant LOL"
I started quadrant up and ran the benchmark: 2309
Then I opened the task manager-> Exit all & Clear memory
Then via long press homebutton back to quadrant to run the benchmark again score: 2453
But since I am a programmer and can imagine all kinds of optimizations and caching I pressed the back button and just ran it again just after it finished
Score: 2675
How the hell could anyone call that a benchmark?^^
just to be sure could anyone confirm that behavior? And does anyone know of a mor reliable alternative? I'd like to collect that knowledge in this thread.
TL;DR: quadrant sucks, you know anything better or want to flame away: do it here
Those are not the actual numbers from my first experiment, I repeated the scenario just now and took the numbers from those runs.
Additional runs scored 2775, 2907 and 2820, that's just silly

I think this behaviour is well known and has to do with JIT optimizations or something like that

allotrios said:
I think this behaviour is well known and has to do with JIT optimizations or something like that
Click to expand...
Click to collapse
The reason is irrelevant. The fact it doesn't provide a reliable benchmark is.

no benchmark is precise if you don't use it as intended. Quadrant produces a reliable comparative benchmark when used as designed: run it five times, remove the lowest and highest scores and average the remaining 3 -- that is your benchmark. You may not like it, but that is how it is designed to be used.
Now if you want to be pedantic, you could reasonably test again, by running quadrant 5 times, removing the outliers and average your 3 remaining scores. Repeat 10 times and then tell me how your average scores do or do not vary: they will in fact be within a narrow range, your actual benchmark.
Alternatively, tell us which benchmark produces the same score each run, as that appears to be the sum total of your objection to quadrant.
There are other benchmarks, such as Caffiene Mark, AnTuTu and NenaMark, but they are all apps just as Quadrant is and all require several runs and averaging to produce a comparable benchmark.
Moreover, the primary use of any benchmark is to compare firmware (kernel and rom) builds on the same phone to see relative performance gain and drop.

A benchmark is supposed to give way of comparing the capabilities of a given device. This means that a device with a high average score implies a better device than a lower score.
But the Quadrant score does nothing of this sort! In a competition with a friend I achieved an average Quadrant score of about 4300, with a peak of 4462. According to Quadrant my device is a lot better than the OP! Which is just not true.
Quadrant is unreliable as a benchmark, no matter how it is "designed to be used".
Sent from my GT-I9000 using XDA App

whaave said:
But the Quadrant score does nothing of this sort! In a competition with a friend I achieved an average Quadrant score of about 4300, with a peak of 4462. According to Quadrant my device is a lot better than the OP! Which is just not true.
Click to expand...
Click to collapse
You're doing it wrong.
lgsshedden said:
Moreover, the primary use of any benchmark is to compare firmware (kernel and rom) builds on the same phone to see relative performance gain and drop.
Click to expand...
Click to collapse

Quadrant scores are useless. I've used custom roms with scores of 2500+ but they aren't as smooth as stock roms, which only have scores of 1600-1800.

Antutu is indeed quite reliable imho. My results never fluctuate more than +-5% on the same config. That's an acceptable range, considering I don't set cpu governor to performance before running my tests.
Sent from my GT-I9000 using XDA App

upichie said:
You're doing it wrong.
Click to expand...
Click to collapse
w00t?
Quadrant does not reflect performance, and therefore can not be used as a comparison parameter.

It can't be much worse than I thought.
My phone with 2.1 and 'lag fix' scored 2200 and lagged so bad I wanted to throw it against a wall multiple times a day.
With stock 2.3 quadrant can be ~1000 but the phone runs much smoother.
Other than the obvious file systems I/O 'cheats' that resulted in the above, there is also the frame rate cap that makes the GPU tests useless as well.

if your trying to measure height with a scale , u wont get your answer .
The only benchmark tool that ever reflected how the phone felt in my hands , in real life usage is linpack .
changing OC / kernel is mainly the only thing that will affect linpack if your trying to use it to compare roms ill efer you to my first statement .
In order to have a good feel of a rom / set up on the phone , use some apps that will use lots of ressources , for example TW4 launcher , go in there scroll a lot open gallery (if you have many pics) scroll thru them and repeat ... Any benchmark tools will basically tell you the 'ability of your device ' ( comparing 2 different models like an inspire and an sgs2 for example will be accurate )

ZioGTS said:
Antutu is indeed quite reliable imho. My results never fluctuate more than +-5% on the same config. That's an acceptable range, considering I don't set cpu governor to performance before running my tests.
Click to expand...
Click to collapse
I recently tried Passmark Mobile. Still a beta version, but I like it. Test results reflect real performance improvement and degradation pretty closely, particularly for what concerns I/O and memory speed.

[Q] Extremely low quadrant score. What do i do?

Okay, so the other day, i ran quadrant on my gs2 against my brohters gs4g and i got 3259 and my brother got 938. I tried again today, and i got 1208 and the gs4g got 1190. i ran the test seversl times and the same result, nothing more than 1300. I didn't change anything from the first time i ran the test, so is my phone messed up? I know most people don't really like quadrant, but i use it because it kind of gives me peace of mind knowing my phone is better thssn everyone else's

Don't worry about it. It really doesnt matter.
Sent From The Best Phone of 2011

Reboot your phone then wait for about 30 seconds while everything settles (dont run any other apps) then run quadrant. What score did you get this time?

I tried the reboot thing after the first low score and pretty much the same score
Sent from my SGH-T989 using Tapatalk

kkg720 said:
I tried the reboot thing after the first low score and pretty much the same score
Sent from my SGH-T989 using Tapatalk
Click to expand...
Click to collapse
I would just throw it out the window bro.......thats too low to be in hand.

KillaHurtz said:
I would just throw it out the window bro.......thats too low to be in hand.
Click to expand...
Click to collapse
Hilarious man
Sent from my SGH-T989 using Tapatalk

kkg720 said:
Okay, so the other day, i ran quadrant on my gs2 against my brohters gs4g and i got 3259 and my brother got 938. I tried again today, and i got 1208 and the gs4g got 1190. i ran the test seversl times and the same result, nothing more than 1300. I didn't change anything from the first time i ran the test, so is my phone messed up? I know most people don't really like quadrant, but i use it because it kind of gives me peace of mind knowing my phone is better thssn everyone else's
Click to expand...
Click to collapse
Quadrant doesnt go well with Dual Core Snapdragon Processors. I've been getting 2000 each time a ran it. Try ANTUTU Benchmark and see what you get.
Quandrant doesnt define a phone of how great your phone is.

Or CF bench works well!
SGSII T989~ juggernaut v4.1~synergy v0.10

Quadrant is a horribly outdated benchmark, don't rely on it ever.
-Sent from my Droid 2-

iExpliziT said:
Quadrant doesnt go well with Dual Core Snapdragon Processors. I've been getting 2000 each time a ran it. Try ANTUTU Benchmark and see what you get.
Quandrant doesnt define a phone of how great your phone is.
Click to expand...
Click to collapse
I'll take what AnTuTu Benchmark gave me.. Listed me at 6327... Just over the Nexus ;-)
Main thing that brought me down as my stupid SD card because I have some generic piece of junk --- EDIT: Just ran it against the internal SD, and I got 6577. Even above the Galaxy Note ;-)

I know that quadrant is not optimized for dual core, but that doesnt explain why i was getting 3000+ and it just dropped to less than 1000
Btw antutu was 6492 idk if thats good or bad so...
Sent from my SGH-T989 using Tapatalk

Freeze Google maps with titanium back up and see what u get ...
Sent from my EVO 3D Running Ganii's ICS

maek_it_happen said:
Freeze Google maps with titanium back up and see what u get ...
Sent from my EVO 3D Running Ganii's ICS
Click to expand...
Click to collapse
Maps is really that big of a culprit?
Sent from my SGH-T989 using xda premium

These threads get so old. Does everything work on your phone? Is it smooth and responsive? Then who cares what some outdated benchmark says. You are the benchmark.

I changed the rom on my friend's mytouch 4g and he scored 3246 on miui oc to 1.6 ghz lagfree, It beat my gs2 running zombie oc to 1.72 ghz lagfree 2976 i dont think quadrant is an accurate benchmark at all, so markus is right, YOU are the benchmark.

[Q] T-Mobile S3 - Low Benchmark Scores/Processor not going 100%?

I've had my Galaxy S3 from T-Mobile now for about a week. The phone seems to run perfectly fine. However when I attempt to run Quadrant for example my friends are averaging anywhere from 4900 - 5300 for their stock T-Mobile S3. My same phone, stock, everything closed/killed from task manager, only manages to score 3100 - 3600.
When I check system information in Quadrant sometimes it says my current clock speed is 912Mhz, 1134Mhz etc.. Rarely do I see it at the full 1512Mhz..
Power saving mode is definitely turned off. Even after doing a factory reset and running quadrant on a fresh install it still scores the same. Beginning to get frustrated this phone isn't living up to the hype.
Ideas anyone? Thanks.

I did a battery pull, killed all my apps, cleared my memory, waited a few seconds, ran Quadrant again and still can only manage to score 3100.

No offence but who cares about benchmarks...... the phones soooo smooth and awsome

Sent from my SGH-T999 using xda premium

Don't have mine yet but I have seen someone have the same problem on here and what was wrong with theirs was they didn't know they could set the CPU Max mhz in the settings. Perhaps yours is not set to Max? Not sure where exactly in the settings it is but I'm sure it isn't hard to find
Sent from my SPH-D700 using XDA

I've been through the settings menu in the phone numerous times. Have not seen an option like that nor do I think it exists? For the most part the phone runs smooth I do notice it lags/hangs up sometimes but I'm guessing that's because the processor isn't going to full potential.
Anyone else have any ideas or where I can find this mysterious setting?

I've noticed on my sprint phone that as the phone gets warm the upper limit that the CPU can scale to seems to be reduced. The day I got it the first quad run was about 5k, but after 30min of nonstop usage the next run was down to 3.5 or so. Check it yourself with setcpu, it will start capping at 1100-1200.

OgremustCrush said:
I've noticed on my sprint phone that as the phone gets warm the upper limit that the CPU can scale to seems to be reduced. The day I got it the first quad run was about 5k, but after 30min of nonstop usage the next run was down to 3.5 or so. Check it yourself with setcpu, it will start capping at 1100-1200.
Click to expand...
Click to collapse
I will check it out but considering even after a battery pull and letting the phone sit power off for an hour I still get crappy scores. Then theres others who can run Quadrant back to back to back to back to back and achieve 4500 - 5100 constantly. I can't even get it on one try! I'll download SetCPU and see.

If you are that concerned, wipe and try it all over again.
EDIT - Damn my spotty reading, you already tried that. Sorry. Not much else to do, unless you try to restore the OS through Kies. A fresh install from a fresh source?
Or sell it on craigslist :laugh:

The phone works fine. Everything is very smooth and responds fast. It's pretty much just the benchmark scores that are throwing me off. I'll see what happens with the second S3 I will receive tomorrow. I can't imagine the heat playing a factor because this S4 is supposed to be one of the coolest running processors out there. There's a huge video on it somewhere where they did a butter melting test. Anyways time will tell I suppose.

Why do you care so much about benchmarks? Also the phone won't ramp the processor up to 1500 if it has no reason to..
Sent from my GT-I9300 using Tapatalk 2

joshnichols189 said:
Why do you care so much about benchmarks? Also the phone won't ramp the processor up to 1500 if it has no reason to..
Sent from my GT-I9300 using Tapatalk 2
Click to expand...
Click to collapse
What's the point of having the newest hottest phone right away? To be able to say it can do what others cant. One of those things being blowing away other phones in the benchmark arena. I just don't see why others are able to hit 5000+ scores and my device can't seem to even come close. It just makes me think there's something wrong with my handset. Do you blame me for being skeptical after the price we pay for these things?

Chicago281 said:
What's the point of having the newest hottest phone right away? To be able to say it can do what others cant. One of those things being blowing away other phones in the benchmark arena. I just don't see why others are able to hit 5000+ scores and my device can't seem to even come close. It just makes me think there's something wrong with my handset. Do you blame me for being skeptical after the price we pay for these things?
Click to expand...
Click to collapse
This is why I buy awesome phones. I love bleeding edge hardware lol.
Sent from my Axiom MAXX!!

Chicago281 said:
What's the point of having the newest hottest phone right away? To be able to say it can do what others cant. One of those things being blowing away other phones in the benchmark arena. I just don't see why others are able to hit 5000+ scores and my device can't seem to even come close. It just makes me think there's something wrong with my handset. Do you blame me for being skeptical after the price we pay for these things?
Click to expand...
Click to collapse
Considering you say it works just fine and are basing these problems that you have off of Quadrant..
Also I guess I will try and help since you are from Chicago they're should definitely be a setting somewhere that allows you to change the CPU settings. I'm not positive where but I did read in a review this was the first phone out of the box able to adjust governors. I would look but I'm on CM9
Sent from my GT-I9300 using Tapatalk 2

I've searched AndroidForums, Here, Google, can't find anything on being able to change the cpu speed without rooting.

Someone have the setcpu xda download page need the link, thx.
Sent from my SAMSUNG-SGH-I747 using xda app-developers app

Chicago281 said:
I've searched AndroidForums, Here, Google, can't find anything on being able to change the cpu speed without rooting.
Click to expand...
Click to collapse
It might only be the international version.. I read it in a review from one of the tech sites.
Sent from my GT-I9300 using Tapatalk 2

Do you perhaps have the CPU Power Saving option on in the Power Saving menu?

Diviance said:
Do you perhaps have the CPU Power Saving option on in the Power Saving menu?
Click to expand...
Click to collapse
First post last paragraph.. No
Sent from my GT-I9300 using Tapatalk 2

Chicago281 said:
I've been through the settings menu in the phone numerous times. Have not seen an option like that nor do I think it exists? For the most part the phone runs smooth I do notice it lags/hangs up sometimes but I'm guessing that's because the processor isn't going to full potential.
Anyone else have any ideas or where I can find this mysterious setting?
Click to expand...
Click to collapse
There is a setting for this, it's actually incorporated in the power savings mode. If you read through it, there is an area that says the max cpu speed is actually lowered when it's enabled. But, if you're not using the power saver mode, then it's irrelevant to your situation. It could have to do with the bloat though, and also the international is almost double the scores as ours yet there is minimal difference in speed presented in side by side real usage testing. I wouldn't get too hung up on the benchmarks, it kind of ruins the enjoyment you get from just using the phone, and seeing how smooth and flawless most everything is. Also, once debated and tweaked, those scores will most definitely improve and will probably ease your mind a bit.

Low benchmark scores after flashing faux007u...

On the stock kernal with the ViperX rom.
I got 12500.
Now with faux kernal I get like 9000.
Why?
I have a MUCH lower CPU score in benchmarks...

Geez you already asked this, why another identical thread? Benchmark means nothing, case closed.
Sent from my HTC One X using Tapatalk 2

yeah what is everyones obsession with benchmarks its just a number..

Benchmarks just a number? Tell that to people who drive race cars.
0-60 in 3 seconds is still better performance than 0-60 in 3.5 seconds. If both cars have the same engine, same gearing and weigh the same, then something else must determine why one is slower than the other.
Same way with our phones. Benchmark tests, especially AnTuTu, do offer insights into the performance of our machines. So, instead of giving the snarky answer that 'benchmarks don't matter', just say you have no idea or don't post at all.

yes maybe numbers are more applicable to cars. but this is a mobile phone forum not a car forum.....
the benchmarks mean sod all. the actual experience of using the phone is what matters.
benchmarks can vary from the slightest differences in environment, so yeah it is just a number, and will change for almost anything..

lawrence750 said:
yes maybe numbers are more applicable to cars. but this is a mobile phone forum not a car forum.....
the benchmarks mean sod all. the actual experience of using the phone is what matters.
benchmarks can vary from the slightest differences in environment. they mean absolutely sod all.
Click to expand...
Click to collapse
Really? A Mobile Phone Forum? How did I miss that part??
Sure experience matters, but to a lot of people, so do numbers. And 3000 points would make a difference, both numerically and perceptually.

I think this thread should be closed and open a new one talking about 10b5.
Enviado desde mi HTC One X usando Tapatalk 2

Baldilocks said:
Sure experience matters, but to a lot of people, so do numbers. And 3000 points would make a difference, both numerically and perceptually.
Click to expand...
Click to collapse
well seeing as the benchmarks can change literally from one minute to the next, and the fact they aren't that consistent, that essentially proves that the numbers don't mean as much as everyone makes out.
And therefore instead of basing how good your phone is operating on benchmark scores, just judge it on the actual performance of flicking through screens, loading apps , playing games etc etc - if there isn't a problem there, which i bet in the OPs case, there isn't any general performance issues, just a lower score. if there isn't any actual performance issues, then what the hell is the problem?

Hold on everyone. Small fluctuations in benchmark scores are nothing but a consistent drop of 25% is more meaningful, especially if all that changed was the kernel.
Why does everyone on here seem to just repeat what they read last week only with attitude added and regardless of factors such as severity.
To the OP... might be an idea to go back to stock and run several benchmarks to get a good overall picture. Note all the different scores for different tests (CPU, GPU, memory etc). Flash this kernel again and run several benchmarks. See which tests are affected and report your findings to Faux.

lawrence750 said:
well seeing as the benchmarks can change literally from one minute to the next, and the fact they aren't that consistent, that essentially proves that the numbers don't mean as much as everyone makes out.
And therefore instead of basing how good your phone is operating on benchmark scores, just judge it on the actual performance of flicking through screens, loading apps , playing games etc etc - if there isn't a problem there, which i bet in the OPs case, there isn't any general performance issues, just a lower score. if there isn't any actual performance issues, then what the hell is the problem?
Click to expand...
Click to collapse
Numbers shouldn't be inconsistent. My scores fall within 0.5-1% of each other every time I run them.
Certain ROMs can have a major impact on performance. Just like going to 4.0.4 and Sense 4.1 made a huge positive performance difference. Guess what, you could actually see that difference with 'useless' benchmark tests too.

Op,
Did you cry? Did your heart feel broken? I've heard benchmark scores are everything, they define smartphones these days.

ben-fisher-bro said:
On the stock kernal with the ViperX rom.
I got 12500.
Now with faux kernal I get like 9000.
Why?
I have a MUCH lower CPU score in benchmarks...
Click to expand...
Click to collapse
I found a fix for you! Just read this post here! http://forum.xda-developers.com/announcement.php?a=81
Sent from my HTC One X using xda app-developers app

Processor Question

Hey I was wondering, does my processor take breaking in? I just returned my other s3 because of defects but with this completely new one the quadrant scores and antutu scores are very low.. only like 3300 on quadrant and something low on antutu also.
The average stock benchmark scores are around 4800 and 6800 in antutu. So will my phone start breaking in and speeding up?

Those benchmarks mean nothing
As long as the phone feels smooth ignore benchmarks
Sent from my Samsung Galaxy Emperor using psycommu

True but I'd like it to run like everyone else's...

Honestly it's just a matter of feeling like your on the top of the food chain =p and for the record, after afew hours of getting my processor warmed up and kicked in, my scores jumped way up

I don't need benchmark scores to know I'm better than every one else
Being stuck up is great
Sent from my Samsung Galaxy Emperor using psycommu

So how exactally are u better than everyone else?
Sent from my SPH-L710 using xda app-developers app

Please share with the class
Sent from my SPH-L710 using xda app-developers app

Squirt oil in all phone openings... That'll make it slick as whale poo....
Sent from my SPH-L710 using xda premium

ReapersDeath said:
Hey I was wondering, does my processor take breaking in? I just returned my other s3 because of defects but with this completely new one the quadrant scores and antutu scores are very low.. only like 3300 on quadrant and something low on antutu also.
The average stock benchmark scores are around 4800 and 6800 in antutu. So will my phone start breaking in and speeding up?
Click to expand...
Click to collapse
Not to be rude. So sorry if this comes off all flame-Ish but you do realize that a CPU is made out of soi transistors those don't hold state if anything the heat slowly changes the resistance in negative ways over 1000s of hours of use leads to heat increase and eventual failure over a very long time line well that's how it goes in x86 computing only if stuff gets hot though. If it stays below 100Ish you don't see the heat increase part.
No seriously though Linux uses a cache so does dalvik and beyond that Android also uses a cache so benchmarks being subjective aside yes it takes a bit.
Just remember benchmarks are there to test changes and compare hardware. On Android....really Linux in general its almost impossible to compare hardware sometimes. The difference are not something that translates and. All of Android operates through a vm pretty much. ..
So your stuck back at comparison .using them to create a applicable baseline for say ....those tweaks you make in setcpu or maybe that shiny new kernel you flash vs the last version.
Sent from my SPH-L710 using Tapatalk 2
---------- Post added at 01:12 AM ---------- Previous post was at 01:06 AM ----------
gtuansdiamm said:
Those benchmarks mean nothing
As long as the phone feels smooth ignore benchmarks
Sent from my Samsung Galaxy Emperor using psycommu
Click to expand...
Click to collapse
I'm totally not with him though. If you are altering settings in your phone like cache stuff. You want to quantify the effects of the changes any way you can not that linpack our antutu or that complete piece of garbage quadrant are the defacto meter for that
Sent from my SPH-L710 using Tapatalk 2

freecharlesmanson said:
Not to be rude. So sorry if this comes off all flame-Ish but you do realize that a CPU is made out of soi transistors those don't hold state if anything the heat slowly changes the resistance in negative ways over 1000s of hours of use leads to heat increase and eventual failure over a very long time line well that's how it goes in x86 computing only if stuff gets hot though. If it stays below 100Ish you don't see the heat increase part.
No seriously though Linux uses a cache so does dalvik and beyond that Android also uses a cache so benchmarks being subjective aside yes it takes a bit.
Just remember benchmarks are there to test changes and compare hardware. On Android....really Linux in general its almost impossible to compare hardware sometimes. The difference are not something that translates and. All of Android operates through a vm pretty much. ..
So your stuck back at comparison .using them to create a applicable baseline for say ....those tweaks you make in setcpu or maybe that shiny new kernel you flash vs the last version.
Sent from my SPH-L710 using Tapatalk 2
---------- Post added at 01:12 AM ---------- Previous post was at 01:06 AM ----------
I'm totally not with him though. If you are altering settings in your phone like cache stuff. You want to quantify the effects of the changes any way you can not that linpack our antutu or that complete piece of garbage quadrant are the defacto meter for that
Sent from my SPH-L710 using Tapatalk 2
Click to expand...
Click to collapse
I don't understand how I go from 3800 in quadrant to 5089 after a few hours of nothing but stock lol.. then 4200 antutu to 6867.. but it goes up and down a little. I like to think processors are getting more advanced and are becoming aware lol. Turning into brains. Give it 5 years and hardware for phones will be so good we can play wow on them and then later on s voice will have a real personality and be living XD she will shock you for even thinking about looking at an iPhone or HTC =p. I know I'm sleep typing or something and that this is all random XD

Could it be possible that your phone was busy building cache files ? If you get quadrant pro it give you a breakdown of each score if you notice your I/o score is low then the filesystem is busy doing god knows what .
Sent from my SPH-L710 using xda app-developers app

Buddy a15 arch power savings aside isn't that stellar AMD is the only one talking innovation in cpus. Trust me you will be looking through your cellphone soon not at looking at it