Quick question:
Occasionally and sometimes after a reboot, my stock ATT GS3 pulls a Quadrant score of 5300-5400. But, I can randomly run the benchmark and get about 3300. Now, I know the community doesn't put a lot into these scores, but I wanted to ensure there isn't an application sapping power.
Does anyone know which applications I should be wary of or what could be causing this? Is there a better benchmark app to gauge performance (besides real-world use) ?
Benchmarks are pointless. They can be faked all too simply. Never put faith in them
Wayne Tech S-III
Related
Has anyone looked in to this to see if it is something that we could possibly port over/modify to work on our phones and benefit from?
http://forum.xda-developers.com/showthread.php?t=749495
I did a search and didn't see any topics started on this so I figured it was worth asking.
It increased the quadrant scores tremendously for the Galaxy S.
Non applicable.
Galaxy class phones have serious lag problems due to installation of apps on slow sd.
Sent from my ADR6300 using XDA App
If someone wants to compile a new kernel with the "fix" implemented, sure. You would need a class 6 sdcard to see any benefit from it though. The EVO guys are doing it and seeing numbers over 2000. There's no reason it wouldn't work. Though, you probably wouldnt notice any practical difference in the phone in regular use. I'm assuming the quadrant increase is due to the read write speeds being much higher using a card that fast, thus increasing the I/O score on quadrant.
What people don't seem to realize is that quadrant score is an average of the different tests, and not just a blanket score to use for comparisons...on the galaxy s it does fix the lag problem they have. Watch one do a quadrant test. The I/O test takes over a minute. A nice aide effect of the fix is that it inflates quadrant scores. The incredible is far from laggy, so you'd basically be doing it just to see a higher quadrant score.
You'd also have to figure out a way to use the Amon-RA recovery instead of clockwork, as only it can partition the sdcard in the correct way to implement this...
I'd actually love to see people try more things with the kernels, the ones we have are nice, but pretty basic. The EVO guys also have one that actively adjusts the cpu voltage based on speed and temperature, it's supposed to really increase the battery life. I guess that's probably due to the EVO not having the same supply shortages, and the fact they gave out a ton of them free to developers at the google conference, lol.
Is it possible for an app to monitor and benchmark battery performance against a wide range on factors including usage and ultimately come up with a score based on the Kernal/Rom etc?
Does anything like that already exist?
If not it would be a great idea to come up with general scores to see which combination would provide the best battery life.
What do you guys think?
No app like that exists; however, skydeaner put a lot of work into getting a working list of kernel performance (which is the leading contributor to battery life). Check it out, it's very useful.
After racking my brain recently thinking that possibly my evo was somehow screwed up internally, I found the culprit.
Here's what happened....
I got bored and curious so I flashed a CM7 nightly to assure myself I have been missing nothing. Did the usual battery of testing after I update all my apps. Pathetically slow. I was left wondering how people could be bragging of 1600+ quadrant scores on aosp roms while I was barely getting over 1000. On VP I was accustomed to getting high 1400's. It seemed smooth enough but I wanted my VP back. I did a fresh install after a few wipes, same routine, updated apps and found I was getting 951 on the quadrant score, topping out at maybe 1100. That is 400 points below what I was getting before using the same kernel etc!
This has gone on for days now, been reading xda, even bugged a developer to see if he had insight. Thought I was onto something earlier when people reported problems with DarkTremors a2sd since I had upgraded to latest beta 4. So I wiped no less than 7 times, even once in the bootloader. Installed VaelPak2.3a and my scores were back! Thought I solved it but nope. Installed VaelPak3.1 found my scores back to normal. Did update 2, scores still normal. Upgraded apps. Scores dropped 300+ points.
Pulled the Quadrant from the VaelPak rom, and guess what? My scores are back to normal! So, am I the only one who has upgraded Quadrant and seen a difference in scores?
[quadrant version 1.1.5 is the good working version, 1.1.7 currently in the Market is causing the low scores]
Damn you're right, I was getting 1800+ before and now with 1.1.7 it's down to 1500+. Not quite the drop you were getting but it's still a significant difference.
Is it possible that newer Quadrant is supposed to give lower scores now on same hardware because it takes into account better hardware/newer phones? Idk, just posing the question
The most recent quadrant update apparently shows lower score results. Same thing happened with Linpack awhile back, they say it's more accurate now :/
same here, dropped by almost 200.
I can't say for sure whether they wanted the drop to "be more accurate" since the change log for 1.1.7 only claims gingerbread compatibility fixes. If you notice, the comparison chart levels have not changed along with the result ranges. A standard EVO is still something like 1200 or 1250 like it has always been. So, I do not see these numbers being more accurate if I am scoring significantly less than a stock unrooted phone.
I guess what I am saying is that if a benchmark decides to adjust their rating scale, the comparison scale should follow suit.
Don't you hate it when you spend time trying to fix something and here it not you at all? Like the the sound goes out on the TV. Then you spend the next 10 min. reseting receivers your TV and every device in your house. Just to find you its the cable company. Man I hate that.. let the bashing of cable companies begin...
Also:
I used to run those kinds of tests on my video cards on my pc all the time. Then they started making them for better cards and my scores kept going lower and lower. Eventually I got board with it because I knew I had a good card. It was just their POS software was making it look bad. Haven't ran that app since. Hope that doesn't happen to quadrant.
no wonder, and i cant get 1.1.7 to work on my g1. can anyone kindly share me 1.1.5's apk?
googled for it but couldnt find any
thanks!
First off I'm fairly aware of the fact that benchmarks are not accurate representations of the day to day real life usefulness of the handset.
That said, I used both linpack and quadrant standard edition for the first time tonight while testing another kernel with my current rom (which is cm7, ggingerbread-6).
At the conclusion of my testing it was very obvious that one kernel completely outclassed the other in a benchmarking situation, however something else became apparent that leads to this post.
If I follow and believe everyone else's benchmark scores, even those posted an hour earlier in the same kernel thread, then I might have the slowest Evo on planet earth.
I see other users of the same rom and kernal posting scores which are never below 1500 in quadrant, I saw one instance of 1300 but nonetheless, even overclocking to 1075 I can barely break 1100 and usually fall just below that. Sadly enough on the "slower" of the 2 kernals I was barely surpassing 900.
Now on the linpack side of things I don't have any comparative scores to judge against, but ill post what I received anyhow for information's sake. On the "faster" of the two kernels (the one that came prebuilt into the rom) I was getting between 33-34, on the new kernel I was testing I was getting between 19 and 22, these are all "mflops" of course, whatever that may be.
Someone give me some information or advice here! Do I just happen to have a slow evolution, or are others either exaggerating or using some trick/mod/tweak I'm royalty unaware of??
Thanks in advance!
some people brag, some people cheat, most have low scores, few have high, there isn't a very good baseline and the benchmark programs dont scale very well at all, I have run 1800 scores and I have run 600 scores, guess what. both roms were smooth and you wouldn't have been able to tell a difference, what does that mean? do we believe the benchmark programs? are they spitting a random number at us? who knows! dont believe them, be satisfied with how your evo is running and if it's not running very well then try a different kernel or rom, keep trying new ones until your satisfied, only then will some benchmark program output not mean a thing
Most of my Quadrant benchmarks with aftermarket ROMS+kernels have been in the 1100-1400 range, using VaelPak and various kernels to get most of the better scores there. The highest I've had was CM7RC1 with the SnapTurbo kernel, got an 1821. It was unusable, though.
I've come to the conclusion that the benchmarks aren't as important as battery life, especially with the Evo.
Biggest reason for the huge difference in numbers? Different versions of the app. The dev changed how it rates phones.
Sent from my PC36100 using XDA App
Thanks!
Explained. I knew they were totally not concurrent with the outward performance and usability of the device, and for what its worth while I've only ever flashed a total of 3 roms, this one is perfect for me and I seem to be one of the rare few with no problems whatsoever, everything works exactly as I would expect it to. So yes, l never feared my device was suddenly slower now that I knew the all knowing superultrabenchmark number.
As we all know quadrant is no reliable measure for speed. At least I knew this for a while now and it was repeated and quoted many times.
This article tells anybody with a functioning brain (that is used of course) that quadrant means pretty much nothing.
I can't help to run it from time to time anyway
So I sat on the to... in my room in front of my computer with my phone. I9000 with supersonic ROM and the remount script from adrenaline shot 7. I sat there and said to myself "how hight can you score in quadrant LOL"
I started quadrant up and ran the benchmark: 2309
Then I opened the task manager-> Exit all & Clear memory
Then via long press homebutton back to quadrant to run the benchmark again score: 2453
But since I am a programmer and can imagine all kinds of optimizations and caching I pressed the back button and just ran it again just after it finished
Score: 2675
How the hell could anyone call that a benchmark?^^
just to be sure could anyone confirm that behavior? And does anyone know of a mor reliable alternative? I'd like to collect that knowledge in this thread.
TL;DR: quadrant sucks, you know anything better or want to flame away: do it here
Those are not the actual numbers from my first experiment, I repeated the scenario just now and took the numbers from those runs.
Additional runs scored 2775, 2907 and 2820, that's just silly
I think this behaviour is well known and has to do with JIT optimizations or something like that
allotrios said:
I think this behaviour is well known and has to do with JIT optimizations or something like that
Click to expand...
Click to collapse
The reason is irrelevant. The fact it doesn't provide a reliable benchmark is.
no benchmark is precise if you don't use it as intended. Quadrant produces a reliable comparative benchmark when used as designed: run it five times, remove the lowest and highest scores and average the remaining 3 -- that is your benchmark. You may not like it, but that is how it is designed to be used.
Now if you want to be pedantic, you could reasonably test again, by running quadrant 5 times, removing the outliers and average your 3 remaining scores. Repeat 10 times and then tell me how your average scores do or do not vary: they will in fact be within a narrow range, your actual benchmark.
Alternatively, tell us which benchmark produces the same score each run, as that appears to be the sum total of your objection to quadrant.
There are other benchmarks, such as Caffiene Mark, AnTuTu and NenaMark, but they are all apps just as Quadrant is and all require several runs and averaging to produce a comparable benchmark.
Moreover, the primary use of any benchmark is to compare firmware (kernel and rom) builds on the same phone to see relative performance gain and drop.
A benchmark is supposed to give way of comparing the capabilities of a given device. This means that a device with a high average score implies a better device than a lower score.
But the Quadrant score does nothing of this sort! In a competition with a friend I achieved an average Quadrant score of about 4300, with a peak of 4462. According to Quadrant my device is a lot better than the OP! Which is just not true.
Quadrant is unreliable as a benchmark, no matter how it is "designed to be used".
Sent from my GT-I9000 using XDA App
whaave said:
But the Quadrant score does nothing of this sort! In a competition with a friend I achieved an average Quadrant score of about 4300, with a peak of 4462. According to Quadrant my device is a lot better than the OP! Which is just not true.
Click to expand...
Click to collapse
You're doing it wrong.
lgsshedden said:
Moreover, the primary use of any benchmark is to compare firmware (kernel and rom) builds on the same phone to see relative performance gain and drop.
Click to expand...
Click to collapse
Quadrant scores are useless. I've used custom roms with scores of 2500+ but they aren't as smooth as stock roms, which only have scores of 1600-1800.
Antutu is indeed quite reliable imho. My results never fluctuate more than +-5% on the same config. That's an acceptable range, considering I don't set cpu governor to performance before running my tests.
Sent from my GT-I9000 using XDA App
upichie said:
You're doing it wrong.
Click to expand...
Click to collapse
w00t?
Quadrant does not reflect performance, and therefore can not be used as a comparison parameter.
It can't be much worse than I thought.
My phone with 2.1 and 'lag fix' scored 2200 and lagged so bad I wanted to throw it against a wall multiple times a day.
With stock 2.3 quadrant can be ~1000 but the phone runs much smoother.
Other than the obvious file systems I/O 'cheats' that resulted in the above, there is also the frame rate cap that makes the GPU tests useless as well.
if your trying to measure height with a scale , u wont get your answer .
The only benchmark tool that ever reflected how the phone felt in my hands , in real life usage is linpack .
changing OC / kernel is mainly the only thing that will affect linpack if your trying to use it to compare roms ill efer you to my first statement .
In order to have a good feel of a rom / set up on the phone , use some apps that will use lots of ressources , for example TW4 launcher , go in there scroll a lot open gallery (if you have many pics) scroll thru them and repeat ... Any benchmark tools will basically tell you the 'ability of your device ' ( comparing 2 different models like an inspire and an sgs2 for example will be accurate )
ZioGTS said:
Antutu is indeed quite reliable imho. My results never fluctuate more than +-5% on the same config. That's an acceptable range, considering I don't set cpu governor to performance before running my tests.
Click to expand...
Click to collapse
I recently tried Passmark Mobile. Still a beta version, but I like it. Test results reflect real performance improvement and degradation pretty closely, particularly for what concerns I/O and memory speed.