Best tools for Android automated testing - Testing

Hi all,
(Hoping this is the right forum - its description includes "Discuss functional and performance testing tools and services" but if I select the Question box when posting I'm told "This is not a Q&A forum. By posting questions in this forum you are breaking the forum rules.")
I'm looking for tools to automate functional testing of Android apps. Robotium looks promising but I'd be interested to hear what more experienced Android developers think and if they have other recommendations. Cloud-based services may be interesting too.
Thanks
Mirko

Robotium is a nice choice. I've written a guide about it which you can take a look at.
Also the standard tool provided by the SDK, called the MonkeyRunner Tool, is also a great option.

EatHeat said:
Robotium is a nice choice. I've written a guide about it which you can take a look at.
Also the standard tool provided by the SDK, called the MonkeyRunner Tool, is also a great option.
Click to expand...
Click to collapse
Good tutorial, thanks.
The ability to control more than one device with MonkeyRunner sounds very interesting. I wonder if you could even simulate two users interacting with each other.

mikron said:
Good tutorial, thanks.
The ability to control more than one device with MonkeyRunner sounds very interesting. I wonder if you could even simulate two users interacting with each other.
Click to expand...
Click to collapse
You can do it with Robotium itself. Just specify the actions which you want sequentially, depending on your needs.

EatHeat said:
You can do it with Robotium itself. Just specify the actions which you want sequentially, depending on your needs.
Click to expand...
Click to collapse
I recently uese monkeyrunner on automated test. I really think it's easier than Robotium. Espesially the additional tool hierarchy to view perspective provided by the SDK is very useful.

fishwy said:
I recently uese monkeyrunner on automated test. I really think it's easier than Robotium. Espesially the additional tool hierarchy to view perspective provided by the SDK is very useful.
Click to expand...
Click to collapse
Maybe I should try using MonkeyRunner once after all. Though I never felt the urge to use anything else after Robotium.

Unit tests
Robotium
Initial set-up
Creating Robotium tests
Running your Robotium test
MonkeyRunner
Putting it all together

Android TV using monkey runner
Hi All,
Im new to this forum and Monkeyrunner as well. I tried connecting my cell phone using monkey runner and it works fine, but when I try to connect my Android TV, it doesn't connect. When I type adb devices, it gives output as <IP ADRRESS>:<port>, but waitforconnection command is not able to connect to my TV. My TV and my PC is on same network, I use ADB commands like logcat, push and pull works fine, but this connection using monkey runner is not working out.
Any suggestion guys.
Regards
Devendra

Alternative Tool: Ranorex
I have another recommendation for you: Ranorex. It supports both iOS and Android testing. You needn’t write any code because Ranorex lets you record your tests on real Android devices. It’s even possible to manage your devices over Wi-Fi from a desktop PC without having a USB connection to the devices.

Robotium is no doubt the best tool for android automation. To use it effectively, you can use this app: Robotium
Playstore link: https://play.google.com/store/apps/details?id=com.appzdevelopers.robotiumf

mikron said:
Hi all,
(Hoping this is the right forum - its description includes "Discuss functional and performance testing tools and services" but if I select the Question box when posting I'm told "This is not a Q&A forum. By posting questions in this forum you are breaking the forum rules.")
I'm looking for tools to automate functional testing of Android apps. Robotium looks promising but I'd be interested to hear what more experienced Android developers think and if they have other recommendations. Cloud-based services may be interesting too.
Thanks
Mirko
Click to expand...
Click to collapse
Hey Mirko,
Have you ever opted to try Cocos Test Automation? Cocos is providing a test automation service which allow you to automate your testing across over 80% of the relevant market share of real mobile devices. All you need to do is submit your games IPA or APK and our QA specialists will perform and record test flows. A test script will then be automatically replicated on hundreds of devices. A test report will be created and sent within 48 hours. Please feel free to contact me if you want to know more about this Cocos Service.
All the best,
Cocos Services

Crash log tool
I am not sure about automated testing by my favourite tool is Acralyzer. Best part is its free to install on your server and generates comprehensive reports.
I guess automated testing is also possible but it is something I have never tried.

thank you guys it's the first I heard of Robotium

Related

Is there a project management app on Android?

I'm unfortunately going to have to post this in a few places because I don't know where it's going to get the most exposure, but I'll start here because I have a Desire.
I need an app to manage the various projects that I have going on. In some ways, you can think of it as a todo application which has subtasks, projects/groups, action lists, sorting via time till due/time taken/project/importance etc.
If you have the time, check out this site where there guy goes through a number of options and criteria for a good GTD (getting things done) application.
I've tested:
OpenLoops
Astrid
Goals Todo
For starters, Astrid cannot sort by tag/project and non have sub-tasks.
I'm really hoping there is a project management app out there and someone can help me.
http://www.appbrain.com/app/com.ssi.gtasksbeta
mouth said:
http://www.appbrain.com/app/com.ssi.gtasksbeta
Click to expand...
Click to collapse
i use it also...
Is this purely an offline GTasks? (I couldn't really see much in the way of features.) If it is, GTasks is quite a basic todo manager, let alone project manager.
Projects feature for Astrid is currently in development. Don't expect it all that soon though.
-------------------------------------
Sent from my HTC Desire
Someone suggested Got To Do + Toodledo. Seems to have much going for it. Testing it out now.
ActionComplete?
Cleargrey said:
I'm unfortunately going to have to post this in a few places because I don't know where it's going to get the most exposure, but I'll start here because I have a Desire.
I need an app to manage the various projects that I have going on. In some ways, you can think of it as a todo application which has subtasks, projects/groups, action lists, sorting via time till due/time taken/project/importance etc.
If you have the time, check out this site where there guy goes through a number of options and criteria for a good GTD (getting things done) application.
I've tested:
OpenLoops
Astrid
Goals Todo
For starters, Astrid cannot sort by tag/project and non have sub-tasks.
I'm really hoping there is a project management app out there and someone can help me.
Click to expand...
Click to collapse
Have you tested 'action complete'
Their website: http://actioncomplete.com/android
Review of it: http://www.androidtapp.com/actioncomplete/
It's free and was built around GTD system afaik. It allows for sorting and projects with 'actions' and 'waits' within them. Good luck
You might want to try Thymer.
It's online and I believe it's not free but they have a beta running for Android.
I'm also looking for something similar. My only real requirement is sub-tasks though.
Got To Do seems nice, but to get sub tasks you have to pay for a pro license for ToodleDo, after you pay for the app itself. I'm sure its not expensive, but still...
Action Complete doesn't seem to have sub tasks.
I've also checked out Astrid and OpenLoops.
I thought the whole idea of GTD was to break a task down into smaller sub-tasks so you can "get things done".
I think I was actually using ToodleDo on my iPhone, but I don't recall having to pay to get sub tasks. Perhaps that is new.
Any methodologies in mind? Any back end software like MS Project Server or open source PM apps / servers in mind? I.e. how far down the rabbit hole do you want to go?
cosmicharade said:
Any methodologies in mind? Any back end software like MS Project Server or open source PM apps / servers in mind? I.e. how far down the rabbit hole do you want to go?
Click to expand...
Click to collapse
How far? Currently I don't need to intergrate with any MS software (or similar) though if it was offered and useful I wouldn't say no.
What I would like to have as a high priority are:
1. Ability to manage different projects (obviously).
2. Ability to plan projects - form project trees, some may have tasks, some may have notes, some may have links etc.
3. An online/desktop backup/sync. Simply because I hate the thought of all my information being in one place.
4. A UI that is easy to assess what is going on and what is coming up.
There are obviously other things required but it's difficult to list everything which is why I attached that site that pretty much lists most of them.
@Powder79, I just looked at Thymer. It looks a bit too basic but thanks for the suggestion.
Currently I'm trying out Toodledoo + Got To Do. It seems to have most but it really does suck for giving your tasks/actions a quick look to see the state (active/waiting/reference/etc). It just has date due and overdue.
If you imagine a bastard love child of Astrid (Android), Goals ToDo (Android), Toodledoo (online), Projekt (Symbian) and Mindmanager (desktop) then you will have my perfect project manager application.
Oh, and a desktop widget for easy viewing.
I don't ask for much obviously.
Project manager
Me too is looking for a good project manager.
Yet haven't found one that handles Values/Visions.
Values/vision is like a direction, it can't be completed. There should be mesure of how well one is living the value/vision.
And projects/goals to hold the direction.
And tasks to complete project. Yet, have only seen project at best.
Need, with overview:
Value1>Projects>Tasks
Value2>Projects>Tasks
Value3>Projects>Tasks
and so..
Sorry to bump up an old thread but anyone interested in a simple project management / to do list tool might like to take a look at my app: Organisa Lite. Its on Android Market currently. It offers some features I think some people might find useful - such as updating the android calendar with chosen dates and organising notes in a hierarchical manner. The user interface is unique too: you can minimise and maximise projects and sub-notes as required and manage everything from the one screen.
I'd like to make Organisa Lite completely free and add a Pro version with much more advanced features but I need some initial confirmation and support that it is what people want in the way of paid downloads.
Mobile Project Manager (MPM)
Have youbtried MPM (Mobile Project Manager)? It has unlimited subtasks, predecessor tasks, and allows importing and exporting to/from project management desktop software. Search MPM in the market.
try MPM
Try MPM if you haven't already
Project Management App on Android
I have been a user of Project Management App, "Project Planning Pro" on iOS for long. This award winning app has helped me significantly in my Project management needs.
However, the app is now available on Android platform & I have given it a try. It works like a breeze. Its a totally worthy app to have.
Best of Luck:good::good:
Cleargrey said:
I'm unfortunately going to have to post this in a few places because I don't know where it's going to get the most exposure, but I'll start here because I have a Desire.
I need an app to manage the various projects that I have going on. In some ways, you can think of it as a todo application which has subtasks, projects/groups, action lists, sorting via time till due/time taken/project/importance etc.
If you have the time, check out this site where there guy goes through a number of options and criteria for a good GTD (getting things done) application.
I've tested:
OpenLoops
Astrid
Goals Todo
For starters, Astrid cannot sort by tag/project and non have sub-tasks.
I'm really hoping there is a project management app out there and someone can help me.
Click to expand...
Click to collapse
Did you gave a try to "Project Planning Pro"
It meets all your requirements. Give it a try..

Multitasking on wp7

Have a look at this
http://www.youtube.com/watch?v=jipqNtc_PPU&feature=player_embedded
Our own developer Jaxbot is up to his tricks again, and this time, it's paid off: Finder of a hack that brought instant resuming to Windows Phone 7, he has been developing a full fast app switching/multitasking solution for Windows Phone 7.
Shown in the video above, the preview of the application is triggered by pressing the half shutter button (pressing the camera button halfway, not to the point where the camera would launch), and allows the user to jump between desired applications. The applications run in the background to a degree, as shown with the timer, and no time is required to bring the applications back from their background state. It's all very technical, but rest assure that he will be sharing some more details in the future.
Keep in mind the application is a preview-the interface and how it functions may very well change before release, and no release date is available. But if you're interested in having it on your device, share some thoughts about how you think it should look, function, etc, in the comments below.
Without further ado, watch the video above for a demonstration of fast app switching on Windows Phone 7.
Click to expand...
Click to collapse
Source
Yeah, checked the article out yesterday. Hopefully he'll be releasing something to everyone soon. It's the one thing I'm dying without No third party alarms (as the one MS provides kind of blows) and all this resuming nonsense is becoming slightly annoying.
That looks pretty slick! Would love to see this one in the wild soon, now if I could only get Pandora I would be set.
kwill said:
Yeah, checked the article out yesterday. Hopefully he'll be releasing something to everyone soon. It's the one thing I'm dying without No third party alarms (as the one MS provides kind of blows) and all this resuming nonsense is becoming slightly annoying.
Click to expand...
Click to collapse
Its still work in progress, there are limitations... people work as hard as they can to overcome these limitations and make it work for all devices.
Currently I believe its not possible to automatically find the titles of the running apps, so they are manually added (hardcoded) however they can find the paths to the files... so the remaining thing is getting access to the files in these paths to extract the title and tile image.
However what you see really works, its real and should work with every app once this is accomplished.
Marvin_S said:
Its still work in progress, there are limitations... people work as hard as they can to overcome these limitations and make it work for all devices.
Currently I believe its not possible to automatically find the titles of the running apps, so they are manually added (hardcoded) however they can find the paths to the files... so the remaining thing is getting access to the files in these paths to extract the title and tile image.
However what you see really works, its real and should work with every app once this is accomplished.
Click to expand...
Click to collapse
Thanks a lot for the insight on the operation and the obstacles they have to overcome. I'll be monitoring for any progress or assistance I can provide. Although, I'm sure my non-expert help won't be of much help.
kwill said:
Thanks a lot for the insight on the operation and the obstacles they have to overcome. I'll be monitoring for any progress or assistance I can provide. Although, I'm sure my non-expert help won't be of much help.
Click to expand...
Click to collapse
you can monitor here: http://forum.xda-developers.com/showthread.php?t=1006331
and here: http://forum.xda-developers.com/showthread.php?t=1021135
btw I have nothing to do with the developments, I was just excited by the news and kept contact with Jaxbot who made this app/video with the code wrapper provided by fiinix
Looks great! I like the idea of a half shutter press. Very elegant solution.
Want that app!
Is this app on the video released?????? I can't find any link to download, although it was introduces about 2 years

The WindowBreak Project

"What am I doing, why am I here, what is this about, and where am I going with this."
Hello all. After dealing with rumors, rumors of rumors, and the like, I've finally managed to disclose something I hold very dear to me: the WindowBreak project.
As the name suggests, this is a jailbreak project for Windows Phone 7. I started something a while back that had little success, but through the months, I've managed to figure out something that should bring light into the Windows Phone jailbreaking scene.
Real quick, though:
What this is: A project, with information about some interesting exploits I found, and a call for the community to dive in.
What this isn't: A full fledged jailbreak. Please don't post replies such as "when will XXXX device be supported". It just wastes time, and I assure you, I want every device included.
It also is not a full unlock. Just interop.
The details
So here's the sitch. We all know how Heathcliff74's interop unlock works. XAP files are just ZIP files, and ZIP files can have entries that allow extracting in parent directories.
Interesting thing is, this can be done using the ZipView application, which normally stores data in \Application Data\Volatile\Zipview\<random id>
Thus, creating a directory in a ZIP file called ../../../../provxml will copy all those files into the \provxml\ folder upon extraction.
See what I did there?
Limitations
Of course, there are limitations.
1) We cannot extract into \Windows\. There's a policy that prevents it.
2) The bad one: We can only extract known MIME types, at least to my knowledge. This is because the files are only extracted when they are clicked on in ZipView. And clicking on a .dbz file, for example, will just say the file type is not supported. Bummer.
What we can do...
As mentioned above, this can be used for a fresh out of the box jailbreak for Samsung devices, using provxml. Here's a video of that:
Try it yourself: with a Samsung device, go to http://windowsphonehacker.com/windowbreak and press WindowBreak Me.
In theory, this would be all we need to jailbreak most Windows Phone devices. Unfortunately, Nokia and HTC devices block the registry entries in provisioning files. I'm not sure what the extent of this "whitelist" (or is it a blacklist?) is, and details/tests on this would be appreciated.
What needs to be done...
Nokia: I don't have a Nokia device, but I've been working a great deal on figuring out how to crack it's shell, and have a couple of ideas. If I'm able to get my hands on a Nokia device soon, I'll try some of these unorthodox exploits out, otherwise I'll need some daring volunteers.
HTC: I do have an HTC device, but I can't figure out how to extract the files for the Connection Setup program. If someone can give me details on what the password encryption is on it, etc, for the HTC interop unlock, that would be much appreciated.
Other devices: Not a lot of demand for these (and LG needs no jailbreak, since it has MFG), but if something comes up, feel free to share where the provisioning files exist and I'll see about "windowbreaking" them.
So this is my little project, and I hope the details I'm sharing will lead to further development. My personal device (Samsung Focus) is easily interop unlocked now, without costing me a cent. I'd really like this to be the case for everyone; I'm not saying the $9 unlock for Chevron Labs is bad, in fact, it's greatly supported homebrew. What I am saying, though, is that freedom is still possible, and regardless, any developments made here will further support interop unlocking on Chevron/apphub unlocked devices. With that in mind...
Merry Christmas.
Special thanks to: Heathcliff74 for much of the research and idea behind the exploit
All the supporting members of XDA, who bring appreciation for what we do. Thank you.
Cool! Ridiculous that I didn't think of this myself
I will send you the password of the dbz files when I get home. I don't have it here.
But the real problem for HTC and NOKIA are the whitelists. I've been working on this for the past time. And today I made more progress. I developed a new way of debugging native 3rd party dll's/drivers. U can isolate functions and call them from a test app for unit-testing. This makes testing a lot easier. This will help me find exploits much faster. I can even call the whitelist functions of HTC and NOKIA on my Samsung now Working on it right now.
Good find!!
Heathcliff74
Heathcliff74 said:
Cool! Ridiculous that I didn't think of this myself
I will send you the password of the dbz files when I get home. I don't have it here.
But the real problem for HTC and NOKIA are the whitelists. I've been working on this for the past time. And today I made more progress. I developed a new way of debugging native 3rd party dll's/drivers. U can isolate functions and call them from a test app for unit-testing. This makes testing a lot easier. This will help me find exploits much faster. I can even call the whitelist functions of HTC and NOKIA on my Samsung now Working on it right now.
Good find!!
Heathcliff74
Click to expand...
Click to collapse
Haha, I knew you would say that when you saw this. Most credit of this goes to your work, in fact, which gave me much of the idea.
As for the whitelists, do you know exactly how it's blocking? Is just registry blocked, or all non-APN related settings?
^-- This is why I nominated you guys for those free Nokia Lumias. Keep up the good work!
Jaxbot said:
Haha, I knew you would say that when you saw this. Most credit of this goes to your work, in fact, which gave me much of the idea.
As for the whitelists, do you know exactly how it's blocking? Is just registry blocked, or all non-APN related settings?
Click to expand...
Click to collapse
Both brands have very similar mechanisms. They both have a driver dedicated to provisioning. The whitelists are implemented in those drivers. HTC has whitelisted only specific registry keys for APN's and stuff. NOKIA does not have the registry on the whitelist at all.
Heathcliff74
Heathcliff74 said:
Both brands have very similar mechanisms. They both have a driver dedicated to provisioning. The whitelists are implemented in those drivers. HTC has whitelisted only specific registry keys for APN's and stuff. NOKIA does not have the registry on the whitelist at all.
Heathcliff74
Click to expand...
Click to collapse
Shame it's a whitelist instead of a blacklist :\
Do you know which CSPs are allowed? I've managed to move files around using provxml on my Samsung, but it seems to allow just about anything.
Yeah I want you two to get the free Nokia Lumia's too! You both do great work - thank you keep giving love to the Omnia 7 too please since it's my girlfriend who has the Lumia 800 (dammit!)
Indeed a very cool solution ! Thanks button pressed.
Is it limited to 1st gen samsung devices or does it work on 2nd gen devices too ?
contable said:
Indeed a very cool solution ! Thanks button pressed.
Is it limited to 1st gen samsung devices or does it work on 2nd gen devices too ?
Click to expand...
Click to collapse
Both first and 2nd gen will work, though some interop won't work on 2nd gen devices (e.g., last I heard, registry editors were all read only)
DBZ Password
030D681B-1DFC-4bd0-A72A-A9B3CCCDA653
---------- Post added at 10:30 AM ---------- Previous post was at 10:29 AM ----------
Oh and it was found here http://forum.xda-developers.com/showthread.php?p=18916888
When I go to http://windowsphonehacker.com/windowbreak/
I don't get any buttons, just these text :
WindowBreak
By WindowsPhoneHacker
WindowBreak is a project with the goal of enabling true jailbreaking on Windows Phones.
We believe in freedom, both in gratis and in libre.
For details on WindowBreak, see here [add XDA link here].
BTW Nice worrk, hope to see more nice hacks these Xmas!
DJSave said:
When I go to http://windowsphonehacker.com/windowbreak/
I don't get any buttons, just these text :
WindowBreak
By WindowsPhoneHacker
WindowBreak is a project with the goal of enabling true jailbreaking on Windows Phones.
We believe in freedom, both in gratis and in libre.
For details on WindowBreak, see here [add XDA link here].
BTW Nice worrk, hope to see more nice hacks these Xmas!
Click to expand...
Click to collapse
You visit it on your phone, silly
And thanks, forgot to add the XDA links to that page.
Damn, purchased yesterday chevron labs-.-
But great work! I appreciate that!
Jaxbot said:
You visit it on your phone, silly
And thanks, forgot to add the XDA links to that page.
Click to expand...
Click to collapse
Yep Had desctop version in settings
10X it worked like a charm
Wait and hope for a solution for HTC devices
yea I can't wait for this. I have a HTC HD7.
Well done man, exactly what worked also with calling provxml files from the iso storage in DiagProvXML. Just do "../the entire path to iso storage/provxml.ext" this worked also on htc. Although of course your mechanism is different as you have to copy it to the correct folders while we just needed to change the default paths of the drivers to a custom folder.
Well done and nice creative solution. It would be great if you manage to get this working for more devices.
Maybe somebody can somehow find the exact provisioning the Developer Registration Tool makes, so you can use that one as that should be white listed I assume.
The Registration tool actually uses a socket connection to send a partly modified cookie string to the phone. The phone then goes and checks that against microsoft's servers and goes OK.
hey ii was just wonderiing, as nobody rarely mentions the dell venue pro, if something was going to be actually done for it. I know you said dont post stuff liike this but its just that ive never heard something like this or anything done about the Dell Venue Pro.
Thanks
Is the point of this for it to go from out of the box to interop unlocked, or from chevron/dev unlock to interop?
In /Classes Root/ registry part you can add the file type so it's known...

[TOOL] AppThwack - Easily test apps on real devices we host

Hi, I'm Trent and I co-created AppThwack, a service for on-demand automated testing of Android apps on real devices. Basically, you upload an apk and a couple minutes later you get screenshots and logcat dumps from actual phones and tablets we host.
Beta Users Needed
We're currently looking for devs to join our private beta. You can message me or sign up on our website and I'll get a beta code to you shortly. We're trying to stress the system and make additions/adjustments so the service is as useful as possible for developers.
Current Features
Real phones and tablets: About 20 high- and low-end devices and we add a few more every week.
Fast: See results in real time. Full test runs on all devices takes a minute or two to complete.
Selectable default tests: Install, launch, UI Monkey, Cleanup
Configurable tests: For example, specify the number of UI Monkey events and seed the randomizer
JUnit including Robotium support via uploaded test packages
Screenshots in portrait and landscape on all devices
Logcat and filterable logcat viewer
High-level results sortable by device or test
Full stack-traces for any exception that occurs
Future
We're working on adding more test frameworks like monkeyrunner and more default tests, particularly performance tests that measure battery consumption, CPU usage, etc. We're also adding more data visualization and charting so it's easy to see what some of the gathered statistics mean.
We plan to launch soon, but the beta program will remain in effect even after that. The service will follow a freemium model. Again, the beta is free and we're going to keep it in place even after we eventually launch.
Inter-device automation
Our back-end supports device-to-device automation, so if you have an app or scenario that you'd like to test that involves multiple devices or interaction with other devices, even non-Android devices, let me know. We're looking for people to help us develop how this service will be exposed.
Edit: I can't post links, but if you search for "appthwack" you'll find it. Btw, I've apparently lurked since October, 2008. Yikes.
Just wondering will this support of testing apps that require root?
Also any chance we can see like a live pic of the device when it installs the app and opens it?
Sent from my VS910 4G using xda premium
motodroidfreak said:
Just wondering will this support of testing apps that require root?
Click to expand...
Click to collapse
Right now it does not and all of our phones are as close to stock as possible. I'll look into making it an option so we automatically root before your app installs and then un-root after the tests are complete. Root opens up some new possibilities, both good and bad, so I'll need to think about it.
motodroidfreak said:
Also any chance we can see like a live pic of the device when it installs the app and opens it?
Click to expand...
Click to collapse
Yes! The launch test takes a screenshot in both landscape and portrait. You can see all screenshots sorted by device by clicking "By Device" or "By Test" and clicking the "Screenshots" link in the blue box at the top.
Screenshots also show up in each launch test log so you can see the context as the shot was captured. Logcat dumps show up in the same place (Link from the blue box at the top will open a filterable and highlighted log viewer).
Alright thanks I'll try it out tonight
Sent from my VS910 4G using xda premium
Holy cow. Didn't realize such web service existed.
I just signed up and currently having a look around. Is it possible for me to join the beta? Thanks!
Is it possible for a "free" upgrade for my account? Heh just asking
EDIT : Created a new project. Then I'm stuck. The "Runs" tab is empty.
EDIT again : Oh.. uploading had error previously. Uploading again.
Very interesting project. Good luck to your team and I hope I can be a good beta tester
Realy interesting, for us, almost of our apps needs root access, so please think about adding root to your service
Test on my Sensation
Will test on my sensation
test
nullFactory said:
Hi, I'm Trent and I co-created AppThwack, a service for on-demand automated testing of Android apps on real devices. Basically, you upload an apk and a couple minutes later you get screenshots and logcat dumps from actual phones and tablets we host.
Beta Users Needed
We're currently looking for devs to join our private beta. You can message me or sign up on our website and I'll get a beta code to you shortly. We're trying to stress the system and make additions/adjustments so the service is as useful as possible for developers.
Current Features
Real phones and tablets: About 20 high- and low-end devices and we add a few more every week.
Fast: See results in real time. Full test runs on all devices takes a minute or two to complete.
Selectable default tests: Install, launch, UI Monkey, Cleanup
Configurable tests: For example, specify the number of UI Monkey events and seed the randomizer
JUnit including Robotium support via uploaded test packages
Screenshots in portrait and landscape on all devices
Logcat and filterable logcat viewer
High-level results sortable by device or test
Full stack-traces for any exception that occurs
Future
We're working on adding more test frameworks like monkeyrunner and more default tests, particularly performance tests that measure battery consumption, CPU usage, etc. We're also adding more data visualization and charting so it's easy to see what some of the gathered statistics mean.
We plan to launch soon, but the beta program will remain in effect even after that. The service will follow a freemium model. Again, the beta is free and we're going to keep it in place even after we eventually launch.
Inter-device automation
Our back-end supports device-to-device automation, so if you have an app or scenario that you'd like to test that involves multiple devices or interaction with other devices, even non-Android devices, let me know. We're looking for people to help us develop how this service will be exposed.
Edit: I can't post links, but if you search for "appthwack" you'll find it. Btw, I've apparently lurked since October, 2008. Yikes.
Click to expand...
Click to collapse
I would like to test the tool, can you share with me..
Tested
Tested the tool, seems too good..
Suggestion : In-case if you want to reach maximum number of developer. Allow developer to use has free.
IDEA : You can request developer to post about you're tool on there app page, website & play store... As you're giving the tool as free you will get enough number of people to view & use the tool.. if the developer agree then you will allow him to use the tool for free of cost..
As a developer am ready to use the tool & post about you in my app & other places too...
This is a great tool, just uploaded an apk of my app that's in my signature and it worked, with a couple of NullPointers from the Play Store's licence service. That shouldn't happen, and doesn't on any of my devices, so I suspect it's an issue on your end. Any idea why?
HTML:
java.lang.NullPointerException at com.google.android.vending.licensing.LicenseValidator.verify(LicenseValidator.java:99) at com.google.android.vending.licensing.LicenseChecker$ResultListener$2.run(LicenseChecker.java:228) at android.os.Handler.handleCallback(Handler.java:605) at android.os.Handler.dispatchMessage(Handler.java:92) at android.os.Looper.loop(Looper.java:137) at android.os.HandlerThread.run(HandlerThread.java:60)
HTML:
FATAL EXCEPTION: background thread java.lang.NullPointerException at com.google.android.vending.licensing.LicenseValidator.verify(LicenseValidator.java:99) at com.google.android.vending.licensing.LicenseChecker$ResultListener$2.run(LicenseChecker.java:228) at android.os.Handler.handleCallback(Handler.java:608) at android.os.Handler.dispatchMessage(Handler.java:92) at android.os.Looper.loop(Looper.java:156) at android.os.HandlerThread.run(HandlerThread.java:60)
Borland
We are using Silk Mobile for end to end applications testing. Do you ever used this tool?
What an interesting service! I'll look into this from work tomorrow.
Not having used this at all, the first things which do spring to mind are:
-streaming realtime logcat
-a (skype?) connection with live streaming video of the app running, so you can see layouts/animations etc.
Anyway, I'm going to check this out tomorrow!
Quinny899 said:
This is a great tool, just uploaded an apk of my app that's in my signature and it worked, with a couple of NullPointers from the Play Store's licence service. That shouldn't happen, and doesn't on any of my devices, so I suspect it's an issue on your end. Any idea why?
Click to expand...
Click to collapse
Thanks for checking out the service. My immediate guess it that this is caused by the absence of a default Play account. Many devices have no account as one of our supported frameworks, calabash, removes accounts upon cleanup after script completion. On the plus-side, if you were to write scripts you should be able to add a temporary account from the test itself.
Highly unlikely you'd hit this bug in a real world situation, but it is a bug nonetheless.
Really awesome service! Can't test it for the moment as it doesn't support root apps, but this is really a great concept :good:
Maybe you could build a superuser permissions manager which would grant root access but makes sure to keep /system mounted as read-only, this way no harm could be done to the devices and us root apps devs could use your awesome service.
I actually got quite a few ideas, you could delete the mount binary in /system/xbin and use it in an internal appthwack app's private data, so that it's the only app able to call this binary and thus to mount /system.
I'd definitely subscribe to AppThwack if it had root support.
If you want help with developing this kind of secure root environment for the testing, I'd gladly contribute.
EDIT : Strangely enough, I just tested it with my app (which asks for root in the launcher activity, so I really didn't expect it to work) and had 0 failures, 75 pass.
How comes ? Have you already added root support ?^^
Either way this is really cool, I'm going to spread the words and most likely subscribe a paid account :good:
Is there somewhere we can see pictures of your device lab? Gotta be one hell of a device museum you got over there^^
Androguide.fr said:
Really awesome service! Can't test it for the moment as it doesn't support root apps, but this is really a great concept :good:
Click to expand...
Click to collapse
Awesome, thanks for the kind words!
EDIT : Strangely enough, I just tested it with my app (which asks for root in the launcher activity, so I really didn't expect it to work) and had 0 failures, 75 pass.
How comes ? Have you already added root support ?^^
Click to expand...
Click to collapse
This is pretty interesting. The only two rooted devices are a couple running CM. I'll look into this further, and if you have any ideas I'd love to hear them as well.
Either way this is really cool, I'm going to spread the words and most likely subscribe a paid account :good:
Is there somewhere we can see pictures of your device lab? Gotta be one hell of a device museum you got over there^^
Click to expand...
Click to collapse
Sweet, I really appreciate it. As soon as I have the number of posts to do so I'll post a pic of the lab.
Some things can not be automated, like scanning a QR code or reading/writing to an NFC tag. Do you plan on adding "manual tests" for a fee?
This would be really great to test apps on specific hardware.
worldtiki said:
Some things can not be automated, like scanning a QR code or reading/writing to an NFC tag. Do you plan on adding "manual tests" for a fee?
This would be really great to test apps on specific hardware.
Click to expand...
Click to collapse
Thanks for the question! Our primary focus is on automation. There are existing test houses and services that will execute manual tests like those you describe, but of course because of the manual component they're slow and expensive.
We often push folks to break their testing down into more granular chunks. For instance, verify you can take a picture and deal with the image, even if it's not the QR code or whatever your app usually consumes. This will find problems with simply using the camera and resulting image location. Now, have a separate test that processes a photo of a QR code, but feed the image in as part of the test. This removes the camera component from the analysis part, meaning it's now possible to benchmark the image analysis algorithm on all devices.
With a combination of a service like ours where you test very, very quickly on tons of devices, you can now do some more UX/end-to-end tests on a handful of devices yourself. This hybrid approach is great for finding the vast majority of issues before release.
nice post
Realy interesting, for us
Awesome tool !
No Developer can test his/her app on many devices. But your tool ... A W E S O M E ! ... I Used it yesterday to test two of my new apps ... Found some error is my app on certain devices which i'd have never found without AppThwack .. :good: :good:

Calabash-Android

Hello, I'm new on XDA and I am working on test for an application. And I made scenarios using calabash-android, cucumber and ruby.
Before I explain my problem, I want to know if I am in the right section and if somebody use or know calabash-android ?
Thanks, Momo James
Momo James said:
Hello, I'm new on XDA and I am working on test for an application. And I made scenarios using calabash-android, cucumber and ruby.
Before I explain my problem, I want to know if I am in the right section and if somebody use or know calabash-android ?
Click to expand...
Click to collapse
I think that you don't found a thread about your test application, but you can develop here what you want to do or explain.
Good luck :good:
For info for other members
Calabash enables you to write and execute automated acceptance tests of mobile apps. Calabash is cross-platform, supporting Android and iOS native apps. It is open source and free, and has a company, LessPainful, backing and developing it.
Calabash consists of libraries that enable test-code to programmatically interact with native and hybrid apps. The interaction consists of a number of end-user actions.
Click to expand...
Click to collapse

Categories

Resources