Develop Bugs

HELP NEEDED: Trying to make my own custom ROM!

Hi,
So, I'm trying to make my own custom ROM. Nothing too fancy, just a couple of simple and easy modifications to fit my needs but there's somethings I don't know how they work nor I understand them and I need some help. The only thing I was able to achieve as of yet was to remove some apps (I only tried to remove "Learn More" and QuickOffice to be honest-- but it worked). I just removed the .apk and .odex file directly from the .zip file, signed it and flashed it as usual. It worked fine.
Now, a few questions...
1) I also tried to add a few apps, just like in MoDaCo Custom ROM (MCR), to the /data/app folder instead of /system/app. Actually, I just copied all of .apk files from MCR /data/app into my own (with same folder structure of course). But they didn't get installed... I know I'm missing something and I can almost bet it has something to do with the /META-INF/com/google/android/update-script file. I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file? It has to do with the fact that my /data/apps weren't installed, right?
2) I don't use the Gmail app, just the HTC Mail one. Is it safe to delete Gmail.apk and GmailProvider.apk? I believe Gmail.apk will be fine, but the provider one, I guess it's used by the data synchronization thing. Maybe it's ok if I don't sync my mail (that's what I want actually) but maybe there's something else that I'm missing and the system will not work properly, somehow, without the GmailProvider.apk or is it safe to remove it?
3) When we first install a new ROM, starting clean, there's an introductory guide allowing to setup some things initially. One of those is the language. There is a bunch of languages, specially like English (A), English (B), English © where the letters represent other languages. I don't need all that, I only need English (Portugal). Is there anyway I could remove the ones I don't want from the ROM or I would need to dig into the Android source for that?
3.1) How about adding my own language? Is there a way I could do that to my ROM?
NOTE: Funny thing, the Android system is localized on my country's language when I first start the device but as soon as I change it to English, I cannot change it back to Portuguese cause that option doesn't exist. How come the device is in Portuguese when I first start it up after flashing a new ROM? This also happens every time I swap the SIM card, but I can never select Portuguese language by myself. Why?
4) I noticed that after flashing my custom ROM, after booting the device for the first time, the mobile network was active. This is not good... For now I have 1 month free of mobile networking but that won't last forever. I don't want to waste money every time I'm trying a new ROM. Is there anyway to disable this thing from automatically starting up connected when I first boot?
For now that's it...

1. You need to edit "META-INF/com/google/android/update-script"

That's what I said... :/ But I also said:
I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file?
Click to expand...
Click to collapse
If anyone knows anything helpful of course...

Nazgulled said:
That's what I said... :/ But I also said:
If anyone knows anything helpful of course...
Click to expand...
Click to collapse
I am also interested in a guide or walk through for making a ROM for the Hero.

1) I also tried to add a few apps, just like in MoDaCo Custom ROM (MCR), to the /data/app folder instead of /system/app. Actually, I just copied all of .apk files from MCR /data/app into my own (with same folder structure of course). But they didn't get installed... I know I'm missing something and I can almost bet it has something to do with the /META-INF/com/google/android/update-script file. I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file? It has to do with the fact that my /data/apps weren't installed, right?
Click to expand...
Click to collapse
You're missing copy_dir PACKAGE:data DATA:, so you're really not writing /data/app even if it's on the package. If you read through the update-script, you'll see that it starts by formating BOOT: and SYSTEM: and later on writes the contents of /system to SYSTEM: (with copy_dir) and writes the raw boot.img to BOOT: (with write_raw_image).
Add the line to the very end of the permissions settings (but before format and write boot, most likely, before the show_progress line). You also need to set a specific permission after writing anything to /data/app from an update-script, otherwise your market won't be able to install anything (as you'll have changed the permissions for /data/app through the update-script).
In all, your update script will look like this (assuming a regular update-script, not the mcr 2.9 one):
Code:
...
symlink toolbox SYSTEM:bin/xxxxyyyy
...
set_perm 0 0 04755 SYSTEM:bin/su
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
show_progress 0.1 10
show_progress 0.2 0
write_raw_image PACKAGE:boot.img BOOT:
show_progress 0.2 10
2) I don't use the Gmail app, just the HTC Mail one. Is it safe to delete Gmail.apk and GmailProvider.apk? I believe Gmail.apk will be fine, but the provider one, I guess it's used by the data synchronization thing. Maybe it's ok if I don't sync my mail (that's what I want actually) but maybe there's something else that I'm missing and the system will not work properly, somehow, without the GmailProvider.apk or is it safe to remove it?
Click to expand...
Click to collapse
It's safe to remove both, though the provider doesn't do much without the app, so it's also safe to leave it there.
3) When we first install a new ROM, starting clean, there's an introductory guide allowing to setup some things initially. One of those is the language. There is a bunch of languages, specially like English (A), English (B), English © where the letters represent other languages. I don't need all that, I only need English (Portugal). Is there anyway I could remove the ones I don't want from the ROM or I would need to dig into the Android source for that?
Click to expand...
Click to collapse
Hero roms take their settings from xml files in /system/customize. You'll see a bunch of xmls in there. Some are references to other xmls to be used (mns_map and cid_map), some are default settings (default.xml, COMMON.xml) and some are carrier specific settings (/CID/HTC_*.xml and /MNS/Lan_*.xml). The files are very redundant, and it's hard to tell which one is being used when, though, as a rule, you should understand that Hero loads a default from the /customize folder and then looks at mns_map and cid_map for the file to use inside /CID and /MNS. I'm not yet aware how it chooses the file, but, if you narrow down it's choices, then you can control which ones it uses and modify the settings there. If you crack open the xmls, you'll immediately see the languages defined in there (en_US, en_GB, etc), so removing languages is a matter of deleting the lang/region and adding your own (en_PL?). Again, there's so many xmls in there that it's hard to know which one to use, so either make your own and delete the rest, or edit them all. I'll give you the batch that I use in my /customize, reading through them should help you understand what's going on with those xmls.
3.1) How about adding my own language? Is there a way I could do that to my ROM?
NOTE: Funny thing, the Android system is localized on my country's language when I first start the device but as soon as I change it to English, I cannot change it back to Portuguese cause that option doesn't exist. How come the device is in Portuguese when I first start it up after flashing a new ROM? This also happens every time I swap the SIM card, but I can never select Portuguese language by myself. Why?
Click to expand...
Click to collapse
Look above, it's because the language is obtained from the carrier and used during first boot, but it's not defined in an xml, so it's unavailable, make it available by adding it to the list.
4) I noticed that after flashing my custom ROM, after booting the device for the first time, the mobile network was active. This is not good... For now I have 1 month free of mobile networking but that won't last forever. I don't want to waste money every time I'm trying a new ROM. Is there anyway to disable this thing from automatically starting up connected when I first boot?
Click to expand...
Click to collapse
This I'm not sure about. You could try emptying your /system/etc/apns-conf.xml so that it won't be able to register to any data service (just leave the <apns version="x"> </apns> tags, remove everything in between).
Good luck.

jubeh said:
You're missing copy_dir PACKAGE:data DATA:, so you're really not writing /data/app even if it's on the package. If you read through the update-script, you'll see that it starts by formating BOOT: and SYSTEM: and later on writes the contents of /system to SYSTEM: (with copy_dir) and writes the raw boot.img to BOOT: (with write_raw_image).
Add the line to the very end of the permissions settings (but before format and write boot, most likely, before the show_progress line). You also need to set a specific permission after writing anything to /data/app from an update-script, otherwise your market won't be able to install anything (as you'll have changed the permissions for /data/app through the update-script).
Click to expand...
Click to collapse
That's what I thought but I'm still missing one thing...
The generic HTC "update.zip" has format: DATA for the last line of the update script, which I suppose will delete everything becuse the copy_dir command was executed before.
Should I not use the format data option? Won't that leave traces of a previous installation if I want to start clean? What's the recommendations or suggestions?
jubeh said:
It's safe to remove both, though the provider doesn't do much without the app, so it's also safe to leave it there.
Click to expand...
Click to collapse
What will happen to the data-sync when i tries to sync the data and finds the provider for gmail is not there? Will it simply be ignored? Will there be any errors changing any of the options (specially the gmail one) on the data synchronization settings?
jubeh said:
Hero roms take their settings from xml files in /system/customize. You'll see a bunch of xmls in there. Some are references to other xmls to be used (mns_map and cid_map), some are default settings (default.xml, COMMON.xml) and some are carrier specific settings (/CID/HTC_*.xml and /MNS/Lan_*.xml). The files are very redundant, and it's hard to tell which one is being used when, though, as a rule, you should understand that Hero loads a default from the /customize folder and then looks at mns_map and cid_map for the file to use inside /CID and /MNS. I'm not yet aware how it chooses the file, but, if you narrow down it's choices, then you can control which ones it uses and modify the settings there. If you crack open the xmls, you'll immediately see the languages defined in there (en_US, en_GB, etc), so removing languages is a matter of deleting the lang/region and adding your own (en_PL?). Again, there's so many xmls in there that it's hard to know which one to use, so either make your own and delete the rest, or edit them all. I'll give you the batch that I use in my /customize, reading through them should help you understand what's going on with those xmls.
Click to expand...
Click to collapse
I'm familiar with XML files and have no problems editing them but this whole thing was a little confused specially because there are no documentation and like you said, it's hard to tell which one is which.
I know you kinda suggested that but I'm no seeing how to do it (maybe with the file you attached?) but, how could I start clean, only have default XML files and nothing else, no carrier specific stuff, no bulk, just plain default settings that I can tweak to my own liking.
Any hints on how to do that, or it's hard to tell and you don't know how to do it either?
jubeh said:
This I'm not sure about. You could try emptying your /system/etc/apns-conf.xml so that it won't be able to register to any data service (just leave the <apns version="x"> </apns> tags, remove everything in between).
Click to expand...
Click to collapse
Unfortunately, this did not work

The generic HTC "update.zip" has format: DATA for the last line of the update script, which I suppose will delete everything becuse the copy_dir command was executed before.
Should I not use the format data option? Won't that leave traces of a previous installation if I want to start clean? What's the recommendations or suggestions?
Click to expand...
Click to collapse
I've never understood why the default update script does that, but yeah, format DATA: will erase userdata available. It might not be such a bad thing if you hadn't written anything to it before. For a wipe before install, move format DATA: (and, optionally, also add format CACHE: ) right after format SYSTEM: and before you start writing anything to disk. It should give you a clean install without having to wipe from the recovery menu.
What will happen to the data-sync when i tries to sync the data and finds the provider for gmail is not there? Will it simply be ignored? Will there be any errors changing any of the options (specially the gmail one) on the data synchronization settings?
Click to expand...
Click to collapse
I've not tried it yet. Worst case scenario, you'll get the current problem we have with eclair builds (sync will just keep running and eat out battery because it's looking for a service). That's why I'd suggest leaving gmail provider there, it's just a broadcast service, but without an app to receive the data, it'll do nothing. If your concern is data usage due to sync, then you should turn off gmail sync in settings, that way only calendar and contacts are synchronized (although you'll still be using data).
I'm familiar with XML files and have no problems editing them but this whole thing was a little confused specially because there are no documentation and like you said, it's hard to tell which one is which.
I know you kinda suggested that but I'm no seeing how to do it (maybe with the file you attached?) but, how could I start clean, only have default XML files and nothing else, no carrier specific stuff, no bulk, just plain default settings that I can tweak to my own liking.
Any hints on how to do that, or it's hard to tell and you don't know how to do it either?
Click to expand...
Click to collapse
The files I attached are pretty generic. It's meant to replace your /customize folder, not add to it, with the exception being /resources inside /customize, that one I use a lighter version (no footprints) but I didn't include it in the archive for size. I have no carrier stuff there, just settings for language (I think I made it all en_US, replace or add what you need) and layouts for the different scenes, some mail app behaviors (added back the Yahoo! mail and AOL mail options, I think), some footprint, stock, and weather information. Again, it's meant to replace your /customize directory and the smaller amount of files makes it way easier to edit.
Unfortunately, this did not work
Click to expand...
Click to collapse
Blame the xmls in /customize for that. I mentioned earlier I don't exactly know how the system picks which xmls to use, I think it looks at your carrier, which is then assigned a number (for example, let's assume T-Mobile is given number 238, so xmls with 238 on them will be loaded). Problem is inside those xmls are also apn settings (which I've removed from the files I provided so that the system looks instead for apn settings in apns-conf.xml in /etc).
You could flash your system erasing the whole /customize directory (and removing all mentions of it and it's sub-directories from the update-script so that install doesn't fail). Doing that will give you ALL languages available to the rom, but you'll also lose all your pre-loaded settings (like scenes, weather locations, footprints, stocks) and HTC Setup will take an ungodly amount of time to start (about 4-5 minutes after it's done dexopting) while it builds it's own default settings. It's another route you could take, though. Experiment and you'll eventually hit what you want/need.

OK, I guess I'll have to waste some time testing all this stuff, but I suppose I'll leave that to some other day and do more basic things for now.
But just to be sure, if I use your "customization" I'm basically starting from scratch (clean, defaults) and I can use the "old" customization stuff to understand how am I supposed to configured everything and adapt the clean customization folder to my needs. Right?
jubeh said:
You could flash your system erasing the whole /customize directory (and removing all mentions of it and it's sub-directories from the update-script so that install doesn't fail). Doing that will give you ALL languages available to the rom, but you'll also lose all your pre-loaded settings (like scenes, weather locations, footprints, stocks) and HTC Setup will take an ungodly amount of time to start (about 4-5 minutes after it's done dexopting) while it builds it's own default settings. It's another route you could take, though. Experiment and you'll eventually hit what you want/need.
Click to expand...
Click to collapse
This gave me an idea, dunno if it could work...
What if I did that, erase everything inside customize, let the system build its own default settings and then access that directory (through ADB maybe?) and see what the system has created. It should be much more cleaned, without all those carrier XMLs and stuff. Then I could start from there, what do you think?
I have a few more questions about the update-script if you don't mind, but I'm very tired to write them and discuss them now. I'll leave them for another day if you don't mind coming back here and help me out some more.
Thanks for all your help so far

I should probably have mentioned, some of the stuff on those xmls is pretty straightforward (setting language, region, weather, footprints, etc), but a lot of it (setting up default shortcuts, default widgets, system settings) requires working knowledge of android, or at least of how activities are handled and how xmls come into play and how android references the different activities in a package.
On a different note, the xmls are read-only configuration files, actually, anything in system is read-only data (program data, configuration data, resource data), so if you don't have a customize directory the system won't recreate it. What the system does do during normal boot (factory) is to take those carrier-specific settings and apply them to a file(s) in /data where configuration settings are ultimately written/re-written and accessed at will. The xmls are just a database of the settings that will ultimately get picked. When you don't have a /customize directory the system writes the settings straight to -rw enabled /data partition based on things like your current carrier, carrier provided information for locale, etc.
I don't think you can take what's written to /data and use it on a default install package, well, you could, but it's much easier (and a more correct approach) to learn how those xmls in /customize work.

jubeh said:
(...)but it's much easier (and a more correct approach) to learn how those xmls in /customize work.
Click to expand...
Click to collapse
I wouldn't mind doing that if there was some kind of documentation... As it is right now, it's too confusing and there are too many XMLs to begin with...
jubeh said:
I've not tried it yet. Worst case scenario, you'll get the current problem we have with eclair builds (sync will just keep running and eat out battery because it's looking for a service). That's why I'd suggest leaving gmail provider there, it's just a broadcast service, but without an app to receive the data, it'll do nothing. If your concern is data usage due to sync, then you should turn off gmail sync in settings, that way only calendar and contacts are synchronized (although you'll still be using data).
Click to expand...
Click to collapse
But if I leave the gmail provider there and do not turn off gmail sync in settings, my e-mail will still be synced right? There's just no Gmail app to view them but they are still going to be downloaded, correct?

Ok, I finally got a change to test the Gmail/GmailProvider thing and it didn't look like there any problems like trying to sync and draining the battery. Actually, if auto-sync was enabled, there was always a message saying the mail was never synced and no matter how many times you check/uncheck it, it wouldn't sync. If auto-sync was disabled, you could press the gmail button many times but nothing happened, no sync at all (of course).
I think this behavior is fine and I can safely delete the Gmail and provider from the phone without worrying that something will go wrong
Ok, now my other questions that I said I was going to post later...
As we talked about, you suggested to add a format data and cache right after a format to system. Comparing that to the MoDaCo ROM (and other ROMs too) it's common for them to, instead of formatting data, they do this:
Code:
delete DATA:app
delete DATA:init.sh
delete DATA:local
delete DATA:dalvik-cache
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
Why are they doing this? If they are not formatting, it means there's something inside data that they want to keep. Have any idea what and why?
The other question I had about the update-script is that the following two lines are found in the original HTC update but not on MoDaCo's:
Code:
symlink dumpstate SYSTEM:bin/bugreport
symlink dumpstate SYSTEM:bin/dumpcrash
Are they relevant somehow? Should I just ignore and leave them there or remove them just as MCR did, for some reason?
There's a file named build.prop inside the system folder. I've already searched countless times and can't find any proper documentation about this file... It seems it can hold various default properties for the system and it would be nice to know what I can configure there. Maybe I could workaround problem #4 (top of the topic) in this file? Maybe some file in the Android source has comments about the possible properties on this file? Do you have any idea?
One last thing...
I've read about odex and dex files but found it a little confusing... All I know is that original HTC updates come mostly with .apk files and a corresponding .odex file. But most (if not all) custom ROM's, do not have the .odex file but have a classes.dex inside the apk. What's the real difference between the two? And why custom ROMs prefer to have the .dex file instead of the .odex, is it better somehow for some reason? If so, what should I do to "change" the .odex files into classes.dex ones and put them inside the .apk files?

As we talked about, you suggested to add a format data and cache right after a format to system. Comparing that to the MoDaCo ROM (and other ROMs too) it's common for them to, instead of formatting data, they do this:
Code:
delete DATA:app
delete DATA:init.sh
delete DATA:local
delete DATA:dalvik-cache
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
Why are they doing this? If they are not formatting, it means there's something inside data that they want to keep. Have any idea what and why?
Click to expand...
Click to collapse
Yeah, I mentioned on the earlier post that doing "format DATA:" and "format CACHE:" would be the equivalent of running "Wipe data/factory reset" from the recovery menu since you mentioned about making a build that installs fresh. Paul's (MoDaCo) roms are no-wipe, so they preserve that program data and instead format all other directories inside /data that might be problematic during an upgrade; his roms use apps2sd, so he assumes that the user's downloaded apps are in mmcblk0p2 (the memory card's ext partition), so by formatting /data/app, he's only erasing the apps he included in his update package, if the user doesn't use a2sd, well, this erases their downloaded apps (one of the reasons I dislike the current a2sd implementation, it's not suited for all situations). Then he deletes data/init.sh, which is basically a script he includes in /data that runs during a rom's boot and starts things like a2sd, compcache, linux-swap, and other custom rom's "advanced" features. DATA:local is also app storage (I believe) and functions much like in windows' /users/x/application data/local, it can be problematic if not done between builds. Last, he erases /data/dalvik-cache, this is where the temporary application code resides (the extracted classes.dex from inside an apk) for rapid execution (for apks that haven't been pre-odexed). If you wish to make your rom no-wipe, add those lines instead of "format DATA:", but I'd remove "delete DATA:app" since it's a bit problematic for people who don't use a2sd.
The other question I had about the update-script is that the following two lines are found in the original HTC update but not on MoDaCo's:
Code:
symlink dumpstate SYSTEM:bin/bugreport
symlink dumpstate SYSTEM:bin/dumpcrash
Are they relevant somehow? Should I just ignore and leave them there or remove them just as MCR did, for some reason?
Click to expand...
Click to collapse
dumpstate is a debugging tool that android uses, but I don't know exactly where or how (I think it's an HTC-specific log for their HTCLog.apk). It contains two tools, dumpcrash and bugreport, much like toolbox is a binary that contains all the tools you see symlinked to it in update-script (bin, df, cp, mv, ls, insmod, etc.). It's not a problem at all if you remove it, but it's not a problem either to leave it there. Whenever something is symlinked, though (if it's something you added, for example, busybox tools), you have to make sure that the binary or a link to it or a previous symlink to it doesn't exist, otherwise your update-script will fail (like symlinking toolbox's insmod and then busybox's insmod, or busybox's ping if a ping binary already exists in the target symlink directory (for example /system/bin)).
There's a file named build.prop inside the system folder. I've already searched countless times and can't find any proper documentation about this file... It seems it can hold various default properties for the system and it would be nice to know what I can configure there. Maybe I could workaround problem #4 (top of the topic) in this file? Maybe some file in the Android source has comments about the possible properties on this file? Do you have any idea?
Click to expand...
Click to collapse
I guess that could work if you modify network type (there's a property somewhere in there about HSXPA, I think), but like you said, there's no documentation so it calls for some experimenting.
Those properties are also overrides, so, for example, if your rom doesn't do adb root (because ro.secure is set to 1 in init.rc), you can add ro.secure=0 in build.prop to enable adb root again.
Most properties are self explanatory, and pretty much, if you can't tell what it is, you probably shouldn't change it (for example, build.fingerprint allows you to change what kind of apps Market presents to you).
I'd say, don't mess around too much with the first batch (default build properties), with the exception of build.fingerprint.
One last thing...
I've read about odex and dex files but found it a little confusing... All I know is that original HTC updates come mostly with .apk files and a corresponding .odex file. But most (if not all) custom ROM's, do not have the .odex file but have a classes.dex inside the apk. What's the real difference between the two? And why custom ROMs prefer to have the .dex file instead of the .odex, is it better somehow for some reason? If so, what should I do to "change" the .odex files into classes.dex ones and put them inside the .apk files?
Click to expand...
Click to collapse
The classes.dex files is the dalvik bytecode for the application. It contains everything inside /appnamespace/src and /appnamespace/R if you're working from the source. It's basically all the java code already compiled for the dalvik virtual machine.
Dalvik doesn't do jit, so the classes.dex have to be extracted (if they're not already) and run through the virtual machine in a process called dexopt that produces code that can be much more rapidly excecuted by the dvm (dalvik virtual machine). The extracted code is then placed in /data/dalvik-cache (mentioned earlier) as [email protected]@[email protected]. All this pre-cached code eats away at your space in your /data partition (where you store your system data, like your email, messages, browser cache, and downloaded applications) and can be quite a big chunk of space gone (framework classes specially, since all jars inside /system/framework are nothing but java code, about 20MB for all framework code).
The solution to prevent usage of /data storage on an initial build is to have the code pre-extracted. This is done at compile-time running the created system through an emulator and then creating the odex files, which is almost exactly the same as the *@classes.dex found inside /data/dalvik-cache except that it's much better aligned to the device's memory configuration and offers (marginally) better excecution, plus when the package manager first encounters the apk, it's first step is to check if an .odex of it is available, if it is, then it skips checking and dexopting a classes.dex inside it and proceeds to the next package.
Odex files increase the space in your /system partition (it was already read-only, so it's not like you're missing out on something) so those 20 MB of framework that would have been tossed into your userdata partition are now out of the way into the useless (in an available-storage sense) /system partition so you have more free space for your apps.
There's a tool that can create odex files for apks that don't have them, it's called dexopt-wrapper and it has to be run from a running phone. However, this tool won't strip the (now unnecessary) classes.dex from inside an apk, so it has to be done manually. Removing the classes.dex once you've created an odex is a good idea merely on the basis of space savings. However, once a classes.dex has been odexed, it's very hard (used to be impossible) to turn it back to a classes.dex and place it inside an apk.
Usually, us rom cooks preffer the classes.dex inside the apk because odexes create conflicts when you try to mix and match apps between builds. Since the odex files are tied to the framework where they were created, adding an apk with it's odex to another framework (even one that's only ever-so-slightly different) will be very problematic (the odex code is no longer protected or verified, so the rom won't even boot because it will fail verification for all available packages).
The tool for reverting odex files is called deodexerant by jesus freke. It requires working knowledge of how the system goes through the dexopting process so you can run it in the proper order.
Another thing to know is that even if your /system is pre-odexed, your downloaded apps will still be run through the package manager and have their classes.dex dexopted and placed in /data/dalvik-cache.
Oh, and never, ever put odexes anywhere but /system/framework or /system/app (or /data/app_s if you use a2sd). The package manager currently can't handle odexes in /data/app or /data/app_private (the other two locations where it looks for packages other than /system/app).
In the end, whether to use/have odexes comes down to personal choice (and available space). There's two hero builds, those based in 2.73.405.61 and 2.73.405.66, that have not been pre-odexed, so they're perfect for experimenting, mix/matching, etc...

jubeh said:
Yeah, I mentioned on the earlier post that doing "format DATA:" and "format CACHE:" would be the equivalent of running "Wipe data/factory reset" from the recovery menu since you mentioned about making a build that installs fresh. Paul's (MoDaCo) roms are no-wipe, so they preserve that program data and instead format all other directories inside /data that might be problematic during an upgrade; his roms use apps2sd, so he assumes that the user's downloaded apps are in mmcblk0p2 (the memory card's ext partition), so by formatting /data/app, he's only erasing the apps he included in his update package, if the user doesn't use a2sd, well, this erases their downloaded apps (one of the reasons I dislike the current a2sd implementation, it's not suited for all situations). Then he deletes data/init.sh, which is basically a script he includes in /data that runs during a rom's boot and starts things like a2sd, compcache, linux-swap, and other custom rom's "advanced" features. DATA:local is also app storage (I believe) and functions much like in windows' /users/x/application data/local, it can be problematic if not done between builds. Last, he erases /data/dalvik-cache, this is where the temporary application code resides (the extracted classes.dex from inside an apk) for rapid execution (for apks that haven't been pre-odexed). If you wish to make your rom no-wipe, add those lines instead of "format DATA:", but I'd remove "delete DATA:app" since it's a bit problematic for people who don't use a2sd.
Click to expand...
Click to collapse
So, to make it clear, I could:
Option A) Add format DATA and format CACHE to my ROM and this would make my ROM to automatically wipe everything and start clean.
Option B) I could not add anything (no formats and no deletes) and warn the users to explicitly perform a wipe before flashing.
Option C) Add delete DATA:local and DATA:dalvik-cache and have a no-wipe ROM where the apps would be preserved from the last ROM, right?
Ok, I don't use a2sd and for now have no intention in doing so. My best guess is to start with option A for now and maybe opt by using some app to backup everything and restore everything back after flashing a wipe ROM. Or I could go with option C and everything could work fine but there's a chance I could run into some issues in the long run.
jubeh said:
The classes.dex files is the dalvik bytecode for the application. It contains everything inside /appnamespace/src and /appnamespace/R if you're working from the source. It's basically all the java code already compiled for the dalvik virtual machine.
Dalvik doesn't do jit, so the classes.dex have to be extracted (if they're not already) and run through the virtual machine in a process called dexopt that produces code that can be much more rapidly excecuted by the dvm (dalvik virtual machine). The extracted code is then placed in /data/dalvik-cache (mentioned earlier) as [email protected]@[email protected]. All this pre-cached code eats away at your space in your /data partition (where you store your system data, like your email, messages, browser cache, and downloaded applications) and can be quite a big chunk of space gone (framework classes specially, since all jars inside /system/framework are nothing but java code, about 20MB for all framework code).
The solution to prevent usage of /data storage on an initial build is to have the code pre-extracted. This is done at compile-time running the created system through an emulator and then creating the odex files, which is almost exactly the same as the *@classes.dex found inside /data/dalvik-cache except that it's much better aligned to the device's memory configuration and offers (marginally) better excecution, plus when the package manager first encounters the apk, it's first step is to check if an .odex of it is available, if it is, then it skips checking and dexopting a classes.dex inside it and proceeds to the next package.
Odex files increase the space in your /system partition (it was already read-only, so it's not like you're missing out on something) so those 20 MB of framework that would have been tossed into your userdata partition are now out of the way into the useless (in an available-storage sense) /system partition so you have more free space for your apps.
There's a tool that can create odex files for apks that don't have them, it's called dexopt-wrapper and it has to be run from a running phone. However, this tool won't strip the (now unnecessary) classes.dex from inside an apk, so it has to be done manually. Removing the classes.dex once you've created an odex is a good idea merely on the basis of space savings. However, once a classes.dex has been odexed, it's very hard (used to be impossible) to turn it back to a classes.dex and place it inside an apk.
Usually, us rom cooks preffer the classes.dex inside the apk because odexes create conflicts when you try to mix and match apps between builds. Since the odex files are tied to the framework where they were created, adding an apk with it's odex to another framework (even one that's only ever-so-slightly different) will be very problematic (the odex code is no longer protected or verified, so the rom won't even boot because it will fail verification for all available packages).
The tool for reverting odex files is called deodexerant by jesus freke. It requires working knowledge of how the system goes through the dexopting process so you can run it in the proper order.
Another thing to know is that even if your /system is pre-odexed, your downloaded apps will still be run through the package manager and have their classes.dex dexopted and placed in /data/dalvik-cache.
Oh, and never, ever put odexes anywhere but /system/framework or /system/app (or /data/app_s if you use a2sd). The package manager currently can't handle odexes in /data/app or /data/app_private (the other two locations where it looks for packages other than /system/app).
In the end, whether to use/have odexes comes down to personal choice (and available space). There's two hero builds, those based in 2.73.405.61 and 2.73.405.66, that have not been pre-odexed, so they're perfect for experimenting, mix/matching, etc...
Click to expand...
Click to collapse
Well, I'm currently basing my ROM on 2.73.405.38 as the .66 one has "test" on the name and it feels like a beta version, which I'd rather not use. The .38 version uses .odex files for most of the packages (only a few has classes.dex inside) and I guess I'll leave them like that. If future ROMs, like that .66 one, comes with classes.dex vs .odex files, well, then I'll just leave it like that...
That looks like a very important thing in Android apps and at the same time something that I should be very knowledgeable about before messing with it, so I'll just stay put
Thanks once again for taking your time with such complete answers

So, to make it clear, I could:
Option A) Add format DATA and format CACHE to my ROM and this would make my ROM to automatically wipe everything and start clean.
Option B) I could not add anything (no formats and no deletes) and warn the users to explicitly perform a wipe before flashing.
Option C) Add delete DATA:local and DATA:dalvik-cache and have a no-wipe ROM where the apps would be preserved from the last ROM, right?
Ok, I don't use a2sd and for now have no intention in doing so. My best guess is to start with option A for now and maybe opt by using some app to backup everything and restore everything back after flashing a wipe ROM. Or I could go with option C and everything could work fine but there's a chance I could run into some issues in the long run.
Click to expand...
Click to collapse
Sounds like you got it.
Well, I'm currently basing my ROM on 2.73.405.38 as the .66 one has "test" on the name and it feels like a beta version, which I'd rather not use. The .38 version uses .odex files for most of the packages (only a few has classes.dex inside) and I guess I'll leave them like that. If future ROMs, like that .66 one, comes with classes.dex vs .odex files, well, then I'll just leave it like that...
That looks like a very important thing in Android apps and at the same time something that I should be very knowledgeable about before messing with it, so I'll just stay put
Click to expand...
Click to collapse
Test only refers to the build being signed with test-keys instead of release-keys. In my testing, it makes no difference whatsoever for a rooted user, and there's many theories floating around about signatures, I haven't researched signatures a whole lot because test-keys have worked fine enough for me, so I'm not so sure, but they seem to be working fine for all.
The reason those releases are un-odexed (and signed with test-keys) is because of the problem with odexes I mentioned earlier. When HTC is testing their rom packages and they need to make, say, a framework change, all the apps with odexes would fail as the framework has changed, so, for testing purposes, it's best to leave the apks and jars in the build full (with classes.dex inside). Once you know your build is solid and you won't be changing much, it's safer to odex the whole thing (sort of like me, I'm waiting for the final release of eclair for ADP1 so I can put out a Dream rom odexed and fully working).
Thanks once again for taking your time with such complete answers
Click to expand...
Click to collapse
No problem, I get bored at work

So, version .66 is not beta or anything? I know MoDaCo has his latest ROM based on that and you're saying they are only different in the signing key, still, it's somewhat a "leaked" build, not quite "official".
But hey, if others are using it without any issues, maybe I should be using it too

On second thought...
1) I can't find a pre-rooted .66 version and I don't know how to root it :/
2) There's lots of files in system/xbin that I don't see referenced anywhere (not even in the update-script) and I don't know what should I do with them.
I'm finding it much more confusing to base my ROM on .66 than on .38...

It is a beta, of sorts, but between it and the final shipping product there's not much difference. It's just much, much easier to work with it.
1.) The simplest form of root is to change ro.sercure to 0 in default.prop in the boot ramdisk. Adding a su binary/Superuser.apk is optional, those you can pull from one of paul's builds.
2.) The system/xbin is added to the boot classpath in the boot ramdisk in init.rc. The only mention of it in in update-script is the permissions for the whole folder and then just a couple permissions for some tools, not all of them.

[HOWTO] Install Latest WaveSecure in ROM

I found that [email protected]'s ROM had a good idea of adding WaveSecure to the system partition (preventing listing in the My Downloads part of market, and preventing uninstallation through normal means), but his version is slightly out of date now (latest version is 3.0.0.43)
As a result, I set about finding a means to install WaveSecure to ROM myself. Here are my findings for anyone interested in doing the same.
Install the latest version from the market (3.0.0.43 at this time). Now use adb pull to get it off the device onto your pc
Code:
adb pull /data/app/com.wsandroid.apk D:\com.wsandroid.apk
Now open Market back up from the menu, go to My Downloads, and choose Wavesecure Mobile Security Beta and uninstall it for just now (to get it off the data/A2SD location that normal apps are stored in) - thanks, my-space!
Then push the saved apk to the system partition after a remount (to make it read/write)
Code:
adb remount
Code:
adb push D:\com.wsandroid.apk /system/app/com.wsandroid.apk
Then set it all up as usual (will appear in apps list immediately)
and remount system as read only again
Code:
adb remount
And that seems to be it so far. Remember to change the D:\com.wsandroid.apk path to whatever you actually used.
Let me know if anyone finds any problems with this, but I've done it and, fingers crossed, it's worked OK for me.
Obviously, this is only for root users, and there are no guarantees for this.
Couple of questions that might need looked into -
- Do settings carry across after a wipe (as Paul claims Modaco's version does. I've never tried it so can't confirm)
- Is there any disadvantage to using this method? (I guess this is all Modaco's update.zip does, but I don't know)

you forgot to metion to uninstall wavesecure before it is pushed back into system....

my_space said:
you forgot to metion to uninstall wavesecure before it is pushed back into system....
Click to expand...
Click to collapse
Oops! Knew I'd forget something, as I always seem prone to do. Well spotted, and OP corrected.
Thanks

No worries I got a bit confused when i pushed it back onto the phone and was still in my downloads...

I've rooted my phone already but whenever i try to use the command adb remount I get "remount failed: operation not permitted". Suggestions?

I see more and more often, redundant threads.
What How-to will you post next time? How to change backlight settings?
You're pointlessly spamming the board.

I can't say i agree with the 'pointlessly spamming the board' comment, but i would have thought this would at least be better in the applications and themes subforum rather than in development.
Don't forget that an awful lot of android users (and more recently all HTC devices) are more and more 'newbs' and need stuff like this.
While this is good and provides info that people like that need (and myself cos i'm crap at adb and stuff like that so wouldn't have had a clue how to do this previously), maybe the development forum is not the best place for it....

I'm guessing one won't be notified via Market if there's an update available if you push an app to /system/app/, right?

usb0 said:
I'm guessing one won't be notified via Market if there's an update available if you push an app to /system/app/, right?
Click to expand...
Click to collapse
You won't be notified, correct

If anybody doesn't already know, WaveSecure have started hosting update.zip files. This means it's now easy to update your "baked-in" version of WaveSecure without much messing around.
https://www.wavesecure.com/installations/update.zip
Download the file, save it to your SD card, reboot into the recovery console and choose the option to apply an update.zip file.
Voila! Your version of WaveSecure will be updated to the very latest version

DJBenson said:
[...]
Click to expand...
Click to collapse
That's really awesome! Thanks for the tip!
Just a question of curiosity: If I push an app to /system/app/ and then issue the rm-command to remove its apk, won't there be lying a bunch of files associated with the program and with absolutely no function, since the app itself is removed? How do I know the name of these files and where they are located for removal?

I'm a bit confused by that question. My understanding (which may be incorrect but from what I've seen of the "guts of a ROM" appears to be the case) is that the applications reside in the apk files, they are not extracted. If you list the content of any of the app folders (/system/app, /data/app or /data/app-private) then all you get is a bunch of apk files (and some odex files). So when you 'push' an apk to the phone, that application is then "installed", when you rm/remove an application, you do so by removing the apk.

if you remove the apk you have left something in /data/dalvik-cache. wiping the dalvik-cache every now and then helps reclaiming that space, though it is not much.
the app settings and data are stored in /data/data, you could delete the files manually by checking their names (no idea if/what convention the names follow), imho not worth the trouble as it is only a few kb.
odex files aren't created if you don't do in a PITA process manually. don't worry about them, don't touch them, then you're good. odex files are only for system apps.

I bought a used phone and it had WaveSecure already installed. I couldn't find it in the applications list to uninstall, so I did a factory reset on the phone. The application was still there and it still didn't show up in the applications list.
I have the Superuser Permissions application, so someone must have rooted it.
Is there any way to uninstall this?

motomeup said:
I bought a used phone and it had WaveSecure already installed. I couldn't find it in the applications list to uninstall, so I did a factory reset on the phone. The application was still there and it still didn't show up in the applications list.
I have the Superuser Permissions application, so someone must have rooted it.
Is there any way to uninstall this?
Click to expand...
Click to collapse
""Just (re)flash a ROM................""

I just noticed that WaveSecure now points to this post for instructions to install as system application, and I am not sure that the update.zip maintained with them is up-to-date.
However, you can now select to download the .apk directly to your PC thus eliminating the first Market step in this guide.

strife242 said:
I just noticed that WaveSecure now points to this post for instructions to install as system application, and I am not sure that the update.zip maintained with them is up-to-date.
However, you can now select to download the .apk directly to your PC thus eliminating the first Market step in this guide.
Click to expand...
Click to collapse
I believe it is kept up to date, as VillainROM kitchen uses it as a source for the WaveSecure app (fetched each night to keep it up-to-date).
I've certainly never had any problems with it.

Excellent guide Pulser,... I should really reinstall Wavesecure now Im not using a MoDacO Custom ROM. Nice one bruv.

Found this to be helpful.
https://www.wavesecure.com/blog/how-to-make-wavesecure-hard-reset-proof.aspx

I just did a search in the Market fro 'wavesecure' and two things popped up WaveSecure and WaveSecure UninstallProtection Add-on which needs to be uninstalled before WaveSecure and if the add-on is uninstalled it is supposed to lock the phone. (all this is in the description I have yet to try)

[MOD] Xoom: symlink /system/apps to /data for GApps on CM10-based ROMs (11/28 update)

Disclaimer: This mod is provided as is. I am not responsible for you or your actions, I can barely handle myself most days. This will probably kill your dog and frame your spouse, so if your not feeling up for it, maybe sit this out.
What is this?
This addresses the lack of space on the /system partition that has been forcing people to compromise on GApps packages, and generally be subjected to wonky behavior.
How do I install it?
First you must make a nandroid. This is a flashable zip that you apply in recovery *in between* flashing cm10 and GApps. *NOTE* There are -NO- GApps in this download. This is just a shim to make room for them!
How does this work?
It's mostly edify, with a shell script that shouldn't have been necessary, but I got impatient, and hell, it works.
What it does is it will create folder on your /data partition called /data/system.app.
It makes sure that the permissions match the original.
Then it will migrate the *entire* contents of the traditional /system/app folder into the new folder, also preserving the owners, permissions and attributes of files
Finally, it deletes the original, and creates a symlink in its place, so that anything that tries to read, or write, to that folder, will end up on the /data partition.
You can now install a full GApps package, and you'll have a lofty 25MB or so to spare!
Is it safe?
I have tested this on my 4G Stingray using CM10 nightlies and Rogue Recovery as both a clean flash and an upgrade. It works under that configuration. See "Before you download" below for currently known issues.
Is this only for the Stingray?
Yes and no. Yes, I only have that one device that suffers the partition issue, and on which to test this. No, I don't see why it couldn't work for at least one other device. But still probably don't try it unless you really know what you're doing.
Why did you make this?
I wanted to solved this problem, and now it's my gift to you all! Enjoy!
Can I incorporate this in my ROM or on my website or at my grandmother's bingo party?
Knock yourselves out. I hope this ends up being super helpful for everybody.
Something broke and it's all your fault!
Before you download!
Make a nandroid! This is twice that I'm telling you, so I mean it!
As of right now, if you were to factory reset your device, it would be necessary to restore or reflash from recovery. Do not factory reset if you apply this mod! (Thanks runandhide05!)
This mod does not work with TWRP recovery at the moment (But I'm working on it). (Thanks dodgefan67!)
Face unlock currently fails to install. I am working to address this.
Some future version will use addon.d so that you don't have to flash this every time. This will be implemented once the above issues have been dealt with.
The zip file was updated on 11/28 to fix an issue where the shell script had windows-style line endings, causing it not to run in TWRP and possibly some other recoveries.
The download:
http://www.tangerinepulsar.com/files/cm10-xoom-system-symlink-mod-signed.zip
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}

the only problem i see here is that you have no way to backup you rom, well, if you run a backup it will say its successful, but you have nothing to get /data/system.app backed up and re symlinked when you restore, and if you preform a factory reset you will loose all of your system apps,
you should look at how EOS symlinks there gapps data at /data/ you have to include addon.d scripts so that if you run a backup it will backup your new folder too, otherwise problems may/will happen.
edit:
for example for the /usr/srec stuff ( google now hotword and talk back files) you have to do something like this
Code:
. /tmp/backuptool.functions
list_files() {
cat <<EOF
EOF
}
case "$1" in
backup)
list_files | while read FILE DUMMY; do
backup_file $S/$FILE
done
;;
restore)
list_files | while read FILE REPLACEMENT; do
R=""
[ -n "$REPLACEMENT" ] && R="$S/$REPLACEMENT"
[ -f "$C/$S/$FILE" ] && restore_file $S/$FILE $R
done
ln -s /data/eos-srec /system/usr/srec
;;
pre-backup)
# Stub
;;
post-backup)
# Stub
;;
pre-restore)
# Stub
;;
post-restore)
# Stub
;;
esac

runandhide05 said:
the only problem i see here is that you have no way to backup you rom, well, if you run a backup it will say its successful, but you have nothing to get /data/system.app backed up and re symlinked when you restore, and if you preform a factory reset you will loose all of your system apps,
you should look at how EOS symlinks there gapps data at /data/ you have to include addon.d scripts so that if you run a backup it will backup your new folder too, otherwise problems may/will happen.
Click to expand...
Click to collapse
I'll definitely look at incorporating this approach, thanks!
At the time I didn't think that backups were that big of a deal so long as you backed up both system and data, all of the symlinks would still be intact. I did write the edify script in such a way that it allows for an existing /system.app folder to already be there and will re-symlink it. But there are always times when you want to restore one and not the other, or switch ROMs without losing data, and this doesn't handle that at all. I definitely agree that a factory reset would trash you pretty badly, it's not a scenario that I accounted for in any way.
Thanks for the nudge in the right direction, I'll keep working on this.

Thanks for workings on this, very exciting news.

I'll start cranking out eos test builds soon. Then I'll take a look at what kind of gapps solutions are available. One thing for sure is it will be a bare bones package. That means anything that can be acquired from the Store will be.

this is good news indeed. i will try this out on my wifi xoom in the next day or two
i understand about doing a factory reset, but if i just update from one nightly to another (dirty flash) i would have to run this each time as well, correct?

dodgefan67 said:
this is good news indeed. i will try this out on my wifi xoom in the next day or two
i understand about doing a factory reset, but if i just update from one nightly to another (dirty flash) i would have to run this each time as well, correct?
Click to expand...
Click to collapse
You should not have to.
Sent from my Galaxy Nexus using Tapatalk 2

runandhide05 said:
You should not have to.
Sent from my Galaxy Nexus using Tapatalk 2
Click to expand...
Click to collapse
doesn't the script actually move the contents of /system/apps though? installing a new version would put apps there right? and overwrite the symlinks?

dodgefan67 said:
doesn't the script actually move the contents of /system/apps though? installing a new version would put apps there right? and overwrite the symlinks?
Click to expand...
Click to collapse
Well it depends on how/if he updated it.
But if its the same as earlier then ya you will need to reflash.
Sent from my Galaxy Nexus using Tapatalk 2

runandhide05 said:
You should not have to.
Sent from my Galaxy Nexus using Tapatalk 2
Click to expand...
Click to collapse
dodgefan67 said:
this is good news indeed. i will try this out on my wifi xoom in the next day or two
i understand about doing a factory reset, but if i just update from one nightly to another (dirty flash) i would have to run this each time as well, correct?
Click to expand...
Click to collapse
runandhide05 said:
Well it depends on how/if he updated it.
But if its the same as earlier then ya you will need to reflash.
Sent from my Galaxy Nexus using Tapatalk 2
Click to expand...
Click to collapse
I haven't released an update yet. Please reflash per the current instructions. I'll make it obvious when the script has changed.

ok, i used CM10 stable from 11/13/2012 and the official GApps JB package from 10/11/2012
flashed CM10
flashed this mod
flashed gapps
on first boot, i did not get any google account sign on screens, went straight to homescreen, no play store or any google apps installed
reflashed gapps and rebooted and this time i got the play store and google search, but that's it. is there more to gapps? i think they took out gmail, but would expect more for a 90meg file. never really looked at this, i have been using the EOS version of gapps with Zigackly's rom
by the way, according to TB i have 29.4mb left on /system and that is before i remove the other stuff i dont want. so it seems to have worked on a MZ604 wifi device
i will use it this way for a couple of days, restoring some apps and installing things and changing settings and see how it goes. then i will update to a nightly and see how that goes
so far so good

dodgefan67 said:
ok, i used CM10 stable from 11/13/2012 and the official GApps JB package from 10/11/2012
flashed CM10
flashed this mod
flashed gapps
on first boot, i did not get any google account sign on screens, went straight to homescreen, no play store or any google apps installed
reflashed gapps and rebooted and this time i got the play store and google search, but that's it. is there more to gapps? i think they took out gmail, but would expect more for a 90meg file. never really looked at this, i have been using the EOS version of gapps with Zigackly's rom
by the way, according to TB i have 29.4mb left on /system and that is before i remove the other stuff i dont want. so it seems to have worked on a MZ604 wifi device
i will use it this way for a couple of days, restoring some apps and installing things and changing settings and see how it goes. then i will update to a nightly and see how that goes
so far so good
Click to expand...
Click to collapse
I'm a little perturbed about the way that first flash went. If you experience that again, will you please post the log from /cache/recovery prior to the re-flash? I want to make sure that there's not some scenario that I'm not handling properly with just one flash.
BTW, Did you flash it all from recovery or did you use something like ROM Manager or ROM Toolbox to script the recovery actions? Just so I cover my bases.

oscillot said:
I'm a little perturbed about the way that first flash went. If you experience that again, will you please post the log from /cache/recovery prior to the re-flash? I want to make sure that there's not some scenario that I'm not handling properly with just one flash.
BTW, Did you flash it all from recovery or did you use something like ROM Manager or ROM Toolbox to script the recovery actions? Just so I cover my bases.
Click to expand...
Click to collapse
i will do it again and get you the log file. i'm using twrp and did it all from there. i even did each zip one at a time even though twrp lets you do multiple ones in one go

good news and bad, i got the log file but i had to dump it to a plain text file so it is hard to read, but if you search for symlink-mod you will find the section where your mod starts and right after that is where i flashed gapps ***edit***never mind about the formatting, i clicked the attachment below using Chrome and it came up fine
i was able to duplicate my issue. i wiped everything three times and installed a different rom and different gapps package and everything was fine. then i wiped three more times and installed cm10 stable, your mod, then the official jb gapps, rebooted and same thing. no google apps installed. this log file is from right after the first reboot
then i went back and flashed gapps again, rebooted and have the play store
i can put CWM on it and try it with that recovery if you want

Here is the problem, u can not have anything in the /system/app folder.. Nothing at all, so the problem lays that if you try to flash a ROM, then the sy link it will copy everything in system/app to data, remove the directory (system/app) then link it to the new directory in /data. Then you are trying to flash gapps which tries to put stuff in /system/app which that directory does not exist so its going to create it. Which will undo the symlink or will not be able to flash anything to /app.
So for the sy link to work u must flash both rom and gapps, which is where the problem of limited space cause now u won't get all your files over to the /system because not enough room. So to fix this you can do one of several things, one of which would be...
Change the gapps package to where the system apps flash to the new /data directory created by the sysmlink.
So if the new directory is /data/sysapp/ open the gapp package and take all the system/app/ and move it to a new folder on the root of the zip called data then in there create a folder called sysapp and put all the /system/app/ content in there. And remember to update the updater script for the resource "data" "/data" and that "should" fix it
Sent from my Galaxy Nexus using Tapatalk 2

dodgefan67 said:
good news and bad, i got the log file but i had to dump it to a plain text file so it is hard to read, but if you search for symlink-mod you will find the section where your mod starts and right after that is where i flashed gapps ***edit***never mind about the formatting, i clicked the attachment below using Chrome and it came up fine
i was able to duplicate my issue. i wiped everything three times and installed a different rom and different gapps package and everything was fine. then i wiped three more times and installed cm10 stable, your mod, then the official jb gapps, rebooted and same thing. no google apps installed. this log file is from right after the first reboot
then i went back and flashed gapps again, rebooted and have the play store
i can put CWM on it and try it with that recovery if you want
Click to expand...
Click to collapse
Okay, I see a couple of things that are downright strange. To start off, the reason it looks like a flat text file is because the line-breaks are unix-style ('\n') and windows apps generally don't show linebreaks unless they have a carriage return preceding a newline character ('\r\n'). Open files from android in Notepad++ and they'll render much more nicely. Turn on viewing end of line characters and you'll see the difference between the Windows and Unix styles of newlines.
In practice, these are invisible control characters that don't visibly render, except in so far as their introduction of a line-break. '\r' and '\n' are aliases of them used in programming. They are actually the hex values 0A and 0D. But, since they are also characters, unexpected ones will have some kind of effect.
On lines 4066-4067 I see an error in the copy.sh: "/tmp/copy.sh: line 2: '\r' : not found"
There is a stray carriage return in the middle of this log file. There should be none. Line 2 in that file is BLANK. The source *should* have a single unix-newline '\n' and nothing else. I would bet that busybox is trying to interpret it and as a result, the shell script never runs. Nothing is getting copied out of /system/app.
I checked the zip I uploaded, and sure enough, the shell script has windows-style line endings. Oops. I've replaced it with a version that does not, but is identical other than that. So that part is fixed. Not sure why it works in CWM and not in TWRP, as far as I'm concerned, it shouldn't have worked in either. They must have slightly different busybox binaries is all that I can think.
The other issue is that both the deletion of /system/app and the subsequent symlinking both claim to have worked. However it's clear that /system/app was not deleted because if it were, you wouldn't have booted correctly. The symlink couldn't have worked either because you can't symlink a location that has files in it. So what in the world happened here? I'm kind of not sure, especially since those steps each returned 0, or else we wouldn't have gotten the success messages.
The wrinkle that you got *no* gapps after a flash is even stranger. But There is one more bit of interest here. I see an error message on line 4135 "cp: target '/system/app' is not a directory". At this point, this is the gapps trying to install face-unlock. I went ahead and looked in my own log files and here's what I found: "cp: target '/system/app' is not a directory." I never used it, but further inspection confirms, no portion of the face-unlock is installed using my mod, and the reason is the syntax of the gapps install-optional.sh: "cp -a /tmp/face/* /system/". Because /system/app is a symlink, you cannot copy the folder /tmp/face/app over it successfully.
This begs the question, how is it displaying the properties of being both a symlink and NOT a symlink?
I'm going to work on these issues, and I'll probably flash TWRP and test against it as well and try to come up with some answers and a one-size-fits-all solution. In the mean time, I would appreciate it if you could confirm whether you have a /data/system.app directory, and if so, what the contents are. I have a hunch that you'll find the gapp apks (minus face-unlock) in that directory, but none of the system apps. I'm not sure how that would be possible, or what it means, but it makes the most sense from where I'm standing. In the meantime, I'm going to add some notes to the main post.
I'll hopefully get some of this figured out by this weekend.

oscillot said:
Okay, I see a couple of things that are downright strange. To start off, the reason it looks like a flat text file is because the line-breaks are unix-style ('\n') and windows apps generally don't show linebreaks unless they have a carriage return preceding a newline character ('\r\n'). Open files from android in Notepad++ and they'll render much more nicely. Turn on viewing end of line characters and you'll see the difference between the Windows and Unix styles of newlines.
In practice, these are invisible control characters that don't visibly render, except in so far as their introduction of a line-break. '\r' and '\n' are aliases of them used in programming. They are actually the hex values 0A and 0D. But, since they are also characters, unexpected ones will have some kind of effect.
On lines 4066-4067 I see an error in the copy.sh: "/tmp/copy.sh: line 2: '\r' : not found"
There is a stray carriage return in the middle of this log file. There should be none. Line 2 in that file is BLANK. The source *should* have a single unix-newline '\n' and nothing else. I would bet that busybox is trying to interpret it and as a result, the shell script never runs. Nothing is getting copied out of /system/app.
I checked the zip I uploaded, and sure enough, the shell script has windows-style line endings. Oops. I've replaced it with a version that does not, but is identical other than that. So that part is fixed. Not sure why it works in CWM and not in TWRP, as far as I'm concerned, it shouldn't have worked in either. They must have slightly different busybox binaries is all that I can think.
The other issue is that both the deletion of /system/app and the subsequent symlinking both claim to have worked. However it's clear that /system/app was not deleted because if it were, you wouldn't have booted correctly. The symlink couldn't have worked either because you can't symlink a location that has files in it. So what in the world happened here? I'm kind of not sure, especially since those steps each returned 0, or else we wouldn't have gotten the success messages.
The wrinkle that you got *no* gapps after a flash is even stranger. But There is one more bit of interest here. I see an error message on line 4135 "cp: target '/system/app' is not a directory". At this point, this is the gapps trying to install face-unlock. I went ahead and looked in my own log files and here's what I found: "cp: target '/system/app' is not a directory." I never used it, but further inspection confirms, no portion of the face-unlock is installed using my mod, and the reason is the syntax of the gapps install-optional.sh: "cp -a /tmp/face/* /system/". Because /system/app is a symlink, you cannot copy the folder /tmp/face/app over it successfully.
This begs the question, how is it displaying the properties of being both a symlink and NOT a symlink?
I'm going to work on these issues, and I'll probably flash TWRP and test against it as well and try to come up with some answers and a one-size-fits-all solution. In the mean time, I would appreciate it if you could confirm whether you have a /data/system.app directory, and if so, what the contents are. I have a hunch that you'll find the gapp apks (minus face-unlock) in that directory, but none of the system apps. I'm not sure how that would be possible, or what it means, but it makes the most sense from where I'm standing. In the meantime, I'm going to add some notes to the main post.
I'll hopefully get some of this figured out by this weekend.
Click to expand...
Click to collapse
Any new news on this?

A little update on what I tried on my MZ-601 Everest. Any feedback will be welcome
Basically what I did was:
1. Trimmed down the CM10 Stable a little (132MB vs the original 141MB) by removing mainly the wallpapers.
2. Flashed the modified CM10 Stable zip (after a full wipe - factory plus cache).
3. Modified the Mod Script to point to "/data/system/app" (just to satisfy my OCD ) then flashed the zip. On first try it aborted with Status 7 on "Fixing permissions" so I had to manually create the directory "/data/system/app" after which the flash was successful.
4. Modified the GApps package (GApps from 11th Oct) by:
a. Removing the "optional" directory, install script and removing the relevant lines from "updater-script",
b. Moving the "/system/app" directory to "/data/system" and
c. Modifying the "updater-script" to reflect the change.
5. Flashed the GApps zip package which went uneventfully.
6. I now have 51.4MB free in "/system" according to Titanium Backup
PS: I am including the modified "Mod Script" and "updater-script" from GApps package if anyone else wants to try it. Of course you mileage may vary and I cannot claim any responsibility for any loss of data that may occur
UPDATE: Tried it with the CM10.1 nightlies and it works flawlessly. Flashed the nightly over the last nightly, flashed this script then the official gpaps and rebooted. All the settings and customizations retained! Still have some 56MB left in /system

[TRICK] How to avoid Google Play Service update

In /data/app folder make a new folder with name com.google.android.gms-1.apk
(to make the folder use a file manager app or the command "su" followed by "mkdir /data/app/com.google.android.gms-1.apk" with a Terminal Emulator app and Busybox installed.
NOTE: Making a new folder in /data partition require root permissions so the Andriod system must be rooted first)
That's it!
------------------------------------------------------------------
Hello everyone,
owners of smartphone with very limited internal storage, like Galaxy Y, will likely end up to be concerned of every single MB of free space lost; and the more the concern as the free space is getting closer to zero.
So there I am too.
I've put lot of effort to optimize the use of my Galaxy Y internal storage (/system, /data and /cache partitions all together) and I'm proud to have more than 50% of internal storage free and I like to have it that way for as long time as possibile.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Yesterday I had a very bad surprise... I suddenly lost 55% (actually 55MB) of free space for some unknow reason
Later, it turned out that the reason was that the Google Play Service had been silently updated in the background without any notification.
Actually the footprint of newer version of Google Play Service has a huge increase of 28MB... almost +78%!
The new apk size is 15MB bigger and a system integration of the update not only might likely eats up all the free space available in the /system partition, but still will need to use additional 13MB in /data partition.
Uninstalling the update was just a matter of tapping one button... but then another background update would have happen soon after.
So, to prevent the update from happening again, I used the same kind of trick used to avoid the self update of Google Play Store, as explained here [TRICK] How to stop Google Play Store self update - easy!
That's all folks!
Bye!

What I used to do was uninstall the Google play services
Then open YouTube
It will say you have to install Google play services
Click the link to install it & it will allow you to download it from the playstore
You can then move or link it to sd card with link2sd
I prefer always having the latest versions of apps as they tend to eliminate bugs - yes it might be a larger file size but you can move or link the apk to sd

Thanks for the hint Marcus
I simply hate when my device is modified without my previous consent.
I want to be the one who decide what to update and when to update because each update might have a cost (in terms of storage space, in terms of data plan usage, in terms of risks of unexpected apps or system errors).
Furthermore I've put lot of effort to make my phone system configuration in such a way that it can still be fully functional even without SD (as in case of a SD failure) or with a brand new empty one; so I've put all the most important applications (Internet browser, Youtube, Google Play Store, Google Play Service, Shake Calc, QuickPic, Maps, GMail, QuickOffice, FM radio, Camera, Root Explorer, Tethering Widget, Terminal Emulator, Flashlight, VNC_Server, SSHDroid, VoipCheapGlobal and few others) in /system and /data partitions and there I want them to stay
All the best!

marcussmith2626 said:
What I used to do was uninstall the Google play services
Then open YouTube
It will say you have to install Google play services
Click the link to install it & it will allow you to download it from the playstore
You can then move or link it to sd card with link2sd
I prefer always having the latest versions of apps as they tend to eliminate bugs - yes it might be a larger file size but you can move or link the apk to sd
Click to expand...
Click to collapse
how to stop play services auto update???????????

What I found was that on my custom rom (cm 10.1.3) google play services was installed as /system/app/GmsCore.apk which was the version that I had installed when I flashed a gapps zip file right after I had initially installed the rom. If I only had this version installed then the play store app would not auto-update google play services. However as soon as I updated google play services from the play store manually it was installed as /data/app/com.google.android.gms-1.apk and as long as it was installed here the play store would auto-update it whenever it wanted without asking or informing me. I even tried doing chattr +i com.google.android.gms-1.apk as some other people have mentioned but then it would just install the new version as com.google.android.gms-2.apk, etc...
I clicked on "uninstall updates" for the google play services app and it removed the /data/app/com.google.android.gms-1.apk file and went back to not auto updating. The problem is that I wanted to use some google apps that would not work with the older /system/app/GmsCore.apk version but I wanted to install a slightly newer version but not have it get auto-updated.
So what I ended up doing was to get the slightly newer version and copy it to /system/app/GmsCore.apk. As long as there is no google services apk in /data/app/ it seems that it will not get auto-updated. This allows me to maintain control over when/if It ever gets updated.
Also note that I have applied a no self-update patch from lucky patcher to my google play store (v4.3.11). I don't know if this is also needed to prevent googleplay services from auto updating.
The reason that I do not want to use the recent (>=4.4) versions of google play services is that they now include a firmware updater service that was transferring a ton of data in the background for no reason as mentioned here. The slightly older version that I am now using (4.3.25) does not have this.
I could have just disabled the service but if it is allowed to auto-update itself then it is possible that it will just get re-enabled or some other battery draining/data using feature will get added.
I would rather maintain control over all the software that is installed on my device instead of allowing google to do whatever they want without asking or even informing me.
Basically if you have the google play store app installed on your phone it allows google to install whatever they want on your phone whenever they want without your consent or notification. (unless you do a lot of hacking to disable it)
This seems pretty crazy to me and I am not sure if most people realize this. Who owns your phone? Is it you or google?
Sorry about the rant...

Hi garbb,
thanks for posting your rant it provides some interesting infos on different scenario than mine which is stock rom with Google Play service installed in system folder.
I also like to express my agreement on the reasons you stated that had led you to find a way to stop the self auto update.
I guess that the patch you applied with lucky patcher is what prevents the auto update to work.
But basically the trick that I posted here should work on your system too as it actually prevent the update process to copy the new updated version of the app in the destination folder that is /data/app
If Google Play Service is been updated once already (so there's a apk in /system and one in /data/app) then the the latest installed version is already named com.google.android.gms-1.apk and placed in /data/app so the update process will name the new updated apk as com.google.android.gms-2.apk.
So, in order to make the update process to fail you just have to make a /data/app/com.google.android.gms-2.apk folder.
Or you might want to integrate to /system the updated version that you like to have (eventually by using link2SD or Titanium Backup), then make /data/app/com.google.android.gms-1.apk folder to make to fail future auto update attempts.

rahul54321 said:
how to stop play services auto update???????????
Click to expand...
Click to collapse
Trick doesn't work.

Verrocio said:
Trick doesn't work.
Click to expand...
Click to collapse
Well, it's possible that the trick doesn't work for you (there's also a chance that as for garbb user you have to change the folder name digit to something greater than 1) but that doesn't mean that it doesn't work for everyone.
For me it works and I proved it on a youtube video that shows how to use the same trick to make the Google Play Store update to fail.
It would be nice if you can provide some more infos on your smartphone OS and Google Play Service version, just for the record and to get some clues on why eventually the trick doesn't work for you.

halnovemila said:
Well, it's possible that the trick doesn't work for you (there's also a chance that as for garbb user you have to change the folder name digit to something greater than 1) but that doesn't mean that it doesn't work for everyone.
For me it works and I proved it on a youtube video that shows how to use the same trick to make the Google Play Store update to fail.
It would be nice if you can provide some more infos on your smartphone OS and Google Play Service version, just for the record and to get some clues on why eventually the trick doesn't work for you.
Click to expand...
Click to collapse
I've tried both 5.0.89 and 6.1.88 (034) , nexus one, with latest euroskank rom from october. Also tried disable from play store on top of this "trick". The gms updates itself every time. I wouldn't mind it so much but it doesn't ask me and i want full control over whats happening on my device but still keep access to device manager.
edit: also tried with stock cm and marketupdater frozen on both, same result.

Verrocio said:
I've tried both 5.0.89 and 6.1.88 (034) , nexus one, with latest euroskank rom from october. Also tried disable from play store on top of this "trick". The gms updates itself every time. I wouldn't mind it so much but it doesn't ask me and i want full control over whats happening on my device but still keep access to device manager.
edit: also tried with stock cm and marketupdater frozen on both, same result.
Click to expand...
Click to collapse
Well, I'm not really an expert on custom roms, and I never heard of euroskank so I did what I always do... a Google search
What I found is a site that provides cyanogenmod roms, so if that's the case we can wait for garbb and see if he has any good news that can be useful for you too Verrocio
In my case I'm using stock rom for Galaxy Y, and Google Play Service ver. 5.0.89 hasn't got an auto update anymore since I used the trick I posted above.

halnovemila said:
Well, I'm not really an expert on custom roms, and I never heard of euroskank so I did what I always do... a Google search
What I found is a site that provides cyanogenmod roms, so if that's the case we can wait for garbb and see if he has any good news that can be useful for you too Verrocio
In my case I'm using stock rom for Galaxy Y, and Google Play Service ver. 5.0.89 hasn't got an auto update anymore since I used the trick I posted above.
Click to expand...
Click to collapse
You may want to consider going to 6.1.88 since it was the last cleanup version in its "class" to include http>>s<< api support. (from what we can tell by the changelogs)
But the jury is still out on that.

Verrocio said:
You may want to consider going to 6.1.88 since it was the last cleanup version in its "class" to include http>>s<< api support. (from what we can tell by the changelogs)
But the jury is still out on that.
Click to expand...
Click to collapse
Well, actually on devices as Galaxy Y with only 222MB storage space for system partition and only 190MB for data partition, apps footprints (use of storage space) is a concern and that's actually why I didn't like the update in first place as I stated in my 1st post here.
Moreover I don't really need much Google Play Services api(s) as I have only three apps that can't work properly if Google Play Service isn't available and I almost never use those apps so I even keep Google Play Service in "disabled" state (frozen) so it doesn't even use RAM and CPU for its many background services that I don't need.

halnovemila said:
Well, actually on devices as Galaxy Y with only 222MB storage space for system partition and only 190MB for data partition, apps footprints (use of storage space) is a concern and that's actually why I didn't like the update in first place as I stated in my 1st post here.
Moreover I don't really need much Google Play Services api(s) as I have only three apps that can't work properly if Google Play Service isn't available and I almost never use those apps so I even keep Google Play Service in "disabled" state (frozen) so it doesn't even use RAM and CPU for it's many background services that I don't need.
Click to expand...
Click to collapse
Well the next two cleanup versions after are 6.5 and 6.7(which just came out last week). Both are more memory hoggy than 6.1.88
If you don't want to install it i might suggest keeping a archived copy of it around with system panel.
My device has limited re-sources same as yours so i FEEL YOUR PAlN.
EDIT: By the way i retried the folder trick with both -1 and -2 + disabled hopefully this time it sticks!

This is a very interesting topic to me. Subscribing. I currently have all google apps disabled or uninstalled and using exchange for calendar and contacts sync. No report yet on how well it works tho. I dont like googles tactics in keeping hold of my android so I hope to see more from this thread.

ElwOOd_CbGp said:
This is a very interesting topic to me. Subscribing. I currently have all google apps disabled or uninstalled and using exchange for calendar and contacts sync. No report yet on how well it works tho. I dont like googles tactics in keeping hold of my android so I hope to see more from this thread.
Click to expand...
Click to collapse
Hi Elwood,
nice to see you are interested on the topic but I wonder what more you could find on this thread as it's just about a trick that makes to fail the automatic background updates of Google Play Service app.
I suppose the posts you'll find here will be somewhat like "Good! it works" or "too bad, it doesn't work"
My personal choice is to use the same trick for both Google Play Service and Google Play Store.
Then I usually keep Google Play Service "disabled/frozen", and I unfreeze it at the need.
I made a couple of Terminal Emulator app shortcuts so that with a single touch I can have the following commands executed
Freeze
Code:
clear ; pm disable com.google.android.gms ; echo ; echo Press any key to continue... ; read anykey ; exit
UnFreeze
Code:
clear ; pm enable com.google.android.gms ; echo ; echo Press any key to continue... ; read anykey ; exit
Those commands have to be executed as root so Terminal Emulator must have the "Initial command" preference set as
Code:
su ; exit

You'll cry after you read this
My Google Play Services app got a whopping internal memory of 114 MB!
Can i still updates apps if i do this TRICK? Not rooted tried with KingRoot but failed at least 3 times hence the name of my username/profile.
Was before but unrooted and uninstalled a few weeks earlier.

Failed Root Advocate said:
My Google Play Services app got a whopping internal memory of 114 MB!
Can i still updates apps if i do this TRICK? Not rooted tried with KingRoot but failed at least 3 times hence the name of my username/profile.
Was before but unrooted and uninstalled a few weeks earlier.
Click to expand...
Click to collapse
I'm sorry but as stated in the 1st post you need root priviliges in order to make a new folder in the internal memory /data partition.
Anyway the answer to your question is "yes"; you can still update all other apps because making the empty folder in /data/app that exactly matches the app package filename will make the update to fail only on that one app only.

Hello,
I've been searching for days to find out where GooglePlayServices apk located but failed. But i could not find it anywhere, even in /system/app/ folder and /system/pri-app/
Could you help me with this?
Thanks with best regards,

Cuong Phan said:
Hello,
I've been searching for days to find out where GooglePlayServices apk located but failed. But i could not find it anywhere, even in /system/app/ folder and /system/pri-app/
Could you help me with this?
Thanks with best regards,
Click to expand...
Click to collapse
/system/priv-app/GmsCore

ElwOOd_CbGp said:
/system/priv-app/GmsCore
Click to expand...
Click to collapse
Gratefully thank you, you save my time. In Android 6.0, it is PrebuiltGmsCore.

android.process.media keeps stopping - so annoying

Dear all,
I have been using RMN5 for a while and this is the most f**cking annoying error in this phone (or MIUI's error - i got it in my LG F180L running MIUI7 a long time ago and it happened too)
Every time i connected the phone to PC, i got this problem, I've followed the tips on XDA about disable media storage and download (force stop only in RMN5) and reboot, ok it fixed for a while, but not for long, next time i connect, it happen AGAIN.
I'm so tired of this sh*t and want to get rid of it.
Please, somebody please show me the way.
Thank you

Don't disable those apps instead go to the app settings media storage and download and clear both cache and data, hopes this will solve the problem and move this post to question and answer thread. It belongs there.

you disabled critical system app and complained why you get error messages

This is the Android Media Scanner, looking for files to index on your storage.
Some options are:
Try creating a file called '.nomedia' in every folder, and the problem should dissapear. In that case you may have a corrupted file somewhere, so try to delete the .nomedia file in some of the suspected folders to see if the problem comes back (or google something like 'android.process.media .nomedia' for a better explanation or solution.
You can also enable debuggind (via developer options), to inspect the problem with adb logcat, or to install a logcat related app from Play Store, in case the dump that should appear at the time of the error contains some hint.

rgawenda said:
This is the Android Media Scanner, looking for files to index on your storage.
Some options are:
Try creating a file called '.nomedia' in every folder, and the problem should dissapear. In that case you may have a corrupted file somewhere, so try to delete the .nomedia file in some of the suspected folders to see if the problem comes back (or google something like 'android.process.media .nomedia' for a better explanation or solution.
You can also enable debuggind (via developer options), to inspect the problem with adb logcat, or to install a logcat related app from Play Store, in case the dump that should appear at the time of the error contains some hint.
Click to expand...
Click to collapse
I'm having this issue after updating my Samsung GT-N8013 to lineageos 16.0. When you say to put a .nomedia file in every folder, do you litterally mean every single folder on both the tablet and sd card or a certain level of folder?

tcfulmer said:
I'm having this issue after updating my Samsung GT-N8013 to lineageos 16.0. When you say to put a .nomedia file in every folder, do you litterally mean every single folder on both the tablet and sd card or a certain level of folder?
Click to expand...
Click to collapse
The .nomedia flag file will stop the scan in the directory it's placed and all of the directories contained in it.

I ended up reinstalling the the OS 16.0 rom for my Samsung N8013. It took a total of running the installation 3 times but it finally became stable.