Develop Bugs

HELP NEEDED: Trying to make my own custom ROM!

Hi,
So, I'm trying to make my own custom ROM. Nothing too fancy, just a couple of simple and easy modifications to fit my needs but there's somethings I don't know how they work nor I understand them and I need some help. The only thing I was able to achieve as of yet was to remove some apps (I only tried to remove "Learn More" and QuickOffice to be honest-- but it worked). I just removed the .apk and .odex file directly from the .zip file, signed it and flashed it as usual. It worked fine.
Now, a few questions...
1) I also tried to add a few apps, just like in MoDaCo Custom ROM (MCR), to the /data/app folder instead of /system/app. Actually, I just copied all of .apk files from MCR /data/app into my own (with same folder structure of course). But they didn't get installed... I know I'm missing something and I can almost bet it has something to do with the /META-INF/com/google/android/update-script file. I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file? It has to do with the fact that my /data/apps weren't installed, right?
2) I don't use the Gmail app, just the HTC Mail one. Is it safe to delete Gmail.apk and GmailProvider.apk? I believe Gmail.apk will be fine, but the provider one, I guess it's used by the data synchronization thing. Maybe it's ok if I don't sync my mail (that's what I want actually) but maybe there's something else that I'm missing and the system will not work properly, somehow, without the GmailProvider.apk or is it safe to remove it?
3) When we first install a new ROM, starting clean, there's an introductory guide allowing to setup some things initially. One of those is the language. There is a bunch of languages, specially like English (A), English (B), English © where the letters represent other languages. I don't need all that, I only need English (Portugal). Is there anyway I could remove the ones I don't want from the ROM or I would need to dig into the Android source for that?
3.1) How about adding my own language? Is there a way I could do that to my ROM?
NOTE: Funny thing, the Android system is localized on my country's language when I first start the device but as soon as I change it to English, I cannot change it back to Portuguese cause that option doesn't exist. How come the device is in Portuguese when I first start it up after flashing a new ROM? This also happens every time I swap the SIM card, but I can never select Portuguese language by myself. Why?
4) I noticed that after flashing my custom ROM, after booting the device for the first time, the mobile network was active. This is not good... For now I have 1 month free of mobile networking but that won't last forever. I don't want to waste money every time I'm trying a new ROM. Is there anyway to disable this thing from automatically starting up connected when I first boot?
For now that's it...

1. You need to edit "META-INF/com/google/android/update-script"

That's what I said... :/ But I also said:
I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file?
Click to expand...
Click to collapse
If anyone knows anything helpful of course...

Nazgulled said:
That's what I said... :/ But I also said:
If anyone knows anything helpful of course...
Click to expand...
Click to collapse
I am also interested in a guide or walk through for making a ROM for the Hero.

1) I also tried to add a few apps, just like in MoDaCo Custom ROM (MCR), to the /data/app folder instead of /system/app. Actually, I just copied all of .apk files from MCR /data/app into my own (with same folder structure of course). But they didn't get installed... I know I'm missing something and I can almost bet it has something to do with the /META-INF/com/google/android/update-script file. I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file? It has to do with the fact that my /data/apps weren't installed, right?
Click to expand...
Click to collapse
You're missing copy_dir PACKAGE:data DATA:, so you're really not writing /data/app even if it's on the package. If you read through the update-script, you'll see that it starts by formating BOOT: and SYSTEM: and later on writes the contents of /system to SYSTEM: (with copy_dir) and writes the raw boot.img to BOOT: (with write_raw_image).
Add the line to the very end of the permissions settings (but before format and write boot, most likely, before the show_progress line). You also need to set a specific permission after writing anything to /data/app from an update-script, otherwise your market won't be able to install anything (as you'll have changed the permissions for /data/app through the update-script).
In all, your update script will look like this (assuming a regular update-script, not the mcr 2.9 one):
Code:
...
symlink toolbox SYSTEM:bin/xxxxyyyy
...
set_perm 0 0 04755 SYSTEM:bin/su
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
show_progress 0.1 10
show_progress 0.2 0
write_raw_image PACKAGE:boot.img BOOT:
show_progress 0.2 10
2) I don't use the Gmail app, just the HTC Mail one. Is it safe to delete Gmail.apk and GmailProvider.apk? I believe Gmail.apk will be fine, but the provider one, I guess it's used by the data synchronization thing. Maybe it's ok if I don't sync my mail (that's what I want actually) but maybe there's something else that I'm missing and the system will not work properly, somehow, without the GmailProvider.apk or is it safe to remove it?
Click to expand...
Click to collapse
It's safe to remove both, though the provider doesn't do much without the app, so it's also safe to leave it there.
3) When we first install a new ROM, starting clean, there's an introductory guide allowing to setup some things initially. One of those is the language. There is a bunch of languages, specially like English (A), English (B), English © where the letters represent other languages. I don't need all that, I only need English (Portugal). Is there anyway I could remove the ones I don't want from the ROM or I would need to dig into the Android source for that?
Click to expand...
Click to collapse
Hero roms take their settings from xml files in /system/customize. You'll see a bunch of xmls in there. Some are references to other xmls to be used (mns_map and cid_map), some are default settings (default.xml, COMMON.xml) and some are carrier specific settings (/CID/HTC_*.xml and /MNS/Lan_*.xml). The files are very redundant, and it's hard to tell which one is being used when, though, as a rule, you should understand that Hero loads a default from the /customize folder and then looks at mns_map and cid_map for the file to use inside /CID and /MNS. I'm not yet aware how it chooses the file, but, if you narrow down it's choices, then you can control which ones it uses and modify the settings there. If you crack open the xmls, you'll immediately see the languages defined in there (en_US, en_GB, etc), so removing languages is a matter of deleting the lang/region and adding your own (en_PL?). Again, there's so many xmls in there that it's hard to know which one to use, so either make your own and delete the rest, or edit them all. I'll give you the batch that I use in my /customize, reading through them should help you understand what's going on with those xmls.
3.1) How about adding my own language? Is there a way I could do that to my ROM?
NOTE: Funny thing, the Android system is localized on my country's language when I first start the device but as soon as I change it to English, I cannot change it back to Portuguese cause that option doesn't exist. How come the device is in Portuguese when I first start it up after flashing a new ROM? This also happens every time I swap the SIM card, but I can never select Portuguese language by myself. Why?
Click to expand...
Click to collapse
Look above, it's because the language is obtained from the carrier and used during first boot, but it's not defined in an xml, so it's unavailable, make it available by adding it to the list.
4) I noticed that after flashing my custom ROM, after booting the device for the first time, the mobile network was active. This is not good... For now I have 1 month free of mobile networking but that won't last forever. I don't want to waste money every time I'm trying a new ROM. Is there anyway to disable this thing from automatically starting up connected when I first boot?
Click to expand...
Click to collapse
This I'm not sure about. You could try emptying your /system/etc/apns-conf.xml so that it won't be able to register to any data service (just leave the <apns version="x"> </apns> tags, remove everything in between).
Good luck.

jubeh said:
You're missing copy_dir PACKAGE:data DATA:, so you're really not writing /data/app even if it's on the package. If you read through the update-script, you'll see that it starts by formating BOOT: and SYSTEM: and later on writes the contents of /system to SYSTEM: (with copy_dir) and writes the raw boot.img to BOOT: (with write_raw_image).
Add the line to the very end of the permissions settings (but before format and write boot, most likely, before the show_progress line). You also need to set a specific permission after writing anything to /data/app from an update-script, otherwise your market won't be able to install anything (as you'll have changed the permissions for /data/app through the update-script).
Click to expand...
Click to collapse
That's what I thought but I'm still missing one thing...
The generic HTC "update.zip" has format: DATA for the last line of the update script, which I suppose will delete everything becuse the copy_dir command was executed before.
Should I not use the format data option? Won't that leave traces of a previous installation if I want to start clean? What's the recommendations or suggestions?
jubeh said:
It's safe to remove both, though the provider doesn't do much without the app, so it's also safe to leave it there.
Click to expand...
Click to collapse
What will happen to the data-sync when i tries to sync the data and finds the provider for gmail is not there? Will it simply be ignored? Will there be any errors changing any of the options (specially the gmail one) on the data synchronization settings?
jubeh said:
Hero roms take their settings from xml files in /system/customize. You'll see a bunch of xmls in there. Some are references to other xmls to be used (mns_map and cid_map), some are default settings (default.xml, COMMON.xml) and some are carrier specific settings (/CID/HTC_*.xml and /MNS/Lan_*.xml). The files are very redundant, and it's hard to tell which one is being used when, though, as a rule, you should understand that Hero loads a default from the /customize folder and then looks at mns_map and cid_map for the file to use inside /CID and /MNS. I'm not yet aware how it chooses the file, but, if you narrow down it's choices, then you can control which ones it uses and modify the settings there. If you crack open the xmls, you'll immediately see the languages defined in there (en_US, en_GB, etc), so removing languages is a matter of deleting the lang/region and adding your own (en_PL?). Again, there's so many xmls in there that it's hard to know which one to use, so either make your own and delete the rest, or edit them all. I'll give you the batch that I use in my /customize, reading through them should help you understand what's going on with those xmls.
Click to expand...
Click to collapse
I'm familiar with XML files and have no problems editing them but this whole thing was a little confused specially because there are no documentation and like you said, it's hard to tell which one is which.
I know you kinda suggested that but I'm no seeing how to do it (maybe with the file you attached?) but, how could I start clean, only have default XML files and nothing else, no carrier specific stuff, no bulk, just plain default settings that I can tweak to my own liking.
Any hints on how to do that, or it's hard to tell and you don't know how to do it either?
jubeh said:
This I'm not sure about. You could try emptying your /system/etc/apns-conf.xml so that it won't be able to register to any data service (just leave the <apns version="x"> </apns> tags, remove everything in between).
Click to expand...
Click to collapse
Unfortunately, this did not work

The generic HTC "update.zip" has format: DATA for the last line of the update script, which I suppose will delete everything becuse the copy_dir command was executed before.
Should I not use the format data option? Won't that leave traces of a previous installation if I want to start clean? What's the recommendations or suggestions?
Click to expand...
Click to collapse
I've never understood why the default update script does that, but yeah, format DATA: will erase userdata available. It might not be such a bad thing if you hadn't written anything to it before. For a wipe before install, move format DATA: (and, optionally, also add format CACHE: ) right after format SYSTEM: and before you start writing anything to disk. It should give you a clean install without having to wipe from the recovery menu.
What will happen to the data-sync when i tries to sync the data and finds the provider for gmail is not there? Will it simply be ignored? Will there be any errors changing any of the options (specially the gmail one) on the data synchronization settings?
Click to expand...
Click to collapse
I've not tried it yet. Worst case scenario, you'll get the current problem we have with eclair builds (sync will just keep running and eat out battery because it's looking for a service). That's why I'd suggest leaving gmail provider there, it's just a broadcast service, but without an app to receive the data, it'll do nothing. If your concern is data usage due to sync, then you should turn off gmail sync in settings, that way only calendar and contacts are synchronized (although you'll still be using data).
I'm familiar with XML files and have no problems editing them but this whole thing was a little confused specially because there are no documentation and like you said, it's hard to tell which one is which.
I know you kinda suggested that but I'm no seeing how to do it (maybe with the file you attached?) but, how could I start clean, only have default XML files and nothing else, no carrier specific stuff, no bulk, just plain default settings that I can tweak to my own liking.
Any hints on how to do that, or it's hard to tell and you don't know how to do it either?
Click to expand...
Click to collapse
The files I attached are pretty generic. It's meant to replace your /customize folder, not add to it, with the exception being /resources inside /customize, that one I use a lighter version (no footprints) but I didn't include it in the archive for size. I have no carrier stuff there, just settings for language (I think I made it all en_US, replace or add what you need) and layouts for the different scenes, some mail app behaviors (added back the Yahoo! mail and AOL mail options, I think), some footprint, stock, and weather information. Again, it's meant to replace your /customize directory and the smaller amount of files makes it way easier to edit.
Unfortunately, this did not work
Click to expand...
Click to collapse
Blame the xmls in /customize for that. I mentioned earlier I don't exactly know how the system picks which xmls to use, I think it looks at your carrier, which is then assigned a number (for example, let's assume T-Mobile is given number 238, so xmls with 238 on them will be loaded). Problem is inside those xmls are also apn settings (which I've removed from the files I provided so that the system looks instead for apn settings in apns-conf.xml in /etc).
You could flash your system erasing the whole /customize directory (and removing all mentions of it and it's sub-directories from the update-script so that install doesn't fail). Doing that will give you ALL languages available to the rom, but you'll also lose all your pre-loaded settings (like scenes, weather locations, footprints, stocks) and HTC Setup will take an ungodly amount of time to start (about 4-5 minutes after it's done dexopting) while it builds it's own default settings. It's another route you could take, though. Experiment and you'll eventually hit what you want/need.

OK, I guess I'll have to waste some time testing all this stuff, but I suppose I'll leave that to some other day and do more basic things for now.
But just to be sure, if I use your "customization" I'm basically starting from scratch (clean, defaults) and I can use the "old" customization stuff to understand how am I supposed to configured everything and adapt the clean customization folder to my needs. Right?
jubeh said:
You could flash your system erasing the whole /customize directory (and removing all mentions of it and it's sub-directories from the update-script so that install doesn't fail). Doing that will give you ALL languages available to the rom, but you'll also lose all your pre-loaded settings (like scenes, weather locations, footprints, stocks) and HTC Setup will take an ungodly amount of time to start (about 4-5 minutes after it's done dexopting) while it builds it's own default settings. It's another route you could take, though. Experiment and you'll eventually hit what you want/need.
Click to expand...
Click to collapse
This gave me an idea, dunno if it could work...
What if I did that, erase everything inside customize, let the system build its own default settings and then access that directory (through ADB maybe?) and see what the system has created. It should be much more cleaned, without all those carrier XMLs and stuff. Then I could start from there, what do you think?
I have a few more questions about the update-script if you don't mind, but I'm very tired to write them and discuss them now. I'll leave them for another day if you don't mind coming back here and help me out some more.
Thanks for all your help so far

I should probably have mentioned, some of the stuff on those xmls is pretty straightforward (setting language, region, weather, footprints, etc), but a lot of it (setting up default shortcuts, default widgets, system settings) requires working knowledge of android, or at least of how activities are handled and how xmls come into play and how android references the different activities in a package.
On a different note, the xmls are read-only configuration files, actually, anything in system is read-only data (program data, configuration data, resource data), so if you don't have a customize directory the system won't recreate it. What the system does do during normal boot (factory) is to take those carrier-specific settings and apply them to a file(s) in /data where configuration settings are ultimately written/re-written and accessed at will. The xmls are just a database of the settings that will ultimately get picked. When you don't have a /customize directory the system writes the settings straight to -rw enabled /data partition based on things like your current carrier, carrier provided information for locale, etc.
I don't think you can take what's written to /data and use it on a default install package, well, you could, but it's much easier (and a more correct approach) to learn how those xmls in /customize work.

jubeh said:
(...)but it's much easier (and a more correct approach) to learn how those xmls in /customize work.
Click to expand...
Click to collapse
I wouldn't mind doing that if there was some kind of documentation... As it is right now, it's too confusing and there are too many XMLs to begin with...
jubeh said:
I've not tried it yet. Worst case scenario, you'll get the current problem we have with eclair builds (sync will just keep running and eat out battery because it's looking for a service). That's why I'd suggest leaving gmail provider there, it's just a broadcast service, but without an app to receive the data, it'll do nothing. If your concern is data usage due to sync, then you should turn off gmail sync in settings, that way only calendar and contacts are synchronized (although you'll still be using data).
Click to expand...
Click to collapse
But if I leave the gmail provider there and do not turn off gmail sync in settings, my e-mail will still be synced right? There's just no Gmail app to view them but they are still going to be downloaded, correct?

Ok, I finally got a change to test the Gmail/GmailProvider thing and it didn't look like there any problems like trying to sync and draining the battery. Actually, if auto-sync was enabled, there was always a message saying the mail was never synced and no matter how many times you check/uncheck it, it wouldn't sync. If auto-sync was disabled, you could press the gmail button many times but nothing happened, no sync at all (of course).
I think this behavior is fine and I can safely delete the Gmail and provider from the phone without worrying that something will go wrong
Ok, now my other questions that I said I was going to post later...
As we talked about, you suggested to add a format data and cache right after a format to system. Comparing that to the MoDaCo ROM (and other ROMs too) it's common for them to, instead of formatting data, they do this:
Code:
delete DATA:app
delete DATA:init.sh
delete DATA:local
delete DATA:dalvik-cache
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
Why are they doing this? If they are not formatting, it means there's something inside data that they want to keep. Have any idea what and why?
The other question I had about the update-script is that the following two lines are found in the original HTC update but not on MoDaCo's:
Code:
symlink dumpstate SYSTEM:bin/bugreport
symlink dumpstate SYSTEM:bin/dumpcrash
Are they relevant somehow? Should I just ignore and leave them there or remove them just as MCR did, for some reason?
There's a file named build.prop inside the system folder. I've already searched countless times and can't find any proper documentation about this file... It seems it can hold various default properties for the system and it would be nice to know what I can configure there. Maybe I could workaround problem #4 (top of the topic) in this file? Maybe some file in the Android source has comments about the possible properties on this file? Do you have any idea?
One last thing...
I've read about odex and dex files but found it a little confusing... All I know is that original HTC updates come mostly with .apk files and a corresponding .odex file. But most (if not all) custom ROM's, do not have the .odex file but have a classes.dex inside the apk. What's the real difference between the two? And why custom ROMs prefer to have the .dex file instead of the .odex, is it better somehow for some reason? If so, what should I do to "change" the .odex files into classes.dex ones and put them inside the .apk files?

As we talked about, you suggested to add a format data and cache right after a format to system. Comparing that to the MoDaCo ROM (and other ROMs too) it's common for them to, instead of formatting data, they do this:
Code:
delete DATA:app
delete DATA:init.sh
delete DATA:local
delete DATA:dalvik-cache
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
Why are they doing this? If they are not formatting, it means there's something inside data that they want to keep. Have any idea what and why?
Click to expand...
Click to collapse
Yeah, I mentioned on the earlier post that doing "format DATA:" and "format CACHE:" would be the equivalent of running "Wipe data/factory reset" from the recovery menu since you mentioned about making a build that installs fresh. Paul's (MoDaCo) roms are no-wipe, so they preserve that program data and instead format all other directories inside /data that might be problematic during an upgrade; his roms use apps2sd, so he assumes that the user's downloaded apps are in mmcblk0p2 (the memory card's ext partition), so by formatting /data/app, he's only erasing the apps he included in his update package, if the user doesn't use a2sd, well, this erases their downloaded apps (one of the reasons I dislike the current a2sd implementation, it's not suited for all situations). Then he deletes data/init.sh, which is basically a script he includes in /data that runs during a rom's boot and starts things like a2sd, compcache, linux-swap, and other custom rom's "advanced" features. DATA:local is also app storage (I believe) and functions much like in windows' /users/x/application data/local, it can be problematic if not done between builds. Last, he erases /data/dalvik-cache, this is where the temporary application code resides (the extracted classes.dex from inside an apk) for rapid execution (for apks that haven't been pre-odexed). If you wish to make your rom no-wipe, add those lines instead of "format DATA:", but I'd remove "delete DATA:app" since it's a bit problematic for people who don't use a2sd.
The other question I had about the update-script is that the following two lines are found in the original HTC update but not on MoDaCo's:
Code:
symlink dumpstate SYSTEM:bin/bugreport
symlink dumpstate SYSTEM:bin/dumpcrash
Are they relevant somehow? Should I just ignore and leave them there or remove them just as MCR did, for some reason?
Click to expand...
Click to collapse
dumpstate is a debugging tool that android uses, but I don't know exactly where or how (I think it's an HTC-specific log for their HTCLog.apk). It contains two tools, dumpcrash and bugreport, much like toolbox is a binary that contains all the tools you see symlinked to it in update-script (bin, df, cp, mv, ls, insmod, etc.). It's not a problem at all if you remove it, but it's not a problem either to leave it there. Whenever something is symlinked, though (if it's something you added, for example, busybox tools), you have to make sure that the binary or a link to it or a previous symlink to it doesn't exist, otherwise your update-script will fail (like symlinking toolbox's insmod and then busybox's insmod, or busybox's ping if a ping binary already exists in the target symlink directory (for example /system/bin)).
There's a file named build.prop inside the system folder. I've already searched countless times and can't find any proper documentation about this file... It seems it can hold various default properties for the system and it would be nice to know what I can configure there. Maybe I could workaround problem #4 (top of the topic) in this file? Maybe some file in the Android source has comments about the possible properties on this file? Do you have any idea?
Click to expand...
Click to collapse
I guess that could work if you modify network type (there's a property somewhere in there about HSXPA, I think), but like you said, there's no documentation so it calls for some experimenting.
Those properties are also overrides, so, for example, if your rom doesn't do adb root (because ro.secure is set to 1 in init.rc), you can add ro.secure=0 in build.prop to enable adb root again.
Most properties are self explanatory, and pretty much, if you can't tell what it is, you probably shouldn't change it (for example, build.fingerprint allows you to change what kind of apps Market presents to you).
I'd say, don't mess around too much with the first batch (default build properties), with the exception of build.fingerprint.
One last thing...
I've read about odex and dex files but found it a little confusing... All I know is that original HTC updates come mostly with .apk files and a corresponding .odex file. But most (if not all) custom ROM's, do not have the .odex file but have a classes.dex inside the apk. What's the real difference between the two? And why custom ROMs prefer to have the .dex file instead of the .odex, is it better somehow for some reason? If so, what should I do to "change" the .odex files into classes.dex ones and put them inside the .apk files?
Click to expand...
Click to collapse
The classes.dex files is the dalvik bytecode for the application. It contains everything inside /appnamespace/src and /appnamespace/R if you're working from the source. It's basically all the java code already compiled for the dalvik virtual machine.
Dalvik doesn't do jit, so the classes.dex have to be extracted (if they're not already) and run through the virtual machine in a process called dexopt that produces code that can be much more rapidly excecuted by the dvm (dalvik virtual machine). The extracted code is then placed in /data/dalvik-cache (mentioned earlier) as [email protected]@[email protected]. All this pre-cached code eats away at your space in your /data partition (where you store your system data, like your email, messages, browser cache, and downloaded applications) and can be quite a big chunk of space gone (framework classes specially, since all jars inside /system/framework are nothing but java code, about 20MB for all framework code).
The solution to prevent usage of /data storage on an initial build is to have the code pre-extracted. This is done at compile-time running the created system through an emulator and then creating the odex files, which is almost exactly the same as the *@classes.dex found inside /data/dalvik-cache except that it's much better aligned to the device's memory configuration and offers (marginally) better excecution, plus when the package manager first encounters the apk, it's first step is to check if an .odex of it is available, if it is, then it skips checking and dexopting a classes.dex inside it and proceeds to the next package.
Odex files increase the space in your /system partition (it was already read-only, so it's not like you're missing out on something) so those 20 MB of framework that would have been tossed into your userdata partition are now out of the way into the useless (in an available-storage sense) /system partition so you have more free space for your apps.
There's a tool that can create odex files for apks that don't have them, it's called dexopt-wrapper and it has to be run from a running phone. However, this tool won't strip the (now unnecessary) classes.dex from inside an apk, so it has to be done manually. Removing the classes.dex once you've created an odex is a good idea merely on the basis of space savings. However, once a classes.dex has been odexed, it's very hard (used to be impossible) to turn it back to a classes.dex and place it inside an apk.
Usually, us rom cooks preffer the classes.dex inside the apk because odexes create conflicts when you try to mix and match apps between builds. Since the odex files are tied to the framework where they were created, adding an apk with it's odex to another framework (even one that's only ever-so-slightly different) will be very problematic (the odex code is no longer protected or verified, so the rom won't even boot because it will fail verification for all available packages).
The tool for reverting odex files is called deodexerant by jesus freke. It requires working knowledge of how the system goes through the dexopting process so you can run it in the proper order.
Another thing to know is that even if your /system is pre-odexed, your downloaded apps will still be run through the package manager and have their classes.dex dexopted and placed in /data/dalvik-cache.
Oh, and never, ever put odexes anywhere but /system/framework or /system/app (or /data/app_s if you use a2sd). The package manager currently can't handle odexes in /data/app or /data/app_private (the other two locations where it looks for packages other than /system/app).
In the end, whether to use/have odexes comes down to personal choice (and available space). There's two hero builds, those based in 2.73.405.61 and 2.73.405.66, that have not been pre-odexed, so they're perfect for experimenting, mix/matching, etc...

jubeh said:
Yeah, I mentioned on the earlier post that doing "format DATA:" and "format CACHE:" would be the equivalent of running "Wipe data/factory reset" from the recovery menu since you mentioned about making a build that installs fresh. Paul's (MoDaCo) roms are no-wipe, so they preserve that program data and instead format all other directories inside /data that might be problematic during an upgrade; his roms use apps2sd, so he assumes that the user's downloaded apps are in mmcblk0p2 (the memory card's ext partition), so by formatting /data/app, he's only erasing the apps he included in his update package, if the user doesn't use a2sd, well, this erases their downloaded apps (one of the reasons I dislike the current a2sd implementation, it's not suited for all situations). Then he deletes data/init.sh, which is basically a script he includes in /data that runs during a rom's boot and starts things like a2sd, compcache, linux-swap, and other custom rom's "advanced" features. DATA:local is also app storage (I believe) and functions much like in windows' /users/x/application data/local, it can be problematic if not done between builds. Last, he erases /data/dalvik-cache, this is where the temporary application code resides (the extracted classes.dex from inside an apk) for rapid execution (for apks that haven't been pre-odexed). If you wish to make your rom no-wipe, add those lines instead of "format DATA:", but I'd remove "delete DATA:app" since it's a bit problematic for people who don't use a2sd.
Click to expand...
Click to collapse
So, to make it clear, I could:
Option A) Add format DATA and format CACHE to my ROM and this would make my ROM to automatically wipe everything and start clean.
Option B) I could not add anything (no formats and no deletes) and warn the users to explicitly perform a wipe before flashing.
Option C) Add delete DATA:local and DATA:dalvik-cache and have a no-wipe ROM where the apps would be preserved from the last ROM, right?
Ok, I don't use a2sd and for now have no intention in doing so. My best guess is to start with option A for now and maybe opt by using some app to backup everything and restore everything back after flashing a wipe ROM. Or I could go with option C and everything could work fine but there's a chance I could run into some issues in the long run.
jubeh said:
The classes.dex files is the dalvik bytecode for the application. It contains everything inside /appnamespace/src and /appnamespace/R if you're working from the source. It's basically all the java code already compiled for the dalvik virtual machine.
Dalvik doesn't do jit, so the classes.dex have to be extracted (if they're not already) and run through the virtual machine in a process called dexopt that produces code that can be much more rapidly excecuted by the dvm (dalvik virtual machine). The extracted code is then placed in /data/dalvik-cache (mentioned earlier) as [email protected]@[email protected]. All this pre-cached code eats away at your space in your /data partition (where you store your system data, like your email, messages, browser cache, and downloaded applications) and can be quite a big chunk of space gone (framework classes specially, since all jars inside /system/framework are nothing but java code, about 20MB for all framework code).
The solution to prevent usage of /data storage on an initial build is to have the code pre-extracted. This is done at compile-time running the created system through an emulator and then creating the odex files, which is almost exactly the same as the *@classes.dex found inside /data/dalvik-cache except that it's much better aligned to the device's memory configuration and offers (marginally) better excecution, plus when the package manager first encounters the apk, it's first step is to check if an .odex of it is available, if it is, then it skips checking and dexopting a classes.dex inside it and proceeds to the next package.
Odex files increase the space in your /system partition (it was already read-only, so it's not like you're missing out on something) so those 20 MB of framework that would have been tossed into your userdata partition are now out of the way into the useless (in an available-storage sense) /system partition so you have more free space for your apps.
There's a tool that can create odex files for apks that don't have them, it's called dexopt-wrapper and it has to be run from a running phone. However, this tool won't strip the (now unnecessary) classes.dex from inside an apk, so it has to be done manually. Removing the classes.dex once you've created an odex is a good idea merely on the basis of space savings. However, once a classes.dex has been odexed, it's very hard (used to be impossible) to turn it back to a classes.dex and place it inside an apk.
Usually, us rom cooks preffer the classes.dex inside the apk because odexes create conflicts when you try to mix and match apps between builds. Since the odex files are tied to the framework where they were created, adding an apk with it's odex to another framework (even one that's only ever-so-slightly different) will be very problematic (the odex code is no longer protected or verified, so the rom won't even boot because it will fail verification for all available packages).
The tool for reverting odex files is called deodexerant by jesus freke. It requires working knowledge of how the system goes through the dexopting process so you can run it in the proper order.
Another thing to know is that even if your /system is pre-odexed, your downloaded apps will still be run through the package manager and have their classes.dex dexopted and placed in /data/dalvik-cache.
Oh, and never, ever put odexes anywhere but /system/framework or /system/app (or /data/app_s if you use a2sd). The package manager currently can't handle odexes in /data/app or /data/app_private (the other two locations where it looks for packages other than /system/app).
In the end, whether to use/have odexes comes down to personal choice (and available space). There's two hero builds, those based in 2.73.405.61 and 2.73.405.66, that have not been pre-odexed, so they're perfect for experimenting, mix/matching, etc...
Click to expand...
Click to collapse
Well, I'm currently basing my ROM on 2.73.405.38 as the .66 one has "test" on the name and it feels like a beta version, which I'd rather not use. The .38 version uses .odex files for most of the packages (only a few has classes.dex inside) and I guess I'll leave them like that. If future ROMs, like that .66 one, comes with classes.dex vs .odex files, well, then I'll just leave it like that...
That looks like a very important thing in Android apps and at the same time something that I should be very knowledgeable about before messing with it, so I'll just stay put
Thanks once again for taking your time with such complete answers

So, to make it clear, I could:
Option A) Add format DATA and format CACHE to my ROM and this would make my ROM to automatically wipe everything and start clean.
Option B) I could not add anything (no formats and no deletes) and warn the users to explicitly perform a wipe before flashing.
Option C) Add delete DATA:local and DATA:dalvik-cache and have a no-wipe ROM where the apps would be preserved from the last ROM, right?
Ok, I don't use a2sd and for now have no intention in doing so. My best guess is to start with option A for now and maybe opt by using some app to backup everything and restore everything back after flashing a wipe ROM. Or I could go with option C and everything could work fine but there's a chance I could run into some issues in the long run.
Click to expand...
Click to collapse
Sounds like you got it.
Well, I'm currently basing my ROM on 2.73.405.38 as the .66 one has "test" on the name and it feels like a beta version, which I'd rather not use. The .38 version uses .odex files for most of the packages (only a few has classes.dex inside) and I guess I'll leave them like that. If future ROMs, like that .66 one, comes with classes.dex vs .odex files, well, then I'll just leave it like that...
That looks like a very important thing in Android apps and at the same time something that I should be very knowledgeable about before messing with it, so I'll just stay put
Click to expand...
Click to collapse
Test only refers to the build being signed with test-keys instead of release-keys. In my testing, it makes no difference whatsoever for a rooted user, and there's many theories floating around about signatures, I haven't researched signatures a whole lot because test-keys have worked fine enough for me, so I'm not so sure, but they seem to be working fine for all.
The reason those releases are un-odexed (and signed with test-keys) is because of the problem with odexes I mentioned earlier. When HTC is testing their rom packages and they need to make, say, a framework change, all the apps with odexes would fail as the framework has changed, so, for testing purposes, it's best to leave the apks and jars in the build full (with classes.dex inside). Once you know your build is solid and you won't be changing much, it's safer to odex the whole thing (sort of like me, I'm waiting for the final release of eclair for ADP1 so I can put out a Dream rom odexed and fully working).
Thanks once again for taking your time with such complete answers
Click to expand...
Click to collapse
No problem, I get bored at work

So, version .66 is not beta or anything? I know MoDaCo has his latest ROM based on that and you're saying they are only different in the signing key, still, it's somewhat a "leaked" build, not quite "official".
But hey, if others are using it without any issues, maybe I should be using it too

On second thought...
1) I can't find a pre-rooted .66 version and I don't know how to root it :/
2) There's lots of files in system/xbin that I don't see referenced anywhere (not even in the update-script) and I don't know what should I do with them.
I'm finding it much more confusing to base my ROM on .66 than on .38...

It is a beta, of sorts, but between it and the final shipping product there's not much difference. It's just much, much easier to work with it.
1.) The simplest form of root is to change ro.sercure to 0 in default.prop in the boot ramdisk. Adding a su binary/Superuser.apk is optional, those you can pull from one of paul's builds.
2.) The system/xbin is added to the boot classpath in the boot ramdisk in init.rc. The only mention of it in in update-script is the permissions for the whole folder and then just a couple permissions for some tools, not all of them.

Understanding the difference.

I'm running 2.1 rooted. I have both 2.2 odex and deodexed versions sitting on my laptop. Not sure which to use am I'm unfamiliar with what the difference between the 2 are.
And when I update to Froyo do I need to flash a custom rom or can I use as comes?
Sent from my Epic 4G - Rooted for My Pleasure!!

I think the odexed and dedexed have to do with the stuff being signed or not, but it doesn't really matter if that's right, because I don't know what the real world diff is. I see that brought up the most in terms of themes not working on one or the other.
When an update comes peole will work that into their stuff. So, whatever rom you choose will probably just get updated itself within a day or two, and then you just flash it more or less the same as any other update to midnight, bonzai, etc.

Searching google turned this up in less than 30 seconds. Since finding the answer was so quicky and easy, I'm just going to copy here instead of taking the time to explain it myself.
WHAT IS AN ODEX FILE?
In Android file system, applications come in packages with the extension .apk. These application packages, or APKs contain certain .odex files whose supposed function is to save space. These ‘odex’ files are actually collections of parts of an application that are optimized before booting. Doing so speeds up the boot process, as it preloads part of an application. On the other hand, it also makes hacking those applications difficult because a part of the coding has already been extracted to another location before execution.
THEN COMES DEODEX
Deodexing is basically repackaging of these APKs in a certain way, such that they are reassembled into classes.dex files. By doing that, all pieces of an application package are put together back in one place, thus eliminating the worry of a modified APK conflicting with some separate odexed parts.
In summary, Deodexed ROMs (or APKs) have all their application packages put back together in one place, allowing for easy modification such as theming. Since no pieces of code are coming from any external location, custom ROMs or APKs are always deodexed to ensure integrity.
HOW THIS WORKS
For the more geeky amongst us, Android OS uses a Java-based virtual machine for running applications, called the Dalvik Virtual Machine. A deodexed, or .dex file contains the cache used by this virtual machine (referred to as Dalvik-cache) for a program, and it is stored inside the APK. An .odex file, on the other hand, is an optimized version of this same .dex file that is stored next to the APK as opposed to inside it. Android applies this technique by default to all the system applications.
Now, when an Android-based system is booting, the davlik cache for the Davlik VM is built using these .odex files, allowing the OS to learn in advance what applications will be loaded, and thus speeds up the booting process.
By deodexing these APKs, a developer actually puts the .odex files back inside their respective APK packages. Since all code is now contained within the APK itself, it becomes possible to modify any application package without conflicting with the operating system’s execution environment.
ADVANTAGES & DISADVANTAGES
The advantage of deodexing is in modification possibilities. This is most widely used in custom ROMs and themes. A developer building a custom ROM would almost always choose to deodex the ROM package first, since that would not only allow him to modify various APKs, but also leave room for post-install theming.
On the other hand, since the .odex files were supposed to quickly build the dalvik cache, removing them would mean longer initial boot times. However, this is true only for the first ever boot after deodexing, since the cache would still get built over time as applications are used. Longer boot times may only be seen again if the dalvik cache is wiped for some reason.
For a casual user, the main implication is in theming possibilities. Themes for android come in APKs too, and if you want to modify any of those, you should always choose a dedoexed custom ROM.
Was this article helpful? If you are confused with some other terms and want us to help explain them, please let us know in the comments.
Click to expand...
Click to collapse
In the future, searching will answer your basic questions a whole lot faster than posting a new thread.

Kernel With Separate Cache Partition

Hello everyone,
I've compiled a kernel which separates the NAND memory into 3 partitions (well more then that but anyway). These partitions are system, data and cache like a native Android phone (which also has recovery but we don't need that). Previously cache was linked to a directory on the data partition which was constantly becoming corrupt causing errors on start-up. Often clearing dalvik-cache would allow Android to boot but data was lost. There's nothing to say that this will solve the problem but i'm giving it a go so thought I would post it up for you to try also.
The cache partition is 20mb which means the data partition is only 90mb or so. If you like a million apps on your phone either put some on your SD card or don't use it
I haven't completely incorporated this partition into the initrd which means that currently it will only mount if data is also on NAND but this can be changed easily enough, but for testing purposes I want everything on the NAND chip anyway.
Downloads from market on all versions of Android, unless the init.rc directory is changed in Gingerbread, all go to the cache partition and are deleted on restart. What this means is that you can't download and install an app larger then 20mb (which isn't recommended anyway with the small amount of data space available). This is a drawback but for those that use data on SD it should be possible in the future to utilise more of the NAND memory for cache so you can install whatever you want. I don't know of many apps larger then 20mb anyway though.
I have included a modified version of ATools in the zip. The standard version will overwrite the partition values and the kernel won't work properly so if you need to modify the kernel use the version included. It is also important that you DO NOT change the system or data partition sizes in ATools as this hasn't been modified to account for the cache partition so will mess it all up.
I have also included an androidupdate.tar modules update for the kernel in case your wifi doesn't work after flashing the kernel.
I think that's it for now, let me know how it goes!
EDIT::
One last thing, you will need to wipe your DATA partition to use this kernel because some of your data may be positioned where the cache partition is now, so if you load this kernel and android is heavily corrupt you know what you have to do!
To check the kernel is working correctly you can type "df -h" into a terminal and it should show all of the current partitions including one called cache on mtdblock4.
Should now work with polaris, kaiser and vogue. There is a VOGUEIMG.NBH included for those who have trouble with ATools. It is completely untested on Vogue so sorry if it doesn't work.

EDIT
I have re-uploaded a modified kernel because there was some issues with data with the last one. If you have already applied the last one flash the new one and apply the module update again

testing will leave feedback

Ok, using your Scoot CyanogenMod 6.1.2 (RLS6) [16.03.2011], overclock via atools+roguetools to 529, gpu oc, battery units tweaked to suit my battery.
No SIM card yet, don't have one to put there, sync with google to get contacts.
Installed GoLauncher. Enabled JIT and Keep Launcher in memory.
Once in a while, Android is killing all apps, launcher included... strange...
Nevermind that, it was Cyanogen and the LongPress BackKey. LongPress on our Kaisers during lag can become short press

First couple of days of heavy use I've had in ages with no corruption. First day of my new job today so it's been on Google Navigation for 3 hours and used extensively during the day. Had to do a hard reset at one point and everything was fine. had a few soft reset's too. Don't get me wrong, i'd be surprised if this is a fix, but it's been a while since I've used my Kaiser this much and not had to wipe my data partition!

Update: CM is a bit slow, trying:
VaniljEclair RLS11 - A fast & stable CM 5.0.8 for Vogue/Kaiser/Polaris [2010-08-19]
And everyting seems good. I've pushed it a little, several normal reboots.
Had to reflash system (didn't reflash data). No errors, FC's, whatevar. Everything works, yet didn't try to make/receive calls. Its 1am here...
Now this particular version, has a god damn bug that is get the best of me. The Power Button and the red button won't make the phone go to standby. If i long press, i get the normal reboot/power off/airplane etc menu, on both keys. But single press is ignore... Any idea ?

scooter1556 , i have a question. It is not the purpose of the thread or kernel, but how difficult would it be to activate scalling of the cpu ?
Right now, the two 3 biggest issues with Android on the Kaiser are, IMHO, Data corruption, Battery life, performance.
Please share your thoughts

Using this kernel with Valentine 1.02. GPU overclocked and CPU @ 520mhz. Super fast and for the first time... no data corruption! Verry happy so far!

daedric said:
scooter1556 , i have a question. It is not the purpose of the thread or kernel, but how difficult would it be to activate scalling of the cpu ?
Right now, the two 3 biggest issues with Android on the Kaiser are, IMHO, Data corruption, Battery life, performance.
Please share your thoughts
Click to expand...
Click to collapse
Well in theory not too hard, there is a feature built into the linux kernel called cpufreq which is currently disabled in our kernel because our cpu and chipset code doesn't support it but krazy-killa has been working on getting it working in his kernels with a little success as far as i'm aware but i'm not that sure. I haven't personally looked into it but if I get some spare time I definitely will. I've started a new job now which i'm sure will keep me quite busy but I still intend to so some work on the Kaiser when I get time

Best of luck on your new job! I wish i had enough knowledge to continue the kernel development, but i'm quite a ignorant
Still, thank you, till now it's one of the best kernels i've used.

Hi Scoot!
I love your kernel. No data corruption so far (two days). did a few stylus resets and stuff.. Magical =D. Now IMO we only need better sleeping and autofocus to call kaiser a fully android device! Please Keep up the awesome job, scoot! thanx, danke, sposeeba, bedankt, aciu, dzenkuja!

Nice job scooter, I've been using Scoot CyanogenMod 7.0.2 RLS2 [28/04/11] @ 500MHz, and it seems fine so far.
I tried Fat Free Froyo before CyanogenMod, which also worked great for the few hours it was running. I'm not sure which one to stick with though.
Keep up the good work. I wish I could help with the development, but I have yet to learn programming (been ten years now ).

This kernel looks pretty stable, but just a few quick questions:
1) What filesystem is used for the cache? If YAFFS, could it get corrupted? If Yes, what would happend then?
2) Is it really needed to have 20MB for the cache? ATM it seems to be used only by a like 1-2MB on the cache partition, would not it be enough to have for example 10MB for the cache leaving more space for apps?
3) What is the difference using apps in the system and data partitions? If I would run off the space on the data partition, can I put some apps in the system apps folder to save the space without wandering about how it works?
Thanx for your great work, finally it seems to be fully usable for me and if you and Krazy-Killa would be able to work together to manage the power consumption, then it would be amazing, because I am going to get new phone (running some new extensive business and I can not afford to be not reachable when something happends on Kaiser) and give this lovely piece of HW to my son... And I really do not want him to have a WM machine But Android needs to be stable for him to use Well... he will get it for his 10th birthday at 6.9. so we still have some time to tweak it a bit more

if this works then it would be magical, your CM builds were so fast and awesome and if there's no data loss then... <3

Made 2 mistakes when building kernel (selected tilt instead of normal and forgot to OC to 480MHz), and running kingshui's 12/15/2010 build 2.2.1. seems ROCK solid and fast! I applaud your use of ramzswap for the extra 20mb, bravo!
I'm just hoping I can go back and rebuild the kernel with the correct settings without messing up the system/data/cache partitioning.

PoXFreak said:
Made 2 mistakes when building kernel (selected tilt instead of normal and forgot to OC to 480MHz), and running kingshui's 12/15/2010 build 2.2.1. seems ROCK solid and fast! I applaud your use of ramzswap for the extra 20mb, bravo!
I'm just hoping I can go back and rebuild the kernel with the correct settings without messing up the system/data/cache partitioning.
Click to expand...
Click to collapse
They aren't mistakes, those are the settings I need for my Kaiser. I included a modified version of ATools for you all to modify the kernel for your devices as I stated in the first post

a.s.j said:
1) What filesystem is used for the cache? If YAFFS, could it get corrupted? If Yes, what would happend then?
2) Is it really needed to have 20MB for the cache? ATM it seems to be used only by a like 1-2MB on the cache partition, would not it be enough to have for example 10MB for the cache leaving more space for apps?
3) What is the difference using apps in the system and data partitions? If I would run off the space on the data partition, can I put some apps in the system apps folder to save the space without wandering about how it works?
Click to expand...
Click to collapse
1) The cache partition is still using Yaffs2 because it is the easiest filesystem to configure at the minute, but the idea of making it a separate partition like on a native android device is that it can be changed to another filesystem in the initrd relatively easily. If it gets corrupted it will do what it used to I guess, but I haven't had any corruption as of yet so it's difficult to say. Hopefully if the cache partition get's corrupted it shouldn't affect data so you should just be able to wipe and format dalvik-cache in the install menu.
2) I made it 20mb because most android devices have 30mb or bigger and this is mainly because market app downloaded get downloaded here before being installed, so if you installed google maps for instance it would use up 6mb, plus the standard 1-2mb normally being used, but if you only had a 10mb cache for instance, you would only be able to install apps less then 8mb which is a little limiting really. I don't really care much about installing lots of apps in data or on the phone at all for that matter so having 95mb for data is more then enough for me.
3) You can put apps in the system partition in /system/app. The system partition is read only so all data for the app still goes to the data partition. You can still save a little data partition space though if you need it. Most installs of android are 80-95mb which should leave you 5mb or so to play with. Some all language builds pretty much use up all of the partition though so you need to look and see what space you have to play with before putting apps on the system partition. If you overload it you will probably end up corrupting it and having to reinstall. But to answer your last question, the system partition works in the same way as the data partition apart fro the fact that it is read-only and data is read/write.

I've modified my Kernel to use the NAND mtdblock3 (/data for everyone else) as the cache partition, since my NAND is pretty much shot to begin with, and have system and data on SD Card. So far it's made downloading Market Apps twice as fast, loading webpages pretty fast, and basically anything else that the cache is used for.

scooter1556 said:
They aren't mistakes, those are the settings I need for my Kaiser. I included a modified version of ATools for you all to modify the kernel for your devices as I stated in the first post
Click to expand...
Click to collapse
Scoot:
I understand that the settings built into the .nbh are for your device. What I was meaning was I needed to modify the CPU clock up to 480MHz, and for some odd reason my device swaps the "@" and "!" buttons if I set it to "tilt", among other buttons being set wrong.
Also, I am used to having the left and right softkeys set as "vol up" and "vol down" respectively. Is this something I need to make an androidupdate for, or do I have to write a new NBH?

PoXFreak said:
Scoot:
I understand that the settings built into the .nbh are for your device. What I was meaning was I needed to modify the CPU clock up to 480MHz, and for some odd reason my device swaps the "@" and "!" buttons if I set it to "tilt", among other buttons being set wrong.
Also, I am used to having the left and right softkeys set as "vol up" and "vol down" respectively. Is this something I need to make an androidupdate for, or do I have to write a new NBH?
Click to expand...
Click to collapse
Can you not do all of this using the ATools i provided in the attachment in the first post? You should be able to load the nbh into ATools and then set the CPU speed, change the keyboard type and remap the buttons and then save it before flashing. Just don't change the partition sizes as this will mess up the cache partition I added until I make it more permanent.

[Q] dalvik cache (in laymans terms)

hi! i'm not a complete noob or anything. but i do like to understand the things i know.
please dont start insulting me and shout "just google it" at me, as i have and it all gets my head in a spin. and to make matters worse, i have a medical condition that hinders my understanding of "big words". lol.
but quite simply, in an easy to understand way, could someone please explain to me what the dalvik cache is, what it does, why it's important, the need (or lack of need) to clean it, and the difference's between it and other cache types?
any info would be appreciated. i would love to help support dev's more, and understand what phones do (os wise) and how to tweak them. and this is just one of the questions that bugs me a fair bit, for some reason
thanks in advance

You have 5 apps and yet for some reason you have more used memory than you should
Sent from my R800i using XDA App

Got this from another fourm here
Dalvik cache is a program cache area for the program dalvik. Dalvik is a java based virtual machine that is the bases for running your programs (the ones that have the .apk extension). In order to make access times faster (because there's not JIT (just in time) compiler installed by default), the dalvik-cache is the result of dalvik doing a optimization of the running program. Sounds confusing. It's similar to the prefetch files in Windows.
Click to expand...
Click to collapse
or even this from here
When Android starts up, the DalvikVM looks thru all of your applications (.apk files) and frameworks, and builds a tree of dependencies. It uses this dependency tree to optimize the bytecode for every application and stores it in the Dalvik cache. The applications are then run using the optimized bytecode.
Click to expand...
Click to collapse
or read about it fully on wiki http://en.wikipedia.org/wiki/Dalvik_virtual_machine

Mozza2k11 said:
Got this from another fourm here
or even this from here
or read about it fully on wiki http://en.wikipedia.org/wiki/Dalvik_virtual_machine
Click to expand...
Click to collapse
Yep the 2nd quote from mozza sums up the dalvik cache. The dalvik vm builds a tree of dependencies on boot, this is used to run applications with optimal bytecode

so it's basically a storage space that tells the phone the best way of running an app through the OS you have installed? take it it takes the hardware into account too? i'd like to understand the prefetch thing, but that confuses me too

[Q] Tiny's CM10.1 Nightlies and bcache

Hiya Folks,
Long time listener, first time poster.
I have a few questions:
a) Tiny, thanks a lot for your CM10.1+evervolv device tree builds, these are awesome. One question for you: Is there any chance you might be willing to post a short changelog when you post new builds? I understand that's a bit of an administrative headache, but it would be really nice to be able to get a sense, at least on a coarse level, as to what's new in each build... (ie: merged new display driver, picked up CM10.1 changes from x date to x date, explicitly tried to fix x)
b) With the initial jump to these builds, I took a full backup and started over. I've found that for some of the nightlies, I can just wipe caches and install the new OS image and everything seems to work. This, of course, is largely dependent on binary compatibility between sqlite versions, etc and high level compatibility between database schemas. Is there any way you could perhaps give a sense as to whether or not anything has changed that would definitely require a full wipe versus this build contains stuff that probably won't make a difference with each build along with a big disclaimer absconding any responsibility if things go wrong?
c) The camcorder is recording at a very low framerate, this is a known issue. Is there a fix coming down the line any time soon? I'm proficient with both userland and kernel hacking, and know my way around Android's internals pretty well. Do you have any information on what's causing this bug. I might be able to hack around a bit with it, but some sense of the history and the components involved would be helpful. (I have a sneaking suspicion this has to do with the video drivers and the sync to vsync stuff that is new in 4.1)
d) I've long been fascinated by the strange situation involving the scantily documented /datadata partition. My understanding is that this is some kind of faster chunk of hardware flash that is used by application databases with the idea being that it speeds them up. I believe ext4all mod moves the stuff onto the sdcard or regular internal storage and then symlinks. Has anyone considered playing with the Linux bcache module that allows faster storage block devices to serve as a cache over slower storage block devices in the storage hierarchy? I've been thinking about hacking on this myself, but I wanted to see if anyone has considered it and if it would cause problems that I don't see now (I could see it causing a problem with the current way nandroid backups are taken, or something along those lines, for example).
In any event, thanks for listening!

I'll try to summarize a, b, and c.
A. No official change log. I don't have time to write one. I try to post significant changes when I can.
B. I almost never wipe with cm. In theory you can even upgrade from cm10 to cm10.1. Wiping is overrated and its usually only useful to troubleshoot really weird issues.
C. The camcorder lag is likely a software or codec issue. Technically its not a bug but more of optimization issue. Preview glitching I'd call a bug.
Sent from my Galaxy Nexus using Tapatalk 2

a-dub said:
Hiya Folks,
Long time listener, first time poster.
I have a few questions:
a) Tiny, thanks a lot for your CM10.1+evervolv device tree builds, these are awesome. One question for you: Is there any chance you might be willing to post a short changelog when you post new builds? I understand that's a bit of an administrative headache, but it would be really nice to be able to get a sense, at least on a coarse level, as to what's new in each build... (ie: merged new display driver, picked up CM10.1 changes from x date to x date, explicitly tried to fix x)
b) With the initial jump to these builds, I took a full backup and started over. I've found that for some of the nightlies, I can just wipe caches and install the new OS image and everything seems to work. This, of course, is largely dependent on binary compatibility between sqlite versions, etc and high level compatibility between database schemas. Is there any way you could perhaps give a sense as to whether or not anything has changed that would definitely require a full wipe versus this build contains stuff that probably won't make a difference with each build along with a big disclaimer absconding any responsibility if things go wrong?
c) The camcorder is recording at a very low framerate, this is a known issue. Is there a fix coming down the line any time soon? I'm proficient with both userland and kernel hacking, and know my way around Android's internals pretty well. Do you have any information on what's causing this bug. I might be able to hack around a bit with it, but some sense of the history and the components involved would be helpful. (I have a sneaking suspicion this has to do with the video drivers and the sync to vsync stuff that is new in 4.1)
d) I've long been fascinated by the strange situation involving the scantily documented /datadata partition. My understanding is that this is some kind of faster chunk of hardware flash that is used by application databases with the idea being that it speeds them up. I believe ext4all mod moves the stuff onto the sdcard or regular internal storage and then symlinks. Has anyone considered playing with the Linux bcache module that allows faster storage block devices to serve as a cache over slower storage block devices in the storage hierarchy? I've been thinking about hacking on this myself, but I wanted to see if anyone has considered it and if it would cause problems that I don't see now (I could see it causing a problem with the current way nandroid backups are taken, or something along those lines, for example).
In any event, thanks for listening!
Click to expand...
Click to collapse
d) The /datadata partion is just a normal partition but is formated yaffs2, which is a little faster than ext 3 which is what /data and /cache normally use out of the box. However this partition is small, and fills up quickly. The ext4 mods do 2 things, they convert /data and /cache from ext3 to the slightly faster ext4. It also moves all of the files from the /datadata partition and instead placed the files on the /data partition in a data folder. So it still ends up having the path /data/data just as it did before the mod. This leaves the yaffs2 /datadata partion empty and unused.
I havent heard of bcache, so im not sure on that question. If you could get bcache to use the old /data/data partition as cache for the other partitions it might provide better performance. It shouldn't cause backup issues as /datadat is already backed up by nandroid, so all required filed should get backed up. If you get it working, i would definitley check it out.