xdandroid what are .ext2 extentions for? can you delete old one?

xdandroid what are .ext2 extentions for? can you delete old one? - Touch Pro, Fuze Android Development

hi, ive been downloading xdandroid for diam500 for awhile already and i always wondered what are some of these files that constantly get updated for?
From wat i think i know:
rootfs - file is for the system core files?
ext - no idea (i have a bunch of them in my root folder. can i delete old ones?)
data.img - where my data is stored
what else am i missing?
i would imagine it would be totally confusing for a newbie to figure out if they need this or that since things get updated independently. i hope others find this post useful.

We do have a dedicated thread for asking questions you know....
rootfs - Contains file we use to copy over to system.ext2 during boot. Such as wifi, bluetooth, apns-lists and configurations.
ext - Is a filesystem. As sqsh did not provide us with read/write capabilities we have moved to the ext2 fs.
data.img - Pretty much as you summed up. Its where we store data. Think of it as your sd i guess. System stores information/data there along with your downloaded apps.
Might aswell explain the others
zImage - Is the kernel which we use it to control hardware and how it functions.
initrd.gz - dont know much about this one myself as i've never touched it

Related

HELP NEEDED: Trying to make my own custom ROM!

Hi,
So, I'm trying to make my own custom ROM. Nothing too fancy, just a couple of simple and easy modifications to fit my needs but there's somethings I don't know how they work nor I understand them and I need some help. The only thing I was able to achieve as of yet was to remove some apps (I only tried to remove "Learn More" and QuickOffice to be honest-- but it worked). I just removed the .apk and .odex file directly from the .zip file, signed it and flashed it as usual. It worked fine.
Now, a few questions...
1) I also tried to add a few apps, just like in MoDaCo Custom ROM (MCR), to the /data/app folder instead of /system/app. Actually, I just copied all of .apk files from MCR /data/app into my own (with same folder structure of course). But they didn't get installed... I know I'm missing something and I can almost bet it has something to do with the /META-INF/com/google/android/update-script file. I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file? It has to do with the fact that my /data/apps weren't installed, right?
2) I don't use the Gmail app, just the HTC Mail one. Is it safe to delete Gmail.apk and GmailProvider.apk? I believe Gmail.apk will be fine, but the provider one, I guess it's used by the data synchronization thing. Maybe it's ok if I don't sync my mail (that's what I want actually) but maybe there's something else that I'm missing and the system will not work properly, somehow, without the GmailProvider.apk or is it safe to remove it?
3) When we first install a new ROM, starting clean, there's an introductory guide allowing to setup some things initially. One of those is the language. There is a bunch of languages, specially like English (A), English (B), English © where the letters represent other languages. I don't need all that, I only need English (Portugal). Is there anyway I could remove the ones I don't want from the ROM or I would need to dig into the Android source for that?
3.1) How about adding my own language? Is there a way I could do that to my ROM?
NOTE: Funny thing, the Android system is localized on my country's language when I first start the device but as soon as I change it to English, I cannot change it back to Portuguese cause that option doesn't exist. How come the device is in Portuguese when I first start it up after flashing a new ROM? This also happens every time I swap the SIM card, but I can never select Portuguese language by myself. Why?
4) I noticed that after flashing my custom ROM, after booting the device for the first time, the mobile network was active. This is not good... For now I have 1 month free of mobile networking but that won't last forever. I don't want to waste money every time I'm trying a new ROM. Is there anyway to disable this thing from automatically starting up connected when I first boot?
For now that's it...

1. You need to edit "META-INF/com/google/android/update-script"

That's what I said... :/ But I also said:
I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file?
Click to expand...
Click to collapse
If anyone knows anything helpful of course...

Nazgulled said:
That's what I said... :/ But I also said:
If anyone knows anything helpful of course...
Click to expand...
Click to collapse
I am also interested in a guide or walk through for making a ROM for the Hero.

1) I also tried to add a few apps, just like in MoDaCo Custom ROM (MCR), to the /data/app folder instead of /system/app. Actually, I just copied all of .apk files from MCR /data/app into my own (with same folder structure of course). But they didn't get installed... I know I'm missing something and I can almost bet it has something to do with the /META-INF/com/google/android/update-script file. I tried countless times to search for some guide, how-to or just a piece of documentation about this file but I can't find anything. I don't want to just change it and see if it works and then screw things up. Can anyone give me some hints about understanding this file? It has to do with the fact that my /data/apps weren't installed, right?
Click to expand...
Click to collapse
You're missing copy_dir PACKAGE:data DATA:, so you're really not writing /data/app even if it's on the package. If you read through the update-script, you'll see that it starts by formating BOOT: and SYSTEM: and later on writes the contents of /system to SYSTEM: (with copy_dir) and writes the raw boot.img to BOOT: (with write_raw_image).
Add the line to the very end of the permissions settings (but before format and write boot, most likely, before the show_progress line). You also need to set a specific permission after writing anything to /data/app from an update-script, otherwise your market won't be able to install anything (as you'll have changed the permissions for /data/app through the update-script).
In all, your update script will look like this (assuming a regular update-script, not the mcr 2.9 one):
Code:
...
symlink toolbox SYSTEM:bin/xxxxyyyy
...
set_perm 0 0 04755 SYSTEM:bin/su
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
show_progress 0.1 10
show_progress 0.2 0
write_raw_image PACKAGE:boot.img BOOT:
show_progress 0.2 10
2) I don't use the Gmail app, just the HTC Mail one. Is it safe to delete Gmail.apk and GmailProvider.apk? I believe Gmail.apk will be fine, but the provider one, I guess it's used by the data synchronization thing. Maybe it's ok if I don't sync my mail (that's what I want actually) but maybe there's something else that I'm missing and the system will not work properly, somehow, without the GmailProvider.apk or is it safe to remove it?
Click to expand...
Click to collapse
It's safe to remove both, though the provider doesn't do much without the app, so it's also safe to leave it there.
3) When we first install a new ROM, starting clean, there's an introductory guide allowing to setup some things initially. One of those is the language. There is a bunch of languages, specially like English (A), English (B), English © where the letters represent other languages. I don't need all that, I only need English (Portugal). Is there anyway I could remove the ones I don't want from the ROM or I would need to dig into the Android source for that?
Click to expand...
Click to collapse
Hero roms take their settings from xml files in /system/customize. You'll see a bunch of xmls in there. Some are references to other xmls to be used (mns_map and cid_map), some are default settings (default.xml, COMMON.xml) and some are carrier specific settings (/CID/HTC_*.xml and /MNS/Lan_*.xml). The files are very redundant, and it's hard to tell which one is being used when, though, as a rule, you should understand that Hero loads a default from the /customize folder and then looks at mns_map and cid_map for the file to use inside /CID and /MNS. I'm not yet aware how it chooses the file, but, if you narrow down it's choices, then you can control which ones it uses and modify the settings there. If you crack open the xmls, you'll immediately see the languages defined in there (en_US, en_GB, etc), so removing languages is a matter of deleting the lang/region and adding your own (en_PL?). Again, there's so many xmls in there that it's hard to know which one to use, so either make your own and delete the rest, or edit them all. I'll give you the batch that I use in my /customize, reading through them should help you understand what's going on with those xmls.
3.1) How about adding my own language? Is there a way I could do that to my ROM?
NOTE: Funny thing, the Android system is localized on my country's language when I first start the device but as soon as I change it to English, I cannot change it back to Portuguese cause that option doesn't exist. How come the device is in Portuguese when I first start it up after flashing a new ROM? This also happens every time I swap the SIM card, but I can never select Portuguese language by myself. Why?
Click to expand...
Click to collapse
Look above, it's because the language is obtained from the carrier and used during first boot, but it's not defined in an xml, so it's unavailable, make it available by adding it to the list.
4) I noticed that after flashing my custom ROM, after booting the device for the first time, the mobile network was active. This is not good... For now I have 1 month free of mobile networking but that won't last forever. I don't want to waste money every time I'm trying a new ROM. Is there anyway to disable this thing from automatically starting up connected when I first boot?
Click to expand...
Click to collapse
This I'm not sure about. You could try emptying your /system/etc/apns-conf.xml so that it won't be able to register to any data service (just leave the <apns version="x"> </apns> tags, remove everything in between).
Good luck.

jubeh said:
You're missing copy_dir PACKAGE:data DATA:, so you're really not writing /data/app even if it's on the package. If you read through the update-script, you'll see that it starts by formating BOOT: and SYSTEM: and later on writes the contents of /system to SYSTEM: (with copy_dir) and writes the raw boot.img to BOOT: (with write_raw_image).
Add the line to the very end of the permissions settings (but before format and write boot, most likely, before the show_progress line). You also need to set a specific permission after writing anything to /data/app from an update-script, otherwise your market won't be able to install anything (as you'll have changed the permissions for /data/app through the update-script).
Click to expand...
Click to collapse
That's what I thought but I'm still missing one thing...
The generic HTC "update.zip" has format: DATA for the last line of the update script, which I suppose will delete everything becuse the copy_dir command was executed before.
Should I not use the format data option? Won't that leave traces of a previous installation if I want to start clean? What's the recommendations or suggestions?
jubeh said:
It's safe to remove both, though the provider doesn't do much without the app, so it's also safe to leave it there.
Click to expand...
Click to collapse
What will happen to the data-sync when i tries to sync the data and finds the provider for gmail is not there? Will it simply be ignored? Will there be any errors changing any of the options (specially the gmail one) on the data synchronization settings?
jubeh said:
Hero roms take their settings from xml files in /system/customize. You'll see a bunch of xmls in there. Some are references to other xmls to be used (mns_map and cid_map), some are default settings (default.xml, COMMON.xml) and some are carrier specific settings (/CID/HTC_*.xml and /MNS/Lan_*.xml). The files are very redundant, and it's hard to tell which one is being used when, though, as a rule, you should understand that Hero loads a default from the /customize folder and then looks at mns_map and cid_map for the file to use inside /CID and /MNS. I'm not yet aware how it chooses the file, but, if you narrow down it's choices, then you can control which ones it uses and modify the settings there. If you crack open the xmls, you'll immediately see the languages defined in there (en_US, en_GB, etc), so removing languages is a matter of deleting the lang/region and adding your own (en_PL?). Again, there's so many xmls in there that it's hard to know which one to use, so either make your own and delete the rest, or edit them all. I'll give you the batch that I use in my /customize, reading through them should help you understand what's going on with those xmls.
Click to expand...
Click to collapse
I'm familiar with XML files and have no problems editing them but this whole thing was a little confused specially because there are no documentation and like you said, it's hard to tell which one is which.
I know you kinda suggested that but I'm no seeing how to do it (maybe with the file you attached?) but, how could I start clean, only have default XML files and nothing else, no carrier specific stuff, no bulk, just plain default settings that I can tweak to my own liking.
Any hints on how to do that, or it's hard to tell and you don't know how to do it either?
jubeh said:
This I'm not sure about. You could try emptying your /system/etc/apns-conf.xml so that it won't be able to register to any data service (just leave the <apns version="x"> </apns> tags, remove everything in between).
Click to expand...
Click to collapse
Unfortunately, this did not work

The generic HTC "update.zip" has format: DATA for the last line of the update script, which I suppose will delete everything becuse the copy_dir command was executed before.
Should I not use the format data option? Won't that leave traces of a previous installation if I want to start clean? What's the recommendations or suggestions?
Click to expand...
Click to collapse
I've never understood why the default update script does that, but yeah, format DATA: will erase userdata available. It might not be such a bad thing if you hadn't written anything to it before. For a wipe before install, move format DATA: (and, optionally, also add format CACHE: ) right after format SYSTEM: and before you start writing anything to disk. It should give you a clean install without having to wipe from the recovery menu.
What will happen to the data-sync when i tries to sync the data and finds the provider for gmail is not there? Will it simply be ignored? Will there be any errors changing any of the options (specially the gmail one) on the data synchronization settings?
Click to expand...
Click to collapse
I've not tried it yet. Worst case scenario, you'll get the current problem we have with eclair builds (sync will just keep running and eat out battery because it's looking for a service). That's why I'd suggest leaving gmail provider there, it's just a broadcast service, but without an app to receive the data, it'll do nothing. If your concern is data usage due to sync, then you should turn off gmail sync in settings, that way only calendar and contacts are synchronized (although you'll still be using data).
I'm familiar with XML files and have no problems editing them but this whole thing was a little confused specially because there are no documentation and like you said, it's hard to tell which one is which.
I know you kinda suggested that but I'm no seeing how to do it (maybe with the file you attached?) but, how could I start clean, only have default XML files and nothing else, no carrier specific stuff, no bulk, just plain default settings that I can tweak to my own liking.
Any hints on how to do that, or it's hard to tell and you don't know how to do it either?
Click to expand...
Click to collapse
The files I attached are pretty generic. It's meant to replace your /customize folder, not add to it, with the exception being /resources inside /customize, that one I use a lighter version (no footprints) but I didn't include it in the archive for size. I have no carrier stuff there, just settings for language (I think I made it all en_US, replace or add what you need) and layouts for the different scenes, some mail app behaviors (added back the Yahoo! mail and AOL mail options, I think), some footprint, stock, and weather information. Again, it's meant to replace your /customize directory and the smaller amount of files makes it way easier to edit.
Unfortunately, this did not work
Click to expand...
Click to collapse
Blame the xmls in /customize for that. I mentioned earlier I don't exactly know how the system picks which xmls to use, I think it looks at your carrier, which is then assigned a number (for example, let's assume T-Mobile is given number 238, so xmls with 238 on them will be loaded). Problem is inside those xmls are also apn settings (which I've removed from the files I provided so that the system looks instead for apn settings in apns-conf.xml in /etc).
You could flash your system erasing the whole /customize directory (and removing all mentions of it and it's sub-directories from the update-script so that install doesn't fail). Doing that will give you ALL languages available to the rom, but you'll also lose all your pre-loaded settings (like scenes, weather locations, footprints, stocks) and HTC Setup will take an ungodly amount of time to start (about 4-5 minutes after it's done dexopting) while it builds it's own default settings. It's another route you could take, though. Experiment and you'll eventually hit what you want/need.

OK, I guess I'll have to waste some time testing all this stuff, but I suppose I'll leave that to some other day and do more basic things for now.
But just to be sure, if I use your "customization" I'm basically starting from scratch (clean, defaults) and I can use the "old" customization stuff to understand how am I supposed to configured everything and adapt the clean customization folder to my needs. Right?
jubeh said:
You could flash your system erasing the whole /customize directory (and removing all mentions of it and it's sub-directories from the update-script so that install doesn't fail). Doing that will give you ALL languages available to the rom, but you'll also lose all your pre-loaded settings (like scenes, weather locations, footprints, stocks) and HTC Setup will take an ungodly amount of time to start (about 4-5 minutes after it's done dexopting) while it builds it's own default settings. It's another route you could take, though. Experiment and you'll eventually hit what you want/need.
Click to expand...
Click to collapse
This gave me an idea, dunno if it could work...
What if I did that, erase everything inside customize, let the system build its own default settings and then access that directory (through ADB maybe?) and see what the system has created. It should be much more cleaned, without all those carrier XMLs and stuff. Then I could start from there, what do you think?
I have a few more questions about the update-script if you don't mind, but I'm very tired to write them and discuss them now. I'll leave them for another day if you don't mind coming back here and help me out some more.
Thanks for all your help so far

I should probably have mentioned, some of the stuff on those xmls is pretty straightforward (setting language, region, weather, footprints, etc), but a lot of it (setting up default shortcuts, default widgets, system settings) requires working knowledge of android, or at least of how activities are handled and how xmls come into play and how android references the different activities in a package.
On a different note, the xmls are read-only configuration files, actually, anything in system is read-only data (program data, configuration data, resource data), so if you don't have a customize directory the system won't recreate it. What the system does do during normal boot (factory) is to take those carrier-specific settings and apply them to a file(s) in /data where configuration settings are ultimately written/re-written and accessed at will. The xmls are just a database of the settings that will ultimately get picked. When you don't have a /customize directory the system writes the settings straight to -rw enabled /data partition based on things like your current carrier, carrier provided information for locale, etc.
I don't think you can take what's written to /data and use it on a default install package, well, you could, but it's much easier (and a more correct approach) to learn how those xmls in /customize work.

jubeh said:
(...)but it's much easier (and a more correct approach) to learn how those xmls in /customize work.
Click to expand...
Click to collapse
I wouldn't mind doing that if there was some kind of documentation... As it is right now, it's too confusing and there are too many XMLs to begin with...
jubeh said:
I've not tried it yet. Worst case scenario, you'll get the current problem we have with eclair builds (sync will just keep running and eat out battery because it's looking for a service). That's why I'd suggest leaving gmail provider there, it's just a broadcast service, but without an app to receive the data, it'll do nothing. If your concern is data usage due to sync, then you should turn off gmail sync in settings, that way only calendar and contacts are synchronized (although you'll still be using data).
Click to expand...
Click to collapse
But if I leave the gmail provider there and do not turn off gmail sync in settings, my e-mail will still be synced right? There's just no Gmail app to view them but they are still going to be downloaded, correct?

Ok, I finally got a change to test the Gmail/GmailProvider thing and it didn't look like there any problems like trying to sync and draining the battery. Actually, if auto-sync was enabled, there was always a message saying the mail was never synced and no matter how many times you check/uncheck it, it wouldn't sync. If auto-sync was disabled, you could press the gmail button many times but nothing happened, no sync at all (of course).
I think this behavior is fine and I can safely delete the Gmail and provider from the phone without worrying that something will go wrong
Ok, now my other questions that I said I was going to post later...
As we talked about, you suggested to add a format data and cache right after a format to system. Comparing that to the MoDaCo ROM (and other ROMs too) it's common for them to, instead of formatting data, they do this:
Code:
delete DATA:app
delete DATA:init.sh
delete DATA:local
delete DATA:dalvik-cache
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
Why are they doing this? If they are not formatting, it means there's something inside data that they want to keep. Have any idea what and why?
The other question I had about the update-script is that the following two lines are found in the original HTC update but not on MoDaCo's:
Code:
symlink dumpstate SYSTEM:bin/bugreport
symlink dumpstate SYSTEM:bin/dumpcrash
Are they relevant somehow? Should I just ignore and leave them there or remove them just as MCR did, for some reason?
There's a file named build.prop inside the system folder. I've already searched countless times and can't find any proper documentation about this file... It seems it can hold various default properties for the system and it would be nice to know what I can configure there. Maybe I could workaround problem #4 (top of the topic) in this file? Maybe some file in the Android source has comments about the possible properties on this file? Do you have any idea?
One last thing...
I've read about odex and dex files but found it a little confusing... All I know is that original HTC updates come mostly with .apk files and a corresponding .odex file. But most (if not all) custom ROM's, do not have the .odex file but have a classes.dex inside the apk. What's the real difference between the two? And why custom ROMs prefer to have the .dex file instead of the .odex, is it better somehow for some reason? If so, what should I do to "change" the .odex files into classes.dex ones and put them inside the .apk files?

As we talked about, you suggested to add a format data and cache right after a format to system. Comparing that to the MoDaCo ROM (and other ROMs too) it's common for them to, instead of formatting data, they do this:
Code:
delete DATA:app
delete DATA:init.sh
delete DATA:local
delete DATA:dalvik-cache
copy_dir PACKAGE:data DATA:
set_perm 1000 1000 0771 DATA:app
Why are they doing this? If they are not formatting, it means there's something inside data that they want to keep. Have any idea what and why?
Click to expand...
Click to collapse
Yeah, I mentioned on the earlier post that doing "format DATA:" and "format CACHE:" would be the equivalent of running "Wipe data/factory reset" from the recovery menu since you mentioned about making a build that installs fresh. Paul's (MoDaCo) roms are no-wipe, so they preserve that program data and instead format all other directories inside /data that might be problematic during an upgrade; his roms use apps2sd, so he assumes that the user's downloaded apps are in mmcblk0p2 (the memory card's ext partition), so by formatting /data/app, he's only erasing the apps he included in his update package, if the user doesn't use a2sd, well, this erases their downloaded apps (one of the reasons I dislike the current a2sd implementation, it's not suited for all situations). Then he deletes data/init.sh, which is basically a script he includes in /data that runs during a rom's boot and starts things like a2sd, compcache, linux-swap, and other custom rom's "advanced" features. DATA:local is also app storage (I believe) and functions much like in windows' /users/x/application data/local, it can be problematic if not done between builds. Last, he erases /data/dalvik-cache, this is where the temporary application code resides (the extracted classes.dex from inside an apk) for rapid execution (for apks that haven't been pre-odexed). If you wish to make your rom no-wipe, add those lines instead of "format DATA:", but I'd remove "delete DATA:app" since it's a bit problematic for people who don't use a2sd.
The other question I had about the update-script is that the following two lines are found in the original HTC update but not on MoDaCo's:
Code:
symlink dumpstate SYSTEM:bin/bugreport
symlink dumpstate SYSTEM:bin/dumpcrash
Are they relevant somehow? Should I just ignore and leave them there or remove them just as MCR did, for some reason?
Click to expand...
Click to collapse
dumpstate is a debugging tool that android uses, but I don't know exactly where or how (I think it's an HTC-specific log for their HTCLog.apk). It contains two tools, dumpcrash and bugreport, much like toolbox is a binary that contains all the tools you see symlinked to it in update-script (bin, df, cp, mv, ls, insmod, etc.). It's not a problem at all if you remove it, but it's not a problem either to leave it there. Whenever something is symlinked, though (if it's something you added, for example, busybox tools), you have to make sure that the binary or a link to it or a previous symlink to it doesn't exist, otherwise your update-script will fail (like symlinking toolbox's insmod and then busybox's insmod, or busybox's ping if a ping binary already exists in the target symlink directory (for example /system/bin)).
There's a file named build.prop inside the system folder. I've already searched countless times and can't find any proper documentation about this file... It seems it can hold various default properties for the system and it would be nice to know what I can configure there. Maybe I could workaround problem #4 (top of the topic) in this file? Maybe some file in the Android source has comments about the possible properties on this file? Do you have any idea?
Click to expand...
Click to collapse
I guess that could work if you modify network type (there's a property somewhere in there about HSXPA, I think), but like you said, there's no documentation so it calls for some experimenting.
Those properties are also overrides, so, for example, if your rom doesn't do adb root (because ro.secure is set to 1 in init.rc), you can add ro.secure=0 in build.prop to enable adb root again.
Most properties are self explanatory, and pretty much, if you can't tell what it is, you probably shouldn't change it (for example, build.fingerprint allows you to change what kind of apps Market presents to you).
I'd say, don't mess around too much with the first batch (default build properties), with the exception of build.fingerprint.
One last thing...
I've read about odex and dex files but found it a little confusing... All I know is that original HTC updates come mostly with .apk files and a corresponding .odex file. But most (if not all) custom ROM's, do not have the .odex file but have a classes.dex inside the apk. What's the real difference between the two? And why custom ROMs prefer to have the .dex file instead of the .odex, is it better somehow for some reason? If so, what should I do to "change" the .odex files into classes.dex ones and put them inside the .apk files?
Click to expand...
Click to collapse
The classes.dex files is the dalvik bytecode for the application. It contains everything inside /appnamespace/src and /appnamespace/R if you're working from the source. It's basically all the java code already compiled for the dalvik virtual machine.
Dalvik doesn't do jit, so the classes.dex have to be extracted (if they're not already) and run through the virtual machine in a process called dexopt that produces code that can be much more rapidly excecuted by the dvm (dalvik virtual machine). The extracted code is then placed in /data/dalvik-cache (mentioned earlier) as [email protected]@[email protected]. All this pre-cached code eats away at your space in your /data partition (where you store your system data, like your email, messages, browser cache, and downloaded applications) and can be quite a big chunk of space gone (framework classes specially, since all jars inside /system/framework are nothing but java code, about 20MB for all framework code).
The solution to prevent usage of /data storage on an initial build is to have the code pre-extracted. This is done at compile-time running the created system through an emulator and then creating the odex files, which is almost exactly the same as the *@classes.dex found inside /data/dalvik-cache except that it's much better aligned to the device's memory configuration and offers (marginally) better excecution, plus when the package manager first encounters the apk, it's first step is to check if an .odex of it is available, if it is, then it skips checking and dexopting a classes.dex inside it and proceeds to the next package.
Odex files increase the space in your /system partition (it was already read-only, so it's not like you're missing out on something) so those 20 MB of framework that would have been tossed into your userdata partition are now out of the way into the useless (in an available-storage sense) /system partition so you have more free space for your apps.
There's a tool that can create odex files for apks that don't have them, it's called dexopt-wrapper and it has to be run from a running phone. However, this tool won't strip the (now unnecessary) classes.dex from inside an apk, so it has to be done manually. Removing the classes.dex once you've created an odex is a good idea merely on the basis of space savings. However, once a classes.dex has been odexed, it's very hard (used to be impossible) to turn it back to a classes.dex and place it inside an apk.
Usually, us rom cooks preffer the classes.dex inside the apk because odexes create conflicts when you try to mix and match apps between builds. Since the odex files are tied to the framework where they were created, adding an apk with it's odex to another framework (even one that's only ever-so-slightly different) will be very problematic (the odex code is no longer protected or verified, so the rom won't even boot because it will fail verification for all available packages).
The tool for reverting odex files is called deodexerant by jesus freke. It requires working knowledge of how the system goes through the dexopting process so you can run it in the proper order.
Another thing to know is that even if your /system is pre-odexed, your downloaded apps will still be run through the package manager and have their classes.dex dexopted and placed in /data/dalvik-cache.
Oh, and never, ever put odexes anywhere but /system/framework or /system/app (or /data/app_s if you use a2sd). The package manager currently can't handle odexes in /data/app or /data/app_private (the other two locations where it looks for packages other than /system/app).
In the end, whether to use/have odexes comes down to personal choice (and available space). There's two hero builds, those based in 2.73.405.61 and 2.73.405.66, that have not been pre-odexed, so they're perfect for experimenting, mix/matching, etc...

jubeh said:
Yeah, I mentioned on the earlier post that doing "format DATA:" and "format CACHE:" would be the equivalent of running "Wipe data/factory reset" from the recovery menu since you mentioned about making a build that installs fresh. Paul's (MoDaCo) roms are no-wipe, so they preserve that program data and instead format all other directories inside /data that might be problematic during an upgrade; his roms use apps2sd, so he assumes that the user's downloaded apps are in mmcblk0p2 (the memory card's ext partition), so by formatting /data/app, he's only erasing the apps he included in his update package, if the user doesn't use a2sd, well, this erases their downloaded apps (one of the reasons I dislike the current a2sd implementation, it's not suited for all situations). Then he deletes data/init.sh, which is basically a script he includes in /data that runs during a rom's boot and starts things like a2sd, compcache, linux-swap, and other custom rom's "advanced" features. DATA:local is also app storage (I believe) and functions much like in windows' /users/x/application data/local, it can be problematic if not done between builds. Last, he erases /data/dalvik-cache, this is where the temporary application code resides (the extracted classes.dex from inside an apk) for rapid execution (for apks that haven't been pre-odexed). If you wish to make your rom no-wipe, add those lines instead of "format DATA:", but I'd remove "delete DATA:app" since it's a bit problematic for people who don't use a2sd.
Click to expand...
Click to collapse
So, to make it clear, I could:
Option A) Add format DATA and format CACHE to my ROM and this would make my ROM to automatically wipe everything and start clean.
Option B) I could not add anything (no formats and no deletes) and warn the users to explicitly perform a wipe before flashing.
Option C) Add delete DATA:local and DATA:dalvik-cache and have a no-wipe ROM where the apps would be preserved from the last ROM, right?
Ok, I don't use a2sd and for now have no intention in doing so. My best guess is to start with option A for now and maybe opt by using some app to backup everything and restore everything back after flashing a wipe ROM. Or I could go with option C and everything could work fine but there's a chance I could run into some issues in the long run.
jubeh said:
The classes.dex files is the dalvik bytecode for the application. It contains everything inside /appnamespace/src and /appnamespace/R if you're working from the source. It's basically all the java code already compiled for the dalvik virtual machine.
Dalvik doesn't do jit, so the classes.dex have to be extracted (if they're not already) and run through the virtual machine in a process called dexopt that produces code that can be much more rapidly excecuted by the dvm (dalvik virtual machine). The extracted code is then placed in /data/dalvik-cache (mentioned earlier) as [email protected]@[email protected]. All this pre-cached code eats away at your space in your /data partition (where you store your system data, like your email, messages, browser cache, and downloaded applications) and can be quite a big chunk of space gone (framework classes specially, since all jars inside /system/framework are nothing but java code, about 20MB for all framework code).
The solution to prevent usage of /data storage on an initial build is to have the code pre-extracted. This is done at compile-time running the created system through an emulator and then creating the odex files, which is almost exactly the same as the *@classes.dex found inside /data/dalvik-cache except that it's much better aligned to the device's memory configuration and offers (marginally) better excecution, plus when the package manager first encounters the apk, it's first step is to check if an .odex of it is available, if it is, then it skips checking and dexopting a classes.dex inside it and proceeds to the next package.
Odex files increase the space in your /system partition (it was already read-only, so it's not like you're missing out on something) so those 20 MB of framework that would have been tossed into your userdata partition are now out of the way into the useless (in an available-storage sense) /system partition so you have more free space for your apps.
There's a tool that can create odex files for apks that don't have them, it's called dexopt-wrapper and it has to be run from a running phone. However, this tool won't strip the (now unnecessary) classes.dex from inside an apk, so it has to be done manually. Removing the classes.dex once you've created an odex is a good idea merely on the basis of space savings. However, once a classes.dex has been odexed, it's very hard (used to be impossible) to turn it back to a classes.dex and place it inside an apk.
Usually, us rom cooks preffer the classes.dex inside the apk because odexes create conflicts when you try to mix and match apps between builds. Since the odex files are tied to the framework where they were created, adding an apk with it's odex to another framework (even one that's only ever-so-slightly different) will be very problematic (the odex code is no longer protected or verified, so the rom won't even boot because it will fail verification for all available packages).
The tool for reverting odex files is called deodexerant by jesus freke. It requires working knowledge of how the system goes through the dexopting process so you can run it in the proper order.
Another thing to know is that even if your /system is pre-odexed, your downloaded apps will still be run through the package manager and have their classes.dex dexopted and placed in /data/dalvik-cache.
Oh, and never, ever put odexes anywhere but /system/framework or /system/app (or /data/app_s if you use a2sd). The package manager currently can't handle odexes in /data/app or /data/app_private (the other two locations where it looks for packages other than /system/app).
In the end, whether to use/have odexes comes down to personal choice (and available space). There's two hero builds, those based in 2.73.405.61 and 2.73.405.66, that have not been pre-odexed, so they're perfect for experimenting, mix/matching, etc...
Click to expand...
Click to collapse
Well, I'm currently basing my ROM on 2.73.405.38 as the .66 one has "test" on the name and it feels like a beta version, which I'd rather not use. The .38 version uses .odex files for most of the packages (only a few has classes.dex inside) and I guess I'll leave them like that. If future ROMs, like that .66 one, comes with classes.dex vs .odex files, well, then I'll just leave it like that...
That looks like a very important thing in Android apps and at the same time something that I should be very knowledgeable about before messing with it, so I'll just stay put
Thanks once again for taking your time with such complete answers

So, to make it clear, I could:
Option A) Add format DATA and format CACHE to my ROM and this would make my ROM to automatically wipe everything and start clean.
Option B) I could not add anything (no formats and no deletes) and warn the users to explicitly perform a wipe before flashing.
Option C) Add delete DATA:local and DATA:dalvik-cache and have a no-wipe ROM where the apps would be preserved from the last ROM, right?
Ok, I don't use a2sd and for now have no intention in doing so. My best guess is to start with option A for now and maybe opt by using some app to backup everything and restore everything back after flashing a wipe ROM. Or I could go with option C and everything could work fine but there's a chance I could run into some issues in the long run.
Click to expand...
Click to collapse
Sounds like you got it.
Well, I'm currently basing my ROM on 2.73.405.38 as the .66 one has "test" on the name and it feels like a beta version, which I'd rather not use. The .38 version uses .odex files for most of the packages (only a few has classes.dex inside) and I guess I'll leave them like that. If future ROMs, like that .66 one, comes with classes.dex vs .odex files, well, then I'll just leave it like that...
That looks like a very important thing in Android apps and at the same time something that I should be very knowledgeable about before messing with it, so I'll just stay put
Click to expand...
Click to collapse
Test only refers to the build being signed with test-keys instead of release-keys. In my testing, it makes no difference whatsoever for a rooted user, and there's many theories floating around about signatures, I haven't researched signatures a whole lot because test-keys have worked fine enough for me, so I'm not so sure, but they seem to be working fine for all.
The reason those releases are un-odexed (and signed with test-keys) is because of the problem with odexes I mentioned earlier. When HTC is testing their rom packages and they need to make, say, a framework change, all the apps with odexes would fail as the framework has changed, so, for testing purposes, it's best to leave the apks and jars in the build full (with classes.dex inside). Once you know your build is solid and you won't be changing much, it's safer to odex the whole thing (sort of like me, I'm waiting for the final release of eclair for ADP1 so I can put out a Dream rom odexed and fully working).
Thanks once again for taking your time with such complete answers
Click to expand...
Click to collapse
No problem, I get bored at work

So, version .66 is not beta or anything? I know MoDaCo has his latest ROM based on that and you're saying they are only different in the signing key, still, it's somewhat a "leaked" build, not quite "official".
But hey, if others are using it without any issues, maybe I should be using it too

On second thought...
1) I can't find a pre-rooted .66 version and I don't know how to root it :/
2) There's lots of files in system/xbin that I don't see referenced anywhere (not even in the update-script) and I don't know what should I do with them.
I'm finding it much more confusing to base my ROM on .66 than on .38...

It is a beta, of sorts, but between it and the final shipping product there's not much difference. It's just much, much easier to work with it.
1.) The simplest form of root is to change ro.sercure to 0 in default.prop in the boot ramdisk. Adding a su binary/Superuser.apk is optional, those you can pull from one of paul's builds.
2.) The system/xbin is added to the boot classpath in the boot ramdisk in init.rc. The only mention of it in in update-script is the permissions for the whole folder and then just a couple permissions for some tools, not all of them.

[stopped] by formatting the internal /data partition

Hmm
sorry guys. by now, I'm going to stop fixing this, sigh, I'm not good enough to fix this at the moment.
Can not garantee you the safety of this method but hopefully, the fantastic voodoo lagfix is coming.
Let's hope to see it soon!
I'll leave the update.zip here, if anyone who is intersted in this method, feel free.
FILE LINK :
http://www.multiupload.com/DU013PW7LW
Hi
I have made an update.zip which fixes I9000's lag.
The idea is to format the /data partition, which is the same as /dev/block/mmcblk0p2, to the EXT4 or NILFS2 File Systems.
It's worth because, at least, we don't need an external MicroSD card.
### Before getting started... ###
This method can damage your phone, so you should know what you are attempting to do. All risks are on your own. Also, I strongly recommend you to have a nandroid backup.
### It contains... ###
- absolutely pure JM2 kernel but a tiny modification of the ramdisk. It's from JM2 firmware but should be working on every ECLAIR devices. This kernel helps mounting the data partition to be mounted in every booting and the reason why this is nearly identical to the stock is because some people mind using user-built kernels for some reasons.
- an application called "sl4a" which has shell scripts inside named "EXT4_Lagfix.sh", "NILFS2_Lagfix.sh" and "Openvpn.sh".
※It will install the kernel in the process of applying update.zip so you don't need the Odin.
### More information about this kernel... ###
- As I said, the only thing I've modified is the ramdisk. In details, I've put two files, user_init.sh and user_early_init.sh which are from Unhelpful's kernel, and these files are essential for mounting the /data partition.
- The kernel is almost pure, so it uses modules.
- The user_init.sh, user_early_init.sh support user scripts are very useful. You can see the idea in below thread, OP of Unhelpful.
http://forum.xda-developers.com/showthread.php?t=762171
### About the scripts... ###
- If you touch the EXT4_Lagfix.sh script or NILFS2_Lagfix.sh, it will ask you a superuser permission, back up the entire /data folder in your /sdcard directory, and finally, reboot and do the rest works automatically.
- You can go over to the other file system whenever you want, for instance, it can be changed to the nilfs2 while you are using the ext4 FS.
- There also is a script called Openvpn.sh which will insert tun.ko module and this is for the people who want to use openvpn.
### What you need to do is... ###
- Needs busybox installed and rooted
- Make enough space in /sdcard to make the backing up process work properly.
- In order to use NILFS2 FS, a bit of studying is needed.
About nilfs2 binaries and the Gabage Collector.
go to this site
http://www.nilfs.org/
### What I want to say is...###
- I'm not actually a goot developer so the scripts must be very messy. But it works well, at least with my device.
- Unfortunately, I haven't got the going-back-to-rfs script. I'm working on it recently so if you want to restore the all fixes, you need to flash the firmware.
- Quadrant scores : NILFS2 ~ around 1300
EXT4 ~ around 1700
Do you really concern about the score? I don't. The NILFS2 is exactly what I've needed.
### What I MUST say is... ###
- Big thanks to these people!
## Tayutama -- the beautiful sl4a application is from his rom!
## Unhelpful -- used his idea to mount the /data partition. And the NILFS2 as well!
## supercurio -- He's the one who found how to flash kernels from the phone!
Also, I uploaded some useful utils.
- mkyaffs2image / unyaffs
- chcp / lscp / lssu / mkcp / rmcp
Hope this will help you guys.

So what's different between this and the voodoo lagfix? Is the the partition? I know that voodoo ext4 is /dev/block/mmcblk0p4 while this one is /dev/block/mmcblk0p2

Nice dkcldark.
Will you share your method to unpack-repack the ramdisk from a stock kernel ?

Very nice! Great job!

NeoXTC said:
So what's different between this and the voodoo lagfix? Is the the partition? I know that voodoo ext4 is /dev/block/mmcblk0p4 while this one is /dev/block/mmcblk0p2
Click to expand...
Click to collapse
Oh, I've been busy so I couldn't have a look at the voodoo lagfix. I'm gonna check it. It could be the same as mine or much better
mmcblk0p2 is equal to /data partition, BTW.
supercurio said:
Nice dkcldark.
Will you share your method to unpack-repack the ramdisk from a stock kernel ?
Click to expand...
Click to collapse
Why not? To do this, I've done a bunch of experiments and it made me get older, I think. Will write a reply or a new thread very soon.
RyanZA said:
Very nice! Great job!
Click to expand...
Click to collapse
Thanks!!

dkcldark said:
Why not? To do this, I've done a bunch of experiments and it made me get older, I think. Will write a reply or a new thread very soon.
Click to expand...
Click to collapse
Yeah, theorically it's simple, we have the smallest piece of code building the kernel so why should extracting then reinserting the ramdisk difficult ?
And there comes the difference between the theory and the practice
Congrats for your research and the result, i hope your method can be applied to any kernel including a ramdisk!
This is a major breakthrough in our Opensource Toolbox.

So do you recommend NILFS or ext4 for phones with nand disk? Just reading up a bit on nilfs and seems to create lots of checkpoints? Is it good for nand disk to be writing that much?
I'm tempted to try this tonight

NeoXTC said:
So do you recommend NILFS or ext4 for phones with nand disk? Just reading up a bit on nilfs and seems to create lots of checkpoints? Is it good for nand disk to be writing that much?
I'm tempted to try this tonight
Click to expand...
Click to collapse
Personally, I recommend the NILFS2 File system. I've searched some infomations.
NILFS2 is log-structured File System and known that it's suitable for SSD which is based on nand flash. Normally, Flash has a limited read-write times but nilfs2 tries to write to the entire space equally, and has got the smallest removing interval so the NILFS2 file system makes SSD work very nice. And a great Wear-Leveling as well.
Click to expand...
Click to collapse
I'm not a good developer, as I already said on my post, so I'm not sure it'd also be nice for SGS but it sounds good to me. Actually, I'm using NILFS2 and it's bloody fast!

How difficult would this be to adapt to the Captivate variant of the Galaxy S?

dkcldark said:
Personally, I recommend the NILFS2 File system. I've searched some infomations.
I'm not a good developer, as I already said on my post, so I'm not sure it'd also be nice for SGS but it sounds good to me. Actually, I'm using NILFS2 and it's bloody fast!
Click to expand...
Click to collapse
on which firmware do u recommend to use it?
I'm actually on JM6 but tomorrow i want to try the JM7....
(Any battery drain? )

andars05 said:
How difficult would this be to adapt to the Captivate variant of the Galaxy S?
Click to expand...
Click to collapse
It's not difficult at all.
My kernel is originally from Unhelpful's idea so, people who use his kernel will easily be able to do it. All you need is just remove the zImage and redbend_ua files in the update.zip. Every kernel which supports Unhelpful's rules about /system/etc/init.d will be OK. Or, you can simply modifiy the scripts inside.
One more, if you use Unhelpful's kernel, you don't need the modules any more.
Narcissus85 said:
on which firmware do u recommend to use it?
I'm actually on JM6 but tomorrow i want to try the JM7....
(Any battery drain? )
Click to expand...
Click to collapse
Any firmware should be OK. I'm using JM2 at the moment but I'll try the JM7 as well. So as the result, I'm going to come up with JM7-based pure kernel, although I believe there won't be big differences between JM2 and JM7.
And the battery.. I'm sorry I haven't tested it yet since I've got 4 batteries. I'm not bothered by them lol.

dkcldark said:
It's not difficult at all.
My kernel is originally from Unhelpful's idea so, people who use his kernel will easily be able to do it. All you need is just remove the zImage and redbend_ua files in the update.zip. Every kernel which supports Unhelpful's rules about /system/etc/init.d will be OK. Or, you can simply modifiy the scripts inside.
One more, if you use Unhelpful's kernel, you don't need the modules any more.
Any firmware should be OK. I'm using JM2 at the moment but I'll try the JM7 as well. So as the result, I'm going to come up with JM7-based pure kernel, although I believe there won't be big differences between JM2 and JM7.
And the battery.. I'm sorry I haven't tested it yet since I've got 4 batteries. I'm not bothered by them lol.
Click to expand...
Click to collapse
4 batteries? ahah cool i'll test it tomorrow with my only one battery! :>
thanks for ur lagfix..
mmm just another question...my english isnt so good (i'm italian)....with this "I'm going to come up with JM7-based pure kernel, although I believe there won't be big differences between JM2 and JM7." do u mean that u are going to update ur lagfix with another kernel? so another version comes out soon?
thanks...
ivan

dkcldark said:
It's not difficult at all.
My kernel is originally from Unhelpful's idea so, people who use his kernel will easily be able to do it. All you need is just remove the zImage and redbend_ua files in the update.zip. Every kernel which supports Unhelpful's rules about /system/etc/init.d will be OK. Or, you can simply modifiy the scripts inside.
One more, if you use Unhelpful's kernel, you don't need the modules any more.
Click to expand...
Click to collapse
So, I should flash unhelpfuls kernel. I then next remove the zImage and redbend_ua files from the update.zip, install it, and then run the sl4a script?

Narcissus85 said:
4 batteries? ahah cool i'll test it tomorrow with my only one battery! :>
thanks for ur lagfix..
mmm just another question...my english isnt so good (i'm italian)....with this "I'm going to come up with JM7-based pure kernel, although I believe there won't be big differences between JM2 and JM7." do u mean that u are going to update ur lagfix with another kernel? so another version comes out soon?
thanks...
ivan
Click to expand...
Click to collapse
Yes, I am. Will Just come with a new kernel. Every else is the same.
andars05 said:
So, I should flash unhelpfuls kernel. I then next remove the zImage and redbend_ua files from the update.zip, install it, and then run the sl4a script?
Click to expand...
Click to collapse
In order to reuse after modifying the update.zip, you need to sign it again. If you want to use a plain kernel, I can make it which is not a big deal.

dkcldark said:
In order to reuse after modifying the update.zip, you need to sign it again. If you want to use a plain kernel, I can make it which is not a big deal.
Click to expand...
Click to collapse
That would be most helpful! Thanks for your work! I can't wait to test it out on my captivate
Edit:
I figured out how to resign the update.zip. I'm running the sl4a script now. I'll post the results shortly.
Edit 2:
It appears to back up the data partition, but upon reboot its still rfs.

You may want to make some other modifications I've since learn are needed. Specifically, the stock init will overwrite the MBR, removing any changes you've made to the partition table, and will then write some data to mmcblk0p2 if it does not find a valid RFS filesystem. You might not even see anything wrong at first, but with enough reboots, this will eventually corrupt your reformatted /data partition.
The edit I'm using will be in my next kernel release, and is very simple - open the init binary in a hex editor, find the string "/dev/block/mmcblk0" (make sure there's a NULL after the 0 so that it's not a substring of some longer string) and replace the "m" with a NULL. Init will then fail to rewrite the partition table (because it tries to open "/dev/block/", which is a directory). Then you simply need to change the partition table - you can use exactly the same region of the disk, you won't even need to reformat, just remove the existing partition 2 and make a new partition 3 or 4. Partitions 3 or 4 won't be checked by init on reboot.

Unhelpful said:
You may want to make some other modifications I've since learn are needed. Specifically, the stock init will overwrite the MBR, removing any changes you've made to the partition table, and will then write some data to mmcblk0p2 if it does not find a valid RFS filesystem. You might not even see anything wrong at first, but with enough reboots, this will eventually corrupt your reformatted /data partition.
The edit I'm using will be in my next kernel release, and is very simple - open the init binary in a hex editor, find the string "/dev/block/mmcblk0" (make sure there's a NULL after the 0 so that it's not a substring of some longer string) and replace the "m" with a NULL. Init will then fail to rewrite the partition table (because it tries to open "/dev/block/", which is a directory). Then you simply need to change the partition table - you can use exactly the same region of the disk, you won't even need to reformat, just remove the existing partition 2 and make a new partition 3 or 4. Partitions 3 or 4 won't be checked by init on reboot.
Click to expand...
Click to collapse
I really appreciate for this information!! Just wondering about one thing.
Should I replace the "m" to NULL once in "/dev/block/mmcblk0" or its friend "/dev/block/mmcblk0p2" as well?

Unhelpful said:
You may want to make some other modifications I've since learn are needed. Specifically, the stock init will overwrite the MBR, removing any changes you've made to the partition table, and will then write some data to mmcblk0p2 if it does not find a valid RFS filesystem. You might not even see anything wrong at first, but with enough reboots, this will eventually corrupt your reformatted /data partition.
The edit I'm using will be in my next kernel release, and is very simple - open the init binary in a hex editor, find the string "/dev/block/mmcblk0" (make sure there's a NULL after the 0 so that it's not a substring of some longer string) and replace the "m" with a NULL. Init will then fail to rewrite the partition table (because it tries to open "/dev/block/", which is a directory). Then you simply need to change the partition table - you can use exactly the same region of the disk, you won't even need to reformat, just remove the existing partition 2 and make a new partition 3 or 4. Partitions 3 or 4 won't be checked by init on reboot.
Click to expand...
Click to collapse
Yeah more like curios setup.

What needs to be done in order to adapt this to the Captivate?
I deleted the zImage and redbend_ua files and resigned the zip. I tried both of the lagfix scripts included and neither one worked. It backed up /data and rebooted. It doesn't reformat the fs, and it is still rfs. There doesn't appear to be a /system/etc/init.d directory. I'm running Unhelpful's kernel (v1.2).
I'm rooted and have busybox installed.

If only supercurio can adapt your method of editing the ramdisk inside the stock kernel, it would be awesome! As he already knows how to go back to rfs! We would then have a complete fix!

Kaiser Kernel Thread - ONLY FOR KERNEL AND BUGS

I start this new thread because i want highlight only kaiser changes
Before posting use search button and look in this thread.
This is the kernel thread so all problem like "how to install","i can't install", "miss module", ecc ecc are not allowed. If you need to ask this probably you didn't read correctly the guide in atools.
Atools is multyplatform and need python.
2.6.32 kernel changelog
nand fixed
irq problem is now fixed
TODO:
finish battery support
rewrite camera driver
speed up gps
Latest releases are here
Atools thread

Maybe now kernel development will be a little easier to keep track of. I had a random freeze in the kernel before with a message i've never seen before. Didn't think about exporting the logs. I will upload if I see it again.
It was with the 11-29 .32 kernel and only happened after I just switched to it, installed my barebones build and rebooted for the 1st time. I have never seen it since so it could have been an anomaly. Like I said, it was an error I have never seen before and will catch a log if I ever see it again.

It would be helpful if someone could make a list of which types of problem are kernel related, which are build related and which could be either.
I am fairly new to android and not at all clear on what does what .

I belive it's kind of dificult to explain properly whats a kernel bug, and whats a build bug. At least with a few lines of text anyway...
We probably will have to deal with _no_kernel_related_ questions, but we must stick to the topic. If it's not kernel related, point to another thread. Or perhaps instruct to open a new one.
Also, it would be nice to have this stickied again, and if those asking questions would be so nice to inform about device ; kernel version+builddate ; type of instalation (nand VS sd img file VS sd partion VS whatever) ; radio, it would probably make it easier to understand why is the user facing issues.
IMHO of course

Ipv6 support in kernel
Hi Liquid,
Could you please include ipv6 support in kernel? It's needed by some apps, like google goggles, i compiled my own kernel with ipv6 and worked fine.
I hope there is enough kernel space to include ipv6 in the current kernel version
thanks

i try to compile a ipv6 6 month ago but i have some problems with wlan.
I change some part so i'm going to do new test on it

Question, not directly kernel related, but must start here.
If data using Yaffs is so unstable... why not another FS ?

I noticed that 2.6.25 kernel doesn't mount the partitions on relatime. Is there any chance we could do the same for 2.6.32 (which defaults to relatime). In 2.6.32, I tried to put it to atime but it doesn't work and just defaults to relatime. Could this possibly be the reason we have an unstable data partition in 2.6.32?

clemsyn said:
I noticed that 2.6.25 kernel doesn't mount the partitions on relatime. Is there any chance we could do the same for 2.6.32 (which defaults to relatime). In 2.6.32, I tried to put it to atime but it doesn't work and just defaults to relatime. Could this possibly be the reason we have an unstable data partition in 2.6.32?
Click to expand...
Click to collapse
Could you point me where are the partitions mounted ? normaly i would search /etc/fstab, but i fearin droid they're mounted elsewhere.
One thing i found in kerneltrap is nice...
"It's also perhaps the most stupid Unix design idea of all times. Unix is really nice and well done, but think about this a bit: 'For every file that is read from the disk, lets do a ... write to the disk! And, for every file that is already cached and which we read from the cache ... do a write to the disk!'"
Click to expand...
Click to collapse
This is explaining why relatime is better than atime. But if we could force a write to the disk on every access...

daedric said:
Could you point me where are the partitions mounted ? normaly i would search /etc/fstab, but i fearin droid they're mounted elsewhere.
One thing i found in kerneltrap is nice...
This is explaining why relatime is better than atime. But if we could force a write to the disk on every access...
Click to expand...
Click to collapse
its on /system/bin/userinit.sh

clemsyn said:
its on /system/bin/userinit.sh
Click to expand...
Click to collapse
You sure ? i only find a remount there ... to fix su

Has anyone recently synchronized our yaffs2 tree with the main one on www.yaffs.net? I noticed there are quite a few differences. I'm testing, too soon to tell if there are any notable differences.

I think scooter synchronised it a little over a month ago? I know its been about that long.

daedric said:
You sure ? i only find a remount there ... to fix su
Click to expand...
Click to collapse
you can add this to userinit.sh
busybox mount -o remount,rw,nouid,nodiratime,nodev /data
I think it depends on the rom but my ROM have those options. I'll upload my userinit.sh
Here is a copy of my userinit.sh
BTW, I added noatime on 2.6.25 kernel (which is suppose to be stable) and I got a Reboot and data corruption. I think if we mount 2.6.32 like 2.6.25's default mount, it could be better but it has to be done in the kernel itself coz 2.6.32 defaults to relatime.

aceoyame said:
I think scooter synchronised it a little over a month ago? I know its been about that long.
Click to expand...
Click to collapse
I synchronized my yaffs sources yesterday. It survived several battery-pulls yesterday with no corruption, but one single clean shutdown modified email settings for one account and made Maps FC. A dalvik clear cured maps but not the email settings.
Yaffs has data recovery capabilities, as proven by the battery pulls that gave no corruption at all.
So i think yaffs is good and nand is good. The data corruption problem lies outside the kernel and is a subject for a different thread.
Sent from my Android on HTC Kaiser/Kaiser/Polaris using XDA App

n2rjt said:
I synchronized my yaffs sources yesterday. It survived several battery-pulls yesterday with no corruption, but one single clean shutdown modified email settings for one account and made Maps FC. A dalvik clear cured maps but not the email settings.
Yaffs has data recovery capabilities, as proven by the battery pulls that gave no corruption at all.
So i think yaffs is good and nand is good. The data corruption problem lies outside the kernel and is a subject for a different thread.
Sent from my Android on HTC Kaiser/Kaiser/Polaris using XDA App
Click to expand...
Click to collapse
I read this quote about atime since you had problems with email on reboot
1. atime – This option causes Linux to record the last (or latest) time when a particular file was accessed. This information is particularly helpful for sysadmins or some programs (specially mail programs) to know when a particular file was last used/accessed.
So I think here is were atime option would help (email programs)

clemsyn said:
I read this quote about atime since you had problems with email on reboot
1. atime – This option causes Linux to record the last (or latest) time when a particular file was accessed. This information is particularly helpful for sysadmins or some programs (specially mail programs) to know when a particular file was last used/accessed.
So I think here is were atime option would help (email programs)
Click to expand...
Click to collapse
I think you're confused... the e-mail problem n2rjt was having was complete random. He had problems in e-mail and maps.. could be browser and settings. It's random.
Now:
1. atime – This option causes Linux to record the last (or latest) time when a particular file was accessed. This information is particularly helpful for sysadmins or some programs (specially mail programs) to know when a particular file was last used/accessed.
2. noatime – This option stops recording the last file access time when the file is just read. The noatime option eliminates all the writes to the disk each time a file was just read which previously used to happen with atime. However a write is made to a disk in case if a file is being changed/written.
3. relatime – A filesystem mount with this option causes the access time to be updated if they are (before the update has occurred) earlier than the modification time. This significantly cuts down the writes caused by atime updates. However not many people use this option because they are simply not aware of it.
source: http://linux.koolsolutions.com/2009...sb-part-4-noatime-and-relatime-mount-options/
Click to expand...
Click to collapse
Indeed relatime is the most usefull option for us.
Yet... reading this for the 2nd time:
Yaffs has data recovery capabilities, as proven by the battery pulls that gave no corruption at all.
So i think yaffs is good and nand is good. The data corruption problem lies outside the kernel and is a subject for a different thread.
Click to expand...
Click to collapse
Could it be... a proc atempting to unmount /data or remount,ro that causes a defect ?
One other thing... you're sync'ing to yaffs.net, would it be too bad/dificult to use the yaffs2 provided with usual HTC sources ?
I'm looking at Supersonic... it uses our 2.6.32 kernel....

daedric said:
One other thing... you're sync'ing to yaffs.net, would it be too bad/dificult to use the yaffs2 provided with usual HTC sources ?
Click to expand...
Click to collapse
The yaffs in the kernel at the minute is from the official android source, so what all other HTC devices are using

I tried the yaffs.net latest just as a comparison. It didn't change anything one way or the other.
Sent from my Android on HTC Kaiser/Kaiser/Polaris using XDA App

Trying to setup 2.6.32 kernel over 2.6.25 on my tytnII with Warm Donut on board.
Using Atools. Doing everything step-by-step, but when i reach this steps:
33 then flash from sd (gray)
34 reset device when it finished
it's booting from sd, (gray), show "Loading.." and never complete :-(
have wait more than hour.
What's possible reason?
Using zImage-11-27-10 from l1qu1d sourceforge.net site.
Thanks in advance for help.

[BUILD] [GingerBread] CM7 from Neopeek!! Dev Preview ClockworkMod!

Hey all,
Wanted to post CM7 here for everyone to enjoy.
Neopeek has been working hard to get this build up and running, he's been doing great work!
Neopeek's Original Post - if you wish to see Neopeek's original post about the CM7 topic.
Also, since now I am using his new ClockWordMod method - CWM Recovery thread
As is stated in the title, this is mainly for devs/testers. Lots of things are broken, and there will be frequent updates.
There are two methods to using this build:
1) ClockWorkMod (requires 3 separate ext2 partitions)
2) Loop'd system.ext2 image (like XDAndroid) [In progress]
The benefit of #1 is a dedicated system/data partition, and *partial* USB mass storage support.
The benefit of #2 is, it's easier .
I still have to work to get #2 up, but I wanted to post #1 ASAP. Lots of work to flesh out the post to, be patient!
I've been holding off on releasing this CWM version of CM7, because I'm still having a lot of bugs with it. Still needs quite a bit of work, but it's getting late for me tonight - I am going to release it, let me know if you guys make improvements, I'd love to incorporate them!
Main issues seem to be related to my durn CDMA device. Need to break into some logs to see what's goin on, it may be a little while! I think GSM folk will be fine. Let me know!
Downloads
Download #1
How-to
Method #1 (ClockWorkMod)
****WARNING****
If you setup dedicated partitions using this method, normal XDAndroid builds will think that you want to use dedicated partitions. I had to modify the init to get it to work correctly (I basically broke the partition detection in the init). It's not pretty, but I don't know of a better way to resolve that issue - let me know what ideas you guys have!
Setup:
You will need to create three (3) ext2 partitions on your SD card. This is not extremely simple, but by no means impossible .
If you're using Windows, I like the Mini Partition Tool. If you're in Linux, I usually use gparted.
This takes a little bit of planning - but in total, I would say you need a minimum of 450mb of space for all three ext2 partitions. So work out how much free space is left (if you want a bigger data partiton, or system partition account for it!) and make that the first partition - FAT32. Make sure all the partitions are "primary", not "logical"! See the partition layout below:
1 - FAT32 Primary (remainder of card)
2 - ext2 Primary (at least 150mb, system partition)
3 - ext2 Primary (at least 250mb, data partition - I wouldn't go over 1gb here)
4 - ext2 Primary (at least 50mb, cache partition)
Don't worry about putting anything in these ext2 partitions, everything is populated by Android and CWM. The FAT32 partition is used to kick off Android, while system, data and cache are all ext2 partitions. Reason for this is, hopefully if things go haywire we can blast out cache, not data . Also, it should be easier to update using this method than previously.
Extract the file. Look at the folder it created (npkboot). Copy the entire npkboot folder to the root of your SD (for now this is required - the init assumes there's a /sdcard/npkboot/tmp folder). There will also be an update.zip, in this case update-cm-7.0.0-neopeek.zip. Put this at the root of your SD card (this makes finding it in the menu much easier!)
Copy the appropriate startup.txt from the STARTUPS folder.
Find haret.exe in th npkboot folder. Run it, and watch the boot process carefully! It will ask you to hold the vol down button to enter recovery, and you have 10 seconds to do so.
Assuming you manage to do this correctly (I had some trouble at first ) you should see an icon like a hat. Wait for the menu.
Buttons:
End key - back (back button seems to do this as well...)
Send key - enter
Vol up - up
Vol down - down (menu button seems to do this as well...)
If this is the first install, go to "install zip from sdcard". Then "choose zip from sdcard" This will install the system image to the system partition from the update.zip.
If you need to wipe the data partition, you can do that under "mounts and storage". You shouldn't need to do this unless you're going from gb -> froyo or visa-versa. Froyo -> froyo should 'just work', as always YMMV.
There are A LOT of options in replimenu. The only ones I have used are install, format /data, /system or /cache and Reboot. According to Neopeek, there are several things that should not be used:
Neopeek said:
* Don't use "factory wipe" in CWM (you have been warned )
* Don't use "format sdcard" unless you know what you do (you have been warned )
* Don't use "partition sdcard" unless you know what you do (you have been warned )
Click to expand...
Click to collapse
Once you've installed Android, go back to the main menu and "reboot system now" There are some bugs, for example "reboot recovery" and "power off" all just reboot the phone to WinMo.
Boot haret again, but this time don't hold the vol down to enter recovery - boot as normal!
#2: Much simpler method, similar to XDAndroid builds.
Extract the file to your desktop. Go into the CM7 folder, look for the STARTUPS folder - grab the appropriate startup.txt, move it to where haret.exe is.
Copy everything in the CM7 folder to the root of your SD card - if you want to run it from a folder, create a rel_path=CM7 statement in your startup.txt after the set cmdline (if you want a different folder, change the CM7.)
Grab the ts-calibration file attached to this post (for RHOD), or go to this post for a list of ts-calib's. You can also create your own on boot, but may have to revert to an older kernel to do so.
Run haret.exe!

****Not Working****
CDMA seems broken - still! I did some testing with hyc's RIL and it's still busted, so there's something in the system image messing with it - Airplane mode was checked, and I could not uncheck it.
In addition, phone.apk seems like a forceclosefest. Neopeek did point this out in his original thread...
Mobile data - broken (Fixed soon)
Wifi - broken (Fixed soon)
System needs optimization for RHOD - was originally build with DIAM in mind - so it's a little slow.
Failed wakes seem pretty bad - sometimes it is quite difficult to wake the phone.
Boot animation seems to be not present - I'm looking into fixing it, probably graphics libs if anything that's broke it.

Dev Content
This new method should be great for anyone who wants to make their own updates - you can easily peek into the system image and make changes at will, then just install the new system image! You can even rolll updates instead of an entire system image if you wish.
The script language in ClockWorkMod is called "Edify", see this thread to learn about the syntax. This is related to the META-INF folder you see in the update zip.
There's a few differences in the rootfs', the main being the init. There's also some additional files it looks like neopeek added for Recovery specifically.

This is amazing work. Props to all involved.

Does anyone have any feedback on this build?

arrrghhh said:
Long story short, you need a ext2 partition and a fat32 partition - ext2 needs to be between 400-600mb, fat32 can just take the remainder of the card's space.
Click to expand...
Click to collapse
Is there a reason for the 600mb upper limit? If I use 1 gb, will it cause problems or will it not use it and just waste card space?

PM message
Arrrgh,
Disregard the PM I sent you.
I read the instruction and like you said, its not simple. So I will wait until you upload the easy to run load.
Thanks

Ltilt2 said:
Arrrgh,
Disregard the PM I sent you.
I read the instruction and like you said, its not simple. So I will wait until you upload the easy to run load.
Thanks
Click to expand...
Click to collapse
Sorry, I need access to a Linux box to do the 'easy' build... I will when I get home tonight from work, promise .
vinceweis said:
Is there a reason for the 600mb upper limit? If I use 1 gb, will it cause problems or will it not use it and just waste card space?
Click to expand...
Click to collapse
You can make it as big as you want I guess... I'm not sure how neopeek sets up the data partition to be honest, I'd have to ask him.
Edit - in lieu of the post below, I adjusted the first post to say it needs to be at least 400mb .

vinceweis said:
Is there a reason for the 600mb upper limit? If I use 1 gb, will it cause problems or will it not use it and just waste card space?
Click to expand...
Click to collapse
I have 700 mb and is working well,you can use as much as you want
Ltilt2 said:
Arrrgh,
Disregard the PM I sent you.
I read the instruction and like you said, its not simple. So I will wait until you upload the easy to run load.
Thanks
Click to expand...
Click to collapse
Is easy just use mini tool partition wizard,delete the main partition on the sd-card
and do two,one fat32 primary and one ext2 primary,then extract the archive and put the proper startups in npkinstall and in root
from the phone run install.exe
and after the reboot use haret.exe

I wonder if CDMA data is working with this...that is my biggest issue with GB right now...

HELICOPTER88 said:
I have 700 mb and is working well,you can use as much as you want
Is easy just use mini tool partition wizard,delete the main partition on the sd-card
and do two,one fat32 primary and one ext2 primary,then extract the archive and put the proper startups in npkinstall and in root
from the phone run install.exe
and after the reboot use haret.exe
Click to expand...
Click to collapse
yeah easy for you to say, that sounds complicated.
I will wait for the other one.
Do you know the ETA?

i have
- downloaded the file cm7new.tar.gz
- made primary fat32 and ext2 partitions
- extracted the file cm7new.tar.gz to the root of the fat32 partition
- got a lot of linux folders
but i am missing
- npkinstall folder
- startup.txt
- install.exe
- haret.exe
- startup folder
Where are these files?

Tippfehler said:
i have
- downloaded the file cm7new.tar.gz
- made primary fat32 and ext2 partitions
- extracted the file cm7new.tar.gz to the root of the fat32 partition
- got a lot of linux folders
but i am missing
- npkinstall folder
- startup.txt
- install.exe
- haret.exe
- startup folder
Where are these files?
Click to expand...
Click to collapse
Still need to upload, sorry!

Tippfehler said:
i have
- downloaded the file cm7new.tar.gz
- made primary fat32 and ext2 partitions
- extracted the file cm7new.tar.gz to the root of the fat32 partition
- got a lot of linux folders
but i am missing
- npkinstall folder
- startup.txt
- install.exe
- haret.exe
- startup folder
Where are these files?
Click to expand...
Click to collapse
I was looking at that too when extracted the files and didn't see what you are also missing.

Sorry about the confusion guys, I probably shouldn't have put the download link up before I was done .
I just fixed it, so #1 is good to go! I'll get #2 up later, thanks.

Another noob question, but the rootfs, kernels, oc, etc from the xdandroid builds will work the same on these?

(...solved...)

it's necessary create a swap partition?
What works in this build?

fishingmedic said:
Another noob question, but the rootfs, kernels, oc, etc from the xdandroid builds will work the same on these?
Click to expand...
Click to collapse
The rootfs is more or less collapsed in these.
Kernels are straight from XDAndroid repo.
OC is the same, startup.txt is fairly similar .
Quincux said:
it's necessary create a swap partition?
What works in this build?
Click to expand...
Click to collapse
Swap is not necessary.
Most things that work in XDAndroid work in here. Lots of other things are broken here tho .

Okay, downloaded and installed without problems.
First impressions:
Absolutely great build in terms of response!
No delays in menus, etc.
Also, this looks like the first build that may have little or no clock delay, so no need for clocksync !!! (to be confirmed since I did not use it very long, die to issues below)
However...
1) No WIFI
2) No Mobile data (2G/3G)
3) camera not working (could be just blackstone)
- maybe other issues, decided to wait for more feedback before continuing, without market access there's little point...
Device: Blackstone
Kernel: extensive testing with provided one (1276), later switched to 1277, to fix audiostuttering, no wifi or mobile data in either...
As a reference, used many NeoPeek / XDAndroid builds, generally got WIFI/Mobile data working until now...
Love to hear if someone else (blackstone?) got WIFI/2G/3G working...
Oh, well, as I understood, this was an early "developer" release, still plenty of room for improvements!
Cheers,
Robbert

[Q] Question about build sizes

Hi All...
Maybe a dumb question but I need an answer.
I was looking through the rest of the builds on kingshui's sourceforge site and something caught me off-guard.
Apparently he has some /system builds exceeding 100mb (the one I downloaded was 103mb, 98mb in a tarball).
I know with standard kernels we can slide the partition marker to the right and gain some /system space, but I'm currently using Scoot's cache kernel: No partition changing allowed (I guess), but the kernel is so smooth and fluid with my kingshui build.
I guess my question would be this...
Can I remove apps from the app folder to shrink this build to the 100mb limit, or will this cause issues in the build itself?
Edit; it appears to be CGM 5.0.8 RLS11, which is Android 2.1 -update1
DURRR!

Still... it's better to answer for future generations.
The androidinstall.tar has at least on folder in it, named System.
It might have another folder in it, named Data.
It might even have other folders, ignore them for now.
Inside both system, there might be alot of folders (expecially in system), but one of them is called "App". Now, all the apps will be there, along with perhaps a .odex file for everyone.
You can move certains apps from /system/app to /data/app. (if /data does not exists, create /data and app inside it.) You can also remove things from /system/app, if you're not going to use them. Like, CMwallpapers.apk.
PoXFreak said:
Hi All...
Maybe a dumb question but I need an answer.
I was looking through the rest of the builds on kingshui's sourceforge site and something caught me off-guard.
Apparently he has some /system builds exceeding 100mb (the one I downloaded was 103mb, 98mb in a tarball).
I know with standard kernels we can slide the partition marker to the right and gain some /system space, but I'm currently using Scoot's cache kernel: No partition changing allowed (I guess), but the kernel is so smooth and fluid with my kingshui build.
I guess my question would be this...
Can I remove apps from the app folder to shrink this build to the 100mb limit, or will this cause issues in the build itself?
Edit; it appears to be CGM 5.0.8 RLS11, which is Android 2.1 -update1
DURRR!
Click to expand...
Click to collapse

Hey, I learn something new everyday!
I assume this would mean that if I needed to move apps from /system to /data, I would copy the files I wanted from /system/app and make a new folder for /data and put the apps in /data/app, along with the odex files if needed?
I did not know it was that easy. I always had a fear of ruining a build due o mis-matched apps being installed.

PoXFreak said:
Hey, I learn something new everyday!
I assume this would mean that if I needed to move apps from /system to /data, I would copy the files I wanted from /system/app and make a new folder for /data and put the apps in /data/app, along with the odex files if needed?
I did not know it was that easy. I always had a fear of ruining a build due o mis-matched apps being installed.
Click to expand...
Click to collapse
Be advised!!! Not every single package SHOULD/CAN be moved to DATA.
Examples:
Vending.apk -- Market... shouldn't.
Google*.apk -- Can't/Shouldn't
LatinIME.apk - I belive CAN'T.
Also... you never bumped into a release with the data folder already present ?
It's usually made so that you can remove apps by uninstalling them (Default launcher for example, in case you replaced it).
Also... why didn't you experiment ??

Sorry for the late reply, I got hung up in trying to memory map the HTC Raphael...
Anyway, I tend to think of it as "sacrelige" (sp) to mess with what others have already worked hard to build.
I will however take the "system_froyo_us_odex" package and modify it by changing the bootanimation.zip or wallpapers.apk to what I like. I have a CM theme pack (5.0.8 I believe), but I haven't gotten around to messing with it. It does however have a couple of certificates along with it, one being "manifest.cert", whic I really don't want to play with.
My Ubuntu machine doesn't like me since the latest kernel update to 11.04 and working on it has become a bear.
Que sera sera, another laptop needed...

PoXFreak said:
Sorry for the late reply, I got hung up in trying to memory map the HTC Raphael...
Anyway, I tend to think of it as "sacrelige" (sp) to mess with what others have already worked hard to build.
I will however take the "system_froyo_us_odex" package and modify it by changing the bootanimation.zip or wallpapers.apk to what I like. I have a CM theme pack (5.0.8 I believe), but I haven't gotten around to messing with it. It does however have a couple of certificates along with it, one being "manifest.cert", whic I really don't want to play with.
My Ubuntu machine doesn't like me since the latest kernel update to 11.04 and working on it has become a bear.
Que sera sera, another laptop needed...
Click to expand...
Click to collapse
It would be sacrilegio if you would dessecate a rom, and publish it as your own without credits. But for out own use, tweak away Just remember not to come asking on it's respective thread if the errors happening might be introduced with your tampering.
As for themes, it's not really my topic
About ubuntu... what machine are we talking about ?

An old Toshiba with an intel proc and realtek audio. I think it has to do with the touchpad (alps) because the system will freeze until I touch it.
As far as that CGM build, I can't use it becaus I'm set up for Froyo and that build is 2.1 update 1.

Develop Bugs

welcome