Develop Bugs

[DOC][DEV] More free space on /data for the Hero

Hey Devs,
As you may have noticed, I recently released a ROM with 276 MB free space on /data.
I accomplished this by using the MTD hack discovered by firerat and lbcoder.
It changes the kernel cmdline and includes addresses where the partitions are located.
This way, we are able to change the sizes.
That's exactly what I did.
The bad thing is that you need a recovery with a cmdline, which matches the ROM's boot.img's one!
So, a user needs to flash a recovery first and can then flash the ROM after a reboot into the new recovery.
Now, if the user wants to change his ROM to something else, he will need to revert the recovery.
It would be a lot easier if some other developers here would be interested in the MTD hack, so there is no need to revert.
I hope that some other developers will jump on the bandwagon now.
Here is the command I use for creating boot.img's, with modified partition sizes:
Code:
mkbootimg --kernel kernel --ramdisk ramdisk --cmdline 'mtdparts=msm_nand:[email protected](misc),[email protected](recovery),[email protected](boot),[email protected](system),[email protected](cache),[email protected](userdata)' -o boot.img --base 0x19200000
This reduces the /cache partition to 4 MB and makes /data as big as possible (the space which is left).
Now, the Market would normally fail to download huge APKs, because /cache is so small.
The problem can be easily solved by symlinking /cache to /data/cache and mounting the native cache partition on /dev/cache.
This can be done even without modifying the ramdisk, just do it in a script which runs before the Android frameworks start, like a2sd scripts.
Here are the necessary commands:
Code:
mount -o rw,remount /
umount /cache
rm -rf /cache
mkdir /data/cache
ln -s /data/cache /cache
mkdir /dev/cache
mount /dev/block/mtdblock4 /dev/cache
chmod 777 /data/cache
mount -o ro,remount /
Regarding the recovery:
Klothius from the Sapphire section created an update.zip which automatically patches the current recovery to use the provided cmdline.
You can find it here and include it in your instructions:
http://www.megaupload.com/?d=FY6CBAEE
I hope this little documentation will help you and will make you interested in doing this for your ROM.

Nice work. We'll look at implementing this into the next FroydVillain release. My only reservation would be keeping a wee bit more space on /cache than 4MB since our OTA app uses it. We could always hack it to use a dir on /data I suppose.

Hacre said:
Nice work. We'll look at implementing this into the next FroydVillain release. My only reservation would be keeping a wee bit more space on /cache than 4MB since our OTA app uses it. We could always hack it to use a dir on /data I suppose.
Click to expand...
Click to collapse
Yea, that would be better.. The whole purpose of this thread is to make things more compatible, means same cmdline Also you wll need to write the instructions for the recovery to /dev/cache then, keep that in mind
I will also add the RecoveryMod and an explanation in the documentation now.

I think the limitation of needing to switch recovery when changing ROMs is quite bothersome. Is there anyway we can get a recovery which supports both modes?

erasmux said:
I think the limitation of needing to switch recovery when changing ROMs is quite bothersome. Is there anyway we can get a recovery which supports both modes?
Click to expand...
Click to collapse
Sure, it can be done, but that would take a looot of work!

maxisma said:
Sure, it can be done, but that would take a looot of work!
Click to expand...
Click to collapse
More than I think it would be worth, to be honest. It'd make more sense for all the ROM developers to pull together on this and just make it a fact that we're giving more space to data, since it also eliminates the need to create fancy hacks moving dalvik to cache or to sdcard or splitting it between 2-3 file systems to cater for people who like to have the entire Android Market on their phone.
This has my/Villain's full support Maxisma, it makes perfect sense. Might I suggest you PM the other ROM developers here, or at least the popular ones like Fusion etc, and refer them to this thread so we can all work together on it.
Now that it's safe to assume that HTC have pretty much ditched the Hero, it makes sense for us to fix/improve -everything- including partition layout. There is a lot of wasted space in /cache that only gets used if a ROM developer starts fudging about with where dalvik lives, but /cache isn't quite big enough to completely migrate dalvik cache to for people who like to have a lot of apps.

Is there any more work on this? :d

shinyfong said:
Is there any more work on this? :d
Click to expand...
Click to collapse
I believe maxisma stopped work on it due to it breaking something or causing something not to work. I think he mentioned it in his Rom thread...

pulser_g2 said:
I believe maxisma stopped work on it due to it breaking something or causing something not to work. I think he mentioned it in his Rom thread...
Click to expand...
Click to collapse
It was me who broke it ;-) I accdidentally booted with a different mtd layout and thus killed all my data.
This method is perfectly fine though.
But I switched to a Nexus and I don't have a Hero anymore.

samsung clever but dirty hack used to fix "lag" in the epic 4g

they mount /dbdata/ as a tempfs in the epic 4g
epic 4g's init.rc: http://www.mediafire.com/?267r9xm77c6m3cn
################################################## #################
# create dbdata partition mount rfs /dev/block/stl7 /dbdata
mkdir /dbdata
# mount rfs /dev/block/stl10 /dbdata
mkdir /dbdata/databases
chown system system /dbdata
chown system system /dbdata/databases
chmod 0771 /dbdata
chmod 0777 /dbdata/databases

Good idea?
noobnl said:
they mount /dbdata/ as a tempfs in the epic 4g
epic 4g's init.rc: http://www.mediafire.com/?267r9xm77c6m3cn
################################################## #################
# create dbdata partition mount rfs /dev/block/stl7 /dbdata
mkdir /dbdata
# mount rfs /dev/block/stl10 /dbdata
mkdir /dbdata/databases
chown system system /dbdata
chown system system /dbdata/databases
chmod 0771 /dbdata
chmod 0777 /dbdata/databases
Click to expand...
Click to collapse
It would be good if someone could try this out and see if it is as advantageous as the currect lag fixes. It could also reduce the risk of damaging/shortening the life of the internal sd. If anyone is willing to try and post feedback + noob friendly adb instructions then it would be much appreciated.
PS. Good find

Sorry to go slightly OT, but since you have init.rc available, do you happen to know where we could download the Epic 4G kernel zImage as a whole? Thanks.

Case_ said:
Sorry to go slightly OT, but since you have init.rc available, do you happen to know where we could download the Epic 4G kernel zImage as a whole? Thanks.
Click to expand...
Click to collapse
we tried to extract the zimage ... permission denied

noobnl said:
we tried to extract the zimage ... permission denied
Click to expand...
Click to collapse
I see. Thanks anyway.

If Samsung is doing this I would've expected more excitement. Is it not a possible fix?

This is also in the JP3 firmware.
I checked last night after seeing this thread.

bratfink said:
It could also reduce the risk of damaging/shortening the life of the internal sd.
Click to expand...
Click to collapse
How does/do the current lagfix options available damage or shorten the life of the int sd?
Like to hear your logic.
Sent from my GT-I9000 using XDA App

Probably to do with the ext2 fs not being too reliable.

Ummm... maybe I'm missing something, but /dbdata is also mounted onto fast nand on all other galaxy s variants too. At least it is on mine.
What am I missing?

Ah nevermind! It became clearer after downloading the file. I missed the #comment
So they get around /dbdata being on slow RFS with a really quick dirty method.
I don't think they should be storing /dbdata on tmpfs/rootfs!
Do they really have something against modern filesystems..?
EDIT: I don't really get why they do this. /dbdata is used to store settings for the built in android apps, like maps etc, correct?
Won't this just mean that all of these settings will be lost whenever you restart your Epic?
Plus, since all 3rd-party apps are in /data, this doesn't really help anything?
Explanations, someone?!

Also, is there a reason samsung makes such a huge huge ram disk?
We only have 500mb of ram, and samsung is blowing 100mb+ on a stupid ram disk! Which is barely used!
Waarggh?!
I'm sure there's a good reason for all this..

RyanZA said:
Also, is there a reason samsung makes such a huge huge ram disk?
We only have 500mb of ram, and samsung is blowing 100mb+ on a stupid ram disk! Which is barely used!
Waarggh?!
I'm sure there's a good reason for all this..
Click to expand...
Click to collapse
There is a way to recover that 100mb(or at least a part) to every day use?

cba1986 said:
There is a way to recover that 100mb(or at least a part) to every day use?
Click to expand...
Click to collapse
mount /dbdata on /dev/block/stl10
i just saw that there are (commented) yaffs2 mounts?! so they changed it from yaffs2 to rfs??
@OP: is it fast? how much ram is available during normal use? any experiences you can share?

jodue said:
mount /dbdata on /dev/block/stl10
i just saw that there are (commented) yaffs2 mounts?! so they changed it from yaffs2 to rfs??
@OP: is it fast? how much ram is available during normal use? any experiences you can share?
Click to expand...
Click to collapse
Yeah I saw that too! I can't believe they were testing yaffs2 and then changed to RFS! I'd like to know the story behind that mess up!
As far as I can tell, the next project we should be doing is SHRINKING the stupid tmpfs ramdisk as low as we can get it, and trying to get the ram back into use by the system. I have no idea how to do that, or if it is even possible, but it should end up increasing our total ram from 300something to 400something!
Anybody have any extra info on this stupid tmpfs/ramfs, that is being horribly wasted on stuff like maps?

RyanZA said:
Also, is there a reason samsung makes such a huge huge ram disk?
We only have 500mb of ram, and samsung is blowing 100mb+ on a stupid ram disk! Which is barely used!
Waarggh?!
I'm sure there's a good reason for all this..
Click to expand...
Click to collapse
that is really, really stupid......
so they waste our limited 512mb of ram (thank you for solving the whole "why does it not show 512mb" mystery)
AND they were thinking of yaffs2 and didn't use it? I can understand why people hate samsung...
I hope someone can fix this crap.

BTW, the real reason for speed differences between epic and other galaxy devices is this line:
Galaxy:
mount rfs /dev/block/mmcblk0p2 /data nosuid nodev check=no
Epic:
mount rfs /dev/block/stl10 /data nosuid nodev check=no
So here is how epic and SGS differ:
SGS:
/dev/block/stl10 is mounted on /dbdata (127mb)
/dev/block/mmcblk0p2 is mounted on /data (2GB)
Epic:
ramfs for /dbdata (size unknown)
/dev/block/stl10 is mounted on /data (size unknown)
ie. The epic is using the fast nand for all data storage, while SGS is using the larger internal SD card. The epic will run faster than other SGS devices, but have less space. The epic should still experience the same lags that the SGS does, but the lags will be shorter due to increased hardware speed.
The Epic will benefit from an EXT2/other lagfix just as much as the SGS will.
Stuff to find out: NAND size in the epic, how much bigger is it than the NAND in SGS?
Epic+EXT2 lag fix, how high is the quadrant score?
How do we get rid of the stupid ram fs?!?

INeedYourHelp said:
AND they were thinking of yaffs2 and didn't use it?
Click to expand...
Click to collapse
After a bit more digging, it appears the yaffs2 is from the reference script for android. So the yaffs2 is probably not something that samsung was testing, but rather the recommendation from google. Samsung just changed it to their own filesystem... because they could, I guess.

jodue said:
mount /dbdata on /dev/block/stl10
i just saw that there are (commented) yaffs2 mounts?! so they changed it from yaffs2 to rfs??
@OP: is it fast? how much ram is available during normal use? any experiences you can share?
Click to expand...
Click to collapse
Normally i get between 110 and 125 Mb available of 326. With Widget locker, beatiful widget and mixget 1.1 running.
I have no complain about the speed in any application.
RyanZA said:
Yeah I saw that too! I can't believe they were testing yaffs2 and then changed to RFS! I'd like to know the story behind that mess up!
As far as I can tell, the next project we should be doing is SHRINKING the stupid tmpfs ramdisk as low as we can get it, and trying to get the ram back into use by the system. I have no idea how to do that, or if it is even possible, but it should end up increasing our total ram from 300something to 400something!
Anybody have any extra info on this stupid tmpfs/ramfs, that is being horribly wasted on stuff like maps?
Click to expand...
Click to collapse
If there is a way to get 400 Mb of RAM it would be great!!
I'm very interested in this.
I think Samsung wanted to break the Android limitations on the size of apps partitions and you know how it ended.

Ugh, it seems everyday that there are further revelation about how Samsung crippled their flagship Android phone. I believe the correct response is facepalm.gif
I really hope that Samsung are addressing this in their 2.2 update. I love the phone, I love the hardware and I love that the community is working hard to address these issues, but its up to Samsung now to provide a long term fix now.
Otherwise.... *looks around at other 4" Super AMOLED Android's*..... Oh.... darn.

Some possible insight on the cause of the RFS lag and work-around

Since I installed JPK on a test phone after claims that the RFS related lag has been fixed.. well.. I'd like to share my impressions:
- I have installed approx 100 apps after the flash (flashed / wiped / factory reset btw)
A this point, the phone while slightly slower than one with Voodoo is pretty fast
A few days later, it's still good enough.
- 5 days later, lag starts to be felt, and it's stronger every next day until it becomes annoying to use
That's exactly the same experience as I had on various Eclair roms without any lag-fix. I also had this on JP3 (early Froyo).
So yeah. I looked into the /init binary from samsung and it's supposed to make some file system checks from time to time when you restart the phone, but it does not appear to really carry on with that.
So I ran the checks myself:
- you need a rooted phone, and adb or a terminal
- find all RFS partitions:
$ su
# mount|grep -i rfs
- kill all processes, go flight mode and remount them read-only
$ su (if you're not root.. not going to repeat it again for subsequent commands)
# kill -9 <pid> of anything that use the patrition
# mount -oremount,ro /dev/block/....
- check the RFS filesystem and correct errors
# /system/bin/fsck_msdos -p -f /dev/block/....
Surprise, a million of RFS errors fixed such as:
/path/to/file starts with free cluster FIXED
Cluster XXX continues with cluster XXX in FAT 0 but is marked free in FAT 1 FIXED
Detected cluster chain loop head XXXX for p XXXX FIXED
FSNext block XXX is correct NumClusters XXX FIXED (weird one)
No LOST.DIR FIXED
Lost cluster chain at cluster XXX YY clusters lost FIXED (prolly lost files/data here!)
Repeat for each partition
reboot the phone at the end
SURPRISE! It doesn't lag much anymore.
Use the phone an hour, do the check again and you will see it's already full of errors.
Conclusion:
RFS is bugged (we knew that didn't we) but it looks like it's fixable, if ever Samsung figures out what is corrupting the file system exactly (it's closed source so we can't really find out easily)
It might be possible to figure it out by looking at VFAT sources too.
I would be very interested to see if that fixes the lag for everyone or if i'm an isolated case and all my RFS partitions are on bad hardware or if it's really software corruption as I am guessing
Small disclaimer:
im not responsible for any data loss etc. no warranties etc. fixing file system even readonly might cause data loss due to the bugs in RFS. you've been warned lol.

My God,
I think a lot of eyes will be on the BOUNTY

Could someone just create an app that does this?

bilboa1 said:
Since I installed JPK on a test phone after claims that the RFS related lag has been fixed.. well.. I'd like to share my impressions:
- I have installed approx 100 apps after the flash (flashed / wiped / factory reset btw)
A this point, the phone while slightly slower than one with Voodoo is pretty fast
A few days later, it's still good enough.
- 5 days later, lag starts to be felt, and it's stronger every next day until it becomes annoying to use
That's exactly the same experience as I had on various Eclair roms without any lag-fix. I also had this on JP3 (early Froyo).
So yeah. I looked into the /init binary from samsung and it's supposed to make some file system checks from time to time when you restart the phone, but it does not appear to really carry on with that.
So I ran the checks myself:
- you need a rooted phone, and adb or a terminal
- find all RFS partitions:
$ su
# mount|grep -i rfs
- kill all processes, go flight mode and remount them read-only
$ su (if you're not root.. not going to repeat it again for subsequent commands)
# kill -9 <pid> of anything that use the patrition
# mount -oremount,ro /dev/block/....
- check the RFS filesystem and correct errors
# /system/bin/fsck_msdos -p -f /dev/block/....
Surprise, a million of RFS errors fixed such as:
/path/to/file starts with free cluster FIXED
Cluster XXX continues with cluster XXX in FAT 0 but is marked free in FAT 1 FIXED
Detected cluster chain loop head XXXX for p XXXX FIXED
FSNext block XXX is correct NumClusters XXX FIXED (weird one)
No LOST.DIR FIXED
Lost cluster chain at cluster XXX YY clusters lost FIXED (prolly lost files/data here!)
Repeat for each partition
reboot the phone at the end
SURPRISE! It doesn't lag much anymore.
Use the phone an hour, do the check again and you will see it's already full of errors.
Conclusion:
RFS is bugged (we knew that didn't we) but it looks like it's fixable, if ever Samsung figures out what is corrupting the file system exactly (it's closed source so we can't really find out easily)
It might be possible to figure it out by looking at VFAT sources too.
I would be very interested to see if that fixes the lag for everyone or if i'm an isolated case and all my RFS partitions are on bad hardware or if it's really software corruption as I am guessing
Small disclaimer:
im not responsible for any data loss etc. no warranties etc. fixing file system even readonly might cause data loss due to the bugs in RFS. you've been warned lol.
Click to expand...
Click to collapse
VOLD is what does the RFS check at boot. It seems to run, although maybe it fails? It definitely does run though, you can check it running using 'ps' on phone boot. Maybe it's only cleaning up /sdcard though.
At any rate, running the disk check does help with speed, but it doesn't help that much. It's still slow. I think if you want to stick with RFS, you need to do a defrag as well as the filesystem check. After some use, RFS must be very very fragemented on most people's phones.
Even in perfect condition though, RFS still has some very nasty properties such as locking the entire disk when a write occurs, not doing buffering, etc etc.

INeedYourHelp said:
Could someone just create an app that does this?
Click to expand...
Click to collapse
An app can't do this, since the app would have to be running off RFS and would crash/have to be killed to perform the FS checks. It could be done on boot using some trickery and the playlogos / replace binary trick. Or it can be done by replacing the init script with some kernel hackery. But at that point, you might as well just use a real filesystem.
I guess a PC .bat file could be made that uses adb to do this for you and then reboot the phone... Doesn't seem worth the trouble though!

RFS is journelled. You sure the filesystem doesn't do the journaling properly?

Nice job figuring this out. Is someone forwarding all these findings to that Samsung dev?

It's not about RFS vs the other filesystems. I'm well aware of the performance of RFS. But it's decent enough when it's working properly. It's not nearly as fast as ext but fast enough that you don't get annoyed.
Thus fixing RFS would make the life of many people who aren't technically inclined better. If the RFS do get all corrupted everywhere, and Samsung figure that out and fixes it, it means a good thing for most people.
The rest of us will end up on ext anyways
And yeah I think the fscheck at boot fails, it must fail actually, i don't see how else it would happen.

bilboa1 said:
It's not about RFS vs the other filesystems. I'm well aware of the performance of RFS. But it's decent enough when it's working properly. It's not nearly as fast as ext but fast enough that you don't get annoyed.
Thus fixing RFS would make the life of many people who aren't technically inclined better. If the RFS do get all corrupted everywhere, and Samsung figure that out and fixes it, it means a good thing for most people.
The rest of us will end up on ext anyways
And yeah I think the fscheck at boot fails, it must fail actually, i don't see how else it would happen.
Click to expand...
Click to collapse
What firmware did you test on, btw? I've noticed that JPK does seem to take longer on the FS checks, so maybe they have it fixed already (doubt it though)?

RyanZA said:
What firmware did you test on, btw? I've noticed that JPK does seem to take longer on the FS checks, so maybe they have it fixed already (doubt it though)?
Click to expand...
Click to collapse
on JPK actually
im going to flash JM8 to see if its the same in fact, but i expect so

RyanZA said:
VOLD is what does the RFS check at boot. It seems to run, although maybe it fails? It definitely does run though, you can check it running using 'ps' on phone boot. Maybe it's only cleaning up /sdcard though.
Click to expand...
Click to collapse
I think VOLD only does the checks for the Internal SD and External SD, not the RFS partitions.
Interesting find though, about the FS errors.

I’ve been wondering why the lag appears after a couple of days. I suspected corruption in one way or another myself, as it stays after a reboot it could not have been RAM and there are no signs of running out of space. Thanks for your research and I hope it will lead to new fixes. Sad but true my fix is to reflash my phone almost weekly.
I run Doc's JPK super slim ROM which is really nice but still it lags, even with OCLF installed.
Yesterday I take my phone out of my pocket to take a quick photo. My phone wakes up and I sweep the glass. Halfway through the sweeping the animation stops. I wait patiently for a second or three and there my home screen is. (No widgets, no animations, just a single home screen with 12 icons on it of the applications I actually use). I click on the camera icon and I wait another 5 seconds for the camera to realize it is not supposed to sleep on duty. I make a photo, the actual moment is already gone by now but hey I have the thing in my hand. It just takes another 5 seconds to store the photo.
It is like being in a hurry with a toddler with you. You want to go quicker but you can’t get angry cause the little thing just can’t go faster. It has to stop and wonder about life every once in a while.
I like my phone and I am sure it will grow up.

Very interesting findings! Sure hope Samsung sees this or this is forwarded to some Samsung techs.

Maybe move this to the development forum then it might get more traffic from people that can actually help.

borchgrevink said:
Very interesting findings! Sure hope Samsung sees this or this is forwarded to some Samsung techs.
Click to expand...
Click to collapse
do you really think Samsung is reading every thread here on XDA?
Sent from my GT-I9000 using XDA App

matty___ said:
do you really think Samsung is reading every thread here on XDA?
Sent from my GT-I9000 using XDA App
Click to expand...
Click to collapse
No need to be uppity my friend ;-)

Some further info on possible RFS fixes here:
http://forum.xda-developers.com/showpost.php?p=8445244&postcount=143
With the sync issue fixed, and corruption fixed with checks, RFS might just work 'okay'! It won't be as fast as EXT, but it should still work fine if we can sort out all the bugs! And we have a much better chance of getting Samsung to include these fixes in a new firmware too.
If we could get hold of Samsung somehow, and manage to convince them that they should
1) disable the always sync behaviour
2) do full filesystem checks at boot up (or provide a utility to do the checks)
then RFS should be A LOT more usable!

Could someone perhaps turn this into a .bat-file?

RyanZA said:
Some further info on possible RFS fixes here:
http://forum.xda-developers.com/showpost.php?p=8445244&postcount=143
With the sync issue fixed, and corruption fixed with checks, RFS might just work 'okay'! It won't be as fast as EXT, but it should still work fine if we can sort out all the bugs! And we have a much better chance of getting Samsung to include these fixes in a new firmware too.
If we could get hold of Samsung somehow, and manage to convince them that they should
1) disable the always sync behaviour
2) do full filesystem checks at boot up (or provide a utility to do the checks)
then RFS should be A LOT more usable!
Click to expand...
Click to collapse
That quite right, I noticed the sync issue also.
As for the corruption, 2 things should be fixed:
- fsck from init should actually fsck the partitions properly on boot (we can sort of fix that by calling it in a script again)
- corruption should in theory not even happen so Samsung would have to work on RFS for that one
I ran JM8 for 2 days now and my rfs partitions are full of errors, like in JPK, just for confirmation if there needed to be one.
In fact it's still running fsck on /system as I'm writting and issues are filling the terminal .. its been running for 30s already lol

woeds said:
Could someone perhaps turn this into a .bat-file?
Click to expand...
Click to collapse
I haven't tried that but it might be easier to enable adb over usb (in development settings) then .. make sure you are rooted, type that:
> adb shell
$ su
#
<prompt on the phone for root, click allow>
reboot into recovery
> adb shell
$ su
# mount -oremount,ro /dev/block/stl9
# mount -oremount,ro /dev/block/stl10
# mount -oremount,ro /dev/block/stl11
<for safety i'm not including stl3 it's the EFS)
if there's any error due to "filesystem busy" them stop there, it means it doesn't work
otherwise:
#/system/bin/fsck_msdos -p -f /dev/block/stl9
#/system/bin/fsck_msdos -p -f /dev/block/stl10
#/system/bin/fsck_msdos -p -f /dev/block/stl11
# reboot

[HOWTO] Convert ext4 partitions to No-Journaling mode

WARNING: This procedure is risky and may result in loss of data.
This is a follow up to findings made in this thread: http://forum.xda-developers.com/showthread.php?t=819580
Many people have been asking for an ext2 lagfix. You can get something similar but (arguably) better: ext4 with no journaling. Ext4 no-journal performs as fast or faster than ext2 because of performance improvements made in ext4.
Quadrant comparison for ext4 /data-only lagfix:
ext4 scores about 1500, ext4 no-journal scores about 1650.
This should work with existing kernels that support ext4 lagfixes. It's tested on a ULFK kernel (SpeedMod).
WARNING: Turning off journaling makes your data more susceptible to getting corrupted, although the risk is small.
Disclaimer: No promises that this will work for you, or that it won't corrupt your data. Try this at your own risk.
Step 0: You start off by applying an ext4 lagfix. If you are already using an ext4 lagfix, you can skip this step.
For ULFK kernels, this is either:
- "Voodoo" ext4 /data
- No-RFS advanced ext4
After the lagfix has been successfully applied and your phone is up and running properly, then you can proceed to convert the ext4 partitions.
Step 1: Make a backup of your data, using CWM (recommended). If anything goes wrong, you can restore the backup later.
Step 2: Download the tune2fs file attached to this post (works for FROYO roms only), and copy it to /data as /data/tune2fs:
adb push tune2fs /sdcard/
adb shell
# su
# cp /sdcard/tune2fs /data/
Procedure if your kernel has ro.debuggable enabled:
Step 3: If your kernel has ro.debuggable enabled, then boot your phone into recovery mode. Then run adb in root mode:
adb root
(wait for adbd to restart)
adb shell
Copy tune2fs to the /tmp folder.
# cp /data/tune2fs /tmp/
If you don't have ro.debuggable enabled, "adb root" will give you an error. Go to Step 3A in the next section.
Step 4: Now in ADB shell, find out which partitions are ext4:
# mount | grep ext4
mount | grep ext4
/dev/block/mmcblk0p2 on /data type ext4 (rw,noatime,barrier=0,data=writeback,noauto_da_alloc)
/dev/block/stl10 on /dbdata type ext4 (rw,noatime,barrier=0,data=writeback,noauto_da_alloc)
/dev/block/stl11 on /cache type ext4 (rw,noatime,barrier=0,data=writeback,noauto_da_alloc)
In this example, the 3 partitions are:
/dev/block/mmcblk0p2 (/data)
/dev/block/stl10 (/dbdata)
/dev/block/stl11 (/cache)
Repeat Steps 5 to 9 for every partition you want to remove the journal from.
The next steps show the procedure for /dev/block/mmcblk0p2 (/data).
Step 5: Unmount the partition:
umount partition_mount_point
for example:
# umount /data
Step 6: Check if there is a journal:
# /tmp/tune2fs -l /dev/block/mmcblk0p2 | grep features
You should see something like this:
Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
You should see "has_journal" in the features. It means this partition has a journal.
Step 7: Fsck the partition:
# e2fsck -f /dev/block/mmcblk0p2
Step 8: Remove the journal:
# /tmp/tune2fs -O ^has_journal /dev/block/mmcblk0p2
(this is a capital "O"!)
Step 9: Check if the journal was removed:
# /tmp/tune2fs -l /dev/block/mmcblk0p2 | grep features
You should see something like this:
Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
You should see "has_journal" is NOT there.
Done for this partition.
Step 10: After you've remove the journal from all the partitions you wanted to, shutdown the phone by pressing the power button.
DONE. You only need to do this procedure once and it'll "stick" until the next time you re-format the partition.
--------------------------------------------------
Procedure if you don't have ro.debuggable enabled:
Step 3A: If you kernel does not have ro.debuggable enabled, then you can try doing this using normal adb with su while the phone is running. But this is much more risky.
To lower the risk, do this right after booting, wait for the Media Scan to complete.
adb shell
# su
Step 4A: Now in ADB shell, find out which partitions are ext4:
# mount | grep ext4
mount | grep ext4
/dev/block/mmcblk0p2 on /data type ext4 (rw,noatime,barrier=0,data=writeback,noauto_da_alloc)
/dev/block/stl10 on /dbdata type ext4 (rw,noatime,barrier=0,data=writeback,noauto_da_alloc)
/dev/block/stl11 on /cache type ext4 (rw,noatime,barrier=0,data=writeback,noauto_da_alloc)
In this example, the 3 partitions are:
/dev/block/mmcblk0p2 (/data)
/dev/block/stl10 (/dbdata)
/dev/block/stl11 (/cache)
Repeat Steps 5A to 8A for every partition you want to remove the journal from.
The next steps show the procedure for /dev/block/mmcblk0p2 (/data).
Step 5A: Check if there is a journal:
# /data/tune2fs -l /dev/block/mmcblk0p2 | grep features
You should see something like this:
Filesystem features: has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
You should see "has_journal" in the features. It means this partition has a journal.
Step 6A: Fsck the partition:
# e2fsck -f /dev/block/mmcblk0p2
WARNING!!! Running e2fsck on a mounted filesystem may cause
SEVERE filesystem damage.
Do you really want to continue (y/n)?
Answer yes.
Step 7A: Remove the journal:
# /data/tune2fs -O ^has_journal /dev/block/mmcblk0p2
(this is a capital "O"!)
Step 8A: Check if the journal was removed:
# /data/tune2fs -l /dev/block/mmcblk0p2 | grep features
You should see something like this:
Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
You should see "has_journal" is NOT there.
Done for this partition.
Step 9A: After you've remove the journal from all the partitions you wanted to, shutdown the phone by pressing the power button. Reboot the phone and hope everything works.
DONE. You only need to do this procedure once and it'll "stick" until the next time you re-format the partition.

Nice one!
Going to try this tomorrow!
__________________________
Device: GT-I9000 16GB
Rom: Doc's V8 Nude&Raw v3
Kernel: Hardcore's Speedmod K9A - 500 Hz
Modem: JPP
Lagfix: No RFS, ext4-ext2, no binds
Tweaks: I/O Sched, Kernel VM Management, Kernel sched

Please ignore.

danzgrace said:
http://www.mjmwired.net/kernel/Documentation/filesystems/ext4.txt#364
Data Mode
=========
There are 3 different data modes:
* writeback mode
In data=writeback mode, ext4 does not journal data at all. This mode provides a similar level of journaling as that of XFS, JFS, and ReiserFS in its default mode - metadata journaling. A crash+recovery can cause incorrect data to appear in files which were written shortly before the crash. This mode will typically provide the best ext4 performance.
* ordered mode
In data=ordered mode, ext4 only officially journals metadata, but it logically groups metadata information related to data changes with the data blocks into a single unit called a transaction. When it's time to write the new metadata out to disk, the associated data blocks are written first. In general,
this mode performs slightly slower than writeback but significantly faster than journal mode.
* journal mode
data=journal mode provides full data and metadata journaling. All new data is written to the journal first, and then to its final location.
In the event of a crash, the journal can be replayed, bringing both data and metadata into a consistent state. This mode is the slowest except when data needs to be read from and written to disk at the same time where it outperforms all others modes. Currently ext4 does not have delayed allocation support if this data journalling mode is selected.
Click to expand...
Click to collapse
Applying writeback mode is the best option.

danzgrace said:
Applying writeback mode is the best option.
Click to expand...
Click to collapse
Hi danzgrace. Those options you quoted are for journaled modes. data=writeback is still a journaled mode.
This procedure is for non-journaled mode (none of the above). Its faster than all of the above because there is *no* journal.

hardcore said:
Hi danzgrace. Those options you quoted are for journaled modes. data=writeback is still a journaled mode.
This procedure is for non-journaled mode (none of the above). Its faster than all of the above because there is *no* journal.
Click to expand...
Click to collapse
Yes. Just stumbled upon lots of docu about ext4 and you are right about the performance gain when there's no journal involve but kind of risky.
https://ext4.wiki.kernel.org/index.php/Ext4_Howto#.22No_Journaling.22_mode

danzgrace said:
Yes. Just stumbled upon lots of docu about ext4 and you are right about the performance gain when there's no journal involve but kind of risky.
https://ext4.wiki.kernel.org/index.php/Ext4_Howto#.22No_Journaling.22_mode
Click to expand...
Click to collapse
It should be no more risky than ext2. Been using it for a few days now and so far so good.

danzgrace said:
Yes. Just stumbled upon lots of docu about ext4 and you are right about the performance gain when there's no journal involve but kind of risky.
https://ext4.wiki.kernel.org/index.php/Ext4_Howto#.22No_Journaling.22_mode
Click to expand...
Click to collapse
But its not much more risky because writeback journal does not prevent data loss - it just removes the need of a filecheck after unclean unmounting. Data still can be lost or?
Sent from my GT-I9000 using XDA App

Can you implement this tweak in speedmod kernel?
Sent from my GT-I9000 using XDA App

IMO, keeping the journal, with the default data=ordered option, is a good safety vs. speed compromise.
However, have you tried fiddling with the commit= mount option? The default commit interval is 5 s - this could be what's detrimental to battery life (a wake every 5 seconds). Maybe upping this to 60, or even more, seconds? I myself wouldn't care if it were 5 minutes on a phone, but upping it to one minute should already show a significant battery life increase.

corgar said:
Can you implement this tweak in speedmod kernel?
Sent from my GT-I9000 using XDA App
Click to expand...
Click to collapse
Not that easy because I have to modify the recovery executable. Maybe at a later stage.

AnnihilatorSC said:
IMO, keeping the journal, with the default data=ordered option, is a good safety vs. speed compromise.
However, have you tried fiddling with the commit= mount option? The default commit interval is 5 s - this could be what's detrimental to battery life (a wake every 5 seconds). Maybe upping this to 60, or even more, seconds?
Click to expand...
Click to collapse
I'm not sure if this has any effect when there's no journal. But I guess it's worth a shot.

I hope there is a way to do this in a script and apply by update.zip in recovery. Please tell me this can be done...
In general use, just how much faster does this feel? I'm asking because regular ext4 does not feel much faster in general use than RFS using speedmod. Back when I was using JM8 + OCLF, k-9 Email was so much faster.

hardcore said:
I'm not sure if this has any effect when there's no journal. But I guess it's worth a shot.
Click to expand...
Click to collapse
It doesn't. But I aim to hang on to the journal.
You could try releasing a SpeedMod revision which mounts all ext4 fs's with commit=60. If you do, i'll be sure to report the battery life - the effect should already be obvious to me after a night on battery (I currently lose about 20% battery overnight - no-rfs advanced ext4, w/ defaults).

hardcore said:
I'm not sure if this has any effect when there's no journal. But I guess it's worth a shot.
Click to expand...
Click to collapse
I guess commit=60 or more and journal_async_commit, can save us a lot of battery power.

you can change the commit options by remounting the disk.. in adb or terminal with root..
mount -oremount,data=writeback,journal_async_commit,... /dev/block/... /data (for example)
you will lose it at phone reboot, too - but good for testing

bilboa1 said:
you can change the commit options by remounting the disk.. in adb or terminal with root..
mount -oremount,data=writeback,journal_async_commit,... /dev/block/... /data (for example)
you will lose it at phone reboot, too - but good for testing
Click to expand...
Click to collapse
not unless it was on your playlogos hack.

danzgrace said:
journal_async_commit, can save us a lot of battery power.
Click to expand...
Click to collapse
I sincerely doubt journal_async_commit would bring about better power utilization. Maybe a slight performance increase, but nothing significant. Personally, I wouldn't enable that option.

danzgrace said:
not unless it was on your playlogos hack.
Click to expand...
Click to collapse
SpeedMod and ULFK supports init scripts, without the playlogos hack.
Just name the script E_something.sh and place it in /system/etc/init.d/
For example /system/etc/init.d/E_startup.sh

hardcore said:
SpeedMod and ULFK supports init scripts, without the playlogos hack.
Just name the script E_something.sh and place it in /system/etc/init.d/
For example /system/etc/init.d/E_startup.sh
Click to expand...
Click to collapse
Thanks for the info. imho. I feel more safe putting the script on /data partition so if it fails I just factory reset rather than putting it on /system dir.

[Guide]Functional ext4 for external microSD with just a few bumps left

Y.G. said:
I formated my sd card to Ext4 and when insert it in to my phone, it says that's it's blank and has unsupported files. Any reasons for that?
Sent from my SPH-L710 using xda premium
Click to expand...
Click to collapse
The SPH-L710 Samsung Stock LJ7 TW 4.1.1 Android doesn't understand/support the change to ext4 external SD card (microSD) without a few things being done.
I'm working this out right now. So far I have manually been able to mount the newly created ext4 partition on the microSD card through adb, and after some chown/chmod I was able to go back to "Settings and Storage" and the "Mount SD Card" picked it up, and I was up and running ext4. But this didn't persist after a restart. So I'm looking into: /etc/vold.fstab MODS to keep it after restart right Now !!
If Some one else already has this perfected please chime in. I'm wanting to do most of the devices in the house this way when I get time because better performance, having a file system with a journal, and getting rid of thins like 4 Gig per file limitations is pretty Sweet in my humble opinion *Grin*
0) Assuming you already have your microSD card formatted ext4. I also happened to label mine extSdCard for the volume label within gparted
1) Can mount with:
mount -w -t ext4 /dev/block/mmcblk1p1 /storage/extSdCard/
2) To get the correct owner and permissions run:
chown root:sdcard_rw /storage/extSdCard
chmod 775 /storage/extSdCard
3) Should make the extSdCard owner/permissons match the regular internal sdcard you can verify this like so:
cd /storage/ && ls -l
drwxrwxr-x root sdcard_rw 2013-01-12 18:16 extSdCard
drwxrwxr-x root sdcard_rw 2013-01-12 17:05 sdcard0
4) After that you can go to the "Settings and Storage" to run "Mount SD" and you will have ext4 extSdCard Show up and it bring up the File System Status !! --> Until you reboot and it goes to crap because I don't have the vold.fstab edit/MOD complete _yet_ ... So, for now a boot script has been put in place to bring our external SD card back online during restart, so the system will acknowledges it, making the world a better place.
Example of how things look file system wise: mount | grep extSdCard
/dev/block/mmcblk1p1 /storage/extSdCard ext4 rw,relatime,user_xattr,barrier=1,data=ordered 0 0
5) Have not been able to resolve the vold.fstab to make this ext4 extSdCard matter fully Legit (in my opinion), but I did manage to make it remount the card on boot, so its online when the system comes up instead of having to manually mount it. Did this by -->
Added the following lines to the very bottom of: /etc/init.qcom.post_fs.sh
## sponix MOD to match with ktoonz kernel for better power management
stop mpdecision
## sponix MOD to mount extSdCard prior to GUI work around to make ext4 function
## read and write extSdCard mount
chown root:sdcard_rw /storage/extSdCard
chmod 775 /storage/extSdCard
mount -w -t ext4 /dev/block/mmcblk1p1 /storage/extSdCard
chown root:sdcard_rw /storage/extSdCard
chmod 775 /storage/extSdCard
## if you want read only extSdCard mount
## mount -r -t ext4 /dev/block/mmcblk1p1 /storage/extSdCard
Still attempting to automate the process so the Stock+root LJ7 can pick up the extSdCard _normally_ without having to do the mount command manually, but so far its kicking my butt. Also this is more a "General, or Question and Answer type Topic" the Kernel(s) obviously support ext4 the system fs uses/requires it *Grin*.. So we might get Our Friendly Neighborhood Moderator to Migrate it to the proper place to help others. Just hoping to get the last few bumps smoothed out, or find someone that already documented the process that I've overlooked *Grin*..
Current Known Issues: If you unmount the card through the "Settings | Storage | Umount SD" or by hand with umount, you will either need to reboot for it to reattach through the /etc/init.qcom.post_fs.sh boot script script addition, or will have to mount it manually if you want to keep the system up and running. Guess you could also probably just run the /etc/init.qcom.post_fs.sh as root from a terminal emulator (or adb).
Still searching for vold.fstab bits of wisdom but that will have to continue next weekend -->
Sexy and You Know it,
Keep on Flashing,
sponix2ipfw (sponix
:fingers-crossed:

Ha! Sorry. Deleted: Didn't understand that you had it running on boot (can't read properly )

Great idea
Am I really the only one who also thinks this idea is the nuts?
Am I the only one who longs to transform the mess that passes for a filing system on the internal sd using symbolic links into a beautifully organized, encrypted and cloud synced system on my external sd?
Is it just me and a few others that want to be able to achieve the above so that we can move from one ROM to another or recover from a lost phone with the minimum of fuss?
Are we freaks? :cyclops:
Say it isn't so XDA!! :crying:
I'm gonna try this on my international S3 running Null_ Rom 25 JB 4.1.2
PS do you have any idea how the entire ExtSD or just a folder can be enrypted using Cryptonite and automatically mounted at boot time?

emp111 said:
PS do you have any idea how the entire ExtSD or just a folder can be enrypted using Cryptonite and automatically mounted at boot time?
Click to expand...
Click to collapse
Is this along the lines of what you're looking for?
http://forum.xda-developers.com/showthread.php?t=1141467
Also your idea is pretty insane, but also genius
If you get that to work please do come back and share

Insane ideas are the best lol
CNexus said:
Is this along the lines of what you're looking for?
http://forum.xda-developers.com/showthread.php?t=1141467
Also your idea is pretty insane, but also genius
If you get that to work please do come back and share
Click to expand...
Click to collapse
Thank you for your prompt reply, yes it is asking a lot I know but I think that it can be done.
Now if you really thought that idea was insane......check this out:
Imagine that we asked every Android app developer to submit the various paths used for their config (and config backup) files to a central database and had the ability to add or own custom paths (which could be added to the central database once approved).
We could build an script/app that would retrieve a list of currently installed apps on your phone then automatically build a symbolically linked file system (and/or backup file system) in the location of your choice that you could either encrypt and/or sync using your current tools or even incorporate this functionality into the app itself along with the ability to choose what was encrypted/backed up and how i.e. either synced to the Cloud or (S)FTP or SMB as either a dd copy or even a cwm flashable zip.
Could I dare hope for a Tasker module or the ability to add custom scripts?
I wish I could do this myself but my coding skills are non existent
Anyway the LUKS manager app won't automatically mount a file system, but I really like it anyways, thank you for pointing me to it!
And on the Ext4 front, the mount command (yes the 1st one ) failed, maybe the op could offer a suggestion. :angel:
BTW is there a place for people to suggest ideas for apps here?

Wait really it wont? I couldve sworn I remembering that it did
But dude....
You need to learn yourself some java and start whipping stuff up
Idk about the whole central database thing, but the rest could definitely be done with root access
I think the main problem with that is the proprietary aspects...i mean even here on XDA where binaries released are supposed to be GPL compliant, many of them arent and its sad because it deteriorates the overall quality of work thats released afterward

This whole thing is just hard work!
CNexus said:
Wait really it wont? I couldve sworn I remembering that it did
Click to expand...
Click to collapse
Doesn't seem to unfortunately
But dude....
You need to learn yourself some java and start whipping stuff up :D :D[/QUOTE said:
You make it sound soooo easy lol, and at another point in my life maybe it would have been but right now I'm operating at a reduced level due to some unforeseen circumstances that have left me lacking focus, motivation etc
You know all the things you need to be creative, learn etc lol
Anyway back to the matter at hand, I have got my ext4 SD card to the stage where I have to manually mount it from within the Settings/Storage as I'm using the international S3 and don't have the init.qcom.post_fs.sh, I think the qcom refers to Qualcomm chipset in US S3's.
As for modifying vold.fstab so we can avoid the above workaround it would seem that maybe thats a dead end as according to a German guy on android-hilfe. de, Vold may have been modified by Samsung to only deadl with exFAT on External SD's.
Looks like I'm not gonna be in Android nirvana for a while :crying:
Unless anyone else on XDA fancies getting in on this !!!!
Click to expand...
Click to collapse

Got it working ..... kinda
Got an app called ezymount (by ezynow) that automounts my ext4 64GB microSD at boot time.
I have to wait a few seconds for the boot process to complete but it's automatic, am pretty happy!!
Now gotta get symlinks, encryption and cloud synchronization sorted :/