Related
Since I installed JPK on a test phone after claims that the RFS related lag has been fixed.. well.. I'd like to share my impressions:
- I have installed approx 100 apps after the flash (flashed / wiped / factory reset btw)
A this point, the phone while slightly slower than one with Voodoo is pretty fast
A few days later, it's still good enough.
- 5 days later, lag starts to be felt, and it's stronger every next day until it becomes annoying to use
That's exactly the same experience as I had on various Eclair roms without any lag-fix. I also had this on JP3 (early Froyo).
So yeah. I looked into the /init binary from samsung and it's supposed to make some file system checks from time to time when you restart the phone, but it does not appear to really carry on with that.
So I ran the checks myself:
- you need a rooted phone, and adb or a terminal
- find all RFS partitions:
$ su
# mount|grep -i rfs
- kill all processes, go flight mode and remount them read-only
$ su (if you're not root.. not going to repeat it again for subsequent commands)
# kill -9 <pid> of anything that use the patrition
# mount -oremount,ro /dev/block/....
- check the RFS filesystem and correct errors
# /system/bin/fsck_msdos -p -f /dev/block/....
Surprise, a million of RFS errors fixed such as:
/path/to/file starts with free cluster FIXED
Cluster XXX continues with cluster XXX in FAT 0 but is marked free in FAT 1 FIXED
Detected cluster chain loop head XXXX for p XXXX FIXED
FSNext block XXX is correct NumClusters XXX FIXED (weird one)
No LOST.DIR FIXED
Lost cluster chain at cluster XXX YY clusters lost FIXED (prolly lost files/data here!)
Repeat for each partition
reboot the phone at the end
SURPRISE! It doesn't lag much anymore.
Use the phone an hour, do the check again and you will see it's already full of errors.
Conclusion:
RFS is bugged (we knew that didn't we) but it looks like it's fixable, if ever Samsung figures out what is corrupting the file system exactly (it's closed source so we can't really find out easily)
It might be possible to figure it out by looking at VFAT sources too.
I would be very interested to see if that fixes the lag for everyone or if i'm an isolated case and all my RFS partitions are on bad hardware or if it's really software corruption as I am guessing
Small disclaimer:
im not responsible for any data loss etc. no warranties etc. fixing file system even readonly might cause data loss due to the bugs in RFS. you've been warned lol.
My God,
I think a lot of eyes will be on the BOUNTY
Could someone just create an app that does this?
bilboa1 said:
Since I installed JPK on a test phone after claims that the RFS related lag has been fixed.. well.. I'd like to share my impressions:
- I have installed approx 100 apps after the flash (flashed / wiped / factory reset btw)
A this point, the phone while slightly slower than one with Voodoo is pretty fast
A few days later, it's still good enough.
- 5 days later, lag starts to be felt, and it's stronger every next day until it becomes annoying to use
That's exactly the same experience as I had on various Eclair roms without any lag-fix. I also had this on JP3 (early Froyo).
So yeah. I looked into the /init binary from samsung and it's supposed to make some file system checks from time to time when you restart the phone, but it does not appear to really carry on with that.
So I ran the checks myself:
- you need a rooted phone, and adb or a terminal
- find all RFS partitions:
$ su
# mount|grep -i rfs
- kill all processes, go flight mode and remount them read-only
$ su (if you're not root.. not going to repeat it again for subsequent commands)
# kill -9 <pid> of anything that use the patrition
# mount -oremount,ro /dev/block/....
- check the RFS filesystem and correct errors
# /system/bin/fsck_msdos -p -f /dev/block/....
Surprise, a million of RFS errors fixed such as:
/path/to/file starts with free cluster FIXED
Cluster XXX continues with cluster XXX in FAT 0 but is marked free in FAT 1 FIXED
Detected cluster chain loop head XXXX for p XXXX FIXED
FSNext block XXX is correct NumClusters XXX FIXED (weird one)
No LOST.DIR FIXED
Lost cluster chain at cluster XXX YY clusters lost FIXED (prolly lost files/data here!)
Repeat for each partition
reboot the phone at the end
SURPRISE! It doesn't lag much anymore.
Use the phone an hour, do the check again and you will see it's already full of errors.
Conclusion:
RFS is bugged (we knew that didn't we) but it looks like it's fixable, if ever Samsung figures out what is corrupting the file system exactly (it's closed source so we can't really find out easily)
It might be possible to figure it out by looking at VFAT sources too.
I would be very interested to see if that fixes the lag for everyone or if i'm an isolated case and all my RFS partitions are on bad hardware or if it's really software corruption as I am guessing
Small disclaimer:
im not responsible for any data loss etc. no warranties etc. fixing file system even readonly might cause data loss due to the bugs in RFS. you've been warned lol.
Click to expand...
Click to collapse
VOLD is what does the RFS check at boot. It seems to run, although maybe it fails? It definitely does run though, you can check it running using 'ps' on phone boot. Maybe it's only cleaning up /sdcard though.
At any rate, running the disk check does help with speed, but it doesn't help that much. It's still slow. I think if you want to stick with RFS, you need to do a defrag as well as the filesystem check. After some use, RFS must be very very fragemented on most people's phones.
Even in perfect condition though, RFS still has some very nasty properties such as locking the entire disk when a write occurs, not doing buffering, etc etc.
INeedYourHelp said:
Could someone just create an app that does this?
Click to expand...
Click to collapse
An app can't do this, since the app would have to be running off RFS and would crash/have to be killed to perform the FS checks. It could be done on boot using some trickery and the playlogos / replace binary trick. Or it can be done by replacing the init script with some kernel hackery. But at that point, you might as well just use a real filesystem.
I guess a PC .bat file could be made that uses adb to do this for you and then reboot the phone... Doesn't seem worth the trouble though!
RFS is journelled. You sure the filesystem doesn't do the journaling properly?
Nice job figuring this out. Is someone forwarding all these findings to that Samsung dev?
It's not about RFS vs the other filesystems. I'm well aware of the performance of RFS. But it's decent enough when it's working properly. It's not nearly as fast as ext but fast enough that you don't get annoyed.
Thus fixing RFS would make the life of many people who aren't technically inclined better. If the RFS do get all corrupted everywhere, and Samsung figure that out and fixes it, it means a good thing for most people.
The rest of us will end up on ext anyways
And yeah I think the fscheck at boot fails, it must fail actually, i don't see how else it would happen.
bilboa1 said:
It's not about RFS vs the other filesystems. I'm well aware of the performance of RFS. But it's decent enough when it's working properly. It's not nearly as fast as ext but fast enough that you don't get annoyed.
Thus fixing RFS would make the life of many people who aren't technically inclined better. If the RFS do get all corrupted everywhere, and Samsung figure that out and fixes it, it means a good thing for most people.
The rest of us will end up on ext anyways
And yeah I think the fscheck at boot fails, it must fail actually, i don't see how else it would happen.
Click to expand...
Click to collapse
What firmware did you test on, btw? I've noticed that JPK does seem to take longer on the FS checks, so maybe they have it fixed already (doubt it though)?
RyanZA said:
What firmware did you test on, btw? I've noticed that JPK does seem to take longer on the FS checks, so maybe they have it fixed already (doubt it though)?
Click to expand...
Click to collapse
on JPK actually
im going to flash JM8 to see if its the same in fact, but i expect so
RyanZA said:
VOLD is what does the RFS check at boot. It seems to run, although maybe it fails? It definitely does run though, you can check it running using 'ps' on phone boot. Maybe it's only cleaning up /sdcard though.
Click to expand...
Click to collapse
I think VOLD only does the checks for the Internal SD and External SD, not the RFS partitions.
Interesting find though, about the FS errors.
I’ve been wondering why the lag appears after a couple of days. I suspected corruption in one way or another myself, as it stays after a reboot it could not have been RAM and there are no signs of running out of space. Thanks for your research and I hope it will lead to new fixes. Sad but true my fix is to reflash my phone almost weekly.
I run Doc's JPK super slim ROM which is really nice but still it lags, even with OCLF installed.
Yesterday I take my phone out of my pocket to take a quick photo. My phone wakes up and I sweep the glass. Halfway through the sweeping the animation stops. I wait patiently for a second or three and there my home screen is. (No widgets, no animations, just a single home screen with 12 icons on it of the applications I actually use). I click on the camera icon and I wait another 5 seconds for the camera to realize it is not supposed to sleep on duty. I make a photo, the actual moment is already gone by now but hey I have the thing in my hand. It just takes another 5 seconds to store the photo.
It is like being in a hurry with a toddler with you. You want to go quicker but you can’t get angry cause the little thing just can’t go faster. It has to stop and wonder about life every once in a while.
I like my phone and I am sure it will grow up.
Very interesting findings! Sure hope Samsung sees this or this is forwarded to some Samsung techs.
Maybe move this to the development forum then it might get more traffic from people that can actually help.
borchgrevink said:
Very interesting findings! Sure hope Samsung sees this or this is forwarded to some Samsung techs.
Click to expand...
Click to collapse
do you really think Samsung is reading every thread here on XDA?
Sent from my GT-I9000 using XDA App
matty___ said:
do you really think Samsung is reading every thread here on XDA?
Sent from my GT-I9000 using XDA App
Click to expand...
Click to collapse
No need to be uppity my friend ;-)
Some further info on possible RFS fixes here:
http://forum.xda-developers.com/showpost.php?p=8445244&postcount=143
With the sync issue fixed, and corruption fixed with checks, RFS might just work 'okay'! It won't be as fast as EXT, but it should still work fine if we can sort out all the bugs! And we have a much better chance of getting Samsung to include these fixes in a new firmware too.
If we could get hold of Samsung somehow, and manage to convince them that they should
1) disable the always sync behaviour
2) do full filesystem checks at boot up (or provide a utility to do the checks)
then RFS should be A LOT more usable!
Could someone perhaps turn this into a .bat-file?
RyanZA said:
Some further info on possible RFS fixes here:
http://forum.xda-developers.com/showpost.php?p=8445244&postcount=143
With the sync issue fixed, and corruption fixed with checks, RFS might just work 'okay'! It won't be as fast as EXT, but it should still work fine if we can sort out all the bugs! And we have a much better chance of getting Samsung to include these fixes in a new firmware too.
If we could get hold of Samsung somehow, and manage to convince them that they should
1) disable the always sync behaviour
2) do full filesystem checks at boot up (or provide a utility to do the checks)
then RFS should be A LOT more usable!
Click to expand...
Click to collapse
That quite right, I noticed the sync issue also.
As for the corruption, 2 things should be fixed:
- fsck from init should actually fsck the partitions properly on boot (we can sort of fix that by calling it in a script again)
- corruption should in theory not even happen so Samsung would have to work on RFS for that one
I ran JM8 for 2 days now and my rfs partitions are full of errors, like in JPK, just for confirmation if there needed to be one.
In fact it's still running fsck on /system as I'm writting and issues are filling the terminal .. its been running for 30s already lol
woeds said:
Could someone perhaps turn this into a .bat-file?
Click to expand...
Click to collapse
I haven't tried that but it might be easier to enable adb over usb (in development settings) then .. make sure you are rooted, type that:
> adb shell
$ su
#
<prompt on the phone for root, click allow>
reboot into recovery
> adb shell
$ su
# mount -oremount,ro /dev/block/stl9
# mount -oremount,ro /dev/block/stl10
# mount -oremount,ro /dev/block/stl11
<for safety i'm not including stl3 it's the EFS)
if there's any error due to "filesystem busy" them stop there, it means it doesn't work
otherwise:
#/system/bin/fsck_msdos -p -f /dev/block/stl9
#/system/bin/fsck_msdos -p -f /dev/block/stl10
#/system/bin/fsck_msdos -p -f /dev/block/stl11
# reboot
Hi guys,
Now that several lagfixes are using the ext4 filesystem, perhaps we should look at optimizing ext4 for performance and lifespan of the flash memory.
One of the main things we can change is the journaling method to data=writeback. This should reduce writes and improve performance at a slight expense of reliability. Quote:
"In data=writeback mode, ext4 does not journal data at all. This mode provides a similar level of journaling as that of XFS, JFS, and ReiserFS in its default mode - metadata journaling. A crash+recovery can cause incorrect data to appear in files which were written shortly before the crash. This mode will typically provide the best ext4 performance."
One explanation here:
http://blog.smartlogicsolutions.com...ions-to-improve-ext4-file-system-performance/
Reference here:
http://git.kernel.org/?p=linux/kern...02ac5fa36d7f4c07856fe9cf89391e08986f7;hb=HEAD
HOW TO
1. Change the default mount option of the partition using tune2fs. You need to use a version of tune2fs that supports ext4, the one in busybox 1.17 does NOT. A working version is attached to this post.
- Push the tune2fs file to the phone's SD CARD:
adb push tune2fs /sdcard/
- Copy the tune2fs file to /data/
adb shell
su
cp /sdcard/tune2fs /data/
Now change the options:
/data/tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
To verify:
/data/tune2fs -l /dev/block/mmcblk0p2
Look for the line that says:
Default mount options: journal_data_writeback
* Repeat the above for all other ext4 partitions.
It should take effect after a reboot. The next time it will be mounted automatically with data=writeback. You might have to do the tune2fs settings and reboot twice to get it to stick - I'm not sure why, but I had to.
You can verify this using the mount command:
# busybox mount | grep ext4
/dev/block/mmcblk0p2 on /data type ext4 (rw,noatime,barrier=0,nobh,data=writeback,noauto_da_alloc)
/dev/block/stl10 on /dbdata type ext4 (rw,noatime,barrier=0,nobh,data=writeback,noauto_da_alloc)
2. In addition to that, you can also edit the mount options to include the nobh option, which is a further minor optimization for data=writeback mode.
I personally use the options noatime,barrier=0,nobh,data=writeback. Voodoo already uses some of them like noatime and barrier=0.
I do it using a startup script (you need to know how to use/modify a startup script), with the following commands:
for k in $(busybox mount | grep ext4 | cut -d " " -f3)
do
sync
busybox mount -o remount,barrier=0,nobh $k
done
EDIT: Attached tune2fs that supports ext4. Works on Froyo kernels with ext4 support.
I posted the same request in another thread and i though it will be better if i do it here.
I found e2fsprogs-ext4.zip, but not sure if this is the correct zip.
it will be great if you can provide it here.
For others willing to try it, please change the /dev/block/mmcblk0pX to whatever you are using now.
my system is using /dev/block/mmcblk0p4
s88 said:
I posted the same request in another thread and i though it will be better if i do it here.
I found e2fsprogs-ext4.zip, but not sure if this is the correct zip.
it will be great if you can provide it here.
Click to expand...
Click to collapse
I've attached it here, it worked for me with the Universal Lagfix Froyo kernel that supports ext4.
hardcore said:
I've attached it here, it worked for me with the Universal Lagfix Froyo kernel that supports ext4.
Click to expand...
Click to collapse
And did you notice any improvements with new mount options?
busybox 1.17.1 says invalid option -o on tune2fs
==
i need the version attached to this thread, busybox version only supports ext2 and ext3
danzel said:
busybox 1.17.1 says invalid option -o on tune2fs
Click to expand...
Click to collapse
copy tune2fs into /system/xbin and chmod 755
Write back will cause data loses on crashes and it will increase read time on read miss since you need to write the block back to the main mem before reading a new block.
how & where do i edit mount options?
mdalacu said:
And did you notice any improvements with new mount options?
Click to expand...
Click to collapse
I didn't do any benchmarks, but in theory it should be better - reduced periodic journal writes (better battery life, flash lifespan and performance).
I don't know about performance but my SGS feels like a little bit snappier.
Thanks for this info.
chanw4 said:
Write back will cause data loses on crashes and it will increase read time on read miss since you need to write the block back to the main mem before reading a new block.
Click to expand...
Click to collapse
There is always a risk of slight data loss during crash, even with data=ordered. And we are already using some even more 'risky' options like barrier=0 and noauto_da_alloc anyway.
This setting increases the risk a bit more.
s88 said:
copy tune2fs into /system/xbin and chmod 755
Click to expand...
Click to collapse
He's referring to the busybox 1.17.1 version. It doesnt support the ext4 options. For the version attached here, you have to copy it to somewhere else like /data/ before u can execute it. You don't really need to copy it to /system/xbin.
hardcore said:
There is always a risk of slight data loss during crash, even with data=ordered. And we are already using some even more 'risky' options like barrier=0 and noauto_da_alloc anyway.
This setting increases the risk a bit more.
Click to expand...
Click to collapse
Yeah but I think at some point you're just throwing away the safety of EXT4, and might as well just use EXT2 since it writes faster even with these options on. (Can anybody confirm this? I did a check and EXT2 is still faster even with these mount options... while EXT4 reads faster. But writes are more important than reads on MoviNAND from my usability experience, since the reads are generally fast enough anyway.)
RyanZA said:
Yeah but I think at some point you're just throwing away the safety of EXT4, and might as well just use EXT2 since it writes faster even with these options on. (Can anybody confirm this? I did a check and EXT2 is still faster even with these mount options... while EXT4 reads faster. But writes are more important than reads on MoviNAND from my usability experience, since the reads are generally fast enough anyway.)
Click to expand...
Click to collapse
Hi Ryan, it's different. ext4 with data=writeback still uses journaling but only for metadata, which supposedly "provides a similar level of journaling as that of XFS, JFS, and ReiserFS".
So it's still safer than ext2 but without the speed (and more importantly, battery or flash write lifespan) impact full data+metadata journaling.
More detailed information in the sources here:
http://git.kernel.org/?p=linux/kern...02ac5fa36d7f4c07856fe9cf89391e08986f7;hb=HEAD
hardcore,
how do you change the mount options? like buffer from 1 to 0 and nobh, etc
s88 said:
hardcore,
how do you change the mount options? like buffer from 1 to 0 and nobh, etc
Click to expand...
Click to collapse
I've made a clearer procedure in the first post. To change some options you might need a startup script though.
hardcore said:
I didn't do any benchmarks, but in theory it should be better - reduced periodic journal writes (better battery life, flash lifespan and performance).
Click to expand...
Click to collapse
That sounds very promising
Followed the procedure - had to do it twice as well to make it stick...dont see any obvious changes in the last 5 minutes. Will update after using it for a while.
PS: how persistent is that setting? what circumstance does it get reset? When a new kernel is installed?
Does it work with 2.1 too?
bobbel said:
Does it work with 2.1 too?
Click to expand...
Click to collapse
yes, it does. I am using JM9, and it feel faster.
take note i only change the data to write_back, haven't figure out a way to do the rest.
for write back, it can be done on the terminal emulator.
s88 said:
yes, it does. I am using JM9, and it feel faster.
take note i only change the data to write_back, haven't figure out a way to do the rest.
for write back, it can be done on the terminal emulator.
Click to expand...
Click to collapse
Cool, I give it a try.
One Question:
When I enter the mount cmd I have:
/dev/block/mmcblk0p4 /data ext4 rw,noatime,...
So I have to use
/data/tune2fs -o journal_data_writeback /dev/block/mmcblk0p4 (instead mmcblk0p2)
Right?
Hi all, i've bought an HTC Hero with HTC's 2.1 ROM, and since it's still in warranty i don't want to flash it with a custom ROM now
I can't find a way to install apps on the SD, because all i find are cusom roms which already have apps2sd enabled...
What can i do if i want to keep my warranty?
Cero92 said:
Hi all, i've bought an HTC Hero with HTC's 2.1 ROM, and since it's still in warranty i don't want to flash it with a custom ROM now
I can't find a way to install apps on the SD, because all i find are cusom roms which already have apps2sd enabled...
What can i do if i want to keep my warranty?
Click to expand...
Click to collapse
no. you need root. If you like your warranty, and your phone gets broken in some way while rooting, you can just run the official RUU and return it back to stock state, and your provider will never know.
Cero92 said:
Hi all, i've bought an HTC Hero with HTC's 2.1 ROM, and since it's still in warranty i don't want to flash it with a custom ROM now
I can't find a way to install apps on the SD, because all i find are cusom roms which already have apps2sd enabled...
What can i do if i want to keep my warranty?
Click to expand...
Click to collapse
You can't enable app2sd without root, and if I am not mistaken, it's the rooting the will void your warranty....
I don't think you can mess with anything at the system level without voiding your warranty, which makes sense.
But, Andyt95 is right, if you mess something up, the RRU should restore it to the unrooted state.
I rooted it using this guide: http://forum.xda-developers.com/showthread.php?t=645253
I followed the first part up to the fourth point (i think recovery img is needed to flash, not to root, am i wrong?)
But, now?
ya recovery image needs when you want to flash something in your mobile . But to root your phone it don't need recovery
there are many apps out there which can give you root & most of them are compatible for Hero
Yes i rooted it
But what to do next to install apps on sd?
I can't find a tutorial, my mistake?
Cero92 said:
Yes i rooted it
But what to do next to install apps on sd?
I can't find a tutorial, my mistake?
Click to expand...
Click to collapse
You need to partition your card with a EXT 2 partition. Then install any custom ROM. Most support app2SD
nirmalv said:
You need to partition your card with a EXT 2 partition. Then install any custom ROM. Most support app2SD
Click to expand...
Click to collapse
Yes, but i'd like to jeep the default ROM for a while
Sent from my HTC Hero using XDA App
Cero92 said:
Yes, but i'd like to jeep the default ROM for a while
Sent from my HTC Hero using XDA App
Click to expand...
Click to collapse
The problem is, its not a simple update, it needs support in the kernel ramdisk, busybox installed and some scripts.
Usually I don't like to recommend the Darktremor app2sd, because its a bit too bloated in my eyes, and I don't like things which are so complicated I can't understand them (and I have tried, but he has so much script for a relatively simple task...).
Anyway in this case, I think its your only bet.
DO A NANDROID BACKUP BEFORE YOU TRY ANYTHING - ESPECIALLY THIS!
EDIT: Taking another look, even it (the Darktremor app2sd) does not seem to fix the kernel's ramdisk. You might want to test it after a nandroid backup on the odd chance I am wrong (i.e. if your original ROM does run the init.d scripts - mine did not). I am afraid if I am right, the only way you can keep the original ROM and enable app2sd, is if you unpack the kernel ramdisk and edit the init.rc to run the init.d scripts - which is a bit more complicated than it sounds, but not that hard if you have the proper background....
erasmux said:
The problem is, its not a simple update, it needs support in the kernel ramdisk, busybox installed and some scripts.
Usually I don't like to recommend the Darktremor app2sd, because its a bit too bloated in my eyes, and I don't like things which are so complicated I can't understand them (and I have tried, but he has so much script for a relatively simple task...).
Anyway in this case, I think its your only bet.
DO A NANDROID BACKUP BEFORE YOU TRY ANYTHING - ESPECIALLY THIS!
EDIT: Taking another look, even it (the Darktremor app2sd) does not seem to fix the kernel's ramdisk. You might want to test it after a nandroid backup on the odd chance I am wrong (i.e. if your original ROM does run the init.d scripts - mine did not). I am afraid if I am right, the only way you can keep the original ROM and enable app2sd, is if you unpack the kernel ramdisk and edit the init.rc to run the init.d scripts - which is a bit more complicated than it sounds, but not that hard if you have the proper background....
Click to expand...
Click to collapse
I'm down to give this a try, my girl wants the stock rom and a2sd. Dam cake eaters! but is there a process to follow with this? I'm good enough to follow a process but not create it for a2sd. Any links to reference??
da-pharoah said:
I'm down to give this a try, my girl wants the stock rom and a2sd. Dam cake eaters! but is there a process to follow with this? I'm good enough to follow a process but not create it for a2sd. Any links to reference??
Click to expand...
Click to collapse
why stock though? have you considered that villainrom 13 is just like stock but better and has apps2sd? it's a full HTC sense 2.1 rom but rooted with apps2sd and a few small additions to ensure it runs as fast as possible and gives as much room as possible and runs a custom kernel ***
and you can revert by flashing an RUU which is super easy to do (I've done it before for warranty reasons) and the RUU (stock rom) works exactly the same, in fact IS exactly the same except for the features you're asking for.
***(which is pretty much required for apps2sd unless you want to hack it for a while and know how to do it yourself (it's a lot to ask the devs here to tell you everything you need to do if you're not a kernel developer already imho and even then there isn't a guarantee you wont' mess up your phone's stock rom).
dkelley said:
why stock though? have you considered that villainrom 13 is just like stock but better and has apps2sd? it's a full HTC sense 2.1 rom but rooted with apps2sd and a few small additions to ensure it runs as fast as possible and gives as much room as possible and runs a custom kernel ***
and you can revert by flashing an RUU which is super easy to do (I've done it before for warranty reasons) and the RUU (stock rom) works exactly the same, in fact IS exactly the same except for the features you're asking for.
***(which is pretty much required for apps2sd unless you want to hack it for a while and know how to do it yourself (it's a lot to ask the devs here to tell you everything you need to do if you're not a kernel developer already imho and even then there isn't a guarantee you wont' mess up your phone's stock rom).
Click to expand...
Click to collapse
Hi I am trying to find the step by step guide about how to setup Apps2SD working. But having tough time. I have done this with my HD2 but cant find much help for Hero.
I have Recovery image flashed with 8GB (class 6) SD Card.
Next I am looking to partition the card from recovery, but what should be the swap size?
Once partitioned I will try flashing Villainrom 13 ROM.
Thanks
nbhor said:
Hi I am trying to find the step by step guide about how to setup Apps2SD working. But having tough time. I have done this with my HD2 but cant find much help for Hero.
I have Recovery image flashed with 8GB (class 6) SD Card.
Next I am looking to partition the card from recovery, but what should be the swap size?
Once partitioned I will try flashing Villainrom 13 ROM.
Thanks
Click to expand...
Click to collapse
Swap does not seem to work well for the hero, therefore most ROMs will not enable it and I use 0 for the swap size when partitioning. If you still prefer to reserve room for it use something like 32 or 64.
dkelley said:
why stock though? have you considered that villainrom 13 is just like stock but better and has apps2sd? it's a full HTC sense 2.1 rom but rooted with apps2sd and a few small additions to ensure it runs as fast as possible and gives as much room as possible and runs a custom kernel ***
and you can revert by flashing an RUU which is super easy to do (I've done it before for warranty reasons) and the RUU (stock rom) works exactly the same, in fact IS exactly the same except for the features you're asking for.
***(which is pretty much required for apps2sd unless you want to hack it for a while and know how to do it yourself (it's a lot to ask the devs here to tell you everything you need to do if you're not a kernel developer already imho and even then there isn't a guarantee you wont' mess up your phone's stock rom).
Click to expand...
Click to collapse
thanks for the reply! but to answer your in short, My girl doesnt speak geek and gets pissed right away when the ROM she uses acts up. So in short stock roms is whats she likes. I have been trying to convince her to do another ROM but she is hard headed. I think i'll just flash the villainrom 13 and tell her its stock lol. Thanks again i'll check it out!
Tried multiple attempts from scratch every time.
Attempt 1 - Failed
Partitioned my SD Card for EXT 2 (512MB).
Tried flashing of Villainrom 13 ROM along with Dalvik2SD patch.
Installed bunch of apps. Checked size of EXT2 partition. Nothing changed. Its still same size as it was before installing apps. System memory reduced from its original value.
Attempt 2 - Failed
Partitioned my SD Card for EXT 3 (512MB).
Tried flashing of Villainrom 13 ROM along with Dalvik2SD patch.
Installed bunch of apps. Checked size of EXT3 partition. Nothing changed. Its still same size as it was before installing apps. System memory reduced from its original value.
Any suggestions?
nbhor said:
Tried multiple attempts from scratch every time.
Attempt 1 - Failed
Partitioned my SD Card for EXT 2 (512MB).
Tried flashing of Villainrom 13 ROM along with Dalvik2SD patch.
Installed bunch of apps. Checked size of EXT2 partition. Nothing changed. Its still same size as it was before installing apps. System memory reduced from its original value.
Attempt 2 - Failed
Partitioned my SD Card for EXT 3 (512MB).
Tried flashing of Villainrom 13 ROM along with Dalvik2SD patch.
Installed bunch of apps. Checked size of EXT3 partition. Nothing changed. Its still same size as it was before installing apps. System memory reduced from its original value.
Any suggestions?
Click to expand...
Click to collapse
System memory is still reduced with app2sd, since the /data/data folder is not moved to the SD.
Assuming you have adb working, check these two commands:
Code:
adb shell df -h
adb shell ls -ld /data/app
If you think you still have a problem please post the result of these two commands.
erasmux said:
System memory is still reduced with app2sd, since the /data/data folder is not moved to the SD.
Assuming you have adb working, check these two commands:
Code:
adb shell df -h
adb shell ls -ld /data/app
If you think you still have a problem please post the result of these two commands.
Click to expand...
Click to collapse
Do I run this command in terminal emulator on droid or use droid explorer > Terminal Command?
Can Root Explorer help? I am getting adb permission denied from terminal emulator.
nbhor said:
Do I run this command in terminal emulator on droid or use droid explorer > Terminal Command?
Can Root Explorer help? I am getting adb permission denied from terminal emulator.
Click to expand...
Click to collapse
well, I actually meant connecting the device w/ a USB cable to your computer and then from a command prompt (i.e. start -> run -> "cmd") using the "adb.exe" utility that comes with the android SDK (under the tools folder)....
You can run the same commands in a terminal windows just without the "adb shell":
Code:
df -h
ls -ld /data/app
Only problem is that then it you can't copy paste your result here.
for the "df -h" command take note of the mount of your sdcard which should start like this line:
Code:
/dev/block/mmcblk0p2 720.9M 506.3M 178.0M 74% /sd-ext
I would assume that for you the mount point (last value) would be /system/sd.
next check the "ls -ld /data/app" and make sure it is linked to the same location, in my case it is:
Code:
lrwxrwxrwx 1 root root 11 Apr 20 09:59 /data/app -> /sd-ext/app
erasmux said:
well, I actually meant connecting the device w/ a USB cable to your computer and then from a command prompt (i.e. start -> run -> "cmd") using the "adb.exe" utility that comes with the android SDK (under the tools folder)....
You can run the same commands in a terminal windows just without the "adb shell":
Code:
df -h
ls -ld /data/app
Only problem is that then it you can't copy paste your result here.
for the "df -h" command take note of the mount of your sdcard which should start like this line:
Code:
/dev/block/mmcblk0p2 720.9M 506.3M 178.0M 74% /sd-ext
I would assume that for you the mount point (last value) would be /system/sd.
next check the "ls -ld /data/app" and make sure it is linked to the same location, in my case it is:
Code:
lrwxrwxrwx 1 root root 11 Apr 20 09:59 /data/app -> /sd-ext/app
Click to expand...
Click to collapse
My results from above 2 commands
C:\Program Files (x86)\Android\android-sdk\platform-tools>adb shell df -h
Filesystem Size Used Available Use% Mounted on
tmpfs 95.6M 0 95.6M 0% /dev
tmpfs 4.0M 0 4.0M 0% /sqlite_stmt_journals
/dev/block/mtdblock3 170.0M 162.8M 7.2M 96% /system
/dev/block/mtdblock5 165.8M 11.2M 154.5M 7% /data
/dev/block/mtdblock4 130.0M 58.8M 71.2M 45% /cache
/dev/block/mtdblock4 130.0M 58.8M 71.2M 45% /data/dalvik-cache
/dev/block//vold/179:1
6.8G 4.0G 2.8G 59% /sdcard
C:\Program Files (x86)\Android\android-sdk\platform-tools>adb shell ls -ld /data/app
drwxrwx--x 1 1000 1000 2048 Apr 22 01:34 ?[1;34m/data/app?[0m
nbhor said:
My results from above 2 commands
C:\Program Files (x86)\Android\android-sdk\platform-tools>adb shell df -h
Filesystem Size Used Available Use% Mounted on
tmpfs 95.6M 0 95.6M 0% /dev
tmpfs 4.0M 0 4.0M 0% /sqlite_stmt_journals
/dev/block/mtdblock3 170.0M 162.8M 7.2M 96% /system
/dev/block/mtdblock5 165.8M 11.2M 154.5M 7% /data
/dev/block/mtdblock4 130.0M 58.8M 71.2M 45% /cache
/dev/block/mtdblock4 130.0M 58.8M 71.2M 45% /data/dalvik-cache
/dev/block//vold/179:1
6.8G 4.0G 2.8G 59% /sdcard
C:\Program Files (x86)\Android\android-sdk\platform-tools>adb shell ls -ld /data/app
drwxrwx--x 1 1000 1000 2048 Apr 22 01:34 ?[1;34m/data/app?[0m
Click to expand...
Click to collapse
Well, you really do *not* have app2sd enabled (I am sorry for my previous disbelief but I have seen many users in the past which only thought they did not have app2sd enabled).
From your previous posts I understand you are using VR13, so either ask on the VR13 thread how to enable app2sd (you can post the above as proof it is not working for you), or change to a ROM with more "stable" app2sd....
I've tested this and it works. I only have a class 2 SDcard, so my testing shows it's really slow, a faster card would probably help.
With just this class 2 SD card, the boot-from-sd process is really only useful to see if the ROM being tested will straight-up brick your device or not.
Edit!
A class 6 card works fabulous! I'm posting from a tweaked version of mmarz's port of ath3nos' port of cm7 running from my brand-new class 6 card right now.
This process is hacked together from multiple other devices' howtos here on xda, sorry I don't know who to credit for the bits and parts. All of it is pretty generic, actually, and might well be applicable to other devices when tried as a whole.
For the moment, I'm only posting a brief skeleton how-to without specific walkthroughs for the steps, and I don't really want to upload many files until more of the bugs are swatted.
The following info should be enough for you, the savvy dev, to put the ROM of your choice on your SD card and boot it (or watch it fail to boot) without risk of bricking your device!
noobs, don't you dare, bricking is always a risk if you don't know your way around fastboot and adb!
Of course, savvy dev or not, a fresh nandroid backup is MANDATORY before hacking at your phone like this.
Standard disclaimer:
There is always a possibility of bricking your phone when messing with adb and fastboot.
If you're not willing to take that risk, don't try this at home (or at work, or school....)
Here goes:
The basic plan I followed was:
1) make and format 3 extra ext2 partitions after the default fat32 on the SD card, in this order: data (at least 180Mb) , system (170Mb to match stock), and cache (102Mb stock).
2) split up the boot.img from your chosen ROM so you can mod the ramdisk.
edit: turns out this next step in the quote is not required, there is an easier way.
A nifty command called devwait for init.rc
3) compile a modified init, adding a "pause (5);" after the ANDROID text. It goes on line 569 in the gingerbread init.c from a recent repo of google source.
use the newly compiled init in place of the init from the ramdisk.
since getting the android source uses so much time and bandwidth, I'm being nice and attaching a modified gingerbread init. If you test it with a froyo ROM and it doesn't work, don't blame me. If it does work with froyo ROMS, let me know!
Click to expand...
Click to collapse
3) change your init.rc from the ramdisk as follows:
replace the "mount yaffs2 [email protected] /system" , cache, and data lines with
Code:
devwait /dev/block/mmcblk0p3
mount ext2 /dev/block/mmcblk0p3 /system nodev noatime nodiratime
mount ext2 /dev/block/mmcblk0p3 /system nodev noatime nodiratime ro
devwait /dev/block/mmcblk0p2
mount ext2 /dev/block/mmcblk0p2 /data nodev nosuid noatime nodiratime
devwait /dev/block/mmcblk0p4
mount ext2 /dev/block/mmcblk0p4 /cache nodev nosuid noatime nodiratime
be sure to comment out any other mounts which go to /system anywhere, like the "mount squashfs [email protected]/system/blahblah/blah.sqf /blah/blah" lines in the aospCmod init.rc
4) repack your boot.img, with the correct cmdline ("mem=477M console=ttyMSM2,115200n8 androidboot.hardware=thunderc" works,) etc.
5) prepare the ext2 /system partition with your target ROM /system files.
How to load the ROM:
steps a) to f) done on your PC. step g) is done on the phone.
a) Unzip the ROM on your PC, to get at the files to copy to the new /system directory in adb, and allow you to modify the updater-script.
b) Mod that ./META-INF/com/google/android/updater-script as follows:
As an example, I removed the following from the aospCmodOV-5-16-11 updater-script:
everything except the "ui_print" "symlink" and "set_perm" lines, including the "unmount" line at the end of the script.
I thinned it down because the update-binary wants to write to MTD partition for /system, and I didn't want or need that to happen.
I did need it to install the symlinks and permissions, though.
Without those, the keyboards kept FCing, and the phone couldn't connect to the network.
c) rezip the ROM with the modified updater-script.
d) boot phone into recovery and connect to PC with cable.
e) adb push or otherwise copy the modified ROM.zip to your sdcard.
f) next, copy /system from the unzipped ROM to the new partition with adb
Code:
adb shell
mount -t ext2 /dev/block/mmcblk0p3 /system
exit
adb push /path/to/unzipped/ROM/system /system
g) on phone, in recovery: install .zip from sdcard
select the new ROM.zip you reassembled and pushed to the SD card
select yes, you really want to install the .zip
this should write the /system symlinks and permissions to the new /system partition.
Click to expand...
Click to collapse
6) back on your PC, still connected to phone with cable in recovery;
Code:
adb reboot bootloader
fastboot boot /path/to/boot.img
Step 6 is important!
this will boot from your modified boot.img without actually flashing it into the phone's NAND, so a reboot (or restart after battery pull in case of issues) just goes to your regular installed ROM in the phone.
That's about it for now. It's pretty much a hack as yet, but it's cool in a geeky way to be able to do this.
<reserved for future use>
Thanks! This should be very helpful to devs and those of us who are adventurous with our phones!
Very interesting concept, I'd love to see how many uses this could have!
bump...
with a class 6 card, ath3nos/blarf cm7 runs great off sd.
even feels a little snappier than running from the phone mtd partitions.
I must benchmark soon to verify.
I'm really surprised no-one seems interested, I guess this is too old-hat, like a debian chroot
bigsupersquid said:
bump...
with a class 6 card, ath3nos/blarf cm7 runs great off sd.
even feels a little snappier than running from the phone mtd partitions.
I must benchmark soon to verify.
I'm really surprised no-one seems interested, I guess this is too old-hat, like a debian chroot
Click to expand...
Click to collapse
This is good work, but I think people require a bit more hand holding. How about publishing some of the modded files?
I personally still have problems recompiling a ramdisk.
mmarz said:
This is good work, but I think people require a bit more hand holding. How about publishing some of the modded files?
Click to expand...
Click to collapse
I can post files one rom at a time, because the ramdisk and kernel will be a bit different for each rom, and so will the permission/symlink updater-script.
first, I'll have to get rom-dev permission, because they get modded, and also cause it's nice to have permission.
I personally still have problems recompiling a ramdisk.
Click to expand...
Click to collapse
fastboot provides a fantastic shortcut for that, if you cpio-gzip the ramdisk together after editing it to your specs
from http://android-dls.com/wiki/index.php
in a command line linux shell, from the directory containing your unpacked ramdisk,
Code:
find . | cpio -o -H newc | gzip > ../newramdisk.cpio.gz
will put newramdisk.cpio.gz in the directory above the one containing your ramdisk
Click to expand...
Click to collapse
fastboot can make the boot.img for you and either dump it into the phone (assuming your shell current directory contains your kernel (zImage) and ramdisk (newramdisk.cpio.gz))
Code:
fastboot -c "mem=477M console=ttyMSM2,115200n8 androidboot.hardware=thunderc" flash:raw boot zImage newramdisk.cpio.gz
or boot your phone from the assembled image without actually flashing it in.
Code:
fastboot -c "mem=477M console=ttyMSM2,115200n8 androidboot.hardware=thunderc" boot zImage newramdisk.cpio.gz
those fastboot lines assume an optimus v (or s.)
looking at all the great work you've posted on the forums, I'm surprised you have trouble handling anything to do with these phones!
I'm slowly working up to a modded recovery to install-to-sd directly and avoid all this hacking.
Nice, thanks for that. I had no idea fastboot could do that.
mmarz said:
Nice, thanks for that. I had no idea fastboot could do that.
Click to expand...
Click to collapse
happy to share.
I only learned fastboot could do that when I seriously started in kernel testing after finally getting one to compile without errors. I didn't feel like bricking my phone to test the fresh kernel, and researching fastboot showed me its capabilities... I could've just typed fastboot with no arguments and that makes it list its parameters.
it's really helpful for this mod, because you can just reboot into your stock rom without having to reflash anything.
plus it's nice to just tweak the ramdisk and let fastboot do the work making a new boot.img for you when testing weird stuff like this.
bigsupersquid said:
happy to share.
I only learned fastboot could do that when I seriously started in kernel testing after finally getting one to compile without errors. I didn't feel like bricking my phone to test the fresh kernel, and researching fastboot showed me its capabilities... I could've just typed fastboot with no arguments and that makes it list its parameters.
it's really helpful for this mod, because you can just reboot into your stock rom without having to reflash anything.
plus it's nice to just tweak the ramdisk and let fastboot do the work making a new boot.img for you when testing weird stuff like this.
Click to expand...
Click to collapse
Very cool i loved this idea when it was executed in a way over at Ubuntdroid for the intercept when i was doing alpha testing. Very convenient for many of the scarier tests when i wanted to keep my existing system intact. The big difference was that on the intercept the bootable partitions were written into the kernel. Downside you couldn't use it to test kernels but could switch kernels from the phone and flip-flop between completely isolated systems on the go without a computer. Which was great when i wanted to test a new ROM on the way to work but i needed my phone when i got there and had to have a painless method of switching to my usual running system. So i'd flash a "non SD" kernel and i was back up and running in just a minute or so.
I'm excited to play with this when i have some free time.
Thanks dude!
you could easily use an update.zip for each of the sd-boot and regular phone boot.img files and swap between working systems with those in recovery.
before getting the fast sd card, I was looking at this more as for testing foreign roms than two-system operation, but now I see that would work too.
for a second-system option I was looking at a market-enabled rom on sd since I don't use g-apps on my daily driver.
note: the GB rom I'm running on won't 'see' a card with more than four partitions properly, and won't mount the vfat partition with vold, although you can manually mount and access the first four partitions including the vfat. android just won't admit it's there. and I couldn't get at a fifth partition through the phone os, even though my pc could see it fine (to get 5 partitions I had to use an extended partition to contain most of the virtual partitions)
android could read virtual partitions in an extended partition just fine as long as the total number on the card didn't exceed four.
I learned a neat trick today.
going through the init.rc from the stock rom, I noticed a section at the beginning which was labeled 'on emmc' which would imply running from a card if the device hardware lacked nand flash memory.
I'd read on xda while researching this concept of boot-from-sd about an unspecified wait-for-device command in init.rc, but most of the info I've gotten on init scripting language has come from picking apart init scripts, as google has not documented the available commands very well.
long story short, I tried using devwait with the regular init and it worked just fine to mount the ext partitions. yay!
first post edited to reflect the improved method.
I've also been testing a recovery mod. if recovery.fstab is modified in the recovery ramdisk (or a fakeflash recovery) then the recovery will copy a tweaked rom.zip to the ext /system partition with some minimal modifications to the updater-script and a few commands in the adb shell.
the recovery with the modded recovery.fstab can also back up from and restore to the sdcard ext /data and /system partitions.
I will update the first post with more specific instructions as I hammer them into something consistent.
soon I'll be ready to release a modded fakeflash recovery to make experimentation easier, as well.
I posted this here - http://forum.xda-developers.com/showpost.php?p=18647176&postcount=14 - but never got an answer so I'm posting it to its own thread. I would post to the development section but apparently until I clutter up the place with 10 posts I'm considered a drooling idiot.
I was playing around and I noticed that the problem is with s-on I cannot write to /system meaning no superuser etc. So I remembered from way back when I started linux you could mount over a mounted folder. So I made a small file, put ext3 on on it, transferred it to the sdcard and mounted over /system with
mount -t ext3 -o rw,loop /mnt/sdcard/test.dump /system
it worked and then of course I lost all commands (ls, reboot...) in the shell.
Would it be possible to dump /system with "cat /dev/mtd/mtd3 > /mnt/sdcard/system.img", extract it, put it in an ext3 file, mount it as readable over system and just go to town?
Somebody MUST have tried this in the past. Before I go any further can anyone tell me why this wouldn't work and save me a few hours noodling around lol
baconbacon said:
I posted this here - http://forum.xda-developers.com/showpost.php?p=18647176&postcount=14 - but never got an answer so I'm posting it to its own thread. I would post to the development section but apparently until I clutter up the place with 10 posts I'm considered a drooling idiot.
I was playing around and I noticed that the problem is with s-on I cannot write to /system meaning no superuser etc. So I remembered from way back when I started linux you could mount over a mounted folder. So I made a small file, put ext3 on on it, transferred it to the sdcard and mounted over /system with
mount -t ext3 -o rw,loop /mnt/sdcard/test.dump /system
it worked and then of course I lost all commands (ls, reboot...) in the shell.
Would it be possible to dump /system with "cat /dev/mtd/mtd3 > /mnt/sdcard/system.img", extract it, put it in an ext3 file, mount it as readable over system and just go to town?
Somebody MUST have tried this in the past. Before I go any further can anyone tell me why this wouldn't work and save me a few hours noodling around lol
Click to expand...
Click to collapse
S-on itself got nothing to do with rooting. It just means you can't flash unsigned images as much as I know.
That system is mounted as ro is also at rooted phones the standard behaviour, you have to remount it as rw, but without root it won't accomplish I think.
Cat is for text files not for binaries or whole partitions, to extract the image use the dd command.
Do you have an /dev/mtd folder? My partitions are at /dev/block and there is no /dev/mtd present.
Swyped from my HTC Desire S
Tectas said:
S-on itself got nothing to do with rooting. It just means you can't flash unsigned images as much as I know.
That system is mounted as ro is also at rooted phones the standard behaviour, you have to remount it as rw, but without root it won't accomplish I think.
Cat is for text files not for binaries or whole partitions, to extract the image use the dd command.
Do you have an /dev/mtd folder? My partitions are at /dev/block and there is no /dev/mtd present.
Swyped from my HTC Desire S
Click to expand...
Click to collapse
This issue is that when you remount it as rw you cannot write to it. I want to know if I make my own partitions and mount them after boot if that will work, probably slower but work. This is so lojack I doubt it will work, I just want to know if anyone tried it.
Also "Cat is for text files not for binaries or whole partitions" is wrong, it works fine.
baconbacon said:
This issue is that when you remount it as rw you cannot write to it. I want to know if I make my own partitions and mount them after boot if that will work, probably slower but work. This is so lojack I doubt it will work, I just want to know if anyone tried it.
Also "Cat is for text files not for binaries or whole partitions" is wrong, it works fine.
Click to expand...
Click to collapse
Then it's not rw, because rw stands for read write, but like I said without root...
And no I think no one tried it before, because using an xtc-clip is compared to your solution much less work
But if you want to try it, do it and share your experiences, maybe it helps others
That cat works doesn't mean that it's made for it, you can also hunt for birds with a bazooka
Take a look at cat and dd and choose yourself.
Tectas said:
Then it's not rw, because rw stands for read write, but like I said without root...
And no I think no one tried it before, because using an xtc-clip is compared to your solution much less work
But if you want to try it, do it and share your experiences, maybe it helps others
That cat works doesn't mean that it's made for it, you can also hunt for birds with a bazooka
Take a look at cat and dd and choose yourself.
Click to expand...
Click to collapse
is xtc-clip better? yes, not arguing that. is cat is better then dd? no, I said cat works in this instance the same as dd, the results will be the identical.
I didn't ask this question to argue unrelated linux 101. I'm just going to look into this myself. iow screw you guys, im going home...
Dude, I use Linux too though I'm probably not a guru. I think I understand what you're saying... but if I do, aren't you just talking about editing a copy of /system? How would you mount your edited partition at boot?
Time permitting, I could try a few tricks but I'm not willing to risk bricking my phone though. I'd be stranded without it.
Sent from my HTC Wildfire S A510e using XDA App
First don't blame others, because of things I wrote.
I never said it couldn't work, I just wanted to show you that there are better ways to do it, cause cat also tries to interpret the file and not only copies it, what could mess up your image in some cases.
Also be aware that if your using the same data and cache partition with both of your systems, it could be easy bricked, if you change to much at your 2. system and remember that there aren't many ways left to get it working again cause your still s-on.
I think he wants to mount it himself every time after boot.
If he can root his 2. system (but all by hand, I think) he maybe can mount his 1. system as rw and also root it, but the chance that something goes wrong is pretty high, even mounting itself could freeze the device, cause the first system is still in memory.
I'm also not even close to be a Linux guru.
Swyped from my HTC Desire S