Hi!
I was experimenting with the data2ext script made for the Desire in following thread:
http://forum.xda-developers.com/showthread.php?t=859419
Now I managed to get it working with HTC Hero and Flykernel12a and Amon RA-s recovery v1.7.0.1 .
MAKING THIS WORKING IS A RISKY OPERATION! DO NOT START IF YOU DO NOT UNDERSTAND WHAT IS WRITTEN BELOW! I'M NOT RESPONSIBLE IF YOU DAMAGE YOUR PHONE OR YOU LOOSE YOUR DATA!
So here is what I made:
- make backup of your entire SD card, because we are going to repartition it, so all data will be lost on your SD card
- reboot to recovery and MAKE NANDROID BACKUP of your existing configuration!!! Then stay in recovery mode, and copy your NANDROID BACKUP to a safe place on your computer from the SD card!!!!! DURING THE OPERATION WE WILL ERASE THE WHOLE SD CARD!!!
- wipe your cache and dalvik-cache (only this 2, NO factory/data reset!!)
- connect to your phone with "adb shell" (while in recovery!!)
- in the shell execute following commands:
(do this only if you have active app2sd and your app/app-private/dalvik-cache directories are on the SD card EXT partition)
mount /data
rm -rf /data/app
rm - rf /data/app-private
rm - rf /data/dalvik-cache
mkdir /data/app
mkdir /data/app-private
mkdir /data/dalvik-cache
chmod 777 /data/app*
chmod 777 /data/dalvik-cache
chown 1000:1000 /data/app*
chown 1000:1000 /data/dalvik-cache
- now we will copy back the apps on the nand
mount /sd-ext
cp /sd-ext/app/* /data/app
cp /sd-ext/app-private/* /data/app-private
(no need to copy the dalvik-cache content!)
umount /sd-ext
- so now we have all our app on the nand again. next step is to put the 04data2extV4 script into the right place
umount /data
mount /system
exit
(now leaving shell to copy file)
in the command prompt execute:
adb push pathtothescript/04data2extV4 /system/etc/init.d/
adb shell
(back to shell again)
chmod 755 /system/etc/init.d/04data2extV4
chown 0:2000 /system/etc/init.d/04data2extV4
rm /system/etc/init.d/05mountsd
(remove script which mounts SD card differently)
rm /system/etc/init.d/10apps2sd
(remove script which makes app2sd)
umount /system
exit
(now leaving shell)
now we have to repartition your SD card to 2 ext and 1 fat partition. For this use the utility1 I uploaded which is modified version of the original utility in /sbin.
in command prompt:
adb push pathtoyourscript/utility1 /sbin/
adb shell
(back to shell)
chmod +x /sbin/utility1
(now execute the utility)
/sbin/utility1
choose 0 from menu: Partition SD 2 EXT 1 FAT (can set sizes)
enter the first EXT partition size (min. 3-400, I use 1300 - values are in MB)
enter second EXT partition size (min. 2-300, I use 300 - values are in MB)
the rest will be configured for the FAT32 file system
On the first EXT partition you will store the whole /data
On the second EXT partition you will store the /data/data in a loop file
Now we are hopefully ready to reboot, the rest will be managed by the 04data2extV4 script, wich creates the ext2 file systems, converts them to ext4, stets file system option such as journaling where needed, etc and copies everything from nand to ext4.
I hope this works for all. I reached Quadrant score for total system score about 455, without data2ext I had about 400. But I have a Class4 SD card. On class6 I'm sure it will be even faster.
Again, it is risky, so take care.
The boot process can be long, because at the first time it will copy the whole /data directory to the first EXT partition, the data/data directory on the 2nd partition, and will recreate the whole dalvik-cache (remember we wiped!).
Enjoy
Regards,
byr2
PS: in the RAR file you will find an FSTAB file. I removed the /sd-ext mount option from it. If you have trouble, try to replace the fstab file in /etc to this fstab file or remove the /sd-ext mount option from your fstab
Thank you for this.
Just a few questions. The EXT partitions, are they EXT 2, 3 or 4?
Are you using exactly the same script as ownhere? As in, you use exactly his method, aka the following one:
"I would like to share my Data2EXT script, a collection of data2sd(Thanks to sibere) and the advantages of ext4, you can let the phone run at peak efficiency.
I use EXT4 for whole SD 2nd partition, and create a ext2 loopback device with 1/5 size of 2nd partition.
The files in /data/app,/data/.systemapp it not updated frequently. so store them in EXT4.
In /data/data and /data/app-private, a lot of small files and sqlite database, is updated frequently. so store thenm in EXT2 loopback device.
/data/dalvik-cache is stored in phone memory /mnt/asec/mtddata for distributed IO pressure.
/data/misc, /data/system, /data/property is stored in phone memory /mnt/asec/mtddata for system settings restore(or you will lost your timezone/language settings after reboot).
use this script, I can get quandrant score:2000+, IO:4000+, my phone is smooth and quickly."
Are you running this stable? Reboots? All apps show after reboots?
Hi!
The two EXT partition is EXT4 partition. The script I provided is based on ownhere's script, wich is modified only in a few places.
For example on hero I do not have a /system/xbin/mkfs.ext4 like he had, but only a mkfs.ext2, so I have to create the EXT4 partition on a different way. Also the loopback device is set up in a different way than in the original script.
But what you quoted remains so, which means I did not modified any location of storage files, nor did modified filesystem parameters. First EXT4 ha no journaling ,second EXT4 has journaling, ext4 loop device has no journaling.
You can diff the two scripts to see the difference, but an example here:
Original part:
/system/xbin/mkfs.ext4 -b 4096 -m 0 -F -L userdata /dev/block/mmcblk0p2
/system/bin/tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
/system/bin/e2fsck /dev/block/mmcblk0p2
/system/bin/tune2fs -O ^has_journal /dev/block/mmcblk0p2
Modified part:
/system/xbin/mkfs.ext2 -b 4096 -m 0 -F -L extdata /dev/block/mmcblk0p2
/system/bin/e2fsck -y -v -f /dev/block/mmcblk0p2
/system/bin/tune2fs -O extents,uninit_bg,dir_index /dev/block/mmcblk0p2
/system/bin/e2fsck -fpDC0 /dev/block/mmcblk0p2
/system/bin/tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
/system/bin/e2fsck /dev/block/mmcblk0p2
/system/bin/tune2fs -O ^has_journal /dev/block/mmcblk0p2
As you can see I create an ext2 partition, check it, and with tune2fs I convert it to EXT4, check it again, and set the same parameters ownhere has set.
for loopback device I use:
$BUSYBOX losetup /dev/block/loop1 /mnt/asec/extdata/ext4
$BUSYBOX mount -t ext4 -o loop,sync,commit=3,barrier=1,nouser_xattr,errors=continue,noatime,nodiratime,nosuid,nodev /dev/block/loop1 /mnt/asec/tmpdata
the losetup is needed, because for some reason it won't mount as loop. But this way it is working.
Other parts of the script are untouched.
Regards,
byr2
"Are you running this stable? Reboots? All apps show after reboots?"
Till now everything runs OK, no reboot, all widgets, apps, settings were preserved. After reboot also everything OK. Made some phone calls, no problems so far.
Just with Class4 SD card the speed gain is not as huge as expected. The IO performance is slightly better, but not too much. I've read posts on CDMA Hero forum, where they managed to get quadrant score over 700, but I'm sure not with a Class4 SD card
But the funniest thing in settings is: Available internal phone storage: 1,17Gb
Excuse me for the noob question, but what this process basically does is make the entire phone run off of an ext4 partition on the SD card?
No, not the entry phone, just the /data directory is moved to ext4 partition (wich is accessed frequently by the OS, and where applications and application related data are stored). The kernel, firmware, etc. remains on the NAND.
I wonder what one does when flashing a new ROM =/ You have to go through this procedure every single time? What does AmonRA do when you Wipe Data/Factory Reset?
So essentially the differentiations are due to the fact that we lack a /system/xbin/mkfs.ext4 (since we only have an ext2) and you turn the first EXT4 partition into a non journaling one to fill the lack of that, if i understand correctly.
If you flash a new rom, then all this is gone, since if will overwrite the /system/etc/init.d where our data2ext script is located. So you're right, you have to do this every time you flash a new ROM. A solution would be to pack this script into the ROMs, so it would work out of the box without this hack mentioned in the first post.
The data/factory reset wont work I think, because it deletes from the nand, but since our data/settings are not on the nand but the SD, it will have no effect.
the first ext4 partition is non journaling because ownhere write the script so. You can read in his post why he choosed to do so.
The first EXT pratition where /data is stored is a EXT4 part without journaling
The second EXT partition where the loop file is stored is an EXT4 part with journaling
and the LOOP file which contains an EXT4 partition without journaling
from ownhere's post:
"EDIT:2010/12/12 update V4 script:
/data/:ext4, disable journaling
/data/data:ext4 loopfile on ext4 device, enable journaling on ext4 device, disable journaling on ext4 loopfile, for fast and stable sqlite3 access.
really smooth this time. please trying..."
This is the setup here also.
byr2
I see, the script is essentially the same, with your own critical changes to fit the Hero.
Now what i am trying to understand is what happens when you want to flash a new ROM.
Say you place the 04data2extV4 script in the new ROM's /system/etc/init.d and remove the other two scripts if they exit (05mountsd and 10apps2sd). Does this mean the partitions and this whole process won't have to be repeated?
Also when installing a new rom it is recommended you wipe data and system, if AmonRA doesn't wipe, what do we do?
Well, I've not tested to install a new rom, but let think:
- the partitions are preserved on your SD, so repartitioning is not needed.
- but as you mention all data/settings are currently on your SD, so a wipe does nothing with it, and if you simply put the 04data2extV4 script back to init.d I think it will mess things just up... but if you just delete everything off the two EXT partitions I think it will be fine
That's the only limitation i have so far as to not attempting this.
I wonder if i can mod the AmonRA Recovery to delete the content of those 2 partitions instead =/
I'm going to have a look.
Nice work!
By the way, I was thinking. Why not just make an ext partition, add a line for it in fstab and set the mountpoint for it to be /data. What would happen if I tried this?
Byr2.... just wanted to say this is first thing i read today and without coffee all I heard was
Blah blah blah dangerous blah blah blah faster blah blah risk blah blah even faster
Lmao
Once I get my head working it should prove interesting though
Thanks for this experiment
Sent from my HTC Hero using Tapatalk
ok, sitting down at pc now, drinking espresso...
this is starting to sink in
Instant benefit is huge storage space which, coupled with the method used, ought to have a general performance improvement over previous ext-based apps2sd, right? if so then removing ext-based apps2sd and using this script concept instead seems like a good plan if, for example, every rom builder switched to this concept and removed old apps2sd stuff simultaneously. Does that sound right or wrong?
if you have a fast sd card then the speed benefit should be very good, better than using ext-based "apps2sd" since it's the actual data partition on the ext card.
and I too would like to know the answer to the brief question someone already asked - what about just setting a mount point to data on the sd card and doing it that way in simple linux fashion? I'm not bothering to think this through - coffee's still kicking in - I'll leave the intelligence to you folks who are already awake.
IMHO this idea of yours is quite brilliant as a goal (well ok, it's not your idea, but you did it first on the gsm hero . The thing that slows my phone the most is reads and writes to phone memory and to my stock sd card. I can upgrade the sd card but the phone memory writes are still slow. This would improve the speed of the largest file writes when managing my apps and installing from the market which we all know brings our phones to a standstill briefly during installs and so on.
How's the performance when installing apps? does your phone still crawl for a bit at those crucial moments or is it all somewhat less of a dramatic slowdown now?
Hi All!
Well, If you do not change the ROMs frequently, it should not be a problem that ROM developers do not include this script. However till now only one ROM developers have taken care about this opportunity, I think it was Cronos. They have a modified ROM wich supports this kind of functionality.
The speed gain over app2sd comes from storing the whole data partition on a faster SD card than the internal NAND. As my tests show a class4 SD card is slightly faster than the internal NAND (not much, say about 5-10%). A class6 SD card could be much more faster. All application access the data partition to read and write settings/data. Even the Frimware stores things there, so if you have a fast SD card, overall system performance could be great.
Not to mention about stored data space. With app2sd you only store the downloaded apk-s on the EXT. With data2ext you have extra space for the application data/settings/cache wich size is determined by you and your SD card size ! (See screenshot!)
A simple fstab modification could work, however the script does do a lot of things beside copying everything to EXT. It also sets up some symlinks during every boot, so a part of the script is needed always. Also there is 2 EXT partitions, and the /data/data is stored on a 2nd EXT partition in an EXT4 loop file image So it's not that easy to simply mount it via fstab.
byr2
dkelley said:
Byr2.... just wanted to say this is first thing i read today and without coffee all I heard was
Blah blah blah dangerous blah blah blah faster blah blah risk blah blah even faster
Lmao
Click to expand...
Click to collapse
Hahaha. I've read this several times now and still all I get is what you wrote above.
I got this after all
Code:
$mount
rootfs on / type rootfs (ro)
tmpfs on /dev type tmpfs (rw,mode=755)
devpts on /dev/pts type devpts (rw,mode=600)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
none on /acct type cgroup (rw,cpuacct)
tmpfs on /mnt/asec type tmpfs (rw,mode=755,gid=1000)
none on /dev/cpuctl type cgroup (rw,cpu)
/dev/block/mtdblock3 on /system type yaffs2 (ro)
/dev/block/mtdblock4 on /cache type yaffs2 (rw,nosuid,nodev,noatime)
/dev/block/mtdblock5 on /mnt/asec/mtddata type yaffs (rw,nosuid,nodev,relatime)
/dev/block/mmcblk0p2 on /data type ext4 (rw,nosuid,nodev,noatime,nodiratime,commit=19,barrier=0,nobh,data=writeback)
/dev/block/mmcblk0p3 on /mnt/asec/extdata type ext4 (rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered)
/dev/block/loop1 on /data/data type ext4 (rw,sync,nosuid,nodev,noatime,nodiratime,commit=3,barrier=1,data=writeback)
/dev/block/vold/179:1 on /mnt/sdcard type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
/dev/block/vold/179:1 on /mnt/secure/asec type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
tmpfs on /mnt/sdcard/.android_secure type tmpfs (ro,size=0k,mode=000)
All correct?
This is mine:
Code:
rootfs on / type rootfs (ro,noatime)
tmpfs on /dev type tmpfs (rw,mode=755)
devpts on /dev/pts type devpts (rw,mode=600)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
none on /acct type cgroup (rw,cpuacct)
tmpfs on /mnt/asec type tmpfs (rw,mode=755,gid=1000)
none on /dev/cpuctl type cgroup (rw,cpu)
/dev/block/mtdblock3 on /system type yaffs2 (ro)
/dev/block/mtdblock4 on /cache type yaffs2 (rw,nosuid,nodev,noatime)
/dev/block/mtdblock5 on /mnt/asec/mtddata type yaffs (rw,nosuid,nodev,relatime)
/dev/block/mmcblk0p2 on /data type ext4 (rw,nosuid,nodev,noatime,nodiratime,commit=19,barrier=0,nobh,data=writeback)
/dev/block/mmcblk0p3 on /mnt/asec/extdata type ext4 (rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered)
/dev/block/loop1 on /data/data type ext4 (rw,sync,nosuid,nodev,noatime,nodiratime,commit=3,barrier=1,data=writeback)
/dev/block/vold/179:1 on /mnt/sdcard type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0
020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
/dev/block/vold/179:1 on /mnt/secure/asec type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_ut
ime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
tmpfs on /mnt/sdcard/.android_secure type tmpfs (ro,size=0k,mode=000)
/dev/block/dm-0 on /mnt/asec/com.qualcomm.qx.neocore-1 type vfat (ro,dirsync,nosuid,nodev,noexec,uid=1000,fmask=0222,dmask=0222,code
page=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
Seems your's almost the same...
The key is this:
Code:
/dev/block/mmcblk0p2 on /data type ext4 (rw,nosuid,nodev,noatime,nodiratime,commit=19,barrier=0,nobh,data=writeback)
/dev/block/mmcblk0p3 on /mnt/asec/extdata type ext4 (rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered)
/dev/block/loop1 on /data/data type ext4 (rw,sync,nosuid,nodev,noatime,nodiratime,commit=3,barrier=1,data=writeback)
How is your phone performing? Was everything OK?
byr2
Everything seems to be ok )
There is no more sticking when new apk installed. Quadrant score around 500.
I have Class 10 sd card (prooflink).
But I have had a little problem - not enough nand space to hold all of my apks, so I've decided to push most of them via adb from my laptop after Hero's reboot.
Quadrant score not so high as I expected, but anyway I'm happy =)
if you push it via ADB after reboot, than it should get to the right place
The quadrant score is affected by 3D and CPU also, but can you feel the "speed" while testing the IO? Does it run fast? How long is the DB read/write, FS read/write?
I'm happy someone else could get this thing working
byr2