Develop Bugs

[Dev Kernel] 2.6.29 Support for ext2/ext3/ext4/yaffs2

Here is the kernel with yaffs2 and MTD block support, I am missing some mtd util tools like nandwrite so i cannot continue. Those that have the tools like mkyaffs2image, nandwrite, flash_eraseall can try to create an image and mount it
here are the steps that i think should work:
# mknod /dev/mtd0 c 90 0
# flash_eraseall /dev/mtd0
# mkyaffs2image /data/data yaffs2.img
# nandwrite -p /dev/mtd0 yaffs2.img
# mkdir /data/data2
# mount -t yaffs2 /dev/mtdblock0 /data/data2
If you have the tools, please attach so we can continue.. Its getting late so I am going to bed, have fun
Download
yaffs2_support.tar

Oh my god! I was just about to post similar findings, what a coincidence!
I just had compiled a kernel with yaffs2 support. My idea was to modify /init.rc in the kernel to mount the partitions with yaffs2 instead of rfs. Problem: we can't just change the filesystem without backing up and restoring all the files inside. My idea was to do that inside the recovery. Mount a mountpoint, backup with tar to /sdcard, unmount, format with dd (in theory a partition filled with 0's is ok for yaffs2), mount again with yaffs2 and restore from tar.
I'm stuck at the point that I can't seem able to mount any partition with yaffs2 previously formatted (with /system/xbin/busybox dd if=/dev/zero of=/dev/block/stl10). If I mount (/system/xbin/busybox mount -t yaffs2 /dev/block/stl10 /dbdata) there is always an "Invalid argument" error. It happens with /dev/block/stl*, /dev/block/bml* and /dev/block/mmcblk*
I'm no kernel expert, neither I know a lot about yaffs2, but I think I read you can't mount a yaffs2 partition anywhere, it has to be an MTD flash device.
Anyway, I'm going to try with the tools you said. Wish me luck.

Thanks for your work rasdeep! Is it only YAFFS2 and MTD support that you added? Because the kernel already had support for EXT2 and EXT3.
darksnarfer said:
I think I read you can't mount a yaffs2 partition anywhere, it has to be an MTD flash device.
Click to expand...
Click to collapse
Does the kernel that rasdeep posted load the existng flash devices/partitions as MTD devices?
I think as a start you only have to worry about formatting the /dev/block/mmcblk0p2 device which is mapped to the /data directory. With the new lag fix we are only moving directories stored in /data and that is what gives a big performance improvement. RyanZA did tests and found that the /dbdata partition was already quite fast and I don't think we have to worry about the /system partition etc. because we don't really write to them.
I think after a new firmware flash the /dev/block/mmcblk0p2 (mounted as /data) will be safe to format into YAFFS2 without loosing anything important.
I think these are the steps we can do:
1. Flash new firmware with repartition option on to erase everything.
2. Boot into recovery and format /dev/block/mmcblk0p2 as YAFFS2 filesystem.
3. Copy on new a init.rc script which will mount /dev/block/mmcblk0p2 to /data as a YAFFS2 filesystem and not a RFS filesytem to get much faster speeds.

bub181 said:
Does the kernel that rasdeep posted load the existng flash devices/partitions as MTD devices?
Click to expand...
Click to collapse
Just checked, there isn't any /dev/block/mtd* so I guess the kernel still can't see them as mtd.

/data is on the internal SD not in NAND so will we be able to use yaffs2 at all? I was working on formatting to EXT4 and adding the init.rc to match as well as compiling with EXT4 support, but if yaffs2 works, I'm fine with that.

modifying the init.rc is secondary because that is easy.. the main problem now is to be able to manually setup yaffs2 to mount properly.. if everything works like it should, init.rc can be change in matter of seconds
its funny now u said mtd not found.. i know we can mknod /dev/mtd* but there should be /dev/mtdblock* when booting with kernel right? hmm.. something to investigate further..
keep suggestions and finding coming, we may hit something that i've missed
problem is im lacking those tools that will do write to flash directly.. for example nandwrite.. anyone has that can share so i can try more stuff?
also, are my steps for "preparing" yaffs2 partitions/image correct? if anyone familiar with yaffs, please give some insight

raspdeep said:
modifying the init.rc is secondary because that is easy..
Click to expand...
Click to collapse
Do you need to recompile the kernel to modify the init.rc? If not could you post some instructions on how to do it please? I would like to try some things in there before /data gets mounted as a RFS filesystem.

bub181 said:
Do you need to recompile the kernel to modify the init.rc? If not could you post some instructions on how to do it please? I would like to try some things in there before /data gets mounted as a RFS filesystem.
Click to expand...
Click to collapse
i'm afraid so for now until someone found a way to extract the initramfs from later kernel and restore it back after modification..

I think it would be easier to start out with EXT2 since we can easily format that. Just as a proof of concept. I'm planning to do so tonight. Then I'll add support for EXT4 formatting and try that.

raspdeep said:
problem is im lacking those tools that will do write to flash directly.. for example nandwrite.. anyone has that can share so i can try more stuff?
Click to expand...
Click to collapse
Have you seen these pages rasdeep? They seem to describe how to create the YAFFS filesystem. Also, are there any /dev/mtdblock* devices using the kernel that you created? According to what I read elsewhere it is supposed to create both /dev/mtd* and /dev/mtdblock*. I think we have to use /dev/mtdblock* to create the new filesystem.
http://processors.wiki.ti.com/index.php/Create_a_YAFFS_Target_Image
http://processors.wiki.ti.com/index.php/MTD_Utilities
http://processors.wiki.ti.com/index.php/Put_YAFFS_Image_to_Flash
Also, I know that the Nexus One uses YAFFS2 for its internal filesystem. Maybe someone has worked out a way to write a new image file for that and we can use what they have learnt?

Can we potentially use this kernal (with the right tools) to format the NAND in YAFFS2 instead of RFS? Then still have way of flashing firmware onto it?
If so, that would be awesome and it would seem to have the least draw backs.
Currently the A2SD lag fix requires an external microsd card, and the one click lag fix, mounts a EXT2 image ontop of the RFS NAND, and that could drastically affect NAND life.

This is great !!!
Thank you raspdeep and all the devs here, this is one step closer to a stable kernel and a permanent genuine solution for the lag problem as far as a n00b like me understands
cheers !

Daemos said:
Can we potentially use this kernal (with the right tools) to format the NAND in YAFFS2 instead of RFS? Then still have way of flashing firmware onto it?
If so, that would be awesome and it would seem to have the least draw backs.
Click to expand...
Click to collapse
Not if we can't mount the NAND as a MTD-device instead of STL-device.
Daemos said:
Currently the A2SD lag fix requires an external microsd card, and the one click lag fix, mounts a EXT2 image ontop of the RFS NAND, and that could drastically affect NAND life.
Click to expand...
Click to collapse
Say what? /data/ is mounted on the internal SSD, not the NAND. /dbdata/ is however mounted on the NAND. Just do a df -h and you can see all the mountpoints.
Also, mounting and using a ext2-image that reside on an rfs-partition will not damage your drive. The rfs filesystem is still the one used to store the image itself (the ones and the zeros), even though you write it inside an image that uses another filesystem.

I stumbled upon these documents while Googling for STL-devices:
http://www.samsung.com/global/busin...y/downloads/LinuStoreII_GPL Compliance_10.pdf
http://www.samsung.com/global/busin...ionmemory/downloads/RFS_130_Porting_Guide.pdf
I haven't had the time to read them through yet. But if there's a porting guide, there must be a way to reverse it
Go go reverse engineers!

Is it really necessary to format the NAND with another filesystem? /dbdata seems fast enough as it is. Wouldn't it be enough (and easier?) to only convert the internal SD to yaffs2?

Quote from someone that knows a lot more than I do:
yaffs2 won't work on a sdcard
Click to expand...
Click to collapse
http://twitter.com/wesgarner/status/20805997718
So right now we have /data that is mounted in the second partition of the internal sdcard (mmcblk0p2), where all the critical files are stored (data, dalvik-cache, etc..) but we can't use yaffs2 there because it's not MTD.
We could, however, format and mount the other partitions (/system, /dbdata, /cache, etc...) with yaffs2, but that might not help because there isn't many heavy read/write there. And we should figure out how to make the kernel see them as /dev/block/mtdblock* instead of /dev/block/stl*.
rasdeep, can you share the .config for the kernel you used? I managed to get some mtd devices (atleast in /proc/mtd) but they were virtual devices. Maybe we need some more options enabled in the kernel.
Also, anybody knows why there are /dev/block/stl* and /dev/block/bml* they seem the same though.

darksnarfer said:
Also, anybody knows why there are /dev/block/stl* and /dev/block/bml* they seem the same though.
Click to expand...
Click to collapse
According to http://www.samsung.com/global/busin...ducts/fusionmemory/Products_XSR_Brochure.html
STL and BML are independent layers for managing I/O to the NAND devices
STL manages the sector translations while BML manages the invalid blocks (whatever that means)

Did any of you notice that in init.rc there are many references to yaffs2 ? Perhaps in engineering Samsung did use it ?

Chainfire said:
Did any of you notice that in init.rc there are many references to yaffs2 ? Perhaps in engineering Samsung did use it ?
Click to expand...
Click to collapse
Yeah, I think they are just android standard (maybe they took them from a template) and commented to use their better(?) filesystem.

darksnarfer said:
Quote from someone that knows a lot more than I do:
http://twitter.com/wesgarner/status/20805997718
So right now we have /data that is mounted in the second partition of the internal sdcard (mmcblk0p2), where all the critical files are stored (data, dalvik-cache, etc..) but we can't use yaffs2 there because it's not MTD.
We could, however, format and mount the other partitions (/system, /dbdata, /cache, etc...) with yaffs2, but that might not help because there isn't many heavy read/write there. And we should figure out how to make the kernel see them as /dev/block/mtdblock* instead of /dev/block/stl*.
rasdeep, can you share the .config for the kernel you used? I managed to get some mtd devices (atleast in /proc/mtd) but they were virtual devices. Maybe we need some more options enabled in the kernel.
Also, anybody knows why there are /dev/block/stl* and /dev/block/bml* they seem the same though.
Click to expand...
Click to collapse
Ok, so what filesystem does the internal SD use currently? Would it help if we change that to EXT4? That would emulate the current mimocan fix, but without having to use an external SD card.

[Q] Apps2SD +++

I've been trying to add all apps+data+dalvik to my sdcard but only got the apps+dalvik to the sdcard.
After installing a few apps i was out of space internally and tried to mount a 4th sd partition as /data/data.
The problem was:
1. when all was done in recovery it was not used when rebooting the phone, this also applies to /etc/fstab
2. still canot figure out how to permanently mount /dev/block/mmcblk0p4 as /data/data or as an alternative mount it as /sd-ext2 and mount -o bind /sd-ext2 with /data/data
3. the froyo style apps2sd seems to me like a bunch of crap, the went to my fat32 partition ?
4. not the problem ? Partition1=fat32, partition3=ext3(apps), partition3=swap, partition4=ext3(data)
5. the phone totally crashed when trying to combine partition 2 and 4 as one with 2gb storage and mounting it to /data/
If anyone can figure out a solution to this it would be great
Btw, tried this on modaco r9, pinkolin pinky, auraxtsense and leedroid roms, all with sense. (I want to use sense just because!)
PLZ come up with a solution/patch/mod, this would be the lifesaver for us who want's alot of apps and still lots of free storage space
Iknow linux, but it made no difference with android in some ways.

I've tried several attempts to ln-s /sd-ext/data to /data/data as well, but ever boot it resets it and does it's thing.
Nothing I do seems to permanently change this thing.
Where can I change it ?
It's the same as with /data/app and /data/dalvik-cache ?
And if so, where is that ?
SYmlink works with /data/app and dalvik-cache, why not on /data/data ?

This is strange because I've moved my dalvik-cache to /cache without any problem.
Maybe froyo doesn't like being symlinked to external storage?

hawkn said:
I've tried several attempts to ln-s /sd-ext/data to /data/data as well, but ever boot it resets it and does it's thing.
Nothing I do seems to permanently change this thing.
Where can I change it ?
It's the same as with /data/app and /data/dalvik-cache ?
And if so, where is that ?
SYmlink works with /data/app and dalvik-cache, why not on /data/data ?
Click to expand...
Click to collapse
the symlink wont exist after reboot unless you add a correct script to the /system/ect/init.d/ directory!! but be warned i have tried this and its not worth the effort, even if your device dosent bootloop when you try to boot, it will be as slow as a stoned wildfire, if you have s-off, i suggest changing the partiton tables on your device, you can gain an extra 50mb+ on the data partition that way, and its alot safer!

[TUTORIAL][Link2SD][ROOT]Updated! How to prep your SD card for it...Windoze Style

Rather than recreate the wheel...
Here is the developer's thread on Link2SD
Read it thoroughly!!!
Here are some links on Ext2, Ext3 and Ext4 file systems and why Ext 4 is preferable over Ext2 if you can do it with your ROM's kernel.
Ext2
Ext3
Ext4
Here's a debate over which to use. You be the judge and go with what makes sense for you.
What I'm going to take you through is creating the second partition on your SD card to be able to use it. We are going to use Minitool Partition Wizard Pro Home Edition (Freeware) to do this on a Windows machine.
There are other ways to do this including Linux, from recovery, etc, but this is a free and simple method that DOES NOT LOSE YOUR EXISTING SD CARD DATA!!!
First things first!
Repeat after me...
1/ I have a stock kernel in my ROM. My ROM only supports FAT32 and Ext2 for the second partition and I will not format it Ext4 and then come asking why it's not mounting correctly! I will format it Ext2 for best performance.
2/ I am not S-OFF or HTCDev unlocked AND rooted. I will NOT come asking if this will work or WHY it's not working at all!
3/ I WILL NOT TRY MOUNTING MY PHONE AS A HARD DRIVE TO DO THIS. I WIL READ THE INSTRUCTIONS AND USE A CARD READER!
So the prerequisites are:
1/ HTC Dev Unlocked OR S-OFF
2/ Permanently rooted
3/ A windows machine with Minitool Partition Wizard Pro Home Edition (Freeware) Installed or Hirens Boot CD which has a bootable windows version with PWPH installed on it.
4/ An SD card appropriate for the job. The faster the better! For 16 and 32 GB I really do recommend splurging on the Class 10...
Steps:
1/ Boot up your Hirens or Windows machine
2/ Make sure that Show Hidden Files and Folders is ON so you can see some of the hidden folders present on an android system's SD card
3/ Insert your Micro SD card into a reader and attach it to the computer
4/ Make a folder on your computer and back up your entire SD card's main partition to it including hidden files and folders JUST TO BE SAFE. (Remember if you do this right you won't have to lose your card's contents, unlike the recovery method.
5/ Install Partition Wizard Pro Home Edition or check that its on your Hiren's bootable windows cd
6/ Run Partition Wizard Home Edition.
7/ Look for your SD card along the bottom and identify your FAT32 Partition
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
8/ Right click on it and choose "Move/Resize Partition"
9/ Use the down arrow in the middle field "Partition Size:" to change the value and consequently the value of the "Free Space After: field as well until you have created around 1027 MB of free space. *Warning - Do not create free space before the first primary FAT32 partition!!! If you do discard changes and start over!
10/ Click "OK"
http://dl.xda-developers.com/attachdl/97c7411fd1fdf03bee85f54f0203ebd8/4f4cec58/8/4/5/7/6/5/Shot_1_Resizing_FAT32.PNG/IMG]
11/ Identify the new unallocated space on the drive map.
[IMG]http://dl.xda-developers.com/attachdl/a02e4cb4e27a14095eaa8347118e31e7/4f4cec81/8/4/5/7/6/6/Shot_1_Basic_FAT32_and_unallocated_space.jpg
12/ Right click on it and choose "Create"
13/ When you get the warning about the partition not being able to be used in Windows click "Yes" to continue
14/ Choose Ext2, Ext3 or Ext4 for partition type based on whether your kernel supports Ext4. If it does choose it! If you don't know choose Ext2 to be safe. I would avoid Ext3 unless you have to. 2 or 4 are preferred
*NOTE!!! You cannot resize an Ext partition after you make it. You can only move it around so choose your size wisely! I find a GB to be plenty and I have 150+ linked apps with 360 MB free on SD card partition 2 but you can have up to 2 GB.
15/ Make sure it is a primary not a logical partition
16/ Name it if you want.
17/ Click "OK" to continue
18/ Click the BIG "Apply" button in the top left of the window
19/ Choose "Yes" to continue from the pop up
20/ Let it do it's work.
Congratulations! Step 1 Complete!
Now to step 2:
1/ Eject and place your SD card back into your phone
2/ Boot up your phone
3/ Install Link2SD from the Android Market
4/ Run it
5/ On first run it will ask you the type of partition your second SD card partition is. Choose Ext2 or Ext4 depending on which you made it.
6/ Follow the directions to reboot your phone
Congratulations! Link2SD is now working on your phone!
When you click the menu button in Link2SD and choose "Storage Info" you should see three memory spaces. Internal, SD Card, and SD Card 2nd Part.
At this point I recommend you Menu click and go to Settings and choose "Autolink" to automatically link programs (.APK), dalvik cache files (.DEX) and library files (libs) to your SD Card 2'nd partition when a new program is installed.
Now for some strategy!
You say, "I've got all this external space! Why bother putting any programs on the internal space?"
Well...SPEED MAN! SPEED!
I keep Launchers, Widgets, Background processes like Titanium Backup, etc, on the internal as space allows to allow for speed when booting. The internal memory is faster than your SD card and keeping these programs here helps a lot in speeding up your boot time. This is an option with Link2SD that you DON'T have with Data2SD or Data2Whatever!
You can also keep commonly accessed programs like Tapatalk for example, on this internal fast memory.
You can also mass move by multi selecting, several programs to and from either internal, SD card, or SD card second partition. Ideal for your initial move to your new second partition of your apps pre-installed prior to link2sd.
People wanted instructions for a third swap so see post two for additional steps.
Troubleshooting:
*If you have issues with your primary SD card partition not working right or perhaps you decided to wipe the whole card and create them both/or all three from scratch, remember that this FAT32 partition 1 has to be set ACTIVE and all partitions have to be primary not logical.
*If you cannot mount Ext2 or Ext4 make the second partition FAT32. It will work for all Wildfire S Models and all ROM's. Try the others first as they are preferred for performance and stability.
*If you don't have a card reader it's been reported that you can use your phone in hard drive mode to achieve the same thing, followed by a reboot but I have heard many cases where this causes troubles so I personally don't recommend it but hey. You gotta do what you gotta do.
*Some users are reporting with MarvelC's that they needed to use Ext3 so I have updated the tutorial to reflect this.
*If you don't immediately see a change in internal memory after linking apps relax. It will show up. Sometimes it just is a little delayed.
That's pretty much it for now.
Enjoy and if this tutorial helped you or you like it, please click "THANKS"!

Add On's and FAQ's
Adding in a Linux Swap If Wanted, Supported, and Needed:
Personally I don't see the use for it but hey. Sometimes people want the kitchen sink included as well right?
1/ Increase your unallocated space in the FAT32 resize to around 1550 MB
2/ Right click on the unallocated space and make your Ext2/Ext4 partition of around 1027 MB
3/ Right click on the remaining unallocated space at the end of the drive and choose Linux swap for partition type. Again make sure it's a Primary and not a Logical parition!
Apply.
Have fun...(And click thanks!

How would I know which partition my kernel supports? I'm using alquez's CM7.

i wana do the partitions for this
but this topic said u need "3rd-Linux Swap - mine is 512MB" its a 3rd partitio? or how shall i do this? or the 3rd partition is not necessary?and just stay relax, make my sd card as d33ps1x wrote in this thread, and i can easily install that linked ROM with 800mhz processor?

bingzz said:
How would I know which partition my kernel supports? I'm using alquez's CM7.
Click to expand...
Click to collapse
+1 same
i got cm7 but link2sd didnt work should i flash kernel or something like that

hanynemesis said:
+1 same
i got cm7 but link2sd didnt work should i flash kernel or something like that
Click to expand...
Click to collapse
I used the ext2 partition and it worked fine for me, I'd recommend that for you too.

bingzz said:
I used the ext2 partition and it worked fine for me, I'd recommend that for you too.
Click to expand...
Click to collapse
so after create ex2 partition setup link2sd and choose ex2 and all ok ?

Used ext4 for alquez' cm7.

Ext4 is fine for CM7. Recommended. Just make sure you choose it when you first run Link2SD.
I am using it without issues.
Regards,
DS

szuladam said:
i wana do the partitions for this
but this topic said u need "3rd-Linux Swap - mine is 512MB" its a 3rd partitio? or how shall i do this? or the 3rd partition is not necessary?and just stay relax, make my sd card as d33ps1x wrote in this thread, and i can easily install that linked ROM with 800mhz processor?
Click to expand...
Click to collapse
You don't NEED a linux swap but just add into the equation
1/ Created 1.5 gb of free space
2/ Create second Ext4 partition
3 Create linux swap partition with remainder of space at end of disk.

szuladam said:
i wana do the partitions for this
but this topic said u need "3rd-Linux Swap - mine is 512MB" its a 3rd partitio? or how shall i do this? or the 3rd partition is not necessary?and just stay relax, make my sd card as d33ps1x wrote in this thread, and i can easily install that linked ROM with 800mhz processor?
Click to expand...
Click to collapse
Here you go...Instructions for swap.

d33ps1x said:
At this point I recommend you Menu click and go to Settings and choose "Autolink" to automatically link program, dalvik and libs to your SD Card 2'nd partition when a new program is installed.
Click to expand...
Click to collapse
Suggestion (since this is so noob-friendly otherwise): Instead of mentioninf dalvik, refer to .dex files, synonymous in this context (at least my version of link2sd refers to apk, dex and lib files)
Regards,
Rich

rkleinhenz said:
Suggestion (since this is so noob-friendly otherwise): Instead of mentioninf dalvik, refer to .dex files, synonymous in this context (at least my version of link2sd refers to apk, dex and lib files)
Regards,
Rich
Click to expand...
Click to collapse
Appreciate the constructive input. Will do.

OK guys i followed all steps and after setup link2sd and open it it ask for permission sure i press ok and choose ex4 partition type then got this error
Mount script error
mount script cannot be created
mount:mounting/dev/block/vold/179:2on /data/sdex2
failed :device or resource busy.
of-course i have ex4 partition after my fat partition both are primary and the fat one is active my memory is 2 GB
tried to restard and try again same error so what i had done wrong ?

hanynemesis said:
OK guys i followed all steps and after setup link2sd and open it it ask for permission sure i press ok and choose ex4 partition type then got this error
Mount script error
mount script cannot be created
mount:mounting/dev/block/vold/179:2on /data/sdex2
failed :device or resource busy.
of-course i have ex4 partition after my fat partition both are primary and the fat one is active my memory is 2 GB
tried to restard and try again same error so what i had done wrong ?
Click to expand...
Click to collapse
Hmm............Does your primary SD partition still function right?

Did this but was stuck on cm7 boot screen after a reboot.

*se-nsei. said:
Did this but was stuck on cm7 boot screen after a reboot.
Click to expand...
Click to collapse
Wipe Cache and Dalvik cache and try reboot again.

d33ps1x said:
You don't NEED a linux swap but just add into the equation
1/ Created 1.5 gb of free space
2/ Create second Ext4 partition
3 Create linux swap partition with remainder of space at end of disk.
Click to expand...
Click to collapse
thx for help. so actualy the linux swap patition it not necessary. than what for?
without it that rom wont work? (the apps will go to ext4 partition, so what goes for 3rd one?)
or does is have any advantage/disadvantage?
(this questions apply only for HTC WFS ofc)

szuladam said:
thx for help. so actualy the linux swap patition it not necessary. than what for?
without it that rom wont work? (the apps will go to ext4 partition, so what goes for 3rd one?)
or does is have any advantage/disadvantage?
(this questions apply only for HTC WFS ofc)
Click to expand...
Click to collapse
It's what it says it is. Memory swap space.
It swaps out rarely used memory pages to the swap to make room for immediate use of active ones.
If your ROM kernel supports it give it a go with and without and see if you notice performance gains in your everyday use.
Google will provide you with lots of info on what a linux swap partition is and it's benefits/detriments.

d33ps1x said:
Wipe Cache and Dalvik cache and try reboot again.
Click to expand...
Click to collapse
Thank you. Great tutorial btw. One question - what would you recommend that I keep on the SD card to avoid boot problems?

[DEV][19NOV] Native Mount DataOnEXT with DalvikOnNAND (Test #2)

Index
Post 1 - Android Boot Process, Space Restrictions in /data, Usual Workaround scripts that load from init.d
Post 2 - The Permanent Solution for Internal space issues in /data, and how it is executed (Step by Step)
Post 3 - Credits, Benefits, Note on Cache, Note to ROM Makers on ClockWorkMod Updater Script, FAQ
Post 568 - Test ROM for Native Mount DataOnEXT with DalvikOnNAND
Post 622 - Test #2 ROM for Native Mount DataOnEXT with DalvikOnNAND
Thanks to quite a lot of inputs, ideas, testing and feedback, I have been able to give a working solution to reliably mount /data to the EXT partition in our HD2’s SD Card before Android initializes. I am redoing the first three posts of this thread to give a complete info that answers the what, why, how questions.
Before I even start, let me make my objectives clear:
Have a system partition that is as big as I want in NAND, so that I can install any bloated ROM I want without chopping down anything, and yet have a /data partition for all my apps and their bloated data
Avoid using internal (NAND) userdata partition completely, so that if the day comes where my /system is full of bad blocks, I will still be able to create a system partition in the ~900+ MB internal NAND my TMoUS HD2 has
With this method, this is how I am using my phone:
Device - TMoUS HD2
Boot Loader - cLK 1.5.0.9
Partitions (in order) and Sizes - Recovery - 10 MB, Boot - 10 MB, System - 500 MB, Userdata - <I didn't bother >, Misc - 1 MB, Cache - 2 MB.
Recovery - CWM Based Touch Recovery B8
SD Card - 16GB Transcend Class 6 with 4 GB EXT4 Partition (mounted as /data, of course!)
My idea for this setup is that a user should not be having to get into the boot loader or changing partition sizes for every ROM. With this setup, I am having a close-to-native-Android device setup and can flash any ROM I want, without worrying if my partitions will accommodate it.
So what exactly does this method do? Before I answer that, a brief on the Android booting process – Note that this is simplified to meet this thread’s requirement, and not necessarily completely accurate. When you power on the phone the following actions happen…
The Android Boot Process
Bootloader – In HD2’s case, Magldr or cLK – loads the kernel based on how you have configured the phone.
Kernel – The kernel (zImage) is loaded into RAM along with an initial ramdisk (initrd.gz), which initializes various devices (IO, memory, GPU, etc.), interrupts, and mounts the root file system (/). After this, the first user-space process called init is started.
Init – this is a binary file that is contained within the initrd.gz. The init binary processes init.rc and init..rc , along with other .rc files that are called by these two .rc files. Some of the key functions (from this thread’s perspective) in the order of their initialization/ execution are:
The init process follows the instructions in the init.rc and init.xyz.rc files and creates empty directories including /data. It then mounts the storage devices (partitions in the internal NAND (MTD)) to these empty directories. The NAND partition for system is mounted to /system, followed by the partitions specified for data, cache, etc. The directories for dalvik-cache (/data/dalvik-cache) are also created by the init process after mounting the specified device to /data.
The init process then starts various services including adb, service manager, Volume Daemon (vold) for media like SD Card (FAT partition). Most importantly, the zygote service which initiates the Dalivk-Cache is loaded in this sequence.
As we all know, Android is based on Linux. The boot sequence described above is common for all Linux machines – until the zygote stage. Core Android file like core, framework, services, IME, policy, etc. are executed from the Dalvik-Cache and hence Initialization of the Dalvik Cache is pretty much where Android comes into the picture
The sysinit/ run-parts part, which runs scripts from the /system/etc/init.d later the Zygote stage. No matter how this is done, Android has already started loading by the time the boot process comes to executing scripts in /system/etc/init.d
Now that you know what happens when Android boots, you may also have realized that mounting a device to /data happens early in the boot process.
Space Restriction in HD2, and the twist with bad blocks
The 'normal' way to store all OS data (dalvik-cache, Configurations & Settings, Accounts, etc.) and User data (Apps, app data, mails and messages, call logs, etc.) is to have a partition named userdata in the HD2’s NAND, and have it mounted to /data during the init process. The drawback in using the userdata partition is that the space you can have for userdata is inversely affected by the size of your system partition, and whether your phone is an EU/ International HD2 or a TMoUS HD2.
Best case is, if you have a ~100 MB system partition for a really scaled-down, space-optimized OS, you will get about 800+ MB for userdata partition on TMoUS and ~400 MB on an International/ EU HD2. This space might be good enough for a lot of us who only have light-to-medium use of our phones. But for quite a lot of us, this is a bottleneck. As if that is not enough, we have these villainous bad-blocks that creep up slowly, determined to swallow up the internal NAND completely, though it would be several years down the line before that happens
The Workaround
The workaround for this method is to have a script in the /system/etc/init.d that mounts the EXT partition in the SD Card to sd-ext, moves a part of /data from the userdata (NAND) to the EXT partition (sd-ext) and creates symlinks (symbolic link) in /data (userdata) to point to the new location in the EXT partition. In this method you will not see an increase in what the phone shows for internal space. The apps you install and then some more will be installed in the EXT partition, which will be symlinked as /data/app and so on.
The other way is to move the mount point (and the contents) for the /data directory from userdata (NAND) to the EXT partition in the SD Card, mount the userdata (NAND) as sd-ext or some other directory, create dalvik-cache (and maybe /data/data - where all user apps' data are stored) in the userdata and symlink that location back to the EXT partition (which is now /data). In this method, you will get the size of your EXT partition as the internal space. Moving parts of /data to NAND will in theory give you better performance, as the most read/ written part of the phone resides in NAND.
The benefit of these methods is that it is very simple and easy to install on any rooted ROM through the recovery. This method balances performance by moving the static part of /data (like the .apk you install) to the EXT partition, keeping the dynamic part (/data/data – where all app data are stored, or /data/dalvik-cache – from where all apks are executed) in NAND.
The drawbacks with this workaround:
This method is initiated AFTER Android boots. No matter how it is executed or what it does, anything that loads from /system/etc/init.d is executed only after at least some parts of Android loading process is initialized. This leads to weird outcomes – low sound issues, unpredictable behavior of the OS or some apps (Notification Widget toggles not working, etc.), and then some more. There is no reasonable way to predict what issues you may get. In fact you cannot be sure if the issue is with the ROM or if the issue is occurring because of the move2sd script you are using. If you are facing an issue that none others are facing and if you have a move2sd script, then you MUST undo what the script does before you can start troubleshooting.
In some cases, an in place upgrade (upgrading from one version of the same ROM to a higher/ newer version, or upgrading from one ROM to another ROM of the same Android version/ build) will corrupt the apps' data as well as configuration, etc.
Moreover, one move2sd script that works fine with one ROM will not work exactly the same way with another ROM or even an updated version of the same ROM.
There is always some amount hit or miss involved in this workaround!!!

The Solution
…is what this thread is all about. Like we saw, /data directory is mounted to a storage device pretty early in the boot process – so early that it is only Linux at the time /data gets mounted and Android is still yet to be loaded. The storage device that is traditionally mounted to /data directory is the internal NAND’s userdata partition.
The simple solution I have for unrestricted and reliable internal space is to directly mount the EXT partition of the SD Card to /data during the Linux init process (instead of first mounting the NAND’s userdata, loading Android, and moving the /data mount point or some contents within /data to the EXT partition while Android is loading).
How is this done?
The Easy way - Thanks to @Kokotas (Note that I have not updated Kokotas' .zip file to include the recent changes (20th October 2012) (See The Long Way below)
Visit the post (quoted below), thank @Kokotas, download the .zip from the post, flash it in recovery and lets us know how it goes. Did I tell you to make sure you have a NANDROID backup?
This .zip can be used to install DataOnEXT mod on the fly - your ROM will be the same when you reboot after flashing this .zip, except that /data will be mounted from the EXT partition with all your apps, settings, etc. intact.
kokotas said:
If you have a normal NAND installation(and an ext2/3/4 partition on your card) and want to give ph03n!x's DataOnEXT method a try,
BUT you think it is difficult THEN you can try the attached recovery package.
It follows all the steps of the described process AND at the end it copies all of your /data to the ext2/3/4 partition. <= This means that after rebooting, you'll have your system set up as before but with DataOnEXT!
And if you want to go back to your previous setup (that is a normal NAND installation without DataOnEXT), just flash the original kernel to your device.
Have in mind that I have only tested it on my phone. If you do try it, please leave a comment. Especially MAGLDR users! Cause it is not tested with MAGLDR.
Thank you ph03n!x for a great concept!
Click to expand...
Click to collapse
The Long Way (or this is how I started off doing this mod )
The following steps show how to modify a ROM’s boot process to mount the EXT partition as /data.
IMPORTANT: THIS PROCESS INVOLVES USING DISXDA’s KITCHEN in CYGWIN. I EXPECT YOU TO BE ABLE TO SETUP CYGWIN AND USE DSIXDA KITCHEN. EVEN WHILE I AM TRYING TO GIVE A STEP-BY-STEP HERE, I WILL NOT ANSWER QUESTIONS ON HOW TO SETUP AND TROUBLESHOOT DSIXDA’s KITCHEN OR CYGWIN.
Assumption:
For the sake of simplicity, I am going to assume that you have a ROM that is installed in your phone, and you want to modify the same ROM for natively mounting EXT to /data.
I am also going to assume that your HD2 has a SD Card in it, and the SD Card has been partitioned in such a way that the first partition is the FAT partition that Windows sees, and the second partition is a EXT2/3/4 partition. You may or may not have a Swap partition, but I strongly recommend NOT enabling Swap.
Prerequisites:
Take a NANDROID backup using ClockWorkMod
Back up all user apps and system app data using Titanium Backup
Install Cygwin
Install @dsixda’s Kitchen
Install Notepad++ (or any other UNIX compatible text editor)
@dsixda’s Kitchen – Setting up a working folder
While these steps will involve the entire ROM, note that you will only using the boot directory or boot.img for this mod.
You need a rom.zip file that has at least these three directories when you open the .zip – META-INF, system, boot (or boot.img)
If your rom.zip file has a different structure, just extract the three directories alone from it and create a new zip. Essentially, when open the rom.zip you should see only these three directories
An Example:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Place this zip file in the original_update directory under Kitchen. A sample directory structure of the Kitchen is as follows:
From Command Prompt, run cygwin.bat, browse to the directory where you have installed the kitchen and run it using ./menu
Select option 1 for setting up a Working Folder, follow instructions. Congratulation! You should have a working folder now. After the Working Folder is setup, navigate to the WORKING_091712_123456 directory. Make sure you have a directory named boot, or a boot.img here. To be double sure, you can replace the boot directory or boot.img with the one from the ROM you want to modify and flash.
Steps for mounting EXT to /data
Back in the Kitchen Menu, select 0 for Advanced Options
Select 20 for Tools for boot image
Hit z for Extract kernel+ramdisk from NAND boot folder (or w for Extract kernel+ramdisk from boot.img)
Now navigate to the BOOT-EXTRACTED directory that got created under the directory where you installed the kitchen. You will see something similar to this:
Edit init.rc and init.htcleo.rc to find all mentions of sd-ext and comment them out with # in the begining. This step is to ensure there is no directory called sd-ext when your phone boots, avoiding any confusions
In init.rc, look for the on fs part. Under this part there will be a line that reads mount yaffs2 [email protected] /system noatime nodiratime. Below this will be another line that reads mount yaffs2 [email protected] /data nosuid nodev noatime nodiratime. Comment this line out with a # (#mount yaffs2 [email protected] /data nosuid nodev noatime nodiratime). Also, if you have mount yaffs2 [email protected] /system ro remount, comment that out too (make it #mount yaffs2 [email protected] /system ro remount)
Do all the following changes AFTER mount yaffs2 [email protected] /system noatime nodiratime, else the process will fail.
Code:
# Wait to ensure SD Card is available, and scanning file system of EXT
wait /dev/block/mmcblk0p2
wait /dev/block/mmcblk0p2
wait /dev/block/mmcblk0p2
wait /dev/block/mmcblk0p2
exec /system/xbin/busybox sh -c "/system/bin/e2fsck -f -p -t -t -v /dev/block/mmcblk0p2 > /system/fsck.log"
# Native Mount DataOnEXT
mount ext4 /dev/block/mmcblk0p2 /data wait nosuid nodev noatime nodiratime barrier=0 data=writeback
Leave everything else in this section as is - you will have another line to mount cache, do not remove or modify it.
Next, scroll down to the section that reads on boot. Add the following like under it:
Code:
# SD Card Cache
write /sys/devices/virtual/bdi/179:0/read_ahead_kb 2048
Build the NAND boot folder (for Magldr yboot) or boot.img (for cLK / Magldr rboot) as per your requirements
Head back to the Working folder, copy the boot directory or the boot.img and include it in your original ROM’s zip file (not the one you created for the kitchen, in case you had to). Check the init.d directory of the ROM’s zip to ensure there is no script that mounts the EXT partition to sd-ext directory, or has a move2sd/ app2sd/ int2sd script – delete such scripts!. Also make sure that your ROM's inatallation does not copy anything to /data during Installation. If it does, move those parts to /system. Or use the instructions I had given under Note to ROM Makers on ClockWorkMod Updater Script to mount the EXT partition as /data in Recovery.
Format your phone’s /system, /data, / sd-ext and /boot from clockworkmod and flash this modified rom.zip
Let your phone boot into Android and settle down.
Configure your Google Account
Download and install Titanium Backup
In Titanium Backup, restore all apps and their data from the backup you had taken earlier
You phone must now have an internal storage that is roughly equal to the size of your SD Card
Note:
I have tested this method with CM7, CM9, CM10 (ParanoidAndroid JB) and Sense ROMs. There has also been feedback that this is working on other AOSP based ROMs like MIUI.
The mount wait command is available in the init binary from CM7 onwards (not sure if it is there in CM6). If you are not sure (or cannot check the source code for init.c)

Misc & Changelog
Credits
All the Devs who are contributing their valuable time to get Android working on our phones
Proz0r for implementing something very similar to what I had wanted. He has essentially implemented the same thing, I am implementing this in a slightly different way, that's all. (To be clear, I did not borrow the idea even though a few others have had the same brainwave that I thought I got first - I only borrowed the part where we have the phone wait until the sd-ext partition is ready for being mounted)
Rick_1995 for the first hint to get this modification to work
Securecrt for showing that this modification is indeed possible
Xylograph for pointing me to another thread where this modification is already done
marco.palumbi - for showing me that I need not have an elaborate script to ensure /dev/block/mmcblk0p2 is available, and a mount... wait option is available in the CM source for init. Also for doing away with the need for extprep script and including the e2fsck routine complete with a fsck.log in init.rc itself.
kokotas - For the recovery-flashable version of this mod that simplifies migrating to DataOnEXT
Some of the benefits of this mod are:
Space - Obvisiouly. The 1GB/ 2GB/ 4GB sd-ext partition will be mounted as internal memory, instead of the usual /data partition from NAND
Reliability - EXT4 partition is arguably more reliable than YAFFS2
Working around Bad Blocks - Most of our phones would be having a bunch of bad blocks by now (lucky you, Hynix NAND folks!). With this method, the /data partition is mounted directly to the EXT partition in your SD Card, and the internal userdata NAND partition is untouched. This means you can create bigger system partitions to work your way around bad blocks. Incidently all the bad blocks my phone has is in the userdata partition - this might be so because that is the area that is written to the most.
Performance - A fast SD Card gives better transfer speeds than NAND (My ParanoidAndroid build is noticeably faster after this modification, with my Class 6 Transcend SD Card with 4GB sd-ext partition)
Simplicity - Don't have to worry about hoards of the a2sd scripts and their variants, and carry-out trial and error as to which ones work reliably with your ROM of choice. /data is transparently and natively mounted to sd-ext, so no more low-sound or other odd issues
/data/data can be in the sd-ext partition, and ROM updates will not corrupt it like in some a2sd scripts
Hassle-free - Whether an EU HD2 or TMoUS HD2, you set a large size (445 MB?!!) for /system and forget about partitioning, as the internal userdata partition is not used at all and can be of whatever size. I am using 300 MB system in both my phones. This is close to native Android phones, where we do not keep changing the system partition size for every ROM that is installed.
A proper multi-user setup - if you had to share the phone with someone else, all you have to do is swap out the SD Card you usually have in the phone to a fresh one that has the required FAT and EXT partitions. The other user will see a fresh Android install and configure whatever they want. When you get the phone back, all you have to do is put your usual SD Card back in, and voila! you have all your stuff just the way it was (Thanks to @teemoo for the tip!)
Trying a new ROM? Make sure you do not botch up your data! Just remove the SD Card you usually use and place one for testing. You can always mod the initrd.gz of your new ROM, flash the boot image or place the initrd,gz in /boot and replace your usual SD Card when you want all your apps, data and settings!
A Note on Cache
As many may remember (and some have pointed out), I had the initial trial with 8 MB of cache for SD Card interface. Over a period of time, I tried various sizes from 512 KB to 16 MB and found out that the cache size is dependant on the phone AND the SD Card that is in it. This experiment involves:
Setting a cache size in the extprep script
Booting from the boot.img using fastboot boot command
Waiting for my phone to connect to Google servers
Opening Root Explorer and confirming the cache size
Running SD Tools, testing the SD Card Read/ Write speeds thrice in a row
Changing the cache size in extprep script, recreating a boot,img, and starting from fastboot boot...
And here are my test results:
As you can see, the higher the cache, higher the difference between the Minimum and Maximum values for Reads. While this test does not consider ALL aspects (like random reads), this gives a good idea of what is the cache size that is the best for my phone and my SD Card. Note that I have two HD2 with 2 different brand/ class of SD Cards, and bot gave pretty different results. You may have to find the right cache size for your phone! Once you got the right cache, even a Class 2 card (the stock Sandisk 16 GB Class 2 that comes with the TMoUS HD2) performa pretty well - no deal-breaking lags even in games like Temple Run or Ninjump.
Also, you should be worried more about read speeds - that is what the phone does most of the time. Write speeds comes into picture mostly when you are installing something. All writes that happen after that are mostly incremental, and never something a major as what happens when you install an app.
Note to ROM Makers on ClockWorkMod Updater Script
While one of the benefits of this mod is that you can set a huge /system partition and forget trimming down anything, some ROMs mounts /data during CWM install to copy certain apps or parts of /system to the /data partition. For such a setup to work well with this mod, the /data partition should be mounted to the EXT partition in the SD Card. I am not sure if there is a straightforward mount command that can be used in updater-script to mount an MMC partition. Thanks to Looki75's post, I have made the script simpler...
The code in updater-script...
Code:
package_extract_dir("work", "/tmp");
format("MTD", "system");
format("MTD", "boot");
mount("MTD", "system", "/system");
mount("ext4","/dev/block/mmcblk0p2","/data");
ui_print("Copying files to /system");
package_extract_dir("system", "/system");
...
...
ui_print("Copying files to /data");
package_extract_dir("data", "/data");
FAQ
I will try and compile some sensible Questions from this thread in the FAQ Section. I request all to read the first three posts at least before asking any questions.
Q: I have an EXT2/ EXT3 partition. You are mounting it as EXT4. What gives?
A: Mounting as EXT4 is backwards compatible with EXT2/3
Q: Where are the initrd.gz that you had posted earlier?
A: I have moved them to this post
Q: Will you modify the initrd.gz from the <name of ROM>?
A: I took more than 6 Hours on a Sunday Night to re-do the first three posts, and give a step-by-step guide. Do you still want me to modify the initrd.gz for you?!! I might post the initrd.gz of ROMs I use or try from time to time, but please don't post requests for modding <name of your ROM>'s initrd.gz. What is that saying about giving a man a fish vs. teaching how to fish...??!
Q: Is ClockWorkMod or 4EXT or TWRP compatible with this mod?
A: Before I answer, note that almost all Recovery tools available for HD2 will mount the SD Card's EXT partition to the sd-ext directory. After doing this mod, /data (in Android) = /sd-ext (in Recovery)
Any backup that includes the sd-ext partition will backup your /data.
At least CWM based recovery will delete dalvik-cache from NAND's data and /sd-ext. After this mod, /data/dalvik-cache (Android) = sd-ext/dalvik-cache (Recovery), so yes deleting Dalvik Cache works
CWN Recovery's Fix Permissions runs in both internal data and sd-ext, so yes that works too
Q: Will installing <name of script/ mod for ROM>, which involves flashing something in Recovery, work after this mod?
A: If whatever you are flashing is limited to modifications in /system, no problems - it will work.
If what you are flashing involves changing or adding content to any part of /data then it will probably not. The reason is when you mount data in Recovery, you will by default mount the internal NAND. After this method, the internal NAND is untouched. You must either modify the updater-script (See Note to ROM Makers on ClockWorkMod Updater Script), or use QtADB/ ADB shell to do what the flashing is supposed to do.
Q: I am getting stuck at boot animation/ getting "Encryption Unsuccessful" message when Android boots...
A: Your ROM's init binary may not support the mount... wait option that I am using. There are other variants like wait <device>, and then mount <device>, see if you can find what it is from the source code of your ROM's init binary (init.c). If all else fails, you may have to include a copy of busybox in sbin, point all commands in extprep script to that busybox, and increase the sleep time using /sbin/busybox sleep 25 (that is an example). In any case, you DO NOT HAVE TO WIPE DATA or WIPE ANYTHING AS SUGGESTED BY THE INSTRUCTIONS IN "Encryption Unsuccessful" message
Q: Can I move part of /data from EXT partition to NAND?
A: You don't have to move anything - that is the beauty of this mod. If, for example, you want dalvik-cache to be in NAND, all you have to do is create a directory, mount NAND to that directory, and symlink /data/dalvik-cache to that directory. Example: In init.rc, make sure /data/dalvik-cache is NOT getting created (comment that line out with a #)
Code:
mkdir /dalvik 0771 system system
mount yaffs2 [email protected] /dalvik nodev nosuid noatime nodiratime
symlink /dalvik /data/dalvik-cache
Q: How do I improve performance?
SD Card interface by design cannot handle multiple read/ write activities simultaneously. An example: Put the SD Card in a card reader attached to your computer. Copy a ~500MB file to it, see what write speeds your get (Win 7 shows an approx value). Now, while the first file is still being copied, copy another similar sized file - what happens to the write speed? It crawls, right?
When you have DataOnEXT, the SD Card is continuously being read from and written to because all your apps, their data, system data and configuration, etc. resides in the EXT partition. You SHOULD NOT expect the same speeds like a full-NAND setup. That said, there a few things you can do to improve performance:
Get a performance SD Card like SanDisk Ultra® microSDXC™UHS-I card
ODEX your ROM
See the previous question's answer for moving dalvik cache to NAND
Increase the NAND's read ahead cache from 4KB to 128 ot 256KB (in init.htcleo.rc)
Set your IO Scheduler to Deadline (use No frills CPU Control)
Q: My ROM slows to a crawl after installing this mod when I install or remove an app...
A: That is normal behavior, let it be for a min or two and it will get back to the usual speeds. This is because installing or removing an app involves creating the DEX file in the dalvik cache, extracting the components (lib files, config files, etc.) for the app and placing them in appropriate locations in /data and so on. This will slow down the system responsiveness because all of these are in the EXT partition, and the SD Card interface is a serial interface that will slow down with multiple simultaneous read/ write requests. However, doing what I have said for the previous question will improve performance quite a bit
FAQ is still WIP, I will keep adding to this section as and when...

ph03n!x said:
Devs - Would be great if you can help me with a way to reliably mount the ext partition to /data in init.rc!!!
Click to expand...
Click to collapse
The problem is with the slow sd cards, Try something of this sort.
Code:
mount ....
while [ $? -ne 0]
do
echo "Waiting for slow sd card!"
sleep 5
mount ....
done
You might have to check a2sd script since I'm not sure what is the success test case for mount, most unix programs return zero so i've based it on that assumption.

Rick_1995 said:
The problem is with the slow sd cards, Try something of this sort.
Code:
mount ....
while [ $? -ne 0]
do
echo "Waiting for slow sd card!"
sleep 5
mount ....
done
You might have to check a2sd script since I'm not sure what is the success test case for mount, most unix programs return zero so i've based it on that assumption.
Click to expand...
Click to collapse
I tried a sleep command before the part where I mount the ext partition to /data, but that doesn't seem to work. I am still figuring out the script language used in Android's init.rc, including a if loop or a whole loop.
Looks like I have to create a service that calls an external script- at least that is how far I have come with my searching
Swyped from my HTC HD2 using xda premium

ph03n!x said:
Space - Obvisiouly. The 1GB/ 2GB/ 4GB sd-ext partition will be mounted as external memory
Click to expand...
Click to collapse
Think you mean internal here, I take it internal space is still used as cache (have an unanswered question in one of Xylo's threads as to why sdcard isn't mounted as cache).
Could you post just the init.rc file to see what modifications have been made?
In terms of loop, while trying to get a squashed system up and running (last squashed rom for HD2 was on gingerbread), the squashed files mounted fine (but couldn't get the system to fully boot, probably a libs issue), the mount command used in init.rc were:
Code:
mount squashfs [email protected]/system/fonts.sfs /system/fonts ro
placed after "on fs".
So modifying for this:
Code:
mount ext4 [email protected]/dev/block/mmcblk0p2 /data rw
, failing that you may need to mount it first as say /mnt/ext then loop.
The issue with the ext mount scripts that I've found (at least with link2sd) is that there is a delay in being granted superuser permissions, using init.rc should overcome that issue.

HypoTurtle said:
Think you mean internal here, I take it internal space is still used as cache (have an unanswered question in one of Xylo's threads as to why sdcard isn't mounted as cache).
Could you post just the init.rc file to see what modifications have been made?
In terms of loop, while trying to get a squashed system up and running (last squashed rom for HD2 was on gingerbread), the squashed files mounted fine (but couldn't get the system to fully boot, probably a libs issue), the mount command used in init.rc were:
Code:
mount squashfs [email protected]/system/fonts.sfs /system/fonts ro
placed after "on fs".
So modifying for this:
Code:
mount ext4 [email protected]/dev/block/mmcblk0p2 /data rw
, failing that you may need to mount it first as say /mnt/ext then loop.
The issue with the ext mount scripts that I've found (at least with link2sd) is that there is a delay in being granted superuser permissions, using init.rc should overcome that issue.
Click to expand...
Click to collapse
I meant internal indeed - corrected it
SD Card as Cache - Not sure if I checked back on the question, but I do remember the discussion. In most recent ROMs, /cache is a part of /data - this is to make space for large apps from the market that may otherwise be limited to the size of the cache partition. Instead of /data, we can have any other writable partition - SD Card or even SD-EXT - but symlinking it to /data seems hassle free and does not require anything more. Linking to SD Card may mean that the SD Card has to always be in the phone, and it should be mounted soon enough.
I haven't checked the loop part you have given - let me do some reading about it (or you can try it and let us know!). I will driving for the next several hours, will post the init.rc later today/ tomorrow...

HypoTurtle said:
Think you mean internal here, I take it internal space is still used as cache (have an unanswered question in one of Xylo's threads as to why sdcard isn't mounted as cache).
Could you post just the init.rc file to see what modifications have been made?
In terms of loop, while trying to get a squashed system up and running (last squashed rom for HD2 was on gingerbread), the squashed files mounted fine (but couldn't get the system to fully boot, probably a libs issue), the mount command used in init.rc were:
Code:
mount squashfs [email protected]/system/fonts.sfs /system/fonts ro
placed after "on fs".
So modifying for this:
Code:
mount ext4 [email protected]/dev/block/mmcblk0p2 /data rw
, failing that you may need to mount it first as say /mnt/ext then loop.
The issue with the ext mount scripts that I've found (at least with link2sd) is that there is a delay in being granted superuser permissions, using init.rc should overcome that issue.
Click to expand...
Click to collapse
ph03n!x said:
I meant internal indeed - corrected it
SD Card as Cache - Not sure if I checked back on the question, but I do remember the discussion. In most recent ROMs, /cache is a part of /data - this is to make space for large apps from the market that may otherwise be limited to the size of the cache partition. Instead of /data, we can have any other writable partition - SD Card or even SD-EXT - but symlinking it to /data seems hassle free and does not require anything more. Linking to SD Card may mean that the SD Card has to always be in the phone, and it should be mounted soon enough.
I haven't checked the loop part you have given - let me do some reading about it (or you can try it and let us know!). I will driving for the next several hours, will post the init.rc later today/ tomorrow...
Click to expand...
Click to collapse
Alright the mount loop command you mentioned and the looping I was discussing with Rick are different. The mount loop command mounts a file (eg: a .iso image) as a device.
The loop I was looking to create is a logical if... else or a while... do loop that will check if /dev/block/mmcblk0p2 is available, else will halt the boot process until it becomes available. The challenge (for me) its that the scripting language used in init.rc its not like the scripts we create for init.d
I am still doing some reading up...
Swyped from my HTC HD2 using XDA Premium

Will this work??
Starting your own service at boot.
...If you have a reason to add something or just what to try things out however it is relatively straight forward to add a native service to that init sequence. I have created a simple example that writes the time elapsed since it started to the log. It wakes up every three seconds. The code for exampleservice.c looks like:
Code:
#define LOG_TAG "Example Service"
#include < utils/log.h >
#include < unistd.h >
int main(int argc, char **argv)
{
LOGI("Service started");
int elapsed = 0;
while(1)
{
sleep(3);
elapsed += 3;
LOGI("Service elapsed time is %d", elapsed);
}
}
Place this somewhere in the platform build system and build as an executable. The build system will place the output in the /system/bin folder on the device. Now all that is needed is to add the following to the init.rc script:
Code:
service exampleservice /system/bin/exampleservice
user exampleservice
group exampleservice
oneshot
and your service will start on init. It is also possible to write your own file using the Android init language that enables your services and variables. Then just add a line to import the file in init.rc and you are done.
Source
Click to expand...
Click to collapse
Will this method work to pause the boot process until /dev/block/mmcblk0p2 becomes available? I do not have a dev environment setup - all I have is Cygwin that I use to edit boot.img, port some ROMs to test etc. Can someone help me create such a binary please? Again, I am good at logic and bad at writing/ compiling stuff
Alternatively, can cLK be made to wait until /dev/block/mmcblk0p2 is available? Thoughts?

What is the difference to this methods:
http://forum.xda-developers.com/showthread.php?t=1716124
Are you sure we need more I/O performance with sequencial r/w? Isn't the task with multitasking sutuations to have more perfomance with random r/w? Sdcards won't give that much improvement then: http://www.tomshardware.com/reviews/microsdhc-memory-card-performance,3011-12.html

ReinerK said:
What is the difference to this methods:
http://forum.xda-developers.com/showthread.php?t=1716124
Are you sure we need more I/O performance with sequencial r/w? Isn't the task with multitasking sutuations to have more perfomance with random r/w? Sdcards won't give that much improvement then: http://www.tomshardware.com/reviews/microsdhc-memory-card-performance,3011-12.html
Click to expand...
Click to collapse
In the CronMod as well as all A2SD methods, the Linux boot init process will mount /data to the internal NAND partition (userdata). During Android boot, /data or a part of /data is remounted or symlinked to the SD-EXT partition. This method has its own set of issues, both known and unpredictable. Read up the ROM threads with A2SD or similar stuff in HD2 Android NAND forum and HD2 Android General forum and you will know. This is a trial and error method which is a hit or a miss.
What I am trying to achieve is to have the Linux boot init process mount /data directly to the EXT partition in the SD Card, making this transparent to Android. As far as Android is concerned, it will see a /data partition like it sees a NAND /data partition when you do not use any of these scripts. With this method, there is no miss that I have experienced so far in my testing, except not being able to get the SD Card device ready in time for the Linux boot init process to mount it as /data. Once that is fixed, am sure many will see the benefits.
While the benchmark you have linked to gives a comparison of random and sequential transfer rates of Flash Memory from 9 months ago (it does not even have all that many decent class6 cards that are available now), have you find anything on the transfer rates of NAND? While I remember a HD2 user benchmarking NAND transfer rates, I do not remember the exact values but do remember it was not good. From my personal opinion, the same ROM felt faster with this modification - note that after the first boot all the apps (system or user) that you see and use will be loaded from dalvik-cache which is located in /data. All app data and databases are read/ written in the /data partition. Hence the responsiveness of the phone will be directly proportional on how fast the data partition is - in my case it is my class6 SD Card's EXT partition.
There is also the issue with bad block NAND making several flashaholics' phones unusable or unstable. EXT2/3/4 partitions have better data consistency management and also have e2fsck - it would not be without a reason that Google moved to EXT4 NAND format for their recent phones
Even if you have an International HD2 with 512 MB NAND, with this method you can use almost all of that space (except spaces reserved for bootloader, recovery, misc, etc.) for /system partition which will definitely make the phone live a lot longer even with bad blocks, what with recent cLK versions letting you manipulate partition layouts.
I can keep going on the benefits of having EXT partitions and the benefits of this method, but that would beat the purpose of this thread. Let's see if we can get EXT mounted during boot to /data reliably and consistently. Whether anyone wants to use that method is, of course, their choice
By the way, have anyone used the test ROMs? Any feedback?

I think yaffs file system is used by HTC only, other droid phones already using EXT file system on NAND.
Thats why some phones has good I/O benchmark
Even if it was possible to use EXT4 on HTC HD2 nand would be big improvment

ph03n!x said:
...This is a trial and error method which is a hit or a miss.
Click to expand...
Click to collapse
That's true. Sine even tytung Roms are bigger than 150MB I used 40a2sdx, and I noticed performance problems and sometimes low sound bug. Well I can live with this, but without of cource would be better
ph03n!x said:
While the benchmark you have linked to gives a comparison of random and sequential transfer rates of Flash Memory from 9 months ago
Click to expand...
Click to collapse
I am sorry, I did not found any newer tests in english language. But I think 9 month old test are okay, cause most people use their µSDs several years (my 2009 SanDisk 16GB one just died ).
ph03n!x said:
There is also the issue with bad block NAND making several flashaholics' phones unusable or unstable
Click to expand...
Click to collapse
They should just stop doin' task29 whenever they only wanna flash a rom or change the partition layout.
ph03n!x said:
I can keep going on the benefits of having EXT partitions and the benefits of this method, but that would beat the purpose of this thread. Let's see if we can get EXT mounted during boot to /data reliably and consistently. Whether anyone wants to use that method is, of course, their choice
Click to expand...
Click to collapse
I am pleased to hear that you wanna find a way that way. I just don't have an extra phone to test your rom atm.
Anyway, thaks a lot for your explanations!

ph03n!x said:
In the CronMod as well as all A2SD methods, the Linux boot init process will mount /data to the internal NAND partition (userdata). During Android boot, /data or a part of /data is remounted or symlinked to the SD-EXT partition. This method has its own set of issues, both known and unpredictable. Read up the ROM threads with A2SD or similar stuff in HD2 Android NAND forum and HD2 Android General forum and you will know. This is a trial and error method which is a hit or a miss.
Click to expand...
Click to collapse
Reason I believe it is hit or miss is because during boot init.d scripts in most of the roms runs in parallel. hence while scripts is trying to re-mount/re-link the data partition Android boot-up continues and causes problem. if we pause the android bootup while scripts finishes its job of setting up links correctly I think problem should go away. take a look at this
Code:
# Execute files in /system/etc/init.d before booting
# service sysinit /system/bin/logwrapper /system/xbin/busybox run-parts /system/etc/init.d
# class main
# disabled
# oneshot
if we change that to
Code:
exec /system/bin/sysinit
problem with A2SD scripts should be solved of-course you will have to modify any scripts which sits in infinite loop.
take a look at my latest JB AOSP roms or miui ICS roms.
-----Update----
what if we leave both the method in there
service method to run init.d scripts
and
exec method to pause the boot-up till sd-card is mounted and sd-ext is available.
lets says
make sysinit-1 which will execute the script from init.d1 directory which will have script to check the sd-ext availability

I am close to a solution - looks like a few other devices have already mounted EXT partitions in the SD Card to /system or /data. Here is the command:
Code:
devwait /dev/block/mmcblk0p2
mount ext2 /dev/block/mmcblk0p2 /data nodev nosuid noatime nodiratime
(Check this thread).
But the problem is it is still not mounting. Does this need anything in the kernel/ init? Even the batter-pull technique is not working! The phone is booting as if it is a fresh installation, though I do not see any mounted /data partition
The other option from the same thread is to have
Code:
pause (5);
in init.c and compile is at a binary.
Another bit of information I have come across:
Code:
on emmc-fs
# mount mmc partitions
wait /dev/block/mmcblk0p13
exec /system/bin/e2fsck -p /dev/block/mmcblk0p13
mount ext4 /dev/block/mmcblk0p13 /data nosuid nodev barrier=1 noauto_da_alloc,noinit_itable,nodelalloc
(from here).
With this, the phone is going back to the Encryption Unsuccessful mode.
Pretty sleepy already, gonna call it a day for now - my poor phone has undergone a bunch of boots and reboots for a day
smokin901 said:
Reason I believe it is hit or miss is because during boot init.d scripts in most of the roms runs in parallel. hence while scripts is trying to re-mount/re-link the data partition Android boot-up continues and causes problem. if we pause the android bootup while scripts finishes its job of setting up links correctly I think problem should go away. take a look at this
Code:
# Execute files in /system/etc/init.d before booting
# service sysinit /system/bin/logwrapper /system/xbin/busybox run-parts /system/etc/init.d
# class main
# disabled
# oneshot
if we change that to
Code:
exec /system/bin/sysinit
problem with A2SD scripts should be solved of-course you will have to modify any scripts which sits in infinite loop.
take a look at my latest JB AOSP roms or miui ICS roms.
Click to expand...
Click to collapse
Thanks, I will check your ROM. I would still like to get /data to sd-ext working before Android boots though I thought I was close today, but unfortunately no!

Looks like there had been some progress to this cause, thanks to @securecrt! Check this post.
Swyped from my HTC HD2 using XDA Premium

ph03n!x said:
Mounting - /dev/block/mmcblk0p2 is not getting mounted to /data at every boot/ reboot reliably. This results is the phone not booting at all (Gingerbread) or giving an Encryption Failed, wipe data prompt (This is a feature of ICS/ JB, if /data is not accessible, the OS will boot with a temporary /data file system to show this error).
I have not been able to find a way to reliably carry out this mount - my best guess is that the SD Card Reader interface is not getting ready in time to be mounted to /data. I do not know how to make the boot process wait until the device is available.
ClockWorkMod/ Other recoveries - As the recovery boot images are still mounting userdata to /data and the ext partition to sd-ext, you will have to access sd-ext to access your data. What this means is that if you are moving ROMs, a direct CWM backup restore won't work unless the new ROM also has this modification
Performance- is only as good or bad as your SD Card. My second HD2 has a class 2 SD Card, and I do not find any performance difference. While this is not bad, the SD Card I am talking about is a decent SanDisk card. If you have something with poor transfer rates, performance will suffer
SD Card cannot be removed from the system while Android is running!!!
Click to expand...
Click to collapse
This is, obviously, not compatible with other setups, such as those using some type of ad2sd script (i.e. Amarulls), correct?
CWM Recovery backup should still work since it does backup sd-ext, no?
I need another sd card to play..

Ok, I need some help-
securecrt said:
the trick is:
1. rename the init to init.android
2. create a script named init, mount the system/data in this script, you can add a few sleep time in this to confirm the mount is alreay done and then execut init.android
3. disable the system/data mount in init.rc
Click to expand...
Click to collapse
Ever since @securecrt posted this, I have been running in circles. Here is what I have done:
Renamed init to init.android
Commented out in init.rc:
Code:
#mkdir /data 0771 system system
Commented out in init.rc:
Code:
#mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime
Created a script named init with
Code:
#!/sbin/busybox sh
export PATH=/sbin
sleep 10
mkdir /dev
mkdir /dev/block
mknod /dev/block/mmcblk0p2 b 179 2
mkdir /data 0771 system system
mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime
/init.android
Copied busybox binary to /sbin/busybox
Created the initrd.gz and flashed
The phone is booting to second boot (logo.rle loads), and then reboots.
Like I've said I have very limited Linux/ coding skills. Would be great if someone can nudge me in the right direction!

ny_limited said:
This is, obviously, not compatible with other setups, such as those using some type of ad2sd script (i.e. Amarulls), correct?
CWM Recovery backup should still work since it does backup sd-ext, no?
Click to expand...
Click to collapse
Yes, you need to remove all variants of a2sd/ link2sd scripts.
CWM Recovery will back up SD-EXT as /sd-ext, but Android will load it as /data after this mod. So if you are going to try this mod, you need to titanium backup all you system/ user app+data, install the modified mod or initrd.gz, make sure you have a ext partition, boot to Android and restore the backup from titanium. And the exact same if you are removing the mod (in which case it will be original initrd.gz).
I can sense that I am pretty close to getting this to work reliably, except for being a dunce in Linux/ Scripting. With little help this mod is a go for daily use

ph03n!x said:
Ok, I need some help-Ever since @securecrt posted this, I have been running in circles. Here is what I have done:
Renamed init to init.android
Commented out in init.rc:
Code:
#mkdir /data 0771 system system
Commented out in init.rc:
Code:
#mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime
Created a script named init with
Code:
#!/sbin/busybox sh
export PATH=/sbin
sleep 10
mkdir /dev
mkdir /dev/block
mknod /dev/block/mmcblk0p2 b 179 2
mkdir /data 0771 system system
mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime
/init.android
Copied busybox binary to /sbin/busybox
Created the initrd.gz and flashed
The phone is booting to second boot (logo.rle loads), and then reboots.
Like I've said I have very limited Linux/ coding skills. Would be great if someone can nudge me in the right direction!
Click to expand...
Click to collapse
I am sure securecrt will reveal his secret soon, m8.. I wouldn't 'waste' my time on it since he is already running it and knows how to do it
Maybe you can send him a PM if the wait is killing you ?

[Q] Error formatting /data! Error formatting /cache!

Using CWM 5.0.2.0, when I try to format /data, I get:
Error formatting /data!
When I try to format /cache, I get:
Error formatting /cache!
This all started because I decided to try Albinoman's AOSP 4.2 ROM. This is what happened:
1. Booted into CWM 5.0.2.0, made a nandroid backup of stock GB ROM.
2. Installed Albinoman's ROM using CWM, wiped /cache and /data.
3. Booted Albinoman's ROM, no problems. Decided I wanted to backup and restore some apps in the old ROM using Titanium Backup.
4. Restored nandroid backup of stock ROM. When it rebooted, all was well.
5. Used Titanium Backup to back up my apps.
6. Installed Albinoman's ROM again.
7. Cleared /data, it was fine. Cleared /cache. Stuck at "formatting" for half an hour.
8. Rebooted back to recovery and tried it again. Couldn't find E:.
9. Tried letting it boot. Stuck on the boot animation for 30-minutes.
10. Wipe everything and re-flash. Tried again. Still stuck on boot animation.
11. Manually unmount and tried to format /data and /cache, and I get the error messages.
I had installed Convert2Ext4_no_data_limit_normal_dalvik.zip to the stock ROM about 4-months ago, not sure if that makes a difference or not.
When I search, it looks like the news is bad... most people that had this error gave up and believed it to be hardware NAND failure. Anything I can do to confirm or rule out NAND failure?
UPDATE: Managed to flash RUU, but results were interesting. It worked and it boots, home screen is blank (no HTC widgets), and apps I'd installed are still there. Clear Storage and Factory Reset does not get rid of them! I can manually uninstall apps, but when I reboot, they come back. I guess the /data must be stuck in some kind of read-only state.

GnatGoSplat said:
Using CWM 5.0.2.0, when I try to format /data, I get:
Error formatting /data!
When I try to format /cache, I get:
Error formatting /cache!
This all started because I decided to try Albinoman's AOSP 4.2 ROM. This is what happened:
1. Booted into CWM 5.0.2.0, made a nandroid backup of stock GB ROM.
2. Installed Albinoman's ROM using CWM, wiped /cache and /data.
3. Booted Albinoman's ROM, no problems. Decided I wanted to backup and restore some apps in the old ROM using Titanium Backup.
4. Restored nandroid backup of stock ROM. When it rebooted, all was well.
5. Used Titanium Backup to back up my apps.
6. Installed Albinoman's ROM again.
7. Cleared /data, it was fine. Cleared /cache. Stuck at "formatting" for half an hour.
8. Rebooted back to recovery and tried it again. Couldn't find E:.
9. Tried letting it boot. Stuck on the boot animation for 30-minutes.
10. Wipe everything and re-flash. Tried again. Still stuck on boot animation.
11. Manually unmount and tried to format /data and /cache, and I get the error messages.
I had installed Convert2Ext4_no_data_limit_normal_dalvik.zip to the stock ROM about 4-months ago, not sure if that makes a difference or not.
When I search, it looks like the news is bad... most people that had this error gave up and believed it to be hardware NAND failure. Anything I can do to confirm or rule out NAND failure?
UPDATE: Managed to flash RUU, but results were interesting. It worked and it boots, home screen is blank (no HTC widgets), and apps I'd installed are still there. Clear Storage and Factory Reset does not get rid of them! I can manually uninstall apps, but when I reboot, they come back. I guess the /data must be stuck in some kind of read-only state.
Click to expand...
Click to collapse
Did you reflash the Convert2Ext4 mod after you flashed the new rom? See this thread and read the top part after 'IMPORTANT:' There is a link in there to the old thread as well with a lot of info.

I did try that, but my /data and /cache are still unable to be written to, wiped, or formatted.
e2fsck won't touch /data, it has a superblock error. It finds errors on /cache and appears to be fixing them, but when I re-run it, the same errors are still there.

GnatGoSplat said:
I did try that, but my /data and /cache are still unable to be written to, wiped, or formatted.
e2fsck won't touch /data, it has a superblock error. It finds errors on /cache and appears to be fixing them, but when I re-run it, the same errors are still there.
Click to expand...
Click to collapse
Since you flashed the RUU, did you install an EXT4 compatible kernel or are you using the stock kernel? Maybe try restoring your working nandroid backup, then run the .zip to revert Convert2Ext4. Or even better, if you have a nandroid from before you installed the EXT4 mod.

DaggerDave said:
Since you flashed the RUU, did you install an EXT4 compatible kernel or are you using the stock kernel? Maybe try restoring your working nandroid backup, then run the .zip to revert Convert2Ext4. Or even better, if you have a nandroid from before you installed the EXT4 mod.
Click to expand...
Click to collapse
Wiping /data and /cache are enough to revert the mod.
Sent from my Galaxy Nexus using Tapatalk 2

Uggh, definitely doesn't sound so good. Since the RUU won't work, I'd say check out this thread:
http://forum.xda-developers.com/showthread.php?t=2123966

Yep, my partitions are no longer EXT4. #1 is EXT3 and #2 is EXT2. I guess that means Convert2Ext4 is no longer in play and the problem is unrelated.
Here is my parted:
Code:
parted /dev/block/mmcblk0 print
Model: MMC SEM08G (sd/mmc)
Disk /dev/block/mmcblk0: 7944MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 8192B 797MB 797MB primary ext3
2 797MB 1007MB 210MB primary ext2
3 1007MB 7944MB 6937MB primary fat32 lba
Yeah, I read that thread and it didn't sound encouraging! I get pretty much the same errors.
Even rm 1 and rm 2 don't remove the partitions. It's like they're read-only. I'm using the parted from CWM 5.0.2.0. Should I use a different one?
Also when I try e2fsck on it:
Code:
~ # e2fsck -pf /dev/block/mmcblk0p1
e2fsck -pf /dev/block/mmcblk0p1
/dev/block/mmcblk0p1: recovering journal
e2fsck: unable to set superblock flags on /dev/block/mmcblk0p1
~ # e2fsck -pf /dev/block/mmcblk0p2
e2fsck -pf /dev/block/mmcblk0p2
/dev/block/mmcblk0p2: Unconnected directory inode 34817 (/???)
/dev/block/mmcblk0p2: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
(i.e., without -a or -p options)
~ # e2fsck -f /dev/block/mmcblk0p2
e2fsck -f /dev/block/mmcblk0p2
e2fsck 1.41.6 (30-May-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Unconnected directory inode 34817 (/???)
Connect to /lost+found<y>? y
yes
Pass 4: Checking reference counts
Inode 2 ref count is 2, should be 3. Fix<y>? yes
Inode 34817 ref count is 3, should be 2. Fix<y>? y
yes
Pass 5: Checking group summary information
Block bitmap differences: +274 +(449--510) +(645--701) -(139524--139541) -(139
85--139800)
Fix<y>? yes
Free blocks count wrong for group #0 (7919, counted=7799).
Fix<y>? y
yes
Free blocks count wrong for group #17 (7934, counted=7929).
Fix<y>? yes
Free blocks count wrong (198324, counted=198199).
Fix<y>? y
yes
Free inodes count wrong for group #17 (2048, counted=2045).
Fix<y>? yes
Directories count wrong for group #17 (0, counted=1).
Fix<y>? y
yes
Free inodes count wrong (51189, counted=51186).
Fix<y>? yes
/dev/block/mmcblk0p2: ***** FILE SYSTEM WAS MODIFIED *****
/dev/block/mmcblk0p2: 14/51200 files (14.3% non-contiguous), 6601/204800 blocks
When I tried it without the -p is when it looks above like it's fixing something, but it really didn't fix anything because it does the same thing when I try e2fsck again (as if it's read-only).
Is MMCBLK0 one physical NAND chip? If so, how come I can read/write/erase the /emmc with no problems? Isn't that just another partition on the same NAND disk?
If /data and /cache are really bad, is there any possible way to do an equivalent of a low-level format on the NAND and repartition?

Is booting CWM 5.0.2.0 and using adb shell supposed to have root privileges?
I can't modify the partition table using either parted or fdisk. I even tried zeroing the partitions using dd if=/dev/zero of=/dev/block/mmcblk0p1, p2, and even the whole of mmcblk0, but nothing actually gets written.

Since it seems like the NAND is probably irreparably borked, can I approach this problem from a different angle?
Can I partition my sdcard with an ext3 and ext2 partition, and tell Android to use those as /data and /cache respectively?
What in Android maps /dev/block/mmcblk0p1 to /data and /dev/block/mmcblk0p2 to /cache, and can I change those to point to partitions on sdcard?
Or is that hard-coded into the kernel?

GnatGoSplat said:
Since it seems like the NAND is probably irreparably borked, can I approach this problem from a different angle?
Can I partition my sdcard with an ext3 and ext2 partition, and tell Android to use those as /data and /cache respectively?
What in Android maps /dev/block/mmcblk0p1 to /data and /dev/block/mmcblk0p2 to /cache, and can I change those to point to partitions on sdcard?
Or is that hard-coded into the kernel?
Click to expand...
Click to collapse
It is hard coded into the kernel. To get root in adbshell try typimg su. This seems to be happening to more and more people as these phones get up there in age.

GnatGoSplat said:
Since it seems like the NAND is probably irreparably borked, can I approach this problem from a different angle?
Can I partition my sdcard with an ext3 and ext2 partition, and tell Android to use those as /data and /cache respectively?
What in Android maps /dev/block/mmcblk0p1 to /data and /dev/block/mmcblk0p2 to /cache, and can I change those to point to partitions on sdcard?
Or is that hard-coded into the kernel?
Click to expand...
Click to collapse
The answer is yes to both actually. Thrle boot.img would need to be customized. The best way is a flashable zip in recovery. Data and cache would need to be wiped manually ad recovery will not pick this up. Expect poorer performance though as SD cards tend to be slower than built in storage.
Sent from my Galaxy Nexus using Tapatalk 2

Thanks! So it sounds like the way to make this work is to compile a custom kernel? I don't know the first thing about it, but I think I could learn.
This is only in the kernel, so I could flash any 3rd party D-Inc ROM and just flash my customized kernel and it will see /data and /cache on the sdcard?
I could probably get one of those fast Class 10 cards if it turns out this is too slow, but I do recall changing vold.fstab once to fool Android into seeing /emmc as /sdcard and the /emmc (as /sdcard) didn't benchmark all that much faster than a real sdcard. I'm assuming /data and /cache are the same speed as /emmc since they're on the same mmcblk0 device, so I'm thinking maybe sdcard performance would be acceptable.
This actually sounds like it could be a better long-term solution than replacing the motherboard because no telling if a new motherboard will fail soon too.

The motherboard should be reliable. Emmc may die. Especially considering how old current dincs are.
I was talking about the boot.img which is not ROM independent and kernels aren't always either. Best idea is a flashable zip to modify the ram disk but I need the partition names if you decide to do it. I recommend keeping cache and data in the same order.
Also there would need to be a flashable wipe script to wipe data and cache.
Sent from my Galaxy Nexus using Tapatalk 2

Sorry, I wasn't very clear. By motherboard failing, I was assuming NAND is soldered to the motherboard, and if that's the case, ruined NAND = ruined motherboard. I had thought maybe I could buy a DInc with smashed LCD and good motherboard and just swap into mine (because mine is MINT condition), but I thought if I do that, I could very well end up in the same place as I'm in now and using sdcard would be more of a guarantee I wouldn't have to worry about this again.
I think I understand, the boot.img and kernel aren't exactly the same thing, but the kernel is part of boot.img? Or is it only the RAMDisk that needs changed, or both? You mentioned partition names, are you offering to make these modifications for me?
By same order, do you mean because mmcblk0p1 is /data, mmcblk0p2 is /cache, and mmcblkp3 is /emmc, I should partition my sdcard the same way with data 1st, cache 2nd, and sdcard 3rd?
Should I partition it with the phone, PC, or does it matter? Should the sizes of the new /data and /cache be the same size as original, or is there any reason I should make them larger?
Could I wipe the new /data and /cache simply by booting into Recovery and using adb shell and format them?
Also, the commands I was issuing in adb shell were from booting into CWM Recovery. Is it possible that mode can't write because I didn't login as root using the su command? I hadn't thought that would be the problem seeing as how the RUU and Android itself can't write to /data and /cache and other people managed to fix partition problems without needing to run su.

GnatGoSplat said:
Sorry, I wasn't very clear. By motherboard failing, I was assuming NAND is soldered to the motherboard, and if that's the case, ruined NAND = ruined motherboard. I had thought maybe I could buy a DInc with smashed LCD and good motherboard and just swap into mine (because mine is MINT condition), but I thought if I do that, I could very well end up in the same place as I'm in now and using sdcard would be more of a guarantee I wouldn't have to worry about this again.
I think I understand, the boot.img and kernel aren't exactly the same thing, but the kernel is part of boot.img? Or is it only the RAMDisk that needs changed, or both? You mentioned partition names, are you offering to make these modifications for me?
By same order, do you mean because mmcblk0p1 is /data, mmcblk0p2 is /cache, and mmcblkp3 is /emmc, I should partition my sdcard the same way with data 1st, cache 2nd, and sdcard 3rd?
Should I partition it with the phone, PC, or does it matter? Should the sizes of the new /data and /cache be the same size as original, or is there any reason I should make them larger?
Could I wipe the new /data and /cache simply by booting into Recovery and using adb shell and format them?
Also, the commands I was issuing in adb shell were from booting into CWM Recovery. Is it possible that mode can't write because I didn't login as root using the su command? I hadn't thought that would be the problem seeing as how the RUU and Android itself can't write to /data and /cache and other people managed to fix partition problems without needing to run su.
Click to expand...
Click to collapse
The NAND is likely soldered to the motherboard, so you're right about that.
To answer your first question, yeah, the kernel is part of the boot.img. The other part is the RAMdisk which is the only thing that needs to be changed.
Partition it in the same order. Probably parted via adb shell. 100MB for cache and at least 750MB for data. Sizes don't matter too much but you want enough.
adb shell has root access automatically in recovery. The issue is hardware most likely so your proposed workaround may be the only option.
First plan of attack is to find the partition names which would be mmcblk1p1 and mmcblk1p1 if you partition the sdcard the same way as the ROM is partitioned. Ideally you need to be able to fix the partitions themselves. I'll try to give more info later.

I copied the partition sizes from mmcblk0, so here's what I have on my 16GB SD card:
Code:
Model: SD SA16G (sd/mmc)
Disk /dev/block/mmcblk1: 16.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 8192B 797MB 797MB primary ext2
2 797MB 1007MB 210MB primary ext2
3 1007MB 16.0GB 15.0GB primary fat32 lba
mmcblk1p1 for /data
mmcblk1p2 for /cache
mmcblk1p3 for /sdcard
Will that work? I couldn't get mmcblk1p1 to show up as ext3. I did use mke2fs -j /dev/block/mmcblk1p1 to format it and I've read ext3 = ext2+journal, so I guess the fact it's not saying "ext3" under filesystem is okay?

GnatGoSplat said:
I copied the partition sizes from mmcblk0, so here's what I have on my 16GB SD card:
Code:
Model: SD SA16G (sd/mmc)
Disk /dev/block/mmcblk1: 16.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 8192B 797MB 797MB primary ext2
2 797MB 1007MB 210MB primary ext2
3 1007MB 16.0GB 15.0GB primary fat32 lba
mmcblk1p1 for /data
mmcblk1p2 for /cache
mmcblk1p3 for /sdcard
Will that work? I couldn't get mmcblk1p1 to show up as ext3. I did use mke2fs -j /dev/block/mmcblk1p1 to format it and I've read ext3 = ext2+journal, so I guess the fact it's not saying "ext3" under filesystem is okay?
Click to expand...
Click to collapse
You would be correct. Yes, that's fine.

Thanks, what should be my next steps?

GnatGoSplat said:
Thanks, what should be my next steps?
Click to expand...
Click to collapse
I'll have to find time to make scripts that will modify the ramdisk, unless you can just extract it and make the changes yourself.

That would be great, I really appreciate all the help so far.
Do you think ramdisk is the only place a change will need to be made?
I was trying to see if I could figure out how to extract ramdisk from boot.img and found a perl script, but I haven't gotten around to playing with it.