Here is the kernel with yaffs2 and MTD block support, I am missing some mtd util tools like nandwrite so i cannot continue. Those that have the tools like mkyaffs2image, nandwrite, flash_eraseall can try to create an image and mount it
here are the steps that i think should work:
# mknod /dev/mtd0 c 90 0
# flash_eraseall /dev/mtd0
# mkyaffs2image /data/data yaffs2.img
# nandwrite -p /dev/mtd0 yaffs2.img
# mkdir /data/data2
# mount -t yaffs2 /dev/mtdblock0 /data/data2
If you have the tools, please attach so we can continue.. Its getting late so I am going to bed, have fun
Download
yaffs2_support.tar
Oh my god! I was just about to post similar findings, what a coincidence!
I just had compiled a kernel with yaffs2 support. My idea was to modify /init.rc in the kernel to mount the partitions with yaffs2 instead of rfs. Problem: we can't just change the filesystem without backing up and restoring all the files inside. My idea was to do that inside the recovery. Mount a mountpoint, backup with tar to /sdcard, unmount, format with dd (in theory a partition filled with 0's is ok for yaffs2), mount again with yaffs2 and restore from tar.
I'm stuck at the point that I can't seem able to mount any partition with yaffs2 previously formatted (with /system/xbin/busybox dd if=/dev/zero of=/dev/block/stl10). If I mount (/system/xbin/busybox mount -t yaffs2 /dev/block/stl10 /dbdata) there is always an "Invalid argument" error. It happens with /dev/block/stl*, /dev/block/bml* and /dev/block/mmcblk*
I'm no kernel expert, neither I know a lot about yaffs2, but I think I read you can't mount a yaffs2 partition anywhere, it has to be an MTD flash device.
Anyway, I'm going to try with the tools you said. Wish me luck.
Thanks for your work rasdeep! Is it only YAFFS2 and MTD support that you added? Because the kernel already had support for EXT2 and EXT3.
darksnarfer said:
I think I read you can't mount a yaffs2 partition anywhere, it has to be an MTD flash device.
Click to expand...
Click to collapse
Does the kernel that rasdeep posted load the existng flash devices/partitions as MTD devices?
I think as a start you only have to worry about formatting the /dev/block/mmcblk0p2 device which is mapped to the /data directory. With the new lag fix we are only moving directories stored in /data and that is what gives a big performance improvement. RyanZA did tests and found that the /dbdata partition was already quite fast and I don't think we have to worry about the /system partition etc. because we don't really write to them.
I think after a new firmware flash the /dev/block/mmcblk0p2 (mounted as /data) will be safe to format into YAFFS2 without loosing anything important.
I think these are the steps we can do:
1. Flash new firmware with repartition option on to erase everything.
2. Boot into recovery and format /dev/block/mmcblk0p2 as YAFFS2 filesystem.
3. Copy on new a init.rc script which will mount /dev/block/mmcblk0p2 to /data as a YAFFS2 filesystem and not a RFS filesytem to get much faster speeds.
bub181 said:
Does the kernel that rasdeep posted load the existng flash devices/partitions as MTD devices?
Click to expand...
Click to collapse
Just checked, there isn't any /dev/block/mtd* so I guess the kernel still can't see them as mtd.
/data is on the internal SD not in NAND so will we be able to use yaffs2 at all? I was working on formatting to EXT4 and adding the init.rc to match as well as compiling with EXT4 support, but if yaffs2 works, I'm fine with that.
modifying the init.rc is secondary because that is easy.. the main problem now is to be able to manually setup yaffs2 to mount properly.. if everything works like it should, init.rc can be change in matter of seconds
its funny now u said mtd not found.. i know we can mknod /dev/mtd* but there should be /dev/mtdblock* when booting with kernel right? hmm.. something to investigate further..
keep suggestions and finding coming, we may hit something that i've missed
problem is im lacking those tools that will do write to flash directly.. for example nandwrite.. anyone has that can share so i can try more stuff?
also, are my steps for "preparing" yaffs2 partitions/image correct? if anyone familiar with yaffs, please give some insight
raspdeep said:
modifying the init.rc is secondary because that is easy..
Click to expand...
Click to collapse
Do you need to recompile the kernel to modify the init.rc? If not could you post some instructions on how to do it please? I would like to try some things in there before /data gets mounted as a RFS filesystem.
bub181 said:
Do you need to recompile the kernel to modify the init.rc? If not could you post some instructions on how to do it please? I would like to try some things in there before /data gets mounted as a RFS filesystem.
Click to expand...
Click to collapse
i'm afraid so for now until someone found a way to extract the initramfs from later kernel and restore it back after modification..
I think it would be easier to start out with EXT2 since we can easily format that. Just as a proof of concept. I'm planning to do so tonight. Then I'll add support for EXT4 formatting and try that.
raspdeep said:
problem is im lacking those tools that will do write to flash directly.. for example nandwrite.. anyone has that can share so i can try more stuff?
Click to expand...
Click to collapse
Have you seen these pages rasdeep? They seem to describe how to create the YAFFS filesystem. Also, are there any /dev/mtdblock* devices using the kernel that you created? According to what I read elsewhere it is supposed to create both /dev/mtd* and /dev/mtdblock*. I think we have to use /dev/mtdblock* to create the new filesystem.
http://processors.wiki.ti.com/index.php/Create_a_YAFFS_Target_Image
http://processors.wiki.ti.com/index.php/MTD_Utilities
http://processors.wiki.ti.com/index.php/Put_YAFFS_Image_to_Flash
Also, I know that the Nexus One uses YAFFS2 for its internal filesystem. Maybe someone has worked out a way to write a new image file for that and we can use what they have learnt?
Can we potentially use this kernal (with the right tools) to format the NAND in YAFFS2 instead of RFS? Then still have way of flashing firmware onto it?
If so, that would be awesome and it would seem to have the least draw backs.
Currently the A2SD lag fix requires an external microsd card, and the one click lag fix, mounts a EXT2 image ontop of the RFS NAND, and that could drastically affect NAND life.
This is great !!!
Thank you raspdeep and all the devs here, this is one step closer to a stable kernel and a permanent genuine solution for the lag problem as far as a n00b like me understands
cheers !
Daemos said:
Can we potentially use this kernal (with the right tools) to format the NAND in YAFFS2 instead of RFS? Then still have way of flashing firmware onto it?
If so, that would be awesome and it would seem to have the least draw backs.
Click to expand...
Click to collapse
Not if we can't mount the NAND as a MTD-device instead of STL-device.
Daemos said:
Currently the A2SD lag fix requires an external microsd card, and the one click lag fix, mounts a EXT2 image ontop of the RFS NAND, and that could drastically affect NAND life.
Click to expand...
Click to collapse
Say what? /data/ is mounted on the internal SSD, not the NAND. /dbdata/ is however mounted on the NAND. Just do a df -h and you can see all the mountpoints.
Also, mounting and using a ext2-image that reside on an rfs-partition will not damage your drive. The rfs filesystem is still the one used to store the image itself (the ones and the zeros), even though you write it inside an image that uses another filesystem.
I stumbled upon these documents while Googling for STL-devices:
http://www.samsung.com/global/busin...y/downloads/LinuStoreII_GPL Compliance_10.pdf
http://www.samsung.com/global/busin...ionmemory/downloads/RFS_130_Porting_Guide.pdf
I haven't had the time to read them through yet. But if there's a porting guide, there must be a way to reverse it
Go go reverse engineers!
Is it really necessary to format the NAND with another filesystem? /dbdata seems fast enough as it is. Wouldn't it be enough (and easier?) to only convert the internal SD to yaffs2?
Quote from someone that knows a lot more than I do:
yaffs2 won't work on a sdcard
Click to expand...
Click to collapse
http://twitter.com/wesgarner/status/20805997718
So right now we have /data that is mounted in the second partition of the internal sdcard (mmcblk0p2), where all the critical files are stored (data, dalvik-cache, etc..) but we can't use yaffs2 there because it's not MTD.
We could, however, format and mount the other partitions (/system, /dbdata, /cache, etc...) with yaffs2, but that might not help because there isn't many heavy read/write there. And we should figure out how to make the kernel see them as /dev/block/mtdblock* instead of /dev/block/stl*.
rasdeep, can you share the .config for the kernel you used? I managed to get some mtd devices (atleast in /proc/mtd) but they were virtual devices. Maybe we need some more options enabled in the kernel.
Also, anybody knows why there are /dev/block/stl* and /dev/block/bml* they seem the same though.
darksnarfer said:
Also, anybody knows why there are /dev/block/stl* and /dev/block/bml* they seem the same though.
Click to expand...
Click to collapse
According to http://www.samsung.com/global/busin...ducts/fusionmemory/Products_XSR_Brochure.html
STL and BML are independent layers for managing I/O to the NAND devices
STL manages the sector translations while BML manages the invalid blocks (whatever that means)
Did any of you notice that in init.rc there are many references to yaffs2 ? Perhaps in engineering Samsung did use it ?
Chainfire said:
Did any of you notice that in init.rc there are many references to yaffs2 ? Perhaps in engineering Samsung did use it ?
Click to expand...
Click to collapse
Yeah, I think they are just android standard (maybe they took them from a template) and commented to use their better(?) filesystem.
darksnarfer said:
Quote from someone that knows a lot more than I do:
http://twitter.com/wesgarner/status/20805997718
So right now we have /data that is mounted in the second partition of the internal sdcard (mmcblk0p2), where all the critical files are stored (data, dalvik-cache, etc..) but we can't use yaffs2 there because it's not MTD.
We could, however, format and mount the other partitions (/system, /dbdata, /cache, etc...) with yaffs2, but that might not help because there isn't many heavy read/write there. And we should figure out how to make the kernel see them as /dev/block/mtdblock* instead of /dev/block/stl*.
rasdeep, can you share the .config for the kernel you used? I managed to get some mtd devices (atleast in /proc/mtd) but they were virtual devices. Maybe we need some more options enabled in the kernel.
Also, anybody knows why there are /dev/block/stl* and /dev/block/bml* they seem the same though.
Click to expand...
Click to collapse
Ok, so what filesystem does the internal SD use currently? Would it help if we change that to EXT4? That would emulate the current mimocan fix, but without having to use an external SD card.
I've noticed that everywhere in the ROM threads people using apps2sd seem to partition their SD-cards with ext3. I have a bit of experience with Linux filesystems and I am quite baffled by the choice, especially for flash media, where wear is a very real concern.
Some of the reasons I've seen for people choosing ext3 over ext2 are just plain wrong. E.g. speed; people say that ext3 is faster because it does journaling as opposed to ext2 that doesn't. This is not true, journaling actually makes file operations slower because the journal needs to be updated for every single operation. Journaling makes filesystem recovery faster, because the journal will show exactly which operations did not complete successfully, but this is not very useful due to the relatively small sizes of the partitions and the fact that the phone doesn't crash that often.
Keeping the journal also increases wear on the SD-card as the journal is written to on every operation, and its very specific location means that the SD-card is worn at this location and may make it unusable sooner than normal use would.
So if anyone has some insights into why ext3 seems to be preferred over ext2, then I'd love to hear them, as this has been bothering me for some time.
- RK
Due to the fact that the SD is removable/un-mountable media and its tied in and recognized as "internal memory" by the system, journaling safeguards the data as it would during a system crash under Linux, not sure on the speed my self but the rest is true to the best of my knowledge, its always been the "preffered" format.
Sent from LeeDrOiD loaded HTC Desire using Tapatalk
Moved to Q&A. Please don't post question threads in Development.
Journaling does not actually keep your data safer, it only speeds up the recovery time, as the recovery operation does not have to walk through the entire inode structure to confirm that the file system is consistent. As I said, the size and number of files on the ext-partition make a journal pretty pointless, especially since random access times are very low on flash media, which makes the recovery walk even faster.
So I still don't agree with the choice of ext3 over ext2.
PS. Thanks for correcting my mistake, rofld.
Has anyone successfully reformatted either /sdcard or /sdcard/external_sd/ /emmc as ext3? I'm not talking about an Apps2SD partition, I'm talking about reformatting the whole volume as one partition, transparent to apps. I do not use Windows so I do not need interoperability, I am fine with ext filesystems.
My goal is to be able to use symlinks (not to mention a more robust filesystem) to redirect certain hard-coded items to store on a different storage volume than it defaults to, but symlinks are not permitted on vfat (FAT32) filesystems.
Thanks!
Anything that I look an answer for, is just not answered. :-(
I am looking for the same information. Wonder why the native ext file systems are not allowed, given that its a Linux kernel running.
You could format it to ext3 and see how it works. Worst case scenario is your phone will tell that sdcard to fu*k off. Given that our phones can and do read the ext format I'm sure you can do it. I'm just not sure how well it will work as far as mounting and such.
On top of that, Windows can read ext formats. You just need a program to do it. I use it all the time.
Thanks for reviving this thread. Surely there must be someone out there who has tried this, and I was hoping to hear from them before I gave it a go, but looks like I'll just have to give it a shot myself.
When I find the time to backup & wipe my sdcard to try and reformat & mount it, I'll report back here.
With my Galaxy Note N7000 I can format a USB Stick to EXT3, and the phone uses it perfectly well as USB Storage...
(rooted, w/w SpeedMod kernel)
Hi,
I have searched for the answer to my question so if I have just missed a posting please direct me to it...
I have CWM 5.0.2.8 on my Wildfire S. This phone uses yaffs2 filesystems.
I can backup and restore via CWM with no problems.
If I backup my phone (yaffs2) and then flash a custom ROM that uses a different type of filesystem (ext3 or ext4) will CWM be able to reformat my phone as yaffs2 as it restores it to its original state??
I have seen various posts, some saying this feature is included in CWM and works, others saying it doesnt.
Has anyone actually done this and if so can you let me know if it works or not with CWM 5.0.2.8.
Otherwise I assume I could try to flash a ROM that uses yaffs2 and then restore but Im reluctant to mess about too much in case I make a fatal bodge that renders my phone either unable to get back to its current state or end up flashing an RUU which (from some research) would maybe relock the bootloader on my phone and make it impossible to re-unlock again via HTCDEV.
Or is there a way to revert ext3/4 back to yaffs2 before a restore takes place, eg some other recovery, Ive seem a recovery that can go from ext3 to ext4 and vice versa but havent found one that can go yaffs2 to ext3/4 and vice versa.
Regards,
Scott
forum.xda-developers.com/showthread.php?t=1497086
1. I know no ROM which uses extX file system.
2. I would not recommend using such a rom either because it would wear out the phone's memory.
Why? ext is a filesystem optimized for hard disks which can be written a billion times. The nand is based on flash and you can only write from 1.000-10.000 cycles on each page.
especially ext3 and ext4 are journalling filesystems and those tend to need many many many write accesses. even if you do not use your phone. all the day. So the memory "grows older" much quicker.
this was the initial reason to create special filesystems for flash disks.
My DInc NAND went bad, it became read-only and could not be written to. This is the 8GB NAND that houses /data, /cache, and /emmc partitions.
Tiny came out with a nice little patch for me, to patch custom ROMs in order to use partitions on the physical SD card as /data and /cache. That worked well for awhile, though with an occasional glitch where I'd get random factory resets. Anyway, I don't use the DInc anymore, thinking to fix it with a new motherboard from a cracked screen phone or something and selling it. Probably won't hardly go for anything, but it's not doing me much good sitting around. With that plan in mind, I decided to pull the bad NAND chip just to see how hard that is. Got a new SMD rework station and figured it's something I could goof off with on a boring Saturday morning. It came out without much fuss.
Anyhow, now the phone won't boot CM11 with the NAND chip missing. The phone does try to boot because /boot and /system are on a different NAND chip, but now it's not seeing /data or /cache on the physical SD card, at least that's my guess. I do plan to replace the motherboard, but mainly just curious why it no longer works. Is because of partition order? Because I lost 3 bad read-only partitions, the partition count is now off?
I knew it was possible I'd end up killing my DInc by pulling the bad NAND and I'm not too broken up about it. Mainly just curious why it no longer works, and kind of wondering if it's possible to fix.
Figured out some stuff.
With bad NAND chip removed, MicroSD takes its place as /dev/block/mmcblk0.
In a way, this is kind of good. If I make 3 partitions on MicroSD, I no longer need to use Tiny's patch or a custom recovery to map everything to mmcblk1. I can now use the same TWRP recovery everyone else uses and it recognizes mmcblk0p1 as /data, mmcblk0p2 as /cache, and mmcblk0p3 as /emmc.
Even CM11 and Evervolve 4.4.4 work without any patch and recognizes mmcblk0p1 as /data, mmcblk0p2 as /cache.
However, neither ROM maps mmcblk0p3 to anything. /sdcard, /emmc, /storage/sdcard0, /storage/sdcard1, none of those are mapped to mmcblk0p3.
UPDATE: Figured it out. Had to change /devices/platform/msm_sdcc.3/mmc_host/mmc2 to /devices/platform/msm_sdcc.2/mmc_host/mmc1 in fstab.inc.
just wondering, how are you determining that you have a bad nand chip? are you running any sort of test that can tell you definitely?
I have 3 dincs, 2 that I bought refurbished and used on eBay that seem to have been repaired(the water damage sticker has been tripped). I always assumed that the HTC Dinc was just a weak, limited and terrible phone due to bad 1st generation design. My phones had been prone to rebooting and errors due to the small system app partition. But they have worked reasonably well for me the last 3-4 years.
I upgraded all 3 to the latest evervolv 4.0 version with TWRP 2.6.3 and 2 of the 3 seem to have taken the upgrade well. have the 3rd one constantly reboots when trying to access anything via the play store. The internal 8 gig EMMC card had disappeared from it and also at times, the wifi controller disappears. I down graded it all the way back to Tiny 5-16(which it had run for the last 2 years) and it stabilized for the most part with the EMMC being seen again. Do you think the missing EMMC partition is a sure indicator that the memory on the phone is bad?
Here is the full thread:
http://forum.xda-developers.com/showthread.php?t=2175346
But long story short, I couldn't write to the eMMC chip at all. Not only my /emmc partition was read-only like some people had happen and could fix with fsck_msdos, but my entire eMMC chip, which means /data, /cache, and /emmc were read-only. RUU couldn't fix it, fsck couldn't fix it, and even attempting to remove and rebuild partitions using parted couldn't fix it.
I'm not sure if losing the /emmc partition by itself means the chip is bad. The chip also has the /data and /cache partitions. I'd think if either of those go offline, all your apps will start to FC. Although you did say it constantly reboots, and I think that's possible when /data and /cache go offline. I think mine actually did reboot an awful lot on its own before the eMMC chip went completely bad. You might try running e2fsck -c to check for bad blocks.