Related
What I need...
Someone to repackage a boot.img for me. Specifically, I need a boot.img unpacked, replace the file "/init.inc.rc" with my version, and repack it. (I'm stuck on all Windows, and cygwin is not an option.)
Purpose/Goal
Fix the annoying fact that /emmc is not accessible to almost all market apps, without resorting to using the modified media scanner from CyanogenMod or anything else drastic.
Do not change fstab. /mnt/emmc and /mnt/sdcard will not change.
NOTE: I am not trying to create a symlink on the FAT32 of the SD card.
My idea
Modify /init.inc.rc to change the EMMC settings. (Changes made, need someone with tools. See above.)
Modify /system/etc/vold.fstab to relocate EMMC mount point.
Directory structure
/sdcard (unchanged)
/sdcard/emmc (new location of emmc)
/emmc (Retarget the symbolic link to new location.)
/mnt/emmc (Change to symlink for compatibility, just in case.)
What I want to know...
This arrangement makes /emmc redundant. Can it be safely removed?
When you connect to a PC, how/what does Android map to USB drives? (I assume it is the mount blocks, not the symlinks.)
What odd behaviors may occur? (e.g. will "Settings->SD & phone storage" freak out?)
Maybe, just maybe: How to bottle this into a flashable .zip? (I'll worry about that later.)
If anyone has any pull with the ROM devs (i.e. Koush, rmk40, et al.), I really want to hear from them.
+1. I'd love to make emmc usable in Winamp
Bzzzt... Sorry.
ARGH!
The internal storage is VFAT also. No symlinks allowed.
Blast you, HTC!
What??
I hate doing this, but editing the original post doesn't bump the thread.
I did not want to create a new thread, but what I need has completely changed.
So. BUMP.
Why HTC, WHY?!
Would there be a way to format emmc to NOT be VFAT?
Progress!!!!
I'm close to getting a boot.img ready to test out on my phone.
I spent yesterday hacking away. I've made a Windows tool for handling boot images. I'm about 80% done with the required features. I have all the unpacking finished. I've repacked the ramdisk. I can generate the SHA hash to sign the image. It's just a matter to gluing the last stages together.
ppd0526 said:
Why HTC, WHY?!
Would there be a way to format emmc to NOT be VFAT?
Click to expand...
Click to collapse
I'm sure it could be done, but this is way more work/hassle that I want to deal with. Major obstacle here is that ALL data on emmc is wiped, and I'm not willing to that.
Assuming there are no hidden "features," my approach should be invisible to the Android layer (i.e. will not break any HTC software). HOWEVER... I'm pretty much Senseless, so I'm not too concerned (for my own use) about HTC's apps.
My biggest concern is that the media scanner will generate duplicates of all files found on emmc. But a/the big motivator for this work is to use Winamp, so I don't really give a whoop.
This was tried by koush when we couldn't get cm6 to scan emmc. It didn't work.
Sent from my ADR6300 using XDA App
distINCtINC said:
This was tried by koush when we couldn't get cm6 to scan emmc. It didn't work.
Sent from my ADR6300 using XDA App
Click to expand...
Click to collapse
What didn't work? Or how did it not work?
I'm not sure. I wasn't involved in the effort. Koush would be the person to talk to but island how to get a hold of him. I just think the OS rejected mounting a physical drive within a physical drive. But don't let that discourage you. It might be still be possible.
Sent from my ADR6300 using XDA App
Any progress on this? I really would love to have /emmc accessible as /sdcard.
Why HTC decided to make /data/data so small and put the rest of the space as /emmc is still a mystery to me. This is my only gripe with this phone.
Progress update
I made my first attempt on Friday, which resulted in a boot loop. So, obviously, I'm missing something important. (Battery pull and recover boot fixed the problem.)
I couldn't work on it this weekend. But I'm going to start digging into it some today.
More info:
It didn't get past the slash screen. Since I'm flashing a boot image, this is obviously where the problem is located. I just don't have any error logging to read over. Since I'm cooking on Windows, I'm wondering if my file permissions and ownership are messed up.
Also, while running my update, I observed that the boot image flashing did not seem to take long at all. So, I'm concerned that I didn't even get a complete flash.
weareallkosh said:
I made my first attempt on Friday, which resulted in a boot loop. So, obviously, I'm missing something important. (Battery pull and recover boot fixed the problem.)
I couldn't work on it this weekend. But I'm going to start digging into it some today.
More info:
It didn't get past the slash screen. Since I'm flashing a boot image, this is obviously where the problem is located. I just don't have any error logging to read over. Since I'm cooking on Windows, I'm wondering if my file permissions and ownership are messed up.
Also, while running my update, I observed that the boot image flashing did not seem to take long at all. So, I'm concerned that I didn't even get a complete flash.
Click to expand...
Click to collapse
That doesn't bode well so far.
just some info
i'm not sure if this will help any of you, but i read before that it said koush tried to get the emmc mounting.. etc... and it didn't work. but - i'm running cm 6.1 stable and winamp can and does read my music found on the internal memory. so it seems to work for me. however - i cannot take picture and have them stored on the internal memory. anyway, good luck.
OK.... I got the ramdisk sorted out. The boot loop is gone. But it doesn't progress past the splash1. sigh.
I need to see the kernel messages, and typically ADB is not available.
drwndphish said:
just some info
i'm not sure if this will help any of you, but i read before that it said koush tried to get the emmc mounting.. etc... and it didn't work. but - i'm running cm 6.1 stable and winamp can and does read my music found on the internal memory. so it seems to work for me. however - i cannot take picture and have them stored on the internal memory. anyway, good luck.
Click to expand...
Click to collapse
CM6 uses a modified media scanner (Android layer) that uses '/mnt' as its base directory, instead of '/mnt/sdcard.' And, as you say, it has its own problems.
I am trying to modify the underlying file system to make hopefully all software work (e.g. HTC stock, Winamp and other media players, 90% of the apps I've played with).
It "works!"
Alright... I have all the tools made, and bugs squashed. So, I can correctly mod a boot image. (Tip: The boot process has a zero warning or error tolerance.)
It fully booted. BUT... A permanent notification "Preparing phone storage.../Checking for errors." did not go away. All emmc directories were absent.
So, I've found a problem. Google didn't return any useful information... Especially annoying is the face that I don't even get a link to the Android source to even get a hint where this notification comes from.
AHA! Fixed that problem. Now, emmc is visible on the sdcard.
NEXT problem (this one I expected):
Media scanner picked up 2 copies of my pictures (I have them on emmc). So, my thought is to start removing links (and references) to emmc in its new location until things clear up.
weareallkosh said:
Alright... I have all the tools made, and bugs squashed. So, I can correctly mod a boot image. (Tip: The boot process has a zero warning or error tolerance.)
It fully booted. BUT... A permanent notification "Preparing phone storage.../Checking for errors." did not go away. All emmc directories were absent.
So, I've found a problem. Google didn't return any useful information... Especially annoying is the face that I don't even get a link to the Android source to even get a hint where this notification comes from.
AHA! Fixed that problem. Now, emmc is visible on the sdcard.
NEXT problem (this one I expected):
Media scanner picked up 2 copies of my pictures (I have them on emmc). So, my thought is to start removing links (and references) to emmc in its new location until things clear up.
Click to expand...
Click to collapse
The double items in media scanner seems like a small issue compared to what it fixes.
More problems created than fixed. (And WHY this is a mess to begin with.)
Postmortem thus far...
Touching ANY code related to where emmc appears will cause Setting to FC if you view the SD & phone storage usage.
Makes media visible to "other" applications, but shows duplicates in HTC's music and gallery apps.
I didn't look too closely at it, but I question that the camera was playing nice with the new config.
ppd0526 said:
The double items in media scanner seems like a small issue compared to what it fixes.
Click to expand...
Click to collapse
SO... I took some time to understand the double items, and why EMMC is such a problem to begin with.
HTC did not modify the media scanner/provider to support EMMC. They modified their APPS. Do a dump of HTC's music and gallery apps. You'll find a LOT of extra code, and a lot of added symbols/strings for handling the phone storage.
In other words, they didn't create a "public" solution. They just made their own private patch, and brushed it under the rug. (How dare we not use THEIR apps???)
Media Scanner/Provider:
I looked at Koush's code changes. I don't understand his changes. I'm not sure how/why it "works." I don't know why it reportedly breaks the HTC apps. I am also not sure that their is a way to mod it to actually fix the issue... Google's code was not written to be extensible. But, I think it may be the way to go in the long run.
What other phones have EMMC? Do they have a fix?
My tools...
There is a lot of code duplication, no GUI, or options. It ain't exactly pretty, but it works.
I'm posting this here for future reference by others wishing to work with boot.img.
Note: This specifically targets 1 file on the ramdisk (init.inc.rc), but with some changes this could do almost anything you would want.
License... Oh. GPL. If you make changes, please send me a patch.
REQUIRES: AutoHotkey (Hey.... it's all I had available, but the code should be easy for anyone to follow and port to another language.)
Runs on Windows (XP). Does not require cygwin. Native GZIP included.
Hello everyone,
I've compiled a kernel which separates the NAND memory into 3 partitions (well more then that but anyway). These partitions are system, data and cache like a native Android phone (which also has recovery but we don't need that). Previously cache was linked to a directory on the data partition which was constantly becoming corrupt causing errors on start-up. Often clearing dalvik-cache would allow Android to boot but data was lost. There's nothing to say that this will solve the problem but i'm giving it a go so thought I would post it up for you to try also.
The cache partition is 20mb which means the data partition is only 90mb or so. If you like a million apps on your phone either put some on your SD card or don't use it
I haven't completely incorporated this partition into the initrd which means that currently it will only mount if data is also on NAND but this can be changed easily enough, but for testing purposes I want everything on the NAND chip anyway.
Downloads from market on all versions of Android, unless the init.rc directory is changed in Gingerbread, all go to the cache partition and are deleted on restart. What this means is that you can't download and install an app larger then 20mb (which isn't recommended anyway with the small amount of data space available). This is a drawback but for those that use data on SD it should be possible in the future to utilise more of the NAND memory for cache so you can install whatever you want. I don't know of many apps larger then 20mb anyway though.
I have included a modified version of ATools in the zip. The standard version will overwrite the partition values and the kernel won't work properly so if you need to modify the kernel use the version included. It is also important that you DO NOT change the system or data partition sizes in ATools as this hasn't been modified to account for the cache partition so will mess it all up.
I have also included an androidupdate.tar modules update for the kernel in case your wifi doesn't work after flashing the kernel.
I think that's it for now, let me know how it goes!
EDIT::
One last thing, you will need to wipe your DATA partition to use this kernel because some of your data may be positioned where the cache partition is now, so if you load this kernel and android is heavily corrupt you know what you have to do!
To check the kernel is working correctly you can type "df -h" into a terminal and it should show all of the current partitions including one called cache on mtdblock4.
Should now work with polaris, kaiser and vogue. There is a VOGUEIMG.NBH included for those who have trouble with ATools. It is completely untested on Vogue so sorry if it doesn't work.
EDIT
I have re-uploaded a modified kernel because there was some issues with data with the last one. If you have already applied the last one flash the new one and apply the module update again
testing will leave feedback
Ok, using your Scoot CyanogenMod 6.1.2 (RLS6) [16.03.2011], overclock via atools+roguetools to 529, gpu oc, battery units tweaked to suit my battery.
No SIM card yet, don't have one to put there, sync with google to get contacts.
Installed GoLauncher. Enabled JIT and Keep Launcher in memory.
Once in a while, Android is killing all apps, launcher included... strange...
Nevermind that, it was Cyanogen and the LongPress BackKey. LongPress on our Kaisers during lag can become short press
First couple of days of heavy use I've had in ages with no corruption. First day of my new job today so it's been on Google Navigation for 3 hours and used extensively during the day. Had to do a hard reset at one point and everything was fine. had a few soft reset's too. Don't get me wrong, i'd be surprised if this is a fix, but it's been a while since I've used my Kaiser this much and not had to wipe my data partition!
Update: CM is a bit slow, trying:
VaniljEclair RLS11 - A fast & stable CM 5.0.8 for Vogue/Kaiser/Polaris [2010-08-19]
And everyting seems good. I've pushed it a little, several normal reboots.
Had to reflash system (didn't reflash data). No errors, FC's, whatevar. Everything works, yet didn't try to make/receive calls. Its 1am here...
Now this particular version, has a god damn bug that is get the best of me. The Power Button and the red button won't make the phone go to standby. If i long press, i get the normal reboot/power off/airplane etc menu, on both keys. But single press is ignore... Any idea ?
scooter1556 , i have a question. It is not the purpose of the thread or kernel, but how difficult would it be to activate scalling of the cpu ?
Right now, the two 3 biggest issues with Android on the Kaiser are, IMHO, Data corruption, Battery life, performance.
Please share your thoughts
Using this kernel with Valentine 1.02. GPU overclocked and CPU @ 520mhz. Super fast and for the first time... no data corruption! Verry happy so far!
daedric said:
scooter1556 , i have a question. It is not the purpose of the thread or kernel, but how difficult would it be to activate scalling of the cpu ?
Right now, the two 3 biggest issues with Android on the Kaiser are, IMHO, Data corruption, Battery life, performance.
Please share your thoughts
Click to expand...
Click to collapse
Well in theory not too hard, there is a feature built into the linux kernel called cpufreq which is currently disabled in our kernel because our cpu and chipset code doesn't support it but krazy-killa has been working on getting it working in his kernels with a little success as far as i'm aware but i'm not that sure. I haven't personally looked into it but if I get some spare time I definitely will. I've started a new job now which i'm sure will keep me quite busy but I still intend to so some work on the Kaiser when I get time
Best of luck on your new job! I wish i had enough knowledge to continue the kernel development, but i'm quite a ignorant
Still, thank you, till now it's one of the best kernels i've used.
Hi Scoot!
I love your kernel. No data corruption so far (two days). did a few stylus resets and stuff.. Magical =D. Now IMO we only need better sleeping and autofocus to call kaiser a fully android device! Please Keep up the awesome job, scoot! thanx, danke, sposeeba, bedankt, aciu, dzenkuja!
Nice job scooter, I've been using Scoot CyanogenMod 7.0.2 RLS2 [28/04/11] @ 500MHz, and it seems fine so far.
I tried Fat Free Froyo before CyanogenMod, which also worked great for the few hours it was running. I'm not sure which one to stick with though.
Keep up the good work. I wish I could help with the development, but I have yet to learn programming (been ten years now ).
This kernel looks pretty stable, but just a few quick questions:
1) What filesystem is used for the cache? If YAFFS, could it get corrupted? If Yes, what would happend then?
2) Is it really needed to have 20MB for the cache? ATM it seems to be used only by a like 1-2MB on the cache partition, would not it be enough to have for example 10MB for the cache leaving more space for apps?
3) What is the difference using apps in the system and data partitions? If I would run off the space on the data partition, can I put some apps in the system apps folder to save the space without wandering about how it works?
Thanx for your great work, finally it seems to be fully usable for me and if you and Krazy-Killa would be able to work together to manage the power consumption, then it would be amazing, because I am going to get new phone (running some new extensive business and I can not afford to be not reachable when something happends on Kaiser) and give this lovely piece of HW to my son... And I really do not want him to have a WM machine But Android needs to be stable for him to use Well... he will get it for his 10th birthday at 6.9. so we still have some time to tweak it a bit more
if this works then it would be magical, your CM builds were so fast and awesome and if there's no data loss then... <3
Made 2 mistakes when building kernel (selected tilt instead of normal and forgot to OC to 480MHz), and running kingshui's 12/15/2010 build 2.2.1. seems ROCK solid and fast! I applaud your use of ramzswap for the extra 20mb, bravo!
I'm just hoping I can go back and rebuild the kernel with the correct settings without messing up the system/data/cache partitioning.
PoXFreak said:
Made 2 mistakes when building kernel (selected tilt instead of normal and forgot to OC to 480MHz), and running kingshui's 12/15/2010 build 2.2.1. seems ROCK solid and fast! I applaud your use of ramzswap for the extra 20mb, bravo!
I'm just hoping I can go back and rebuild the kernel with the correct settings without messing up the system/data/cache partitioning.
Click to expand...
Click to collapse
They aren't mistakes, those are the settings I need for my Kaiser. I included a modified version of ATools for you all to modify the kernel for your devices as I stated in the first post
a.s.j said:
1) What filesystem is used for the cache? If YAFFS, could it get corrupted? If Yes, what would happend then?
2) Is it really needed to have 20MB for the cache? ATM it seems to be used only by a like 1-2MB on the cache partition, would not it be enough to have for example 10MB for the cache leaving more space for apps?
3) What is the difference using apps in the system and data partitions? If I would run off the space on the data partition, can I put some apps in the system apps folder to save the space without wandering about how it works?
Click to expand...
Click to collapse
1) The cache partition is still using Yaffs2 because it is the easiest filesystem to configure at the minute, but the idea of making it a separate partition like on a native android device is that it can be changed to another filesystem in the initrd relatively easily. If it gets corrupted it will do what it used to I guess, but I haven't had any corruption as of yet so it's difficult to say. Hopefully if the cache partition get's corrupted it shouldn't affect data so you should just be able to wipe and format dalvik-cache in the install menu.
2) I made it 20mb because most android devices have 30mb or bigger and this is mainly because market app downloaded get downloaded here before being installed, so if you installed google maps for instance it would use up 6mb, plus the standard 1-2mb normally being used, but if you only had a 10mb cache for instance, you would only be able to install apps less then 8mb which is a little limiting really. I don't really care much about installing lots of apps in data or on the phone at all for that matter so having 95mb for data is more then enough for me.
3) You can put apps in the system partition in /system/app. The system partition is read only so all data for the app still goes to the data partition. You can still save a little data partition space though if you need it. Most installs of android are 80-95mb which should leave you 5mb or so to play with. Some all language builds pretty much use up all of the partition though so you need to look and see what space you have to play with before putting apps on the system partition. If you overload it you will probably end up corrupting it and having to reinstall. But to answer your last question, the system partition works in the same way as the data partition apart fro the fact that it is read-only and data is read/write.
I've modified my Kernel to use the NAND mtdblock3 (/data for everyone else) as the cache partition, since my NAND is pretty much shot to begin with, and have system and data on SD Card. So far it's made downloading Market Apps twice as fast, loading webpages pretty fast, and basically anything else that the cache is used for.
scooter1556 said:
They aren't mistakes, those are the settings I need for my Kaiser. I included a modified version of ATools for you all to modify the kernel for your devices as I stated in the first post
Click to expand...
Click to collapse
Scoot:
I understand that the settings built into the .nbh are for your device. What I was meaning was I needed to modify the CPU clock up to 480MHz, and for some odd reason my device swaps the "@" and "!" buttons if I set it to "tilt", among other buttons being set wrong.
Also, I am used to having the left and right softkeys set as "vol up" and "vol down" respectively. Is this something I need to make an androidupdate for, or do I have to write a new NBH?
PoXFreak said:
Scoot:
I understand that the settings built into the .nbh are for your device. What I was meaning was I needed to modify the CPU clock up to 480MHz, and for some odd reason my device swaps the "@" and "!" buttons if I set it to "tilt", among other buttons being set wrong.
Also, I am used to having the left and right softkeys set as "vol up" and "vol down" respectively. Is this something I need to make an androidupdate for, or do I have to write a new NBH?
Click to expand...
Click to collapse
Can you not do all of this using the ATools i provided in the attachment in the first post? You should be able to load the nbh into ATools and then set the CPU speed, change the keyboard type and remap the buttons and then save it before flashing. Just don't change the partition sizes as this will mess up the cache partition I added until I make it more permanent.
So i am here with a new idea. A rescue.zip which can be used to rescue any android device which have a recovery like the famous cwm.
So here is it..
Some times we people screw up our android os like hell, and to reboot the device we usualy do a recovery flash of a new os, flash back our nandroid backup ( both on worst conditions) or even do permission fix, clean cache or dalvic cache( those in 'not that worse' conditions) . So thats are all the options we got. Rit?
Although flashing recovery backups, new roms can fix all, it will also eatup our apps, current setups, contacts, msgs, etc( in case we dont have backups) and will probably screw us. All we can do is say " WTF..WTF..WTF.."
SO here is my idea,
Find out the causes of what causes a reboot, non-boot, hang,fc etc.
And keep a zip that can be flashed through recovery, that has a solution for our problem. They may be including..
1) fix permission of system, data, and user data.
2) zipalign the apps
3) fix the default clock speed of processor
4) defragment memory
5) flash a new copy of su and busy box
6)wipe data or system or ext or cache or dalvic cache
7) flash a new copy of framework.res, system-ui.apk, settings.apk with default permissions( those files are kept in separate "custom" folder on the zip, so that end user can put their own files to that "custom" folder for flashing., the reason behind it is known to all, yap. Not all devices have them in common, every device have its own files)
These are all i got for now, pls post ur ideas and knowledge for any possible cure about any problem u faced/ cured. So that we can make it an ultimate rescue.zip that have a cure for 99% problems android os have. The rest 1% will go with a clean flash.( well we cant avoid that if we did something that bad).
So my plan is to use aroma installer( now on hard learning to find how it works). Throw in some scripts, files etc. Into the zip.
And since its not a device specific .zip file, i want to know how and why any problems are caused in any device( there are many common problems, but that is not what i ask for. I ask for device/os specific problems, and not for a problem that we can cure after booting, but for a problem that can make the device un-bootable) . So u people may help me to find those problems and cures for it. For my knowledge i have experience with wildfire and hd2.
Well i will keep this thread for a week or two, so that u can post ur knowledge, and info. after that i will release the file for u.
To the admin. Of the forum, pls keep this thread as announcement so that all can take a look.
HYPERDROID EXTREEM EDITION-THE NEW BENCHMARK ROM FOR HD2.
If you plan to do this available to any android device, the file size will be so big that it will become useless. Every phone has different apk, and not only that, but those apk are different in different version of os. For example, CM9 framework should not work on google release. Worst, older CM9 framework might not work on newer CM9 and newer framework might not work on older. Also, one of the cause of bootloop that i have been experiencing since i have my GNexus is data corruption of apps. The only way i had was to wipe data. I dont think there is a way to know if your app are corrupted with script. I also seen a lot of strange problem on SGS II like the kernel being erased. Well, in this case this package would be useless. So i guess that having this package would be awesome, but wont happen. My best advice is that you could create a universal guide on how to recover from bootloop/fc/hang with the minimum of impact on the phone. This is just my opinion tho.
Sent from my Galaxy Nexus using xda premium
You could add using flags in the updates filename, see some roms or themes for the lg optimus 2x for more information. It uses sed. For example, "update-wc-wd.zip" would wipe /data and /cache.
You could also merge these features in a customized clockwork mod recovery, the up side would be that you could automatically make a backup of the last flashed full ROM's systemui etc. this would also allow usage of the touch screen/volume keys to choose an repair option. You could even allow users to backup specific applications along with their data, and let users restore it later on after a fresh flash. I have some basic knowledge in modifying the recovery so I might help you out a little if you're interested.
chadouming said:
If you plan to do this available to any android device, the file size will be so big that it will become useless. Every phone has different apk, and not only that, but those apk are different in different version of os. For example, CM9 framework should not work on google release. Worst, older CM9 framework might not work on newer CM9 and newer framework might not work on older. Also, one of the cause of bootloop that i have been experiencing since i have my GNexus is data corruption of apps. The only way i had was to wipe data. I dont think there is a way to know if your app are corrupted with script. I also seen a lot of strange problem on SGS II like the kernel being erased. Well, in this case this package would be useless. So i guess that having this package would be awesome, but wont happen. My best advice is that you could create a universal guide on how to recover from bootloop/fc/hang with the minimum of impact on the phone. This is just my opinion tho.
Sent from my Galaxy Nexus using xda premium
Click to expand...
Click to collapse
I told it already, the "custom" folder is not filled. It will be kept empty. The user can put a file, which ofcourse is the file of the device he/she have or want to get repaired. All he has to do is copy and paste the file from the working zip( zip file of his currently installed rom, that encounter the problem) of his rom to the custom folder inside the rescue.zip.
And the things that are common will be scripts, but those too will contains device specific mound points, paths, etc. I think that will be common( ie, the working of script, once the mound is done). Am i right?
So all i have to figure out is mount points, paths etc.. i got a couple of them, about 15 or so. And pls help me to find the rest.
HYPERDROID EXTREEM EDITION-THE NEW BENCHMARK ROM FOR HD2.
a good idea to add is a file system chech like windows systems has. By installing a rom the installer should first check for bad sectors and mem blocks before installing the rom. After all blocks and sectors are scanned and the bad ones marked as "bad or corrupt" it should run something like defrag and place the bad blocks at the end of the file table. When all is done .. then the true rom install should start.
This will prevent heaps of problems since the curent installs just write over a bad block or sector creating the most weird problems. A fault checker/repair will take away a lot of strange forced closes and othere software/hardware failures.
Most phones wont last that long so that bad blocks or sectors can occure. But for the flashing junkies among us its a serious problem what can occure. I guess after 1000 or more installs bad sectors or blocks will occure and not all are being able to be repaired
Sent from my Galaxy Nexus using XDA App
Mikevhl said:
You could add using flags in the updates filename, see some roms or themes for the lg optimus 2x for more information. It uses sed. For example, "update-wc-wd.zip" would wipe /data and /cache.
You could also merge these features in a customized clockwork mod recovery, the up side would be that you could automatically make a backup of the last flashed full ROM's systemui etc. this would also allow usage of the touch screen/volume keys to choose an repair option. You could even allow users to backup specific applications along with their data, and let users restore it later on after a fresh flash. I have some basic knowledge in modifying the recovery so I might help you out a little if you're interested.
Click to expand...
Click to collapse
I am totaly newbee to lg. I have experience with htc, few samsung, etc. So can u pm me the details? Also is it usable to create recovery? I think a zip file with selectable options is more friendly. The thing is building a recovery wont make it universal( or atleast common for a couple of devices) and we will have to port them for each and every device. Thats the problem.
But any way i want ur help in building it. Can u pm me an example for mounding script in lg devices? And any thing that may become useful. Thank you.
HYPERDROID EXTREEM EDITION-THE NEW BENCHMARK ROM FOR HD2.
wilwilwel said:
a good idea to add is a file system chech like windows systems has. By installing a rom the installer should first check for bad sectors and mem blocks before installing the rom. After all blocks and sectors are scanned and the bad ones marked as "bad or corrupt" it should run something like defrag and place the bad blocks at the end of the file table. When all is done .. then the true rom install should start.
This will prevent heaps of problems since the curent installs just write over a bad block or sector creating the most weird problems. A fault checker/repair will take away a lot of strange forced closes and othere software/hardware failures.
Most phones wont last that long so that bad blocks or sectors can occure. But for the flashing junkies among us its a serious problem what can occure. I guess after 1000 or more installs bad sectors or blocks will occure and not all are being able to be repaired
Sent from my Galaxy Nexus using XDA App
Click to expand...
Click to collapse
Pls pm me the idea how to make the checking script. Or links that have info in this. Thank u in figuring out such a prob. I am unaware of that.
HYPERDROID EXTREEM EDITION-THE NEW BENCHMARK ROM FOR HD2.
showlyshah said:
I am totaly newbee to lg. I have experience with htc, few samsung, etc. So can u pm me the details? Also is it usable to create recovery? I think a zip file with selectable options is more friendly. The thing is building a recovery wont make it universal( or atleast common for a couple of devices) and we will have to port them for each and every device. Thats the problem.
But any way i want ur help in building it. Can u pm me an example for mounding script in lg devices? And any thing that may become useful. Thank you.
HYPERDROID EXTREEM EDITION-THE NEW BENCHMARK ROM FOR HD2.
Click to expand...
Click to collapse
I'll send this as a PM as well, but people might learn from this. I am not talking about any specific mount points for LG phones, I just pointed out that there are some roms which use sed to check the filename of its update.zip and do tasks according to that, you need to have one line in your updater script to run the script which detects what to do. That way a user of a Galaxy Nexus would rename it to update-maguro.zip and it would know to use mount points for the maguro, while if the exact same update.zip was to be named update-p990.zip, it would know to use the mount points for the LG optimus 2x. This way you could easily keep the zip up to date for any device, because they all use the same update.zip
About the recovery, you would need to build it for every phone once, but you could make one change to the recovery source and easily compile the recovery for all phones which are capable of running CWM. I believe this method to be more user friendly, as a recovery image has support for actually choosing what you want to do, instead of having to rename the file. A recovery image also has a better way of communicating with the user. Where a update.zip can only say "Hey, I had an error and I'm quitting now, I won't give you any details what the problem was because that's just how update.zips roll", a recovery image would be able to give more advanced outputs, like "An error occurred when trying to mount /data." And then give you the option to either try again, manually fix it by using a computer with adb, or quitting.
But that's just my personal opinion. The recovery would be way harder to make, but I was the original porter of CM6, CM7 and HTC Sense to the xperia mini pro and mini back in the days. I also made a custom recovery and roms for the HTC desire Z, maintain a CWM port for the HTC Chacha which I don't even own and have used the LG optimus 2x before. (currently a maguro owner) but I'm trying to say that I've been experimenting a lot with different phones and know what the possibilities of Android are. you could even make a live Android build, tailored for recovering your phone, which is ran by an update.zip! How cool is that? That would be VERY device specific though..
let me know what you think is the best way to do this. I was thinking of making a mobile time machine app for some time so it's good I saw this thread.
Is TNN causing your problems?
Ok I have been seeing a bunch of issues lately all around hear lately. I am starting to wonder if it could be linked to TNN? I am not trying to start a panic or bring attentioin to TNN. I think some of the issues could possibly be linked to some things I have done to our roms to enhance them. So if you have used or are using a TNN rom (Dirty Diablo, 4Kornerz, MIUI (HipKatz latests), Merge by Jlmancuso, or possibly a leak of Jelly MIUI some of our tweaks could be causing your issues.
If you properly wipe everything you will not have any issues from our tweaks. Boot, system, data, cache, sd-ext, .android_secure. Now in recovery toggle usb and from your computer erase any .XXXXX folders (this are hidden Linux folders used to store information on your sdcard), and the data folder.
How much of the problems are coming from our roms I am not sure. I have been busy trying to get my workstation all setup and have not been able to look into it real good yet. Again I am sorry if it has caused to many headaches.
So for the next few days if you do use our roms just make sure you have done a complete wipe of your phone and sdcard (sd-ext and temp files). This will keep any of our tweaks from causing issues with other roms.
In second post I will have my thoughts of the issue and fixes as I have them.
You have my greatest apologies if this has caused you any issues.
Peace,
Love, and
Happiness
Here is what I think. Mostly it is just a failure to wipe completely. No calling everyone dumb or anything like that it is just that Android stores files in many places. Some files are on the phone in the typically wiped places. Then you have the sd-ext partition, the built in a2sd by Android that store files in the .android_secure directory. There are also files kept in many other directories on your sdcard. Most of them are hidden Linux directories starting with a period (.data, .bookmark_thumb1, etc) that way the normal user can not accidently erase them and make applications crash and possible render the phone usless.
Now this would not be an issue if we did not keep changing our roms all the time. As we are getting newer users and different versions of Android all the time it can cause major issues. Here on the of evo forums we have froyo, gingerbread, ice cream sandwich, and jelly bean and there are many sub versions of each. So if you dont wipe everything completely these files can mix together and cause all kinds of problems.
Peace,
Love, and
Happiness
OK, this thread is going to be a work in progress, intended to serve as a reference for the work I've been doing on LVM partition remapping.
My work was done initially on a Find 7, but this should eventually be usable on many other devices (I have the Find 5 and N1 in mind for when I return from vacation). Also, this would not have been possible without the work Steven676 did years ago on the Nexus S, which has been used by all AOSP-derivative projects to support the Samsung Aries (Galaxy S) family for quite some time now.
The current state of things is that the patches are solid and work very well for the system side of things, but there is still a bit of work needed on the recovery side of things. This is due to TWRP having an architectural limitation I need to work on - Whether a device uses emulated storage or not is set at compile time, which is a problem if your design requires automatic detection of configuration at run time.
One of the key design goals here was to support both normal and LVM configurations automatically with a single build that detects which configuration is present on a device at run time.
A second key design goal was that the underlying partition table of the device is not touched in any way. Touching the partition table of a mobile device in the field is a fundamentally dangerous operation, as many partitions contain data that is device-unique or will render a device unbootable if altered. Recovery methods that involve DDing partition images to nonstandard partitions is asking for trouble due to typos... There's no protection against a user typoing the name of a critical partition.
Initially, I'm going to dump the contents of an email I wrote to someone giving them documentation on how to integrate LVM into their project. Over time I'll clean up and reorganize this post, including adding some more links. Also, since this email was written, I've added a LOT of comments to each patch explaining what is going on.
For additional documentation, especially a more user-oriented view of things (such as how to set this up if you want to use it with Omni nightlies) - see the Omni nightlies thread on XDA.
So here goes:
How it's implemented - the complete patch set is at:
https://gerrit.omnirom.org/#/q/topic:find7_lvm - Expect this to periodically change as work on this feature continues (Note: All patches required to support nightly builds of Omni have been merged. At this point, all remaining work that I expect is on polishing up TWRP.)
With the rest of this post, I'll talk about each individual patch and what it does.
https://gerrit.omnirom.org/#/c/9273/ - This is a patch against frameworks/base which adds an alternative to storage_list.xml called storage_list_lvm.xml - The frameworks will choose storage_list_lvm.xml instead of storage_list.xml if the property ro.lvm_storage is set to 1 - The device init scripts will set this property if they detect an LVM configuration.
https://gerrit.omnirom.org/#/c/9207/ - This is an Omni-specific patch. Omni builds for both the Find 7 and OnePlus One (also known as find7op) and both share a common device tree. The LVM patches do not apply to the find7op, so we move init.recovery.rc out of the msm8974-common tree - You likely don't have to worry about this unless you also have a -common tree for find7 and find7op
https://gerrit.omnirom.org/#/c/9276/ - Normal Android kernel ramdisks do not include busybox or any form of shell, making it impossible to run shell scripts without /system mounted. Since we need to run a shell script prior to mounting partitions, we need to add busybox to the ramdisk. This patch does that. For legal reasons you may wish to replace busybox with system/core/toolbox and system/core/sh - I have not tried doing so. If you choose to stay with busybox, you will have to provide the busybox source code in order to comply with the GPL.
https://gerrit.omnirom.org/#/c/9205/ - This adds the LVM binary and LVM configuration file to the ramdisks of both normal boot and recovery. This patch does not actually begin doing anything with the binaries, I separated it out from the other patches as a way to keep things organized so I could start working with the binaries when I began this project. The original source code and documentation for the binary is at https://github.com/steven676/android-lvm-mod
One change I made in lvm.conf that differs from the Samsung aries family (galaxysmtd, fascinatemtd, captivatemtd, etc.) is that I changed the filter line to only allow the userdata and sdcard partitions. This prevents LVM's vgscan from accidentally determining another partition is a physical volume, and also prevents users from accidentally running pvcreate on a critical partition.
https://gerrit.omnirom.org/#/c/9206/ - This is where all of the "heavy lifting" is done. I'm going to work on adding more comments to the init scripts and shell scripts to document them tonight and tomorrow, but I'll try to explain things here.
Android's init system is a bit limited in that it's very difficult to have conditional behavior defined in init.rc - which appears to be why Qualcomm loves to use shell scripts called from init. Similarly, much of the LVM magic happens in three shell scripts (which execute at three different phases within the boot sequence).
In the early-init phase, the two "wait" blocks ensure that the underlying block devices are ready before vgscan/vgchange are called. This will probably slow down booting by a few fractions of a second unfortunately.
vgscan will scan the volumes defined in lvm.conf (in this case, only the userdata and sdcard partitions) for LVM physical volumes. If LVM physical volumes are detected and form a proper volume group, vgscan will create appropriate device nodes. With the configuration I'm using, the device node will be /dev/lvpool/userdata - which consists of a single logical volume that merges the sdcard and physical userdata physical volumes (partitions). The configuration of lvm.conf prevents LVM commands (especially pvcreate) from altering partitions we don't want to alter. If someone accidentally tries to, for example, run pvcreate on the system partition, it will give an error indicating that the partition was not part of the filter.
vgchange will activate the physical volumes detected by vgscan
lvm_init.sh will check to see if /dev/lvpool/userdata exists, and copy fstab.qcom.lvm to fstab.qcom, init.fs.rc.lvm to init.fs.rc, and twrp.fstab.lvm to twrp.fstab if it does. If it does not, it selects fstab.qcom.std, etc.
In the "on init" section, the init script exports all environment variables from init.fs.rc, and creates all storage-related directories and symlinks needed for both configurations (except for when they conflict). lvm_symlinks.sh will create directories/symlinks that must be configuration-specific. Just like lvm_init.sh - it decides what to do based on whether /dev/lvpool/userdata exists
In the "on fs" section - we do an SELinux restorecon on /dev/mapper/lvpool-userdata (/dev/lvpool/userdata would probably work here too). If it doesn't exist, this will fail gracefully without causing any issues.
In "on early-boot" - lvm_setprop.sh uses /system/bin/setprop to set ro.lvm_storage to 0 or 1 depending on the detected configuration. The property service is not available until early-boot - so this cannot be in lvm_init.sh or lvm_symlinks.sh This propery is used by the frameworks/base patch above to determine which storage_list to choose.
At the end of the init.qcom.rc, the fuse daemon for emulated storage is added for all configurations. (I could not figure out a good way to make this conditional based on whether LVM was present or not). In a non-LVM configuration, it runs but is harmless - it maps /data/media (which is empty) to /mnt/shell/emulated (which nothing is looking at due to the environment variables and symlinks set in the "on init" section )
You will probably notice that Omni's standard storage configuration is fairly different from ColorOS - this is due to the way KitKat storage works, but it allowed us to get away without using Oppo's ext4 permissions hacks in our kernel (by remapping permissions instead, in a manner similar to how the emulated storage system works) The way we handle our /sdcard partition does interoperate without issues with the ColorOS approach.
https://gerrit.omnirom.org/#/c/9279/ is a patch specifically for TWRP. TWRP currently determines whether to use emulated storage (/sdcard on /data/media) at build time instead of at run time. Until I have time to fix this, the patch here operates as a workaround. It is similar to the behavior of the fuse sdcard daemon in the previous patch - it maps /data/media to /sdcard whether the configuration is actually emulated storage or not. If the device is not using emulated storage (LVM), mapping of /data/media to /sdcard is still mostly harmless. However it does result in undesirable changes to TWRP's user interface. DO NOT USE THIS APPROACH IN PRODUCTION RELEASES. It's a horrible hack. You'll need to figure out how to properly do /data/media handling depending on whether LVM is present or not based on how your own recovery architecture works.
https://gerrit.omnirom.org/#/c/9281/ adds "raw" sdcard and userdata partition entries to the partition table for the LVM configuration. This allows a user to return their device to a standard configuration by formatting the underlying sdcard and userdata partitions directly, instead of the removelvm ZIP at the beginning of this email. - To be abandoned, this patch was squashed into 9206
FAQ
Q: Coldbird already had repartitioning support. Why did you create this different approach?
A: Even before he started work, I strongly recommended that he not touch the partition table of the device. It's a really bad idea and is fundamentally dangerous. It's pure luck that someone hasn't hardbricked yet. (A number of people have come close.) If you read through his thread and the ColorOS 2.0.2 thread, you'll see that the repartitioning approach fails frequently, and in multiple ways. (Missing partition contents, partition table ending early, etc. The latter is really scary, one person had the process fail at mmcblk0p19 - what if someone else's partition table write operation aborted even earlier?.) Also, nearly everyone that has implemented support for that approach has needed a separate build to support it. (Oppo is the first to manage autodetection.) I also provided him all of the reference information from Steven676's work.
LVM is far safer. The underlying partition table is not touched in any way. Instead, LVM remaps sectors on the fly so that two partitions that are not adjacent to each other on the physical storage appear as a single contiguous partition to the filesystem drivers. Linux has supported LVM for on the order of a decade, if not more. I've been using LVM on my file server since 2006. (Yes, the system is 8 years old and still working other than needing a new power supply after a thunderstorm. Nothing to do with LVM. ) In addition, the lvm.conf configuration used here provides protection against accidental typos causing damage. Undoing the changes is as simple as doing a wipe of /data and /sdcard from any standard recovery and can be done in seconds, not of running a special batch file that runs a bunch of fastboot commands and takes 4-5 minutes. Similarly, the LVM setup process currently described in the Omni thread involves flashing a single ZIP from recovery that takes only 10-15 seconds, and most of that process is flashing an LVM-aware recovery. (The only limitation currently is that the ZIP must be on external storage - USB OTG or MicroSD)
To put it simply, it Just Works. No need to back up a pile of partitions other than /data and /sdcard because those partitions are never touched or altered.
Q: I have a device with a ridiculously oversized /system partition, can I get some of that back for /data?
A: Yes, you can. Add the physical /system partition to the lvm.conf filters and add it to the lvpool when creating it, then create a smaller /system LV out of this big pool. (see updater.sh in device/samsung/aries-common/ of any AOSP-derivative for hints here.) Be careful though - leave enough spare space for growth (new Android versions, etc.) While it should be possible to use some of the LVM tools along with ext4 resize tools to reorganize the LVs without wiping, this is very difficult and you'll probably have to make users wipe /data if you want to alter /system.
*reserved 2*
Nice work, I hope all the patches can be widely used on some other devices and other roms.
systop said:
Nice work, I hope all the patches can be widely used on some other devices and other roms.
Click to expand...
Click to collapse
Yup. I know Andre from PA was working on it last week but I haven't heard from him lately.
My priority when I return from vacation will be fixing up the TWRP side of things. It's working for now, but the user interface on non-LVM configs is a little funky thanks to RECOVERY_SDCARD_ON_DATA being compile time. This has never been a problem before since a single TWRP binary never had to support two different configurations before. I plan on either doing a property-based approach or fstab-based like CWM. (It should be possible for someone to make a CWM build that automatically detects configuration without any modifications to CWM, based on reading the code - but I haven't tried it myself.)
Once TWRP is in better shape, I plan on doing the Find 5 and N1. These will have the challenge of not having a MicroSD slot, so I may have to change TWRP so that it use /tmp instead of /sdcard when doing "adb sideload", or at least gives the user that option.
Good stuff :good: I don't really need it as of yet, but when my new device is provided (warranty) I will surely give this a try.
I hope ayysir will merge the LVM support very soon ^^
Find 7u PA 4.6 beta 1
Awesome work mate. I have avoided other methods because I'm always the guy that will have a device fail at very bad timing; like during boatloader or SBL stage.
I'm really glad you have continued to work on this. I have hit thanks a few times but would also like to thank you here
tork987 said:
I hope ayysir will merge the LVM support very soon ^^
Find 7u PA 4.6 beta 1
Click to expand...
Click to collapse
He had issues with merging support, hopefully now that I've added more documentation he can try again.
how are the *.std files created?
atm this is tough for me to port from omni to cm base which AOSPA Oppo trees
ayysir said:
how are the *.std files created?
atm this is tough for me to port from omni to cm base which AOSPA Oppo trees
Click to expand...
Click to collapse
the std files are also part of the device tree
https://github.com/omnirom/android_device_oppo_find7/tree/android-4.4/configs
ayysir said:
how are the *.std files created?
atm this is tough for me to port from omni to cm base which AOSPA Oppo trees
Click to expand...
Click to collapse
For the fstabs - they are simply moves/renames of the fstab files and other storage-related items from the standard Oppo configuration (they should appear as renames/moves in the Gerrit commit...)
For the init.fs.rc file - all of the "export <blah>_STORAGE" lines from init.qcom.rc/init.find7.rc are cut out of the RC file and put into .std
Obviously, the .lvm versions of the files are the ones where the fstab has been altered to support a single data partition with emulated storage.
Amazing work and amazing posts. Thanks a lot for your sharing. ?
I've got a question related to your configuration (/data and /sdcard merged) : are the LV hot-resizables?
Wendigogo said:
Amazing work and amazing posts. Thanks a lot for your sharing. ?
I've got a question related to your configuration (/data and /sdcard merged) : are the LV hot-resizables?
Click to expand...
Click to collapse
In theory, you could probably use some of the ext4 resizing tools to do something like this, but I haven't looked into it as there isn't much point in the current config (since the LVM userdata volume is allocated to use all space on the volume group).
Something like that might be more useful if someone ever uses LVM to regain some of the wasted /system partition space on certain excessively bloated devices (like some GS4 units).
Entropy512 said:
In theory, you could probably use some of the ext4 resizing tools to do something like this, but I haven't looked into it as there isn't much point in the current config (since the LVM userdata volume is allocated to use all space on the volume group).
Something like that might be more useful if someone ever uses LVM to regain some of the wasted /system partition space on certain excessively bloated devices (like some GS4 units).
Click to expand...
Click to collapse
Thanks for your answer.
Seems I misunderstood the way it's implemented here. All space is allocated to /data? So there's no more internal sdcard right?
But in that case an external sdcard is mandatory. How is it managed when there's no sdcard?
Enjoy!
Wendigogo said:
Thanks for your answer.
Seems I misunderstood the way it's implemented here. All space is allocated to /data? So there's no more internal sdcard right?
But in that case an external sdcard is mandatory. How is it managed when there's no sdcard?
Enjoy!
Click to expand...
Click to collapse
Android has supported emulated storage (where /data/media is mapped to /sdcard with a special FUSE daemon that makes /sdcard have DOS-like permissions despite an underlying ext4 partition) since ICS. It's pretty much the standard in all new devices - the Find 7 is to my knowledge the only device launched in 2014 not to use emulated storage. Most devices in 2013 also did - Oppos were again the rare exception.
As I understand it - for some reason Chinese users prefer the legacy pre-ICS partitioning scheme. My guess is due to UMS vs. MTP - MTP is required for access to emulated storage, UMS can't be used, but a lot of older desktop OSes have issues with MTP. So Oppo finds themselves in conflict between their home market (China) and expanding in the West. That said, the Find 7 was kind of a screwup in achieving this goal, since the internal sdcard partition was ext4 which meant UMS was a no-go for it.
Entropy512 said:
Android has supported emulated storage (where /data/media is mapped to /sdcard with a special FUSE daemon that makes /sdcard have DOS-like permissions despite an underlying ext4 partition) since ICS. It's pretty much the standard in all new devices - the Find 7 is to my knowledge the only device launched in 2014 not to use emulated storage. Most devices in 2013 also did - Oppos were again the rare exception.
As I understand it - for some reason Chinese users prefer the legacy pre-ICS partitioning scheme. My guess is due to UMS vs. MTP - MTP is required for access to emulated storage, UMS can't be used, but a lot of older desktop OSes have issues with MTP. So Oppo finds themselves in conflict between their home market (China) and expanding in the West. That said, the Find 7 was kind of a screwup in achieving this goal, since the internal sdcard partition was ext4 which meant UMS was a no-go for it.
Click to expand...
Click to collapse
I've got it now. Thanks for your explanations
I saw that Oppo phones didn't follow Android guidelines (yet?) by not using the emulated_storage mounting method but I didn't know why.
And your right, mtp doesn't work in Windows XP (or is hard to make working) and there's a lot of Asian people still using it. Obvious once you said it...
And that's also why only external sdcard is accessible in UMS mode in recovery.
Thanks again for your enlightenment. ?
Reading some of the comments on G+ it looks like Oppo might be using this solution for their KitKat release. I would be so pleased if they did.
Sent from my X9076 using Tapatalk
kishd said:
Reading some of the comments on G+ it looks like Oppo might be using this solution for their KitKat release. I would be so pleased if they did.
Sent from my X9076 using Tapatalk
Click to expand...
Click to collapse
You could be pleased...
Wendigogo said:
You could be pleased...
Click to expand...
Click to collapse
Had some problems with camera focus on earlier versions of omnirom for find 7. Now those have been addressed. I installed Omni and am on the nightlies with lvm. My find 7 and find 7a will not see another rom again.