Related
WARNING:
Use this on your own risk! I will not be held responsible for any failure that might be caused by using this.
"How should I know if it works? That's what beta testers are for. I only coded it."
- Linus Torvalds
What:
Yet another attempt at breaking free from RFS on the SGS phone (ie, lagfix), for Eclair, Froyo, and hopefully Gingerbread...
Which:
Supported devices are I9000 ATM. Theoretically it should work on all SGS variants, but since I only have I9000 to test on, I can't confirm this. I'd appreciate if other model owners will give it a try and report back.
Why:
KISS. I don't want to compile my own kernel or use a specific kernel (see voodoo) for my lagfix. I'd like to have a lagfix for ANY kernel out there.
z4mod is the 1st attempt in achieving that.
Another issue is standards. We don't need multiple lagfixes, all doing the same thing. One method, easy to customize and implement - thats the way to go.
How:
1. Use the online z4mod patcher to patch your kernel.
2. Download and apply one of the following update.zip files to convert your /data partition to the filesystem of your choice.
z4mod update.zip variants:
z4mod.ext2.update.zip - convert to ext2 - works on ALL kernels.
z4mod.ext3.update.zip - convert to ext3 - works only on modified kernel which supports ext3.
z4mod.ext4.update.zip - convert to ext4 - works only on modified kernel which supports ext4.
z4mod.auto.update.zip - automatically find which fs are supported by the kernel and convert to best suited - works on ALL kernels.
z4mod.rfs.update.zip - convert back to RFS - works on ALL kernels..
These update.zip files can be applied only to kernels which has been patched with z4build - a shell script which patches zImage to allow the init scripts additional non-RFS mount to each RFS partition, or with our online z4mod patcher. If the ROM you wish to install wasn't originally patched with z4build already, you can run z4build yourself on the zImage from the ROM package and put the patched zImage back to the ROM package for flashing, or use the online z4mod patcher to do it for you. It is that easy.
[NOTE:] Kernels (zImage) patched by z4build supports both mounts - RFS and non-RFS. So it doesn't change anything for users who doesn't apply z4mod.
You can also patch your z4mod scripts to enhance it's functionality, add more filesystems support, etc.
Although z4mod allows converting all the partitions from RFS to ext2/3/4, our update.zip files only convert /data - since this is ment for the 'masses'. If you feel brave enough, feel free to experiment with converting more partitions. All the sources are well documented.
Who:
z4ziggy, aka Elia Yehuda.
___
Original post [for reference]:
Although this is a half-baked project and far from complete, im introducing it to the xda community so i might get more ideas, and fix/implement more functionality, as the project advances.
z4mod is the 1st step in breaking free from RFS death grip - or at least tries to achieve this. The project adds ext3 support to a given zImage's init scripts, and creates update.zip for converting all RFS partitions and flash the patched zImage.
...thats the idea anyway.
ext4 support is planned for future official release. for testings purposes ext3 is sufficient.
sources and information can be found on my blog here: http://z4ziggy.wordpress.com
And of course, I'm not the only one interested in your measurements and conclusions
This looks really awesome, I'm loving it!
As far as the kernel patch goes, though:
Does the same kernel patch work for both 2.1 and 2.2 kernels? They seem pretty different.
Instead of requiring a linux box to patch the kernel, do you see any problems with doing the actual patching on the stock device itself? I didn't see anything in the script that should make this too difficult.
At any rate, this looks like The Solution.
love the work all 3 of you are doing with the file system on the i9000!
awesome job.
the patch is for the initscripts only - doesn't mind which kernel version. im doing my testings on Froyo 2.2 but patching should work the same on 2.1 scripts.
I hope to have some time later tonight to update the updater-script to follow all RFS partitions, and test it. ATM it works with my /cache partition only since its easy for testings (or "crashes" should i say? ).
i will update soon.
[edit]
regarding the linux part - no can do. We patch the initramfs scripts (which resides in the zImage), and i don't know of an easy way todo so from recovery mode. The easiest way is to use linux (even in a WM - you don't even need a full blown desktop, just CLI and compile tools) and patch the zImage, but i saw some Windows tools who might be able to do the same - I don't really know since i hardly use windows.
Thanks for your efforts! Keeping a close watch on this thread
z4ziggy said:
the patch is for the initscripts only - doesn't mind which kernel version. im doing my testings on Froyo 2.2 but patching should work the same on 2.1 scripts.
I hope to have some time later tonight to update the updater-script to follow all RFS partitions, and test it. ATM it works with my /cache partition only since its easy for testings (or "crashes" should i say? ).
i will update soon.
[edit]
regarding the linux part - no can do. We patch the initramfs scripts (which resides in the zImage), and i don't know of an easy way todo so from recovery mode. The easiest way is to use linux (even in a WM - you don't even need a full blown desktop, just CLI and compile tools) and patch the zImage, but i saw some Windows tools who might be able to do the same - I don't really know since i hardly use windows.
Click to expand...
Click to collapse
That's what rooting your phone is for! Just root the device, copy over all of the required binaries (arm statically linked if required, but busybox should handle 99% of what you're using in the script) and the kernel itself can be easily exacted using dd, of course.
Whole thing can be wrapped in a UI using java without too much hassle as well.
Most users aren't gonna be up to installing linux in a VM to run the script!
just replacing those scripts on your system won't do. you need to patch the initramfs and flash zImage back.
repackaging zImage with a new initramfs is not that simple. I'm using kernel_repacker todo so.
im sure there must be a better way than kernel_repacker, but until then, this is what we use
and about the Linux usage - the user would probably never use z4build script. only the update.zip. Modders, ROM creators, etc, should use z4build (or similar method to add the appropriate "mount ext3..." command to the "mount rfs", as i showed in my blog) to allow their kernel to support ext3. Simply apply z4build on any rom's zImage, and it's users can now apply the update.zip to have this rom' work with ext3.
I hope this explanation is suffice.
z4ziggy said:
just replacing those scripts on your system won't do. you need to patch the initramfs and flash zImage back.
repackaging zImage with a new initramfs is not that simple. I'm using kernel_repacker todo so.
im sure there must be a better way than kernel_repacker, but until then, this is what we use
and about the Linux usage - the user would probably never use z4build script. only the update.zip. Modders, ROM creators, etc, should use z4build (or similar method to add the appropriate "mount ext3..." command to the "mount rfs", as i showed in my blog) to allow their kernel to support ext3. Simply apply z4build on any rom's zImage, and it's users can now apply the update.zip to have this rom' work with ext3.
I hope this explanation is suffice.
Click to expand...
Click to collapse
Ah... kernel_repacker uses GCC. That would be pretty hard to port over to the SGS's stock kernel I guess.
Guess you're right then - would have to be run externally to the device. Bit of a shame since it adds to the complexity of the whole thing, still.
This does make it nice and quick to patch up any kernel at all though, which should make it much easier to keep the kernel patching up with any leaked kernels that come out!
looks like a very promising project! i liked voodoo but sadly we can not use it for 2.2 until we get the sources and it only converts the /data partition...
thanks for your work and i will be sure to check back here & on your blog soon
z4ziggy said:
Although this is a half-baked project and far from complete, im introducing it to the xda community so i might get more ideas, and fix/implement more functionality, as the project advances.
z4mod is the 1st step in breaking free from RFS death grip - or at least tries to achieve this. The project adds ext3 support to a given zImage's init scripts, and creates update.zip for converting all RFS partitions and flash the patched zImage.
...thats the idea anyway.
ext4 support is planned for future official release. for testings purposes ext3 is sufficient.
sources and information can be found on my blog here: http://z4ziggy.wordpress.com
Click to expand...
Click to collapse
Did you manage to compile ext3 or ext4 support to Froyo kernels? I only found ext2 support in them. (At least in JPK)
Another thing I see that you mount the rfs file on the device, but for me it only got mounted as vfat, so the symlinks in /bin in factoryfs.rfs were gone. (I solved this issue by adding commands to the initramfs that restore these symlinks)
ext3 is built into 2.2 kernel, i assumed the same for 2.1.
ext4 module will have to be rebuilt for each kernel, unless we build a standard kernel with ext4 support built-in, but all this will have to wait until samsung releases the sources.
[EDIT:]
Don't use the method which mounts the rfs files on the device. It uses ext2 and its obsolete already. I've only used the rfs mount on the device since it is the only way to have RFS readable as RFS and not as FAT. if your device mounted the RFS images as FAT, either:
1. you didnt use busybox mount
2. you might need to supply "-t rfs" to your mount command
z4ziggy said:
ext3 is built into 2.2 kernel, i assumed the same for 2.1.
ext4 module will have to be rebuilt for each kernel, unless we build a standard kernel with ext4 support built-in, but all this will have to wait until samsung releases the sources.
[EDIT:]
Don't use the method which mounts the rfs files on the device. It uses ext2 and its obsolete already. I've only used the rfs mount on the device since it is the only way to have RFS readable as RFS and not as FAT. if your device mounted the RFS images as FAT, either:
1. you didnt use busybox mount
2. you might need to supply "-t rfs" to your mount command
Click to expand...
Click to collapse
I know ext4 is missing, but I couldn't even get ext3 to work on JPK. (ext2 works)
for the mounts:
1. I did use busybox mount (using toolbox mount I couldn't even mount it in vfat mode)
2. I did supply -t rfs, but it told me invalid parameter
Some additions:
When I try to mount factoryfs.rfs in rfs mode it won't work (invalid argument). in vfat mode it works, but no symlinks and other rfs stuff. I can also mount it in j4fs mode, where I get some files not available in vfat mode (like param.blk or charging.jpg), the strange thing is these files are actually the same as in /mnt/.lfs
RyanZA said:
Most users aren't gonna be up to installing linux in a VM to run the script!
Click to expand...
Click to collapse
There's an easier workaround for that.
Provide instructions to:
- Copy required files to USB stick
- Download Ubuntu Live CD
- Boot from CD
- Run scripts
If you wanted to get fancy it could be prepackaged into an existing bootable CD distro with the scripts included (checking for updates when they're launched).
From what I've read, Desire owners used bootable CDs to root their phones, so it's obviously a viable option for the standard user.
Alright, that's the extent of my expertise. I'll leave you guys to sort out the real issues now!
I'm also very interested in this so subing myself to this thread.
I quite like the fact you Windows users have to use a Linux install to do it, I get a bit fed up having to find a Windows box somewhere to use Odin and Kies all the time.
RyanZA said:
Most users aren't gonna be up to installing linux in a VM to run the script!
Click to expand...
Click to collapse
Why? Surely the people wanting to do this kind of thing will be the more technical users anyway and us Linux users have to do the more difficult task of installing Windows in a VM to use the normal tools.
my ct-ng seems to be broken, my binaries doesn't execute on my SGS for some reason, so this will have to wait for tomorrow so I can investigate this further.
I need to compile the mkfs.ext3 staticly (which seems to compile fine using several different toolchains, but none works) so i can execute it from z4mod/updater-script.
dirk1978 said:
Why? Surely the people wanting to do this kind of thing will be the more technical users anyway and us Linux users have to do the more difficult task of installing Windows in a VM to use the normal tools.
Click to expand...
Click to collapse
Not true at all!
z4ziggy said:
my ct-ng seems to be broken, my binaries doesn't execute on my SGS for some reason, so this will have to wait for tomorrow so I can investigate this further.
I need to compile the mkfs.ext3 staticly (which seems to compile fine using several different toolchains, but none works) so i can execute it from z4mod/updater-script.
Click to expand...
Click to collapse
I've had so much trouble with statically compiling stuff...
We really need some kind of complete guide on all the ins and outs of statically compiling these system utilities. For a lot of them it doesnt seem to be all that straight forward...
i've a question
i've tried to mount the partitions as ext2/3 before by replacing the rfs in init.rc and recomplied the kernel and pack the zimage, but the system wont boot. it seems it wont accept any other fs other than rfs, rootfs, etc. my init.rc looks like this:
mount ext2 /dev/block/stl9 /system
so how did u simply mount them as ext3 by just changing the init.rc and recompile the kernel?
EDIT: sorry, i just tried it again and it's ok now, but not with the "||" (or) in the middle
Development continues here: http://forum.xda-developers.com/showthread.php?t=881837
Universal lagfix kernel project.
This project is aimed at multiple audiences.
If you think you're an end user read on.
For lagfix and kernel developers check the second post.
Download, changelog, bugs and other links can be found at the third post.
Credits
ChainFire for the original cf-root initramfs
hardcore for the speedup hacks
nikademus for the compiled jfs utils
neldar for BackLightNotification
newmail for some kernel modules
RyanZA for OCLF and for being helpful
supercurio for Voodoo
ykk_five for being helpful (and for the memory tweaks)
z4ziggy for z4mod
and for everyone else I might have missed. (like myself - I'm an egoist you know... and for all the people that helped the guys I credited above. And for the people who helped the guys that helped the guys credited above. And for XDA of course.)
If you only care about points
OhmygodIcanget2kpointsatquadranttoobadI'vealreadysoldSGSbecauseitonlyhas300MBraminsteadof512asadvertisedandboughtadesirebecauseilikehowitcrapsatmultitouch.
For the rest of the people
What is this about?
This project has multiple aims:
A JPM based kernel with ext4, jfs, tun and BLN support
An init script which supports multiple lagfix schemes
Init.d support (without early script support)
A modified ClockWorkMod that
Has better support for some features of the SGS-I9000
Includes support for rooting the device
Includes support for switching the lagfix schemes
Has some tweaking options
Various additions that might help the other lagfix/kernel developers
How does this work?
Simply flash the kernel to any JPM/JP6 based, unlagfixed ROM. By default it won't apply any lagfix scheme, so you might use it as a simple ROM, with built in ClockworkMod + rooting support. If you have a complete ROM package (with dbdata.rfs and an older csc file (it won't work with some of the newer csc files) ) you can also flash it with the other parts, just replace the zImage in the PDA.tar with the one supplied. The ROM was tested with the original JP6 and JPM ROMs, and with my Multinational 3.0 ROM.
How can I use the extra features?
Reboot your phone to recovery mode. Under "Advanced ULK", there are some new options:
Reboot to download: switches to download mode
Run 2e recovery: Switches to 2e recovery
Install superuser: installs busybox and superuser to the phone
Lagfix options: switch between various lagfix schemes
Tweak options: switch between various startup tweaks
BLN options: Turns BackLightNotification on or off
Reset permissions: Resets permissions to the same state as init does
What are the lagfix schemes?
They are ways on how to format and bind the various filesystems. You can chose how you want to format the partitions (data,dbdata and cache), whether you want to use an extra loop device or not, and whether you want to bind some directories from /data to /dbdata.
There are also 8 included pre-defined schemes:
OCLF: /data stays at rfs, but has an ext2 loop device on top of it
Voodoo: /data is ext4, all others are kept at rfs
JFS Voodoo: /data is jfs, all others are still rfs
No-RFS Standard: /data is rfs+ext2, all other partitions are ext4
No-RFS Advanced: all partitions are ext4
No-RFS Advanced JFS: all partitions are jfs
No-RFS Overkill: turns on all options: everything is ext4+ext2 and /data/data is bound to /dbdata
No-RFS Overkill JFS: turns on all options: everything is jfs+ext2 and /data/data is bound to /dbdata
(If using the overkill scheme using the bind option is dangerous if using too much apps, so you should disable it in the advanced menu.)
Although I used the names OCLF and Voodoo, the ROM is not compatible with any of them, it's just a naming convention I used. Hope RyanZA and supercurio won't mind
Which one should I use?
You decide. OCLF or Voodoo might seem simple but they are actually working great (but for those you can actually use the originals, they are probably much more stable), while NO-RFS Overkill might seem cool, it is clearly an overkill.
What happens after I chose a lagfix scheme?
If the chosen one is different from the active one the kernel will reformat your device the next start. Before reformatting it will create a nandroid backup, reformats the device then restores the data from your backup.
Is this ROM safe
alpha 0.2 means no. Switching between lagfix schemes might break, and data loss (mostly database corruption) can happen if you're using a lot of loop devices. And this kernel isn't thorougly tested anyway.
How to switch to another kernel/ROM
After disabling the lagfix everything should be fine (except for the possible data corruption). You can always flash another complete ROM package over this using Odin, you'll lose all your data however.
I've found a bug
And you'll find more of them. I'll try to collect them and answers to them at post #3
Internal workings.
In this post I'll try to describe how this works, and how it might be useful for other people.
The Kernel uses both the pre-init trick and the bootlogos trick to load up, and do it's stuff. In the pre-init phase it does the following:
initializes the devices
checks the actual state of the partitions, and decides which lagfix scheme is active
checks the config (resides in /system/etc/lagfix.conf), and if they differ runs the converter application
if they don't differ then it starts to mount the devices, and let's init run.
if during the mounting a fatal error occurs, it prints the log, waits for a couple of seconds and reboots the device.
It does the filesystem checks by mounting and checking whether mounting succeded or not. To defend against samsung's init's reformatting, it simply removes the fat.format command.
In the post-init phase it removes some files (mainly the symlinks to busybox, so they won't interfere with the busybox installed on the system). It also reinstalls fat.format, because it might be needed later.
The lagfixer is built inside clockworkmod, which has the following advantages:
Can use nandroid backup
Can use the framebuffer to show the progress (this makes it much easier to debug problems)
Besides the lagfixer there is also a "graphsh" application, which will run a specified command and prints its output to the framebuffer (the reasons for this are the same: it's much easier to debug this way than to check a log file copied to the sdcard). If a fatal error occurs this command is run so the end-user can see some logs. (all of this is still pre-init). The only problem with using the framebuffer is that core android won't boot up properly if the framebuffer was initialized beforehand (it switches to QVGA with overlapped screens... it's actually quite fun to try using android that way). That's why I always reboot after using either 'graphsh' or 'lagfixer', except for recovery modes, because CWM is using the same framebuffer anyway.
The initramfs is "ro.debuggable=1" so if anything happens one can connect to the phone in adb root mode, to check for errors. Adb is enabled by default in both CWM and base android mode. I don't know the drawbacks of using debuggable=1.
CWM was modified in order to be able to handle the various lagfix schemes. If it tries to mount, unmount or format /data,/dbdata or /cache, it asks for the lagfixer to do the actual mount, unmount or format. Therefore if one saves or restores a backup it will always see and use the actual files, and not the loop device file.
There were some other minor modifications, like binding the home button too to the enter key, using /mnt/sdcard instead of /sdcard, and the ability to back up the efs partition too. (/efs won't be restored however).
There is also a small modification in how the updates work. If there is an update, the init will always run samsung's 2e recovery, if not it will use CWM (you can use CWM's menu to run an update using CWM if desired). This is to maximize compatibility with updates.
About the compiled files:
I had a hard time getting the binaries to work, even when compiling in static mode. All of the binaries are either compiled by me using bionic (AOSP), or used from other sources (but they seem to be using bionic too). The main advantage is that the recovery binary is actually only around 700k, but has a working busybox implementation, a working recovery and some other small stuff. The main problem is with the other binaries (mainly with e2fsprogs). If I could cut from the size the whole initramfs might fit into an official kernel, so it could be used by anyone.
Links
Da kernel 0.3 + BLN 2.2: http://forum.xda-developers.com/showpost.php?p=9157498&postcount=738
Da kernel 0.3: http://android.sztupy.hu/dl/Universal_lagfix_kernel_JPM_0.3.tar
Da kernel 0.2: http://android.sztupy.hu/dl/Universal_lagfix_kernel_JPM_0.2.tar
Da kernel 0.1: http://android.sztupy.hu/dl/Universal_lagfix_kernel_JPM_0.1.tar
Source of the modified CWM: http://github.com/sztupy/android_bootable_recovery
Source of vendor/samsung/galaxys: http://github.com/sztupy/android_vendor_samsung_galaxys
Kernel sources I'm using: http://github.com/sztupy/universal_lagfix_kernel
Bionic port of jfsutils: http://github.com/sztupy/android_external_jfsutils
The contents of the initramfs: http://github.com/sztupy/universal_lagfix_kernel_initramfs
Changelog
Version 0.3
- before the lagfix gets applied there are now multiple options to chose from:
-- backup+restore (the original way)
-- backup, but don't restore (this eventualy results in a factory reset)
-- no backup (this also results in a factory reset)
-- go to recovery mode (to debug)
-- erase config file (to get back to the old options)
- the screen glitch in the first screen is now gone (because the kernel got smaller)
- jfsutils are ported to bionic (takes less space)
- there is a new option that will do the same restore permissions run, as the init script does
- I also added a few fixes to the CWM fix_permissions script, although I don't tested them yet (so it might brick the phone)
- Added two more rooting option:
-- the old one simply copyes busybox, superuser and su to the phone and adds symlinks to busybox
-- the new ones will also remove some old toolbox commands, so if you use the shell a lot, they won't interfere. The first mode only removes the base utils, like ls,mkdir,rm or ln
-- the second mode removes every toolbox command that has an equivalent busybox command (like mount, lsmod, rmmod, etc.)
- Adeed an option to get back to 2e recovery from CWM
Remarks
It seems that formatting dbdata as rfs only works, if you use the 512 PIT file (more exacty it only works if you have used the 512 PIT file the last time you've ticked re-partition in odin). This is probably due to the fact thet the 512 and 803 pit files reserve different space for the dbdata and for the system partition (the 803 pit file reserves 40 units less for the dbdata than the 512 pit file)
Version 0.2
- efs is not backed up automatically during lagfixing (but it's still backed up during ordinary backups)
- added jfs support and some more lagfix schemes using jfs
- added backlightnotification support, with the option to turn it on and off
- added startup script tweaks support, with options to turn them on and off
- removed ext2 support for the base mounts (might be put back into later)
- bind now only binds /data/data (should use less space)
- some minor bugfixes relating
Remarks
- If Updating from 0.1 to 0.2: If binds were turned on you have to turn them off before updating. After the update has finished you can turn them back on if desired.
Known bugs
After the lagfix conversion is done the phone reboots and the lagfix conversion starts over again
This means that the conversion took place, but the result is not the same as was desired. You should reboot into recovery mode and remove the config file.
After rebooting I have a lot of FC's - After switching lagfix schemes I have a lot of FC's
This is probably because of a database corruption. Try loading an earllier backup.
Can't download anything from market
Some configurations might have this problem, I'm still trying to find out why this happens.
Can't download some apps, like angry birds from the market
Retry, it should work after a while. If it still fails remove the external sd.
Shutdown fails on some configurations
Until I find a way to hook onto the shutdown script this might happen on more complex configurations (like the NO-RFS overkill)
Can't get back to rfs on /dbdata
Reverting the lagfix on /dbdata only works if you used the 512 pit file the last time you checked re-partition in odin.
I for one am excited
Dan
Can't wait to see this.
I am waiting too ++++
Lol 100% teasing ^^
gingerbread?
shahadat said:
gingerbread?
Click to expand...
Click to collapse
would be a bit early
Fantastic
Testing now on a very stripped JP6...
very interresting i think i will give it a try hehehe
any feedback?
Would it be unsavory to say that I got a little motion in my pants after reading that first post?
Nah, I think we all did
Sounds great! And I'll wait for a few more version points on that alpha before I try it... but I'm tempted!
wonderful,i'll try it
Sztupy is this kernel and method usable with your multilingual rom v3.0?
Thanks
kurudisease said:
Sztupy is this kernel and method usable with your multilingual rom v3.0?
Thanks
Click to expand...
Click to collapse
check first post. Tested with JPM, JP6 and Multinational 3.0
Test drive
I just used it, I had no issue. Everything worked fine and there is no lag at all, still too soon to say it's faster than voodoo though(which I was very happy with). All data is still there and no problem! Im using JP6 with NO-RFS Extended. Quadrant score is ~1900 for anyone that want to know .
looks great! gonna try this
ist it possible to add in that also??
Startup script speed tweaks
http://forum.xda-developers.com/showthread.php?t=813309
I applied the no-rfs advanced fix and everything worked fine. No issues so far.
Thanks!
Team Win Recovery Project 2.2, or twrp2 for short, is a custom recovery built with ease of use and customization in mind. It’s a fully touch driven user interface – no more volume rocker or power buttons to mash. The GUI is also fully XML driven and completely theme-able. You can change just about every aspect of the look and feel.
ChangeLogs
CHANGELOG for 2.6.1.0:
Initial SELinux support (only a few devices, need testers so come by IRC if your device doesn't have it and needs it)
Initial support for f2fs file system formatting (Moto X)
Update SuperSU install for 4.3 ROMs
Fixed a permissions bug on files created during backup
Fixed a bug that caused TWRP to not wait for compressed backups to finish causing 0 byte files and md5sums to not match
Fixed decryption of encrypted data so that both TouchWiz and AOSP decryption are possible
Ignore lost+found folder during backup and size calculations
Various other minor bug fixes and tweaks
CHANGELOG for 2.6.0.0:
What's new in 2.6.0.0:
Special Note: If you are running a custom theme, some of the changes in 2.6.0.0 will likely not be visible with your custom theme.
Can encrypt a backup to prevent theft of private data from your backup files
Updated graphics / icon courtesy of shift
Updated exFAT to latest commits
Fixed a problem with Samsung TouchWiz decryption
Update SuperSU binary
Fixed saving of backup partitions list
Fixed saving of last used zip install folder
Fixed backup of datadata on devices that use a separate partition for datadata
Fixed some issues with the advanced wipe list (android_secure, can now wipe internal storage on data/media deivces and wipe data on the advanced list no longer formats the entire data partition)
Fixed some problems with partitioning a SD card
Various other bug fixes and tweaks
Notes about encrypted backups:
Why encrypt your backups? -- Most people store their backups on the device. Any app that has permission to access storage could potentially read your backup files and try to harvest your data. Encrypted backups also provide an added layer of security if you move your backups to other storage devices or to the cloud. The encryption that we're using is probably not strong enough for enterprise level security, but should be strong enough to make it significantly difficult to get to your data.
Encryption is using OpenAES which uses AES 128-bit cbc encryption. If you happen to use a longer password (over 16 characters) then the encryption strength improves to 192 or 256 bits. Do not forget your password. If you forget your password you will be unable to restore your backup. We don't encrypt the entire backup. Encryption is very CPU intensive and can be fairly slow even when we spread the workload over multiple cores even on the latest high-end devices. To ensure that encrypted backups don't take forever, we don't encrypt any other partitions besides /data and in /data we don't encrypt /data/app (or other app related directories where apks are stored) and we don't encrypt dalvik cache.
CHANGELOG for 2.5.0.0:
Special Note: If you are running a custom theme, you will likely need to remove that theme before updating to 2.5.0.0 as your custom theme will likely not be compatible with the new changes!
-Added scrollable partition lists for mount, backup, restore, wipe, and storage selection
-Add new SliderValue GUI element for selecting brightness and screen timeout (thanks to Tassadar)
-Re-work AOSP and TWRP code to improve license compatibility between GPL and Apache
-Added official theme for 1080x1920 portrait devices (HTC One m7, HTC DNA, HTC Butterfly, Oppo Find 5, Sony Xperia Z, etc)
-Fixed a problem with directory permissions on split archive backups (backups usually restored with no app data)
-Fixed a problem with md5 verification of backups
-Added a search function to libtar (thanks to kokotas)
-Improve handling of XML errors (fix permissions)
-Fixed handling of subpartitions
-Improvements to recovery.fstab flags
-Fixed a problem with decryption not being able to locate the decryption key in some situations
CHANGELOG for 2.4.1.0:
Fixed a problem with mkdosfs that formatted sdcards to 2GB
Fixed handoff between vfat and exFAT on devices where blkid didn't detect vfat (fixes some issues with mounting sdcards)
Fixed problems with changing working directory on MD5 creation/checking that may have prevented unmounting
Backups will now store a copy of the backup log after the backup is completed (only if backup is successful)
CHANGELOG for 2.4.0.0:
Using libtar instead of busybox's tar for better control over tar file creation and breaking the 2GB barrier that busybox imposes (thanks to bigbiff)
Support for exFAT formatted sdcards (also thanks to bigbiff)
Support for decrypting Samsung TouchWiz encrypted devices including internal and external storage (special thanks to a3955269 for figuring it out)
Improvements to OpenRecoveryScript including displaying a proper GUI while the script is running
Added wipe cache and dalvik after ADB Sideload
Replaced many system calls with their native C counterparts
Fixed bugs in file manager where it would display an empty list after moving or deleting a folder
Fixed AOSP recovery commands to run after decryption on encrypted devices
Improvements for building TWRP in CM10.1
Other minor bugfixes and improvements
CHANGELOG for 2.3.3.0:
-Fix renaming backups with a space in the name
-Add decrypt button to mount page if you cancel decryption during startup
-Added ignore blkid flag
-Fixed handling of MTD partitions during mount
-Fixed some keyboard mapping issues on 800x1280 layout
[/hide]
This is an unofficial port of TWRP by ME (runandhide05)
Standard disclaimers
blablabla im not responsible for anything.
Download Links
2.7x pulled
*NOTE THAT YOU SHOULD ALWAYS USE THE MOST UP TO DATE RECOVERY, IF YOU ARE PLANING ON FLASHING A 4.2 BASED ROM YOU SHOULD UPDATE TO THE LATEST BEFORE FLASHING ROM.*
R.A.H._TWRPv2.6.3.zip
zip
https://www.androidfilehost.com/?fid=95916177934538765
img
https://www.androidfilehost.com/?fid=95916177934538766
all in one
https://www.androidfilehost.com/?fid=95916177934538767
R.A.H._TWRPv2.6.1.zip
Install_Scripts_R.A.H._TWRPv2.6.1.zip - 11.70 MB
R.A.H._TWRPv2.6.1.img - 5.70 MB
R.A.H._TWRPv2.6.1.zip - 5.79 MB
Install_Scripts_R.A.H._TWRPv2.6.0.zip - 11.60 MB
R.A.H._TWRPv2.6.0.img - 5.65 MB
R.A.H._TWRPv2.6.0.zip - 5.75 MB
TWRPv2.5.0
Install_Scripts_R.A.H.v3_TWRPv2.5.0.zip - 11.61 MB
R.A.H.v3_TWRPv2.5.0.img - 5.64 MB
R.A.H.v3_TWRPv2.5.0.zip - 5.75 MB
TWRPv2.4.1.0
Install_Scripts_v2.4.1.0.zip - 10.24 MB
TWRPv2.4.1.0.img - 4.95 MB
TWRPv2.4.1.0.zip - 5.06 MB
Installation Options
Installation instructions (option 1)
Download zip file from link below
Place zip on Micro SDcard
Reboot into existing recovery
Flash TWRP2.2.0_R.A.H.v0.0.X.zip
Installation instructions (option 2)
download recovery.img
place tablet in to FastBoot
Flash Recovery with
Code:
fastboot flash recovery name-of-recoveryimage.img
Installation instructions (option 3)
Download zip file, extract zip to anywhere, choose the Operating system
linux or windows folder,
For windows:
Simply click the RUNME.bat
For Linux:
Make adb and fastboot files included in the linux folder executable
Make script executable.
click RUNME.sh
Installation instructions (option 4)
download through goomanager app
GooManager
I am proud to announce that today twrp recovery is now officially available through goomanager app.
These are the Stock (non-EOS themed)
Thank you Dees_Troy
Source: runandhide05's Github
For more info on TWRP see the Team Page teamw.in
Bug tracker bug tracker
Nice
I'll try it as soon as possible & report soon
Wow. Dude you got the win factor baby!
Were you able to get open recovery script working? Heck I'll just flash . Epic win bro.
Sweet, thanks runandhide05!
(Not often I thank other people around here!)
bigrushdog said:
Wow. Dude you got the win factor baby!
Were you able to get open recovery script working? Heck I'll just flash . Epic win bro.
Click to expand...
Click to collapse
Not yet, It still errors on opening file, I'm thinking its due to the file location as /storage/.... and not /sdcard...
This is something that I will be working for next release.
Sent from my Galaxy Nexus using Tapatalk 2
solarnz said:
Sweet, thanks runandhide05!
(Not often I thank other people around here!)
Click to expand...
Click to collapse
WOW!!!! Goosebumps!
Thanks, mean a lot from you and brd!
Thanks for everything you guys do!
Sent from my Galaxy Nexus using Tapatalk 2
Big thanks, already do a backup, restore, and move file from internal to external, so many features to get use to again thanks for a very fine software...
since this is working why not change the direction of this thread to xoom android development section?
Thanks in advance
Anyone else having problems with the download links not working.
TheBurgh said:
Anyone else having problems with the download links not working.
Click to expand...
Click to collapse
Dev-host is down it seems. And I'm mobile right now.
They will be back up soon enough
Sent from my Galaxy Nexus using Tapatalk 2
runandhide05 said:
Dev-host is down it seems. And I'm mobile right now.
They will be back up soon enough
Sent from my Galaxy Nexus using Tapatalk 2
Click to expand...
Click to collapse
I love this, but I was unable to backup to external or actually do anything with external storage. Am I doing something wrong?
okantomi said:
I love this, but I was unable to backup to external or actually do anything with external storage. Am I doing something wrong?
Click to expand...
Click to collapse
Must be doing something wrong, I am able to backup to external and internal with 3 1/2 gig /data size.
Look at where it says to backup to, and select internal or external
Sent from my Galaxy Nexus using Tapatalk 2
Any mirrors dev-host seem to go down a lot lately
thisismalhotra said:
Any mirrors dev-host seem to go down a lot lately
Click to expand...
Click to collapse
Mirrors are in OP now
Sent from my Galaxy Nexus using Tapatalk 2
runandhide05 said:
Must be doing something wrong, I am able to backup to external and internal with 3 1/2 gig /data size.
Look at where it says to backup to, and select internal or external
Sent from my Galaxy Nexus using Tapatalk 2
Click to expand...
Click to collapse
Problem is, it won't let me choose external, but that's not my biggest issue right now.
I can't restore from the backup I've just done (on internal), can't install anything from internal and still can't access external. I've tried many different approaches.
I can't reflash a recovery.img because I'm away from my PC so I'm stuck. I'm sure there's something I'm not getting about the file system this recovery sees...it doesn't see "storage" on the root directory. I can't access it that way.
But this is really a cool recovery, and has many great features.
okantomi said:
Problem is, it won't let me choose external, but that's not my biggest issue right now.
I can't restore from the backup I've just done (on internal), can't install anything from internal and still can't access external. I've tried many different approaches.
I can't reflash a recovery.img because I'm away from my PC so I'm stuck. I'm sure there's something I'm not getting about the file system this recovery sees...it doesn't see "storage" on the root directory. I can't access it that way.
Click to expand...
Click to collapse
Look at op where the external and internal are mounted as. I'll look into it when I have a mome t
Sent from my Galaxy Nexus using Tapatalk 2
runandhide05 said:
Look at op where the external and internal are mounted as. I'll look into it when I have a mome t
Sent from my Galaxy Nexus using Tapatalk 2
Click to expand...
Click to collapse
I think I may have resolved my issue of not being able to flash ROM and Gapps...I could do it from Goomanager in internal. If everything works, then I'll experiment some more.
Thanks for adding the locations to internal and external in the OP. It was not something I would have figured out on my own ;^)
okantomi said:
I think I may have resolved my issue of not being able to flash ROM and Gapps...I could do it from Goomanager in internal. If everything works, then I'll experiment some more.
Thanks for adding the locations to internal and external in the OP. It was not something I would have figured out on my own ;^)
Click to expand...
Click to collapse
No problem... FYI they were in op from the beginning... no anything I added.. helps to read.. lol I kid buddy.
Hope all works out for ya
Sent from my Galaxy Nexus using Tapatalk 2
runandhide05 said:
No problem... FYI they were in op from the beginning... no anything I added.. helps to read.. lol I kid buddy.
Hope all works out for ya
Sent from my Galaxy Nexus using Tapatalk 2
Click to expand...
Click to collapse
Haha...thought I read it all.
Seriously, still can't seem to restore from my back-up...seems like it says complete and successful, but hasn't actually changed anything.
Let me explore it a bit more (read more, lol) and see what I've been doing wrong.
Thanks again...all of your work is very high quality and high value for us here on the Xoom.
okantomi said:
Haha...thought I read it all.
Seriously, still can't seem to restore from my back-up...seems like it says complete and successful, but hasn't actually changed anything.
Let me explore it a bit more (read more, lol) and see what I've been doing wrong.
Thanks again...all of your work is very high quality and high value for us here on the Xoom.
Click to expand...
Click to collapse
OK well I have made a backup, flashed a ROM, wiped every partition than can be formated and flashed other recoverys zips with no problem. I haven't restored a backup yet, I am at work but believe me I will be working on this tonight to get open script working and I will do some more extensive testing also.
Sent from my Galaxy Nexus using Tapatalk 2
Intro:
---------------------
Extended TWRP is based on official Team Win Recovery Project, but modified** to address some special methods & features found on The HD2,
a device which has 3 different android loaders (cLK, MAGLDR and Haret) and various Android Rom configurations (DataOnExt, NativeSD, SD, Nand).
** If you are just looking for an unmodified TWRP for the HD2, check here.
I take no credit for this. All the creative hard work was already done!
Credits to:
agrabren, Dees_Troy, ViViDboarder, and AssassinsLament - core twrp 2 development team.
Toastcfh – for his underappreciated dedication to the community.
s0up and kevank - for their hard work on the web side of things.
shift, onicrom, netarchy, kevank, myndwire, bigbiff, dkelle4, shinzul, spiicytuna, and eyeballer - the rest of Team Win.
Amon_RA - for his great Recovery.
Koush - for his dedication to the community and ClockworkMod.
Akmzero, arif-ali, Evil_DevNull, gus6464, Jesusice, _jmz_, NxtGenCowboy, ohshaith55, Pyrostic, [R], and Rootzwiki - for all the help with devices and testing.
Team's supporters and the Android Community.
However, if you think that this extended version is full of unnecessary cr*p or that something is not working as expected anymore,
Credits to:
me
Notice:
---------------------
For me, this thread is about learning while having fun. It would be ideal if it leads to something that is really useful.
So, as you may have already understood, This Is Not a release thread. If you have the time and interest to help in any way, please do. Cause together we can make sure that this project will benefit all of us.
The attached recovery image is provided for people who can't compile it from source, but want to either help in debugging or just experiment.
Have in mind that before attaching it here, it was only tested on one HTC-HD2(Eu) powered by (black) cLK bootloader.
The performance on devices powered by MAGLDR bootloader or running the good-old WM6.X should be considered untested.
In order to exploit the full potentials of this software, Black cLK bootloader is strongly recommended!
As usual, use at your own risk. TeamWin or kokotas is not responsible for anything bad that may occur from using Extended TWRP.
ReadMe
Features:
---------------------
**************Mainline**************
* Ability to save custom recovery settings
* Touchscreen driven with real buttons and drag-to-scroll
* XML-based GUI that allows full customization of the layout – true theming!
* Settings are saved to the sdcard and persist through reboots
* Ability to choose which partitions to back up and which to restore
* Ability to choose to compress backups – now with pigz (multi-core processor support for faster compression times)
* Ability to backup large partitions (size>1.5GB) by splitting the backup archive.
* Basic file manager (copy, move, delete, and chmod any file)
* OpenRecoveryScript scripting engine
* On-screen keyboard in recovery! -- supports long press, backspace repeat, and swipe left deletes everything left of the cursor
* ADB sideload functionality from AOSP
* Pseudo-terminal emulator
* Decryption support
* Ability to set a timeout for auto-turning off the screen
* exFAT support for sdcard's 1st partition
* Manual screen rotation
**************Extended**************
* Proper detection of bootloader(cLK/MAGLDR/haret)
* Support for cLK's extra boot partitions
* Tweaked off-mode charging for cLK (device can wake up by pressing any key)
* Direct rebooting to selected boot partition for cLK bootloader
* Direct rebooting with selected kernel from NativeSD folder for MAGLDR bootloader
* Ability to communicate with cLK in order to change partitions' size if needed
* Built-in NativeSD manager(Backup - Restore - Delete - Fix Permissions - Wipe Data - Wipe Dalvik-Cache - [cLK]Kernel-Restore)
* Option to skip any NativeSD Rom during sd-ext's partition backup
* Option to adjust backup/restore process for DataOnExt method
*
* Nilfs2 support for sdcard's ext(2nd/3rd primary) partition
* NTFS support for sdcard's 1st primary partition
* Option for converting file system [ext2 - ext3 - ext4 - nilfs2] of sdcard's ext partition (without losing any data if there is enough space on the /sdcard)
* Option for adding a 3rd primary partition(mmcblk0p3 as /sdext2)
* Option to skip dalvik-cache during backup
* Ability to restore backups that were made using a CWM Recovery
* Ability to check SD Card's filesystem(s)
* Ability to run shell scripts from your SD Card (script location: /sdcard/TWRP/scripts)
* Ability to "run" recovery (AROMA based) apps with one click (app location: /sdcard/TWRP/app)
* Ability to select current theme (example of theme file location /sdcard/TWRP/theme/MyTheme/ui.zip)
* Ability to check the size of the backup to be restored
* Ability to take screenshot (screenshots location: /sdcard/TWRP/screenshots)
* Configurable haptic feedback
* Configurable system tweaks (cpu gov, cpu freq, i/o sched, drop_caches)
Source:
---------------------
Extended-Team-Win-Recovery-Project
Looks:
---------------------
Screenshots
Compatible themes:
TWRP's Default Style Blue Theme for v2.5.0.5
Dark_Avenger's Holo Dark Theme for v2.6.3.0
Download:
---------------------
HD2_EXTENDED_TWRP_2.6.3.4.zip
md5: 5af088d200a00e0a3efb27860a5168c6
Info: You need to have a recovery partition at least 9 MB to fit the img
HD2_EXTENDED_SD_TWRP_2.6.3.4.zip
Info: Extract to the root of your SD Card
Frequent issues:
---------------------
If you report an issue without a log and/or without details, that's not very helpful in tracking down that issue, so, eventually, don't expect any assistance cause I am not a medium.
If you get a "status 2" error when installing a recovery-update-package(zip) that used to work with your old recovery, change the update-binary inside the zip file with a newer one (API 3).
DON'T FORGET to check the syntax in the updater-script and verify that format() and mount() commands have the correct amount of arguments, otherwise you'll end up having other errors like "status 7".
i.e. if you use the update-binary included in the attached file below, format takes 2: format("MTD", "boot").
If you occasionally can't boot into recovery, try increasing the size of its partition >= 10MB.
Installation:
---------------------
[Basic stuff before installing]
Make sure you have adb and fastboot working (required files here). Please don't ask here questions about this - a simple web search will be enough to help you.
Make sure that the size of the 'recovery' partition is enough for the image to fit.
Make sure the name of the recovery *.img file is the one you use in the commands or in flash.cfg
[Flashing occasions]
[cLK/MAGLDR]
Flashing the zip file from existing Recovery
Download zip file to your computer and copy to /sdcard.
Reboot phone into recovery.
Select to install the zip file from your /sdcard.
Reboot Phone into recovery as normal.
[cLK/MAGLDR]
Flashing the *.img file while booted in Android or in Recovery
Download the recovery file to your computer.
If it is a zip file, extract the *.img.
(Win) Open the folder you have the recovery, do a [Shift+RightClick] and select "Open command window here".
(Lnx) Open the folder you have the recovery, RigthClick and select "Open in Terminal".
Execute the commands:
Code:
adb push recovery.img /sdcard/recovery.img
adb shell erase_image recovery
adb shell flash_image recovery /sdcard/recovery.img
Reboot Phone into recovery as normal.
[cLK]
Flashing the *.img file while in fastboot mode(cLK menu)
Download the recovery file to your computer.
If it is a zip file, extract the *.img.
(Win) Open the folder you have the recovery, do a [Shift+RightClick] and select "Open command window here".
(Lnx) Open the folder you have the recovery, RigthClick and select "Open in Terminal".
Reboot phone into cLK menu (fastboot mode).
If needed, change the size of the 'recovery' partition on the fly (under SETTINGS/RESIZE PARTITIONS).
Execute the commands:
Code:
fastboot erase recovery
fastboot flash recovery recovery.img
fastboot oem boot-recovery
[MAGLDR]
Flashing the *.img file while in USB Flasher mode(using DAF.exe)
Code:
[B][COLOR="Red"]WARNING!!![/COLOR] This way of flashing will erase your current Rom on NAND too!
You will probably want to do this
[COLOR="Red"]ONLY IF[/COLOR] you need to increase the size of the 'recovery' partition![/B]
Download the recovery file to your computer.
If it is a zip file, extract the *.img.
Open the folder you have the recovery and place DAF.exe and flash.cfg in that same folder.
Edit flash.cfg according to required recovery partition size:
Make sure the .img file has the name that flash.cfg specifies!!
i.e. If the *.img file is named recovery.img and min-required-size = 8MB then the flash.cfg could be:
Code:
misc ya 1M
[U]recovery rrecov|ro|nospr 8M recovery.img[/U]
boot yboot|ro 5M
system ya 150M
cache ya 2M
userdata ya|asize|hr allsize
Check out the info about flash.cfg.
Connect the device to the computer via usb, enter MAGLDR and select USB Flasher option.
Right click on DAF.exe and select Run as Administrator.
Click [NEXT] when asked and wait to finish.
Post-Installation:
---------------------
Make sure you delete /sdcard/TWRP/.twrps.
If you're using a theme, update your theme file or temporarily switch to the default theme.
A big Thank-You for supporting/testing the ExtendedTWRP to:
---------------------
Dark_Avenger
NYLimited
DarrenNewsgroup
Nixda99
MarkAtHome
Robbie P
Timeline
Commit History
HD2_Extended_TWRP_2.6.3.4 @ 2013.11.24
Compiled tytung's jellybean kernel with CONFIG_YAFFS_XATTR=y
and replaced existing recovery kernel.
Attempt to fix curtain.jpg loading bug.
HD2_Extended_TWRP_2.6.3.3 @ 2013.11.20
Reverted to 2.x kernel (more stable and smaller size).
HD2_Extended_TWRP_2.6.3.2 @ 2013.11.20
Ported latest mainline commits.
Compiled without ntfs and exFat support (since no existing HD2 Rom really requires any of them).
Switched to 3.x kernel (github.com/TeamNightmare/android_kernel_htc_leo).
HD2_Extended_TWRP_2.6.3.0 @ 2013.09.19
Ported latest mainline commits (update to v2.6.3.0).
Disabled the ability to encrypt the backup of the data partition. Rolled back to old twrpTar code in an attempt to temporarily fix some backup/restore errors.
(The backups made by previous versions might not be good).
HD2_Extended_TWRP_2.6.1.2 @ 2013.09.09
Fix crashing when installing a recovery-package.
Added "bootloader=haret" to startup.txt for haret's case.
HD2_Extended_TWRP_2.6.1.1 @ 2013.09.06
Reverted a commit that messed up the rotation feature.
Attempt to improve bootloader detection.
HD2_Extended_TWRP_2.6.1.0 @ 2013.08.31
Ported latest mainline commits (update to v2.6.1.0).
HD2_Extended_TWRP_2.6.0.0 @ 2013.07.16
Ported (most of the) latest mainline commits (update to v2.6).
Ported Vojtech Bocek's screen rotation.
Old themes are not compatible!
Instead of one ui.xml file, the new theme requires two files portrait.xml and landscape.xml and the appropriate images.
Misc code cleanups/fixes.
HD2_Extended_TWRP_2.5.0.5 @ 2013.05.10
Try to fix libtar bug (present when creating sd-ext's backup).
Reboot options for "recovery" and "bootloader" will be available for cLK only.
Revised nativeSD manager.
HD2_Extended_TWRP_2.5.0.4 @ 2013.05.08
Made partitionlist's selection persist.
Added ability to load selected theme's curtain.jpg.
Fixed slider-touch.png, changed list's header separator color to red and reordered header info in built-in theme.
Revised how twrp handles 'userdata' partition when DataOnExt mode is.
Cause there is a chance 'DalvikOnNand' method is used.
Cleaned up unused code and added a flag for HD2-specific stuff so if anyone would try to build Extended TWRP for another device it would be easier.
HD2_Extended_TWRP_2.5.0.3 @ 2013.05.06
Added "View file" under filemanager options (it's more or less like 'cat /file').
Small fix to off-mode charging.
Attempt to fix progress bar issue.
Changed built-in theme. The new one is based on Dark_Avenger's Holo Dark.
Changed some log messages.
HD2_Extended_TWRP_2.5.0.2 @ 2013.04.22
Changed behavior of "Reset Defaults". Device will reboot after that.
Changed handling of extra boot partitions for cLK.
Now you can backup/restore/wipe any of those partitions just like the default ones!
(In the end this means that you can backup/restore EVERYTHING at once)
Changed versioning way (last digit will be used for the extended revision).
Changed screenshot function:
If the system is busy (perhaps doing something (?) to the sdcard) no screenshot will be taken if you press the top logo.
Otherwise, in order to notify the user that a screenshot was successfully taken the keys backlight will blink.
Reorganized/rewrote some of the code (backup/restore/wipe) around "partitionlist".
Removed sfdisk.
Recompiled tytung_jellybean_r2 kernel with increased MSM_MAX_PARTITIONS to 20 (this will fix the detection of cLK's extra boot partitions).
Misc fixes that I don't remember now...
HD2_Extended_TWRP_2.5.0.0_r1 @ 2013.04.17
Ported latest mainline commits (update to v2.5).
Fixed battery_thread().
Added ability to auto delete logs from cache to free up some space when needed (see Dark_Avenger's post).
Switched kernel to tytung_jellybean_r2.
Misc fixes.
Compiled img with TW_INCLUDE_CRYPTO = true & TW_INCLUDE_JB_CRYPTO = true.
Increased required min size to 9MB.
HD2_Extended_TWRP_2.4.4.0_r7 @ 2013.04.05
Attempt to fix /sdcard mounting issues in case of slow cards(thanks to Darren for helping).
Fixed restoring a tar that was made with a previous version that used busybox tar.
HD2_Extended_TWRP_2.4.4.0_r6 @ 2013.04.03
Attempt to fix archive extraction during restore.
Added separate option for excluding dalvik-cache from a NativeSD Rom's backup under NativeSD Manager Settings.
HD2_Extended_TWRP_2.4.4.0_r5 @ 2013.04.02
Attempt to fix timezone dst settings.
Revised extractTarFork() function.
HD2_Extended_TWRP_2.4.4.0_r4 @ 2013.03.31
3rd attempt to fix sdcard partitioning.
Added option for magldr users to select which NativeSD Rom to reboot to(jskenderi's idea).
Fixed cpu governor selection not getting applied at boot.
Removed "read_ahead_kb" tweaks from init.rc.
Revised LoadValues() function.
HD2_Extended_TWRP_2.4.4.0_r3 @ 2013.03.22
2nd attempt to fix sdcard partitioning.
Changed "Enable MD5 verification" to "Skip MD5 verification" - The default behavior is to verify md5s.
Fixed a bug regarding backing up with "Exclude NativeSD Roms..." option enabled.
HD2_Extended_TWRP_2.4.4.0_r2 @ 2013.03.20
First attempt to fix sdcard partitioning.
Ported latest mainline md5sum fix.
Fixed NativeSD dalvik wipe.
Replaced the option for the preboot scrict with an option to force TWRP to handle boot partition as 'mtd'.
Only MAGLDR users will be able to see it AND in case your "boot" is raw (rboot was used in flash.cfg), enable it!
HD2_Extended_TWRP_2.4.4.0_r1 @ 2013.03.14
Ported latest mainline commits.
Updated/fixed clkpartmgr (needs cLK v1.5.2.0 to work correctly) - partition size is passed in blocks.
Revised storage partitioning method.
Enabled ntfs and exFat formats for card's 1st primary partition.
Switched to libtar for archive handling.
Switched to libblkid for filesystem detection.
Added more options for haptic feedback.
Added option under "Settings" for the preboot scrict that sets the filesystem for "boot" partition before recovery loads recovery.fstab.
If your bootloader is MAGLDR and "boot" is raw, disable it. IN ANY OTHER CASE leave it enabled.
If you don't know what I'm talking about, leave it enabled.
Kernel is compiled with defaults: gov = performance, i/o scheduler = deadline, cpu freq = (245 - 998)MHz
SD Card read_ahead_kb set to 2048, NAND's partitions read_ahead_kb set to 128
Added system tweaks options (cpu gov, i/o scheduler, cpu max/min freq, available memory boost).
If we want to have a theme setting persisted there is a new function <action function="save"> instead of <action function="set">.
HD2_Extended_TWRP_2.4.3.0_r3 @ 2013.03.04
Reverted the way TWRP handles 'boot' partition's file-system.
Backup/restore for MAGLDR should work now.
Ported the dim effect before screen turns off from mainline.
Misc small changes in code.
HD2_Extended_TWRP_2.4.3.0_r2 @ 2013.03.02
Reverted fileselector to clear by default the data variable.
HD2_Extended_TWRP_2.4.3.0_r1 @ 2013.03.01
Ported mainline commits (update to 2.4.3.0).
One of them fixes the fileselector bug.
1st attempt to fix UI's bug (hot-rebooting after theme changing).
HD2_Extended_TWRP_2.4.2.0_r2 @ 2013.02.27
Fixed screen timeout not being loaded at boot (instead it was always 60 sec).
Added haptic feedback and an option to enable it under "Settings".
Fixed screenshot feature.
HD2_Extended_TWRP_2.4.2.0_r1 @ 2013.02.25
Added option for the root checking stuff. If you don't want TWRP messing with your system you have the option to disable it under "Settings".
Added option to select the clock-time format.
Cleaned up blanktimer.
HD2_Extended_TWRP_2.4.1.0_r3 @ 2013.02.24
Ported latest mainline commits.
Some of them were already in Extended TWRP(button highlight and 24h clock)
One of them added a "Screen" button under "Settings" which gives you the option to
(a)set the timeout for auto-turning-off the screen and
(b)set the level of screen brightness.
Created separate thread for monitoring battery capacity and usb status.
Led will work as normal. I've also set it to blink the amber led if battery < 10%.
Tweaked offmode-charging (this is for cLK users that don't use cLK's built-in offmode charging mode).
Fast charging is enabled in kernel and 'Powersave' settings will be used.
Also added the ability to wake up the device while offmode-charging by pressing ANY key.
Removed unused/duplicate options from "Settings".
When you skip dalvik-cache during a backup, a file named .nodalvikcache will be created in the backup folder with the size of dalvik-cache stored inside.
This will be used whenever you restore that backup in order to add that size to the calculations for minimum partition's size.
In case the '.nodalvikcache' file doesn't exist there is an option to set the increment(%) for the size of the partition to which a backup that doesn't contain dalvik-cache will be restored. Default value is 40%.
Changed the way TWRP handles 'boot' partition's file-system. It doesn't depend anymore on which bootloader we have (cause MAGLDR supports both yaffs2 and raw mtd).
Code:
Dropped the logic:
If cLK => mtd
else => yaffs2
Now it really checks the file-system.I think
Added the ability to restore a backup made with cLK as bootloader on MAGLDR.
...
HD2_Extended_TWRP_2.4.1.0_r2 @ 2013.02.11
Moved option for backing up nand's userdata to BACKUP page.
Instead of Data show DataOnExt or DataOnNand accordingly.
Added option under "SETTINGS" page to let TWRP check if a partition's size is enough for its backup to be restored before tar throws an error caused of insufficient space.
MAGLDR users: if the selected part of your backup is larger than the actual size of the partition, this part will be excluded from the restore process.
cLK users: Update to v1.5.1.9 and size mismatches will be no problem, since TWRP and cLK will automatically resize any partition that needs resizing.
Show progressbar during NativeSD backup.
Updated clkpartmgr (the args passed to it have changed too).
HD2_Extended_TWRP_2.4.1.0_r1 @ 2013.02.06
Port mainline commit(TWFuncc::Get_Folder_Size and MD5 checking).
Added function for finding a file on storage using the file's name and used it in OpenRecoveryScript.
Add clkpartmgr to TWRP.
HD2_Extended_TWRP_2.4.0.0_r8 @ 2013.02.01
When restoring a backup with DataOnExt show 'data' instead of 'sd-ext' under Restore page.
Always reset Use_unyaffs_To_Restore to false.
Fix Recreate_DataOnExt_Folder().
HD2_Extended_TWRP_2.4.0.0_r7 @ 2013.02.01
Created separate thread for updating time clock in order to avoid having it frozen.
Fix for sd-ext button disappearing after wiping.
Recreate folder for DataOnExt after wiping sd-ext.
Port last mailine commits.
HD2_Extended_TWRP_2.4.0.0_r6 @ 2013.01.31
Fixed backup part selection when there are split archives.
Corrected free space calculation in FS-converting and NativeSD-backup.
Time format changed to hh:mm:ss. (Sometimes seconds get frozen but it's nothing to worry about)
Misc NativeSD Manager fixes.
HD2_Extended_TWRP_2.4.0.0_r5 @ 2013.01.30
Added ability to select theme.
You must place the theme file(ui.zip) in a folder under /sdcard/TWRP/theme (eg /sdcard/TWRP/theme/Holo_Dark/ui.zip)
Added ability to select and load a recovery package like Aroma-File-Manager or NativeSD-Recovery.
You must place the files (*.zip) under /sdcard/TWRP/app.
Fixed NativeSD backup(tar arguments error).
Moved Update_Size() from Mount() function to Wipe().
HD2_Extended_TWRP_2.4.0.0_r4 @ 2013.01.29
Fixed a mounting issue:
Update_Size() function was called inside Mount().
However inside Update_Size() there was a call for Mount() and UnMount(!?) causing a mess.
Lets see if this could fix the problem with DataOnExt too.
HD2_Extended_TWRP_2.4.0.0_r3 @ 2013.01.29
Ported mainline commits till 29th of Jan.
Fixed boot partition's flag so that it can be wiped
Settings file (.twrps) is now read once at boot.
The TWRP folder structure on /sdcard will be auto-generated.
HD2_Extended_TWRP_2.4.0.0_r2 @ 2013.01.27
Updated to twrp2.4.
Ported all mainline commits till 26th of Jan.
Rewrote most of the extended functions.
Added option to format all main partitions except for /sdcard.
Added new feature in file manager: the size of the selected file or folder will be shown in the operation page.
HD2_Extended_TWRP_2.3.3.0_rev12 @ 2013.01.16
One&only purpose: Attempt to fix all issues for MAGLDR users.
Extensive logging is present in this rev.
HD2_Extended_TWRP_2.3.3.0_rev11 @ 2013.01.15
Second attempt to fix weird sdcard issues.
Added more logging output to help find the problem.
HD2_Extended_TWRP_2.3.3.0_rev10 @ 2013.01.14
Probably fixed the error during "Fix-Permissions" when DataOnExt is used.
Reverted changes in fstab as an attempt to fix weird sdcard issues.
HD2_Extended_TWRP_2.3.3.0_rev9 @ 2013.01.13
Another attempt to fix partitions' sizes for the backup process.
Added option (under "Backup" page) for the user to select if the NativeSD roms will be excluded or not from the sd-ext backup.
Added ability to run shell scripts that you might have on your sdcard (/sdcard/TWRP/scripts).
Added back the missing "Reload Theme" button under "Advanced" page.
HD2_Extended_TWRP_2.3.3.0_rev8 @ 2013.01.06
Changed the path where TWRP will save any taken screenshot to "/sdcard/TWRP/screenshots/".
The name of the png file will have this format "TWRPScr-001.png". The increment will be done automatically.
Unfortunately I haven't fixed the quality yet.
Added an option under "Settings" to "Enable path-checking for DataOnExt".
If enabled the path that will be used for the DataOnExt mode will be checked before any errors occur. Hopefully this will help non-experienced users to use DataOnExt function.
Added ability to automatically skip any NativeSD Rom folder from your sd-ext backup.
If you want to backup a NativeSD Rom do it from the NativeSD Manager.
Probably fixed the way TWRP calculates the partition sizes according to our extended settings(skip dalvik-cache - dataonext - skip nativesd roms).
So now the sizes (shown under "Backup" and) used during backup-process should be correct.
HD2_Extended_TWRP_2.3.3.0_rev7 @ 2013.01.04
Removed the second parsing of recovery.fstab (the one after "postrecoveryboot.sh" run). Instead the script "prerecoveryboot.sh" will run before TWRP parses the recovery.fstab file.
This should also eliminate any weird errors you've been seeing in the logs.
Attempt to resolve problems relevant to "Fix Permissions" when DataOnExt mode is ON.
Added a second step/check in UnMount() in case it fails the first time. Just like I did for the Mount() in previous rev.
Added option to skip saving "dalvik-cache" during backup (under "Backup" menu) in order to minimize the size of the backup file. After-all dalvik-cache is recreated. It will be like you've wiped it when you restore that backup.
Changed the way Backup & Restore work for DataOnExt:
If you have DataOnExt checked and you go ahead and start a backup process then
a (hidden) file called .dataonext will be created in the backup folder.
The path for your /data mount point will be stored inside that file.
Do not delete that file cause you'll have to recreate it manually!
Regardless of the settings you might have(DataOnExt checked or unchecked) if you go ahead and start a restore process of a backup with DataOnExt
(so it has the .dataonext file inside - if not it's easy to manually create it)
the option for "Setup recovery for DataOnExt" will be checked and
the path (DataOnExt button) will be reset to the path you had when the backup was done.
If you were using the entire /sd-ext for /data (original DataOnExt) when you made your backup then
when you restore that backup, the sd-ext will be formatted.
If you were using a subdir of /sd-ext for /data (modified DataOnExt like "/sdext/NexusHD2-JellyBean/data") when you made your backup then
when you restore that backup, the sd-ext will not be formatted - instead only that subdir will be deleted.
Ported Tasssadar's "clicked" effect to all buttons as feedback.
Been experimenting and added some code to be able to take a screenshot (will be saved as /sdcard/TWRPScr.png).
It is done by touching the top LOGO. To enable it, there is a relevant checkbox under "Settings".
It doesn't work as expected yet! The pics taken are crap but it's a wip...
HD2_Extended_TWRP_2.3.3.0_rev6 @ 2013.01.02
Fixed a problem when restoring an sd-ext backup that was made with CWM Recovery.
Added an option to run some filesystem checks on the sd-card under "Partition SD Card" button inside "Advanced".
Enabled power key to lock the screen.
HD2_Extended_TWRP_2.3.3.0_rev5 @ 2012.12.30
1st attempt to fix mounting problem(s) related to ext partition.
TWRP is now able to restore backups that were made using a CWM Recovery.
All you have to do is copy the entire backup folder (i.e. 2012-08-20.09.12.38) from /sdcard/clockworkmod/backup to /sdcard/TWRP/BACKUPS/htcleo
HD2_Extended_TWRP_2.3.3.0_rev4 @ 2012.12.28
1st attempt to fix TWRP's failure on file-system check when a partition has a label.
Changed how TWRP handles the ext partition(s) size while parsing recovery.fstab.
HD2_Extended_TWRP_2.3.3.0_rev3 @ 2012.12.27
3rd attempt to fix the .android_secure bug.
Added an SD version for testing.
HD2_Extended_TWRP_2.3.3.0_rev2 @ 2012.12.26
2nd attempt to fix the .android_secure bug.
1st attempt to fix the crash of sd card's partitioning when adding 2nd ext partition.
Reverted the logo to the default one.
New settings for fs-check and DataOnExt will be saved and not lost after a reset.
Switched the type of release from just an img-file to a flashable-zip with a script to check recovery's partition size before updating.
HD2_Extended_TWRP_2.3.3.0_rev1 @ 2012.12.24
Ported commits from mainline 2.3.3.0.
Changed how the DataOnExt will be handled.
Under "Settings" you'll find a checkbox with which you can "tell" recovery that you're using DataOnExt or not.
In case you're using a modified DataOnExt method you can set the actual path of your /data (i.e. /sd-ext/%RomName%/data) by hitting DataOnExt button.
Probably fixed the .android-secure bug.
Reverted the backup path(BACKUPS/htcleo/).
So you have to create an intermediate folder named "htcleo" under BACKUPS and move your backups inside.
Reverted the max size for splitting archive to 1.5GB.
1st attempt for saving the contents of your ext partition when you change the fs type (will need enough free space on your sdcard for this to work).
Added option to set the number of mounts that will trigger a filesystem check under "Settings".
Added option to wipe Data, Boot and (cLK)sBoot under "wipe" Menu.
Added necessary code for selecting fs type (vfat / ntfs) for /sdcard but since I can't make it function properly I've disabled it in the UI. (Also in case it will work using the ntfsprogs binaries, the size of the recovery will increase to ~ 8MB)
HD2_TWRP_2.3.2.X @ 2012.12.14
Detection of bootloader
Nilfs2 support for sdcard's ext partition
Option for converting file system(formatting) of sdcard's ext partition: [ext2 - ext3 - ext4 - nilfs2]
In-built NativeSD manager
Option for adding a 2nd ext partition(mmcblk0p3)
(cLK)Direct rebooting to selected boot partition
Option to adjust the backup process for DataOnExt case
1st attempt for archive splitting(also lowered the max size to 512MB)
1st attempt for MAGLDR boot restoring
Info
How to report a problem
Provide the log file.
In order to get it connect your device to pc via usb and run in terminal:
Code:
adb pull /tmp/recovery.log
If that fails for some strange reason then you could also try:
Code:
adb pull /cache/recovery/log
Or you could just use the "Copy Log" button under "Advanced" and have the log copied to the root of your card.
Give a short description of what triggered the problem.
If you don't have your device's setup in your signature, post that info too. For example:
Code:
HD2 ver.: EU
SDCard: Sandisk 32GB(c4)
SDCard's Partition Table: mmcblk0p1 [vfat,30GB], mmcblk0p2 [ext4,2GB]
Bootloader: cLK 1.5.1.6
NAND's Partition Table: recovery 8MB, misc 2MB, boot 4MB. userdata auto-size, system 200 MB, cache 2MB
Rom: HyperDroid 6.1 - NAND
Since this is a WIP, you could also use github to report any issue.
Theme related
Information on Official TWRP Theming.
Extended TWRP gives you the ability to choose between many themes.
When booting, it checks for a theme zip file on your card:
/sdcard/TWRP/theme is the location where you must create a folder named after the theme you will use and place the ui.zip inside it (i.e. /sdcard/TWRP/theme/Holo_Dark/ui.zip).
If none found then it will use the built-in theme.
For those who prefer the blue-ish TWRP default theme, I will be attaching it to this post every time something changes, so that you can use it as any external theme.
Since version 2.6.0.0, the theming engine of the Extended TWRP differs from the official.
Instead of one ui.xml file, the new theme requires two files portrait.xml and landscape.xml with the appropriate images.
Thanks for your work. Will try and report back.
Holo Dark theme for version 2.6.3.4 (portrait only).
Nice! will give this a shot
thanks.
Dark_Avenger said:
Updated Holo Dark theme for version 2.4.0.0 rev2
Click to expand...
Click to collapse
How is your theme applied? do you extract it and put it in a folder somewhere?
Re: [DEV][27/Jan/2013]Extended TWRP|v2.4.0.0.r2
Create a folder "/sdcard/TWRP/theme" and place the ui.zip there without unzipping it.
Sent from my HTC HD2 using Tapatalk 2
Re: [DEV][27/Jan/2013]Extended TWRP|v2.4.0.0.r2
v2.4.0.0r2 is not mounting my sd-ext at all, when I checkmark it manually, it vanishes after a short while.
Obviously, fix permissions don't work. Backups menu reports 0 size.
My log: http://ompldr.org/vaDk5aw/recovery.log
System setup:
Boot loader: Black cLK 1.5.1.6
Radio: 2.15.50.14
ROM: NexusHD2-JellyBean-1.3a
SD card: SanDisk class 4 30GB fat32 mmcblk0p1 and 2GB ext4 mmcblk0p2
Internal partitions: system 390M, user data auto, recovery 8M, rest cLK default.
TWRP v2.3.3.0rev7 was the last extended recovery that worked perfectly for me.
Thanks kokotas. =)
Sent with love from me to you via HTC HD2 with Tapatalk 2
sunitknandi said:
v2.4.0.0r2 is not mounting my sd-ext at all, when I checkmark it manually, it vanishes after a short while.
Obviously, fix permissions don't work. Backups menu reports 0 size.
My log: http://ompldr.org/vaDk5aw/recovery.log
System setup:
Boot loader: Black cLK 1.5.1.6
Radio: 2.15.50.14
ROM: NexusHD2-JellyBean-1.3a
SD card: SanDisk class 4 30GB fat32 mmcblk0p1 and 2GB ext4 mmcblk0p2
Internal partitions: system 390M, user data auto, recovery 8M, rest cLK default.
TWRP v2.3.3.0rev7 was the last extended recovery that worked perfectly for me.
Thanks kokotas. =)
Sent with love from me to you via HTC HD2 with Tapatalk 2
Click to expand...
Click to collapse
Did you try to run "Check SD Card" from "Advanced" menu?
Re: [DEV][27/Jan/2013]Extended TWRP|v2.4.0.0.r2
Dark_Avenger said:
Did you try to run "Check SD Card" from "Advanced" menu?
Click to expand...
Click to collapse
Yeah that works fine.
The recovery doesn't seem to function properly with DataOnEXT enabled.
In normal mode, all seems OK.
Log:
http://ompldr.org/vaDlubQ
Sent with love from me to you via HTC HD2 with Tapatalk 2
Re: [DEV][27/Jan/2013]Extended TWRP|v2.4.0.0.r2
Did you set the path properly in DataOnExt page?
Sent from my HTC HD2 using Tapatalk 2
sunitknandi said:
Yeah that works fine.
The recovery doesn't seem to function properly with DataOnEXT enabled.
In normal mode, all seems OK.
Log:
http://ompldr.org/vaDlubQ
Sent with love from me to you via HTC HD2 with Tapatalk 2
Click to expand...
Click to collapse
same here.
sd-ext is not getting mounted if DataOnEXT option enabled.
mounting via shell workes
if DataOnEXT option disabled sd-ext gets mounted vie GUI
Hi guys,
I don't currently use a Rom with the DataOnExt feature, so I can't really test this.
I'll try one tomorrow in order to be able to see what's the problem.
[EDIT]I found a nasty loop in Mount() function which I removed in r4. Try to see if this one solves this problem.[/EDIT]
Also I'd suggest that you either manually delete the /sdcard/TWRP/.twrps file or select to "Restore Defaults" after updating TWRP.
I've seen some really strange issues (like 'buttons disappearing') if I don't remove the previous settings file.
At last, I have ported the latest commits from official TWRP and fixed a couple of things in the r2.
But I didn't check out the problem you reported. Nevertheless I will upload the compiled files to keep things moving...
Regards!
PS:@Dark_Avenger
Just a suggestion: Keep using the 5th post for any updates of your theme so that it will be found easier. I already linked that post in the 3rd one.
kokotas said:
Hi guys,
I don't currently use a Rom with the DataOnExt feature, so I can't really test this.
I'll try one tomorrow in order to be able to see what's the problem.
Also I'd suggest that you either manually delete the /sdcard/TWRP/.twrps file or select to "Restore Defaults" after updating TWRP.
I've seen some really strange issues (like 'buttons disappearing') if I don't remove the previous settings file.
At last, I have ported the latest commits from official TWRP and fixed a couple of things in the r2.
But I didn't check out the problem you reported. Nevertheless I will upload the compiled files to keep things moving...
Regards!
Click to expand...
Click to collapse
FYI
deleting settings file did not help
issue still present in .rev3
kokotas said:
Hi guys,
I don't currently use a Rom with the DataOnExt feature, so I can't really test this.
I'll try one tomorrow in order to be able to see what's the problem.
[EDIT]I found a nasty loop in Mount() function which I removed in r4. Try to see if this one solves this problem.[/EDIT]
Also I'd suggest that you either manually delete the /sdcard/TWRP/.twrps file or select to "Restore Defaults" after updating TWRP.
I've seen some really strange issues (like 'buttons disappearing') if I don't remove the previous settings file.
At last, I have ported the latest commits from official TWRP and fixed a couple of things in the r2.
But I didn't check out the problem you reported. Nevertheless I will upload the compiled files to keep things moving...
Regards!
PS:@Dark_Avenger
Just a suggestion: Keep using the 5th post for any updates of your theme so that it will be found easier. I already linked that post in the 3rd one.
Click to expand...
Click to collapse
kokotas,
thanks for suggestion. I will use the 5th post to keep the Holo Dark theme in sync with the current extended version.
Without having a DataOnExt rom, I think that the problem with mounting the "sd-ext" is still there. You could reproduce it without a DataOnExt rom. Check the "DataOnExt" checkbox in "Settings" and try to mount the "sd-ext" from "Mount" page. The checkbox is selected for a second and then immediately deselected.
[EDIT]Deleting the .twrp settings file fixed it.[/EDIT]
Best
Dark_Avenger said:
kokotas,
thanks for suggestion. I will use the 5th post to keep the Holo Dark theme in sync with the current extended version.
Without having a DataOnExt rom, I think that the problem with mounting the "sd-ext" is still there. You could reproduce it without a DataOnExt rom. Check the "DataOnExt" checkbox in "Settings" and try to mount the "sd-ext" from "Mount" page. The checkbox is selected for a second and then immediately deselected.
Best
Click to expand...
Click to collapse
for me sd-ext mounting issue is fixed in .rev4
ratelutz said:
for me sd-ext mounting issue is fixed in .rev4
Click to expand...
Click to collapse
Yes... I can't reproduce it too after deleting the settings file ".twrp".
Re: [DEV][29/Jan/2013]Extended TWRP|v2.4.0.0.r4
kokotas said:
HD2_Extended_TWRP_2.4.0.0_r4 @ 2013.01.29
Click to expand...
Click to collapse
Looks like I am getting to the party a little late, sorry. Will be catching up soon as time permits.
---
If at first you don't succeed, skydiving is probably not for you... (via Tapatalk)
Re: [DEV][29/Jan/2013]Extended TWRP|v2.4.0.0.r4
kokotas said:
HD2_Extended_TWRP_2.4.0.0_r4 @ 2013.01.29
Click to expand...
Click to collapse
It certainly is better and probably functional now! You know, I used to have 'fun' with circular links and here you go introducing circular mounts!
Okay, let's see. I had some odd things happen after I flashed r4. I did delete the structure and started clean.
Sd-ext was found immediately. However, the xml path was not. I tried to use file explorer to look at the partition but it came up blank.
I tried to calculate partition sizes and it sat there for a long time before returning to the twrp splash screen and restarting. No boot - never saw Magldr.
I loaded Aroma fm which worked fine.
Back to twrp and checked the card. All went fine.
Not sure what else I may have touched but all of the sudden the correct path was populated and verified.
Not enough time to do a full backup but it was very close.
Should you want the log it is on dev-host: recovery.log - 230.13 KB <= Deleted
---
If at first you don't succeed, skydiving is probably not for you... (via Tapatalk)
OK, this thread is going to be a work in progress, intended to serve as a reference for the work I've been doing on LVM partition remapping.
My work was done initially on a Find 7, but this should eventually be usable on many other devices (I have the Find 5 and N1 in mind for when I return from vacation). Also, this would not have been possible without the work Steven676 did years ago on the Nexus S, which has been used by all AOSP-derivative projects to support the Samsung Aries (Galaxy S) family for quite some time now.
The current state of things is that the patches are solid and work very well for the system side of things, but there is still a bit of work needed on the recovery side of things. This is due to TWRP having an architectural limitation I need to work on - Whether a device uses emulated storage or not is set at compile time, which is a problem if your design requires automatic detection of configuration at run time.
One of the key design goals here was to support both normal and LVM configurations automatically with a single build that detects which configuration is present on a device at run time.
A second key design goal was that the underlying partition table of the device is not touched in any way. Touching the partition table of a mobile device in the field is a fundamentally dangerous operation, as many partitions contain data that is device-unique or will render a device unbootable if altered. Recovery methods that involve DDing partition images to nonstandard partitions is asking for trouble due to typos... There's no protection against a user typoing the name of a critical partition.
Initially, I'm going to dump the contents of an email I wrote to someone giving them documentation on how to integrate LVM into their project. Over time I'll clean up and reorganize this post, including adding some more links. Also, since this email was written, I've added a LOT of comments to each patch explaining what is going on.
For additional documentation, especially a more user-oriented view of things (such as how to set this up if you want to use it with Omni nightlies) - see the Omni nightlies thread on XDA.
So here goes:
How it's implemented - the complete patch set is at:
https://gerrit.omnirom.org/#/q/topic:find7_lvm - Expect this to periodically change as work on this feature continues (Note: All patches required to support nightly builds of Omni have been merged. At this point, all remaining work that I expect is on polishing up TWRP.)
With the rest of this post, I'll talk about each individual patch and what it does.
https://gerrit.omnirom.org/#/c/9273/ - This is a patch against frameworks/base which adds an alternative to storage_list.xml called storage_list_lvm.xml - The frameworks will choose storage_list_lvm.xml instead of storage_list.xml if the property ro.lvm_storage is set to 1 - The device init scripts will set this property if they detect an LVM configuration.
https://gerrit.omnirom.org/#/c/9207/ - This is an Omni-specific patch. Omni builds for both the Find 7 and OnePlus One (also known as find7op) and both share a common device tree. The LVM patches do not apply to the find7op, so we move init.recovery.rc out of the msm8974-common tree - You likely don't have to worry about this unless you also have a -common tree for find7 and find7op
https://gerrit.omnirom.org/#/c/9276/ - Normal Android kernel ramdisks do not include busybox or any form of shell, making it impossible to run shell scripts without /system mounted. Since we need to run a shell script prior to mounting partitions, we need to add busybox to the ramdisk. This patch does that. For legal reasons you may wish to replace busybox with system/core/toolbox and system/core/sh - I have not tried doing so. If you choose to stay with busybox, you will have to provide the busybox source code in order to comply with the GPL.
https://gerrit.omnirom.org/#/c/9205/ - This adds the LVM binary and LVM configuration file to the ramdisks of both normal boot and recovery. This patch does not actually begin doing anything with the binaries, I separated it out from the other patches as a way to keep things organized so I could start working with the binaries when I began this project. The original source code and documentation for the binary is at https://github.com/steven676/android-lvm-mod
One change I made in lvm.conf that differs from the Samsung aries family (galaxysmtd, fascinatemtd, captivatemtd, etc.) is that I changed the filter line to only allow the userdata and sdcard partitions. This prevents LVM's vgscan from accidentally determining another partition is a physical volume, and also prevents users from accidentally running pvcreate on a critical partition.
https://gerrit.omnirom.org/#/c/9206/ - This is where all of the "heavy lifting" is done. I'm going to work on adding more comments to the init scripts and shell scripts to document them tonight and tomorrow, but I'll try to explain things here.
Android's init system is a bit limited in that it's very difficult to have conditional behavior defined in init.rc - which appears to be why Qualcomm loves to use shell scripts called from init. Similarly, much of the LVM magic happens in three shell scripts (which execute at three different phases within the boot sequence).
In the early-init phase, the two "wait" blocks ensure that the underlying block devices are ready before vgscan/vgchange are called. This will probably slow down booting by a few fractions of a second unfortunately.
vgscan will scan the volumes defined in lvm.conf (in this case, only the userdata and sdcard partitions) for LVM physical volumes. If LVM physical volumes are detected and form a proper volume group, vgscan will create appropriate device nodes. With the configuration I'm using, the device node will be /dev/lvpool/userdata - which consists of a single logical volume that merges the sdcard and physical userdata physical volumes (partitions). The configuration of lvm.conf prevents LVM commands (especially pvcreate) from altering partitions we don't want to alter. If someone accidentally tries to, for example, run pvcreate on the system partition, it will give an error indicating that the partition was not part of the filter.
vgchange will activate the physical volumes detected by vgscan
lvm_init.sh will check to see if /dev/lvpool/userdata exists, and copy fstab.qcom.lvm to fstab.qcom, init.fs.rc.lvm to init.fs.rc, and twrp.fstab.lvm to twrp.fstab if it does. If it does not, it selects fstab.qcom.std, etc.
In the "on init" section, the init script exports all environment variables from init.fs.rc, and creates all storage-related directories and symlinks needed for both configurations (except for when they conflict). lvm_symlinks.sh will create directories/symlinks that must be configuration-specific. Just like lvm_init.sh - it decides what to do based on whether /dev/lvpool/userdata exists
In the "on fs" section - we do an SELinux restorecon on /dev/mapper/lvpool-userdata (/dev/lvpool/userdata would probably work here too). If it doesn't exist, this will fail gracefully without causing any issues.
In "on early-boot" - lvm_setprop.sh uses /system/bin/setprop to set ro.lvm_storage to 0 or 1 depending on the detected configuration. The property service is not available until early-boot - so this cannot be in lvm_init.sh or lvm_symlinks.sh This propery is used by the frameworks/base patch above to determine which storage_list to choose.
At the end of the init.qcom.rc, the fuse daemon for emulated storage is added for all configurations. (I could not figure out a good way to make this conditional based on whether LVM was present or not). In a non-LVM configuration, it runs but is harmless - it maps /data/media (which is empty) to /mnt/shell/emulated (which nothing is looking at due to the environment variables and symlinks set in the "on init" section )
You will probably notice that Omni's standard storage configuration is fairly different from ColorOS - this is due to the way KitKat storage works, but it allowed us to get away without using Oppo's ext4 permissions hacks in our kernel (by remapping permissions instead, in a manner similar to how the emulated storage system works) The way we handle our /sdcard partition does interoperate without issues with the ColorOS approach.
https://gerrit.omnirom.org/#/c/9279/ is a patch specifically for TWRP. TWRP currently determines whether to use emulated storage (/sdcard on /data/media) at build time instead of at run time. Until I have time to fix this, the patch here operates as a workaround. It is similar to the behavior of the fuse sdcard daemon in the previous patch - it maps /data/media to /sdcard whether the configuration is actually emulated storage or not. If the device is not using emulated storage (LVM), mapping of /data/media to /sdcard is still mostly harmless. However it does result in undesirable changes to TWRP's user interface. DO NOT USE THIS APPROACH IN PRODUCTION RELEASES. It's a horrible hack. You'll need to figure out how to properly do /data/media handling depending on whether LVM is present or not based on how your own recovery architecture works.
https://gerrit.omnirom.org/#/c/9281/ adds "raw" sdcard and userdata partition entries to the partition table for the LVM configuration. This allows a user to return their device to a standard configuration by formatting the underlying sdcard and userdata partitions directly, instead of the removelvm ZIP at the beginning of this email. - To be abandoned, this patch was squashed into 9206
FAQ
Q: Coldbird already had repartitioning support. Why did you create this different approach?
A: Even before he started work, I strongly recommended that he not touch the partition table of the device. It's a really bad idea and is fundamentally dangerous. It's pure luck that someone hasn't hardbricked yet. (A number of people have come close.) If you read through his thread and the ColorOS 2.0.2 thread, you'll see that the repartitioning approach fails frequently, and in multiple ways. (Missing partition contents, partition table ending early, etc. The latter is really scary, one person had the process fail at mmcblk0p19 - what if someone else's partition table write operation aborted even earlier?.) Also, nearly everyone that has implemented support for that approach has needed a separate build to support it. (Oppo is the first to manage autodetection.) I also provided him all of the reference information from Steven676's work.
LVM is far safer. The underlying partition table is not touched in any way. Instead, LVM remaps sectors on the fly so that two partitions that are not adjacent to each other on the physical storage appear as a single contiguous partition to the filesystem drivers. Linux has supported LVM for on the order of a decade, if not more. I've been using LVM on my file server since 2006. (Yes, the system is 8 years old and still working other than needing a new power supply after a thunderstorm. Nothing to do with LVM. ) In addition, the lvm.conf configuration used here provides protection against accidental typos causing damage. Undoing the changes is as simple as doing a wipe of /data and /sdcard from any standard recovery and can be done in seconds, not of running a special batch file that runs a bunch of fastboot commands and takes 4-5 minutes. Similarly, the LVM setup process currently described in the Omni thread involves flashing a single ZIP from recovery that takes only 10-15 seconds, and most of that process is flashing an LVM-aware recovery. (The only limitation currently is that the ZIP must be on external storage - USB OTG or MicroSD)
To put it simply, it Just Works. No need to back up a pile of partitions other than /data and /sdcard because those partitions are never touched or altered.
Q: I have a device with a ridiculously oversized /system partition, can I get some of that back for /data?
A: Yes, you can. Add the physical /system partition to the lvm.conf filters and add it to the lvpool when creating it, then create a smaller /system LV out of this big pool. (see updater.sh in device/samsung/aries-common/ of any AOSP-derivative for hints here.) Be careful though - leave enough spare space for growth (new Android versions, etc.) While it should be possible to use some of the LVM tools along with ext4 resize tools to reorganize the LVs without wiping, this is very difficult and you'll probably have to make users wipe /data if you want to alter /system.
*reserved 2*
Nice work, I hope all the patches can be widely used on some other devices and other roms.
systop said:
Nice work, I hope all the patches can be widely used on some other devices and other roms.
Click to expand...
Click to collapse
Yup. I know Andre from PA was working on it last week but I haven't heard from him lately.
My priority when I return from vacation will be fixing up the TWRP side of things. It's working for now, but the user interface on non-LVM configs is a little funky thanks to RECOVERY_SDCARD_ON_DATA being compile time. This has never been a problem before since a single TWRP binary never had to support two different configurations before. I plan on either doing a property-based approach or fstab-based like CWM. (It should be possible for someone to make a CWM build that automatically detects configuration without any modifications to CWM, based on reading the code - but I haven't tried it myself.)
Once TWRP is in better shape, I plan on doing the Find 5 and N1. These will have the challenge of not having a MicroSD slot, so I may have to change TWRP so that it use /tmp instead of /sdcard when doing "adb sideload", or at least gives the user that option.
Good stuff :good: I don't really need it as of yet, but when my new device is provided (warranty) I will surely give this a try.
I hope ayysir will merge the LVM support very soon ^^
Find 7u PA 4.6 beta 1
Awesome work mate. I have avoided other methods because I'm always the guy that will have a device fail at very bad timing; like during boatloader or SBL stage.
I'm really glad you have continued to work on this. I have hit thanks a few times but would also like to thank you here
tork987 said:
I hope ayysir will merge the LVM support very soon ^^
Find 7u PA 4.6 beta 1
Click to expand...
Click to collapse
He had issues with merging support, hopefully now that I've added more documentation he can try again.
how are the *.std files created?
atm this is tough for me to port from omni to cm base which AOSPA Oppo trees
ayysir said:
how are the *.std files created?
atm this is tough for me to port from omni to cm base which AOSPA Oppo trees
Click to expand...
Click to collapse
the std files are also part of the device tree
https://github.com/omnirom/android_device_oppo_find7/tree/android-4.4/configs
ayysir said:
how are the *.std files created?
atm this is tough for me to port from omni to cm base which AOSPA Oppo trees
Click to expand...
Click to collapse
For the fstabs - they are simply moves/renames of the fstab files and other storage-related items from the standard Oppo configuration (they should appear as renames/moves in the Gerrit commit...)
For the init.fs.rc file - all of the "export <blah>_STORAGE" lines from init.qcom.rc/init.find7.rc are cut out of the RC file and put into .std
Obviously, the .lvm versions of the files are the ones where the fstab has been altered to support a single data partition with emulated storage.
Amazing work and amazing posts. Thanks a lot for your sharing. ?
I've got a question related to your configuration (/data and /sdcard merged) : are the LV hot-resizables?
Wendigogo said:
Amazing work and amazing posts. Thanks a lot for your sharing. ?
I've got a question related to your configuration (/data and /sdcard merged) : are the LV hot-resizables?
Click to expand...
Click to collapse
In theory, you could probably use some of the ext4 resizing tools to do something like this, but I haven't looked into it as there isn't much point in the current config (since the LVM userdata volume is allocated to use all space on the volume group).
Something like that might be more useful if someone ever uses LVM to regain some of the wasted /system partition space on certain excessively bloated devices (like some GS4 units).
Entropy512 said:
In theory, you could probably use some of the ext4 resizing tools to do something like this, but I haven't looked into it as there isn't much point in the current config (since the LVM userdata volume is allocated to use all space on the volume group).
Something like that might be more useful if someone ever uses LVM to regain some of the wasted /system partition space on certain excessively bloated devices (like some GS4 units).
Click to expand...
Click to collapse
Thanks for your answer.
Seems I misunderstood the way it's implemented here. All space is allocated to /data? So there's no more internal sdcard right?
But in that case an external sdcard is mandatory. How is it managed when there's no sdcard?
Enjoy!
Wendigogo said:
Thanks for your answer.
Seems I misunderstood the way it's implemented here. All space is allocated to /data? So there's no more internal sdcard right?
But in that case an external sdcard is mandatory. How is it managed when there's no sdcard?
Enjoy!
Click to expand...
Click to collapse
Android has supported emulated storage (where /data/media is mapped to /sdcard with a special FUSE daemon that makes /sdcard have DOS-like permissions despite an underlying ext4 partition) since ICS. It's pretty much the standard in all new devices - the Find 7 is to my knowledge the only device launched in 2014 not to use emulated storage. Most devices in 2013 also did - Oppos were again the rare exception.
As I understand it - for some reason Chinese users prefer the legacy pre-ICS partitioning scheme. My guess is due to UMS vs. MTP - MTP is required for access to emulated storage, UMS can't be used, but a lot of older desktop OSes have issues with MTP. So Oppo finds themselves in conflict between their home market (China) and expanding in the West. That said, the Find 7 was kind of a screwup in achieving this goal, since the internal sdcard partition was ext4 which meant UMS was a no-go for it.
Entropy512 said:
Android has supported emulated storage (where /data/media is mapped to /sdcard with a special FUSE daemon that makes /sdcard have DOS-like permissions despite an underlying ext4 partition) since ICS. It's pretty much the standard in all new devices - the Find 7 is to my knowledge the only device launched in 2014 not to use emulated storage. Most devices in 2013 also did - Oppos were again the rare exception.
As I understand it - for some reason Chinese users prefer the legacy pre-ICS partitioning scheme. My guess is due to UMS vs. MTP - MTP is required for access to emulated storage, UMS can't be used, but a lot of older desktop OSes have issues with MTP. So Oppo finds themselves in conflict between their home market (China) and expanding in the West. That said, the Find 7 was kind of a screwup in achieving this goal, since the internal sdcard partition was ext4 which meant UMS was a no-go for it.
Click to expand...
Click to collapse
I've got it now. Thanks for your explanations
I saw that Oppo phones didn't follow Android guidelines (yet?) by not using the emulated_storage mounting method but I didn't know why.
And your right, mtp doesn't work in Windows XP (or is hard to make working) and there's a lot of Asian people still using it. Obvious once you said it...
And that's also why only external sdcard is accessible in UMS mode in recovery.
Thanks again for your enlightenment. ?
Reading some of the comments on G+ it looks like Oppo might be using this solution for their KitKat release. I would be so pleased if they did.
Sent from my X9076 using Tapatalk
kishd said:
Reading some of the comments on G+ it looks like Oppo might be using this solution for their KitKat release. I would be so pleased if they did.
Sent from my X9076 using Tapatalk
Click to expand...
Click to collapse
You could be pleased...
Wendigogo said:
You could be pleased...
Click to expand...
Click to collapse
Had some problems with camera focus on earlier versions of omnirom for find 7. Now those have been addressed. I installed Omni and am on the nightlies with lvm. My find 7 and find 7a will not see another rom again.