I have written a updater-script to do this, but I am not sure whether it works. Can anybody give me some suggestion?
Code:
ui_print("Changing Moto Rom");
ui_print("Updating boot");
show_progress(0.330000,2);
assert(package_extract_file("boot.img", "/tmp/boot.img"),
write_raw_image("/tmp/boot.img", "/dev/block/mmcblk1p15"),
delete("/tmp/boot.img"));
ui_print("Updating devtree");
show_progress(0.330000,2);
assert(package_extract_file("devtree.img", "/tmp/devtree.img"),
write_raw_image("/tmp/devtree.img", "/dev/block/mmcblk1p12"),
delete("/tmp/devtree.img"));
ui_print("Updating stock recovery");
show_progress(0.340000,2);
assert(package_extract_file("recovery.img", "/tmp/recovery.img"),
write_raw_image("/tmp/recovery.img", "/dev/block/mmcblk1p16"),
delete("/tmp/recovery.img"));
ui_print("Changed");
And are the three img files generated only by change names of CG35.smg/ CG61.smg/ CG47.smg?
After we extract the smg files from a sbf file, we have many CG??.smg.
CG35.smg refers to the partition of boot.
CG61.smg refers to the partition of devtree.
CG47.smg refers to the partition of recovery.
Now I have two problems.
1. Does the boot&devtree&recovery in zip file have the same format as CG35&CG61&CG47? Such as the yaffs or ext?
2. Shall we use the code above to replace the partitions successfully considering the not-open bootloader and the secure signature?
i will try!
CG35.smg and CG47.smg are different. CG47 is Recovery and contains recovery software does not boot system, CG35 is boot and does not have ny recovery stuff only boots the system.
But I don't know anything about devtree and devtree_backup. but the name indicate that devtree_backup is a exact copy of devtree.
EDIT: CG35 and CG47 have different SHA1 hash and different signatures when extracted from the same (original) SBF file. if CG47 would have been a copy of CG35 the SHA1 hash would have been the same.
Febernacke said:
CG35.smg and CG47.smg are different. CG47 is Recovery and contains recovery software does not boot system, CG35 is boot and does not have ny recovery stuff only boots the system.
But I don't know anything about devtree and devtree_backup. but the name indicate that devtree_backup is a exact copy of devtree.
EDIT: CG35 and CG47 have different SHA1 hash and different signatures when extracted from the same (original) SBF file. if CG47 would have been a copy of CG35 the SHA1 hash would have been the same.
Click to expand...
Click to collapse
I am afraid you misunderstood me. I know that CG35 and CG47 are different.
After we extract the smg files from a sbf file, we have many CG??.smg.
CG35.smg refers to the partition of boot.
CG61.smg refers to the partition of devtree.
CG47.smg refers to the partition of recovery.
Now I have two problems.
1. Does the boot&devtree&recovery in zip file have the same format as CG35&CG61&CG47? Such as the yaffs or ext?
2. Shall we use the code above to replace the partitions successfully considering the not-open bootloader and the secure signature?
iaio72 said:
i will try!
Click to expand...
Click to collapse
I am waiting for your good news!
Just a warning guys, those CGXX.smg files are not the same as the .img files. SBF files pad the partitions with nulls so they reach multiples of 1024 bytes. The image files themselves can be extracted from the .smg files by removing the extra padding nulls from the end. Hex editors are your friend.
Related
This tool can be used to either decompile or compile ROMs for various Asus devices that used the ABI firmware format (Can also be used in O2 XDA Zest ). The current version can support P835 unencrypted ABI, and even encrypted ABI from updater EXE! Current finding is this tool also supports unreleased Garmin-Asus ROMs!
Thanks (Especially )Harshal and Leon in AsusPda for testing.
Usage:
- Decompiling ROM
1. p835abisplit2 <abi/exe file>
2. os.nb0 and extrom.img released (Only for Pre-P835 devices), os.nb0 cab be processed by imgfsfromnb or osnbtool, extrom.img can be processed by WinImage.
- Compiling ROM
1. First rename the new os.nb0 to os-new.nb0, and rename extrom.img to extrom-new.img (If the new files do not exist, then the compiler will use the parts from original ROM)
2. p835abisplit2 /b <abi/exe file>
3. out.abi releases, which can be used to flash directly (Only for Pre-P835 devices or Post-P835 devices with unencrypted ROM).
4. If you input the updater EXE to p835abisplit2, it will also produce out.exe with region locked patched which can be used to flash your new ROM directly on devices with any region!
- Note when building ROM
1. If you need to modify XIP, make sure the modded XIP is the same size as the original one before merging back to nb0, otherwise booting will fail
2. For Pre-P835 devices, current version can create big-storage ROMs with variable size of imgfs. If the new OS is smaller than the original one, the freed space will be allocated to user space (The left part as shown in Memory setting) after flashing. However the user space display will only reflect the change on second flash.
3. For Post-P835 devices, all partitions must be exactly same size with the original one (i.e. you need to pad the partition before putting it back), otherwise the device won't boot.
4. For Pre-P835 devices, you can modify ExtROM as you like, but not remove or rebuild the image file, otherwise you may get a brick! (Not able to enter bootloader)
Final Warning: Customizing a ROM always has risks, I won't be responsible for any damages lead to your custom ROM!
Release Notes:
v2.40:
- Added support for M930
v2.32:
- Support extraction of encrypted ABI file resource from P835 updater exe
- Support reconstruction of P835 updater exe
- When rebuild to exe, the produced out.exe is patched to install in devices of any region.
V2.20:
- Support extration and rebuilding of P835 ABI file (Note that not for ABI inside EXE)
- When rebuild with exe, OUT.EXE will be produced for direct flashing
starkwong said:
This tool can be used to either decompile or compile ROMs for various Asus devices that used the ABI firmware format (Can also be used in O2 XDA Zest ).
Thanks Harshal and Leon in AsusPda for testing.
Usage:
- Decompiling ROM
1. p565abisplit2 <abi/exe file>
2. os.nb0 and extrom.img released, os.nb0 cab be processed by imgfsfromnb or osnbtool, extrom.img can be processed by WinImage.
- Compiling ROM
1. First rename the new os.nb0 to os-new.nb0, and rename extrom.img to extrom-new.img (If the new files do not exist, then the compiler will use the parts from original ROM)
2. p565abisplit2 /b <abi/exe file>
3. out.abi releases, which can be used to flash directly.
- Note when building ROM
1. If you need to modify XIP, make sure the modded XIP is the same size as the original one before merging back to nb0, otherwise booting will fail
2. OS part doesn't need to be the same size as the original. If the new OS is smaller than the original one, the freed space will be allocated to user space (The left part as shown in Memory setting) after flashing. However the user space display will only reflect the change on second flash.
3. You can modify ExtROM as you like, but not remove or rebuild the image file, otherwise you may get a brick! (Not able to enter bootloader)
Final Warning: Customizing a ROM always has risks, I won't be responsible for any damages lead to your custom ROM!. Moreover, don't use it in P835 abi, it won't work
Click to expand...
Click to collapse
Nice we all waiting for it
Many Congrats for successfully ripping through the ROM !!
Partition offsets and checksums reported by the tool are :
Part #0002 sz:0003e000=>0003e000 cs:625f94b2=>625f94b2 of:000003a0=>000003a0
Part #0004 sz:00100000=>00100000 cs:ffe5f731=>ffe5f731 of:0003e3a0=>0003e3a0
Part #0103 sz:00452a8c=>00452a8c cs:99f65978=>99f65978 of:0013e3a0=>0013e3a0
Part #0104 sz:000fffc0=>000fffc0 cs:5e03943f=>5e03943f of:00590e2c=>00590e2c
Part #0005 sz:07e00000=>07e00000 cs:674f4072=>674f4072 of:00690dec=>00690dec
Part #0013 sz:00a00000=>00a00000 cs:56994bf9=>56994bf9 of:08490dec=>08490dec
I am a little scared to use this tool for following reasons :
1. Actually, the IMGFS & ExtROM offsets are '00690dfc' & '08490dfc' respectively.
2. Checksums( 674f4072, 56994bf9....) can not be located in the Header.
3. The Adler32 checksum for the ExtROM is '5ff94bfa', while your tool reports '56994bf9'.
Any clues ?
rishi2504 said:
Many Congrats for successfully ripping through the ROM !!
Partition offsets and checksums reported by the tool are :
Part #0002 sz:0003e000=>0003e000 cs:625f94b2=>625f94b2 of:000003a0=>000003a0
Part #0004 sz:00100000=>00100000 cs:ffe5f731=>ffe5f731 of:0003e3a0=>0003e3a0
Part #0103 sz:00452a8c=>00452a8c cs:99f65978=>99f65978 of:0013e3a0=>0013e3a0
Part #0104 sz:000fffc0=>000fffc0 cs:5e03943f=>5e03943f of:00590e2c=>00590e2c
Part #0005 sz:07e00000=>07e00000 cs:674f4072=>674f4072 of:00690dec=>00690dec
Part #0013 sz:00a00000=>00a00000 cs:56994bf9=>56994bf9 of:08490dec=>08490dec
I am a little scared to use this tool for following reasons :
1. Actually, the IMGFS & ExtROM offsets are '00690dfc' & '08490dfc' respectively.
2. Checksums( 674f4072, 56994bf9....) can not be located in the Header.
3. The Adler32 checksum for the ExtROM is '5ff94bfa', while your tool reports '56994bf9'.
Any clues ?
Click to expand...
Click to collapse
Tool works properrly.No harm is trying
starkwong, is there any tool to decompile P835's ROM in the same way?
New version posted.
rishi2504:
The image checksum is not calculated by plain Adler32, actually is uses the same formula as older Asus ROMs, however it is not a one-time calculation.
Checksums are inside header, given you decoded it correctly.
starkwong, here's what I get when trying to decompile a ROM:
Code:
v2.40 (Jun 16 2009 19:56:06)
ExtractABI(): Trying to load G5_ALL_V4.11.0_V3.6.12.P2_Ship_WWE_app_MYS00_V2.3.6.exe...
Module loaded, searching for BIN resource...
Found matching resource at BIN #211!
GetPartitions(): Getting Partition Information...
*** Encrypted ABI detected
ABI Version 0x00030012
Project Name: G5
Partition Type: 000f [email protected]
Partition Type: 000e [email protected]
Partition Type: 000e [email protected]
Partition Type: 0004 [email protected]
Partition Type: 0004 [email protected]
Partition Type: 000f [email protected]
Partition Type: 000f [email protected]
Partition Type: 0102 damage [email protected]
Partition Type: 0005 UnKnown [email protected]
Partition Type: 0002 [email protected]
Partition Type: 0003 [email protected]
ProcessABI(): Writing OS data...
* BIN(P835) Image Detected
Warning: OS.nb0/flash.bin is not a NB image, not modifying MSFLSH50 headers
ProcessABI(): No ExtROM partition found.
OK!
So it seems that partitions are not detected correctly, there's no os.nb0 at the output, and the flash.bin apperars to be of no use. Even when I found imgfs partition inside of it, there's still something wrong with it, e.g. bad start block offset, and everything else is also broken.
Can you help me with this?
In fact it is correct, as Asus uses B000FF image on P835, not a plain NB0 image.
You need to use osnbtool to get a nb0 with extra bytes, then use nbsplit -data 2048 -extra 8 to get a nb0 with sector size 0x800.
ginkage said:
starkwong, here's what I get when trying to decompile a ROM:
Code:
v2.40 (Jun 16 2009 19:56:06)
ExtractABI(): Trying to load G5_ALL_V4.11.0_V3.6.12.P2_Ship_WWE_app_MYS00_V2.3.6.exe...
Module loaded, searching for BIN resource...
Found matching resource at BIN #211!
GetPartitions(): Getting Partition Information...
*** Encrypted ABI detected
ABI Version 0x00030012
Project Name: G5
Partition Type: 000f [email protected]
Partition Type: 000e [email protected]
Partition Type: 000e [email protected]
Partition Type: 0004 [email protected]
Partition Type: 0004 [email protected]
Partition Type: 000f [email protected]
Partition Type: 000f [email protected]
Partition Type: 0102 damage [email protected]
Partition Type: 0005 UnKnown [email protected]
Partition Type: 0002 [email protected]
Partition Type: 0003 [email protected]
ProcessABI(): Writing OS data...
* BIN(P835) Image Detected
Warning: OS.nb0/flash.bin is not a NB image, not modifying MSFLSH50 headers
ProcessABI(): No ExtROM partition found.
OK!
So it seems that partitions are not detected correctly, there's no os.nb0 at the output, and the flash.bin apperars to be of no use. Even when I found imgfs partition inside of it, there's still something wrong with it, e.g. bad start block offset, and everything else is also broken.
Can you help me with this?
Click to expand...
Click to collapse
B000FF image cannot be modified without osnbtool or viewbin or cvrtbin tool..For Ext ROM there is no partition in the abi file which can b read as .nb0 Ext ROM is inside the OS and is not as a partition.So what the output u get from the tool is correct.Whatever ROMs u saw mine are from using the same tool
I hope this clarifies.
starkwong, Thank you so much, it worked perfectly!
Can't use this tool on my Asus M530w. Getting this message:
Copyright(C) 2009 Studio KUMA(starkwong). All rights reserved
v2.40 (Jun 16 2009 19:56:06)
ExtractABI(): Trying to load nk.abi...
Failed loading as module (193). Perhaps ABI?
Trying as ABI directly...
Creating file mapping...
GetPartitions(): Getting Partition Information...
Error: AES Key not suitable for this ROM
unencrypted vers encrypted
There is little bit mess in description.
If you will use unecrypted rom + p835abisplit2 you will get os.nb0.
With encrypted rom + p835abisplit2 you will get flash.bin.
starkwong said:
4. For Pre-P835 devices, you can modify ExtROM as you like, but not remove or rebuild the image file, otherwise you may get a brick! (Not able to enter bootloader)
Click to expand...
Click to collapse
Hi,
Can i add my own cab/xml files to this Ext_ROM, after removing the files not needed ?
rishi2504 said:
Hi,
Can i add my own cab/xml files to this Ext_ROM, after removing the files not needed ?
Click to expand...
Click to collapse
Don't bother dude , figured it out ...
Is there anyway I can extract the flash.bin file from a .abi file using this tool?
My P835 is unable to upgrade from the SD card with the .abi file on it.. so I wanted to extract the flash.bin and see if I can use the QPST tool to update the image with flash.bin file..
Sorry if I'm being stupid here (not unusual!). I'm trying to use this tool to get a .abi file out of O2's "Xda_Zest Firmware Update_V7.7.0S.WWE20.00_M4.6.5.P7_V2.1.4 GBR20.exe" so I can stick it on the SD card and flash the ROM, but of course I'm only ending up with the two files you mention, os.nb0 and extrom.img, no .ABI file. I can look in the extrom.img file with winrar, but the file only contains FINDMA~1.000 (300 bytes), 000dummy.001 (0 bytes) and _setup.xml (1205 bytes).
Plainly I'm thick - where am I going wrong, and how do I get the .ABI file?
so...i'm behide the curve on on this hero stuff (being a rom cook for the s200 i'm not complete clueless...but thats winmo)
Anyway i'm playing around and would love an answer on these from one of the resident dev's on here
1. i took boot.img from the RUU rom.zip of wwe 2.73.405.61, took the system.img, did the magic to create an update.zip, which seems to flash fine from a recovery rom BUT on reboot it's stuck on the Hero boot screen/logo, any clue as to where to look/debug (yes I did the wipe etc. etc. it must be something about the update.zip / boot.img i'm missing)
2. i assume there is still no way to re-sign the a changed RUU rom.zip ?
Thanks for your time !
1: extract the system with unyaffs
2: zip the contents of the extracted system
3: sign the resulting zip (androsign or whatever)
boot into recovery
4: copy over to sdcard
5: copy over boot.img also to sdcard
6: cat /dev/zero > /dev/mtd/mtd2
or
dd if=/dev/zero of=/dev/mtd/mtd2
whichever you prefer
7: flash_image boot /sdcard/boot.img
8: flash your update.zip or whatever you called it
9: wipe system (format DATA: & CACHE
OR
flash_image system system.img (might not work)
adwinp said:
1: extract the system with unyaffs
2: zip the contents of the extracted system
3: sign the resulting zip (androsign or whatever)
boot into recovery
4: copy over to sdcard
5: copy over boot.img also to sdcard
6: cat /dev/zero > /dev/mtd/mtd2
or
dd if=/dev/zero of=/dev/mtd/mtd2
whichever you prefer
7: flash_image boot /sdcard/boot.img
8: flash your update.zip or whatever you called it
9: wipe system (format DATA: & CACHE
OR
flash_image system system.img (might not work)
Click to expand...
Click to collapse
Thanks, i got that far, but rolling the whole thing (so boot.img + extracted system.img) into update.zip and flashing that from a recovery image is where it goes all tits up and doesn't boot, even tried without any changes and just taking the boot.img and system.img from a offical htc rom and creating an update.zip
so i must be doing something wrong but can't put my finger where it goes wrong.
anycase, i'll keep trying
thx
1: Did you sign your update.zip?
2: An update.zip usually contains an update script; did you write one too?
This is why I suggested you flash manually.
adwinp said:
1: Did you sign your update.zip?
2: An update.zip usually contains an update script; did you write one too?
This is why I suggested you flash manually.
Click to expand...
Click to collapse
both answers yes
still digging, but it could be that "something" in the update script isn't working, however i've copied one from a working rom, with the same results...
going through the script now....i'm sure it's something stupid and i'll smack my head later, but for now no go......thanks for your tips and time !
Edit:
is there something special you need to do with the boot.img if you just copy (unzip thats it) from a rom.zip and put it in the update.zip ??
Jesterz said:
Edit:
is there something special you need to do with the boot.img if you just copy (unzip thats it) from a rom.zip and put it in the update.zip ??
Click to expand...
Click to collapse
Either you push it to sdcard and
#flash_image boot /sdcard/boot.img
or
in the update.zip:
META-INF\com\google\android\update-script
where META-INF is a folder in the root, and update-script is a text file without file extension; its contents should be:
show_progress 0.1 0
write_raw_image PACKAGE:boot.img BOOT:
show_progress 0.1 10
So, the contents should be:
\META-INF\
com\
google\
android\update-script
\boot.img
zip all that into update.zip, and sign the zip.
today i tried restoring a stock rom through cwm which was installed through odin.. while restoring it i got an error saying there is no MD5 files! So can i put the md5 files from other backups in this??
nope...you can't do that. what CWM was you used before? you should note that skin1980's or madteam's is work just fine. the one from merruk's kernel doesn't work. it still under development stage. please be wary of that
Just create an md5sums.txt using notepad++. And generate the md5sums yourself:
1. Download the md5sums-1.2.zip here. Extract md5sums.exe.
2. To generate md5 checksums for your .tar files, run md5sums.exe -u your_nandroid_backup.tar >> md5sums.txt
3. Edit the md5sums.txt in notepad++. Replace each asterisk (*) beside each filename with a space.
4. Save your file. Ensure it is encoded in utf8.
Your md5sums.txt should contain the following:
Code:
7e613b2d2d799aecad4d76931a80928b cache.rfs.tar
ff68ff227b91bb4ab5006df682d31d5f data.rfs.tar
02da391f4dfb511ffb69b7188174b3ee system.rfs.tar
7c294a7d02fcfb81ef79b258d02da705
.android_secure.vfat.tar
Source:
Bumslayer.
raolemo said:
Just create an md5sums.txt using notepad++. And generate the md5sums yourself:
1. Download the md5sums-1.2.zip here. Extract md5sums.exe.
2. To generate md5 checksums for your .tar files, run md5sums.exe -u your_nandroid_backup.tar >> md5sums.txt
3. Edit the md5sums.txt in notepad++. Replace each asterisk (*) beside each filename with a space.
4. Save your file. Ensure it is encoded in utf8.
Your md5sums.txt should contain the following:
Code:
7e613b2d2d799aecad4d76931a80928b cache.rfs.tar
ff68ff227b91bb4ab5006df682d31d5f data.rfs.tar
02da391f4dfb511ffb69b7188174b3ee system.rfs.tar
7c294a7d02fcfb81ef79b258d02da705
.android_secure.vfat.tar
Source:
Bumslayer.
Click to expand...
Click to collapse
cn u explain step 2?
Today i had the same problem on TermaRom with CWM restore failed MD5 missing.Greeeeetz
Hi,
As you probably know, ext4 image can be extracted from system.sin but cannot be mounted. When trying to mount it, it fails with :
[ 1476.821582] EXT4-fs (loop0): bad geometry: block count 262144 exceeds size of device (144631 blocks)
I open this thread just to share what I did around the issue and maybe have some helpful quotes about it
Here is what I did (under linux)
# First create an zero filled file. Size is system partition size (262144 blocks of 4096 each)
dd if=/dev/zero of=/home/xperia/virtualfs bs=4096 count=262144
# Attach file to loopback
sudo losetup /dev/loop0 /home/xperia/virtualfs
# Format it with same features as system partition on phone
sudo mkfs.ext4 -O has_journal,^ext_attr,^dir_index,^flex_bg,^huge_file,resize_inode,filetype,extent,sparse_super,large_file,^uninit_bg,^dir_nlink,^extra_isize -v /dev/loop0
# Write extracted system.sin.ext4 extracted image to loopback
sudo dd if=system.sin.ext4 of=/dev/loop0
# Mount filesystem
sudo mount /dev/loop0 /mnt
It can be mounted and I can have folder structure but I can't work with files. Editing default.prop gives me a non readable file.
But we can go a step ahead as we can now mount the image.
Still some issues have to be worked out.
The poit is, why we cant mount system.img on ICS but we can on GB?
maybe someone can contact with sony t oask
im extracting the .sin to .img like always but its impossible to mount.. what are you using to extract the .sin to a .ext4?
BTW, thanks for the info, i've been trying to modify system.img since ICS appeared.
EDIT: of, ext4 can be extracte with flashtool ~.~
maybe we need to read something from system.partinfo
Yakandu said:
The poit is, why we cant mount system.img on ICS but we can on GB?
maybe someone can contact with sony t oask
im extracting the .sin to .img like always but its impossible to mount.. what are you using to extract the .sin to a .ext4?
BTW, thanks for the info, i've been trying to modify system.img since ICS appeared.
EDIT: of, ext4 can be extracte with flashtool ~.~
maybe we need to read something from system.partinfo
Click to expand...
Click to collapse
Yes Flashtool can extract image from system.sin.
partinfo is partition information used by loader in flashmode to identify where to flash image on phone. (Something like start nand address of system partition)
so, any ideas why ext4 cant be mounted? maybe its encrypted or something..
Sorry for double post, i found a solution
Flash the system through a .ftf with flashtools
Flash a custom kernel with recovery (or the nozomi recovery)
Backup nandroid
We get a system.ext4.tar ··· move it to your developement folder
Create a folder (mkdir system)
Enter nautilus with root acces
Extract system files to the created folder
Modify whatever you want
Make a flashable system.img with: "./mkuserimg.sh -s /system ./system.img ext4 ./temp 1024M"
AND ITS WORKING!
Yakandu said:
Sorry for double post, i found a solution
Flash the system through a .ftf with flashtools
Flash a custom kernel with recovery (or the nozomi recovery)
Backup nandroid
We get a system.ext4.tar ··· move it to your developement folder
Create a folder (mkdir system)
Enter nautilus with root acces
Extract system files to the created folder
Modify whatever you want
Make a flashable system.img with: "./mkuserimg.sh -s /system ./system.img ext4 ./temp 1024M"
AND ITS WORKING!
Click to expand...
Click to collapse
This solution is already known
But my goal is to be able to mod a system partition without having to flash it before. And more, understand why extracted system image cannot be mounted and how to work this out
oh, ok xD
i didnt know that solution, its new for me
Yakandu said:
Sorry for double post, i found a solution
Flash the system through a .ftf with flashtools
Flash a custom kernel with recovery (or the nozomi recovery)
Backup nandroid
We get a system.ext4.tar ··· move it to your developement folder
Create a folder (mkdir system)
Enter nautilus with root acces
Extract system files to the created folder
Modify whatever you want
Make a flashable system.img with: "./mkuserimg.sh -s /system ./system.img ext4 ./temp 1024M"
AND ITS WORKING!
Click to expand...
Click to collapse
Have you already tried flashing this img on your device? I have already tried this solution twice but didn't succeed (@Spectre51 that's why I haven't replied your PM yet). system.img was succesfully created but I got boot loop when I flashed it on my device.
Hi Androxyde,
I figured it out, basically we have to dig further in sin format as new ext4 sins skips part of the file. See my thread for more details.
PS: Thanks for flashtool, it's a great tool!
LeTama
letama said:
Hi Androxyde,
I figured it out, basically we have to dig further in sin format as new ext4 sins skips part of the file. See my thread for more details.
PS: Thanks for flashtool, it's a great tool!
LeTama
Click to expand...
Click to collapse
:good:
After extracting the ramdisk from the stock boot image of OSS5.0 / Android O 8.0 for Oneplus3, I noticed that fstab.qcom file is no longer there in the ramdisk. Until OSS4.5.1 / Android N 7.1 this file was present and could be edited to set the flag for data partition to encryptable and to also remove the verity flag ...both of which I found to be very useful.
Appreciate if someone can please help me understand what has changed in the Android O 8.0 and what would be the method to achieve the above?
Looking at https://forum.xda-developers.com/oneplus-3t/how-to/disable-dm-verity-force-encryption-op3t-t3688748 and https://forum.xda-developers.com/attachment.php?attachmentid=4302571&d=1507993390, it appears that the fstab.qcom file has moved to /system/vendor/etc/fstab.qcom in Oreo in keeping with Android O / 8.0 specifications. Changing that file means modifying the system partition, which I don't mind as I like to modify the hosts file as the barest minimum change even in my daily use device. But still wondering if the same can be achieved by Magisk and how. Appreciate if someone can help me understand this.
Today's learning:
1. Android O kernel specifications include early mounting of some partitions and Oneplus 3 specifies the system partition in its device tree blobs (dtb).
2. These dtb are part of (...appended to) the real kernel image and not on a separate partition / blob.
3. There are 13 dtb appended to Oneplus 3 kernel and each of them has an entry relevant to fstab which mounts system with the verify flag.
4. Again, in keeping with Android O specifications, fstab.qcom (...now located in the /vendor/etc/ directory) does not include a line for the system partition (...commented out) as mounting the system partition is already taken care of through the dtb in the kernel.
5. That means system modifications cannot be done until the dtb are cleaned of verify flag .... or perhaps the whole of the code mounting the system partition in dtb is deleted and then fstab.qcom is restored in the ramdisk.
Haven't yet tried this and it will be helpful if someone more knowledgeable can confirm that these are the steps to be done to modify the system partition.
@rk2612 great findings and write up!
Just saw your post at magisk beta thread. I'm facing same problem on xperia device with Oreo, and I think we need to change the flag as you stated there.
I've already tried flashing magisk, but dm-verity is still being triggered and phone falls in a bootloop after it reboots.
I'd like to try this method out on my xperia, but I don't know how to convert dtb into dts, and vice versa. Any help on this would be great! Thanks in advance!!
serajr said:
@rk2612...
I've already tried flashing magisk, but dm-verity is still being triggered and phone falls in a bootloop after it reboots.
I'd like to try this method out on my xperia, but I don't know how to convert dtb into dts, and vice versa. ...
Click to expand...
Click to collapse
Have you tried patching with Magisk Beta v1456? That seems to take care of the verity flag in the dtb of OnePlus 3.
On Ubuntu, you can install dtc (device tree compiler ...Refer http://manpages.ubuntu.com/manpages/xenial/man1/dtc.1.html ) to handle conversion from dtb to dts or vice versa.
rk2612 said:
Have you tried patching with Magisk Beta v1456? That seems to take care of the verity flag in the dtb of OnePlus 3.
On Ubuntu, you can install dtc (device tree compiler ...Refer http://manpages.ubuntu.com/manpages/xenial/man1/dtc.1.html ) to handle conversion from dtb to dts or vice versa.
Click to expand...
Click to collapse
Thank you bro!
Yes, I had, latest beta v1456!
Gonna try it out and report back here later!!
serajr said:
...
I'd like to try this method out on my xperia, ....
Click to expand...
Click to collapse
serajr said:
Thank you bro!
Yes, I had, latest beta v1456!
Gonna try it out and report back here later!!
Click to expand...
Click to collapse
According to https://github.com/topjohnwu/Magisk/blob/master/scripts/boot_patch.sh (...line 93 onwards) Sony devices may not be handled well by Magisk.
rk2612 said:
According to https://github.com/topjohnwu/Magisk/blob/master/scripts/boot_patch.sh (...line 93 onwards) Sony devices may not be handled well by Magisk.
Click to expand...
Click to collapse
Bro, this is an known issue for all xperia users. Always before we flash magisk (os SuperSU), we do need to convert xperia stock kernel (elf format) to boot.img (aosp format), and fastboot it!
And for the records, this is exactly what I've found with dtc:
Code:
firmware {
android {
compatible = "android,firmware";
fstab {
compatible = "android,fstab";
vendor {
compatible = "android,vendor";
dev = "/dev/block/platform/soc/7464900.sdhci/by-name/vendor";
type = "ext4";
mnt_flags = "ro,barrier=1,discard";
fsmgr_flags = "wait";
status = "disabled";
};
system {
compatible = "android,system";
dev = "/dev/block/platform/soc/7464900.sdhci/by-name/system";
type = "ext4";
mnt_flags = "ro,barrier=1,discard";
fsmgr_flags = "wait[B][COLOR="Red"],verify[/COLOR][/B]";
status = "ok";
};
};
};
};
Removed ,verify and conveted back to dtb. Gonna test it out (report you back later)!
Thank you!
i have a one plus 5T and i dont see an DTB files in the boot.img
can someone post a walk through on how to do this manually without magisk or supersu?
i know i can modify the fstab in system/vendor/etc
but what are the dtb files exactly? and where are they located / how are they extracted?
virtyx said:
i have a one plus 5T and i dont see an DTB files in the boot.img
can someone post a walk through on how to do this manually without magisk or supersu?
i know i can modify the fstab in system/vendor/etc
but what are the dtb files exactly? and where are they located / how are they extracted?
Click to expand...
Click to collapse
DTB are Device Tree Blobs / Binary and not files. These are appended to the kernel in the boot.img.
On Ubuntu, you can install dtc (device tree compiler ...Refer http://manpages.ubuntu.com/manpages/...an1/dtc.1.html ) to handle conversion from dtb to dts or vice versa.
You can also hex edit the boot.img and delete the text ",verify" at each instance it is found. Or use Linux command line to do that if you're familiar with that.
Sorry, don't have the spare time right now to write a detailed walk through. Maybe this weekend.
rk2612 said:
DTB are Device Tree Blobs / Binary and not files. These are appended to the kernel in the boot.img.
On Ubuntu, you can install dtc (device tree compiler ...Refer http://manpages.ubuntu.com/manpages/...an1/dtc.1.html ) to handle conversion from dtb to dts or vice versa.
You can also hex edit the boot.img and delete the text ",verify" at each instance it is found. Or use Linux command line to do that if you're familiar with that.
Sorry, don't have the spare time right now to write a detailed walk through. Maybe this weekend.
Click to expand...
Click to collapse
thank you
is it possible to extracat the DTB files on a windows machine?
i know after OS install the fstab is located at /system/etc/vendor which we can edit before first boot
but id prefer to edit the DTB files in the boot, and add mounting flags for each partition.
virtyx said:
thank you
is it possible to extracat the DTB files on a windows machine?
i know after OS install the fstab is located at /system/etc/vendor which we can edit before first boot
but id prefer to edit the DTB files in the boot, and add mounting flags for each partition.
Click to expand...
Click to collapse
I didn't look around for a windows tool to extract DTBs.
In Oreo, fstab file contains the mount points for all other partitions except /system and you can edit the fstab file as in previous android versions. The only partition which is mounted early through the DTBs is /system and you will see that the /system mount entry in fstab is commented out as that is not needed.
To mount /system with verity disabled, you will need to remove the flag "verify" from the DTBs. I'm attaching a text version of one of the 13 DTBs contained in OnePlus 3's boot image as an example. You will notice the following code
Code:
fsmgr_flags = "wait,verify"
...and you need to remove ",verify" to make it
Code:
fsmgr_flags = "wait"
To remove ",verify" from the DTBs, you can either:
1. Hex edit the boot image but this will leave the verity key in the ramdisk (...may work but I haven't tried).
2. (a) Unpack the boot image; (b) hex edit the kernel (to which dtbs are appended) and remove ",verify"; (c) extract ramdisk and remove verity.key; (d) repack ramdisk; and (e) repack the boot image: should work
3. (a) Unpack the boot image, (b) extract the dtb appended to kernel / zImage; (c) convert the dtb to dts (text format); (d) text edit the dts to remove ",verify"; (e) then reconvert dts to dtb; (f) append all the modified dtbs to the kernel; and finally (g) repack the boot image (...also remove the verity.key from the ramdisk before repacking the boot image): this would be the cleanest approach but more cumbersome.
Tools needed on Ubuntu machine with Python installed:
1. Unpack and repack boot image: use unpackbootimg and mkbootimg scripts (Python) from lineageos code: https://github.com/LineageOS/android_system_core/tree/cm-14.1/mkbootimg
2. Extract kernel and dtbs using python script: https://github.com/PabloCastellano/extract-dtb
3. Convert dtb to dts and back: use Ubuntu package dtc: http://manpages.ubuntu.com/manpages/xenial/man1/dtc.1.html
rk2612 said:
I didn't look around for a windows tool to extract DTBs.
In Oreo, fstab file contains the mount points for all other partitions except /system and you can edit the fstab file as in previous android versions. The only partition which is mounted early through the DTBs is /system and you will see that the /system mount entry in fstab is commented out as that is not needed.
To mount /system with verity disabled, you will need to remove the flag "verify" from the DTBs. I'm attaching a text version of one of the 13 DTBs contained in OnePlus 3's boot image as an example. You will notice the following code
Code:
fsmgr_flags = "wait,verify"
...and you need to remove ",verify" to make it
Code:
fsmgr_flags = "wait"
To remove ",verify" from the DTBs, you can either:
1. Hex edit the boot image but this will leave the verity key in the ramdisk (...may work but I haven't tried).
2. (a) Unpack the boot image; (b) hex edit the kernel (to which dtbs are appended) and remove ",verify"; (c) extract ramdisk and remove verity.key; (d) repack ramdisk; and (e) repack the boot image: should work
3. (a) Unpack the boot image, (b) extract the dtb appended to kernel / zImage; (c) convert the dtb to dts (text format); (d) text edit the dts to remove ",verify"; (e) then reconvert dts to dtb; (f) append all the modified dtbs to the kernel; and finally (g) repack the boot image (...also remove the verity.key from the ramdisk before repacking the boot image): this would be the cleanest approach but more cumbersome.
Tools needed on Ubuntu machine with Python installed:
1. Unpack and repack boot image: use unpackbootimg and mkbootimg scripts (Python) from lineageos code: https://github.com/LineageOS/android_system_core/tree/cm-14.1/mkbootimg
2. Extract kernel and dtbs using python script: https://github.com/PabloCastellano/extract-dtb
3. Convert dtb to dts and back: use Ubuntu package dtc: http://manpages.ubuntu.com/manpages/xenial/man1/dtc.1.html
Click to expand...
Click to collapse
ah i see
i guess i should really dual boot this machine haha
i found the mounting options in hex editor for the boot.img (/system and /vendor, i think 13 instances of both)
also the verity.key can be delete when unpacking the boot.img and repacking it with no verity.key.
thank you, i think im able to modify it on a windows machine, ill try it out as soon as a stable oreo comes out for the 1+5T
I already have the dtb edited. how can I add them back to the zImage or the boot.img?
Thank you very much
kenet said:
I already have the dtb edited. how can I add them back to the zImage or the boot.img?
Thank you very much
Click to expand...
Click to collapse
Actually the appending should be done like
Code:
cat zImage [I][U]<DTB_FILE_NAME>[/U][/I].dtb > zImage-dtb
So you can get a zImage with dtb