Develop Bugs

Flashing Galaxy S kernel from update.zip : template update.zip

NEWS :
Koush's bmlunlock (a simple IOCTL send to the bml device) is just out and can replace redbend_ua !
http://github.com/CyanogenMod/android_device_samsung_bmlunlock
Hey
Is Flashing from update.zip the new trend ?
'Don't know but here is how you can do it quite easily using this template.
If you target a GT-I9000 on Eclair, you'll need to customize one thing :
In build-update-zip.sh, set DROID_HOME to the source code path for your local Android repository.
If you target another Galaxy S phone or another version of Android, you'll need to adjust ro.hardware and ro.build.id properties accordingly.
This template is done for Linux/Unixes/OSX.
If Linux cost too much for you or your employer, please contribute by sending a .bat equivalent to the .sh
Of course, you also need to put a valid zImage to replace the template's empty zImage
Feel free to adjust to your needs. License is WTFPL
Note : requirement are Java and Android source repo, already build.

Looks awesome!
I have a question though... why do you need to flash the kernel in an update.zip at boot?
As far as I know, the system will read the kernel at boot up time and load it into ram. It won't access the on-disk file again until the next boot. If that is true, then flashing while running should be 100% safe, right?

RyanZA it's an update.zip, not a ramdisk.
The point is getting an easy flashing method, requiring no computer.
And you don't even need to root your phone !

curio, as i see this script is quite simple, so dont u think this all could actually be done on the phone directly? i mean if people cant afford a linux machine...
edit:
sorry for major dumbness, but do you think this could also somehow be used to reflash a nandroid backup?

FadeFx said:
curio, as i see this script is quite simple, so dont u think this all could actually be done on the phone directly? i mean if people cant afford a linux machine...
Click to expand...
Click to collapse
I don't think the signing tool works on android -- maybe it does? Anyway editing scripts on a phone seems a bit silly!

FadeFx said:
curio, as i see this script is quite simple, so dont u think this all could actually be done on the phone directly? i mean if people cant afford a linux machine...
edit:
sorry for major dumbness, but do you think this could also somehow be used to reflash a nandroid backup?
Click to expand...
Click to collapse
Nope actually this is not dumb at all
Yes the flashing part run in updater-script can be started manually.
In the script :
Code:
"redbend_ua", "restore", "zImage", "/dev/block/bml7"
The update.zip presented here mainly targets custom kernel creators in order to give them another way to distribute their work.
This is a working example of how to use redbend_ua programmatically, hopefully it may help new ideas coming. redbend_ua usage is not limited at all to kernel flashing.
PS : you can use this template with windows as well, you'll just need to translate the ulta-basic .sh to a .bat script, or do the signing part manually.

supercurio said:
Hey
If you target a GT-I9000 on Eclair, you'll need to customize one thing :
In build-update-zip.sh, set DROID_HOME to the source code path for your local Android repository.
Click to expand...
Click to collapse
quick question, the build-update-zip.sh is not a must if i only want to flash zImage, rite?
looks like that, I need to modify updater-script (if needed), as well as putting a zImage into ur zip file, and finally remove the build-update-zip.sh from ur zip attached
then, ur zip file can be used for flashing
is it correct?
thx

So this will let people flash any rom from an update.zip (once the ROM makers take this into account) via RomManager without ever having to use Odin to get off stock?
Awesome!

Thanks for the update script curio! this looks great.
One quick question - ive noticed several update.zip scripts for the galaxy S
have update-binary included.
Does anyone know what that does?? where did you get yours?
ive had success in using update.zips without that file at all.
Could anyone post information on what that binary is/does?
supercurio said:
Hey
Is Flashing from update.zip the new trend ?
'Don't know but here is how you can do it quite easily using this template.
If you target a GT-I9000 on Eclair, you'll need to customize one thing :
In build-update-zip.sh, set DROID_HOME to the source code path for your local Android repository.
If you target another Galaxy S phone or another version of Android, you'll need to adjust ro.hardware and ro.build.id properties accordingly.
This template is done for Linux/Unixes/OSX.
If Linux cost too much for you or your employer, please contribute by sending a .bat equivalent to the .sh
Of course, you also need to put a valid zImage to replace the template's empty zImage
Feel free to adjust to your needs. License is BSD anyway.
Note : requirement are Java and Android source repo, already build.
I'll add some documentation later.
Click to expand...
Click to collapse

dseo80 said:
Thanks for the update script curio! this looks great.
One quick question - ive noticed several update.zip scripts for the galaxy S
have update-binary included.
Does anyone know what that does?? where did you get yours?
ive had success in using update.zips without that file at all.
Could anyone post information on what that binary is/does?
Click to expand...
Click to collapse
Thats for the updater-script i believe. In most cases, theres a update-script in the zip's as well and the recovery picks that up in which case it doesn't need the binary and hence works.

Okay !
Answers hour
ykk_five said:
quick question, the build-update-zip.sh is not a must if i only want to flash zImage, rite?
looks like that, I need to modify updater-script (if needed), as well as putting a zImage into ur zip file, and finally remove the build-update-zip.sh from ur zip attached
then, ur zip file can be used for flashing
Click to expand...
Click to collapse
When you run ./build-update-zip.sh,
- it produce a temp zip file containing appropriate files in it.
- then there'is the signature part, building another .zip, ready to be used.
- this "final" update.zip is put in the same current directory and you can use it as it is.
No further complication
Brantyr said:
So this will let people flash any rom from an update.zip (once the ROM makers take this into account) via RomManager without ever having to use Odin to get off stock?
Awesome!
Click to expand...
Click to collapse
Yes, to flash a complete ROM (several partitions) one more thing is needed.
On command line, redbend_ua accept only one command.
In order to run several commands successively (ie flash multiple partition like Odin does), you'll need to write them in a file.
The file /cache/ota/command should do the trick, but it's untested right now.
There may be other method to prevent rebooting after flashing (hacking the redbend_ua binary, finding the appropriate command line option or removing the reboot command temporary)
dseo80 said:
Thanks for the update script curio! this looks great.
One quick question - ive noticed several update.zip scripts for the galaxy S
have update-binary included.
Does anyone know what that does?? where did you get yours?
ive had success in using update.zips without that file at all.
Could anyone post information on what that binary is/does?
Click to expand...
Click to collapse
Right, many update.zip done today are made without knowing anything about how it really works
I studied a bit before creating mine, here is a walk-through this fairly undocumented process :
- recovery mounts /sdcard/
- recovery search for a default file named : META-INF/com/google/android/update-binary in the zip and runs it : see the source in bootable/recovery/install.c
- update-binary is actually updater in sources
- updater looks into the zip file to the script file named updater-script, update-script is obsolete
- updater then runs the commands listed in updater-script : here is the list of commands.
- then reboot
The only documentation I know for this command is the recovery/updater/install.c file itself

supercurio said:
When you run ./build-update-zip.sh,
- it produce a temp zip file containing appropriate files in it.
- then there'is the signature part, building another .zip, ready to be used.
- this "final" update.zip is put in the same current directory and you can use it as it is.
Click to expand...
Click to collapse
ok,thx
but one more thing i want to know is, u said the path must be changed to the android repo, so do u mean the source code for the kernel like linux-xxx-2.xxx dir?
Thx

@ykk_five
Content of build-update-zip.sh v1 :
Code:
#!/bin/sh
DROID_HOME="/home/curio/dev/mydroid"
zip -r /tmp/update.zip META-INF/ redbend_ua zImage
java -jar \
$DROID_HOME/out/host/linux-x86/framework/signapk.jar \
$DROID_HOME/build/target/product/security/testkey.x509.pem \
$DROID_HOME/build/target/product/security/testkey.pk8 \
/tmp/update.zip update.zip
rm /tmp/update.zip
adb push update.zip /sdcard/
DROID_HOME="/home/curio/dev/mydroid" : you set here the directory of your Android AOSP directory.
See : http://source.android.com/source/download.html
Create an empty directory to hold your working files:
$ mkdir mydroid
$ cd mydroid
Run "repo init" to bring down the latest version of Repo with all its most recent bug fixes. You must specify a URL for the manifest:
$ repo init -u git://android.git.kernel.org/platform/manifest.git
* If you would like to check out a branch other than "master", specify it with -b, like:
$ repo init -u git://android.git.kernel.org/platform/manifest.git -b cupcake
Click to expand...
Click to collapse
This is this mydroid directory
The one that contains Android git source home directory, and compiled files in the out/ subdir

many thx for ur detailed explanation, supercurio!

Thanks curio,
very helpful explanation!

Thanks supercurio for the template!
However, there is no need to spend many hours (depending on hardware and bandwidth) pulling down 100s of megabytes of source and compiling it all if all you want is to sign an update.zip with test keys (if you already have the zImage)!
Just google around for "signapk.jar test keys" and you will get there.
BTW: I know that koush, leshak and wesgamer have Samsung Galaxy S trees up at Github but are they fully merged with AOSP yet?
I'm planning to go build my own kernel for this beast to try to solve the mono FM radio mystery, but last time I checked around it was said that the SGS tree required the use of a custom toolchain to get it to work at all.
Any comments on this?

Hey miki4242 !
Good to know that signapk.jar doesn't require hundred of megs of dependencies
About toolchain, you can use the one indicated by Samsung (CodeSourcery) but you'll face the big and ugly WakeLag.
I recommend you crosstool-ng or buildroot to build toolchains, with gcc 4.3.x march=arm mcpu=cortex-a8 mtune=cortex-a8 (no 4.4.x with march=armv7-v)
The building tutorial will be a part of the documentation I'll publish with my lagfix opensource release
Right now these info are too hard to find.
PS : I send you a mail with attached .config for ct-ng !

supercurio said:
Hey miki4242 !
Good to know that signapk.jar doesn't require hundred of megs of dependencies
Click to expand...
Click to collapse
supercurio said:
PS : I send you a mail with attached .config for ct-ng !
Click to expand...
Click to collapse
Thanks for the info !

Sorry for my ignorance, but does this mean someone can package Froyo in an update.zip and we could update it directly on our phone without needing Samsung Kies or Odin?

@supercurio:
Thanks for this well working template.
In updater-script:
Code:
line 17: package_extract_dir("zImage", "zImage");
should be
Code:
package_extract_file("zImage", "zImage");
But it did work with package_extract_dir, for some reason, too.
Btw, could you send me the config for ct-ng ?
I'm also struggling with this wakelag.

[02 Nov][RECOVERY][NAND|SD][cLK|MAGLDR] ClockworkMod Recovery 5.0.2.6 "YES/NO" STABLE

Hi, folks
ClockworkMod Recovery (<- source).
"Changelog of ClockworkMod Recovery"
changes are here for some of the work we have done for support in RM
https://github.com/arif-ali/cLK
https://github.com/CyanogenMod/android_device_htc_leo
changes for the yes/no patch, and anything provided in this thread are here
https://github.com/arif-ali/android_bootable_recovery
https://github.com/arif-ali/android_device_htc_leo
If you cannot wait for new builds, then Arifs nightly builds are available here.
Features:
Based on 5.0.2.6 sources
Re-coded recovery and leo gits in few places to have the same CWR working for both MAGLDR and cLK
Fixed cLK with versioning in cLK to add clk=1.4.0.1 to /proc/cmdline
Now require cLK 1.4.0.1, rather than the 1.4 from cedesmith
Compiled using official Offmode-Charging Fix by Koush
Edited confirmation menu to show one "no" only
Re-arranged YES / NO: YES now is on first place
ReiserFS and NILFS Support
Offmode-Charging with cLK ONLY
SD-EXT backup / restore (no matter if Reiser, NILFS, ext 2/3/4)
You can flash this image easily from within Android; simply copy the file onto your sdcard and run following commands:
Code:
$ su
# flash_image recovery /sdcard/name_of_recovery_file.img
Maybe the flash_image command is not available on your build; you can find it attached. Copy it into /system/bin folder and run following in terminal emulator:
Code:
$ su
# chmod 777 /system/bin/flash_image
For cLK:
You need android sdk installed and path variable set in windoze to "*androidsdk*/platform-tools" and "*androidsdk*/tools" (alternatively you can copy the .img into the tools folder of android sdk and operate from there)
Flash appropiate cLK 1.4.0.1 layout from second post
Download the file attached
Boot your phone into fastboot mode (hold back key (left arrow) whilst powering on)
Open a command line by pressing "windoze key+r" and typing "cmd", followed by enter
Navigate to folder, which contains the recovery image
Type "fastboot erase recovery" (<- not really necessary; just to make sure), followed by enter
Type "fastboot flash recovery name_of_recovery_file.img", followed by enter
Type "fastboot reboot" for rebooting your phone
Access recovery as described in cedesmiths thread (hold home key whilst powering on)
You're done
Click to expand...
Click to collapse
For MAGLDR 1.13, booting from NAND:
Download the file attached
Download a partition layout incl. recovery from this thread which fits your ROM you want to use best and unzip it into "C:\recovery" for example
Copy over the recovery image file from inside the zip archive into this folder "C:\recovery", delete existing "recovery-raw.img" and rename copied image file to "recovery-raw.img"
Flash the recovery and partition layout using "DAF.exe" as usual and described in the thread of raiderx or the ROM provider
Boot Recovery with menu point "8. AD Recovery"
You're done
Note: For future updates of CWR it's best to set up a fixed boot partition size of 5M and not use the filesize flag in flash.cfg.
[/list]
Click to expand...
Click to collapse
For MAGLDR up to 1.13, booting from SDCard (SD-files version):
Download the file attached
Extract the files to the root directory of your sdcard
Start recovery within MAGLDR with menu point "AD SD"
Recovery should start
You're done
Click to expand...
Click to collapse
For both via CWR:
Download the *_CWR.zip file onto sdcard
Go into recovery by either holding home key in cLK or using the 8. AD to recovery in MAGLDR
Note: recovery-partition size of minimum 5MB needed
Select Install zip from sdcard
Select Choose zip from sdcard
Navigate/Select the recovery from the sdcard
Select yes to install the recovery
reboot into recovery, and hooray, you are in the new recovery
Click to expand...
Click to collapse

August 7, 2011
Newest sources 4.0.1.4
June 10, 2011
4.0.0.0 sources
Changelog of recovery:
- added the efs partition to the do not format list
- also fix up the /sdcard symlink on startup
- tar nandroid and /data/media support
June 8, 2011
3.2.0.1 sources
Changelog of recovery:
- Rename format_ignore_partitions to a forbid_format
- Fix 6 extentedcommands declaration warnings
June 3, 2011
3.2.0.0 sources
Added changes necessary for compiling "one recovery for all"
ClockworkmodRecovery now works for MAGLDR and cLK, no extra recovery for each loader
Added new cLK 1.4.0.1 partition layouts as attachment
Arif is continuing this for unknown time: thank you very much, Arif
April 18, 2011
3.0.2.4 sources; renamed to 3.0.2.5
Compiled with Koush's Offmode Charging fix
Attention: Filesize is ~ 1MB bigger than previous version due to new Offmode Charge Fix (i believe); it still fits into 5MB Recovery Partition Size
SD Version is only working if power or USB is plugged in -.- Working 3.0.2.4 Version here
April 13, 2011
Updated to newer 3.0.2.4 sources; see "changelog" link in first post
Full support for REISERFS & NILFS filesystems
cLK and MAGLDR will follow
Added zip file containing cedesmith's offmodecharging and my edits as "patch" (diff) file
Added another flash-method and attached the binary which is necessary
March 29, 2011
Added some cLK_1.4_partition layouts; attached on this post
Normal Cache-size Version (usable for most cases):
boot 5M
cache 44MB
recovery 5MB
misc 1MB
system 80-160MB in 10MB steps, 180-400 in 20MB steps
userdata = rest of available NAND
Small Cache-size Version (requires initrd.gz which links /cache to userdata partition):
boot 5M
cache 5MB
recovery 5MB
misc 1MB
system 80-160MB in 10MB steps, 180-400 in 20MB steps
userdata = rest of available NAND
stirkac made an exe installer with these layouts; it installs cLK and the desired layout automatically. You can find it here. Thank him
March 25, 2011
Updated to 3.0.2.4 sources
March 17, 2011
Updated to 3.0.1.9 sources
cLK was updated to latest patches from cedesmith incl. complete offmode charging support (LED color changing, button press behaviour etc)
March 15, 2011
Backup'n Restore does what it is intended to
cLK is running well and was updated to latest patches from cedesmith
MAGLDR is running well if restored backups are from same version
March 14, 2011
Second run, needs testing by MAGLDR users regarding full restore (mainly: Boot partition!)
cLK is running well
March 10, 2011
Initial release, 3.0.1.4, English only
Bug with MAGLDR; removed
Notes:
- Backups from a previous version can't be restored with this one. However, backups from this version are fully working.
- Offmode charging is only possible with cLK; MAGLDR up to 1.13 isn't capable about that at this time, maybe in future releases.
- MAGLDR Versions are untested by me, here i'll need your feedback, especially regarding restorings from earlier versions (e.g 3.0.1.4 backup restored in 3.0.1.9 = working or not).
CWM "ChangeLog", but w/o Version No.

Awesome. Only thing we really need now is a change log between different CWM versions as they come out.
PS>> Gave you an extra post in case you need it later.

hmm, i'll look around if an official changelog already exists, otherwise i have to figure this out and write for myself, yes.
and thx for extra post

Hard to find an actual version history... All I found was ROM Manager's version history:
http://gh-pages.clockworkmod.com/ROMManagerManifest/CHANGELOG.txt

Nice Work ..
Cool.. Will Save On The Poor Vol Down Key!
Nice Work ..

Cool nice job well done...

seadersn said:
hi, folks
after testing this out i've decided to make it public: my compiled and edited version of ClockworkMod Recovery.
download from multiupload.com
features:
based on newest 3.0.1.4
edited confirmation menu to show one "no" only
edited title to show "S.ClockworkMod Recovery 3.0.1.4" (for determination)
SD-EXT backup / restore incl.
this one is usable for both cLK & MAGLDR.
instructions:
this thread / recovery will be updated every 2-4 weeks, if new recovery is available.
have fun!
Click to expand...
Click to collapse
wooow~try it right now ! thank you!!

@ karendar: thx for looking too. but it seems, there isn't one available, not even in the sources... maybe @ forum, but haven't found sth until now.

EDIT
It works! : D
No more "No's": D
And the function "PowerOff" is rly nice
@seadersn,
kommt das auch auf Handy-FAQ?
Thnx
Greetings.

xD hab ich's mir doch gedacht jap, wird's voraussichtlich in den nächsten drei stunden.
jep, it will

Freu mich schon
Könntest du die ganze Recovery übersetzen [E-D]?
Oder ist das unmöglich?
Could you translate the whole recovery?
Or is that impossible?

nope, this isn't impossible; but this is really too much for me atm maybe in about a week, german only.

We can wait
You have so much time
Thnx.
Greetings.

seadersn said:
hi, folks
after testing this out i've decided to make it public: my compiled and edited version of ClockworkMod Recovery (<- source).
download from multiupload.com
features:
based on newest 3.0.1.4
edited confirmation menu to show one "no" only
edited title to show "S.ClockworkMod Recovery 3.0.1.4" (for determination)
SD-EXT backup / restore incl.
this one is usable for both cLK & MAGLDR.
Click to expand...
Click to collapse
BEAUTIFUL!
Can this be used with MAGLDR 1.11!?!?
If yes can you please provide me with the kernel and initrd to replace the existing ones on SDcard!?
Thx

seadersn said:
hi, folks
after testing this out i've decided to make it public: my compiled and edited version of ClockworkMod Recovery (<- source).
download from multiupload.com
features:
based on newest 3.0.1.4
edited confirmation menu to show one "no" only
edited title to show "S.ClockworkMod Recovery 3.0.1.4" (for determination)
SD-EXT backup / restore incl.
this one is usable for both cLK & MAGLDR.
instructions:
this thread / recovery will be updated every 2-4 weeks, if new recovery is available.
have fun!
Click to expand...
Click to collapse
Any chance you can add reiserfs and btrfs support when making a Nandroid Backup and Restore?

https://github.com/CyanogenMod/android_bootable_recovery#
Here's the original Github for the Gingerbread Clockwork recovery... I haven't programmed in so long I don't think I'm of any help with this.
Pongster, I'm digging through gits to see if anyone had this implemented already in a modded version.

hi,
thx for the github link; anyway i've already added it to the first post unfortunately a changelog or sth similar isn't arising from the sources but thx, again.
regarding reiser+btr: i don't know this much of programming, what i did was an (easy) change of existing code, but if it's already present in the source code and one only needs the mount points, this is no problem, i believe
i can have a further, deeper look into that, but it would take a while if it's not present and i have to change the code and i will definitively need help. no assurance about that
regarding magldr 1.11: life engineer, i will pm you a version for testing. maybe tomorrow morning, GMT+1?
@ spyros, from where do you know that i have plenty of time, he?

seadersn said:
hi,
thx for the github link; anyway i've already added it to the first post unfortunately a changelog or sth similar isn't arising from the sources but thx, again.
regarding reiser+btr: i don't know this much of programming, what i did was an (easy) change of existing code, but if it's already present in the source code and one only needs the mount points, this is no problem, i believe
i can have a further, deeper look into that, but it would take a while if it's not present and i have to change the code and i will definitively need help. no assurance about that
regarding magldr 1.11: life engineer, i will pm you a version for testing. maybe tomorrow morning, GMT+1?
@ spyros, from where do you know that i have plenty of time, he?
Click to expand...
Click to collapse
hey seadersn,
I have been looking at the compilation of recovery as well, and a thought about drivers came about. Which kernel are you using are using the same one in device/htc/leo, or the one compiled by cedesmith. I think in order to get the reiserfs or btrfs, then all we need to do is compile our own kernel with the mods, and put it in the right place and it should work.
I will give it a test with my kernel this evening
with respect to the initrd and zImage, that's easy to get, You need the following 2 files
out/target/product/leo/kernel
out/target/product/leo/ramdisk-recovery.img
but again this needs testing, unless someone else can confirm this

seadersn said:
hi,
thx for the github link; anyway i've already added it to the first post unfortunately a changelog or sth similar isn't arising from the sources but thx, again.
regarding reiser+btr: i don't know this much of programming, what i did was an (easy) change of existing code, but if it's already present in the source code and one only needs the mount points, this is no problem, i believe
i can have a further, deeper look into that, but it would take a while if it's not present and i have to change the code and i will definitively need help. no assurance about that
Click to expand...
Click to collapse
seadersn, check this out: Post from the desire forums... Looks like ReiserFS might be supported in Clockwork directly. Just need to edit .config:
http://forum.xda-developers.com/showpost.php?p=10488102&postcount=432
Alot of the kernels already have ReiserFS and btrfs, I believe...

unpack/repack boot.img : Here is the solution ! :)

Hi everynone,
I was building a kernel for the pixel c. I could build it, now the only problem i have is i can't unpack / repack boot.img of the pixel c (i would like to unpack it in order to extract the ramdisk ans generate the boot.img).
How do you generate boot.img after you have the compiled kernel image on a pixel c ?
These informations are very important to understand and mod Pixel C.
I will be very interested too to have some info about how to unpack/repack boot.img for pixel c (as it's a chromeos boot image)
Thanks a lot in advance,
Mathieu

Ok so after some analysis, here is what i understood :
boot.img contains in itself an android boot.img at the adress 0x1000.
So to extract the android boot.img image, we can use dd :
dd if=chromeosBoot.img of=androidBoot.img skip=8192 bs=8
This will extract the androidBoot.img. Then we will be able to use split_bootimg.pl to extract the kernel and the ramdisk (compressed in gz).
Now my questions : from 0x0 to 0x1000 in chromos boot img, there is some data i still don't understand clearly.
I would like to try to replace the kernel and the ramdisk, and then repack all keeping the same data from 0x0 to 0x1000.
Do you think it's gonna work ?
What are the risks to do so (i don't want to brick too much my tablet...)
Thanks a lot for your help,
Hope my researches will be helpful for other developers.
Mathieu

Samt434 said:
Ok so after some analysis, here is what i understood :
boot.img contains in itself an android boot.img at the adress 0x1000.
So to extract the android boot.img image, we can use dd :
dd if=chromeosBoot.img of=androidBoot.img skip=8192 bs=8
This will extract the androidBoot.img. Then we will be able to use split_bootimg.pl to extract the kernel and the ramdisk (compressed in gz).
Now my questions : from 0x0 to 0x1000 in chromos boot img, there is some data i still don't understand clearly.
I would like to try to replace the kernel and the ramdisk, and then repack all keeping the same data from 0x0 to 0x1000.
Do you think it's gonna work ?
What are the risks to do so (i don't want to brick too much my tablet...)
Thanks a lot for your help,
Hope my researches will be helpful for other developers.
Mathieu
Click to expand...
Click to collapse
Thanks for the unpacking help .. works quick and good.
It´s actually the same like ..
Code:
dd if=boot.img of=unpacked.img ibs=65536 skip=1
Have the same problem, as I want to repack Pixel C boot.img ..etc.
For the repacking part I was searching heavily .. like you.
boot-split.pl or mkbootimg is done ... only the signing part is missing ?!?
Here is a part of the solution : ( working on ubuntu 14.04 )
Code:
apt-get install vboot-kernel-utils
Code:
vbutil_kernel --repack boot.img --oldblob xceed-kernel-google-dragon-v6.10-RELEASE-mxc14g.img --signprivate /usr/share/vboot/devkeys/kernel_data_key.vbprivk
the output ...
Code:
Key block:
Size: 0x4b8
Flags: 7 !DEV DEV !REC
Data key algorithm: 4 RSA2048 SHA256
Data key version: 1
Data key sha1sum: d6170aa480136f1f29cf339a5ab1b960585fa444
Preamble:
Size: 0xfb48
Header version: 2.2
Kernel version: 1
Body load address: 0x100000
Body size: 0x57d000
Bootloader address: 0x67c000
Bootloader size: 0x1000
Body verification succeeded.
Just took the xceed kernel as example .. works with all other "CHROMEOS" images too.
But to sign a changed kernel it seems you need more/other parameter.
Still searching ...
Time to boot soon
Cheers

More infos !
Here is where my researches are :
You need to get and compile vboot_reference in order to have the lastest version :
git clone https://chromium.googlesource.com/chromiumos/platform/vboot_reference
after this :
futility vbutil_keyblock --pack boot.img.keyblock --datapubkey kernel_data_key.vbpubk --signprivate kernel_data_key.vbprivk
and next :
echo " " > empty.txt
futility vbutil_kernel --pack boot.img --keyblock boot.img.keyblock --signprivate kernel_data_key.vbprivk --version 1 --vmlinuz boot.img.unsigned --config empty.txt --arch arm --bootloader empty.txt --flags 0x1
And now it seems we have a valid boot.img image !
Thanks a lot for your researches followmsi, i was blocked due to the fact i couldn't generate a signing key, but it seems that the keys are public finally (in tests/devkeys/ of vboot_reference project) !!
Next step for me :
fastboot flash boot boot.img
This comment here helped me a lot :
https://bbs.archlinux.org/viewtopic.php?id=207385
Thanks anatolik if you read this !
Mathieu

I could unpack/repack the stock kernel with this :
From the boot.img :
dd if=boot.img of=androidBoot.img skip=8192 bs=8
Click to expand...
Click to collapse
This gives you the Android boot.img
then, we can use
split_bootimg.pl androidBoot.img
Click to expand...
Click to collapse
in order to extract the kernel.img and the ramdisk.gz.
To repack :
mkbootimg --kernel kernel.img --ramdisk ramdisk.gz --output boot.img.unsigned
futility vbutil_keyblock --pack boot.img.keyblock --datapubkey kernel_data_key.vbpubk --signprivate kernel_data_key.vbprivk --flags 7
futility vbutil_kernel --pack boot.img --keyblock boot.img.keyblock --signprivate kernel_data_key.vbprivk --version 1 --vmlinuz boot.img.unsigned --config empty --arch arm --bootloader empty --flags 0x1
Click to expand...
Click to collapse
We flash the new kernel :
fastboot flash boot boot.img : the kernel repacked is booting correctly !!

Samt434 said:
I could unpack/repack the stock kernel with this :
From the boot.img :
This gives you the Android boot.img
then, we can use
in order to extract the kernel.img and the ramdisk.gz.
To repack :
We flash the new kernel :
fastboot flash boot boot.img : the kernel repacked is booting correctly !!
Click to expand...
Click to collapse
Well done!
Works like a charm .. Thanks !
Just made a new stock rooted encrytable kernel for Pixel C .. to be installed via TWRP.
http://forum.xda-developers.com/pixel-c/development/twrp-flashable-monthly-update-zip-pixel-t3375591
Cheers

How to build pixel c kernel
Hi Samt434,
I have downloaded pixel c kernel code from chrome os git.
But I don't know which build config should I use or set.
When build other nexus device, from example, Nexus 9, I use:
make tegra_defconfig
make -j4
But what I should to do this for pixel c kernel code?
Thanks very much!
Samt434 said:
Ok so after some analysis, here is what i understood :
boot.img contains in itself an android boot.img at the adress 0x1000.
So to extract the android boot.img image, we can use dd :
dd if=chromeosBoot.img of=androidBoot.img skip=8192 bs=8
This will extract the androidBoot.img. Then we will be able to use split_bootimg.pl to extract the kernel and the ramdisk (compressed in gz).
Now my questions : from 0x0 to 0x1000 in chromos boot img, there is some data i still don't understand clearly.
I would like to try to replace the kernel and the ramdisk, and then repack all keeping the same data from 0x0 to 0x1000.
Do you think it's gonna work ?
What are the risks to do so (i don't want to brick too much my tablet...)
Thanks a lot for your help,
Hope my researches will be helpful for other developers.
Mathieu
Click to expand...
Click to collapse

make dragon_defconfig
Cheers
Sent from my Nexus 10 using Tapatalk

Samt434 said:
I could unpack/repack the stock kernel with this :
From the boot.img :
This gives you the Android boot.img
then, we can use
in order to extract the kernel.img and the ramdisk.gz.
To repack :
We flash the new kernel :
fastboot flash boot boot.img : the kernel repacked is booting correctly !!
Click to expand...
Click to collapse
hi Samt434,
I followed the instructions to flash my repacked boot image, and fastboot boot boot.img
After the boot animation, my Pixel C keeps telling me "To start Android, enter your password"
But I didn't setup any password
any clue? thanks

Samt434 said:
Here is where my researches are :
You need to get and compile vboot_reference in order to have the lastest version :
git clone https://chromium.googlesource.com/chromiumos/platform/vboot_reference
Click to expand...
Click to collapse
Is there someplace that I can download this from (already compiled)? Thanks, in advance.
I'm just trying to edit fstab.dragon, changing from force encrypt to encrytable.

cam30era said:
Is there someplace that I can download this from (already compiled)? Thanks, in advance.
I'm just trying to edit fstab.dragon, changing from force encrypt to encrytable.
Click to expand...
Click to collapse
if you have TWRP, do you mind testing https://build.nethunter.com/android-tools/no-verity-opt-encrypt/no-verity-opt-encrypt-5.1.zip for me?
It should do what you want if it succeeds. I'd like a recovery.log either way.

jcadduono said:
if you have TWRP, do you mind testing https://build.nethunter.com/android-tools/no-verity-opt-encrypt/no-verity-opt-encrypt-5.1.zip for me?
Click to expand...
Click to collapse
I will try it and report back, in a day or two. Thank you very much for the response.

@jcadduono,
It works. Thank you very much. Recovery log attached.

cam30era said:
@jcadduono,
It works. Thank you very much. Recovery log attached.
Click to expand...
Click to collapse
Thanks a lot! This is great news It also means Pixel C custom kernel dev can take advantage of https://github.com/jcadduono/lazyflasher now if they like!
It allows you to simply git clone, put an Image.gz-dtb in the folder, and type 'make' and you've got a TWRP flashable kernel!
If you guys want a simpler way to extract the boot images, you can use https://github.com/jcadduono/android_external_libbootimg
"make" will produce bootimg.exe for windows, bootimg for linux & mac. you can use "make bootimg" for just linux/mac.
Unfortunately it can only unpack ChromeOS images (bootimg xv boot.img) and not modify or create them. It will strip the ChromeOS signature and header if you do use the create/update functions of it, but you can just use futility vbutil_kernel to sign it again.
While it would certainly be nice to add the option to pack/modify ChromeOS images without needing futility, the feature alone would make my project increase in size tenfold.
Prebuilt binaries are here: https://build.nethunter.com/possibly-the-coolest-things/bootimg/

jcadduono said:
It will strip the ChromeOS signature and header if you do use the create/update functions of it, but you can just use futility vbutil_kernel to sign it again.
Click to expand...
Click to collapse
Where can I find the futility vbutil_kernel? I'd still like to be able to manually modify fstab.dragon myself.
Thanks for all of this. Much appreciated.

cam30era said:
Where can I find the futility vbutil_kernel? I'd still like to be able to manually modify fstab.dragon myself.
Thanks for all of this. Much appreciated.
Click to expand...
Click to collapse
It depends on Android libc and libcrypto inside Android build environment so it's not exactly portable but might be possible to force it static and include libc. I tried to do that myself but libcrypto doesn't seem to want to build staticly.
I'd suggest doing a shallow depth repo init of Omni 6.0 TWRP manifest if you want a smallish Android build tree to work with, allowing you to build the tool.

jcadduono said:
It depends on Android libc and libcrypto inside Android build environment so it's not exactly portable but might be possible to force it static and include libc. I tried to do that myself but libcrypto doesn't seem to want to build staticly.
I'd suggest doing a shallow depth repo init of Omni 6.0 TWRP manifest if you want a smallish Android build tree to work with, allowing you to build the tool.
Click to expand...
Click to collapse
I didn't understand a word you said.... You might as well have been speaking Greek!
I did get the part about it not being portable though. Obviously, I'm not a developer.
I'll just stick with your TWRP flashable zip!
Thanks for the feedback, though. I do appreciate it.

cam30era said:
I didn't understand a word you said.... You might as well have been speaking Greek!
I did get the part about it not being portable though. Obviously, I'm not a developer.
I'll just stick with your TWRP flashable zip!
Thanks for the feedback, though. I do appreciate it.
Click to expand...
Click to collapse
Another option as well is to flash your modified normal android boot image to your device.
Download no-verity-opt-encrypt zip 5.1+ and add a file in the patch.d folder of the zip, this should be the file's contents:
Code:
#!/sbin/sh
. "$env"
> "$split_img/boot.img-chromeos"
exit 0
(you can remove the verity/opt encrypt scripts from patch.d if you want to as well)
This will convince the zip to sign it for ChromeOS before flashing it back to the device

jcadduono said:
Another option as well is to flash your modified normal android boot image to your device.
Download no-verity-opt-encrypt zip 5.1+ and add a file in the patch.d folder of the zip, this should be the file's contents:
Code:
#!/sbin/sh
. "$env"
> "$split_img/boot.img-chromeos"
exit 0
This will convince the zip to sign it for ChromeOS before flashing it back to the device
Click to expand...
Click to collapse
I'll give it a try and let you know. Thank you.

@jcadduono,
Why don't you post your mods in the Android Development section for Pixel C. This doesn't get much visibility here, in Q&A.

[TOOL] A QUICK Android OTA payload dumper

Made with Go. By utilizing goroutines, this can extract img files from (full) OTA payload.bin really quickly.
See how fast this is: https://i.imgur.com/adpijqf
Source Code: https://github.com/ssut/payload-dumper-go
Prebuilt binaries: https://github.com/ssut/payload-dumper-go/releases/tag/1.0.0 (for macOS and Windows only)
Howto:
1. Copy original image (zip archive or payload.bin) to the same directory as payload-dumper-go exists.
2. ./payload-dumper-go payload.bin
Notes:
- Incremental OTA payloads are currently not supported but definitely will be in near future.

ssssut said:
Made with Go. By utilizing goroutines, this can extract img files from (full) OTA payload.bin really quickly.
See how fast this is: https://i.imgur.com/adpijqf
Source Code: https://github.com/ssut/payload-dumper-go
Prebuilt binaries: https://github.com/ssut/payload-dumper-go/releases/tag/1.0.0 (for macOS and Windows only)
Howto:
1. Copy original image (zip archive or payload.bin) to the same directory as payload-dumper-go exists.
2. ./payload-dumper-go payload.bin
Notes:
- Incremental OTA payloads are currently not supported but definitely will be in near future.
Click to expand...
Click to collapse
Thanks for creating this, I wanted to give it a try on Windows but it came out this error: liblzma-5.dll not found. Do I need to install any per-requisite? Thanks
EDIT: managed to get the dll from here https://tukaani.org/xz/ and it's all working nicely.

Works great. As a Mac user this is extremely helpful!

Another very good option! cheers

zellleonhart said:
Thanks for creating this, I wanted to give it a try on Windows but it came out this error: liblzma-5.dll not found. Do I need to install any per-requisite? Thanks
EDIT: managed to get the dll from here https://tukaani.org/xz/ and it's all working nicely.
Click to expand...
Click to collapse
can i ask how to install liblzma-5 please? in system? in the program?

mixlex said:
can i ask how to install liblzma-5 please? in system? in the program?
Click to expand...
Click to collapse
You just put the .dll in the same directory as the payload-dumper-go .exe; the issue could be pretty easily avoided if it were compiled static.
In fact, I spent some time today figuring out how to static cross-compile payload-dumper-go from my Ubuntu VM to Win32, Linux x86 and armhf, since it's usually better to go for lowest common denominator, and of course having arm since on-device is where payload-dumper-go might be most useful!
After digging into the recent Docker commit for some hints, then adding stripping and disabling DWARF debugging info generation to have the smallest binary possible, here are my notes for Linux x86:
Bash:
# install latest Go (currently 1.16.2) to /usr/local/go per the Linux instructions at https://golang.org/doc/install
export PATH=$PATH:/usr/local/go/bin
git clone https://github.com/ssut/payload-dumper-go
cd payload-dumper-go
apt-get install liblzma-dev
GOOS=linux GOARCH=386 CGO_ENABLED=1 CC=i686-linux-gnu-gcc go build -a -ldflags '-extldflags "-static -s -w"'
Then, I found that payload-dumper-go's go-xz dependency also in turn being dependent on the toolchain hopefully containing liblzma is extremely problematic/frustrating for Go cross-compiling, but was able to hack the MSYS2 mingw-w64-i686-xz liblzma into the Ubuntu mingw-w64 toolchain to make a static Win32 build:
Bash:
apt-get install mingw-w64
# install include and lib from https://packages.msys2.org/package/mingw-w64-i686-xz to /usr/i686-w64-mingw32
GOOS=windows GOARCH=386 CGO_ENABLED=1 CC=i686-w64-mingw32-gcc go build -a -ldflags '-extldflags "-static -s -w"'
And finally, for Android, NDK gcc wasn't cooperating with `go build` but, since we're building static, Linux armhf will still work fine, but we still need a similar trick to get Ubuntu's own armhf liblzma into the armhf toolchain:
Bash:
apt-get install gcc-arm-linux-gnueabihf
# install include and lib from https://launchpad.net/ubuntu/bionic/armhf/liblzma-dev/5.2.2-1.3 to /usr/arm-linux-gnueabihf
GOOS=linux GOARCH=arm GOARM=7 CGO_ENABLED=1 CC=arm-linux-gnueabihf-gcc go build -a -ldflags '-extldflags "-static -s -w"'
I also noticed it doesn't print instructions even though there are some in the code, and have added a PR to fix that: https://github.com/ssut/payload-dumper-go/pull/5
Hopefully @ssssut will still see about adding Incremental OTA support at some point, maybe do something about go-xz to make cross-compiling easier, and ideally add a feature to only dump specific partitions, since extracting the entire payload.bin can be time-consuming (and RAM-consuming!) when all you want is boot.img.
So, without further ado, here are my builds:
[ Attachments removed since they're now superseded by CI releases on GitHub in all major architectures! ]

Made a Magisk module with a wrapper to get the arm build working smoothly on-device: https://forum.xda-developers.com/t/...ices-platforms.2239421/page-149#post-84753275

osm0sis said:
ideally add a feature to only dump specific partitions
Click to expand...
Click to collapse
There is python variant of dumper with this feature (if anyone interested).
GitHub - sabpprook/payload_dumper
Contribute to sabpprook/payload_dumper development by creating an account on GitHub.
github.com

Trying this on a SP7 with Oneplus6 firmware -- I get a crash towards the end "panic: Memory allocation failed" (I use the static compiled version by osmosis)

hayvan96 said:
Trying this on a SP7 with Oneplus6 firmware -- I get a crash towards the end "panic: Memory allocation failed" (I use the static compiled version by osmosis)
Click to expand...
Click to collapse
To quote my module post: "Only issue I've seen so far is that on a HUGE payload.bin it can run out of memory and fail to extract the largest partitions, regardless of platform, so I believe that's more of an issue with payload-dumper-go itself than my compiles. It certainly works very well to get boot.img and recovery.img, etc. from a Full OTA quickly. Generally I've had best results extracting on my OnePlus 8T, which is a decently beefy device."

OK I'll try to extract it on my phone directly then

osm0sis said:
To quote my module post: "Only issue I've seen so far is that on a HUGE payload.bin it can run out of memory and fail to extract the largest partitions, regardless of platform, so I believe that's more of an issue with payload-dumper-go itself than my compiles. It certainly works very well to get boot.img and recovery.img, etc. from a Full OTA quickly. Generally I've had best results extracting on my OnePlus 8T, which is a decently beefy device."
Click to expand...
Click to collapse
Ok fixed by dumping the xz5.2.5 libs in the same directory (as instructed above) -- but I had to rename libzlma.dll to libzlma-5.dll, maybe this should be added to have a working fix.

hayvan96 said:
Ok fixed by dumping the xz5.2.5 libs in the same directory (as instructed above) -- but I had to rename libzlma.dll to libzlma-5.dll, maybe this should be added to have a working fix.
Click to expand...
Click to collapse
Even with my static compiles? Weird..

osm0sis said:
To quote my module post: "Only issue I've seen so far is that on a HUGE payload.bin it can run out of memory and fail to extract the largest partitions, regardless of platform, so I believe that's more of an issue with payload-dumper-go itself than my compiles. It certainly works very well to get boot.img and recovery.img, etc. from a Full OTA quickly. Generally I've had best results extracting on my OnePlus 8T, which is a decently beefy device."
Click to expand...
Click to collapse
Looks like @luca020400 and @LuK1337 from Lineage fixed this today and added the feature to select partitions to extract!
Hopefully @ssssut can make some new official binary release builds (static this time ), and I'll be happy to post some for any architectures not covered and update my Magisk module.

osm0sis said:
Looks like @luca020400 and @LuK1337 from Lineage fixed this today and added the feature to select partitions to extract!
Hopefully @ssssut can make some new official binary release builds (static this time ), and I'll be happy to post some for any architectures not covered and update my Magisk module.
Click to expand...
Click to collapse
Man it was using 7GB of RAM here, I had to fix it.

v1.1.0 is up thanks to some more solid work from @LuK1337! Now it automatically builds in all major architectures.
Releases · ssut/payload-dumper-go
an android OTA payload dumper written in Go. Contribute to ssut/payload-dumper-go development by creating an account on GitHub.
github.com
Updated Magisk module to v1.1.0 as well: https://forum.xda-developers.com/t/...ices-platforms.2239421/page-149#post-84753275
Probably solid now until Incremental OTA support can be looked into.

@ssssut - I tried a few different versions of windows_386 payload-dumper-go up to 1.1.1 on Windows XP Professional with Service Pack 3. Unfortunately, the payload-dumper-go software does not work. When trying to execute payload-dumper-go.exe, I receive the following error message:
[Path the executable]\payload-dumper-go.exe is not a valid Win32 application.
Click to expand...
Click to collapse
Please update the software so that it may work on Windows XP Professional with Service Pack 3.

Hmm I think it worked fine for me on Windows 10 x86 last I checked, so I guess it just doesn't support XP.. Not sure if there's anything to be done for that.

Very useful tool! Updated my firmware on [email protected]
Thank you!

thanks, bro...
very simple but pretty useful tool

Include an individual file in the lineage sources

Hello folks,
I am a beginner. Sorry if it's a stupid question.
I want to include a file when compiling a custom rom with Lineage OS. It should be under "/sys/module/dwc3/parameters", named "aca_enable" and contain a "Y".
How can I build this into the sources so that it is already there after flashing?
I've already searched the web, but couldn't find anything.
Thanks for replies in advance!

This is not a normal file. Kernel modules expose flags under /sys/module/<module-name> which can be changed by file operations. Pretty common under linux to have such interfaces (e.g. procfs).
I can't give you a complete answer, but can probably point you in the right direction: You need to do some research on init.rc. Basically you need to set a rule in an .rc file in /system/etc/init. (Or compile a new kernel / the dwc3 module and set a new default value)

sn00x said:
This is not a normal file. Kernel modules expose flags under /sys/module/<module-name> which can be changed by file operations. Pretty common under linux to have such interfaces (e.g. procfs).
I can't give you a complete answer, but can probably point you in the right direction: You need to do some research on init.rc. Basically you need to set a rule in an .rc file in /system/etc/init. (Or compile a new kernel / the dwc3 module and set a new default value)
Click to expand...
Click to collapse
Hello, many thank you for the answer!
Yes I understand. These files are created and modified at runtime. Then I know where to look. An example would be best. "Copy and Paste" is a popular method. ;-)

First try - quick and dirty - did not work.
I put the following lines into the system/core/rootdir/init.usb.rc (lineageos sources):
# Set /sys/module/dwc3/parameters/aca_enable to "Y"
write /sys/module/dwc3/parameters/aca_enable Y
After compiling the ROM, flashing it on the device and boot the device the file /sys/module/dwc3/parameters/aca_enable contain a "N".
On the Spartphone the file /etc/init/hw/init.usb.rc contains the modified lines.
In the modified file dwc3_otg.c i can't find anything that write a "N".
Maybe someone has an idea or an alternative?
(Here is the source code of dwc3_otg.c https://github.com/sollapse/opo_dwc3_otg/)

Well, just change 0 to 1 in line 42.

sn00x said:
Well, just change 0 to 1 in line 42.
Click to expand...
Click to collapse
Hello, oh yes - I had tomatoes on my eyes. Thank you very much for the tip!

sn00x said:
Well, just change 0 to 1 in line 42.
Click to expand...
Click to collapse
It works!
Thank you so much for making the effort.
fyi: Another option would be here.