[TUTORIAL] Making Flashable ZIPs, EDIFY Script, ZIP Signing & Key Creation [19.01.13]
Recent Updates: Command Line Edify Script Tester, Further 'update-binary' info clarification, helpful links, additional credits. post#1
Hi all,
In the process of creating my [BOOTANIMATIONS - LINKS & INFO] thread I realised I needed to know more about how boot animations worked, as by default people tend to ask questions in that type of thread, so I wanted to be able to help.
This meant I came across the problem of creating flash-able ZIP files and Edify scripting and in the interests of sharing and helping this great community I would like to share what I have found and hopefully learn more through discussion, I certainly am no expert and am always willing to learn.
I have checked xda and various other websites whilst reseraching for this post and there is nothing that really explains all the basic concepts to just get up and running, also a lot of threads are quite old and focus on the old Amend scripting syntax or are a confusing mixture of the two.
I hope this helps people get started and that we can all collaborate to make this a thread a source of valuable information.
Contents:
Post #1: Tutorials
Post #2: Edify Commands
***Please, if there are any mistakes on this thread then let me know what I have done incorrectly in reply to this post and I will update the thread to correct it***
***As always, I accept no responsibiltiy for your handset, you modify it at your own risk, nobody is forcing you...be aware you can seriously bork your phone by flashing its internal partitions! Always make a nandroid back-up and think before you flash!***
***Have you made a nandroid back-up...no? Then make one!!***
You will need the following:
Android SDK, ADB & Fastboot set up for your handset.
Notepad++ [LINK]
7zip [LINK]
Setting up your zip directories:
you will need to create the following folder structure (these are case sensitive):
Code:
/META-INF/com/google/android
All flash-able zips include this file structure, the final folder 'android' will contain two files:
Code:
update-binary
updater-script
update-binary: I have been unable to find much information on the update-binary file other than they seem to be chip set specific (if anyone can shed further light on these I will include it here). For the sake of compatibility I have attached the update binary from the latest CyanogenMod Nightlies for the maguro at the bottom of this thread - You can of course download the latest nightly and extract the update-binary file yourself as the attached one will obviously begin to date (all credit to the great devs at CyanogenMod for this, many thanks).
Update: For more detailed info on the update binary you can find the sources in the bootable/recovery/updater directory at AndroidXRef here: [ xref: /bootable/recovery/updater/ ]. There is also an edify directory which looks like it handles the parsing of the script. Obviously with the sources there's nothing to stop you extending syntax/functionality should you wish given the fact the each zip comes with its own version. It will also help you understand how signature checking is handled.
Update II: For anybody having problems flashing it is worth ensuring you have an up to date version of the update-binary (obviously the one attached to this thread will gradually become out of date).
It is advisable to use the update-binary from the latest OTA image for your device, these images can be found in the following link:
Official Google OTA update URLs - efrant
updater-script: This we can create ourselves, to ensure it works properly we will use Notepad++.
Open Notepad++ and start a new file, with the following settings:
Format: Unix
Encoding: ANSI
Default Language: Normal Text
Save this file as:
File name: updater-script
File type: All types (*.*)
You can now edit this file, add the following text:
Code:
[COLOR="Red"]assert(getprop("ro.product.device") == "maguro" || getprop("ro.build.product") == "maguro");[/COLOR]
[COLOR="Green"]ui_print(" ");[/COLOR]
[COLOR="green"]ui_print("confirming device maguro");[/COLOR]
[COLOR="green"]ui_print(" ");[/COLOR]
[COLOR="green"]ui_print("success");[/COLOR]
[COLOR="green"]ui_print(" ");[/COLOR]
[COLOR="DarkOrange"]show_progress(0.200000, 5);[/COLOR]
[COLOR="green"]ui_print("mounting system");[/COLOR]
[COLOR="red"]mount("ext4", "EMMC", "/dev/block/platform/omap/omap_hsmmc.0/by-name/system", "/system");[/COLOR]
[COLOR="green"]ui_print(" ");[/COLOR]
[COLOR="DarkOrange"]show_progress(0.200000, 5);[/COLOR]
[COLOR="green"]ui_print("updating system files");[/COLOR]
[COLOR="red"]package_extract_dir("system", "/system");[/COLOR]
[COLOR="green"]ui_print(" ");[/COLOR]
[COLOR="DarkOrange"]show_progress(0.200000, 5);[/COLOR]
[COLOR="green"]ui_print("unmounting system");[/COLOR]
[COLOR="red"]unmount("/system");[/COLOR]
[COLOR="green"]ui_print(" ");[/COLOR]
[COLOR="DarkOrange"]show_progress(0.200000, 5);[/COLOR]
[COLOR="green"]ui_print("by yourusername");[/COLOR]
[COLOR="DarkOrange"]show_progress(0.200000, 5);[/COLOR]
[COLOR="green"]ui_print(" ");[/COLOR]
***EMPTY LINE***
Ok, I've colour coded this so we can break it down, the lines of code in RED are the actual commands, everything else is cosmetic.
assert(getprop("ro.product.device") == "maguro" || getprop("ro.build.product") == "maguro"); This is checking you are flashing the correct handset, this is not a requirement, but is best practice. Just insert another device name in the place of "maguro" if you so wish, or remove the command all together.
mount("ext4", "EMMC", "/dev/block/platform/omap/omap_hsmmc.0/by-name/system", "/system"); This is the specific mount point for the maguro system partition, if you wish to flash a different partition, data for instance you can get the mount points for your device by entering the following code over ADB:
Code:
adb shell "mount > /sdcard/PHONENAME_mountinfo.txt"
This will place a text file on your sdcard with the mount points for your specific device (remember to swap 'PHONENAME' with your device name e.g. 'maguro')
package_extract_dir("system", "/system"); This command extracts the files you wish to flash from the zip and flashes them to the phone, it is formatted as follows ("package-path", "/destination-path"). So for this example you are telling it to flash everything from the 'system' folder in your zip to the /system partition of your handset. You can obviously change these values for different partitions.
unmount("/system"); This simply unmounts whatever partition you previously mounted.
ui_print(" "); Shows text in the recovery whilst the flash is ongoing, you can put whatever you please between the speech marks, you must leave a space if you wish to have a blank line.
show_progress(0.200000, 5); Controls what the progress bar in the background is displaying, it is formatted as follows (fragment, seconds).
***EMPTY LINE*** This is not actually text, you simply need to leave a blank line at the end of your script before you save it for it to work.
FYI, this means in fact your script could look like this and still work:
Code:
[COLOR="Green"]ui_print(" ");[/COLOR]
[COLOR="DarkOrange"]show_progress(1.000000, 30);[/COLOR]
[COLOR="red"]mount("ext4", "EMMC", "/dev/block/platform/omap/omap_hsmmc.0/by-name/system", "/system");[/COLOR]
[COLOR="red"]package_extract_dir("system", "/system");[/COLOR]
[COLOR="red"]unmount("/system");[/COLOR]
[COLOR="Green"]ui_print(" ");[/COLOR]
***EMPTY LINE***
But that just isn't that pretty, or as deceptively complicated lol!
Make sure you save your edify script.
Files to flash:
You now need to create a further folder, this needs to be named based on where within the system you are flashing, for the sake of this example we are flashing to the system partition, so:
Code:
/META-INF/com/google/android
/system/app
Place whatever files you wish to flash within this file e.g:
Code:
/META-INF/com/google/android
/system/app/nameofapp.apk
Creating your .zip file:
Select both of your top directories and their contents 'META-INF' and 'system' and package them into a .zip file with 7zip using the following settings:
Archive: name_of_your_file
Archive Format: zip
Compression level: Store
Update mode: Add and replace files
And there you have it, your very own flash-able .zip file.
I have attached an example flashable .zip file that includes this updater-script and the above mentioned CM update-binary.
How to sign your update.zip:
You don't actually need to sign your zip file for it to work as most custom recoveries now support unsigned zips, for aspiring developers and the more cautious among us though, here is how.
You will need to download the following files:
SignApk [LINK]
Java JDK (Java Development Kit) & JRE (Java Runtime Environment) [LINK]
Install Java JRE and JDK and restart your PC.
You then need to create a folder in the root of your c:\ drive named 'signapk' and extract the contents of the SignApk download (signapk.jar, key.pk8, certificate.pem) to this folder.
Place your .zip file into the same folder and then open cmd line. Input the following commands (remeber to change the name of 'myupdate.zip' to the name of your file and 'myupdate-signed.zip' to whatever you want your resulting .zip to be named):
Code:
cd\signapk\
java -jar signapk.jar certificate.pem key.pk8 c:\signapk\myupdate.zip myupdate-signed.zip
You should find that you now have the the following files in your 'META-INF' folder: CERT.RSA, CERT.SF, MANIFEST.MF
This can also to be used to sign .apk files using the below command (same stipulations for file name changing applies as above):
Code:
java -jar signapk.jar certificate.pem key.pk8 c:\signapk\myapplication.apk myapplication-signed.apk
How to create your own private signing key & certificate:
You will need the following download:
OpenSSL [LINK]
The above signapk file includes test keys, if you want to create your own private keys for signing, here's what you need to do.
As before extract the OpenSSL files to a folder in the root of your c:\ drive, preferably named 'openssl' for ease of cmd line navigation. Then input the following:
Code:
cd\openssl\
openssl genrsa -out key.pem 1024
openssl req -new -key key.pem -out request.pem
openssl x509 -req -days 9999 -in request.pem -signkey key.pem -out certificate.pem
openssl pkcs8 -topk8 -outform DER -in key.pem -inform PEM -out key.pk8 -nocrypt
You can then replace 'key.pk8' & 'certificate.pem' with your own files.
I really hope this has helped folks out, if so please consider hitting the 'Thanks' button!
Happy flashing!
Testing your script before you flash:
If you would like to test if your script commands are valid and will run properly on your PC first before you use them on your handset, therefore removing the fear of borking your current install/handset in the process then head to this post:
Command Line Edify Script Tester by: trevd
This great tool developed by trevd currently has the following features:
Validates function name.
Validates function parameter count.
It is available for Linux and Windows, so head over and check it out. Be sure to click trevd's thanks button if you find this useful.
Other helpful links & threads:
xda-developers: Edify script language wiki
Edify scripts in CWM recovery by: NFHimself
Phandroid - Edify Script Language Reference by: Koumajutsu
Introduction to edify updater script by: kurotsugi
Intelligent EDIFY updater-script - (Decides things during flash) by: lotherius
Credits:
CyanogenMod for (update-binary)
Lorenz's Blog for (SignAPK & Info)
JoeSyr for (Help with File Permissions)
efrant for (Help with various commands)
trevd for (further update-binary clarification and source links)
osm0sis for (Helpful info & forum links & help with update-binary info)
Edify Scripting Commands
I would like to try and put together a guide to Edify scripting commands and what their function is to compliment my above tutorial, I hope again we can all collaborate to make this a helpful source of information to all budding devs...we all have to start somewhere right!?
***Please, if there are any mistakes in this post or if you can help by adding further info please post below, thanks***
Ok...
Check you are flashing the correct device:
Code:
assert(getprop("ro.product.device") == "maguro" || getprop("ro.build.product") == "maguro" || getprop("ro.product.board") == "maguro");
This will check your device is showing the correct name in the build.prop, in this case "maguro", you can obviously replace this with your device name.
To explain this a little further, you have told the script to check certain properties within your 'build.prop' file located in your /system folder. The above command ensures the following categories have the correct definitions before proceeding with the flash:
Code:
ro.product.device=
ro.build.product=
ro.product.board=
Here are the main build properties within your build.prop:
Code:
ro.build.id=
ro.build.display.id=
ro.build.version.incremental=
ro.build.version.sdk=
ro.build.version.codename=
ro.build.version.release=
ro.build.date=
ro.build.date.utc=
ro.build.type=
ro.build.user=
ro.build.host=
ro.build.tags=
ro.product.model=
ro.product.brand=
ro.product.name=
ro.product.device=
ro.product.board=
ro.product.cpu.abi=
ro.product.cpu.abi2=
ro.product.manufacturer=
ro.product.locale.language=
ro.product.locale.region=
ro.wifi.channels=
ro.board.platform=
ro.build.description=
ro.build.fingerprint=
ro.build.characteristics=
ro.cm.device=
Print text in the recovery during flashing process:
Code:
ui_print("Your text here");
Code:
ui_print(" ");
This simply prints a line of text in the recovery, it has no actual effect on the flashing process, if you want a blank line just leave a blank space as the second example.
Controlling the progress bar:
I believe I understand this correctly, if not please do post below with further clarification.
Code:
show_progress(0.000000, 0);
or
Code:
set_progress(0.000000);
You have two choices when controlling the progress bar, the first example allows you to define fractions of the progress bar and how long they will take to fill. The second example just allows you to specify that the bar fills to a certain fraction at whatever point during the flashing process.
The command is defined as: (progress bar fraction, duration in seconds to fill defined fraction);
For example the following code lines interspersed with your other commands would fill a fifth of the progress bar every five seconds:
Code:
show_progress(0.200000, 5);
ui_print(" ");
show_progress(0.200000, 5);
ui_print(" ");
show_progress(0.200000, 5);
ui_print(" ");
show_progress(0.200000, 5);
ui_print(" ");
show_progress(0.200000, 5);
This process will only complete if the script takes long enough to flash, therefore you need to be aware of what you are actually flashing and how long it will take when defining these values.
If you wish to just define a fraction without fill without a time scale you can use the following command:
Code:
set_progress(0.000000);
It is also best practice to include this element in your scripts as it will reassure people that their handset hasn't frozen during a flash.
Mount/Unmount a partition:
To mount a partition you need to use the following syntax:
Code:
mount("filesystem-type", "partition-type", "device-specific-location", "mount-point");
filesystem-type: "ext4" or "yaffs2" (maguro is ext4)
partition-type: "EMMC" or "MTD" (maguro is EMMC)
location: Device specific address
mount-point: /system etc
So far I have managed to define the following Maguro specific mount points:
Code:
"/dev/block/platform/omap/omap_hsmmc.0/by-name/system", "/system"
"/dev/block/platform/omap/omap_hsmmc.0/by-name/userdata", "/data"
"/dev/block/platform/omap/omap_hsmmc.0/by-name/cache", "/cache"
"/dev/block/platform/omap/omap_hsmmc.0/by-name/efs" "/factory"
"/dev/block/platform/omap/omap_hsmmc.0/by-name/userdata", "/mnt/sdcard"
"/dev/block/platform/omap/omap_hsmmc.0/by-name/boot"
Example command would be:
Code:
mount("ext4", "EMMC", "/dev/block/platform/omap/omap_hsmmc.0/by-name/system", "/system");
I have taken these mount points from the file output by the maguro when you enter the following adb command:
Code:
adb shell "mount > /sdcard/maguro_mountinfo.txt"
I have to admit I am a little in the dark about the "/factory" partition, any info would be greatly appreciated!
To unmount a partition you need to input the following command:
Code:
unmount("/system");
Obviously replace the mount partition with whatever partition you are working in.
Format a partition:
To format a partition you need to use the following syntax:
Code:
format("filesystem-type", "partition-type", "device-specific-location", "0");
Example:
Code:
format("ext4", "EMMC", "/dev/block/platform/omap/omap_hsmmc.0/by-name/system", "0");
Some scripts include the "0" and some do not, not sure exactly on the function difference when included or not, again clarification would be great.
Flashing the contents of your ZIP file:
To flash an entire directory:
Code:
package_extract_dir("system", "/system");
To flash a single file:
Code:
package_extract_file("boot.img", "/dev/block/platform/omap/omap_hsmmc.0/by-name/boot");
These commands are structured as follows:
Entire directory: ("zipfileSource", "destination-partition");
Single File: ("file", "device-specific-mountpoint");
Deleting folders & files:
You can delete multiple folders or files using just one command, as follows:
To delete files:
Code:
delete("file-path-1", "file-path-2", "file-path-3);
To delete folders/directories:
Code:
delete_recursive("directory-path-1", "directory-path-2", "directory-path-3");
Setting Permissions
Here are the basics for setting permissions.
If you want to research the reasons behind this, there is some useful information on Linux permissions here: chmod wiki, linuxquestions.org linux wiki.
Thanks to JoeSyr for his input and help understanding this section of the tutorial.
Set permissions of a file or set of files:
Code:
set_perm(uid, gid, mode, "filepath1", "filepath2")
Example:
Code:
set_perm(0, 0, 06755, "/system/xbin/su");
uid - user id
gid - group id
mode - permission mode
filepath... - file to set permission on
Set permissions of a directory or set of directories and all files and folders within them:
Code:
set_perm_recursive(uid, gid, dirmode, filemode, "dirpath1", "dirpath2")
Example:
Code:
set_perm_recursive(0, 0, 0755, 0644, "/system");
uid - user id
gid - group id
dirmode - permission to set to directories contained within the specified directory
filemode - permission to set to files contained within the specified directory
dirpath... - directory to set permission on
Permissions syntax explained...so I can understand it lol (if I do then anyone can!):
The following are the pre-defined Android UID's & GID's. Taken from the following link: Android UIDs and GIDs
Code:
AID_ROOT 0 /* traditional unix root user */
AID_SYSTEM 1000 /* system server */
AID_RADIO 1001 /* telephony subsystem, RIL */
AID_BLUETOOTH 1002 /* bluetooth subsystem */
AID_GRAPHICS 1003 /* graphics devices */
AID_INPUT 1004 /* input devices */
AID_AUDIO 1005 /* audio devices */
AID_CAMERA 1006 /* camera devices */
AID_LOG 1007 /* log devices */
AID_COMPASS 1008 /* compass device */
AID_MOUNT 1009 /* mountd socket */
AID_WIFI 1010 /* wifi subsystem */
AID_ADB 1011 /* android debug bridge (adbd) */
AID_INSTALL 1012 /* group for installing packages */
AID_MEDIA 1013 /* mediaserver process */
AID_DHCP 1014 /* dhcp client */
AID_SHELL 2000 /* adb and debug shell user */
AID_CACHE 2001 /* cache access */
AID_DIAG 2002 /* access to diagnostic resources */
/* The 3000 series are intended for use as supplemental group id's only. */
/* They indicate special Android capabilities that the kernel is aware of. */
AID_NET_BT_ADMIN 3001 /* bluetooth: create any socket */
AID_NET_BT 3002 /* bluetooth: create sco, rfcomm or l2cap sockets */
AID_INET 3003 /* can create AF_INET and AF_INET6 sockets */
AID_NET_RAW 3004 /* can create raw INET sockets */
AID_MISC 9998 /* access to misc storage */
AID_NOBODY 9999
AID_APP 10000 /* first app user */
"root", AID_ROOT
"system", AID_SYSTEM
"radio", AID_RADIO
"bluetooth", AID_BLUETOOTH
"graphics", AID_GRAPHICS
"input", AID_INPUT
"audio", AID_AUDIO
"camera", AID_CAMERA
"log", AID_LOG
"compass", AID_COMPASS
"mount", AID_MOUNT
"wifi", AID_WIFI
"dhcp", AID_DHCP
"adb", AID_ADB
"install", AID_INSTALL
"media", AID_MEDIA
"shell", AID_SHELL
"cache", AID_CACHE
"diag", AID_DIAG
"net_bt_admin", AID_NET_BT_ADMIN
"net_bt", AID_NET_BT
"inet", AID_INET
"net_raw", AID_NET_RAW
"misc", AID_MISC
"nobody", AID_NOBODY
You will need to also understand the way file permissions are represented:
Example = drwxrwxrwx
-Use Root Explorer to find the UID, GID, and permissions for a file. Permissions are the string that looks like some variation on 'rwxr-xr--' next to each file. Long press and choose "change owner" to get the UID and GID. You just want the number next to "owner" and "group", respectively.
-If you're replacing a file, look up these settings for the existing copy and use them. If you're adding a file, just find a file that does the same functions and copy what it has (for example, installing an app to /system/app? Just look at these settings for any other app in that directory).
-MODE is technically a 4-bit string, but the first character can be omitted. There doesn't seem to be any android file permissions with the first character set. For good practice, I think it's safe to assume you can always use a leading 0 unless you know otherwise for something specific. Or, just use a 3-digit MODE, which says to leave those settings as they are (disabled by default). (I.e. 0644=644).
The next 9 characters define the file permissions. These permissions are
given in groups of 3 each.
The first 3 characters are the permissions for the owner of the file or directory.
Example = -rwx------
The next 3 are permissions for the group that the file is owned by.
Example = ----rwx---
The final 3 characters define the access permissions for everyone not part of the group.
Example = -------rwx
There are 3 possible attributes that make up file access permissions.
r - Read permission. Whether the file may be read. In the case of a
directory* this would mean the ability to list the contents of the
directory.
w - Write permission. Whether the file may be written to or modified. For
a directory* this defines whether you can make any changes to the contents
of the directory. If write permission is not set then you will not be able
to delete* rename or create a file.
x - Execute permission. Whether the file may be executed. In the case of a
directory* this attribute decides whether you have permission to enter*
run a search through that directory or execute some program from that
directory
You set these permissions using the following binary based numerical system:
Code:
0: --- No Permissions (the user(s) cannot do anything)
1: --x Execute Only (the user(s) can only execute the file)
2: -w- Write Only (the user(s) can only write to the file)
3: -wx Write and Execute Permissions
4: r-- Read Only
5: r-x Read and Execute Permissions
6: rw- Read and Write Permissions
7: rwx Read, Write and Execute Permissions
Default Linux permissions:
For Files:
"Read" means to be able to open and view the file
"Write" means to overwrite or modify the file
"eXecute" means to run the file as a binary
For Directories:
"Read" means to be able to view the contents of the directory
"Write" means to be able to create new files/directories within the directory
"eXecute" means to be able to "Change Directory" (cd) into the directory
Most of the time you set "Read" and "eXecute" together on directories (kind of useless when set by themselves)
Interestingly through further research it seems this system is based on the Octal Binary system...I may be wrong...
Code:
Octal
Binary is also easily converted to the octal numeral system, since octal uses a radix of 8, which is a power of two (namely, 23, so it takes exactly three binary digits to represent an octal digit).
The correspondence between octal and binary numerals is the same as for the first eight digits of hexadecimal in the table above.
Binary 000 is equivalent to the octal digit 0, binary 111 is equivalent to octal 7, and so forth.
Octal Binary
0 000
1 001
2 010
3 011
4 100
5 101
6 110
7 111
Converting from octal to binary proceeds in the same fashion as it does for hexadecimal:
658 = 110 1012
178 = 001 1112
And from binary to octal:
1011002 = 101 1002 grouped = 548
100112 = 010 0112 grouped with padding = 238
And from octal to decimal:
658 = (6 × 81) + (5 × 80) = (6 × 8) + (5 × 1) = 5310
1278 = (1 × 82) + (2 × 81) + (7 × 80) = (1 × 64) + (2 × 8) + (7 × 1) = 8710
Flash boot partition
This uses the following command but with some caveats:
Code:
write_raw_image("path to boot.img", "boot");
An example of this commands use would be to flash a custom kernel, as far as I can tell though you need to place the boot.img in a temp file before the actual flash (clarification on this would be helpful). You would therefore use the following commands as part of your edify script:
Code:
package_extract_file("boot.img","/tmp/boot.img");
write_raw_image("/tmp/boot.img", "device-specific-boot-partition-mount-point");
delete("/tmp/boot.img");
It seems this may only be for specific handsets though e.g HTC, having looked through quite a few updater-script files from maguro zips it seems this can just be flashed directly with the following command:
Code:
package_extract_file("boot.img", "/dev/block/platform/omap/omap_hsmmc.0/by-name/boot");
Many thanks for further clarification on this particular command from efrant below:
I feel more comfortable using something like this:
Code:
package_extract_file("boot.img", "/tmp/boot.img");
run_program("/sbin/busybox", "dd", "if=/tmp/radio.img", "of=/dev/block/platform/omap/omap_hsmmc.0/by-name/boot");
delete("/tmp/boot.img")
The second line is using the "dd" command to flash the image.
The dd command (included in busybox) is:
Code:
dd if=xxx of=yyy
where xxx is the source image/partition and yyy is the target image/partition. So, for example, to flash a radio image stored in /sdcard called xxlf1.img to a GNex, it would be:
Code:
busybox dd if=/sdcard/xxlf1.img of=/dev/block/platform/omap/omap_hsmmc.0/by-name/radio (I've added "busybox" the the beginning, just to be explicit.)
So, to put it in edify format, it becomes:
Code:
run_program("/sbin/busybox", "dd", "if=/sdcard/xxlf1.img", "of=/dev/block/platform/omap/omap_hsmmc.0/by-name/radio");
However, as I said, I believe your one line:
Code:
package_extract_file("boot.img", "/dev/block/platform/omap/omap_hsmmc.0/by-name/boot");
would work.
Further Commands:
In progress...
So, this is by no means an exhaustive list and I would appreciate it that if I have made any mistakes people make me aware in this thread and I can update this tutorial.
I would also like to add further commands and their uses, if people can provide more I will add them too.
I hope this helps people out! Many thanks!
Thx for your work. Will try it when I back @ home.
Mfg Daniel
Gesendet von meinem Galaxy Nexus mit Tapatalk 2
very interesting 3ad.i'm learning right now how to make these,yours guide resolve me a lot of questions!thanks mate!+1
Awesome. Thank you!
Thanks for the feedback folks!
Top post updated with guides for:
Signing ZIP files
Creating private ZIP signing keys
I have updated the second post with edify commands:
Contents:
Post #1: Tutorials
Post #2: Edify Commands
I would like to expand and update this information to help both others and myself learn, if people can help with more commands or any corrections this would be much appreciated!
Thanx a lot . Ive been looking into editing apks and themeing and this answers a lot of my questions. Exactly what I've been looking for.
Sent from my Galaxy Nexus using XDA
graffitiwriter said:
Thanx a lot . Ive been looking into editing apks and themeing and this answers a lot of my questions. Exactly what I've been looking for.
Click to expand...
Click to collapse
Many thanks for the feedback.
Everything I know about Android has been learned from internet research and tutorials. So as far as I can I like to then re-share the things I have learned in way that I understand in the hope it will in turn help others!
wilskywalker said:
Many thanks for the feedback.
Everything I know about Android has been learned from internet research and tutorials. So as far as I can I like to then re-share the things I have learned in way that I understand in the hope it will in turn help others!
Click to expand...
Click to collapse
Me too. What I've been doing is opening other people's flashable .zips and making changes that I needed to make so I can flash what I wanted to flash. I have something that I want to change in a ROM I've been helping to work on, and your guide will help in the setup process.
dsmryder said:
Me too. What I've been doing is opening other people's flashable .zips and making changes that I needed to make so I can flash what I wanted to flash. I have something that I want to change in a ROM I've been helping to work on, and your guide will help in the setup process.
Click to expand...
Click to collapse
Glad my guide helped out buddy! This community has helped me to learn so much, it's nice to give something back.
I'm looking to create a script that deletes system apps/sounds I don't want upon updating CM9 (nightly). I've taken a script from a different device and modified, what I hope is, accordingly. I only edited the mounts to: ext4 /dev/block/platform/omap/omap_hsmmc.0/by-name /system and of course the apps I want to delete upon first boot. Edit removed the detailed path as per someone's advice.
Does this look like it will accomplish my objective?
Edit: Annnnnnnd it doesn't.
Code:
#!/system/bin/sh
#mount system rw
mount -o remount, rw -t /system
#apps to delete
#send installed apps to /data/packages
pm list packages -f > /data/packages
for i in HoloSpiralWallpaper LiveWallpapers LiveWallpapersPicker MagicSmokeWallpapers Music PhaseBeam RomManager VisualizationWallpapers
do
if [ -e /system/app/$i.apk ]
then
rm -f /system/app/$i.apk
grep $i.apk /data/packages > /data/pname
pname2=$(sed 's/\(.*=\)\(.*\)/\2/' /data/pname)
pm uninstall $pname2
fi
done
rm -f /data/packages
rm -f /data/pname
#delete camera_click sound
if [ -e /system/media/audio/ui/camera_click.ogg ]
then
rm -f /system/media/audio/ui/camera_click.ogg
fi
#delete camera_focus sound
if [ -e /system/media/audio/ui/camera_focus.ogg ]
then
rm -f /system/media/audio/ui/camera_focus.ogg
fi
#mount system ro
mount -o remount,ro -t /system
Source: http://forum.xda-developers.com/showthread.php?t=1050288
CMNein said:
I'm looking to create a script that deletes system apps/sounds I don't want upon updating CM9 (nightly). I've taken a script from a different device and modified, what I hope is, accordingly. I only edited the mounts to: ext4 /dev/block/platform/omap/omap_hsmmc.0/by-name /system and of course the apps I want to delete upon first boot. Edit removed the detailed path as per someone's advice.
Does this look like it will accomplish my objective?
Edit: Annnnnnnd it doesn't.
Code:
#!/system/bin/sh
#mount system rw
mount -o remount, rw -t /system
#apps to delete
#send installed apps to /data/packages
pm list packages -f > /data/packages
for i in HoloSpiralWallpaper LiveWallpapers LiveWallpapersPicker MagicSmokeWallpapers Music PhaseBeam RomManager VisualizationWallpapers
do
if [ -e /system/app/$i.apk ]
then
rm -f /system/app/$i.apk
grep $i.apk /data/packages > /data/pname
pname2=$(sed 's/\(.*=\)\(.*\)/\2/' /data/pname)
pm uninstall $pname2
fi
done
rm -f /data/packages
rm -f /data/pname
#delete camera_click sound
if [ -e /system/media/audio/ui/camera_click.ogg ]
then
rm -f /system/media/audio/ui/camera_click.ogg
fi
#delete camera_focus sound
if [ -e /system/media/audio/ui/camera_focus.ogg ]
then
rm -f /system/media/audio/ui/camera_focus.ogg
fi
#mount system ro
mount -o remount,ro -t /system
Source: http://forum.xda-developers.com/showthread.php?t=1050288
Click to expand...
Click to collapse
Hey buddy,
This looks like an init.d script to me, this is different to Edify. init.d scripts are placed in the /system/etc/init.d folder and performs actions when you boot the phone.
Your post has piqued my interest in init.d scripting though, might have to make it my next thing to learn!
wilskywalker said:
Hey buddy,
This looks like an init.d script to me, this is different to Edify. init.d scripts are placed in the /system/etc/init.d folder and performs actions when you boot the phone.
Your post has piqued my interest in init.d scripting though, might have to make it my next thing to learn!
Click to expand...
Click to collapse
Yes init.d, wasn't sure if you'd have insight or not, but glad to have piqued your interest
Sent from my Galaxy Nexus
CMNein said:
Yes init.d, wasn't sure if you'd have insight or not, but glad to have piqued your interest
Click to expand...
Click to collapse
Sorry I couldn't help more buddy.
Interest definitely piqued though...only problem is that I am currently trying to teach myself HTML5, C++ and Edify as well...lol!
Whenever I am working on a particular project I always get sidetracked and end up trying to learn a whole new language...the only reason I started investigating Edify was so I could flash bootanimations and that led to this tutorial!
Hi, I just recently set out to teach myself making flashable zips and this was one of the more helpful guides that I found. Especially helps that it's for my phone, which eliminates a good chunk of confusion, and that it's recent, since a lot of guides that I've found seem to be a bit outdated.
However, you don't say anything about setting permissions and that's the topic that I've had the most trouble finding clear documentation for. I see it included enough that I figure that it's good practice to cover any time you're pushing files with a flash, instead of just assuming that you can run fix permissions afterwards out of habit. Without a background working with linux permissions this was all a lot to absorb, and googling terms was frustratingly unhelpful, but hopefully this goes on to help someone else.
Syntax:
Code:
set_perm (UID, GID, MODE, "PATH");
set_perm_recursive(UID, GID, DIRMODE, FILEMODE, "PATH");
So, I think I get this, but the UID and GID aspects, and the leading digit of mode still leaves me feeling like I can do this right, but not fully understand why. Here's what I'm working with:
-UID and GID set the ownership of the file/directory. List of options in android found here:
http://android-dls.com/wiki/index.php?title=Android_UIDs_and_GIDs
I've only ever seen this specified as 0 (root access) or 1000 (system access) in examples I've opened to look at. I'm not sure I entirely understand when to use 1000, but it seems like a good rule of thumb that pushing anything to a dir that usually requires root to access, set to 0.
-MODE is summed up nicely here, and it seems like for the most part you want to use a leading 0, or omit it (leave settings as is).
-And then path is just the file or directory in question. Multiple paths can be specified in the same command if desired.
So in general, a zip that pushes a file to an existing directory should include a line in the installer-script that sets the right permissions. Android seems to keep things pretty simple, so chances are you can browse to the directory and just check the permissions for anything there and use them. For example, everything in /system/app is set to rw-r--r--, which is 644. So if you wanted to push a file to /system/app, you'd use this:
Code:
set_perm (0, 0, 0644, "/system/app/[filename]");
.
If you are pushing multiple files to /system/app, you could use this:
Code:
set_perm (0, 0, 0644 "/system/app/[filename1]", "system/app/[filename2]");
Edit: I'm not sure I see an easy way to use set_perm_recursive to just cover all bases unless you're dealing with a folder that you know has only one uniform permission setting across the board (such as /system/app or /data/app). But if you're pushing a bunch of system apps, for example, you could use this:
Code:
set_perm_recursive (0, 0, 0755, 0644 "/system/app");
and it should have the same effect on the /system/app folder as just running fix permissions in recovery would do.
If I have anything wrong here please let me know. Gonna go tinker with it and try to make a working zip now as a test.
JoeSyr said:
Hi, I just recently set out to teach myself making flashable zips and this was one of the more helpful guides that I found. Especially helps that it's for my phone, which eliminates a good chunk of confusion, and that it's recent, since a lot of guides that I've found seem to be a bit outdated.
However, you don't say anything about setting permissions and that's the topic that I've had the most trouble finding clear documentation for...
...If I have anything wrong here please let me know. Gonna go tinker with it and try to make a working zip now as a test.
Click to expand...
Click to collapse
Ahh, permissions, this is my next set of commands in post #2. I really don't understand the whole permissions thing currently, so was having trouble writing it in simple terms for all to understand...if that makes sense!?
If you are happy, I would like to include your findings in the second post with all the commands (I may re-word/re-order etc for clarity as we learn more) I will of course credit you for all your help!?
I am literally trying to learn about setting permissions currently as well so its great to collaborate, this was what I was hoping for when I started this tutorial.
Ok, I have been researching Android file permissions and commands all afternoon, there really is not much documentation to be found.
I have found the info I think I need to add this to my tutorial, just need to make sense of it myself first and put it in some sort of understandable order!
Big update coming soon...
wilskywalker said:
Ahh, permissions, this is my next set of commands in post #2. I really don't understand the whole permissions thing currently, so was having trouble writing it in simple terms for all to understand...if that makes sense!?
If you are happy, I would like to include your findings in the second post with all the commands (I may re-word/re-order etc for clarity as we learn more) I will of course credit you for all your help!?
I am literally trying to learn about setting permissions currently as well so its great to collaborate, this was what I was hoping for when I started this tutorial.
Click to expand...
Click to collapse
Yes by all means feel copy and/or adapt anything I've written, and if I give the impression that I know what I'm talking about I'm happy to try and explain things as well.
I'll let you decide for yourself how much of an explanation you want to include in your guide, personally I'm going through and teaching myself a lot here, but I think I've just about figured out what you *need to know* to use the set_perm function in edify. You basically need one tool and the ability to do basic math in your head:
-Use Root Explorer to find the UID, GID, and permissions for a file. Permissions are the string that looks like some variation on 'rwxr-xr--' next to each file. Long press and choose "change owner" to get the UID and GID. You just want the number next to "owner" and "group", respectively.
-If you're replacing a file, look up these settings for the existing copy and use them. If you're adding a file, just find a file that does the same functions and copy what it has (for example, installing an app to /system/app? Just look at these settings for any other app in that directory).
-The permission string is a nine-character string. It is rwx repeated three times, each of which stands for one number. Within each rwx block, read r as 4, w as 2, and x as 1. A dash indicates 0 instead of whichever number. So, learn to read rwxrwxrwx as (4+2+1)|(4+2+1)|(4+2+1)=777; rwxr-xr-- is (4+2+1)|(4+0+1)|(4+0+0)=754, etc. This is what you enter for MODE, with one additional note:
-MODE is technically a 4-bit string, but the first bit is special and can be omitted. I've looked around, and have yet to find a single file on my phone that has anything set for the first bit. For good practice, I think it's safe to assume you can always use a leading 0 unless you know otherwise for something specific. Or, just use a 3-digit MODE, which says to leave those settings as they are (disabled by default). (I.e. 0644=644).
As another note, I personally use ES File Explorer w/ root access in my day-to-day tasks because I like the interface, but it will not give you UID or GID, or let you chance these settings.
For really understanding why this is all done this way, I think it helps to really get a grasp for why linux permissions work the way that they do (read this page) and how they are manipulated (this page). Then, consider that a lot of that is built on the assumption that there are multiple humans in the same environment, but android is built on the premise of one person and their phone. Security is largely reduced to limiting the abilities of apps (which all get their own user id, as though apps are a bunch of people signing into computers and trying to look at each others' files). Groups are pre-defined here, and the usual setup is that apps get to effectively request group permissions when installed, otherwise most things in userland don't get access to much other than writing and reading the sdcard.
Sort of makes you appreciate the work done to get phones rooted and turn them from a pallet of pre-defined actions set by OEMs into real computers.
wilskywalker said:
...
Format a partition:
To format a partition you need to use the following syntax:
Code:
format("filesystem-type", "partition-type", "device-specific-location", "0");
Example:
Code:
format("ext4", "EMMC", "/dev/block/platform/omap/omap_hsmmc.0/by-name/system", "0");
Some scripts include the "0" and some do not, not sure exactly on the function difference when included or not, again clarification would be great.
...
Click to expand...
Click to collapse
How to format boot? There is no that menu in CWM and TWRP
This guide purpose is to introduce you with edify scripting language used by Android to run updater-scripts. this guide is based on an article on this web http://www.freeyourandroid.com/guide/introdution_to_edify
why should you read this?
this guide will be usefull for someone who want to make a patch, install an apk, or modify a rom with a simple update.zip file. most of android devices are using edify script instead of amend script. this guide will be applicable in almost every android devices. a simpe example of these script we can found on installer for theme. while the complicated one is aroma installer which are used for GUI installation for rom.
below are examples of functions used in an Edify Updater-script:
Function Name: mount
Function Syntax: mount(fs_type, partition_type, location, mount_point)
Parameter Details:
fs_type = yaffs2 ext4 ext2 ext3 rfs vfat
partition_type="MTD" | "EMMC"
location = partition | device
mount_point = target folder to mount FS.
Action: Mounts a filesystem to the defined mount point
Returns: The mount point if successful, null if failed
Click to expand...
Click to collapse
/system partition is automatically mounted. this command is needed when you need to mount another partition beside /system or in any condition when recovery mode failed to mount your system partition. the system's location is /dev/stl9, cache /dev/stl10, and data /dev/stl11 (all partition type commonly rfs, will be differ if you have custom kernel or rom). if you want to mount sdcard, the location is mmcblk0pX with x is the partition number.
Function Name: is_mounted
Function Syntax: is_mounted(mount_point)
Parameter Details: mount_point = string, mount point to check if is mounted
Action: Checks if a filesystem is mounted.
Returns: The mount point if mounted, otherwise, returns null
Click to expand...
Click to collapse
Function Name: unmount
Function Syntax: unmount(mount_point)
Parameter Details: mount_point = string, mount point to unmount
Action: Unmounts the filesystem
Returns: The mount point that was dismounted if successful, otherwise it returns null
Click to expand...
Click to collapse
Function Name: format
Function Syntax: format(fs_type, partition_type, location)
Parameter Details:
fs_type = string,yaffs2 ext4 ext2 ext3 rfs vfat
partition_type= string, "MTD" | "EMMC"
location = string, partition | device
Action: Formats a filesystem as specified
Click to expand...
Click to collapse
Function Name: delete
Function Syntax: delete(file1, file2, ..., fileN)
Parameter Details: string, file to delete
Action: Deletes a file. Must specify at least one file; multiple files can be specified as multiple arguments
Click to expand...
Click to collapse
Function Name: delete_recursive
Function Syntax: delete_recursive(dir1, dir2,...,dirN)
Parameter Details: string, directory to recursively delete
Action: Deletes a folder and all of its contents. At least 1 directory must be specified; multiple directories can be specified as additional arguments.
Click to expand...
Click to collapse
Function Name: show_progress
Function Syntax: show_progress(frac, sec)
Parameter Details:
frac = fraction of progress completed
sec = total seconds
Action: Displays flashing progress in the recovery system
Click to expand...
Click to collapse
Function Name: set_progress
Function Syntax: set_prograss(frac)
Parameter Details: frac=fraction of progress
Click to expand...
Click to collapse
Function Name: package_extract_dir
Function Syntax: package_extract_dir(package_path, destination_path)
Parameter Details:
package_path = string, directory in package to extract
destination_path = string, target point to extract files to
Action: Extract the all of the files in a directory in the package to the target specified.
Click to expand...
Click to collapse
this command is the usefull one for app/mod installation. don't worry about same filename, the newer one will automatically delete the old one. if you want to make your file easier to use, you can use 'package_extract_dir("system", "/system")'. in this way, you don't need to make multiple command to copy your files and folders into system partition. everything under system folder will be copied into your system partition. see my example file below for further details.
Function Name: package_extract_file
Function Syntax:
package_extract_file(package_path)
or
package_extract_file(package_path, destination_path)
Parameter Details:
package_path = string, file in the package you want to extract
destination_path, target folder to extract the file to.
Action: Extracts a single file from your update package to the target specified
Click to expand...
Click to collapse
Function Name: file_getprop
Function Syntax: file_getprop(file, key)
Parameter Details:
file = string, filename to check
key = string, key in file to return the value of
Action: Checks for a value in a file?
Click to expand...
Click to collapse
Function Name: symlink
Function Syntax: symlink(target, src1, src2, ..., srcN)
Parameter Details:
target = string, the target of the symbolic link
srcX = the symbolic link to create that points to the target
Action: Unlinks any existing symbolic links before creating the new symbolic links.
Click to expand...
Click to collapse
Function Name: set_perm
Function Syntax: set_perm(uid, gid, mode, file1, file2, ..., fileN)
Parameter Details:
uid = user id
gid = group id
mode = permission mode
fileX = file to set permission on
Action: Sets permissions of a file or set of files specified. At least one file must be specified (the first four parameters are required)
Click to expand...
Click to collapse
Function Name: set_perm_recursive
Function Syntax: set_perm_recursive(uid, gid, dirmode, filemode, dir1, dir2, ...dirN)
Parameter Details:
uid = user id
gid = group id
dirmode = permission to set to directories contained within the specified directory
filemode = permission to set to files contained within the specified directory
dirX = directory to set permission on
Action: Set permissions of a directory or set of directories and all files and folders within them. At least one directory must be specified (The first 5 parameters are required)
Click to expand...
Click to collapse
Function Name: getprop
Function Syntax: getprop(key)
Parameter Details: key = string, the property you want the system to return
Action: This function returns the value of the property specified. This is used to query platform information from the build.props file.
Click to expand...
Click to collapse
Function Name: write_raw_image
Function Syntax: write_raw_image(file, partition)
Parameter Details:
file - string, the source .img file to be read from
partition - string, the destination partition to write the .img file to
Description: This function writes an img file to a partition.
Click to expand...
Click to collapse
this command usually used to install kernel. somehow it doesn't work for sgy. you'll need to run bmlunlock program to install the kernel on sgy. check faqbly's kernel installer for further information.
Function Name: apply_patch
Function Syntax: apply_patch(srcfile, tgtfile, tgtsha1, tgtsize, sha1_1, patch_1, ..., sha1_x, patch1_x)
Parameter Details:
srcfile - string, source file to be patched (file to read in)
tgtfile - string, destination file to write the patched file to
tgtsha1 - string, sha1 hash of the target file as it should hash out after the patches apply properly
sha1_x - string, sha1 hash of the patch data that's to be written to the target file
patch1_x- string, actual patch to apply to the target file
Action: This function is used to apply patches to a file.
Click to expand...
Click to collapse
Function Name: apply_patch_check
Function Syntax: apply_patch_check(file, sha1_1, ..., sha1_x)
Parameter Details:
file - string, file to check
sha1_x - hash to check for?
Action: Either checks if a file has been properly patched, or checks if a file can be patched. Need to check the source code of the "applypatch_check" function that is called from here.
Click to expand...
Click to collapse
Function Name: apply_patch_space
Function Syntax: apply_patch_space(bytes)
Parameter Details: bytes = number of bytes to check for
Action: Checks the cache to verify that there is enough space to write the patched files to it and returns something. Need to test this function to verify.
Click to expand...
Click to collapse
Function Name: read_file
Function Syntax: read_file(filename)
Parameter Details: filename - string, the filename to read the contents of
Action: This function returns the contents of a file.
Click to expand...
Click to collapse
Function Name: sha1_check
Function Syntax:
sha1_check(data)
or
sha1_check(data, sha1_hex, ..., sha1_hexN)
Parameter Details:
data - the contents of file to calculate the sha1 hash of - must be in the read_file format
sha1_hexN - A particular sha1_hex string that you want the file data to match
Action: If only data is specified, then the function returns the sha1_hex string of the data. The optional parameters are used if you want to verify that the file you are checking for is one of a list of hashes. It reutrns the hash it matches, or returns nothing if it doesn't match any of the mentioned hashses.
Click to expand...
Click to collapse
Function Name: ui_print
Function Syntax: ui_print(msg1, ..., msgN)
Parameter Details: msg - String, message to be outputted to the user during the patch process
Action: This function prints/echos a message to the console while the script is running. At least 1 parameter needs to be specified, you can specify additional msg parameters and they will be concatenated to the output.
Click to expand...
Click to collapse
Function Name: run_program
Function Syntax: run_program(prog, arg1, .., argN)
Parameter Details:
prog - string, program to execute
argN - string, arguments for the program that is being executed
Description: Executes a program with the arguments specified. Returns a string, I assume it is the buffer of the stdout of the program executed, need to test.
Click to expand...
Click to collapse
Function Name: ifelse
Function Syntax: ifelse(condition, truecondition, falsecondition)
Parameter Details:
condition - an expression to evaluate
truecondition - Edify script block to execute if true
falsecodnition - Edify script block to execute if false
Description: This is the if-then construct of the Edify scripting language. The truecondition or falsecondition arguments can be a single edify command or a script block. Script blocks can be formed by enclosing the parameter with parenthesis, and seperating the commands with semicolons\
Click to expand...
Click to collapse
Function Name: abort
Function Syntax: abort()
Parameter Details: no paremeters
Description: Aborts script execution.
Click to expand...
Click to collapse
Function Name: assert
Function Syntax: assert(condition)
Parameter Details: condition - boolean
Description: If condition evaluates to false, stops script execution, otherwise continues processing.
Click to expand...
Click to collapse
to run the script you'll need a binary file for edify script. you can find an example of the update.zip with the binary file here http://www.mediafire.com/download.php?rti5lc5358w51nn
this one is an example of updater script for roms
http://www.mediafire.com/download.php?nxgiz0993brp9re
NOTE:
1. that file used to enable bootanimation on dxlb or newer firmware. it is still in progress so don't install it. you can use that file as a base to create your own installer zip with edify script.
2. don't use notepad to edit the file. you can use notepad++ instead.
3. all syntax inside () is always within ". for an example, syntax for mount is 'mount(fs_type, partition_type, location, mount_point)', so if we want to mount the system partition we'll type 'mount("rfs", "EMMC", "/dev/block/stl19", "/system")'.
Wow, it's so long, but i try.
whoaa...you're right. I don't know that this post will be this long (lol)
btw, don't worry about those script. in most cases you only need 2-3 of them.
wow this is really awesome!! I would appreciate if you could give examples of commands commonly used
you're already used to work with custom roms. everything you need to know is already inside your rom. for a simple updater script used to install app or file I always use these script.
Code:
ui_print("Installing");
package_extract_dir("system", "/system");
ui_print("Completed");
ui_print("have a nice day");
ui_print("[email protected]");
bookmarked
will try this sooner or later
this is epic dude! I really needed em! thanx!!
Nice ....tanks
Pressed :thumbup::thumbup::thumbup:
Thanks. Nice one again kuro
Sent from my GT-S5360 using xda premium
thanks ....... really great guide...
Sent from my GT-S6102 using Tapatalk 2
awesome,im waiting so long for a good guide.thanks for this bro
GREEEEEEEETZ!!!
thanks for your support guys
Hi Kuro, great tutorial as usual.
Listen, any idea why in the S5570I stock recovery with the stock kernel the format command requires 4 parameters instead of 3?
Code:
format () expects 4 args. got 3
Do you know which one is the correct syntax? Also, what's the difference between EMMC and MTD??
I feel more and more about throwing this phone out of the window. The iPhone 5 is very tempting.
that problem comes from the binary file. I've found at least three version of the binary file. in my guide the syntax is format(fs_type, partition_type, location) or in an example format("EXT4", "MTD", "/dev/block/stl19") but some binary file need to specify more about the partition. the syntax in an example will be format("EXT4", "MTD", "/dev/block/stl19", "/system"). the differences between EMMC and MTD is the hardware of the memory device. an sdcard is an EMMC device, so that we should use EMMC. however, different phone vendor have different type of internal memory device. MTD is commonly used but in some case they use EMMC.
you can try this one. http://www.mediafire.com/download.php?if08rvj22mw9abp it comes from maroc.
Ok, thanks. I've found another couple of update-binary as well in the meantime. One with the 3 parameters format coming from a CM for the Galaxy Min/Pop/Next. Another with a TWO (!!) parameters format, or at least it seems so, coming straight from the Samsung latest CSC for my phone (you know about that :laugh. In the CSC updater-script the syntax is following (but it is commented out):
Code:
# format("MTD", "system");
# mount("MTD", "system", "/system");
Do they really love to make the most trivial things impossible with Android? Not even a well defined, fixed syntax? At least they could rename the update-binary. This software is blasphemy for a computer scientist!!! (sorry. It really makes me needing to blow out).
Code:
# format("MTD", "system");
# mount("MTD", "system", "/system");
it seems that there's a differences between the binary file comes from the factory the one come from community. the script from a factory always in a form like above. the one comes from community always comes in a form like format(fs_type, partition_type, location). IMO its not the factory fault if we found a lot of binary version. everyone can make their own update binary if they know the source code.
Oh, I get it now. I stupidly thought they all came from Google. Let's say that a mechanism to use different names for the update-binary could be put in place (or there is one?). So one could name the update-binary with its version. Like CM7.2-binary, Merruk-binary and so on.
I'll check if I can run them from the phone, to see if they give out any version info, and maybe an help with a list of commands, so I can get the one I think is the best and use it once for all. We keep getting flashing problems in our phone, so we must investigate the issue at 360 degrees. Thanks again.
I doubt that. we can't rename the update binary into something else. even aroma which is using his own binary file still have to rename it to update-binary. in recovery mode it seems that when we flash a file, system will command "exec update-binary". while in update-binary it has something like "see the content of updaters-script then run it". if we made our own binary file, we can easily modify the second part. in aroma, the dev change the second part into "see the content of aroma-config then run it". in other hand, the first part is stored inside our recovery stuff. modfying it is hard and a lil bit risky so that no one ever interested to do it.
Kuro, do you own a thread where we can freely discuss about kernel compilations? I have 4 or 5 questions, nothing complex, just about the gzip/lzma compression and the RAM disk size limit that are driving me crazy. A couple of kernels compile but then when I install them I get blank screens. With other details that I'd like to specify in a dedicated thread.
/off-topic
you can discuss that everywhere
you can make a new one or simply chat like as we did in maroc's thread. ah yes, our kernel max size is about 5024kb or about 4,88Mb. quite small indeed, but we can gain more spaces when we use lzma compression. btw, I got less problem when compiling a zImage file. as long as we didn't make a major change on the source, our kernel will eventually working. nevertheless, I got a lot of problem when trying to modify ramdisk. even a slight change on ramdisk in several occasieon made the kernel didn't work. you might want to test your zImage with a ramdisk from stock kernel or a working custom kernel available for your device to check whether if the problem comes from zImage or ramdisk. if you want to know more about ramdisk modification you can ask savie or yahya (maroc). they sure know better than me in that aspect.
I was wondering if I could mount an empty.img file so that I could add executable into it and chmod 777 them or what ever the number is maybe 666.
Then I would add the location to my $PATH variable in the "/system/etc/mkshrc" file so I could execute those programs from any directory.
What say you?
Has this been done before?
It works!
Well, I wen ahead and tried it out, I figured "What the hell, its not like I have to format my sd card." It worked!
So what I did
1) I changed directories to Downloads. ("cd ~/Downloads")
2) I created a directory for my image in Downloads, and moved into it. (mkdir image && cd ./image)
3) I created an empty 4 gig image called apps.img using dd ("dd if=/dev/zero of=apps.img bs=1MB count=0 seek=4096")
4) I formated it to ext2 ("mke2fs -F apps.img")
5) I used adb to push it to my phone ("adb push ~/Downloads/image/apps.img /storage/sdcard0/Download/")
6) Then on my phone as su I mounted the image ("mount -o loop '/storage/sdcard0/Download/apps.img' '/data/local/mnt' ") {with single quotes around the directories, the double quotes wrap the whole actual command, you don't need them} [EDIT: I used bash on the phone to do this, ie I "su" [enter] ; "bash" [enter] ; "THE ABOVE COMMAND" [enter]
7) To test I used the python interpreter as my executable so I created a folder in /data/local/mnt called apps,(note* I should have made that folder on my pc before I pushed it to my phone to ensure that the foder was actually in the apps.img file.) I created two more folders "bin" and "lib" using "File Manager" on my phone. I then moved what I needed to run python into those folders (though you'll see I forgot something)
8) I added PYTHONHOME PYTHONPATH and added the bin folder I created to $PATH in the /system/etc/bash/bashrc file (Ask and I'll explain). If you don't have bash the mkshrc file is located "/system/etc/mkshrc" on your phone (if its Sprint SGSIII) adding environment variable there will accomplish the same thing, sorta.
9) I connected my phone to pc w/usb, opened up a teminal on pc, started an adb shell
10)........
Code:
[email protected]:~$ adb devices
List of devices attached
xxxxxxxx device
[email protected]:~$ adb shell
[email protected]:/ $ su
[email protected]:/ # bash
void endpwent()(3) is not implemented on Android
localhost / # which python
/data/local/mnt/apps/bin/python
localhost / # python
'import site' failed; use -v for traceback
Python 2.6.2 (r262:71600, Mar 20 2011, 16:54:21)
[GCC 4.4.3] on linux-armv7l
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import sys
>>> import math
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named math
>>> import io
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/manuel/AptanaStudio3Workspace/python-for-android/python-build/output/usr/lib/python2.6/io.py", line 63, in <module>
ImportError: No module named _fileio
>>>
localhost / # exit
[email protected]:/ # ^D
[email protected]:/ $ ^D
[email protected]:~$
I'm thinking that if I can get my mkbootimg tools to work, I would mod an init script to mount the apps.img and then create links for each file in say '/mnt/apps/bin' create a link*in '/system/bin'. this should allow for phones with small or near full system partition install stuff like busybox or your own pprograms. More usefull for a developer.
I wanna try this with pythonforandroid, if I can make python and its modules. accessible during early init or just before the boot process finishes in general, and use it to run python, maybe python can handle boot in a different way, or maybe just one specific. function you might needs.
one big question I have. Does the pythonforandroid interpreter run ontop of the D VM?
Edge-Case said:
I'm thinking that if I can get my mkbootimg tools to work, I would mod an init script to mount the apps.img and then create links for each file in say '/mnt/apps/bin' create a link*in '/system/bin'. this should allow for phones with small or near full system partition install stuff like busybox or your own pprograms. More usefull for a developer.
I wanna try this with pythonforandroid, if I can make python and its modules. accessible during early init or just before the boot process finishes in general, and use it to run python, maybe python can handle boot in a different way, or maybe just one specific. function you might needs.
one big question I have. Does the pythonforandroid interpreter run ontop of the D VM?
Click to expand...
Click to collapse
I don't think so. All command-line programs I know of interface directly with the kernel.
Sent from my S3 on Sense 5 (you jelly?)
CNexus said:
I don't think so. All command-line programs I know of interface directly with the kernel.
Sent from my S3 on Sense 5 (you jelly?)
Click to expand...
Click to collapse
So getting an extended set of Linux (kernel) cli programs working with Android (kernel) is a matter of having the nessissary libraries, kernel prereq., and being compiled for the target processor?
From what I have read, the Android kernel has been cut back so far from the original Linux kernel that its difficult to port "Linux apps" to Android. Something about a slimmed down version of the GNU C/C++ libraries and the Android kernel being designed to run mostly Dalvik.
I haven't tried directly running any "Linux app" (already compiled for arm) on Android yet, but my game plan for that test was to load up an .img file with the nessissary execs, libs, config, etc files (as ext3 this time) and running some scripts that get the paths variables set up and then execute the script, I wrote a short Bash script that sets up python variables and adds others to PATH etc, and it worked, I had python on the img and the img mounted to /mnt/myside and python ran but with some errors, I need to get the variables right, its driving me mad, if its not this its that, last time it was the basic "help()" command not being declaired or something.
Well thats my plan, either these "Linux apps" run on Android without problem or I am going to A) write my own kernel to be compatible with Android/ cli Linux or I am going to get as much source code as I can and practice the art of compiling against Android and/or patching the code when/where nessissary.
We'll see what happens, I've done enough today/night.
Has anyone installed self signed ROMs using the method described in this blog http://mjg59.dreamwidth.org/31765.html? The goal is to install a self signed image that allows you to only install images signed by a key you generate yourself to avoid the giant security hole caused by custom recoveries that don't check signatures.
Here's the overview :
First: Unlock your bootloader. /data will be wiped.
Second: Get a copy of the stock recovery.img for your device.
Third: Grab mkbootimg from here and build it. Run unpackbootimg against recovery.img.
Fourth: Generate some keys. Get this script and run it.
Fifth: zcat recovery.img-ramdisk.gz | cpio -id to extract your recovery image ramdisk. Do this in an otherwise empty directory.
Sixth: Get DumpPublicKey.java from here and run it against the .x509.pem file generated in step 4. Replace /res/keys from the recover image ramdisk with the output. Include the "v2" bit at the beginning.
Seventh: Repack the ramdisk image (find . | cpio -o -H newc | gzip > ../recovery.img-ramdisk.gz) and rebuild recovery.img with mkbootimg.
Eighth: Write the new recovery image to your device
Ninth: Get signapk from here and build it. Run it against the ROM you want to sign, using the keys you generated earlier. Make sure you use the -w option to sign the whole zip rather than signing individual files.
Tenth: Relock your bootloader
Eleventh: Boot into recovery mode and sideload your newly signed image.
Click to expand...
Click to collapse
I have the 8" Tab (SM-T310) and want to try it. Anyone know if there's an official source of stock recovery images? I have the 4.4.2 update.zip from an OTA update I never installed (my tablet's still on stock 4.2.2), can the recovery somehow be extracted from that? Since I'm still on 4.2.2, I don't need to unlock the bootloader, right? Can I use Heimdall to put the new recovery on the tab?
Here's my step by step if anyone else is crazy enough to try this.
First: Unlock your bootloader. /data will be wiped.
Click to expand...
Click to collapse
Skipped so far, running 4.2.2 which from what I've been reading I think isn't locked.
Second: Get a copy of the stock recovery.img for your device.
Click to expand...
Click to collapse
Grabbed the update from Kies (on Windows VM):
Agreed to Kies update when I plugged in the tablet.
Watched my temp folder (from windows search bar enter %temp%) until tmp*.tmp.zipfolder showed up (wait for tmp*.tmp file to download and get converted to a zip, then it unzips into zipfolder automatically), unplugged the tablet immediately as it booted into download mode, copied tmp*.tmp.zipfolder somewhere else (desktop) before Kies cancelled the update process and deleted the temp files.
Third: Grab mkbootimg from here and build it. Run unpackbootimg against recovery.img.
Click to expand...
Click to collapse
Source is on github: osm0sis/mkbootimg From here on I'm using linux.
cd into mkbootimg folder and run "make"
run
Code:
./unpackbootimg -i recovery.img -o rec
(rec is the folder I unpacked to, make the folder first or it'll segfault)
Fourth: Generate some keys. Get this script and run it.
Click to expand...
Click to collapse
Saved script (posted below, can't post links yet...) to "make_key.sh", then
Code:
chmod +x make_key.sh
to make it executable.
Ran script, usage: /make_key.sh <name> <subject> [<keytype>]
subject must be in the format "/C=GB/ST=London/L=London/O=Global Security/OU=IT Department/CN=example.com" (look up "Create CSR using OpenSSL Without Prompt (Non-Interactive)" for what all that means)
Fifth: zcat recovery.img-ramdisk.gz | cpio -id to extract your recovery image ramdisk. Do this in an otherwise empty directory.
Click to expand...
Click to collapse
Make a directory, cd into it, run
Code:
zcat recovery.img-ramdisk.gz | cpio -id
with recovery.img-ramdisk.gz pointing to where your file is (mine was ../recovery.img-ramdisk.gz).
Sixth: Get DumpPublicKey.java from here and run it against the .x509.pem file generated in step 4. Replace /res/keys from the recover image ramdisk with the output. Include the "v2" bit at the beginning.
Click to expand...
Click to collapse
Not sure why he's using a different DumpPublicKey.java than the one that's in the mkbootimg git... but it's in the "libmincrypt/tools/DumpPublicKey.java" that was in the mkbootimg git, I didn't see any differences using diff.
Running
Code:
javac DumpPublicKey.java
tells me I need BouncyCastleProvider. And now I'm stuck.
Downloaded the Bouncy Castle jar and put it in the same folder as DumpPublicKey.java and ran
Code:
javac -cp bcprov-jdk15on-152.jar DumpPublicKey.java
echo "Main-Class: DumpPublicKey" > manifest.txt
jar cvfm DumpPublicKey.jar manifest.txt DumpPublicKey.java
Then running
Code:
java com.android.dumpkey.DumpPublicKey
gave me various class not found errors and I ended up with a file structure like this trying to make it find the bouncy castle. Annnnnd I'm stuck.
.
├── bcprov-jdk15on-152.jar
├── com
│** └── android
│**** └── dumpkey
│****** ├── bcprov-jdk15on-152.jar
│****** ├── DumpPublicKey.class
│****** ├── DumpPublicKey.jar
│****** ├── DumpPublicKey.java
│****** └── manifest.txt
├── DumpPublicKey.class
├── DumpPublicKey.java
└── manifest.txt
I obviously don't know how to java at all.... any suggestions?
I just needed to put the "-cp" in the java command as well as the javac command.
Code:
java -cp .:./bcprov-jdk15on-152.jar com.android.dumpkey.DumpPublicKey
The full command is
Code:
java -cp .:./bcprov-jdk15on-152.jar com.android.dumpkey.DumpPublicKey /path/to/key.x509.pem > keys
After deleting stuff and testing, I found I only needed files in these locations:
.
├── bcprov-jdk15on-152.jar
├── com
│** └── android
│** └── dumpkey
│** ├── DumpPublicKey.class
│** └── DumpPublicKey.jar
├── DumpPublicKey.java
└── manifest.txt
Then I replaced the "/res/keys" file with the "keys" file I just made.
Seventh: Repack the ramdisk image (find . | cpio -o -H newc | gzip > ../recovery.img-ramdisk.gz) and rebuild recovery.img with mkbootimg.
Click to expand...
Click to collapse
From the folder that the ramdisk image was extracted into, run
Code:
find . | cpio -o -H newc | gzip > ../recovery.img-ramdisk.gz
mkbootimg needs offsets that were printed when I ran unpackbootimg:
BOARD_KERNEL_CMDLINE
BOARD_KERNEL_BASE 10000000
BOARD_NAME
BOARD_PAGE_SIZE 2048
BOARD_KERNEL_OFFSET 00008000
BOARD_RAMDISK_OFFSET 01000000
BOARD_TAGS_OFFSET 00000100
Still in the folder where I unpacked the recovery image (mkbootimg is one directory up)
Code:
../mkbootimg --kernel recovery.img-zImage --ramdisk recovery.img-ramdisk.gz --base 10000000 --pagesize 2048 --kernel_offset 00008000 --ramdisk_offset 01000000 --tags_offset 00000100 -o newrecovery.img
Eighth: Write the new recovery image to your device
Click to expand...
Click to collapse
This thread says Heimdall 1.4.0 works for T31x: http://forum.xda-developers.com/showthread.php?t=2522091
Oh how nice, Heimdall 1.4.1 is in my package manager.
Connect device and restart in download mode (held the "HOME" button, "Volume Down" and "Power" all at the same time (for about 5 seconds). Hit the "Volume Up" to enter download mode when prompted.)
Code:
sudo heimdall flash --verbose --no-reboot --RECOVERY recovery.img
Ninth: Get signapk from here and build it. Run it against the ROM you want to sign, using the keys you generated earlier. Make sure you use the -w option to sign the whole zip rather than signing individual files.
Click to expand...
Click to collapse
signapk is on github in android/platform_build/tools. And it gives more bouncy castle errors. Put bcpkix-jdk15on-152.jar and bcprov-jdk15on-152.jar in the signapk folder and run
Code:
javac -cp .:./bcprov-jdk15on-152.jar:./bcpkix-jdk15on-152.jar SignApk.java
echo "Main-Class: SignApk" > manifest.txt
jar cvfm SignApk.jar manifest.txt SignApk.java
make directory com/android/signapk/ and move in all the *.class files and the SignApk.jar
Code:
java -cp .:./bcprov-jdk15on-152.jar:./bcpkix-jdk15on-152.jar com.android.signapk.SignApk /path/to/key.x509.pem /path/to/key.pk8 ../original.zip ../signed.zip
I just replaced the recovery in the OTA with the recovery with my keys from earlier, then rezipped it before signing. And after starting sideloading I noticed I left the "-w" option off the signing command, so I made another file with "-w".
Sideloaded with:
Code:
adb reboot recovery
(then volume down to select apply update from ADB)
adb sideload signed.zip
Then saw:
loading: 'update.zip'
sending: 'update.zip' <completion percent>
(might have to do some adb stopping/starting or adb usb beforehand to make it work, I was getting "error: closed" at first. If it just sits on "loading" something's wrong, as I found out last night when I waited hours for "loading" to turn into something else.)
After the file finished uploading, In terminal I got the response "* failed to write data 'protocol fault (couldn't read status): Success' *"
And on the device I got (for a few seconds before it showed only the sad robot on its back). Then after a couple minutes it rebooted back into recovery.
E: Footer is wrong
E: Footer is wrong
E: Signature verification failed
Installation aborted
So I tried again, but this time selected the option to wipe the cache before adb sideload. Then I tried again, using the update.zip that I forgot to add the "-w" option when signing. So far all fails.
I have no idea if the issue is how the recovery was flashed or what, so I'm doing it manual...
Code:
adb shell
su
#first backup current recovery
dd if=/dev/block/mmcblk0p10 of=/sdcard/recoverybk.img
adb pull /sdcard/recoverybk.img
# now push and dd my signed recovery
adb push /sdcard/recovery.img
dd if=/sdcard/recovery.img of=/dev/block/mmcblk0p10
And still having the same signature verification issue.
Edit: I went back and unpacked the recovery I pulled off the device (unpackbootimg), and it looks like somehow I managed to run mkbootimg with the old ramdisk image.
New problem with the new recovery: "E:failed to load keys". Maybe it had something to do with using ec keys just for funsies. So I remade the keys with the default, redumped into the ramdisk image, rebuild/reflashed recovery, resigned the update.zip. Now it's back to the "footer is wrong, signature verification failed" error.
But this is still trying to use the OTA 4.4.something update, not a full ROM. What's it do with Cyanogen?
First attempt: Download Cyanogen and sideload it (unmodified)
E:failed to verify whole-file signature
E:signature verification failed
Ok, expected behavior, it's not signed with my keys.
Attempt 2: sign Cyanogen with my keys and sideload it
Install from ADB complete!! Success!!
@user0002,
great thread! have you considered assembling a toolkit/suite for this process?
Thanks.
m
moonbutt74 said:
@user0002,
great thread! have you considered assembling a toolkit/suite for this process?
Thanks.
m
Click to expand...
Click to collapse
Hmmm... that's a good idea.