Develop Bugs

Google Android and Linux for Kaiser Volume II

The original thread:http://forum.xda-developers.com/showthread.php?t=396782 needs an abridged version.
==================================================
Go to http://www.androidonhtc.com/ if you're just starting as it has the latest info. This thread is to highlight the info from the original thread only.
Latest Builds
Port Status
==================================================
Compiling Android Kernel for Kaiser
Modify initrd.gz files and CPIO handling
system.img mounting, editing and rebuilding with ext2/3
system.img mounting, editing and rebuilding with cramfs
Howto: Pull from git (new/update/resync)
[WIP] Configuring WiFi Interface
==================================================

system.img mounting, editing and rebuilding with ext2/3
seidler2547: Post:
Actually I've played with Android a bit for now, and I changed to ext3. It doesn't only work - it's much faster, too! Startup time during the blinking android is about half of what it was before.
How-To:
Code:
Code:
cd /tmp
# prepare dirs
mkdir a-sys
mkdir a-ext
# prepare image
dd if=/dev/zero of=/where/is/sdcard/system.img.new bs=1M count=64
mkfs.ext3 /where/is/sdcard/system.img.new
# mount old image and copy to new
mount -o loop /where/is/the/system.img a-sys
mount -o loop /where/is/sdcard/system.img.new a-ext
cp -a a-sys/* a-ext/
Now you can unmount the old image and happily edit in the new image. Don't forget to rename the system.img.new to system.img (after you have unmounted it).
In your initrd, in file init, where it says
Code:
losetup /dev/block/loop1 /sdcard/system.img
...
mount -t cramfs -o ro,noatime,nodiratime /dev/block/loop1 /system
change "-t cramfs" to "-t ext2" or "-t ext3". You can also change the path (/sdcard/system.img) there.
Click to expand...
Click to collapse

Modify initrd.gz files and CPIO handling
dcordes: Post:
There is no magick in the initrd files. They are .cpio.gz files, gzipped cpio balls. To extract a .cpio.gz file named initrd-android.cpio.gz simply do
Code:
gunzip initrd-android.cpio.gz && cpio -i < initrd.android.cpio
Then you have the extracted rootfs. The reverse way would be, assuming you are inside your rootfs folder:
Code:
find ./ | cpio -H newc -o | gzip > ../my-initr-android-with-custom-stuffs.cpio.gz
And yes, you can remove and add applications you find that way.
Click to expand...
Click to collapse

system.img mounting, editing and rebuilding with cramfs
dzo: Post:
Hi, you can't just use mkcramfs on the system folder because the permissions will be wrong. This is the script I use:
Code:
Code:
out/host/linux-x86/bin/genext2fs -d out/target/product/generic/system -b 80000 -a system.ext2
mount -o loop system.ext2 /mnt/system
cp /mnt/system/usr/keychars/qwerty2.kcm.bin /mnt/system/usr/keychars/vogue-ts.kcm.bin
cp com.google.android.maps.jar /mnt/system/framework
cp Maps.apk Street.apk /mnt/system/app
mkfs.cramfs /mnt/system system.img
umount /mnt/system
#pcp system.img :/Storage\ Card/system.img
This also puts the maps app in (just copy from one of my images) and the vogue keymap. Without the source for the ril you will also need to copy my RIL (libreference-ril.so).
Click to expand...
Click to collapse

[WIP] Configuring WiFi Interface
This has been able to initialize the interface, assign arbitrary IP addresses but can not go further at the moment.
Code:
# ifconfig tiwlan0 192.168.1.100
# ifconfig tiwlan0 up
error: SIOCSIFFLAGS (Cannot assign requested address)
# ifconfig tiwlan0
tiwlan0: ip 192.168.1.100 mask 255.255.255.0 flags (down broadcast multicast)
dmesg will show:
Code:
wlan: no version for "struct_module" found: kernel tainted.
TIWLAN: Driver loading
trout_wifi_power: 1
trout_wifi_reset: 0
trout_wifi_set_carddetect: 1
TIWLAN: Found SDIO control (vendor 0x104c, device 0x9066)
TIWLAN: Driver initialized (rc 0)
TIWLAN: Driver loaded
Android's built-in wireless settings seem to disable the interface beyond just interfering with it, therefore it's best to stay with terminal and using 'ash' will give you a shell with command history (up/down scroll).
markya23: Post:
Need to create a folder in you system image package called /etc/wifi and copy tiwlan.ini, wpa_supplicant.conf and fw1251r1c.bin.
Need to copy the wlan.ko to /lib/modules in the system image (create the dir if required).
Create the new system image and boot Android. Start the dev console and type:
Code:
cp /system/etc/wifi/wpa_supplicant.conf /data/misc/wifi/wpa_supplicant.conf
insmod /system/lib/modules/wlan.ko
wlan_loader -f /system/etc/wifi/Fw1251r1c.bin -e /proc/calibration -i /system/etc/wifi/tiwlan.ini
cd /data/local/tmp
wpa_supplicant -f -Dtiwlan0 -itiwlan0 -c/data/misc/wifi/wpa_supplicant.conf &
ifconfig tiwlan0 192.168.1.100 netmask 255.255.255.0
ifconfig tiwlan0 up
Click to expand...
Click to collapse

Compiling Android Kernel for Kaiser
dwaradzyn: Post:
Here are brief instructions on how to compile android kernel for Kaiser from git.linuxtogo.org repository. I assume that running OS is Linux and it has everything required to build x86 or ia64 kernel. Beside that latest git software should be installed. The shell is assumed to be bash.
1. Let's start with creating a directory for kernel in home directory:
Code:
mkdir ~/android-kernel
cd android-kernel
2. Next thing is to get the sources from repository. To make it happen (this could take a while, it downloads 280MB):
Code:
git clone git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git
OUTPUT:
Code:
Initialized empty Git repository in /home/user/android-kernel/kernel/.git/
remote: Counting objects: 908251, done.
remote: Compressing objects: 100% (153970/153970), done.
remote: Total 908251 (delta 755115), reused 906063 (delta 753016)
Receiving objects: 100% (908251/908251), 281.86 MiB | 292 KiB/s, done.
Resolving deltas: 100% (755115/755115), done.
Checking out files: 100% (22584/22584), done.
3. The htc-msm branch is of our interest (again it could take a few seconds):
Click to expand...
Click to collapse
*** Update, poly_poly-man states we are working off of htc-vogue not htc-msm. I'm leaving the original code here but I would urge you to modify the next line as poly has suggested:
Code:
cd kernel
git checkout -b htc-msm origin/htc-msm
OUTPUT:
Code:
Branch htc-msm set up to track remote branch refs/remotes/origin/htc-msm.
Switched to a new branch "htc-msm"
4. Let's take care of arm toolchain. Download this file (64MB) into ~/android-kernel:
Code:
[url]http://www.codesourcery.com/gnu_toolchains/arm/portal/package2549/public/arm-none-linux-gnueabi/arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2[/url]
Unpack it:
Code:
cd ~/android-kernel
tar xjf arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
5. Compile the kernel
Prepare default .config for Kaiser:
Code:
cd ~/android-kernel/kernel
make htckaiser_defconfig ARCH=arm
OUTPUT:
Code:
........
lots of output
........
# configuration written to .config
#
And finally compile the kernel to get zImage (takes a minute or two):
Code:
export PATH=~/android-kernel/arm-2008q1/bin:$PATH
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
OUTPUT:
Code:
........
lots of output
........
Kernel: arch/arm/boot/zImage is ready
Now copy ~/android-kernel/kernel/arch/arm/boot/zImage to your phone and play with it.
Some ending tips:
A. You can compile earlier versions of sources in repository. To do that click on one of "commit" links on page:
Code:
http://git.linuxtogo.org/?p=groups/mobile-linux/kernel.git;a=summary
and read commit id (for example: f9d1bcea9342348623f5a57588044f76d8b649cd):
Code:
git reset --hard f9d1bcea9342348623f5a57588044f76d8b649cd
It will override any changes you made to files in ~/android-kernel/kernel.
B. Once you have downloaded git repository, you can swallow latest changes by issuing:
Code:
cd ~/android-kernel/kernel
git pull
C. If your machine has more than one cpus/cores you can speed up kernel compilation by adding -j <cores/cpus_number>, for example (dual core):
Code:
make -j 2 zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse

bad internet makes for double post. Please delete

wrong directions for kernel - we are working off of htc-vogue branch... not htc-msm...

can we make the internet work through the usb cable?

how does android know which device node is gps? it's not picking it up for kaiser...
if you enable gps in winmo (or enable it in smd0 - I believe the command is @startgps), smd7 is a nmea stream (acts as a serial GPS).... could a symlink possibly be the right solution to this?

Howto: Pull from git (new/update/resync)
This will download the latest from git:
dwaradzyn: Post:
Here are brief instructions on how to compile android kernel for Kaiser from git.linuxtogo.org repository. I assume that running OS is Linux and it has everything required to build x86 or ia64 kernel. Beside that latest git software should be installed. The shell is assumed to be bash.
1. Let's start with creating a directory for kernel in home directory:
Code:
mkdir ~/android-kernel
cd android-kernel
2. Next thing is to get the sources from repository. To make it happen (this could take a while, it downloads 280MB):
Code:
git clone git://git.linuxtogo.org/home/grou
ps/mobile-linux/kernel.git
OUTPUT:
Code:
Initialized empty Git repository in /home/user/android-kernel/kernel/.git/
remote: Counting objects: 908251, done.
remote: Compressing objects: 100% (153970/153970), done.
remote: Total 908251 (delta 755115), reused 906063 (delta 753016)
Receiving objects: 100% (908251/908251), 281.86 MiB | 292 KiB/s, done.
Resolving deltas: 100% (755115/755115), done.
Checking out files: 100% (22584/22584), done.
3. The htc-msm branch is of our interest (again it could take a few seconds):
Code:
cd kernel
git checkout -b htc-msm origin/htc-msm
OUTPUT:
Code:
Branch htc-msm set up to track remote branch refs/remotes/origin/htc-msm.
Switched to a new branch "htc-msm"
4. Let's take care of arm toolchain. Download this file (64MB) into ~/android-kernel:
Code:
http://www.codesourcery.com/gnu_toolchains/arm/portal/package2549/public/arm-none-linux-gnueabi/arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
Unpack it:
Code:
cd ~/android-kernel
tar xjf arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
5. Compile the kernel
Prepare default .config for Kaiser:
Code:
cd ~/android-kernel/kernel
make htckaiser_defconfig ARCH=arm
OUTPUT:
Code:
........
lots of output
........
# configuration written to .config
#
And finally compile the kernel to get zImage (takes a minute or two):
Code:
export PATH=~/android-kernel/arm-2008q1/bin:$PATH
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
OUTPUT:
Code:
........
lots of output
........
Kernel: arch/arm/boot/zImage is ready
Now copy ~/android-kernel/kernel/arch/arm/boot/zImage to your phone and play with it.
Some ending tips:
A. You can compile earlier versions of sources in repository. To do that click on one of "commit" links on page:
http://git.linuxtogo.org/?p=groups/mobile-linux/kernel.git;a=summary
and read commit id (for example: f9d1bcea9342348623f5a57588044f76d8b649cd):
Code:
git reset --hard f9d1bcea9342348623f5a57588044f76d8b649cd
It will override any changes you made to files in ~/android-kernel/kernel.
B. Once you have downloaded git repository, you can swallow latest changes by issuing:
Code:
cd ~/android-kernel/kernel
git pull
C. If your machine has more than one cpus/cores you can speed up kernel compilation by adding -j <cores/cpus_number>, for example (dual core):
Code:
make -j 2 zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
poly_poly-man: Post:
This will pull from git which will update/resync a git pull:
Code:
mkdir foo
cd foo
git init
git pull git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git htc-vogue
Click to expand...
Click to collapse

I'll follow-up with some nice, full instructions...
1. prerequisites: arm-none-linux-gnueabi toolchain (gentoo users can use crossdev - otherwise.... uhh... idk?), git, a host toolchain (gentoo users have this by default, most other distros have this under "development" in their package managment... if you have gcc, you're probably set).
2. mkdir kernel
cd kernel
git init
git pull git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git htc-vogue
3. make vogue_defconfig
4. make
5. cp arch/arm/boot/bzImage /path/to/sdcard/
6. to update, run the git pull command by itself again, run make (may have to do the config line again if it's changed) and cp.

Thanks for the post poly. I don't know how that's different from what the post I referenced as I'm not +4 at this stuff.. I did what you posted with android kernel from git and got a 1.2mb zImage that crashed HaRET.. I'm guessing this is my bad. What could I have overlooked? Thanks

enatefox said:
Thanks for the post poly. I don't know how that's different from what the post I referenced as I'm not +4 at this stuff.. I did what you posted with android kernel from git and got a 1.2mb zImage that crashed HaRET.. I'm guessing this is my bad. What could I have overlooked? Thanks
Click to expand...
Click to collapse
where'd you get your toolchain?
does building a regular (host arch) kernel work?
Also - what's the proper way to build a system.img by hand? I'm looking to modify that quite a bit, but can't find a persistent source tree besides the main one, which is seriously crippled.

I thought you were one of the experts, lol. I've been left with no support on how dzo, et all are customizing kernels so I've been in read only mode on the 'other thread' looking elsewhere for support.
Just wanted to ask you first, what's with the Dream radio? I know you posted the mods censored it but what's with the sig now? It piqued my interest... as I'm using (shudder) winmo on the regular while Android is being worked on I was hoping it would be worth looking into if it doesn't brick my phone.
Answers to your questions:
As I said, my own zImage is no go. Check this link (not for our phone but the links at the bottom are pretty useful): http://wiki.xda-developers.com/index.php?pagename=BlackstoneLinux#Runningx20.Linuxx20.onx20.blackstone
I got the toolchain from the steps I (re)posted on this thread:
http://forum.xda-developers.com/showpost.php?p=2269384&postcount=184 so that gave me a 1.2mb zImage where everyone's been posting 1.4mb-- I know there's something not right. As far as building a system.img by hand? I've taken existing ones either from posted bundles or from Android src directly. Maybe I suck (real possibility) but cupcake and 1.0 have been pretty flaky for me (there are system.img's included in the source). You should know how to mount and edit them though (look at the first post on this thread). My experience is the git source is useless unless you've got a G1-- I don't know how to make it run on Tilts. If it does work, then the answer to your question about host arch compiling is no-- it has to be ARMv5 for our phones. That's where this line comes in:
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
Seriously, I need help learning how to modify the kernel-- I've been a "google will have the answer for me" Linux bystander for a while and while I'm good at what I've done I'm not so good at this uncharted territory. I'm really looking for help to figure out how to compile modules (saurik and dzo never got back to me about that) and all I've gotten is "wait while I do it myself" which is cool they're working on it but we're obviously here to work on it too.
I've even been trying to get Debian installed (familiar territory for me) so I can at least get stuff working. You do know that Android is nothing but a Java VM layer for Linux and it will (could be) put on any self-respecting distro. Personally, I want Debian on my phone with an Android chroot as that would trump all.
Sorry to rant but you seem more about figuring this out like me and I don't know who else is really working on this besides the dev-gods who have no tutorials.

enatefox said:
I thought you were one of the experts, lol. I've been left with no support on how dzo, et all are customizing kernels so I've been in read only mode on the 'other thread' looking elsewhere for support.
Just wanted to ask you first, what's with the Dream radio? I know you posted the mods censored it but what's with the sig now? It piqued my interest... as I'm using (shudder) winmo on the regular while Android is being worked on I was hoping it would be worth looking into if it doesn't brick my phone.
Click to expand...
Click to collapse
It never actually worked... maybe. My phone was reporting the wrong version on a *different* radio (1.65.21.18, was saying 19) before, and trying to flash this changed the version to be correct. 0x300 radios will never flash, and this as a 0x301 *will* brick your phone. Then again... like 2 people reported epic success... In other words, no, it never really existed.
Answers to your questions:
As I said, my own zImage is no go. Check this link (not for our phone but the links at the bottom are pretty useful): http://wiki.xda-developers.com/index.php?pagename=BlackstoneLinux#Runningx20.Linuxx20.onx20.blackstone
I got the toolchain from the steps I (re)posted on this thread:
http://forum.xda-developers.com/showpost.php?p=2269384&postcount=184 so that gave me a 1.2mb zImage where everyone's been posting 1.4mb-- I know there's something not right. As far as building a system.img by hand? I've taken existing ones either from posted bundles or from Android src directly. Maybe I suck (real possibility) but cupcake and 1.0 have been pretty flaky for me (there are system.img's included in the source). You should know how to mount and edit them though (look at the first post on this thread). My experience is the git source is useless unless you've got a G1-- I don't know how to make it run on Tilts. If it does work, then the answer to your question about host arch compiling is no-- it has to be ARMv5 for our phones. That's where this line comes in:
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
unnecessary - zImage is implied, and the other two are in the Makefile already.
Seriously, I need help learning how to modify the kernel-- I've been a "google will have the answer for me" Linux bystander for a while and while I'm good at what I've done I'm not so good at this uncharted territory. I'm really looking for help to figure out how to compile modules (saurik and dzo never got back to me about that) and all I've gotten is "wait while I do it myself" which is cool they're working on it but we're obviously here to work on it too.
Click to expand...
Click to collapse
we have 0 modules at the moment (but perhaps support - I forget). Just compile stuff in - modules are very bad.
I've even been trying to get Debian installed (familiar territory for me) so I can at least get stuff working. You do know that Android is nothing but a Java VM layer for Linux and it will (could be) put on any self-respecting distro. Personally, I want Debian on my phone with an Android chroot as that would trump all.
Click to expand...
Click to collapse
It's a nice idea, but remember where a lot of the current coding is taking place - the ril, which is part of android (the system.img, at least), and doesn't go across to other distros quite as well. I think running Dalvik alongside whatever you're running in Debian might be too much for this phone's epically slow processor (compared to msm7201a).
Sorry to rant but you seem more about figuring this out like me and I don't know who else is really working on this besides the dev-gods who have no tutorials.
Click to expand...
Click to collapse
my instructions should work - they are what I use, at least.

Someone should change the instructions to (in the Wiki they are correct):
make ARCH=arm vogue_defconfig
I compiled my kernel, booted in Ubuntu, but had no touchscreen at all, I am not sure if the vogue_defconfig file takes care of everything necessary, so now I am checking everything via menuconfig to see whether eveything is all right or not.
DOMy

Do not Use Ext3 on SD cards
enatefox said:
seidler2547: Post:
Click to expand...
Click to collapse
You should NOT be using ext3 on any sort of flash ram device. You will wear out the medium. Journaling is just a bad idea in this sort of situation.
http://www.handhelds.org/hypermail/familiar/273/27320.html
3) ext3 is "very bad" because of the way it does journaling. It does journal, which seems like a good idea, but it also automatically periodically writes a lot of things out to the same secors on disk. I don't have first hand experience with this, but I remember somebody familiar with ext3 writing about this. It's method of journaling is not particularly intended for any kind of wear leveling at all.
Click to expand...
Click to collapse
http://www.mail-archive.com/[email protected]/msg38988.html
There are three disadvantages with the journaled file system:
- lower performance at write time, since there is the extra work of the
journal
- increased chance of damaging the SD card due to extra use of the
journal causing wearing
- increased space usage (for the journal)
Click to expand...
Click to collapse
And this is the general consensus within most of linux on ext3 and wearing mediums. I'm not sure if Andriod's kernel can do ext4, but ext4 can run without a journal.
Yet another reason not to use ext3 is that is near impossible to undelete something, something you can do with ext2.
-edit-
It looks like Android can use Ext4
http://thatsbadass.com/android/tag/ext4/

haha! is a good job, i like it

can't run android on my kaiser
hi folks,
i have read many threads here and on androidonhtc.com, i have read also the install instructions, but it seems im too stupid to run it.
it fails on "can't find system.img". i wil not flash it, i will run it from sd-card.
so, please can anyone attached an actually zip file which i just unpack und run haret.exe to work android on my kaiser ?
thanks and best regards
lenzen

How to reconstruct a binary identical I9000XWJP6 kernel image, and more

The idea of this exercise is (at least) to get a stable starting point for kernel development. The thing which is currently missing is a proper working .config. I have reconstructed it using differential analysis and in the process hoped to find which components have actually been activated and to uncover changes (or Easter eggs) in the sources which have not been advertised. Having a working and identical I9000XWJP6 kernel also means that open development can continue from the current official public release. From there the things possible are only limited by your imagination.
The following is a walk-through on how to build the kernel, description of pitfalls that will cause changes in .config to break, and some annotations on discoveries made in the process.
The things you need are:
Mandatory:
- I9000XWJP6 zImage : from your favorite location
- Source tree : opensource•samsung•com the GT-I9000 OpenSource Froyo Update JPM.zip
- Sourcery G++: www•codesourcery•com/sgpp/lite/arm/portal/release1039
- Tweak-Kit : <attached>
Optional:
- Arm enabled GCC and binutils, including a development libbfd.
- Lots of your favorite beverage
The md5sum of the zImage should be: 26e9d5d206baf1515144c6b8de6f10d2
It is critical that the Sourcery G++ version is 2009q3-67.
The Tweak-Kit contains the following components:
Readme.txt - You're reading it
mkvmlinux.cc - convert zImage to vmlinux and extract the init ramdisk image
I9000XWJP6_defconfig - default .config
stamp.patch - set date/time and such to original
style.patch - fix style related warnings
prototype.patch - fix prototype related warnings
error.patch - Recoverable errors
houston.patch - Unrecoverable errors
shadow.patch - Fixate data structures
I9000XWJP6.h - Fixated macros
I9000XWJP6.c - Entry point stubs
SOME THINGS I ENCOUNTERED IN THE PROCESS:
a) What I absolutely did not expect was that I found two different encodings of the build timestamp. I could deduce that the timezone was central Europe. I had the assumption it would be Asia or America.
b) What was to be expected is that the source tree is incomplete. The directories drivers/fsr and fs/rfs are missing. You can still compile the kernel as the missing files are used to build modules. Problems start when you change the config. Doing so will change entry points and data structures and your kernel might die a horrible death when it loads modules who are unaware on these changes. There is a workaround which I will explain later.
c) The weirdest thing I encountered were the functions enable_hlt() and disable_hlt(). The are located deep in the unwind tables, a section not intended for code. I spent many hours trying to figure out how they got there or why but I still have no clue. I found exactly only one way to reproduce this behaviour and it is certainly not due to a typo, accident or ignorance.
d) The kernel is not a production but a debug version. It has nearly all tracking/tracing/debug bells and whistles switched on. If the energy required to maintain the statistics where to emit light, you could use your Galaxy as a Christmas tree. Function profiling is enabled and has a considerable negative effect on performance, code is not optimized for size but speed, and unwind tables have been enabled which are not used. These have a really bad impact on footprint size. I really hope that the same compiler and settings are not used for the Android layer. Changing the config into a production version will not work (and crash) as the non-native modules expect the debugging hooks which will no longer exist. But the same workaround as above can be used.
e) The functionality of the power management domains have been optimized to oblivion due to the excessive placing of code disabling comments in large parts of the clock, power management and mach-aries.c. Maybe because the Galaxy hardware is too different than the evaluation boards, or the hardware is buggy and disabling the code makes it less unstable, or there was just not enough time to get the code working. Anyway, at this moment I have no oversight into what degree the absence of power domains influence battery usage.
f) When I started examining the binary code I was puzzled by snippets of code I could not reproduce. Even worse, I encountered snippets that were just questionable. Unusual instruction sequences, and resister usage. Thinking I bumped into a GCC bug, I started debugging the compiler and even tweaked instruction scheduling weights but with no satisfying outcome. I know that GCC is very stubborn with regard to saving and clobbering registers in/across function calls and the code I saw was just incorrect. I knew a different compiler was used and I suddenly realized that it may be more different than what first meets the eye. The culprit turned out to be Sourcery G++. It is a private maintained branch of GCC for reasons I have not investigated. Even the Sourcery assembler is tainted as it played a nasty trick on me with the enable_hlt/disable_hlt thing. I do not like the code I see and I am aiming into getting the sources stock GCC friendly with a working kernel. However, GCC and Sourcery generate code which seem difficult to mix, but I'm getting closer.
g) Compiler warnings. Many of the Samsung sources generate warnings, something I really dislike. In my opinion a warning is emitted for a piece of code which can be interpreted in several ways, leaving the compiler to choose which. Usually it will choose the wrong one. Most warnings were related to coding style shortcuts, a couple of incorrect function prototype resulting in functions that should return int to return random or falsely ignoring return values. There were also a couple of nasties like deference of uninitialized pointers, accessing out-of-bound data and mixing clock data-structures of different types. Included are a number of patches to fix them.
h) I looked deeper into why GCC and Sourcery won't mix and discovered that they have different implementations with regard to constant definition within enum declarations. Google points to the staring point "GCC bug 30260" where is written that the behaviour of enumeration constants has changed to becoming signed int. I have noticed that even explicit unsigned values will change to signed.
Here is an example of what is going wrong:
Take following declaration
Code:
enum rt_class_t { RT_TABLE_MAX=0xFFFFFFFF }
. GCC will consider RT_TABLE_MAX to be -1, and Sourcery will consider it 4294967295. Now, in net/ipv4/fib_rules.c there is this code snippet
Code:
for (u32 id = 1; id <= RT_TABLE_MAX; id++)
GCC will skip the loop, and Sourcery will have a hard time doing nothing.
There are more examples like calculating the location of physical memory or signed/unsigned comparisons. The compiler switches -fwrapv and -fstrict-overflow might influence things, but it general the behaviour is hardcoded and both compilers have a different flavour. I think it would be wiser to choose the GCC flavour as it is more widespread and thus better tested (and fixed).
i) GCC. I noticed that early versions of kernels compiled with GCC would not start. At first I thought it was because of Sourcery /GCC code generating differences. After a number of buxfixes (in error.patch) I suddenly noticed that the GCC kernel is working. My phone is running a GCC compiled production configured kernel for nearly a week.
j) "Houston, we've had a problem" with the light sensor. One of the compiler warnings brought me to the file drivers/sensor/optical/gp2a.c. There within are located two routines which read the light and proximity sensor. They seem copy-pasted identical, however the sensor value types are different as the proximity value is a char and the light intensity a double. What is more convenient than to simply change the data type of the supplied buffer in the function prototype. Now headache starts as the semantics of the read (and write) call say that the unit size is byte. So returning "1" indicates that only the first byte of the sensor value is copied. Also, there is no bounds/access checking so supplying an invalid pointer to the call will crash the kernel. So, assuming this is all one big mistake, I redesigned the function to do better (see houston.patch) and built a new kernel with it. To my utter surprise my battery charge extended from <24 hours to 2 days and 20 hours.
However... I also noticed that my backlight intensity level was constant at it's lowest although the setting was set to auto. I needed to know how the caller invokes the call, but after an extensive search of internet and android sources it is still something I have not found. Heuristics show that the reading the light sensor is called with a buffer length of 1, and the returned value is only accepted when returning a 1 and that the sensor value type is a double (8 bytes). This is wrong: read() semantics require that you supply a length of 8, and expect a return value of 8. This may be the base of many light sensor issues I found when Googling.
Anyway, I returned the code to it's original faulty behaviour, and being illuminated I disabled the auto backlight intensity and changed it to it's lowest setting to enjoy a longer life between battery charges.
TO CREATE YOUR KERNEL:
1) Prepare a working environment
1a) Unpack Sourcery G++. No installation needed, unpacking is sufficient
1b) Unpack the Samsung sources and cd to the location of the top-level Makefile.
1c) Unpack the zImage and the contents of the Tweak-Kit to the same location
1d) Make sure the zImage is called zImage.I9000XWJP6
2) The ramdisk image is required and can be extracted from zImage.I9000XWJP6
2a) Create an uncompressed image Image.I9000XWJP6
Code:
gcc scripts/binoffset.c -o scripts/binoffset
ofs=`scripts/binoffset zImage.I9000XWJP6 0x1f 0x8b 0x08 0x00 2>/dev/null`
dd ibs=$ofs skip=1 <zImage.I9000XWJP6 | gzip -c -d >Image.I9000XWJP6
2b) The Tweak-Kit includes mkvmlinux which converts the uncompressed binary image into a bfd object. You need an Arm enabled libbfd to get it working. This does not get installed by default so you need to deeplink into binutils. mkvmlinux locates and decodes the kallsym data and econstructs the symbol table. It then uses the values of __initramfs_start/end to extract the initramfs. If you are not bothered with the hassle, just use dd with hardcoded values.
Code:
g++ mkvmlinux.cc -o mkvmlinux -lbfd -liberty -lz [-I and -L that deeplink into binutils]
./mkvmlinux Image.I9000XWJP6 vmlinux.I9000XWJP6 -r initramfs.cpio
or
Code:
dd if=Image.I9000XWJP6 of=initramfs.cpio bs=1 count=2739712 skip=165568
3) Patch date/time and other environmental issues to the moment of original creation
Code:
patch -p1 <stamp.patch
4) Make your computer happy
Code:
# edit Makefile line 184 and update the macro CROSS_COMPILE=
cp I9000XWJP6_defconfig arch/arm/configs/I9000XWJP6_defconfig
make I9000XWJP6_defconfig
make
5) Verify that the kernel is identical
Code:
diff zImage.I9000XWJP6 arch/arm/boot/zImage
AND NOW FOR SOMETHING COMPLETELY DIFFERENT...
Tweaking the configuration will build you a new kernel but when your Galaxy powers on it will either die silently (hang) or experience a horrible death (reboot). The problem is that there are modules built from sources located in the removed directories drivers/fsr and fs/rfs. These modules were compiled with a specific data structure layout and entry points. These will surely change when re-configuring. The way to keep the non-native modules happy is to keep the structures and entry points intact.
The structure layout is influenced by the CONFIG_ macros. The structures can be fixed to reflect the state of the I9000XWJP6 kernel by replacing the CONFIG_ macro's by something that does not change after reconfiguration. For that I use a collection of 'shadow' macro's which have SHADOW_ as prefix. Because the data structures cannot expand, you cannot (easily) enable configure functionality which require extra fields in the data structures. Reducing functionality is highly seldom a problem.
If changing kernel functionality results in removal of entry points, then stubs are required for those entry points needed by the non-native modules
There are automated methods to verify that a new kernel abides to the above constraints. For the data structures the compiler must generate gstabs debug information. This is human readable and includes detailed structure descriptions. This information should be identical across re-configuration. However, the scripts get confused by anonymous structs which are by product of "typedef struct {" constructions. These need to be named, something shadow.patch also does.
The kernel modules have easily-readable symbol tables containing needed kernel entry points. These should all be present in all re-configured kernels. Validation tests that fail emit enough information to further fix data structures and entry points. The Tweak-Kit contains two files: I9000XWJP6.h containing the SHADOW_ macro's and I9000XWJP6.c for the stubs. Both were constructed in an on-demand basis for the reconfiguration I am currently using and both serve as good examples on what to do when validation fails.
Before reconfiguring, rebuild the kernel for usage as a validation checkpoint.
1) Undo the timestamp patches
Code:
patch -R -p1 <stamp.patch
2) Fix the warnings
Code:
patch -p1 <style.patch # style related issues
patch -p1 <prototype.patch # prototype related issues
patch -p1 <error.patch # bug fixing
3) Apply datastructure fixation, entrypoint stubbing and Makefile tweaking
Code:
patch -p1 <shadow.patch
cp I9000XWJP6.c arch/arm/plat-samsung
4) Before recompiling everything, you need to issue "make clean" first. However, the missing directories will now pose a problem as "make clean" will include their Makefiles and will fail if it can't. Just create empties to keep the build happy.
Code:
mkdir -p drivers/fsr fs/rfs
touch drivers/fsr/Makefile fs/rfs/Makefile
5) Optionally change the Makefile to point to your favorite compiler/toolchain. Please note that I am using GCC 4.4.5. GCC 4.5.1 is bumping into problems I haven't looked into yet.
Code:
# edit Makefile line 184 and update the macro CROSS_COMPILE
6) This build will generate gstab debug information. Unexpectingly, this might bite when combined with function profiling, so disable that. But do not CONFIG_FUNCTION_TRACER yet as that does more.
Code:
# edit Makefile line 553, disable the line containing KBUILD_CFLAGS += -pg
7) Unpack the initramfs image. The directory /lib/modules needs to be examined/updated
Code:
mkdir initramfs.dir
cd initramfs.dir
cpio -i --make-directories --preserve-modification-time --no-absolute-filenames <../initramfs.cpio
cd ..
8) Repack initfs as a tarball, as make clean will erase all the modules
Code:
tar cf initramfs.tar initramfs.dir
9) The initramfs image will contain new kernel modules. Make sure a new version will get generated.
Code:
# in .config line 80 point to the unpacked initram location
CONFIG_INITRAMFS_SOURCE="initramfs.dir"
# in .config lines 86-89, select your favourite compression
CONFIG_INITRAMFS_COMPRESSION_NONE=N
CONFIG_INITRAMFS_COMPRESSION_GZIP=Y
10) Build a new kernel
Code:
# not cleaning will confuse the verification
make clean
make CONFIG_DEBUG_INFO=y
# install the modules
tar xf initramfs.tar
cp `find drivers -name '*.ko'` initramfs.dir/lib/modules
# rebuild with a fresh new ramdisk image
rm usr/initramfs_data.cpio*
make CONFIG_DEBUG_INFO=y
11) Checkpoint structure/entrypoint information. This is architecture independent.
Code:
# extract structures. They are the entries with :T
objdump -G vmlinux | awk '{ print $7 }' | grep :T | sed 's/([^)]*)/()/g' | sed 's/=\*()//g' | sort -u > gstabs.ckp
# extract the entrypoints
nm vmlinux | grep 'r __ksymtab_' | awk '{ print $3 }' | sort > ksymtab.ckp
12) Do a test-run. Pack zImage and flash with Odin. If your Galaxy is up and running, I strongly suggest you make a backup of your environment. If you later change something and it breaks, then this is the best place to restart.
Code:
cp arch/arm/boot/zImage .
tar cf I9000XWJP6-2.6.32.9-test.tar zImage
13) Make your re-configuration. I really suggest you do not make too many changes in one go because it gives more work when the structure/entrypoint verification fails.
Code:
# re-configure. For this exercise, change the kernel to a more production version
CONFIG_CC_OPTIMIZE_FOR_SIZE=Y
CONFIG_DM_DEBUG=N
CONFIG_S3C_KEYPAD_DEBUG=N
CONFIG_DEBUG_FS=N
CONFIG_DEBUG_KERNEL=N
CONFIG_LATENCYTOP=N
CONFIG_FTRACE=N
CONFIG_ARM_UNWIND=N
CONFIG_DEBUG_USER=N
14) Build a new kernel
Code:
# not cleaning will confuse the verification
make clean
make CONFIG_DEBUG_INFO=y
# install the modules
tar xf initramfs.tar
cp `find drivers -name '*.ko'` initramfs.dir/lib/modules
# rebuild with a fresh new ramdisk image
rm usr/initramfs_data.cpio*
make CONFIG_DEBUG_INFO=y
You get "struct has no member named" errors if you have enabled subsystems that require data structures to change which are incompatible with the non-native modules.
15) Verify structure/entrypoint checkpoint
Code:
# extract/verify structures
objdump -G vmlinux | awk '{ print $7 }' | grep :T | sed 's/([^)]*)/()/g' | sed 's/=\*()//g' | sort -u > gstabs.t
# new/changed structures are tagged with '+'. Display only the changed ones
diff -U0 gstabs.ckp gstabs.t | grep '+' | grep ':T' | sed 's/+//' | sed 's/:T.*/:T/' | while read s; do
grep -q "$s" gstabs.ckp
if [ $? -eq 0 ]; then
echo $s;
fi
done
# extract/verify entrypoints
nm vmlinux | grep 'r __ksymtab_' | awk '{ print $3 }' | sort >ksymtab.t
# extract all entrypoints needed by the modules
nm `find initramfs.dir/ -name '*.ko'` >allkosym
# some symbols are referenced in other modules. Cross-reference and remove from list
grep ' U ' allkosym | sort -u | awk '{ print $2 }' | while read s; do
if ! egrep -q "^[0-9a-f]* . $s\$" allkosym; then
echo $s;
fi
done > allusym
# check that kernel has entrypoints for all final undefined symbols
cat allusym | while read s; do
if ! egrep -q "^__ksymtab_$s\$" ksymtab.t; then
echo $s;
fi
done
Both scripts will generate output if you have enabled subsystems that require datastructures to change. These do not necessarily have to be data structures needed by non-native modules. However, missing entrypoints are those used by the modules. If it's about datastructures, your best chance is to lookup the data type and see if there any #ifdef CONFIG_ macros that need to be changed into #ifdef SHADOW_. If it's a missing entry point, you need to add a stub in I9000XWJP6.c.
16) Do a test-run. Pack zImage and flash with Odin.
Code:
cp arch/arm/boot/zImage .
tar cf I9000XWJP6-2.6.32.9-test.tar zImage
17) If you want more, jump to step 13
18) When you are really done, rebuild a final and fresh kernel and initramfs image with debugging stuff removed. The -gstabs compiler switch slightly influences code generation.
Code:
# not cleaning will confuse the verification
make clean
make
# install the modules
tar xf initramfs.tar
cp `find drivers -name '*.ko'` initramfs.dir/lib/modules
# rebuild with a fresh new ramdisk image
rm usr/initramfs_data.cpio*
make
My uncompressed image has now shrunk from 14700623 to 11822559 bytes.
Happy Hacking...
[...and now to find a better workaround for those non-native modules.]

WoW, Nice work !! very good info for beginners like me

thx a lot for this tut and i've learnt a lot
btw, seems there r some typos or something is missing. i did it with (XXJPO):
Hexabit said:
make I9000XWJP6_defconfig
Click to expand...
Click to collapse
make defconfig I9000XWJP6_defconfig
- modified include/linux/a.out.h by removing the 2nd def for SEGMENT
- changed the boolean to lowercase for .config
PS i use the cpio extracted by myself coz i couldnt enable libbfd on my ubuntu x64

Good tips. Thanks.

Really insightful i hope the dev take all the tweaks into consideration and make a new optimized kernel

good job here!

I think it's the most amazing first post ever! It should be sticked or kept somewhere safe.

Awesome first post. Will have to work through this.

Great post with very interesting findings!
I'm no expert, so maybe my question is a bit silly:
Is Samsung's published code just a buggy and incomplete pre-release debug version? Then how can e.g. Voodoo get a good working kernel?
Or is the official firmware really built of this, so possibly full of strange bugs and missing optimizations?

[dev][kernel][kexec]

Last Update : August, 19, 2014
Hi,
I'm still try to bypass the MMU protection.
I have fixe a lot of bug, like memory misalignment, bad adresses allocation, dtb correction, etc...
Last sources and binaries here :
kexec-tools V11.zip : http://forum.xda-developers.com/attachment.php?attachmentid=2902912&stc=1&d=1408401794
kexec-tools binaries V11.zip : http://forum.xda-developers.com/attachment.php?attachmentid=2902913&stc=1&d=1408401794
Sorry, i have always 13 sec reboot after new kernel boot.
"cpu_proc_fin" use a "mcr p15" to init cache and proc that cause freeze.
I try to find solution for that.
Last Update : June, 22, 2014
Hi,
My sources are horrible... but i give something new.
This kexec is for stock kernel only (tested on .757). I thinks theses sources work on other kernel too.
In "kexec-tools V10.zip", you have all my sources. It's highly recommended to mod them to have something OK.
In "kexec binaries.zip", you have binaries to install
=> "kexec_load.ko" and "procfs_rw.ko" must be placed in "/system/lib/modules" folder with "chmod 777"
=> "kexec" must be placed in /system/bin" folder with "chmod 777"
=> cd /system/lib/modules
=> insmod kexec_load.ko
For sources :
Mod and adapt all you want, it's free.
You have 2 scripts in Zip : "./compil-kexec" in "kexec-tools" folder to rebuild and send in device directly (install Adbtcp on device and send by tcp with : adb connect xxx.xxx.xxx.xxx) = work perfectly with me.
"scriptZ1" is for compil stock kernel or another kernel (doomlord kernel for eg)
You must rename "custom_final_files" folder after compil to "final_file" manually ; You can have guest kernel in "custom_final_files" and stock kernel in "final_files" for "kexec-tools" path ... Don't mix a guest and host kernel please ^^
I am tired... i let you test and say if it's ok for you...
Thank a lot to munjeni for his help.
kexec-tools V10.zip : http://forum.xda-developers.com/attachment.php?attachmentid=2811994&stc=1&d=1403456181
kexec binaries.zip : http://forum.xda-developers.com/attachment.php?attachmentid=2811995&stc=1&d=1403456181
Last Update : November, 23, 2013
Hi,
For few days now, i haven't no more kernel panic with my kexec.
I have fixed few stuffs into sources, and add a lot.
These adds are, to include a "dt.img" image file into kexec load process.
This image file is a "device_tree" image to match hardware to software.
So, i assume to don't include atags into boot process, but pass bootloader informations by this DT.
I have programmed a little scan memory to found dynamicly all magic tags, because i found 3 device_tree into memory (magic is "0xd00dfeed").
These 2 device_tree are echo from first and nice structure.
The boot process need to have informations from this DT, and need all informations to initialize hardware (no HDW initialisation by the kernel)
I must first fix issues ; Regroup zImage and dt.img into memory to load a solid bloc to kexec_load module to boot into, and second, fix an offset i can't explain, 0x800 in memory causing misalignment memory
Keep tuned..
Last Update : November, 17, 2013
Hi everybody,
My kexec-tools work for Sony Xperia Z1 stock kernel "3.4.0-perf"
This tools can work on all locked bootloader for all locked device, not only Sony or Z1 models.
This kexec-tools add a kexec_load kernel module (LKM) and use a driver to grant a communication between "kexec" user program and kexec_load.ko module
what is for ?
"kexec" user program load in memory a custom kernel in zImage format, but can load ".tar" image too
This user tool load ramdisk in memory if necessary
This tool is for this purpose only, and don't keep in memory the custom kernel at device reboot.
It is a "user" program, not a "kernel" extension... So, to really do the magic, we need the host kernel (stock sony locked kernel) have a kexec_load capability to reboot in a new gest kernel (custom kernel).
Infortuntly, stock kernel don't have kexec_load capability.
Sony have compiled his stock kernel without this option, and "standard" kexec-tools "need" this option to work.
To see all system call capability of kernel, you can run theses command :
Code:
echo 0 > /proc/sys/kernel/dmesg_restrict
echo 0 > /proc/sys/kernel/kptr_restrict
cat /proc/kallsyms
Do all grep you want here.
The "echo 0" "restrict" is here to unmask logical adresses to "system calls"
Like you can see, "__NR_kexec_load" capability isn't here.
To add kexec_load capability in stock locked kernel, we need to add manualy a kernel module wich add this function into the kernel.
Why ? Because the way to keep in memory a custom kernel need to know a lot of parameters, and keep a specific memory range alive at reboot.
Only kernel can do this.
All user program will be terminated at reboot.
"Standard" kexec_load.ko module use a method to implement the "__NR_kexec_load" function in system call table.
Since 2.6.0 kernel, linux for security reason, have locked in memory the "system_call_table" ; No more add or modification is authorized.
If kexec tool try to add a value, "kexec_load" for us, we causes a kernel panic, and reboot device.
For this reason, i have modify kexec user program and kexec_load module to implement a driver to talk to each other.
this driver replace syscall method, and we no more need to use a system call table.
For this reason, this tool is now compatible with modern kernel like our "3.4.0"
For this reason, this tool must work for other device (Xperia X, P, S, etc...) and another brand
For this reason, if kernel is locked, we can bootstrap to run a new kernel.
Installation
First, you can compil your own kexec tool
Here, sources : http://forum.xda-developers.com/attachment.php?attachmentid=2397299&stc=1&d=1384689174
And here, the binaries : http://forum.xda-developers.com/attachment.php?attachmentid=2397305&stc=1&d=1384689406
(it's not a cwm zip, i have no time to create an installer for now ; use "./compil-kexec" if you want an automatic install)
Install *.ko in /system/lib/modules
Install kexec and kdump in /system/bin
Grant with "chmod 777"
Unzip in kexec-tools folder
Install a toolchain (sudo apt-get install gcc-arm-linux-gnueabi)
launch => ./compil-kexec
what's all
This script can do everythinks for you
- Compilation of tools
- Compilation of modules
- installation in device
This script can compil for every brand you have.
Except you must remove or adapt the patch (see below why)
Patch ??
This patch is because a module must be compiled in the same time the kernel himself.
For this reason a "vermagic", an identifier, is used by system to block every module not compil with kernel
Some custom kernel bypass this to authorize every modules.
But for stock kernel, it is not allowed.
You can easely strapp this by busybox.
"busybox modprobe" for help
"-f" to force load without vermagic
To see this vermagic :
Code:
# uname -r
This "uname -r" must be the same that
Code:
# strings kexec_load.ko | grep vermagic
vermagic=3.4.0-perf-g66807d4-02450-g9a218f1 SMP preempt mod_unload modversions ARMv7
If you want use automaticaly this vermagic, you can modify into the custom kernel this file :
Code:
"include/config/kernel.release" and add :
"3.4.0-perf-g66807d4-02450-g9a218f1"
This file will be use at module compil to match the vermagic.
Infortunatly, it is not enought. :silly:
The infamous "no symbol version for module_layout"
When a module compil is created, it use symbols link to system call function, translate by adresses
Theses symbols are not at same physical adresses in stock kernel and modules (compiled from DooMLoRD kernel).
So, theses adresses must be convert into modules itself to match with stock symbols adress.
A patch is needed.
If you use my script, modules are automatically patched.
Here patches :
Code:
sed -i 's/\x32\x76\x86\x29/\x72\xFF\x5E\x20/' procfs_rw.ko
sed -i 's/\x32\x76\x86\x29/\x72\xFF\x5E\x20/' kexec_load.ko
sed -i 's/\xBB\xD0\xF8\x4D/\x0E\x1C\x63\x77/' kexec_load.ko
sed -i 's/\xA6\x26\x81\x1A/\xD4\x56\x02\x7E/' kexec_load.ko
sed -i 's/\xA3\xD1\xEC\x96/\xEC\x43\x28\x1A/' kexec_load.ko
sed -i 's/\x8C\xE6\x6A\x5F/\x3D\xDF\x02\xF2/' kexec_load.ko
sed -i 's/\x3E\xF3\xEF\xE9/\x18\x7F\xA6\x8A/' kexec_load.ko
sed -i 's/\x8B\xD2\x92\x10/\xC8\x19\x08\x9C/' kexec_load.ko
sed -i 's/\x1C\xE8\x18\xE1/\x7C\x71\x9E\xEF/' kexec_load.ko
sed -i 's/\xAB\x2C\x2F\x8B/\x8E\xD7\x63\xC0/' kexec_load.ko
sed -i 's/\xF5\x62\xAA\x4B/\x34\x80\x1B\x74/' kexec_load.ko
sed -i 's/\x00\x52\xD6\xD7/\x6F\x80\x91\x20/' kexec_load.ko
sed -i 's/\x4F\x77\x57\x6A/\x0C\x57\xC7\x63/' kexec_load.ko
sed -i 's/\xCA\x2F\x65\x71/\x92\xB8\x7F\x53/' kexec_load.ko
sed -i 's/\x0F\xD0\xA0\x91/\xFA\x80\x15\xB4/' kexec_load.ko
sed -i 's/\x29\xA0\x6D\x48/\x6C\x6B\x96\x54/' kexec_load.ko
sed -i 's/\x6D\x1F\x1F\x37/\xCC\x5E\x79\x8B/' kexec_load.ko
sed -i 's/\xFD\x23\xD0\xFB/\xE3\xE3\x68\x52/' kexec_load.ko
You can use hexedit or hexdump to see these adresses :
Code:
hexdump kexec_load.ko | grep ff72
0003d50 b0b0 80ac ff72 205e 6f6d 7564 656c 6c5f
how does it work ?
# kexec --help
For kexec help... nothing more to say.
# lsmod
List loaded modules... You must see
kexec_load 31369 0 - Live 0x00000000 (O)
# rmmod kexec_load.ko
Remove kexec_load module from memory.
# grep kexec /proc/device
To see installed driver.
You must see :
100 kexec_driver
First number is "major" number to identify your driver in system.
# mknod /dev/kexec_driver c 100 0
Install driver.
Major number (here 100), is important for module.
This Major must be the same between module and driver.
By default, 100 is used.
# insmod kexec_load.ko
To install "LKM", kexec_load kernel module.
If another Major is needed, you can use "insmod kexec_load.ko 101" for Major 101
You can use "modprob" if you want, but you must configure the module folder.
How kexec and module exchange informations ?
By the driver.
Normal output for a kernel module is to write in "dmsg" file.
To see kernel output, launch this command :
Code:
# dmesg
To see last kernel log, see in :
Code:
# cat /proc/last_kmsg
For kexec module, this normal way still exist, and give a lot of informations, but to speak with, you must use the driver.
/dev/kexec_driver
You can yourself test communication:
Code:
# cat /dev/kexec_driver
You can send kernel by this communication channel.
Type following commands for help
=> echo help >/dev/kexec_driver
=> dmesg | grep Kexec
Code:
# echo help >/dev/kexec_driver
# cat /dev/kexec_driver
Last command : 'help'
Please type following command :
=> dmesg|grep Kexec
Every command send into driver is receive by kexec_load.ko module and running into the kernel.
The answer can by read thru the driver
Here, you can see that normal way to see messages is allway dmesg.
Code:
# dmesg|grep Kexec
<4>[15050.521628] Kexec: Starting kexec_module...
<6>[15050.521656] Kexec: kexec_driver_contener allocation
<6>[15050.521673] Kexec: kexec_memory_buffer allocation
<4>[15050.521691] Kexec:----------------------------------------------------
<4>[15050.521710] Kexec: kexec_driver created with major : '100'
<4>[15050.521728] Kexec: Please, prepare by typing the following commands :
<4>[15050.521746] Kexec: => mknod /dev/kexec_driver c 100 0
<4>[15050.521761] Kexec: => cat /dev/kexec_driver
<4>[15050.521775] Kexec:-----------------------------------------------------
<4>[15050.521791] Kexec: For help
<4>[15050.521803] Kexec: => echo help >/dev/kexec_driver
(...)
I have add a lot of informations to help to configure kexec.
rdtags, atags ??
Not sure for this part of kernel.
"atags" is the most used method to bootloader to parse commands and informations to kernel at boot.
"atags" is a form of structure in memory to organise informations.
At boot, a address chain is created and can be compulse in /proc/atags file.
This file is read only system.
"rdtags" is another way to bootloader to parse information to kernel.
"rdtags" is not stocked in "/proc"
But, as i see, stock kernel can use "atags" from bootloader.
kexec can substitute bootloader function to create fromscratch a atags chain, and parse to new kernel.
I have change this part to stock atags in "/data/atags", and reuse or change if need.
If this don't work, i must create a rdtags chain to replace atags ; It's not a hard work.
Status
For the moment, kexec tools works.
=> Phase one OK.
I can start Phase Two : new kernel patch.
If you want to help me...
Actually, load a custom kernel and boot into with kexec tools work.
But at boot into, a kernel panic occurs.
It seems, a part of kexec patch is missing in custom kernel.

Hi new thread created for kernel kexec development.
Status: not working: wrong values for mem defines under the kernel is giving segmentation fault as its attempting to write to memory areas that are currently being used byyyyy the system
Instructions:
Make kernel compatible?:
1. Download kernel diff patch from below
2. Terminal - diff patch > diff.txt
How to use:
1. Download kexec-tools (kexec binary) from below
2. Copy into system/bin directory and give it executable permission
3. Download compatible kernel
4. Terminal - kexec --load-hardboot zImage --initrd=initrd.img --mem-min=0x20000000 --command-line="$(cat /proc/cmdline)"
kexec -e
Download links:
Kexec tool- https://db.tt/8DZXQ9eV
Ramdisk firmware 1.548 : https://db.tt/8DZXQ9eV
zImage (kernel):
Source code:
Kernel diff patch: https://db.tt/Xi2htT7Q (currently contains wrong values for mem defines)
Kexec-tools: https://db.tt/I22ofr3b
Special thanks: @delewer @krabappel2548

Reserved

Please move this thread to Xda Devdb, then I can also edit first post etc if I find new stuff
Sent from my C6903 using xda app-developers app

krabappel2548 said:
Please move this thread to Xda Devdb, then I can also edit first post etc if I find new stuff
Sent from my C6903 using xda app-developers app
Click to expand...
Click to collapse
Devdb?
Pm me i dont know what Devdb is lol

Recieved segmentation fault with delewers calculated mem values too
We need to write to memory where we have write access to, maybe lockedbootloader is not allowing us to write? Orrr we are just writing to wrong area of memory

If kexec works on the Z1, can it be ported over to Xperia Z/ZL/T/Ultra? I believe they don't all share the same processor.

Shaky156 said:
Devdb?
Pm me i dont know what Devdb is lol
Click to expand...
Click to collapse
Shaky156 said:
Recieved segmentation fault with delewers calculated mem values too
We need to write to memory where we have write access to, maybe lockedbootloader is not allowing us to write? Orrr we are just writing to wrong area of memory
Click to expand...
Click to collapse
I'll discuss with Kali- today if he's available.
Knucklessg1 said:
If kexec works on the Z1, can it be ported over to Xperia Z/ZL/T/Ultra? I believe they don't all share the same processor.
Click to expand...
Click to collapse
Doesn't need to be same processor, can be ported
Sent from my C6903 using xda app-developers app

Knucklessg1 said:
If kexec works on the Z1, can it be ported over to Xperia Z/ZL/T/Ultra? I believe they don't all share the same processor.
Click to expand...
Click to collapse
Yes it wont matter much, since its not s800 it should be easier for you guys , take the kexec-tool use that, implement the patch write to the correct mem addresses which is free, it should boot if you guys have issues let me know,
I need to calculate the correct addresses.
Ive noticed s800 uses a dt.img, might need to modify kexec-tool to support dt.img, not sure what dt.img does yet, only know it holds values

Shaky156 said:
I need to calculate the correct addresses.
Ive noticed s800 uses a dt.img, might need to modify kexec-tool to support dt.img, not sure what dt.img does yet, only know it holds values
Click to expand...
Click to collapse
the dt.img is needed by the kernel to boot, so I guess we need to load that too in kexec.
EDIT: people that wanna try add kexec patch to their kernel, check github: android_kernel_sony_msm8974/commits/kexec

krabappel2548, i have compil your kernel by my script (fromscratch)
My script (instruction in "DoomLord Build kernel thread" : scriptZ1 http://forum.xda-developers.com/attachment.php?attachmentid=2346163&d=1382568778
(for thoses who want to help us...)
You have a little mod to do here (bad compil) :
In "sound/soc/msm/qdsp6v2/rtac.c"
you must change
#include <q6voice.h>
by
#include "q6voice.h"
btw : no more ideas to load kexec for the moment ...

delewer said:
krabappel2548, i have compil your kernel by my script (fromscratch)
My script (instruction in "DoomLord Build kernel thread" : scriptZ1 http://forum.xda-developers.com/attachment.php?attachmentid=2346163&d=1382568778
(for thoses who want to help us...)
You have a little mod to do here (bad compil) :
In "sound/soc/msm/qdsp6v2/rtac.c"
you must change
#include <q6voice.h>
by
#include "q6voice.h"
btw : no more ideas to load kexec for the moment ...
Click to expand...
Click to collapse
Sorry, I'm trying to get caught up on the forum, but what seems to be the current standing issue to get kexec working?

Knucklessg1 said:
Sorry, I'm trying to get caught up on the forum, but what seems to be the current standing issue to get kexec working?
Click to expand...
Click to collapse
Read the OP
Status paragraph

Memory regions
00000000-07afffff : System RAM
00008000-00b79383 : Kernel code
00d04000-00f0cddb : Kernel data
0ff00000-779fffff : System RAM
7ff00000-7ff3ffff : rdtags_mem
7ff80000-7ffa0fff : last_kmsg
7ffa1000-7ffa5fff : last_amsslog
System RAM MEM = 00000000
So --min-mem=0x20000000
Now need to find a free memory area thatll allow us to write and hopefully the mmu/pmu on locked bootloader wont cancel it
@delewer? @DooMLoRD @kali @Bin4ry

I know I shouldn't disturb, but i must ask: if You achieve Your goal, would it be possible to port it to devices like Xperia P, S, T, U and other NXT? It would be great, many ppl are ready to give a prize for it. Thanks in advance, good luck and sorry again.
Sent from my LT22i using xda app-developers app

king960 said:
I know I shouldn't disturb, but i must ask: if You achieve Your goal, would it be possible to port it to devices like Xperia P, S, T, U and other NXT? It would be great, many ppl are ready to give a prize for it. Thanks in advance, good luck and sorry again.
Sent from my LT22i using xda app-developers app
Click to expand...
Click to collapse
These devices are not 2013 devices, they arent s800 socs, so they are much easier to do, simply take the kexec-tools from op, implement the patch in your kernel, write the correct memory values for your specific device and execute in terminal via the command in op, minmem depends on your device too, good luck

I think some1 tried it already, but it works only for unlocked devices... Anyway, thanks for help.
Sent from my LT22i using xda app-developers app

king960 said:
I know I shouldn't disturb, but i must ask: if You achieve Your goal, would it be possible to port it to devices like Xperia P, S, T, U and other NXT? It would be great, many ppl are ready to give a prize for it. Thanks in advance, good luck and sorry again.
Sent from my LT22i using xda app-developers app
Click to expand...
Click to collapse
Does doing this require having an Unlocked Boot loader prior to implementation?
Sent from my C6603 using xda app-developers app

A few informations about kexec-tools debug
in kexec.c
Fonction :
if (file_type.load(argc, argv, kernel_buf,
kernel_size, &info) < 0) {
fprintf(stderr, "Cannot load %s\n", kernel);
return -1;
}
With a forced execution of kexec (bypass error to see...)
--mem-min=0x90000000
kernel: 0xb6b9d008 kernel_size: 3e9340
debug: 1 - after get memory range
debug: 2 - after type test
debug: 3 - after type test
debug: 4 - after info.kexec
debug: Focus 1 - argc '5' ; argv 'be856774' ; kernel_buf 'b6b9d008' ; kernel_size '3e9340' ; info 'be856548' ; i '1' ; file_type.name 'zImage'
Could not find a free area of memory of 3f1340 bytes...
Cannot load zImage
debug: 10 - before trampoline
debug: 11 - after trampoline
debug: 12 - before segment load
debug: 13 - after segment load
debug: 8 - before sort_segment
debug: 9 - after sort_segment
debug: 6 - before purgatory
debug: 7 - after purgatory
kexec_load: entry = (nil) flags = 280004
nr_segments = 0
kexec_load failed: Function not implemented
entry = (nil) flags = 280004
nr_segments = 0
debug: 5 - return result : ffffffff
With a forced bypass on file_type.load , we have this :
--mem-min=0x20000000
debug: Focus 1 - argc '5' ; argv 'bef18774' ; kernel_buf 'b6bc7008' ; kernel_size '3e9340' ; info 'bef18548' ; i '1' ; file_type.name 'zImage'
Segmentation fault

delewer said:
A few informations about kexec-tools debug
in kexec.c
Fonction :
if (file_type.load(argc, argv, kernel_buf,
kernel_size, &info) < 0) {
fprintf(stderr, "Cannot load %s\n", kernel);
return -1;
}
With a forced execution of kexec (bypass error to see...)
--mem-min=0x90000000
kernel: 0xb6b9d008 kernel_size: 3e9340
debug: 1 - after get memory range
debug: 2 - after type test
debug: 3 - after type test
debug: 4 - after info.kexec
debug: Focus 1 - argc '5' ; argv 'be856774' ; kernel_buf 'b6b9d008' ; kernel_size '3e9340' ; info 'be856548' ; i '1' ; file_type.name 'zImage'
Could not find a free area of memory of 3f1340 bytes...
Cannot load zImage
debug: 10 - before trampoline
debug: 11 - after trampoline
debug: 12 - before segment load
debug: 13 - after segment load
debug: 8 - before sort_segment
debug: 9 - after sort_segment
debug: 6 - before purgatory
debug: 7 - after purgatory
kexec_load: entry = (nil) flags = 280004
nr_segments = 0
kexec_load failed: Function not implemented
entry = (nil) flags = 280004
nr_segments = 0
debug: 5 - return result : ffffffff
With a forced bypass on file_type.load , we have this :
--mem-min=0x20000000
debug: Focus 1 - argc '5' ; argv 'bef18774' ; kernel_buf 'b6bc7008' ; kernel_size '3e9340' ; info 'bef18548' ; i '1' ; file_type.name 'zImage'
Segmentation fault
Click to expand...
Click to collapse
Did you compile this kexec yourself? Or did you get this from krapabbel? I issued krapabbel to compile a new debug version have gave him the code but never heard back from him :/
Anywayz so cannot find free memory is the issue

[Dev] HD2 multi-script

I have a script and tools hosted on dropbox.
Stick with the beta script unless you're interested in testing.
What the script can do.
1. Essentially replace aroma in the most basic sense. [Can be made to run with]
It will take any ROM.zip convert the initrd.gz to the desired type (DataOnExt, NativeSD, DirectSD) and copy them to /boot and /boot_dir
Will strip the updater-script of the ROM.zip down to the symlink/set_perm lines along with extract data and system folders.
(Essentially turning an aroma ROM.zip to a basic CWM ROM.zip)
2. Mount appropriate folders
Just tell the script the type of install and the name of the ROM (i.e the ROM folder name on ext4).
Then you can flash any zip you want (as long as the zip doesn't change the mounts)
3. Modify the ramdisk
If you just want the ramdisk modified that can be arranged
4. System install.
In consideration.
It's difficult to get things running as they should without a device to reliably test things on, so need user feedback to correct stuff.
I have been unsuccessful thus far in getting a portadroid type up and running, I'm not sure why at this point.
If the interest is there I can try and add in the updating mechanic from portadroid to Native/Direct, i.e where you have a named folder(s) (open to suggestion on folder names) present in /sdcard/NativeSD and these folders are copied over to the ext4 folders on boot.
How to run:
Instruction can be seen on the dropbox page but:
Simpliest way is via adb.
Copy the bins folder to /sdcard/bins
Reboot to recovery
Push the script to /tmp for example: adb push *.sh /tmp/
Run the script:
adb shell
cd tmp
chmod 777 *.sh
./*.sh
Then read the prompts.
Alternatively you can run it directly as ./*.sh {opt} $type $ROM_NAME $data_NAME $systeminstall
Feel free to report any issues, suggestions/improvements.
I'll be posting a recovery with most of the needed stuff bundled in, so running it will be easier for devs.
Writing the script took some doing (especially as it's in dash - not bash), any feedback would be appreciated.
Credits due.
The portadroid guys (if/when it's up and running)
[cedesmit, Takaaki, boonbing]
The Native/DirectSD guys
[securecrt,Xylograph]
RobbieP for testing thus far.

I have noticed that I already have the code for an update feature in the script just need to add a loop, what would be the general consensus for the location of the script to flash /sdcard/NativeSD/*.zip or /sdcard/NativeSD/ROM_NAME/*.zip?? The second would mean that you can't rename the foldername (unless I manage to add in startup.txt features).
Update added to gamma script - using ROM folder (subject to change)

is this any good for determining kernel version? http://stackoverflow.com/questions/9535227/getting-uname-information-from-a-compressed-kernel-image

I added a modified version of @macs18max CM11 ROM using a version of my gamma script (will pull on request) to dropbox last night, if anyone is willing to test...
1. Does it boot
---If it doesn't add a blank file as /sdcard/bins/makemountfs (echo "" > /sdcard/bins/makemountfs will do it)---
---And try it again; unfortunately this will remove the update feature---
2. Are update zips installed properly
Diff from original:
only one initrd - script makes the others as needed
script performs the partition mounting for the ROM
script checks for zips in the ROM folder in NativeSD
Need to add:
my build.prop append code - once I where I posted it... to reduce duplicates.
Edit. Whoops didn't mean to edit my post from yesterday - @Robbie P just for reference "strings zImage | grep 'Linux version'" is similar to what you posted and can be run on the leo but doesn't seem to give reliable/usable results - I haven't ran the hexdump check on the latest kernel yet so not sure if the check is still valid.

i did try 1_92 on macs18max's rom last night, the kernel was not recognised as a 3.0 kernel. i added 6mb but it didn't boot. I had the extended battery in throughout
i did use the f2fs sd recovery though to flash it, that booted ok and has 3.0 kernel
i still think a simple "if kernel >3mb => 3.0 kernel" haven't checked the latest .35 kernel yet for size.
did you add the adb fix?
will try yours later:fingers-crossed:
thanks
Edit; in fact, size of kernel is the over-riding factor, surely. it is size-of-magldr's-boot-partition minus size-of-initrd.gz(any-type) gives the max size of a kernel before we have to add 6mb. i guess it is around 3mb.
it is conceivable that we may have to add 6mb to the .35 kernel's ramdisk if more code is added.

Robbie P said:
i did try 1_92 on macs18max's rom last night, the kernel was not recognised as a 3.0 kernel. i added 6mb but it didn't boot. I had the extended battery in throughout
i did use the f2fs sd recovery though to flash it, that booted ok and has 3.0 kernel
i still think a simple "if kernel >3mb => 3.0 kernel" haven't checked the latest .35 kernel yet for size.
did you add the adb fix?
I added the ADB 'fix' to on boot - with gamma 4_4 (the ROM zip is using 4_3) - the line I added is with the variable, not ABCD..., might work, moight not
will try yours later:fingers-crossed:
thanks
Edit; in fact, size of kernel is the over-riding factor, surely. it is size-of-magldr's-boot-partition minus size-of-initrd.gz(any-type) gives the max size of a kernel before we have to add 6mb. i guess it is around 3mb.
it is conceivable that we may have to add 6mb to the .35 kernel's ramdisk if more code is added.
You could be right about the 3mb size always needing 6MB but I can't be sure. My feeling is it is a 3.0.x issue rather than a size one (I could be wrong though). 3.0.x kernels could probably be cut down (esp. recovery kernels) to be less than 3MB (and vice versa with the 2.6.x) that's why I was keen to stay away from size as being the determining factor - I'll switch to using sizes in the next gamma.
Click to expand...
Click to collapse
ADB fix with 4_4 is:
sed -i "/on boot/ a write /sys/class/android_usb/android0/iSerial \${ro.serialno}" /tmp/work/init.htcleo.rc
I could try variable substitution \${ro.serialno:-ABCDEF123456} but not sure if that works with .rc's
Edit. 4_5 kernel check changed to:
Code:
[strike]zI=` busybox ls -la /tmp/zImage | awk '{print (${NF-4)}' `
if [ ${zI} -gt '3145728' ]; then...[/strike]
zI=` du -m /tmp/zImage | awk '{print $(NF-1)}' `
if [ ${zI} -gt '2' ]; then...

mac rom; i have an error from aroma-config line 147 col 40, pretty sure it is missing comma on line 145 after "1"
rezipping and re-flashing. is the rom-name different from original?
it went through aroma options (chose directsd, softkeys,wipe), then installed immediately, no files actually installed, attached recov logg
i did have original rom.zip on sdcard and nativesd rom installed, might have to delete either, or both?
Edit; deleted original rom.zip and used 4ext recovery (f2fs sd previously), but same, installs successfully immediately, log did not save unfortunately

Robbie P said:
mac rom; i have an error from aroma-config line 147 col 40, pretty sure it is missing comma on line 145 after "1"
rezipping and re-flashing. is the rom-name different from original?
it went through aroma options (chose directsd, softkeys,wipe), then installed immediately, no files actually installed, attached recov logg
i did have original rom.zip on sdcard and nativesd rom installed, might have to delete either, or both?
Edit; deleted original rom.zip and used 4ext recovery (f2fs sd previously), but same, installs successfully immediately, log did not save unfortunately
Click to expand...
Click to collapse
Have reupped with aroma fixed, rom-name was the same as original (folder in NativeSD would be different). Reupped version has _test added to it. recov.logg reports that system was installed??? Based on the other stuff in the log I don't think it's made from the zip.
type=NativeSD rom_name=cm11_ma and command run was m(ntstuff) rather then m(odify)r(amdisk)...
Yea - the log is from a different zip install:
Code:
Installing '/sdcard/cm11_ma_NativeSD.zip'...
Checking for MD5 file...
I:Cannot find file /sdcard/cm11_ma_NativeSD.zip.md5
Skipping MD5 check: no MD5 file found.
I:Zip does not contain SELinux file_contexts file in its root.
about to run program [/tmp/multi.sh] with 5 args
There really shouldn't be any conflicts with other zips here... Is there any text reported on screen after/before the system supposedly installs?
Edit. Anyone know where the HD2 HaRET source is located? Anyone trying to run 3.0.x with haret try adding 1MB of zeros to the initrd.gz (PM if you don't know how).

sorry about that log, was the one from previous night's install.
i just tried GAMMA4_5.sh DirectSD cm11_m
get screen.txt attached
i know @gilbert32 was looking at getting haret to boot with 3.0 kernel

Robbie P said:
sorry about that log, was the one from previous night's install.
i just tried GAMMA4_5.sh DirectSD cm11_m
get screen.txt attached
i know @gilbert32 was looking at getting haret to boot with 3.0 kernel
Click to expand...
Click to collapse
Could you try just using the zip on the dropbox page - the sh and bins are already added to it. No need to run any script. I might need to change the bundled script though. Picking through the errors in the log:
sh: 3.1: bad number
is from the kernel check - I added h (so that it reports 3.1M rather than 3, wasn't sure how it would handle the decimal) - reverted 4_5
cp: can't stat '/tmp/mountfs-DirectSD.sh': No such file or directory
copying fsck for f2fs
cp: can't create '/system/bin/mkfs.f2fs': No such file or directory
cp: can't create '/system/bin/fsck.f2fs': No such file or directory
chmod: /system/bin/mkfs.f2fs: No such file or directory
chmod: /system/bin/fsck.f2fs: No such file or directory
gamma doesn't use mountfs.sh's by default
f2fs added in error?? Can't copy to system/bin as there isn't one
/GAMMA4_5.sh: line 1236: /tmp/7za: not found
Should have been /bin/7za - fixed 4_5
sh: porta: unknown operand
Trying to figure this one out - it's line 1197 - fixed 4_5 - still not sure why it reported as an error (nested if??)
​Current 4_5 bundled in ROM zip
My thinking regarding haRET - the source I saw (different device) seems to have the ramdisk offset by 5MB. Offset for 3.0.x is 16MB (magldr default offset is 10MB plus the 6MB of zeroes = 16MB). If the offset in haret is matched to that of magldr (10MB) then 1 more MB is needed to get to to 16 - if haret is compiled to add a 5MB offset. Source I was referencing is here.

Robbie P said:
sorry about that log, was the one from previous night's install.
i just tried GAMMA4_5.sh DirectSD cm11_m
get screen.txt attached
i know @gilbert32 was looking at getting haret to boot with 3.0 kernel
Click to expand...
Click to collapse
funny thing, i just was looking into it (after a long time since being on xda), and saw this mention
still, i can't find any clue on what to do, or what's the cause for the no-boot with haret...

tried latest mac.zip, have attached logs
i have a work assessment coming up in the next couple of days, so need to get my head down.:fingers-crossed:

Robbie P said:
tried latest mac.zip, have attached logs
i have a work assessment coming up in the next couple of days, so need to get my head down.:fingers-crossed:
Click to expand...
Click to collapse
Get your head down, mate. Just some missing ,'s in the updater-script (used to separate arguments). Should be fine now - at least on the non-script side of things.
Correction - it's fine now (sh was wiping /tmp and thus /tmp/aroma before it was needed by updater-script), managed to botch a working device again - currently installing as DirectSD on cLK.

HypoTurtle said:
Anyone trying to run 3.0.x with haret try adding 1MB of zeros to the initrd.gz (PM if you don't know how).
Click to expand...
Click to collapse
Just tried this and it still doesn't boot, same as before, hangs on jumping to kernel
edit; tried changing initrd offset in startup.txt from 0x00a00000 to 0x00b00000 and 0x00900000 but no joy either

Robbie P said:
Just tried this and it still doesn't boot, same as before, hangs on jumping to kernel
edit; tried changing initrd offset in startup.txt from 0x00a00000 to 0x00b00000 and 0x00900000 but no joy either
Click to expand...
Click to collapse
nah, this isn't the problem. it's somewhere else and i can't figure it out we have to stick with clk or mag for 3.0.x atm

gilbert32 said:
nah, this isn't the problem. it's somewhere else and i can't figure it out we have to stick with clk or mag for 3.0.x atm
Click to expand...
Click to collapse
How about mtype? we know that the .35 kernel was originally from desire. is it possible that it still has mtype set to 2215 somewhere? i tried setting it to this in startup.txt but same.
Edit; might have found haret source code https://code.google.com/p/android-kaiser/downloads/detail?name=haret.tar.gz&can=2&q=

@gilbert32 @Robbie P
Just for reference: haretlog from 2.x
Built virtual to physical page mapping
Allocated 3124 pages (tags=5C000000/16368000 kernel=5C001000/16369000 initrd=5C2BB000/2c161000 index=5CC2D000/2b7ef000)
Built kernel tags area
Built page index
Tags will be at offset 0x00000100 (0xf00)
Kernel will be at offset 0x00008000 (0x2b9800) [2.72MB]
Initrd will be at offset 0x00a00000 (0x971ad6)
Video buffer at 49739000 sx=480 sy=800 mx=120 my=133
Video Phys FB=03839000 Fonts=2b7ea0e4
[email protected]/2b7e9000 sj=5CC33278 stack=5CC31000/2b7eb000 data=5CC32000/2b7ea000 exec=2b7e93a0
Reading 2856960 bytes...
Read complete
Reading 9902806 bytes...
Read complete
Launching to physical address 2b7e9288 [695.91MB]
Trampoline setup ([email protected]/10028f68/1323ff68) [256.16MB][306.25MB]
MMU setup: mmu=A04C0000/11cc0000
Go Go Go...​and from 3.x
Built virtual to physical page mapping
Allocated 3308 pages (tags=5C900000/163e0000 kernel=5C901000/1649a000 initrd=5CC73000/16294000 index=5D5E5000/2b87c000)
Built kernel tags area
Built page index
Tags will be at offset 0x00000100 (0xf00)
Kernel will be at offset 0x00008000 (0x3715b0) [3.44MB]
Initrd will be at offset 0x00a00000 (0x971ad6)
Video buffer at 49739000 sx=480 sy=800 mx=120 my=133
Video Phys FB=03839000 Fonts=2b8770e4
[email protected]/2b876000 sj=5D5EB278 stack=5D5E9000/2b878000 data=5D5EA000/2b877000 exec=2b8763a0
Reading 3610032 bytes...
Read complete
Reading 9902806 bytes...
Read complete
Launching to physical address 2b876288 [696.46MB]
Trampoline setup ([email protected]/1a028f68/12ca6f68) [416.16MB][300.65MB]
MMU setup: mmu=A04C0000/11cc0000
Go Go Go...​There's probably nothing there of any help. Physical address is changed by about 1.5MB?? 160MB in trampoline == graphics??

Double post

how about booting a 2.6.32 kernel from haret and then using kexec to change to 3.0 kernel in running rom?

HypoTurtle said:
@gilbert32 @Robbie P
Just for reference: haretlog from 2.x
Built virtual to physical page mapping
Allocated 3124 pages (tags=5C000000/16368000 kernel=5C001000/16369000 initrd=5C2BB000/2c161000 index=5CC2D000/2b7ef000)
Built kernel tags area
Built page index
Tags will be at offset 0x00000100 (0xf00)
Kernel will be at offset 0x00008000 (0x2b9800) [2.72MB]
Initrd will be at offset 0x00a00000 (0x971ad6)
Video buffer at 49739000 sx=480 sy=800 mx=120 my=133
Video Phys FB=03839000 Fonts=2b7ea0e4
[email protected]/2b7e9000 sj=5CC33278 stack=5CC31000/2b7eb000 data=5CC32000/2b7ea000 exec=2b7e93a0
Reading 2856960 bytes...
Read complete
Reading 9902806 bytes...
Read complete
Launching to physical address 2b7e9288 [695.91MB]
Trampoline setup ([email protected]/10028f68/1323ff68) [256.16MB][306.25MB]
MMU setup: mmu=A04C0000/11cc0000
Go Go Go...​and from 3.x
Built virtual to physical page mapping
Allocated 3308 pages (tags=5C900000/163e0000 kernel=5C901000/1649a000 initrd=5CC73000/16294000 index=5D5E5000/2b87c000)
Built kernel tags area
Built page index
Tags will be at offset 0x00000100 (0xf00)
Kernel will be at offset 0x00008000 (0x3715b0) [3.44MB]
Initrd will be at offset 0x00a00000 (0x971ad6)
Video buffer at 49739000 sx=480 sy=800 mx=120 my=133
Video Phys FB=03839000 Fonts=2b8770e4
[email protected]/2b876000 sj=5D5EB278 stack=5D5E9000/2b878000 data=5D5EA000/2b877000 exec=2b8763a0
Reading 3610032 bytes...
Read complete
Reading 9902806 bytes...
Read complete
Launching to physical address 2b876288 [696.46MB]
Trampoline setup ([email protected]/1a028f68/12ca6f68) [416.16MB][300.65MB]
MMU setup: mmu=A04C0000/11cc0000
Go Go Go...​There's probably nothing there of any help. Physical address is changed by about 1.5MB?? 160MB in trampoline == graphics??
Click to expand...
Click to collapse
@HypoTurtle @gilbert32 @Robbie P
How do I get this logs from haret. I have just install win6.1. Could someone guide me?

[DEV] Building a custom kernel and kernel modules for stock kernel

Since fire phone doesn't have a bootloader unlock at the moment. There is no point in building a custom kernel. But By building a kernel we can build kernel modules which work on the stock kernel. And yes you can load unsigned kernel modules without a problem since fire phone doesn't use tz apps to verify kernel modules like Samsung does.
Setup
Source
Download the fire phone sources for firmware 4.6.1 from here. And extract the platfrom.tar inside the archive to somewhere(KERNEL_DIR).
toolchain
You can use the android ndk from google, But it requires some setup. I'm using linaro toolchain from here. You can use compiler version 4.7, 4.8 or 4.9. Kernel I'm using (Firmware 4.6.3 - Linux 3.4-perf-g280c96c) is built with gcc-4.7. But I'm using this gcc-4.9. Download it, extract is somewhere(TOOLCHAIN_DIR) and add the $TOOLCHAIN_DIR/bin to your PATH. Theoretically you would be able to build the kernel on windows using Cygwin or MSYS tools but using Linux is better.
config
Connect your phone trough adb and run
Code:
adb pull /proc/config.gz
zcat config.gz > $KERNEL_DIR/kernel/qcom/3.4/.config
With this config you will run into some problems because of a missing "trapz_generated_kernel.h". I don't know if this is an auto generated file when they build android as a whole or amazon removed this explicitly(can they do that without violating GPL?). Anyway It looks trapz is some low level kernel debugging function(comment here if you know more about it). We can safely disable it. Open $KERNEL_DIR/kernel/qcom/3.4/.config in a text editor and change the lines
Code:
CONFIG_TRAPZ=y
CONFIG_TRAPZ_TP=y
CONFIG_TRAPZ_TRIGGER=y
CONFIG_HAVOK=y
to
Code:
#CONFIG_TRAPZ=y
#CONFIG_TRAPZ_TP=y
#CONFIG_TRAPZ_TRIGGER=y
#CONFIG_HAVOK=y
building
Now edit the $KERNEL_DIR/kernel/qcom/3.4/Makefile and add this changes
Code:
EXTRAVERSION = -perf-g280c96c
This is at the top of the makefile. If we don't add this, vermagic for the modules will differ from stock kernel and they won't load.
ARCH=arm
CROSS_COMPILE=arm-linux-gnueabihf-
Click to expand...
Click to collapse
Here arm-linux-gnueabihf- is my cross compiler frefix. Look in $TOOLCHAIN_DIR/bin/ to find it.
Now cd into $KERNEL_DIR/kernel/qcom/3.4/ and do
Code:
make
The build will fail a few times complaining about missing headers. Most of the time it's just
Code:
#include <myheader.h>
instead of
Code:
#include "myheader.h"
Edit the source file where the build fails and change <>s to ""s. (maybe android ndk ignores the difference and include the headers anyway)
After kernel compiles, we are good to go. We can use this kernel sources to build kernel modules for stock kernel.
Kernel modules
To build the kernel modules, we basically need two things. An approximate kernel source and the Module.symvers file from the original kernel. We can get the Module.symvers file by building the complete kernel as explained above or Just extract it from our stock kernel.
To extract the Module.symvers from the stock kernel, extract the boot.img file from firmware update image. Get mkbootimg tools from here compile it and run
Code:
unmkbootimg --kernel zImage ---ramdisk ramdisk.cpio.gz -i boot.img
After you get the zImage. Download extract-symvers script from here and run
Code:
python2 extract-symvers.py -B 0xc0008000 zImage > Module.symvers
place this file in $KERNEL_DIR/kernel/qcom/3.4/ (You still have to do the changes mentioned above in kernel config and building section run make in the $KERNEL_DIR/kernel/qcom/3.4 and intrupt it after few seconds)
Now you can build loadable modules against this source. Here is a hello world kernel module.
Code:
//hello.c
#include<linux/module.h>
#include<linux/kernel.h>
#include<linux/init.h>
static int __init hello_start(void)
{
printk("hello to the world from module");
return 0;
}
static void __exit hello_end(void)
{
printk("heloo exit");
}
module_init(hello_start);
module_exit(hello_end);
Code:
#Makefile
KERNEL_DIR=<your kernel dir>/kernel/qcom/3.4
obj-m := hello.o
PWD := $(shell pwd)
default:
$(MAKE) ARCH=arm CROSS_COMPILE=armeb-linux-gnueabi- -C $(KERNEL_DIR) SUBDIRS=$(PWD) modules
Put this files in a folder and run make in it. Change the paths and cross compiler prefix according to your setup. and run make.
After the build push the hello.ko to the phone.
Code:
adb push hello.ko /sdcard/
adb shell
su
cd sdcard
insmod hello.ko
run dmesg and you'll see the message.
I'm currently trying to build kexec module from hashcode's sources and USB OTG modules.
I'm attaching a few thing helped me do this.

since they have released this version of the fire os they have to provide the source code
see
http://www.gnu.org/licenses/gpl-faq.html#GPLRequireSourcePostedPublic
you have just shown that the source code they releases for the kernel does not match the one used to build the kernel. This means it is a clear violation of the gpl and amazon is in breach and can be sued.
on another note.
are the drivers for the nfc and camera compiled as a module or into the kernel?

They have yet to provide 4.6.3 and 4.6.4 kernel sources too.
I don't know exactly but in order for NFC and camera to work drivers are required and they are in fact compiled into the kernel.
The problem we currently have with NFC and camera is proprietary hal (hardware abstraction libraries) They are a part of Android and does not subject to GPL. Amazon changed the original android way how hal works and didn't release the sources!
by looking at the kernel drivers maybe we would be able to implement hal from scratch. But I don't see that intense dev support for fire phone. If you are up for it camera sources are at $KERNEL_DIR/kernel/qcom/3.4/drivers/media/platform/msm/camera_v2/

Major MAJOR respect for all of you making the Fire Phone even better!

@madushan1000
Could we do something like this to install a custom boot.img?
http://forum.xda-developers.com/optimus-l9/general/guide-install-custom-roms-locked-t3249828

I don't own this device but has anyone tried to see if kexec works?

spudowiar said:
I don't own this device but has anyone tried to see if kexec works?
Click to expand...
Click to collapse
Nope, I was working on it. But then I got a job. It will be sometime before I can start working on it again.

Could someone please provide the config extracted from /proc/config.gz?
I can't find this on CM11 rom for some reason.

Building the kernel now.
Some bugs are in the code and -Wall and gcc-wrapper.py escalate the warnings.
I wonder if those errors are there on purpose XD
helloworld.ko loaded successfully
I was able to execute kexec without anything. Just the binary.
Will keep you posted - this hacking might take a while to figure it all out.
I already have 3.4 kernel from the amazon sources.
I have the kexec userland program.
What is left is a loadable kexec kernel module (if that is possible at all).

removed

Okarin said:
Are we even sure those Amazon Kernel Sources are correct?
Those errors caught by the wrapper scripts are giving me the creeps.
Git the kexec_load.ko build.
Currently hands on insmod.
Phone doesn't do a reboot any longer:
insmod kexec_load.ko
init_module(0xb6e6c008, 408241, "") = -1 ENOENT (No such file or directory)
write(2, "insmod: init_module '/sdcard/kex"..., 79insmod: init_module '/sdcard/kexec_load.ko' failed (No such file or directory)
) = 79
munmap(0xb6e6c000, 409600) = 0
mprotect(0xb6f8c000, 4096, PROT_READ|PROT_WRITE) = 0
mprotect(0xb6f8c000, 4096, PROT_READ) = 0
close(0) = 0
close(1) = 0
close(2) = 0
futex(0xb6f6cd74, FUTEX_WAKE_PRIVATE, 2147483647) = 0
munmap(0xb6f8c000, 4096) = 0
exit_group(-1) = ?
First goal is to get module loaded.
Goal reached:
kexec_load 27813 0 - Live 0x00000000 (O)
procfs_rw 12770 0 - Live 0x00000000 (O)
wlan 3793980 0 - Live 0x00000000 (O)
Shouldn't be functional at all..
I disabled some function calls just to get the module loaded.
The missing symbols are:
soft_restart
arch_kexec
machine_shutdown
And the version I use does some insane function hooking ..
More rework is needed.
[email protected]:/data/local # ./kexec /sdcard/vmlinux
kernel: 0xaf12d008 kernel_size: 7e1354c
unrecoverable error: could not scan "/proc/device-tree/": No such file or directory
<6>[ 97.681256] Kexec_load: Replacement... :
<6>[ 97.681344] kexec_load : my_syscall_table : c0106244
<6>[ 97.681405] kexec_load : kexec_load before replacement : c01b346c
<6>[ 97.681480] kexec_load : kexec_load after replacement : bf3a5650
<6>[ 97.681546] kexec_load : reboot before replacement : c01a83f0
<6>[ 97.681616] kexec_load : reboot after replacement : bf3a6348
<6>[ 97.681675] Kexec_load: End replacement... :
<6>[ 202.694691] Kexec: - Starting kexec_load...
<6>[ 202.694849] Kexec: - ---- kexec_load - result : '0'
It gets better:
255|[email protected]:/data/local # ./kexec --dtb=/sdcard/zImage-dtb /sdcard/vmlinux
kernel: 0xaf1b1008 kernel_size: 7e1354c
kexec-zImage-arm : dtb.img BEFORE CUT : Start : '0xae66f008' - Length : '0xb411e9' - End : '0xaf1b01f1'
Segmentation fault
More tomorrow.
Click to expand...
Click to collapse
Where are you getting your kexec module sources from? BTW try using the original amazon kernal binary the phone is shipped with (we are sure it works). Don't use the custom kernel for the kexec tests (We don't know the custom kernel actually works)

madushan1000 said:
Where are you getting your kexec module sources from? BTW try using the original amazon kernal binary the phone is shipped with (we are sure it works). Don't use the custom kernel for the kexec tests (We don't know the custom kernel actually works)
Click to expand...
Click to collapse
Here is the thread I used as a starting point.
I will put up my "fork" on github after I get permission to do that
The userland part build like a charm once I took the compiler you recommended.
The kernel-module was tricky because the whole thing is modded like hell.
To be able to load I had to comment out some hard coded addresses and the calls to unresolvable symbols.

removed

Okay the kernel gets loaded.
But
kexec -e shuts off the device.
strace doesn't help.
On the plus side:
Devices are shutdown
Look promising
I need a way to tail dmesg ...
Okay a lot of digging around and I found out that the reboot syscall doesn't work properly..
It doesn't look like it hits the kexec_module it looks more like it hits the actualy sys_reboot
Okay reboot syscall hits my reboot-hook.
But the softreboot doesn't work now.
Okay there is some kind of watchdog runnig which doesn't like my kexec.
I need to kill it - that should happen tomorrow.

removed

I hit the same wall when I tried to isolate the kexec code from the kernel itself to a module. I stopped working on it because I lacked the time. BTW the error you are facing now
<3>[ 80.580644] BUG: scheduling while atomic: kexec/4067/0x00000002
Click to expand...
Click to collapse
is because memory allocator is trying to switch threads while you are in a syscall. it's because of lines like this
image = kzalloc(sizeof(*image), GFP_KERNEL);
Click to expand...
Click to collapse
Try changing GFP_KERNEL to GFP_ATOMIC. Other than that, I have another suggestion. Try to get the kernel to run in a single core mode before running kexec code. This might simplify things. I don't know how to do this though.

madushan1000 said:
I hit the same wall when I tried to isolate the kexec code from the kernel itself to a module. I stopped working on it because I lacked the time. BTW the error you are facing now
is because memory allocator is trying to switch threads while you are in a syscall. it's because of lines like this
Try changing GFP_KERNEL to GFP_ATOMIC. Other than that, I have another suggestion. Try to get the kernel to run in a single core mode before running kexec code. This might simplify things. I don't know how to do this though.
Click to expand...
Click to collapse
The atmic error is gone now. It went away after I disabled the watchtog.
smp_disable() is what you are looking for - but this causes the system to hard_reboot ATM XD

what happens if you kill every userlevel program before smp_disable()?

removed

#define tomorrow
Okay .. I worked out the preemption thing.
At least it does something.
Still a black screen and the MSM_WATCHDOG is a ***** again.
It needs to be suspended .. at least that what I get from the code I read here.
If I remove the driver too early the output in /proc/kmsg stops ..
If I try to remove it too late ... well it causes a resched while atomic.