Develop Bugs

Google Android and Linux for Kaiser Volume II

The original thread:http://forum.xda-developers.com/showthread.php?t=396782 needs an abridged version.
==================================================
Go to http://www.androidonhtc.com/ if you're just starting as it has the latest info. This thread is to highlight the info from the original thread only.
Latest Builds
Port Status
==================================================
Compiling Android Kernel for Kaiser
Modify initrd.gz files and CPIO handling
system.img mounting, editing and rebuilding with ext2/3
system.img mounting, editing and rebuilding with cramfs
Howto: Pull from git (new/update/resync)
[WIP] Configuring WiFi Interface
==================================================

system.img mounting, editing and rebuilding with ext2/3
seidler2547: Post:
Actually I've played with Android a bit for now, and I changed to ext3. It doesn't only work - it's much faster, too! Startup time during the blinking android is about half of what it was before.
How-To:
Code:
Code:
cd /tmp
# prepare dirs
mkdir a-sys
mkdir a-ext
# prepare image
dd if=/dev/zero of=/where/is/sdcard/system.img.new bs=1M count=64
mkfs.ext3 /where/is/sdcard/system.img.new
# mount old image and copy to new
mount -o loop /where/is/the/system.img a-sys
mount -o loop /where/is/sdcard/system.img.new a-ext
cp -a a-sys/* a-ext/
Now you can unmount the old image and happily edit in the new image. Don't forget to rename the system.img.new to system.img (after you have unmounted it).
In your initrd, in file init, where it says
Code:
losetup /dev/block/loop1 /sdcard/system.img
...
mount -t cramfs -o ro,noatime,nodiratime /dev/block/loop1 /system
change "-t cramfs" to "-t ext2" or "-t ext3". You can also change the path (/sdcard/system.img) there.
Click to expand...
Click to collapse

Modify initrd.gz files and CPIO handling
dcordes: Post:
There is no magick in the initrd files. They are .cpio.gz files, gzipped cpio balls. To extract a .cpio.gz file named initrd-android.cpio.gz simply do
Code:
gunzip initrd-android.cpio.gz && cpio -i < initrd.android.cpio
Then you have the extracted rootfs. The reverse way would be, assuming you are inside your rootfs folder:
Code:
find ./ | cpio -H newc -o | gzip > ../my-initr-android-with-custom-stuffs.cpio.gz
And yes, you can remove and add applications you find that way.
Click to expand...
Click to collapse

system.img mounting, editing and rebuilding with cramfs
dzo: Post:
Hi, you can't just use mkcramfs on the system folder because the permissions will be wrong. This is the script I use:
Code:
Code:
out/host/linux-x86/bin/genext2fs -d out/target/product/generic/system -b 80000 -a system.ext2
mount -o loop system.ext2 /mnt/system
cp /mnt/system/usr/keychars/qwerty2.kcm.bin /mnt/system/usr/keychars/vogue-ts.kcm.bin
cp com.google.android.maps.jar /mnt/system/framework
cp Maps.apk Street.apk /mnt/system/app
mkfs.cramfs /mnt/system system.img
umount /mnt/system
#pcp system.img :/Storage\ Card/system.img
This also puts the maps app in (just copy from one of my images) and the vogue keymap. Without the source for the ril you will also need to copy my RIL (libreference-ril.so).
Click to expand...
Click to collapse

[WIP] Configuring WiFi Interface
This has been able to initialize the interface, assign arbitrary IP addresses but can not go further at the moment.
Code:
# ifconfig tiwlan0 192.168.1.100
# ifconfig tiwlan0 up
error: SIOCSIFFLAGS (Cannot assign requested address)
# ifconfig tiwlan0
tiwlan0: ip 192.168.1.100 mask 255.255.255.0 flags (down broadcast multicast)
dmesg will show:
Code:
wlan: no version for "struct_module" found: kernel tainted.
TIWLAN: Driver loading
trout_wifi_power: 1
trout_wifi_reset: 0
trout_wifi_set_carddetect: 1
TIWLAN: Found SDIO control (vendor 0x104c, device 0x9066)
TIWLAN: Driver initialized (rc 0)
TIWLAN: Driver loaded
Android's built-in wireless settings seem to disable the interface beyond just interfering with it, therefore it's best to stay with terminal and using 'ash' will give you a shell with command history (up/down scroll).
markya23: Post:
Need to create a folder in you system image package called /etc/wifi and copy tiwlan.ini, wpa_supplicant.conf and fw1251r1c.bin.
Need to copy the wlan.ko to /lib/modules in the system image (create the dir if required).
Create the new system image and boot Android. Start the dev console and type:
Code:
cp /system/etc/wifi/wpa_supplicant.conf /data/misc/wifi/wpa_supplicant.conf
insmod /system/lib/modules/wlan.ko
wlan_loader -f /system/etc/wifi/Fw1251r1c.bin -e /proc/calibration -i /system/etc/wifi/tiwlan.ini
cd /data/local/tmp
wpa_supplicant -f -Dtiwlan0 -itiwlan0 -c/data/misc/wifi/wpa_supplicant.conf &
ifconfig tiwlan0 192.168.1.100 netmask 255.255.255.0
ifconfig tiwlan0 up
Click to expand...
Click to collapse

Compiling Android Kernel for Kaiser
dwaradzyn: Post:
Here are brief instructions on how to compile android kernel for Kaiser from git.linuxtogo.org repository. I assume that running OS is Linux and it has everything required to build x86 or ia64 kernel. Beside that latest git software should be installed. The shell is assumed to be bash.
1. Let's start with creating a directory for kernel in home directory:
Code:
mkdir ~/android-kernel
cd android-kernel
2. Next thing is to get the sources from repository. To make it happen (this could take a while, it downloads 280MB):
Code:
git clone git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git
OUTPUT:
Code:
Initialized empty Git repository in /home/user/android-kernel/kernel/.git/
remote: Counting objects: 908251, done.
remote: Compressing objects: 100% (153970/153970), done.
remote: Total 908251 (delta 755115), reused 906063 (delta 753016)
Receiving objects: 100% (908251/908251), 281.86 MiB | 292 KiB/s, done.
Resolving deltas: 100% (755115/755115), done.
Checking out files: 100% (22584/22584), done.
3. The htc-msm branch is of our interest (again it could take a few seconds):
Click to expand...
Click to collapse
*** Update, poly_poly-man states we are working off of htc-vogue not htc-msm. I'm leaving the original code here but I would urge you to modify the next line as poly has suggested:
Code:
cd kernel
git checkout -b htc-msm origin/htc-msm
OUTPUT:
Code:
Branch htc-msm set up to track remote branch refs/remotes/origin/htc-msm.
Switched to a new branch "htc-msm"
4. Let's take care of arm toolchain. Download this file (64MB) into ~/android-kernel:
Code:
[url]http://www.codesourcery.com/gnu_toolchains/arm/portal/package2549/public/arm-none-linux-gnueabi/arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2[/url]
Unpack it:
Code:
cd ~/android-kernel
tar xjf arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
5. Compile the kernel
Prepare default .config for Kaiser:
Code:
cd ~/android-kernel/kernel
make htckaiser_defconfig ARCH=arm
OUTPUT:
Code:
........
lots of output
........
# configuration written to .config
#
And finally compile the kernel to get zImage (takes a minute or two):
Code:
export PATH=~/android-kernel/arm-2008q1/bin:$PATH
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
OUTPUT:
Code:
........
lots of output
........
Kernel: arch/arm/boot/zImage is ready
Now copy ~/android-kernel/kernel/arch/arm/boot/zImage to your phone and play with it.
Some ending tips:
A. You can compile earlier versions of sources in repository. To do that click on one of "commit" links on page:
Code:
http://git.linuxtogo.org/?p=groups/mobile-linux/kernel.git;a=summary
and read commit id (for example: f9d1bcea9342348623f5a57588044f76d8b649cd):
Code:
git reset --hard f9d1bcea9342348623f5a57588044f76d8b649cd
It will override any changes you made to files in ~/android-kernel/kernel.
B. Once you have downloaded git repository, you can swallow latest changes by issuing:
Code:
cd ~/android-kernel/kernel
git pull
C. If your machine has more than one cpus/cores you can speed up kernel compilation by adding -j <cores/cpus_number>, for example (dual core):
Code:
make -j 2 zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse

bad internet makes for double post. Please delete

wrong directions for kernel - we are working off of htc-vogue branch... not htc-msm...

can we make the internet work through the usb cable?

how does android know which device node is gps? it's not picking it up for kaiser...
if you enable gps in winmo (or enable it in smd0 - I believe the command is @startgps), smd7 is a nmea stream (acts as a serial GPS).... could a symlink possibly be the right solution to this?

Howto: Pull from git (new/update/resync)
This will download the latest from git:
dwaradzyn: Post:
Here are brief instructions on how to compile android kernel for Kaiser from git.linuxtogo.org repository. I assume that running OS is Linux and it has everything required to build x86 or ia64 kernel. Beside that latest git software should be installed. The shell is assumed to be bash.
1. Let's start with creating a directory for kernel in home directory:
Code:
mkdir ~/android-kernel
cd android-kernel
2. Next thing is to get the sources from repository. To make it happen (this could take a while, it downloads 280MB):
Code:
git clone git://git.linuxtogo.org/home/grou
ps/mobile-linux/kernel.git
OUTPUT:
Code:
Initialized empty Git repository in /home/user/android-kernel/kernel/.git/
remote: Counting objects: 908251, done.
remote: Compressing objects: 100% (153970/153970), done.
remote: Total 908251 (delta 755115), reused 906063 (delta 753016)
Receiving objects: 100% (908251/908251), 281.86 MiB | 292 KiB/s, done.
Resolving deltas: 100% (755115/755115), done.
Checking out files: 100% (22584/22584), done.
3. The htc-msm branch is of our interest (again it could take a few seconds):
Code:
cd kernel
git checkout -b htc-msm origin/htc-msm
OUTPUT:
Code:
Branch htc-msm set up to track remote branch refs/remotes/origin/htc-msm.
Switched to a new branch "htc-msm"
4. Let's take care of arm toolchain. Download this file (64MB) into ~/android-kernel:
Code:
http://www.codesourcery.com/gnu_toolchains/arm/portal/package2549/public/arm-none-linux-gnueabi/arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
Unpack it:
Code:
cd ~/android-kernel
tar xjf arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
5. Compile the kernel
Prepare default .config for Kaiser:
Code:
cd ~/android-kernel/kernel
make htckaiser_defconfig ARCH=arm
OUTPUT:
Code:
........
lots of output
........
# configuration written to .config
#
And finally compile the kernel to get zImage (takes a minute or two):
Code:
export PATH=~/android-kernel/arm-2008q1/bin:$PATH
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
OUTPUT:
Code:
........
lots of output
........
Kernel: arch/arm/boot/zImage is ready
Now copy ~/android-kernel/kernel/arch/arm/boot/zImage to your phone and play with it.
Some ending tips:
A. You can compile earlier versions of sources in repository. To do that click on one of "commit" links on page:
http://git.linuxtogo.org/?p=groups/mobile-linux/kernel.git;a=summary
and read commit id (for example: f9d1bcea9342348623f5a57588044f76d8b649cd):
Code:
git reset --hard f9d1bcea9342348623f5a57588044f76d8b649cd
It will override any changes you made to files in ~/android-kernel/kernel.
B. Once you have downloaded git repository, you can swallow latest changes by issuing:
Code:
cd ~/android-kernel/kernel
git pull
C. If your machine has more than one cpus/cores you can speed up kernel compilation by adding -j <cores/cpus_number>, for example (dual core):
Code:
make -j 2 zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
poly_poly-man: Post:
This will pull from git which will update/resync a git pull:
Code:
mkdir foo
cd foo
git init
git pull git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git htc-vogue
Click to expand...
Click to collapse

I'll follow-up with some nice, full instructions...
1. prerequisites: arm-none-linux-gnueabi toolchain (gentoo users can use crossdev - otherwise.... uhh... idk?), git, a host toolchain (gentoo users have this by default, most other distros have this under "development" in their package managment... if you have gcc, you're probably set).
2. mkdir kernel
cd kernel
git init
git pull git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git htc-vogue
3. make vogue_defconfig
4. make
5. cp arch/arm/boot/bzImage /path/to/sdcard/
6. to update, run the git pull command by itself again, run make (may have to do the config line again if it's changed) and cp.

Thanks for the post poly. I don't know how that's different from what the post I referenced as I'm not +4 at this stuff.. I did what you posted with android kernel from git and got a 1.2mb zImage that crashed HaRET.. I'm guessing this is my bad. What could I have overlooked? Thanks

enatefox said:
Thanks for the post poly. I don't know how that's different from what the post I referenced as I'm not +4 at this stuff.. I did what you posted with android kernel from git and got a 1.2mb zImage that crashed HaRET.. I'm guessing this is my bad. What could I have overlooked? Thanks
Click to expand...
Click to collapse
where'd you get your toolchain?
does building a regular (host arch) kernel work?
Also - what's the proper way to build a system.img by hand? I'm looking to modify that quite a bit, but can't find a persistent source tree besides the main one, which is seriously crippled.

I thought you were one of the experts, lol. I've been left with no support on how dzo, et all are customizing kernels so I've been in read only mode on the 'other thread' looking elsewhere for support.
Just wanted to ask you first, what's with the Dream radio? I know you posted the mods censored it but what's with the sig now? It piqued my interest... as I'm using (shudder) winmo on the regular while Android is being worked on I was hoping it would be worth looking into if it doesn't brick my phone.
Answers to your questions:
As I said, my own zImage is no go. Check this link (not for our phone but the links at the bottom are pretty useful): http://wiki.xda-developers.com/index.php?pagename=BlackstoneLinux#Runningx20.Linuxx20.onx20.blackstone
I got the toolchain from the steps I (re)posted on this thread:
http://forum.xda-developers.com/showpost.php?p=2269384&postcount=184 so that gave me a 1.2mb zImage where everyone's been posting 1.4mb-- I know there's something not right. As far as building a system.img by hand? I've taken existing ones either from posted bundles or from Android src directly. Maybe I suck (real possibility) but cupcake and 1.0 have been pretty flaky for me (there are system.img's included in the source). You should know how to mount and edit them though (look at the first post on this thread). My experience is the git source is useless unless you've got a G1-- I don't know how to make it run on Tilts. If it does work, then the answer to your question about host arch compiling is no-- it has to be ARMv5 for our phones. That's where this line comes in:
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
Seriously, I need help learning how to modify the kernel-- I've been a "google will have the answer for me" Linux bystander for a while and while I'm good at what I've done I'm not so good at this uncharted territory. I'm really looking for help to figure out how to compile modules (saurik and dzo never got back to me about that) and all I've gotten is "wait while I do it myself" which is cool they're working on it but we're obviously here to work on it too.
I've even been trying to get Debian installed (familiar territory for me) so I can at least get stuff working. You do know that Android is nothing but a Java VM layer for Linux and it will (could be) put on any self-respecting distro. Personally, I want Debian on my phone with an Android chroot as that would trump all.
Sorry to rant but you seem more about figuring this out like me and I don't know who else is really working on this besides the dev-gods who have no tutorials.

enatefox said:
I thought you were one of the experts, lol. I've been left with no support on how dzo, et all are customizing kernels so I've been in read only mode on the 'other thread' looking elsewhere for support.
Just wanted to ask you first, what's with the Dream radio? I know you posted the mods censored it but what's with the sig now? It piqued my interest... as I'm using (shudder) winmo on the regular while Android is being worked on I was hoping it would be worth looking into if it doesn't brick my phone.
Click to expand...
Click to collapse
It never actually worked... maybe. My phone was reporting the wrong version on a *different* radio (1.65.21.18, was saying 19) before, and trying to flash this changed the version to be correct. 0x300 radios will never flash, and this as a 0x301 *will* brick your phone. Then again... like 2 people reported epic success... In other words, no, it never really existed.
Answers to your questions:
As I said, my own zImage is no go. Check this link (not for our phone but the links at the bottom are pretty useful): http://wiki.xda-developers.com/index.php?pagename=BlackstoneLinux#Runningx20.Linuxx20.onx20.blackstone
I got the toolchain from the steps I (re)posted on this thread:
http://forum.xda-developers.com/showpost.php?p=2269384&postcount=184 so that gave me a 1.2mb zImage where everyone's been posting 1.4mb-- I know there's something not right. As far as building a system.img by hand? I've taken existing ones either from posted bundles or from Android src directly. Maybe I suck (real possibility) but cupcake and 1.0 have been pretty flaky for me (there are system.img's included in the source). You should know how to mount and edit them though (look at the first post on this thread). My experience is the git source is useless unless you've got a G1-- I don't know how to make it run on Tilts. If it does work, then the answer to your question about host arch compiling is no-- it has to be ARMv5 for our phones. That's where this line comes in:
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
unnecessary - zImage is implied, and the other two are in the Makefile already.
Seriously, I need help learning how to modify the kernel-- I've been a "google will have the answer for me" Linux bystander for a while and while I'm good at what I've done I'm not so good at this uncharted territory. I'm really looking for help to figure out how to compile modules (saurik and dzo never got back to me about that) and all I've gotten is "wait while I do it myself" which is cool they're working on it but we're obviously here to work on it too.
Click to expand...
Click to collapse
we have 0 modules at the moment (but perhaps support - I forget). Just compile stuff in - modules are very bad.
I've even been trying to get Debian installed (familiar territory for me) so I can at least get stuff working. You do know that Android is nothing but a Java VM layer for Linux and it will (could be) put on any self-respecting distro. Personally, I want Debian on my phone with an Android chroot as that would trump all.
Click to expand...
Click to collapse
It's a nice idea, but remember where a lot of the current coding is taking place - the ril, which is part of android (the system.img, at least), and doesn't go across to other distros quite as well. I think running Dalvik alongside whatever you're running in Debian might be too much for this phone's epically slow processor (compared to msm7201a).
Sorry to rant but you seem more about figuring this out like me and I don't know who else is really working on this besides the dev-gods who have no tutorials.
Click to expand...
Click to collapse
my instructions should work - they are what I use, at least.

Someone should change the instructions to (in the Wiki they are correct):
make ARCH=arm vogue_defconfig
I compiled my kernel, booted in Ubuntu, but had no touchscreen at all, I am not sure if the vogue_defconfig file takes care of everything necessary, so now I am checking everything via menuconfig to see whether eveything is all right or not.
DOMy

Do not Use Ext3 on SD cards
enatefox said:
seidler2547: Post:
Click to expand...
Click to collapse
You should NOT be using ext3 on any sort of flash ram device. You will wear out the medium. Journaling is just a bad idea in this sort of situation.
http://www.handhelds.org/hypermail/familiar/273/27320.html
3) ext3 is "very bad" because of the way it does journaling. It does journal, which seems like a good idea, but it also automatically periodically writes a lot of things out to the same secors on disk. I don't have first hand experience with this, but I remember somebody familiar with ext3 writing about this. It's method of journaling is not particularly intended for any kind of wear leveling at all.
Click to expand...
Click to collapse
http://www.mail-archive.com/[email protected]/msg38988.html
There are three disadvantages with the journaled file system:
- lower performance at write time, since there is the extra work of the
journal
- increased chance of damaging the SD card due to extra use of the
journal causing wearing
- increased space usage (for the journal)
Click to expand...
Click to collapse
And this is the general consensus within most of linux on ext3 and wearing mediums. I'm not sure if Andriod's kernel can do ext4, but ext4 can run without a journal.
Yet another reason not to use ext3 is that is near impossible to undelete something, something you can do with ext2.
-edit-
It looks like Android can use Ext4
http://thatsbadass.com/android/tag/ext4/

haha! is a good job, i like it

can't run android on my kaiser
hi folks,
i have read many threads here and on androidonhtc.com, i have read also the install instructions, but it seems im too stupid to run it.
it fails on "can't find system.img". i wil not flash it, i will run it from sd-card.
so, please can anyone attached an actually zip file which i just unpack und run haret.exe to work android on my kaiser ?
thanks and best regards
lenzen

Xoom with ICS 4.0.3 AOSP how to

Now that the code is out, let's start hacking the Xoom.
These instructions are for Linux 64-bit Ubuntu 11.10 / Mint 12, with latest updates.
follow directions on source.android.com to set up your environment
then move to next page to set up the repo and download the code
make sure you use Sun/Oracle Java Development Kit 1.6.x. It WILL NOT work with OpenJDK, which comes with Ubuntu 11.10/Mint 12. (best is remove OpenJDK to avoid hard to track errors)
downloading the code takes ages (a good 2h even with a good connection) and requires at least 2GB of hard disk space. But for compilation, I'd recommend to have 20GB at least.
download proprietary drivers at: http://tinyurl.com/6rfu8rn
extract the drivers from Broadcom and NVidia, this will gives you 2 .sh files
just run these 2 .sh files and copy the created vendor/ directory into the root of your Google code repo.
There are 2 kinds of Xoom
Xoom Verizon LTE, codename stingray
Xoom wifi, codename wingray
Make some corrections in the code for GCC 4.6
external/mesa3d/src/glsl/linker.cpp: l.70 add: #include <cstddef>
To build the code:
. ./build/envsetup.sh
lunch full_stingray-userdebug or full_wingray-userdebug
make -j5
This can take 2h to compile on an average machine like my MacBook Pro with 8GB RAM (very important to have more than 4GB!!!), or even more. And having a quad-core machine (or even more cores), really saves you tons of time!
Once compilation is done, it's time to flash:
go to out/target/product/wingray (or stingray)
put your Xoom in fastboot mode: power off, then volume down + power on
fastboot erase cache
fastboot erase userdata
fastboot flash boot boot.img
fastboot flash recovery recovery.img
fastboot flash system system.img
fastboot reboot
Note: you don't need to flash recovery.img, you should instead flash ClockworkMod recovery (http://download.clockworkmod.com/recoveries/recovery-clockwork-4.0.0.4-stingray.img)
And voila, you have a Xoom with WiFi working and all goodies from AOSP. This means no Google apps like Market...
To install Google Apps:
download this zip file [http://www.mediafire.com/?1l4ravroiwt4ybw[/url]
unzip it and go inside the folder
boot your device normally
rm system/app/Nfc*
adb push system /system
adb reboot
There are some errors with exchange service but Market, GMail, Maps work correctly. From there, go update your apps from Market
Enjoy!
Update 1: Android 4.0.1 issues back in 4.0.3?
Some people noted some compilation errors I didn't get on 4.0.3 but on 4.0.1. Here are solutions for 4.0.1 (emulator) that may work for you:
build/core/combo/HOST_linux-x86.mk:61 HOST_GLOBAL_CFLAGS += -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0
build/core/combo/javac.mk:15 COMMON_JAVAC := javac -J-Xmx512M -target 1.5 -source 1.5 -Xmaxerrs 9999999
frameworks/base/libs/util/Android.mk: 63 LOCAL_CFLAGS += -DLIBUTILS_NATIVE=1 $(TOOL_CFLAGS) -fpermissive
libcore/luni/src/main/java/java/lang/Enum.java: 128 return ordinal - o.ordinal ();
external/gtest/include/gtest/internal/gtest-param-util.h:40 #include <cstddef> (after #include <vector>)

How much space just to tinker with ICS/gingerbread?
I have a 1.5 mbps DSL connection and 200gb would take me 8 day/24 hour of downloading and I cant do that.

Kippui said:
How much space just to tinker with ICS/gingerbread?
I have a 1.5 mbps DSL connection and 200gb would take me 8 day/24 hour of downloading and I cant do that.
Click to expand...
Click to collapse
I do my building in a VM and the virtual disk image that has ubuntu and AOSP has grown to 45 GB after a compile.
Also, question for OP... why don't we see people using "make -j otapackage" to create CWM compatible ROMs? Just make sure it doesn't have a recovery in it (or it will torch your existing recovery). I torched my recovery the first time I did it, but I think I fixed it (haven't reflashed to find out).

It builds just fine with 3 GB of RAM in my VM here. Takes around 1 hour for me, with 6 CPU threads assigned to the VM, and using -j6. I just wish I could make sure it doesn't waste time compiling the various demos and tests.
I'm also using ccache to speed up (re)building. Google has instructions on how to set it up (it's really easy). Copied from their page (http://source.android.com/source/initializing.html):
You can optionally tell the build to use the ccache compilation tool. Ccache acts as a compiler cache that can be used to speed-up rebuilds. This works very well if you do "make clean" often, or if you frequently switch between different build products.
Put the following in your .bashrc or equivalent.
export USE_CCACHE=1
By default the cache will be stored in ~/.ccache. If your home directory is on NFS or some other non-local filesystem, you will want to specify the directory in your .bashrc as well.
export CCACHE_DIR=<path-to-your-cache-directory>
The suggested cache size is 50-100GB. You will need to run the following command once you have downloaded the source code.
prebuilt/linux-x86/ccache/ccache -M 50G
This setting is stored in the CCACHE_DIR and is persistent.
Click to expand...
Click to collapse
I have it set to only 10 GB here, but if you intend to do a lot of rebuilding, 20-40 GB might be a better idea.

The full AOSP tree + out directory takes 24G on my machine using du -sh.
The OTA package is 90MB, including the recovery, which you probably want to remove.
Indeed, I forgot to mention using USE_CCACHE, it really speeds up your builds. Make sure your cache dir is the top of your AOSP tree, it's even better.

Hi,
Huge thanks for those usefull instructions.
Just a hint for Gentoo builders (and maybe others distrib), if you're having this error after the "lunch" command :
Code:
AOSP> lunch full_wingray-userdebug
build/core/config.mk:268: *** Error: could not find jdk tools.jar, please install JDK6, which you can download from java.sun.com. Stop.
** Don't have a product spec for: 'full_wingray'
** Do you have the right repo manifest?
just export env ANDROID_JAVA_HOME with full jdk path :
Code:
export ANDROID_JAVA_HOME=/opt/sun-jdk-1.6.0.29/

winglord said:
There are 2 kinds of Xoom
Xoom Verizon LTE, codename stingray
Xoom wifi, codename wingray
Click to expand...
Click to collapse
There's three kinds of Xoom, the GSM MZ601 as well (Moto's codename everest). This is not supported by Google, but hopefully the hardware is close enough to stingray that the port shouldn't be overly difficult. At worst, the port to everest might have issues with vendor_ril?

please post your finished from.
Sent from my Full Android on Wingray using Tapatalk

Does someone know if there is a way to run builded images into the sdk emulator ?

Not sure what I am doing wrong:
Code:
/ICS_SOURCE# make -j4 otapackage
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=4.0.3
TARGET_PRODUCT=full_wingray
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=IML74K
============================================
build/core/main.mk:324: implicitly installing apns-conf_sdk.xml
No private recovery resources for TARGET_DEVICE wingray
host C++: llvm-rs-cc <= frameworks/compile/slang/llvm-rs-cc.cpp
host C++: llvm-rs-cc <= frameworks/compile/slang/slang_rs.cpp
host C++: llvm-rs-cc <= frameworks/compile/slang/slang_rs_ast_replace.cpp
host C++: llvm-rs-cc <= frameworks/compile/slang/slang_rs_context.cpp
<command-line>:0:0: error: "_FORTIFY_SOURCE" redefined [-Werror]<command-line>:0:0: error: "_FORTIFY_SOURCE" redefined [-Werror]
<built-in>:0:0: note: this is the location of the previous definition
<built-in>:0:0: note: this is the location of the previous definition
<command-line>:0:0: error: "_FORTIFY_SOURCE" redefined [-Werror]
<built-in>:0:0: note: this is the location of the previous definition
<command-line>:0:0: error: "_FORTIFY_SOURCE" redefined [-Werror]
<built-in>:0:0: note: this is the location of the previous definition
cc1plus: all warnings being treated as errors
make: *** [out/host/linux-x86/obj/EXECUTABLES/llvm-rs-cc_intermediates/llvm-rs-cc.o] Error 1
make: *** Waiting for unfinished jobs....
cc1plus: all warnings being treated as errors
make: *** [out/host/linux-x86/obj/EXECUTABLES/llvm-rs-cc_intermediates/slang_rs_ast_replace.o] Error 1
cc1plus: all warnings being treated as errors
make: *** [out/host/linux-x86/obj/EXECUTABLES/llvm-rs-cc_intermediates/slang_rs_context.o] Error 1
cc1plus: all warnings being treated as errors
make: *** [out/host/linux-x86/obj/EXECUTABLES/llvm-rs-cc_intermediates/slang_rs.o] Error 1
[email protected]:~/ICS_SOURCE#
Im using this guide as it has more details: http://www.freeyourandroid.com/guide/compile-ics
Any guidance someone could offer would be great.

Are you in ubuntu 11.10? I was having issues also. Check out this site...
http://www.android-dev.ro/2011/12/13/building-android-4-0-on-ubuntu-11-10/

aceman118 said:
Are you in ubuntu 11.10? I was having issues also. Check out this site...
http://www.android-dev.ro/2011/12/13/building-android-4-0-on-ubuntu-11-10/
Click to expand...
Click to collapse
Thanks, I am useing 11.10 so ill read through this and try again.
EDIT: got my first build completed! Thanks

you can use wingray or stingray on everest. Of course, if you use wingray on everest, you won't have phone app and other phone only features.

aceman118 said:
Are you in ubuntu 11.10? I was having issues also. Check out this site...
http://www.android-dev.ro/2011/12/13/building-android-4-0-on-ubuntu-11-10/
Click to expand...
Click to collapse
I'm using Ubuntu 11.10 64bit and GCC 4.6.
Keep in mind to add '#include <cstddef>' in headers where you have compile errors like indexOf() not found.

popokrew said:
Not sure what I am doing wrong:
Code:
/ICS_SOURCE# make -j4 otapackage
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=4.0.3
TARGET_PRODUCT=full_wingray
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=IML74K
============================================
build/core/main.mk:324: implicitly installing apns-conf_sdk.xml
No private recovery resources for TARGET_DEVICE wingray
host C++: llvm-rs-cc <= frameworks/compile/slang/llvm-rs-cc.cpp
host C++: llvm-rs-cc <= frameworks/compile/slang/slang_rs.cpp
host C++: llvm-rs-cc <= frameworks/compile/slang/slang_rs_ast_replace.cpp
host C++: llvm-rs-cc <= frameworks/compile/slang/slang_rs_context.cpp
<command-line>:0:0: error: "_FORTIFY_SOURCE" redefined [-Werror]<command-line>:0:0: error: "_FORTIFY_SOURCE" redefined [-Werror]
<built-in>:0:0: note: this is the location of the previous definition
<built-in>:0:0: note: this is the location of the previous definition
<command-line>:0:0: error: "_FORTIFY_SOURCE" redefined [-Werror]
<built-in>:0:0: note: this is the location of the previous definition
<command-line>:0:0: error: "_FORTIFY_SOURCE" redefined [-Werror]
<built-in>:0:0: note: this is the location of the previous definition
cc1plus: all warnings being treated as errors
make: *** [out/host/linux-x86/obj/EXECUTABLES/llvm-rs-cc_intermediates/llvm-rs-cc.o] Error 1
make: *** Waiting for unfinished jobs....
cc1plus: all warnings being treated as errors
make: *** [out/host/linux-x86/obj/EXECUTABLES/llvm-rs-cc_intermediates/slang_rs_ast_replace.o] Error 1
cc1plus: all warnings being treated as errors
make: *** [out/host/linux-x86/obj/EXECUTABLES/llvm-rs-cc_intermediates/slang_rs_context.o] Error 1
cc1plus: all warnings being treated as errors
make: *** [out/host/linux-x86/obj/EXECUTABLES/llvm-rs-cc_intermediates/slang_rs.o] Error 1
[email protected]:~/ICS_SOURCE#
Im using this guide as it has more details: http://www.freeyourandroid.com/guide/compile-ics
Any guidance someone could offer would be great.
Click to expand...
Click to collapse
You are not doing anything wrong, this is a GCC 4.6 restriction.
Weird I didn't see it with ics 4.0.3 but with 4.0.1 before.
Here is what you need to do for 4.0.1:
build/core/combo/HOST_linux-x86.mk:61 HOST_GLOBAL_CFLAGS += -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0
build/core/combo/javac.mk:15 COMMON_JAVAC := javac -J-Xmx512M -target 1.5 -source 1.5 -Xmaxerrs 9999999
frameworks/base/libs/util/Android.mk: 63 LOCAL_CFLAGS += -DLIBUTILS_NATIVE=1 $(TOOL_CFLAGS) -fpermissive
libcore/luni/src/main/java/java/lang/Enum.java: 128 return ordinal - o.ordinal ();
external/gtest/include/gtest/internal/gtest-param-util.h:40 #include <cstddef> (after #include <vector>)

winglord said:
You are not doing anything wrong, this is a GCC 4.6 restriction.
Weird I didn't see it with ics 4.0.3 but with 4.0.1 before.
Here is what you need to do for 4.0.1:
build/core/combo/HOST_linux-x86.mk:61 HOST_GLOBAL_CFLAGS += -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0
build/core/combo/javac.mk:15 COMMON_JAVAC := javac -J-Xmx512M -target 1.5 -source 1.5 -Xmaxerrs 9999999
frameworks/base/libs/util/Android.mk: 63 LOCAL_CFLAGS += -DLIBUTILS_NATIVE=1 $(TOOL_CFLAGS) -fpermissive
libcore/luni/src/main/java/java/lang/Enum.java: 128 return ordinal - o.ordinal ();
external/gtest/include/gtest/internal/gtest-param-util.h:40 #include <cstddef> (after #include <vector>)
Click to expand...
Click to collapse
winglord, I was able to get it to build with the guide aceman posted. I believe its because of the modified make: make CC=gcc-4.4 CXX=g++-4.4
Where would I find a log for the entire build? A lot of stuff scrolls by while building.
Now I just need to get the recovery image out (guessing just delete it out of the ota .zip?) and try it out. My Xoom isnt with me right now.

AlEsC said:
Does someone know if there is a way to run builded images into the sdk emulator ?
Click to expand...
Click to collapse
When running lunch you have to chose one of the emulator builds (full_eng if my memory's right). You can't run a build compiled for a specific device.

RMerlin said:
When running lunch you have to chose one of the emulator builds (full_eng if my memory's right). You can't run a build compiled for a specific device.
Click to expand...
Click to collapse
Thanks
This means I'll have to rebuild all !!
Another question, I've made a successfully build, then I've made some modification in the sources, so I've deleted all *.img thinking that "make" will rebuild them... but "make" only rebuild "system.img", "ramdisk.img" and "userdata.img".
Is there a "make boot" or something like that ? (real question is how do I rebuild my boot.img )
Thanks

winglord said:
I'm using Ubuntu 11.10 64bit and GCC 4.6.
Keep in mind to add '#include <cstddef>' in headers where you have compile errors like indexOf() not found.
Click to expand...
Click to collapse
I tried that and it didn't seem to work for me. Once I used GCC 4.4 it worked. I couldn't tell you why, though.
So I finally got a build up and running - is there something special that has to be done with the vendor binaries? I know I have seen this in other builds too but the kernel is still showing 2.x and the camera, mic and gps are not working. Is that because work needs to be done to the kernel before it is 100% with ICS? This is my first time messing with an android build so forgive me if this is a dumb question.

AlEsC said:
Thanks
This means I'll have to rebuild all !!
Another question, I've made a successfully build, then I've made some modification in the sources, so I've deleted all *.img thinking that "make" will rebuild them... but "make" only rebuild "system.img", "ramdisk.img" and "userdata.img".
Is there a "make boot" or something like that ? (real question is how do I rebuild my boot.img )
Thanks
Click to expand...
Click to collapse
Make is usually smart enough to know when something you changed will not result in a different output. It is possible that the changes you made would result in an identical boot.img, therefore it is not rebuilding it.

Requesting help with "make-standalone-toolchain.sh" for AFTV2

Yesterday, I cross-compiled DropBear for the 2nd Gen Fire TV, but I am receiving a linkage error that points to it being compiled with an incorrect toolchain.
I downloaded and extracted the "android-ndk-r10e-linux-x86_64.bin" and used the following command to make the standalone toolchain:
Code:
/home/william/android-ndk/build/tools/make-standalone-toolchain.sh --ndk-dir=/home/william/android-ndk --platform=android-21 --toolchain=arm-linux-androideabi-4.9 --system=linux-x86_64 --install-dir=/home/william/aftv2-toolchain
What are the correct platform and toolchain for the AFTV2 on 5.0.4? I tried to look this up, but I was unable to find it posted in any of the expected places. If I don't hear back, I suppose I'll give it a try with aarch64-linux-android-4.9, because the error I'm receiving when I try to connect to server complains about the bitness of "libc.so" (is 32-bit instead of 64-bit), but I can't even seem to find confirmation that the AFTV2 uses a 64-bit arm processor.
Thanks,
William

fecaleagle said:
Yesterday, I cross-compiled DropBear for the 2nd Gen Fire TV, but I am receiving a linkage error that points to it being compiled with an incorrect toolchain.
I downloaded and extracted the "android-ndk-r10e-linux-x86_64.bin" and used the following command to make the standalone toolchain:
Code:
/home/william/android-ndk/build/tools/make-standalone-toolchain.sh --ndk-dir=/home/william/android-ndk --platform=android-21 --toolchain=arm-linux-androideabi-4.9 --system=linux-x86_64 --install-dir=/home/william/aftv2-toolchain
What are the correct platform and toolchain for the AFTV2 on 5.0.4? I tried to look this up, but I was unable to find it posted in any of the expected places. If I don't hear back, I suppose I'll give it a try with aarch64-linux-android-4.9, because the error I'm receiving when I try to connect to server complains about the bitness of "libc.so" (is 32-bit instead of 64-bit), but I can't even seem to find confirmation that the AFTV2 uses a 64-bit arm processor.
Thanks,
William
Click to expand...
Click to collapse
Yes, you need aarch64. I've used both the NDK and the aarch64 compiler straight from AOSP. Both work.

rbox said:
Yes, you need aarch64. I've used both the NDK and the aarch64 compiler straight from AOSP. Both work.
Click to expand...
Click to collapse
Thanks, I actually just rebuilt the toolchain with aarch64 and re-compiled. I'll follow-up, because I'm assuming that was the only thing preventing the server from functioning properly. Thanks for all your assistance.

fecaleagle said:
Thanks, I actually just rebuilt the toolchain with aarch64 and re-compiled. I'll follow-up, because I'm assuming that was the only thing preventing the server from functioning properly. Thanks for all your assistance.
Click to expand...
Click to collapse
Well, I'm obviously still doing something wrong. The file command on my build-system reports:
Code:
./dropbearmulti: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, stripped
, which looks correct.
But clients are still receiving the following error when I try to connect to the server:
Code:
CANNOT LINK EXECUTABLE DEPENDENCIES: "libc.so" is 32-bit instead of 64-bit
I'll keep digging, but basically, I am building the toolchain like so:
Code:
/home/william/android-ndk/build/tools/make-standalone-toolchain.sh --ndk-dir=/home/william/android-ndk --platform=android-21 --toolchain=aarch64-linux-android-4.9 --system=linux-x86_64 --install-dir=/home/william/aftv2-toolchain
Then running configure:
Code:
./configure --build=x86_64-unknown-linux-gnu --host=aarch64-linux-android --prefix=/home/william/aftv2-toolchain --disable-zlib --disable-largefile --disable-loginfunc --disable-shadow --disable-utmp --disable-utmpx --disable-wtmp --disable-wtmpx --disable-pututline --disable-pututxline --disable-lastlog
Then, I am setting CC and PATH to the following:
Code:
CC=aarch64-none-linux-gnueabi-gcc
PATH=/home/william/aftv2-toolchain/bin:$PATH
Then, I am building as such:
Code:
STATIC=1 MULTI=1 SCPPROGRESS=0 PROGRAMS="dropbear dropbearkey scp dbclient" make strip
No complaints during the build or from the file command, but the issue persists. Surely I am missing something.

Interesting. So when I poke around my toolchain, all of the copies of libc.so are 64-bit, according to the linux file command.
However, just to see if it would tell me anything, I compiled the binary dynamically-linked rather than statically linked and used the ndk-depends tool, and it reports the following:
Code:
Building dependency graph...
dropbearmulti depends on: libc.so libdl.so
Android system library: libc.so
Android system library: libdl.so
Building sorted list of binaries:
dropbearmulti -> ./dropbearmulti
libdl.so -> $ /system/lib/libdl.so
libc.so -> $ /system/lib/libc.so
This obviously indicates that when I build dynamic, it is attempting to link with the 32-bit libraries and not the 64-bit libraries. How can I correct this so that when I do any build, the linkages are correct?

fecaleagle said:
Interesting. So when I poke around my toolchain, all of the copies of libc.so are 64-bit, according to the linux file command.
However, just to see if it would tell me anything, I compiled the binary dynamically-linked rather than statically linked and used the ndk-depends tool, and it reports the following:
Code:
Building dependency graph...
dropbearmulti depends on: libc.so libdl.so
Android system library: libc.so
Android system library: libdl.so
Building sorted list of binaries:
dropbearmulti -> ./dropbearmulti
libdl.so -> $ /system/lib/libdl.so
libc.so -> $ /system/lib/libc.so
This obviously indicates that when I build dynamic, it is attempting to link with the 32-bit libraries and not the 64-bit libraries. How can I correct this so that when I do any build, the linkages are correct?
Click to expand...
Click to collapse
I was compiling kexec dynamically by setting CC and it was working. And I've compiled other things with Android.mk files using the NDK and compiling them static and it worked. Why are you compiling dropbear static to begin with? You say CLIENTS are receiving that message. It almost sounds like dropbear is trying to fork a 32bit shell for them. Since you are running dropbear as the server fine, itself is compiled correctly...

rbox said:
I was compiling kexec dynamically by setting CC and it was working. And I've compiled other things with Android.mk files using the NDK and compiling them static and it worked. Why are you compiling dropbear static to begin with? You say CLIENTS are receiving that message. It almost sounds like dropbear is trying to fork a 32bit shell for them. Since you are running dropbear as the server fine, itself is compiled correctly...
Click to expand...
Click to collapse
I'm compiling static because I'm following the guide posted here:
http://forum.xda-developers.com/nexus-7-2013/general/guide-compiling-dropbear-2015-67-t3142412
Running the dynamically-linked version on the fire tv reports:
Code:
error: only position independent executables (PIE) are supported.
I suppose I should be reaching out to @jocala at this point, since I'd guess he's managed to compile it successfully for the 2nd gen fire tv.

fecaleagle said:
I'm compiling static because I'm following the guide posted here:
http://forum.xda-developers.com/nexus-7-2013/general/guide-compiling-dropbear-2015-67-t3142412
Running the dynamically-linked version on the fire tv reports:
Code:
error: only position independent executables (PIE) are supported.
Click to expand...
Click to collapse
http://stackoverflow.com/questions/...id-l-error-only-position-independent-executab
You need to enable PIE in CFLAGS and LDFLAGS:
CFLAGS = -fPIE
LDFLAGS = -fPIE -pie

fecaleagle said:
I suppose I should be reaching out to @jocala at this point, since I'd guess he's managed to compile it successfully for the 2nd gen fire tv.
Click to expand...
Click to collapse
I'm working on the adbFire update for AFTV2 root; ssh is on the list, but I'm not there yet.

rbox said:
http://stackoverflow.com/questions/...id-l-error-only-position-independent-executab
You need to enable PIE in CFLAGS and LDFLAGS:
CFLAGS = -fPIE
LDFLAGS = -fPIE -pie
Click to expand...
Click to collapse
Doing this allows it startup correctly dynamically linked, but clients still fail to connect. Debian reports the most useful information:
Code:
dispatch_protocol_error: type 51 seq 6
CANNOT LINK EXECUTABLE DEPENDENCIES: "libc.so" is 32-bit instead of 64-bit
As usual, dropbear reports:
Code:
[6329] Jan 04 13:48:35 Child connection from 192.168.1.210:53425
void endusershell()(3) is not implemented on Android
void endusershell()(3) is not implemented on Android
[6329] Jan 04 13:48:42 password auth succeeded for 'root' from 192.168.1.210:53425
[6329] Jan 04 13:48:43 Exit (root): Disconnect received
To me, it appears that the server is throwing up the error to the client, but I still suspected that I am linking the 32-bit version of libc.so when I build dropbear, and I am still at a loss for how to correct that.
Thanks for all of your help. I'll get there eventually.

fecaleagle said:
Doing this allows it startup correctly dynamically linked, but clients still fail to connect. Debian reports the most useful information:
Code:
dispatch_protocol_error: type 51 seq 6
CANNOT LINK EXECUTABLE DEPENDENCIES: "libc.so" is 32-bit instead of 64-bit
As usual, dropbear reports:
Code:
[6329] Jan 04 13:48:35 Child connection from 192.168.1.210:53425
void endusershell()(3) is not implemented on Android
void endusershell()(3) is not implemented on Android
[6329] Jan 04 13:48:42 password auth succeeded for 'root' from 192.168.1.210:53425
[6329] Jan 04 13:48:43 Exit (root): Disconnect received
To me, it appears that the server is throwing up the error to the client, but I still suspected that I am linking the 32-bit version of libc.so when I build dropbear, and I am still at a loss for how to correct that.
Thanks for all of your help. I'll get there eventually.
Click to expand...
Click to collapse
Dropbear itself is 64-bit, because you are actually running it fine. If you could turn up the logging on dropbear, maybe you can get it to tell you what it's doing between password succeeded and disconnect received. Or you could just add a ton of logging to the code. My guess is it's forked the connection and trying to start a secondary program, and that program isn't 64bit. My guess would be the shell, but it should just be using the system shell, so unsure.

jocala said:
I'm working on the adbFire update for AFTV2 root; ssh is on the list, but I'm not there yet.
Click to expand...
Click to collapse
Thanks for your work on adbFire. I'll report back if I'm ever able to get this resolved.

rbox said:
Dropbear itself is 64-bit, because you are actually running it fine. If you could turn up the logging on dropbear, maybe you can get it to tell you what it's doing between password succeeded and disconnect received. Or you could just add a ton of logging to the code. My guess is it's forked the connection and trying to start a secondary program, and that program isn't 64bit. My guess would be the shell, but it should just be using the system shell, so unsure.
Click to expand...
Click to collapse
This suggestion is making more and more sense. I'll take a look at the source and start thinking about my own patchset for the latest source version. I'm annoyed that the verbose flag is not included in the source version of dropbear that the patch was created for, so I think I will end up going that route eventually. I'll take your advice and focus adding debug messages when dropbear initializes the shell, since the authentication goes of without a hitch and it's at that point that the process breaks down. I really appreciate all of your suggestions!

@rbox,
This is rather helpful, and the post from July 31st, 2015 all but confirms your suspicion about forking the shell and points me in the right direction:
http://www.kevinboone.net/kbox3_diary.html
I will start by modifying the patches to use: /bin/sh rather than /system/bin/sh when launching the shell and see if that resolves my issue, then get started on the long slog.

The blog post above seems to have shed some light on the issue. Updating the patches to refer to /bin/sh rather than /system/bin/sh seems to have resolved the "CANNOT LINK EXECUTABLE DEPENDENCIES: "libc.so" is 32-bit instead of 64-bit" error. As @rbox suspected, dropbear seems to be creating the appropriate shell upon connect now on a 64-bit system (aftv2).
Unfortunately, I'm still receiving the "dispatch_protocol_error":
Code:
dispatch_protocol_error: type 51 seq 6
[5761] Jan 04 19:47:46 Exit (root): Child failed
Connection to 192.168.1.213 closed.
Any thoughts on this one @jocala? Googling dispatch_protocol_error type 51 returns zilch.

Linageos 17.1 Build faild Python Error - Use prebuilts python for ninja?

Hello,
my Linux OS have a strange Python Version or setting, so the Build failed with:
Code:
FAILED: //art/build/apex:art-check-release-apex-gen generate art-check-release-apex-gen.dummy
Outputs: out/soong/.intermediates/art/build/apex/art-check-release-apex-gen/gen/art-check-release-apex-gen.dummy
Error: exited with code: 1
Command: rm -rf out/soong/.intermediates/art/build/apex/art-check-release-apex-gen/gen && out/soong/host/linux-x86/bin/sbox --sandbox-path out/soong/.temp --output-root out/soong/.intermediates/art/build/apex/art-check-release-apex-gen/gen -c 'out/soong/host/linux-x86/bin/art-apex-tester --debugfs out/soong/host/linux-x86/bin/debugfs --tmpdir __SBOX_OUT_DIR__ out/soong/.intermediates/art/build/apex/com.android.runtime.release/android_common_com.android.runtime.release/com.android.runtime.release.apex && touch __SBOX_OUT_FILES__' __SBOX_OUT_DIR__/art-check-release-apex-gen.dummy
Output:
.path_interposer: no python-exec wrapped executable found in /usr/lib/python-exec.
sbox command (out/soong/host/linux-x86/bin/art-apex-tester --debugfs out/soong/host/linux-x86/bin/debugfs --tmpdir out/soong/.temp/sbox333717900 out/soong/.intermediates/art/build/apex/com.android.runtime.release/android_common_com.android.runtime.release/com.android.runtime.release.apex && touch out/soong/.temp/sbox333717900/art-check-release-apex-gen.dummy) failed with err "exit status 127"
Its on the Finish-Line of the building Process. ( [ 99% 11533/11603] )
I try to use
Code:
virtualenv --version
virtualenv 20.0.18 from /usr/lib64/python3.6/site-packages/virtualenv/__init__.py
With the Python in:
prebuilts/python/linux-x86/2.7.5/bin/python2.7
Some hints for me?
Building Lineage 16, works fine, sure the build process is different.
Can i just use the Command from lineageos16, which create the zip file if all parts are finnished?

I've been having the same issue on Gentoo since LineageOS 17.1.
I have Python 2.7, 3.6, 3.7 and 3.8 installed.
If nobody has a solution and you find it, please comment with the solution, I'm sure we are not the only two people affected by this issue.

Thank you Yurienu,
easy would be to juse another live-Distro with a build Tree on a mounted file System.
However i think the error is in the tools of google, like apex or how the script use and call this function. Not sure if i like to debug this.
I think it because the path.interposer try to present a list of tools to art-check-release-apex-gen, from where the program to do its job. And a wrapper might failed and put its path there. When art-check do its job, try it and failed, because its not design to handle the error message (and so because of that bug).
For this speaks the error message:
But take a look on my error code: "Error: exited with code: 1" from the command: //art/build/apex:art-check-release-apex-gen generate art-check-release-apex-gen.dummy".
I think there is the bug. Whatever art-check-release-apex-gen do, it fail here. Then check, see the wrapper and say that there is no python.
I try to use python 2.7.18 (Gentoos Python 2.7), and it did not work.
I try to use virtuelenvwrapper*** and it failed but the error was like: "no Module with python 2.7.5, named virtuelenvwrapper". However i got this misconfigured. But it sound compelling. Because a Module for python 2.7.18, is maybe not byte compatible with python 2.7.5.
Think i try to use another distro for building this.
Bye the way, there is a Google Summer of code 2020 project, that's sound very interesting. So there is hope for a proper lineage and gentoo support in the future.
However, if i find a solution i share it here!
I stay a little bit longer with lineageos 16.0, because self build security updates are more important for me, as Android 10 is.
*** I just linked the prebuild binary an path as environment to /usr/local/bin/

Hello everyone,
I too met the same issue, the exact same content in the "out/error.log" than the first post; I am sharing the research I got around this issue, because I don't have the same conclusion.
First I tried to discard the error about ".path_interposer", and seek the error from the "sbox" software output.
I dig around "out/soong/host/linux-x86/bin/art-apex-tester", and got those information:
Code:
% file out/soong/host/linux-x86/bin/art-apex-tester
out/soong/host/linux-x86/bin/art-apex-tester: Zip archive data, made by v?[0x314], extract using at least v2.0, last modified Thu Nov
4 03:39:44 1999, uncompressed size 2619, method=store
% unzip -l out/soong/host/linux-x86/bin/art-apex-tester
Archive: out/soong/host/linux-x86/bin/art-apex-tester
Length Date Time Name
--------- ---------- ----- ----
2619 01-01-2008 00:00 __main__.py
28376 01-01-2008 00:00 art_apex_test.py
--------- -------
30995 2 files
It looks like it’s a Zip archive which can execute the Python script "__main__.py".
The "__main__.py" is a wrapper, which its goal is to call the other python file, "art_apex_test.py" with the correct parameters.
By reading its content, I noticed it searchs Python 3 executable, and calls the following command:
Code:
/usr/bin/python3 -S relative_path/to/art_apex_test.py […]
When I execute this command under a temporary directory, I have this output (still under the current directory as parent of the "out" directory):
Code:
python3 -S /tmp/art_apex_test.py --debugfs out/soong/host/linux-x86/bin/debugfs --tmpdir out/soong/.temp out/soong/.intermediates/art/build/apex/com.android.runtime.release/android_common_com.android.runtime.release/com.android.runtime.release.apex
--bitness=auto, trying to autodetect. This may be incorrect!
Detected multilib
I have the same ouput by either executing the Zip archive with the same arguments.
Unfortunately, the "tmpdir" "out/soong/.temp" remains empty.
What about "path_interposer" then? No idea, I found no match in either the two Python scripts.
And then I don't think it's related to the Python 2.7 embedded program.
Oddly enough, the full command with sbox has the same output, but not error. I start to think this error is in a build system file.
Hope to have help (a little).
Best regards,

thican said:
I too met the same issue, the exact same content in the "out/error.log" than the first post; I am sharing the research I got around this issue, because I don't have the same conclusion.
Click to expand...
Click to collapse
I've been having this issue a while also, turns out it's fixable (not that Gentoo won't override it every time you upgrade python) I linked /usr/bin/python3.7 to /usr/bin/python3 instead of /usr/bin/python-exec2c.
Now the apex tester thinger works fine, I'm guessing that inside the sandbox can't find /usr/lib/python-exec.
Hope this works for you guys as well.

SOLVED LineageOS 17.1: .path_interposer: no python-exec wrapped executable found
superfes said:
I've been having this issue a while also, turns out it's fixable (not that Gentoo won't override it every time you upgrade python) I linked /usr/bin/python3.7 to /usr/bin/python3 instead of /usr/bin/python-exec2c.
Now the apex tester thinger works fine, I'm guessing that inside the sandbox can't find /usr/lib/python-exec.
Hope this works for you guys as well.
Click to expand...
Click to collapse
OH! That's the trick, it finally works!
For an easy fix without needing to touch files hold by system, I just create a bin directory at the root of the home directory of the user, put its path inside the PATH environment variable, at its beginning (PATH="${HOME}/bin:${PATH}"), and then creating the symbolic links.
It was so easy it's so frustrating, having this issue for months, even during 16.0 branch IIRC.
Just to be sure, I create those links (I picked Python 3.8 for my example):
Code:
ln -s /usr/bin/python3.8 ~/bin/python3
ln -s python3 ~/bin/python
Now let's try this new build. Thank you very much for you help! :good:
PS: in your message, the links are backward, it's currently python3 which is linked to python-exec2c, and /usr/bin/python3.7{,m} are there own executable. I advice you to avoid editing your system files

@superfes and @thican you guys deserve three internets!
I was struggling with this error as well (not for 16.0, only for 17.1 -- Gentoo here as well). This fixed it!
So again: Thanks!

thican said:
OH! That's the trick, it finally works!
For an easy fix without needing to touch files hold by system, I just create a bin directory at the root of the home directory of the user, put its path inside the PATH environment variable, at its beginning (PATH="${HOME}/bin:${PATH}"), and then creating the symbolic links.
It was so easy it's so frustrating, having this issue for months, even during 16.0 branch IIRC.
Just to be sure, I create those links (I picked Python 3.8 for my example):
Code:
ln -s /usr/bin/python3.8 ~/bin/python3
ln -s python3 ~/bin/python
Click to expand...
Click to collapse
This is the way that worked fully for me. Thanks all! Running Gentoo latest hardened with python 3.7 as default.

Thank you superfes and thican!
I had not the time do investigate since covid19 blow up my personal workingdesk. Nice to go on with Lineage 17.1!

Tnx! Worked for me too!

With dev-lang/python-exec-2.4.6-r2
/usr/bin/python3 is a symlink to python-exec2c which cause this error.
Workaround is use vanilla python version of python-exec2c
before brunch command you can use like
Code:
[[ -d ~/bin/ ]] || mkdir ~/bin
[[ -e ~/bin/python3 ]] || ln -s /usr/lib/python-exec/python-exec2 ~/bin/python3
export PATH=~/bin:$PATH
Hence, in future shebang support of python or python3 will removed also.
762406 – [TRACKER] Packages that have failures with dev-lang/python-exec[-native-symlinks]
bugs.gentoo.org

[Tool] Samloader (SamFirm / Frija replacement)

Hello,
I recently wanted to download some firmware for my Samsung device, but I realized that there is no 100% open source program to do so. In fact, all the tools that claim to do so require a library that is packed by Themida (so it is difficult to check what this might be doing), in order to authenticate to the server. This is a native DLL, meaning that it is only compatible with Windows x86. Additionally, many of these tools are actually using stolen decompiled code from SamFirm, which, apart from being possibly illegal, means they would be difficult to maintain and run slowly.
So, I decided to reverse engineer Smart Switch to figure out exactly how the download is taking place, and wrote a cross-platform tool that does this without using the Windows DLL that the other tools have, making it compatible with Linux and MacOS. I also realized that the newer versions are actually using a new version of the authentication algorithm, meaning possibly at some point the old tools might stop working as Samsung drops support for it.
You can find it at:
Code:
https://github.com/nlscc/samloader
To install, go to the downloaded repository and run:
Code:
pip3 install .
See the README or look at the code for usage. You might want to know that my old github account, nm111, was unfortunately deleted, and I lost access to my old XDA account. You can see the verified email is the same however.
Feel free to use the algorithms I figured out in your own code, so long as you don't use it in proprietary programs. It is licensed under GNU GPLv3 or later.
This works for all phones, not just S10+, but I couldn't find a better forum and this is where Frija posted.

Thank you. This is pretty cool. I downloaded it now and will check it out later tonight.

Can you please do the same thing for the emergency recovery option.
Will you update your tool if it still working in the future?

This is going to be super useful for me (ATM I'm using SamFirm with wine/proton and my setup breaks often, and GUI gets in my way) since I need to download many firmwares for my reverse-engineering, so thanks!

Late for this great find, thanks to the xda article, really needed a solution outside Windows.
PS, what are the chances for spoofing a request, i.e, trying to fetch only the latest OTA from a certain CSC?

@nn000 Glad this made the front page. I have used this for a little wile now and it works great
@phhusson
This will work on windows if you use the WSL and install Ubuntu. You could probably get this working under Cygwin too.

Great work mate. Helps to gather various download. Unfortunately enough Samshung does not allow older firmware via server fetching..

Hi, thanks for this great tool as samfirm and frija are slow to download firmwares. I am trying to download firmware for galaxy watch. I can only search the firmware version but when trying to download it there is error:
filename = root.find("./FUSBody/Put/BINARY_NAME/Data").text
AttributeError: 'NoneType' object has no attribute 'text'
is there any way to modify this tool to download watch firmware too?

You are genius! Thanks for this tool. Can't wait to try this :good:

nn000 said:
Feel free to use the algorithms I figured out in your own code, so long as you don't use it in proprietary programs. It is licensed under GNU GPLv3 or later.
This works for all phones, not just S10+, but I couldn't find a better forum and this is where Frija posted.
Click to expand...
Click to collapse
Brilliant work!
Thank you very much for developing this tool and removing our dependency on Windows for such basic functionality as checking for and downloading firmware updates. Thank you also for having the foresight and generosity to publish this under the GPL.
This tool will receive a huge amount of use on my machines.

Thanks in advance, I don't know how to download the program, they would be so kind to give me a link. Thank you very much

I tried installing with the command found in the readme file
Code:
pip3 install git+https://github.com/nlscc/samloader.git
But when I type
Code:
$ samloader --help
It says "samloader: command not found"

Beautiful. Thanks for this tool!

4929york said:
I tried installing with the command found in the readme file
Code:
pip3 install git+https://github.com/nlscc/samloader.git
But when I type
Code:
$ samloader --help
It says "samloader: command not found"
Click to expand...
Click to collapse
install python first, before try

rikipy said:
install python first, before try
Click to expand...
Click to collapse
Python was installed.

How to set download location(out) in the command line
samloader download [firmware version] [phone model] [region] [out]
I tried to put \Download but it's not in the folder.

I installed python39 and git on Win10 but sill get errors
Code:
pip3 install git+https://github.com/nlscc/samloader.git
Collecting git+https://github.com/nlscc/samloader.git
Cloning https://github.com/nlscc/samloader.git to c:\users\danie\appdata\local\temp\pip-req-build-s8l3kwc6
Collecting clint
Using cached clint-0.5.1.tar.gz (29 kB)
Collecting pycryptodomex
Using cached pycryptodomex-3.9.8.tar.gz (15.6 MB)
ERROR: Command errored out with exit status 1:
command: 'c:\program files\python39\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\danie\\AppData\\Local\\Temp\\pip-install-vvk574_e\\pycryptodomex\\setup.py'"'"'; __file__='"'"'C:\\Users\\danie\\AppData\\Local\\Temp\\pip-install-vvk574_e\\pycryptodomex\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\danie\AppData\Local\Temp\pip-pip-egg-info-3i7fsfut'
cwd: C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\
Complete output (20 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\setup.py", line 457, in <module>
set_compiler_options(package_root, ext_modules)
File "C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\compiler_opt.py", line 341, in set_compiler_options
clang = compiler_is_clang()
File "C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\compiler_opt.py", line 251, in compiler_is_clang
return test_compilation(source, msg="clang")
File "C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\compiler_opt.py", line 82, in test_compilation
objects = compiler.compile([fname], extra_postargs=extra_cc_options)
File "c:\program files\python39\lib\distutils\_msvccompiler.py", line 323, in compile
self.initialize()
File "c:\program files\python39\lib\distutils\_msvccompiler.py", line 220, in initialize
vc_env = _get_vc_env(plat_spec)
File "c:\program files\python39\lib\site-packages\setuptools\msvc.py", line 314, in msvc14_get_vc_env
return _msvc14_get_vc_env(plat_spec)
File "c:\program files\python39\lib\site-packages\setuptools\msvc.py", line 268, in _msvc14_get_vc_env
raise distutils.errors.DistutilsPlatformError(
distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/
Testing support for clang
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Viper780 said:
I installed python39 and git on Win10 but sill get errors
Code:
distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/
Click to expand...
Click to collapse
The problem is right there in front of you: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/

It's pretty annoying when there are requirements and you find out about them afterwards.
Would be good to write them to the install instructions
- python3
- git
- Buildtools for MS C++

Thanks for this, it works well! Cheers..
:highfive:
Example:
Install
Code:
pip3 install git+https://github.com/nlscc/samloader.git
Check update
Code:
samloader -m SM-G975F -r NZC checkupdate
Download
Code:
samloader -m SM-G975F -r NZC download -v G975FXXS9DTI8/G975FOXM9DTI8/G975FXXS9DTI8/G975FXXS9DTI8 -O /home/hinxnz/Downloads
Decrypt
Code:
samloader -m SM-G975F -r NZC decrypt -v G975FXXS9DTI8/G975FOXM9DTI8/G975FXXS9DTI8/G975FXXS9DTI8 -V 4 -i SM-G975F_1_20200921075534_uii8oafhih_fac.zip.enc4 -o SM-G975F_1_20200921075534_uii8oafhih_fac.zip
---------- Post added at 10:52 PM ---------- Previous post was at 10:41 PM ----------
Viper780 said:
I installed python39 and git on Win10 but sill get errors
Code:
pip3 install git+https://github.com/nlscc/samloader.git
Collecting git+https://github.com/nlscc/samloader.git
Cloning https://github.com/nlscc/samloader.git to c:\users\danie\appdata\local\temp\pip-req-build-s8l3kwc6
Collecting clint
Using cached clint-0.5.1.tar.gz (29 kB)
Collecting pycryptodomex
Using cached pycryptodomex-3.9.8.tar.gz (15.6 MB)
ERROR: Command errored out with exit status 1:
command: 'c:\program files\python39\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\danie\\AppData\\Local\\Temp\\pip-install-vvk574_e\\pycryptodomex\\setup.py'"'"'; __file__='"'"'C:\\Users\\danie\\AppData\\Local\\Temp\\pip-install-vvk574_e\\pycryptodomex\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\danie\AppData\Local\Temp\pip-pip-egg-info-3i7fsfut'
cwd: C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\
Complete output (20 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\setup.py", line 457, in <module>
set_compiler_options(package_root, ext_modules)
File "C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\compiler_opt.py", line 341, in set_compiler_options
clang = compiler_is_clang()
File "C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\compiler_opt.py", line 251, in compiler_is_clang
return test_compilation(source, msg="clang")
File "C:\Users\danie\AppData\Local\Temp\pip-install-vvk574_e\pycryptodomex\compiler_opt.py", line 82, in test_compilation
objects = compiler.compile([fname], extra_postargs=extra_cc_options)
File "c:\program files\python39\lib\distutils\_msvccompiler.py", line 323, in compile
self.initialize()
File "c:\program files\python39\lib\distutils\_msvccompiler.py", line 220, in initialize
vc_env = _get_vc_env(plat_spec)
File "c:\program files\python39\lib\site-packages\setuptools\msvc.py", line 314, in msvc14_get_vc_env
return _msvc14_get_vc_env(plat_spec)
File "c:\program files\python39\lib\site-packages\setuptools\msvc.py", line 268, in _msvc14_get_vc_env
raise distutils.errors.DistutilsPlatformError(
distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/
Testing support for clang
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Click to expand...
Click to collapse
Now install linux