Develop Bugs

Can`t extract files from imgfs from HP iPaq 1950 ROM

Hi, all! Please, help me to solve my problem
Yesterday i tried to build custom ROM image for my PDA and failed .
I made following steps:
1. I made a backup of the existing ROM using mtty and SD card. The result was the 29mb raw file
2. i downloaded the last ROM update from the HP site, extracted it and obtained the ceos.nbf file.
3. Next, i tried to use prepare_imgfs and viewimgfs utilities whith both files.
4. After using prepare_imgfs i got, as expected, the imgfs_raw_data.bin file
5. But when i used viewimgfs, it couldn`t find any data in that file! Here is it's output:
Code:
guidBootSignature: F8 AC 2C 9D E3 D4 2B 4D BD 30 91 6E D8 4F 31 DC
dwFSVersion: 00000001
dwSectorsPerHeaderBlock: 00000001
dwRunsPerFileHeader: 00000001
dwBytesPerHeader: 00000034
dwChunksPerSector: 00000008
dwFirstHeaderBlockOffset: 00000200
dwDataBlockSize: 00001000
szCompressionType: LZX
dwFreeSectorCount: 00013620
dwHiddenSectorCount: 00000100
dwUpdateModeFlag: 00000000
Address: 00000200, dwBlockSignature: 00001D20
dwNextHeaderBlock: FFFBFFFF (size: FFFBFDFF)
Header type: 2F5314CE, Addr: 00000208
Unknown header type, FS_DATA_TABLE??
and more messages like that.
Can anyone tell me, where is the problem?
PS sorry for terrible English

stanru1 said:
Hi, all! Please, help me to solve my problem
Yesterday i tried to build custom ROM image for my PDA and failed .
I made following steps:
1. I made a backup of the existing ROM using mtty and SD card. The result was the 29mb raw file
2. i downloaded the last ROM update from the HP site, extracted it and obtained the ceos.nbf file.
3. Next, i tried to use prepare_imgfs and viewimgfs utilities whith both files.
4. After using prepare_imgfs i got, as expected, the imgfs_raw_data.bin file
5. But when i used viewimgfs, it couldn`t find any data in that file! Here is it's output:
Code:
guidBootSignature: F8 AC 2C 9D E3 D4 2B 4D BD 30 91 6E D8 4F 31 DC
dwFSVersion: 00000001
dwSectorsPerHeaderBlock: 00000001
dwRunsPerFileHeader: 00000001
dwBytesPerHeader: 00000034
dwChunksPerSector: 00000008
dwFirstHeaderBlockOffset: 00000200
dwDataBlockSize: 00001000
szCompressionType: LZX
dwFreeSectorCount: 00013620
dwHiddenSectorCount: 00000100
dwUpdateModeFlag: 00000000
Address: 00000200, dwBlockSignature: 00001D20
dwNextHeaderBlock: FFFBFFFF (size: FFFBFDFF)
Header type: 2F5314CE, Addr: 00000208
Unknown header type, FS_DATA_TABLE??
and more messages like that.
Can anyone tell me, where is the problem?
PS sorry for terrible English
Click to expand...
Click to collapse
i would like to know an answer for this too. thanks

I found solution to extract files, but i couldn't find out how to pack it back...
To unpack files, you need:
1. the Perl interpreter
2. script, which can be found on http://forum.wce.by/viewtopic.php?p=78923#78923
3. You must replace $A and $S variables with your values:
$A is the address of the beginning of the imgfs block. It can be found using WinHex and F8AC2C9DE3D42B4DBD30916ED84F31DC signature. $S is size of imgfs block. It can be found with signature E9FDFF. You must find the table of partitions, and take 4 bytes as it's shown on the screenshot, in the reverse order. (00 C0 BD 00 -> 00BDC000)
For example, for iPaq1950 rom, taken from HP update, values are:
$I = "CEOS.nbf";
$O = "bigbrother.bin";
$A = 0x003BB55A;
$S = 0x0000BDC0;
After Perl script ends it's work you can use imgfsToDump or viewimgfs tools.
For this solution HUGE thanks to Gvr.

stanru1 said:
I found solution to extract files, but i couldn't find out how to pack it back...
To unpack files, you need:
1. the Perl interpreter
2. script, which can be found on http://forum.wce.by/viewtopic.php?p=78923#78923
3. You must replace $A and $S variables with your values:
$A is the address of the beginning of the imgfs block. It can be found using WinHex and F8AC2C9DE3D42B4DBD30916ED84F31DC signature. $S is size of imgfs block. It can be found with signature E9FDFF. You must find the table of partitions, and take 4 bytes as it's shown on the screenshot, in the reverse order. (00 C0 BD 00 -> 00BDC000)
For example, for iPaq1950 rom, taken from HP update, values are:
$I = "CEOS.nbf";
$O = "bigbrother.bin";
$A = 0x003BB55A;
$S = 0x0000BDC0;
After Perl script ends it's work you can use imgfsToDump or viewimgfs tools.
For this solution HUGE thanks to Gvr.
Click to expand...
Click to collapse
Very nice thanks.
If you find a way to repack it, it would be even better. I actually want to use this method for the hp1930. My problem is that there are no official updates for this model, so I have no shipped roms. Only a sd image dump.
What type of image is the sd dump? dnf or .bin (raw) ?

I think, it`s a raw dump. In any case, imgfs is the same on both images.

stanru1 said:
I think, it`s a raw dump. In any case, imgfs is the same on both images.
Click to expand...
Click to collapse
thanks again

any idea on how to get start address and size from a sd image? I can't find those signatures in the dump
thanks

The signatures may differ in case of the specific image. If you whant, i could look at your file, if you upload your rom into the Rapidshare or the same file-uploading service and give me the link

stanru1 said:
The signatures may differ in case of the specific image. If you whant, i could look at your file, if you upload your rom into the Rapidshare or the same file-uploading service and give me the link
Click to expand...
Click to collapse
thanks man,
I'm uploading to rapidshare right now.
the image was created using a 64mb sd so the image size might be a bit longer
edit :
this is the link: http://rapidshare.com/files/25438977/1930.rar.html

any updates on how to modify a sd image?

Help wanted for interesting hack job

I'm trying to hack my in car satnav, it runs CE and I'm able to decompile one of the binary images and see some of the applications that the device runs, however I am trying to decompile the OS binary and the same decompile process will not work.
Anyone got any thoughts on what to try?
Oh it's an SH4 device btw and the ROM etc is loaded from DVD.
Thanks in advance for any help.
Cheers

Quick one,
E:\>ImgfsToDump.exe 07AVNe.bin
ImgfsToDump 2.0 RC 3
guidBootSignature: 42 30 30 30 46 46 A 0 0 6 80 FC C2 4F 0 0
dwFSVersion: 10800600
dwSectorsPerHeaderBlock: 99000000
dwRunsPerFileHeader: 09000002
dwBytesPerHeader: 01000900
dwChunksPerSector: 2B0009D0
dwFirstHeaderBlockOffset: 58000940
dwDataBlockSize: 40880356
szCompressionType:
dwFreeSectorCount: 97000000
dwHiddenSectorCount: 45000002
dwUpdateModeFlag: DC434543
Compression DLL does not support compression type ''!
Anyone that can point me in the right direction would be appreciated

[TUT] Manual Full XIP Porting (& MANY MORE TUTORIALS) [ONLINE]

Special Thanks
Abusalza (for the most initial start off guide)
Cmonex (for the “MOST” important finishing touches)
!Aman! (for all the testing and Hex edit helping)
Noonski (for being the inspiration to keep going )
Ervius (for developing the kitchen tool to perform all the operations)
In this forum there are many many tools from experts and likes for porting XIP, rebuilding dumped ROMs etc. This threads aims at showing or sharing what goes in the background of these automated tools and also aims at answering all the many unanswered questions about various factors of ROM cooking / editing I have come across in this forum
Suggestions / comments always welcome to make these tutorials even better
Index of Tutorials
Manual XIP Porting Guide: CLICK HERE
XIP Porting Updates from members: CLICK HERE
XIP Porting for Himalaya devices & others (Nokser): CLICK HERE
Misc XIP Updates: CLICK HERE
PagePool Changing Guide (for Diamond & Raphael): CLICK HERE
Gain More Storage Memory (Increase imgfs size) Guide: CLICK HERE
ULDR Partition Size Reduction Guide: CLICK HERE
MBR and MSFLSH50 Regions Screenshots: CLICK HERE
Gain More Storage Memory (compress imgfs) with LZX algorithm: CLICK HERE
Get High File System Index (!Aman!): CLICK HERE
Ervius's GUI kitchen thread to perform all operations, Noonski's amazing RunCC & AutoRun & SDAutorun tutorial thread
Ervius's post on patching nk.exe to change the EndRam address for more available RAM in device (original credits to cmonex )
Da_G's amazing new initiative to utilise the ULDR partition to upgrade ROM without re-flash
All the above guides and updates are compiled in pdf file also for offline reading, attached in this post as All Guides.zip
The imgfs Gain.zip is actually the 5th guide with pictorial seperatly put up for members who would want to refer only to that process
The Pictorial.zip is the 7th picture reference for offline reading
Donations for this hard work and research are much appreciated. Below are the links whom you may choose to provide those to
Donation to Abusalza, Donation to Cmonex, Donation to !Aman!, Donation to Ameet
Index of Threads (Manila related)
Ameet's Mode9 script editing ideas thread
l3v5y's tutorial thread for editing Manila files
NisseDILLIGAF's Manila Hash tool

Manual Full XIP Porting
Tools you need: (attached the tools in this thread for easy access)
HEX Calculator (recommended – HEX workshop (Not Free)), suggested Windows Calculator
XIPPort.exe
M’Reloc.exe
NBMerge.exe
Insert.exe
OS.nb.payload from 19965 build (shipped ROM)
Cup of nice strong Coffee (A Must)
Brief:
There are many different ways to port the XIP. Few mention of using the 723*.dsm for the build number, few others mention of using the coredll.dll module to have the latest build numbers. As my friends, Noonski and !Aman! always say “Only numbers are just eye wash, core system is what matters the most” Based on this as inspiration, I am posting this guide for manual XIP porting. A few places you may find colors in this guide, these are to visually link the data for easy understanding
The only files removed from the ported XIP are (these are removed to keep the new XIP within the original size):
osaxst0.dll + osaxst0.dll.imageinfo.txt
hd.dll + hd.dll.imageinfo.txt
bmui.nb0 + bmui.nb0.imageinfo.txt
Process:
Prepare OEMXip base
Dump your original XIP.bin (from 19965 build) with XIPPort.exe, and click “write maps” to get MAP.txt in the OUT folder
Open the MAP.txt and go through what you will need to achieve for a full port. I advice to keep this MAP.txt as a backup, just in case
Click “make pkgs” to get “OEMXipKernel” and “MSXipKernel” folders inside \Files and \Modules
Delete MSXipKernel folders from \Files and \Modules both
Now our base OUT folder is ready with OEMXipKernel
Prepare MSXip donor
Dump your donor XIP.bin (from 20758 build) with XIPPort.exe, and click “make pkgs” to get “MSXipKernel” folder inside \Files and \Modules
Delete osaxst0.dll + osaxst0.dll.imageinfo.txt, hd.dll + hd.dll.imageinfo.txt and bmui.nb0 + bmui.nb0.imageinfo.txt to get the new XIP within the original RAM size. If you don’t wish to delete these files, then you will need to increase the “physlast” in ROMHDR.txt. Process of which is not covered under this guide
Copy the MSXipKernel folders from \Files and \Modules both to the \Files and \Modules in the base OUT folder
Now our OUT folder is ready to be ported with the OEMXipKernel and MSXipKernel
Now to proceed with the reallocing you need to re open the packages which have been created. Open XIPPort.exe and click "undo" then click “realloc P” to re calculate the reallocation addresses. Then click “write maps” to get the new MAP.txt file
Open this MAP.txt and look in the o32_realaddr and e32_vbase addresses. Busenum.dll must be the last entry in both tables. Here you may find overlaps of the modules in a few or most places (seen as !!!!!!!!!!!!!!!!!!)
These are the overlaps which need to be taken care of by reallocating the modules in Initialized Data and Virtual Base addresses
You need to work our way up from the bottom of the list since the busenum.dll is reallocated at the last address of the memory
For example:
03f4c000 03fe3000 L00097000 Virtual base address of coredll.dll
03fe2000 03fe3000 L00001000 !!!!!!!!!!!!!!!!!!
03fe2000 03ff0000 L0000e000 Virtual base address of certmod.dll
03ff0000 03ffb000 L0000b000 Virtual base address of cachefilt.dll
03ffa000 03ffb000 L00001000 !!!!!!!!!!!!!!!!!!
03ffa000 04000000 L00006000 Virtual base address of busenum.dll
Meaning, e32_vbase address of cachefilt.dll is overlapping that of busenum.dll by 1000 (L00001000) Similarly e32_vbase address of coredll.dll is overlapping that of certmod.dll by 1000 (L00001000)
I recommend you use M’Reloc.exe for reallocating the addresses in imageinfo.bin and Notepad to reallocate the addresses in the corresponding imageinfo.txt files. Since the binaries (S000, S001...) must actually be relocated using M'Reloc, it is not enough to just adjust the values in the imageinfo.txt files
To calculate the revised addresses (in below example, the e32_vbase) of the overlapping module, open Hex Calculator. To do that you will need to know the e32_vsize of the overlapped module. To find that out open overlapped module (for e.g. cachefilt.dll) in M’Reloc.exe and see the e32_vsize (0000B000)
Now to correct the e32_vbase of cachefilt.dll, follow this calculation as a base (e32_vbase busenum.dll - e32_vsize cachefilt.dll = e32_vbase cachefilt.dll)
Meaning, (03FFA000 – B000 = 03FEF000) hence the correct e32_vbase address is 03FEF000
03ff0000 03ffb000 L0000b000 Virtual base address of cachefilt.dll
03ffa000 03ffb000 L00001000 !!!!!!!!!!!!!!!!!!
03ffa000 04000000 L00006000 Virtual base address of busenum.dll
Now since the cachefilt.dll is reallocated using the above calculation, the modules next in line above that will also have to be reallocated. Namely, certmod.dll (although not overlapping yet above the cachefilt.dll). To calculate the e32_vbase of certmod.dll you will need the revised e32_vbase address of cachefilt.dll which you got just now
I recommend writing down the e32_vbase, e32_vsize, o32_realaddr and o32_vsize of each module so it will be easier to calculate the correct addresses for reallocation)
Remember, you need to work our way up from the bottom of the list since the busenum.dll is reallocated at the last address of the memory
To reallocate the addresses for o32_realaddr, follow the above calculation, only this time replace the e32_vbase busenum.dll with o32_realaddr and e32_vsize with o32_vsize
Now open the corresponding imageinfo.txt file for each module and change the e32_vbase and o32_realaddr address values in the txt file of the values mentioned with V= and D=, seen for e.g. like this
Module name: cachefilt.dll
e32_vbase: V=03FEF000
o32[1].o32_realaddr: D=01FFE000
You will notice that the FLASHDRV.DLL module has the realaddr at 2 regions. Although I have not found a way to calculate the difference between both regions but I change the values as per Abusalza’s MAP.txt
o32[1].o32_realaddr: D=01FCC000
o32[3].o32_realaddr: D=01FD4000
Since the OEMXipKernel modules never change, I only correct values of the ported MSXipKernel modules
This is helpful if the MSXipKernel modules ported from donor ROMs are similar in the sizes. If not then you will need to do the calculation and correction of values
Once through with the address reallocation, open XIPPort.exe and click “realloc P” to re calculate the addresses for writing maps. It will show you errors regarding some regions, ignore those and click “write maps”. Open the new MAP.txt and recheck for (!!!!!!!!!!!!!!!!!!) If none found that means the XIP has been ported well
Now click “build xip_out.bin” to create the resulting XIP to be inserted into the ROM .payload file. Use this command for inserting the xip_out.bin into the .payload (presuming you already have the shipped OS.nb.payload file in the same working folder
insert.exe -i xip_out.bin -o OS.nb.payload -d 0x00320000 -s 0x004C0000
Check these values with your device imgfs since in Diamond the XIP starts at 0x00320000 and the imgfs starts at 0x007A0000, but for some reason the imgfs signature in Diamond is at 0x007E0000
Build OS.nb for use in the ROM folder from the .payload you just updated with the new XIP. Please note these commands are for Diamond device. Please check with your device on the same before building
nbmerge.exe –kaiser OS.nb
Now put this OS.nb file in the ROM, put the boot.rgu from 19965 (shipped ROM) into the \ROM\XIP folder and do not include any of the OEMXipKernel or MSXipKernel folders in OEM & SYS folder while cooking. I observed for some reason, WinCeNls_WWE folder cannot be taken out of XIP and included in SYS. Device wont boot, so keep that in XIP (found a working solution by spocky12: Here (last quote)
Please note the insertion of xip_out.bin can also be done through XIPPort.exe directly
Before clicking “write xip_out.bin to:” replace the name “nk.nb” with “OS.nb.payload” and the address to “00320000” all without quotes
IMP: There may be chances that although the XIP is working fine, but the windows are seen as QVGA versions. The solution to that is either of the below
XIP & SYS of the same builds or
XIP and the OS\Gwes.exe from same build
Cook the new ROM with your favorite kitchen (whichever doesn’t do anything with the XIP) and use this OS.nb file as base template for the ROM with the new XIP
With this note, I hope this guide will serve many as a guiding light and answer many questions on manual full XIP porting. Happy porting

Members Porting Updates
This is where we showcase the updates on XIP porting provided by our kind forum members
Original quote - Cmonex
Code:
[COLOR=royalblue][B]Quote=ababrekar[/B] - Busenum.dll must be the last entry in both tables[/COLOR]
Actually the values are arbitary, even though Microsoft decided to place coredll.dll as the last entry, i.e. at the highest memory address, it doesn't really matter. So, the values are arbitrary, but of course only within limits: the addresses must be divisible by 0x1000 (pagesize of the platform), and they must be inside the memory range reserved for XIP. part of that is the dllfirst and dlllast values in ROMHDR.txt. The other part (the higher addresses, 0x03xxxxxx) are determined by the following way: IMGFS .VM tells you the limits for IMGFS memory range, and XIP is beyond that range. So, if your OS doesn't want to boot, you can check if IMGFS .VM is overlapping with XIP memory range as per your MAP.txt for xip and dump_memorymap.txt (or .VM folder, etc) for IMGFS.
For example if IMGFS ends at 0x03DE0000, then the higher part of your XIP must start later than 0x03DE0000. You can of course modify this to make more space for XIP
If xipport crashes on writing maps it means you definitely have some overlaps left in. So yes, best to work with the maps from the original XIPs and only use the final XIP map to verify you got everything right
[COLOR=red]Btw, XIPPort's insertion function was found buggy on one device once, but cannot remember the details. It wasn't my device, so just posting this as a possible warning[/COLOR]
Oh, same applies to ROMMaster.exe, it is buggy when you try to use that to extract the XIP some ROMs
[COLOR=royalblue][B]Quote=ababrekar[/B] - Few mention of using the 723*.dsm for the build number, few others mention of using the coredll.dll module to have the latest build numbers[/COLOR]
Btw, coredll.dll replacement only works for that pre-WM6.1
And a last tip for [B]debugging [/B]if your OS doesn't want to boot: if you already checked that the maps are all ok and IMGFS doesn't overlap, etc., then if you have a new enough HTC device (for example HTC Athena and later is new enough), then go to SPL using mtty or putty or qmat and there the "task 37" command (without the quotes) will show KITL log, with lots of debug messages, that can be very helpful. (first you must issue "task 32", for "task 37" to work) - this doesn't appear to work on some Raphaels
Original quote - cruzzmz
Code:
If porting for [B][COLOR=teal]Zinc[/COLOR][/B]. After finish with all the MReloc, you need to Hex the S000 of nk.exe in the MODULES folder. The value can be found in MAP.TXT under the Modules
[COLOR=royalblue][B]Quote=ababrekar[/B] - 802FAA9C - 802faaf0 L00000054 rom_00 header: dlls=01f901fd-02000000 phys=80180000-803dc4fa, 24 modules, 10 files, 2 copyentries ext=8018265c ram=803dd000-83c00000 cputype=000001c2[/COLOR]
Open S000 in ur fav hex editor, then go to [B]Offset 1658[/B]
Change the original value i.e: [COLOR=red][B]802FAA9C [/B][/COLOR]and Hex edit it to [COLOR=blue][B]9CAA2F80[/B][/COLOR]
Original quote - DupinBJK
Code:
The addresses on the 80xxxxxx range should be on a [B]WORD [/B]boundary - Divisible by [B]4[/B]
Original quote - spocky12 (how to move wincenls to IMGF from XIP)
Code:
This is related to BootPhase key in boot.rgu. [URL="http://msdn.microsoft.com/en-us/library/ms885267.aspx"]According to Microsoft[/URL]
If this value is 0, then related filesystem is loaded prior to initialization of locale. But for this to work, the filesystem has to be loaded in Autoload key, like this :
[B][COLOR=red][HKEY_LOCAL_MACHINE\System\StorageManager\AutoLoad\FLASHDRV][/COLOR][/B]
[B][COLOR=red] "DriverPath"="Drivers\\BuiltIn\\FLASHDRV"[/COLOR][/B]
[B][COLOR=red] "LoadFlags"=dword:1[/COLOR][/B]
[B][COLOR=red] "Order"=dword:0[/COLOR][/B]
[B][COLOR=red] "MountAsRoot"=dword:1[/COLOR][/B]
[B][COLOR=red] "MountAsBootable"=dword:1[/COLOR][/B]
[B][COLOR=red] "BootPhase"=dword:0[/COLOR][/B]
With this, autoload will regsiter access to the imgfs filesystem before wince.nls is loaded. Then, when it'll be required, if it's not present in xip, it should be found in imgfs

Here is where we showcase miscellanous updates on XIP porting / MBR / MSFLSH50 which doesnt fall under the above categories. These are the updates which are not harming the system in any ways if left as is. Yet just a know how or just in case
Removing modules from XIP: Original quote - Cmonex
Code:
You can always remove osaxst0.dll, osaxst1.dll, hd.dll, kd.dll, and also bmui.nb0 - the latter is just a SplashScreen saying your OS can't boot and reflash or something (I forget the exact text)
The other files are Kernel debuggers and similar, best to remove them, because it just takes up space and can also cause problems if you somehow manage to use the wrong versions of them. They are mapped directly to the Kernel memory space, and if your device uses a different range (i.e. you didn't keep your original debugger dlls), it will prevent the rom from booting
Also I found it's ok to remove (m)encfilt.dll and cachefilt (put them in IMGFS if you want them)
[I][B]Physlast [/B][/I]can be changed up to [I]ulramstart [/I]value without problems (of course not if you don't have enough space in the flash, but that's not really a real life possibility). Of course that also assumes we are not talking about some older devices that have the xip mapped to a different memory range than [I]ulramstart[/I]
You can move [I]ulramstart/ulramfree [/I]too if you relocate nk.exe data section (usually S002) with M'Reloc-nk. Also relocation is needed for any other modules (such as giisr.dll, on some non HTC devices) that have mappings similar to nk.exe (so they have a data section in the map that points into [I]ulramstart/ulramfree [/I]range). on HTC devices I didn't really see such modules so not a real problem usually
Increasing the free RAM (Part 1): Detailed explanation here by DupinBJK
Simple explaination for easy understanding: (the below values are from a sample MAP.txt and dump_memoryMaps.txt (View attachment Examples.zip)) for trying to explain what comes from where and the actual values may differ from your files
Code:
8019e9a4 - 8019e9f8 L00000054 rom_00 header: dlls=[COLOR=red]01f801fc[/COLOR]-[COLOR=black]02000000[/COLOR] phys=[COLOR=black]80000000[/COLOR]-[COLOR=blue]8030c7b3[/COLOR], 28 modules, 10 files, 1 copyentries ext=80002b4c ram=8030d000-83000000 cputype=000001c2
[COLOR=blue]8030c7b3 [/COLOR]- 8030c7b3 L00000000 End: highest physical address
The blue value that mentioned is the physlast value. In the dump_memoryMaps.txt, you will find:
Code:
01F7F000 - 01F7FFFF (4095 bytes): bthasplugin.dll
after which the dllfirst starts in MAP.txt with a difference of 1FD length and
Code:
03D66000 - 03D6FFFF (40959 bytes): bthasplugin.dll
after which the e32_vbase starts (03dcb000 - 03dd4000 L00009000 Virtual base address of wce_rex.DLL) in MAP.txt with a difference of 5B001 length
Increasing the free RAM (Part 2): by Ameet
Changed the size of ROM in G'Reloc from 83000000 to 83400000 and increased the ulRAMEnd to the same value (83400000) getting free RAM space of L0306C000 (I dont know how to translate this into the actual size in % or in bytes) but with the original of 83000000 the RAM space was L02C6C000
Having done this, I get about 62% free memory without TF3D and approx 54% free memory with TF3D at system start
Extracting the XIP from any ROM: Detailed explanation here by boggsie
More explaination about XIP processes & editing OS version on 1st splash screen: by FormerPalmOS
Code:
[B][COLOR=red]1) [/COLOR][/B]The Initial Program Loader copies the XIP partition from the FLASH to SRAM - in Diamond and Touch Pro there is a custom Samsung chip that includes both NAND FLASH and SRAM. The overall physical RAM space where this is loaded is also hard-coded - see below. The amount of RAM used is variable - this info comes from a header in the XIP section - basically how much RAM does the XIP need? What's left is what you get for program memory.
[B][COLOR=red]2) [/COLOR][/B]IPL executes a jump to a hard-coded address within the SRAM - this should be busenum.dll, which is why busenum.dll has to be at a specific physical and virtual memory location.
[COLOR=red][B]3) [/B][/COLOR]busenum.dll does its thing (not sure entirely what) but eventually calls nk.exe. nk.exe is the kernel. nk.exe loads the other modules, initializes the hardware (that's why nk.exe is device-specific), and initializes the filesystem and basic device drivers (again why those are device-specific).
[COLOR=red][B]4) [/B][/COLOR]Once this process has completed, the filesystem proceeds to load the imgfs filesystem and turn control over to the full OS.
The virtual memory map for WM consists of a number of slots. The memory management unit in the CPU translates virtual memory references into physical memory addresses. Every loaded dll or exe must occupy a portion of virtual memory for its code and will likely also use some of the available RAM for its data. The location within virtual memory where the code for a dll or exe is loaded is determined at load time unless the dll or exe is a module (everything in the XIP is a module) in which case the virtual memory location is specified during cooking. In the XIP, the location of the RAM used is also specified - the process of relocating a module in the XIP specifies the virtual memory location for the code and and data in the case of nk.exe, the physical RAM location.
There are four VM sections we care about (note - I'm taking some liberty here - these don't exactly correspond with what Microsoft refers to as a VM slot). Slot 0 runs from 0x00000000 to 0x01FC0000 (in the CDMA Touch Pro). The end of slot 0 is a function of the number of and size of the data regions for the DLL modules in the XIP. This number plus 0x1FC is stored in the ROM header (and can be examined in ROMHRD.txt) - it is referred to as dllfirst. This is also the slot 0 you see when you do G'Reloc.exe (the value in G'Reloc.exe is the last address of slot 0 plus one). These two must match!!! What the XIP uses must not overlap with what your ROM uses.
The next slot is the XIP DLL initialized data. This runs from dllfirst to dlllast. dlllast is fixed (in the Touch pro) at 0x02000000. The XIP DLL data sections are loaded starting at 0x02000000 and working backwards.
The next slot is again available for the OS and runs from dlllast to wherever the code in the XIP starts. You can see this in your XIP memory map - this again must match (the end of slot 1 in G'reloc.exe must match the first DLL virtual base address in your XIP - in mine this is 0x03DC0000). The XIP DLL and EXE code occupies from this virtual memory address to 0x03FFFFFF.
The OS will load DLLs and EXEs (other than XIP) into this slot starting at 0x03DC0000 and working backwards, then will move to the slot below 0x01FC0000. Recall, I'm using my numbers here. Any modules in the ROM will have their virtual memory slot and address pre-assigned. Any non-module DLL or EXE will be relocated to an available slot and VM address at load (this is why modules load quicker).
So in summary, my VM map looks like this:
0x00000000 - 0x01FC0000 - OS available (G'Reloc.exe slot 0)
0x01FC0000 - 0x01FFFFFF - XIP data
0x02000000 - 0x03DC0000 - OS available (G'Reloc.exe slot 1)
0x03DC0000 - 0x3FFFFFFF - XIP modules code
The actual physical XIP RAM address starts at 0x80000000 in the Touch Pro (this is physfirst in the ROMHDR.txt) and ends at 0x83400000 (in the Verizon Touch Pro - this is ulRAMEnd). The XIP is copied from the NAND flash starting here with the ROM header occupying 0x80000000 - 0x80001000. Then come the various XIP components, hopefully none of which overlap. The XIP should end at or before a ROMHDR.txt value called physlast. Thus physlast - physfirst is the size your XIP has to fit into.
Following physlast comes ulRAMStart - this is where the RAM required for nk.exe is located. This RAM ends at ulRAMFree. What remains after ulRAMFree until ulRAMEnd is available for your OS. Shrinking your XIP and relocating nk.exe will allow you to recover wasted space and give you more program memory, but it buys you nothing to move a module out of the XIP if it is required by the system. Only things that aren't required (like debuggers and hard drive drivers) should be removed.
Also, the least significant 16 bits must be zero (lower four hex digits) of the end of vm slot 0 and slot 1 in G'reloc.exe and in your ROMHDR.txt. The least significant 14 bits must be zero (the lower four digits can only be 0000, 4000, 8000 or C000) of the RAM address (ulRAMStart and ulRAMFree).
Code:
Hex edit the S000 file in the nk.exe module folder and search for the revision string. You can find it by doing a search for the unicode string "[B][COLOR=red]Kernel Built[/COLOR][/B]" (Hex String [B][COLOR=red]4B 00 65 00 72 00 6E 00 65 00 6C 00 20 00 42 00 75 00 69 00 6C 00 74 00[/COLOR][/B]). Shortly after that will be the revision that is displayed on the phase 1 boot screen (small red letters in the lower right corner of the device on CDMA Touch Pro). Change that (make sure to overwrite, not to insert, and limit it to 12 characters in unicode format.
When you rebuild your xip.bin and cook with it, you should see this value on the screen during phase 1 boot. The only other way would be to insert a marker into the boot registry

Change PagePool through Hex editing (for Diamond & Raphael)
I'm putting this up here so to answer one more unanswered question about this especially for Diamond & Raphael ROMs
To change PP of Diamond ROMs:
Open the OS.nb in Hex editing software
1. Go to offset 0x37AD68 to find 03 25 A0 E3 03 15 A0 E3 00 20 83 E5 hex string (If this string is not found at the 37AD68 offset, then search for this hex string)
Replace the string with 20 83 E5 with 00 A0 E1
This will make the string NOP (No Operation) meaning, the ROM wont set the PP to default 12MB but will allow the change in below offset
2. Now go to offset 0x3A7F94 to find E0 E2 04 80 00 00 60 00 hex string
again, if this hex string not found at the 3A7F94 offset, then search for the hex string. Just as a hint, this string is after the second NKKD8 (search for text string)
60 is the size of PP that you can now modify to suit your liking
e.g. I made mine 00 to get 0MB PP. Or change it to 80 to get 8MB PP, so forth and so on
With changing the first hex string and making the Kernel NOP, you can also use the tool to change PagePool and it does work
Also to make it a permanent change you can hex edit the first mentioned string in S000 of nk.exe module in XIP and then modify the PP with the program or by hex on OS.nb
To change PP of Raphael ROMs:
Search for hex string: 03 15 A0 03 02 15 A0 13 00 10 82 E5 and change the last 4 bytes to 03 15 A0 03 02 15 A0 13 00 00 A0 E1 then the normal PP Changer tool will work
This is the 2nd string, ignore the 1st one coz that's in ULDR

Gain more Storage Memory (increase imgfs size)
There are 4 partitions in Diamond ROMs
part00 – ULDR
part01 – XIP
part02 – IMGFS
part03 – FAT (This partition exists only on few devices)
We all port XIP from different devices to exclude few modules to gain space and to upgrade the kernel and make the XIP partition smaller in size. Although the new XIP is smaller in size but because of the insertion addresses of XIP & imgfs, there is a gap of wasted space filled with FF between end of XIP & start of imgfs. Although there is no way we can include this space into XIP as free RAM but make use of this space in imgfs and gain whatever storage space we can
Files used as example for this tutorial
xip_out.bin: My own ported XIP of size (30CA12 in Hex, 3195154 in bytes)
os.nb.payload: My own cooked payload (since I also wanted the final ROM to be a cleaner ROM)
imgfs start: in my payload at 0x7A0000 (unedited)
XIP start: in my payload at 0x320000 (unedited)
Before we move into hex editing, let me give an overall outlook of the MBR & MSFLSH regions of the ROM
MBR is the Master Boot Record of the ROM (512 bytes) from 0x0 to 0x1FF. The infomation of partitions types Flags in hex offsets are called from the registry entry mentioned in boot.rgu below
The starting block (LBA) and number of sectors for each partition are defined as shown below
part00. 1C6 – 1C9 (starting block) 1CA – 1CD (number of sectors)
part01. 1D6 – 1D9 (starting block) 1DA – 1DD (number of sectors)
part02. 1E6 – 1E9 (starting block) 1EA – 1ED (number of sectors)
part03. 1F6 – 1F9 (starting block) 1FA – 1FD (number of sectors)
[HKEY_LOCAL_MACHINE\System\StorageManager\PartitionTable]
"04"="FATFS" ; (hex: 1F2)
"20"="BOOT" ; (hex: 1C2)
"23"="RAWFS" ; (hex: 1D2)
"25"="IMGFS" ; (hex: 1E2)
MSFLSH50 is the Flash region of imgfs from 0x800 (see post #8 for screenshots, shown here is for Diamond) to 0xFFF. The starting block of imgfs is located in MSFLSH at 81C
e.g. if your device ROM's sector size is 200 then the MSFLSH50 region will starts at 0x200 and so on
Moving into the hex editing mode for making use of the wasted space between the actual XIP end & start of imgfs partitions
The new xip_out.bin is 30CA12 in total size (check your actual xip_out.bin size, shown here is just example) starting at 0x320000 (check you device XIP start, shown here is for Diamond) and ideally should end at 62CA12. But since the starting block of imgfs must be divisible by 20000 (see post #8 for screenshots, shown here is for Diamond) the imgfs needs to start at 640000. So the new XIP will have to be inserted into the payload at 0x320000 till 0x640000 with XIP size of 320000 and reduced wastage of 135EE bytes
The imgfs can also start at 630000 since this is directly after the XIP and also divisible by 20000, used here is 640000 as expansion for future xip_out.bin
Open the existing os.nb.payload in hex editor. Delete everything from 0x640000 till 0x79FFFF. This will move the imgfs from 0x7A0000 to 0x640000. Since we are now moving the imgfs partition next to new XIP, the number of sectors in new XIP and new LBA of imgfs needs to be edited to the revised value in the MBR region
To calculate the new starting block of imgfs partition we need the number of sectors in new XIP. To calculate that, use the following method
In Hex calc
Number of sectors = size of partition / sector size
e.g. (new XIP) 320000 (shown above) / 800 (see post #8 for screenshots, shown here is for Diamond) = 0640
since the coding is in little endian, we have to reverse these values to 40 06 00 00
Go to offset 0x1DA and change the values to 40 06 till 1DB and then 00 00
Now realloc the LBA of imgfs since we revised the number of sectors in XIP and to calculate that, use this method
In Hex calc
Logical Block Address (LBA) = Previous Partition LBA + Previous Partition number of sectors
e.g. (XIP LBA) 0640 + (XIP no of sectors) 0640 = 0C80
since the coding is in little endian, we have to reverse these values to 80 0C 00 00
Go to offset 0x1E6 and change the values to 80 0C till 1E7 and then 00 00
Logical Block Address (LBA) should be equal to (Previous Partition LBA + Previous Partition number of sectors * Sector Size)
e.g. (XIP LBA) 0640 + (XIP no of sectors) 0640 * 800 (see post #8 for screenshots, shown here is for Diamond) = 640000 (size of imgfs partition)
Similarly to imgfs calculate and change the LBA of FAT at 1F6 and 1F7 using the default imgfs no of sectors (use these since the cooking tools will change these as per actual size)
We have changed the LBA and number of sectors in MBR, but the OS needs to know the block address of imgfs in MSFLSH50 region
To calculate that, use this method
In Hex calc
MSFLSH50 Block Address = imgfs partition starting address / 20000 (see post #8 for screenshots, shown here is for Diamond)
e.g. (imgfs starting address) 640000 (shown above) / 20000 = 32
Go to offset 0x81C and change the value to 32
Save and close the os.nb.payload file in hex editor. Insert the new XIP into this file using this command
“insert.exe -i xip_out.bin -o OS.nb.payload -d 0x00320000 -s 0x00320000” (check your insert start address, shown here is for Diamond)
To calculate the size of XIP from MBR, use this method
In Hex calc
Size of XIP = Number of Sectors * Sector Size
e.g. (if the no of sectors in little endian) 0640 (shown above) * 800 (see post #8 for screenshots, shown here is for Diamond) = 320000 (sector size for diamonds)
This value shall be the "-s" while using insert.exe tool and to calculate the start of the XIP, use this method
In Hex calc
XIP Start = imgfs Start + "-s"

Reduce ULDR Partition Size
“ULDR” stands for “Update Loader”, and is part of the Image Update system. This system allows deployed devices to be updated with new software after they ship. The Update Loader reads a configuration stored in persistent memory and downloads and installs new versions of operating system or OEM files
Also known as part00 in the ROM, is something we all wish to get rid of and use the space as additional storage memory. This tutorial currently aims at reducing the size of this partition by 3.0 MB
Tools you need
NBSplit.exe
NBMerge.exe
Hex editor
Ervius's Payload Reducer
IMPORTANT NOTES
The template OS.nb used is the same OS.nb in which the XIP is inserted at 320000 and of size 320000
For best results use Ervius's Payload Reducer to reduce the size of payload from shipped ROM use nbmerge.exe to cook OS.nb as template for further cooking
This ROM is assumed to be from Diamond and check your device values as per the guide below
The hex offsets of (L)ogical (B)lock (A)ddress and number of sectors and imgfs block address are mentioned in tutorial above or in the post #8 below
Process
Extract OS.nb.payload from the OS.nb (nbsplit.exe –kaiser (check your device) OS.nb)
Run the OS.nb.payload through Ervius's Payload Reducer tool to remove all files from the imgfs and keep only the partition headers
Open this OS.nb.payload in your hex editor. We need to change LBA values of the partitions and number of sectors of ULDR partition since we are reducing the size
In the MBR region (partition Flag 20) LBA of ULDR partition remains same since we are not moving it anywhere. The existing number of sectors of ULDR is 3E 06 from little endian it will be 063E. We are removing 0600 sectors from this partition (0600 * 800 (size of sector, see post #8 for screenshots) = 300000) so, 063E – 0600 = 00 3E. Write it in little endian at hex offset 1CA and 1CB to 3E 00
To physically reduce the partition, remove all data between hex offsets 0x20000 till 0x31FFFF. This will make the XIP start from hex offset 0x20000 till 0x33FFFF and the imgfs partition start at 0x340000
Now since we have reduced the size of ULDR partition, the LBA values of XIP and imgfs partitions will have to be changed in the MBR region
Now change the LBA of XIP. To calculate that, use this method
In Hex calc
Logical Block Address (LBA) = Previous Partition LBA + Previous Partition number of sectors
e.g. (ULDR LBA) 00 00 00 02 + (ULDR no of sectors) 00 00 00 3E = 00 00 00 40
since the coding is in little endian, we have to reverse these values to 40 00 00 00
Go to offset 0x1D6 and change the values to 40 00 00 00 till 1D9
Now change the LBA of imgfs. To calculate that, use this method
In Hex calc
Logical Block Address (LBA) = Previous Partition LBA + Previous Partition number of sectors
e.g. (XIP LBA) 00 00 00 40 + (XIP no of sectors) 00 00 06 40 = 00 00 06 80
since the coding is in little endian, we have to reverse these values to 80 06 00 00
Go to offset 0x1E6 and change the values to 80 06 00 00 till 1E9
We have changed the LBA and number of sectors in MBR, but the OS needs to know the block address of imgfs in MSFLSH50 region
To calculate that, use this method
In Hex calc
MSFLSH50 Block Address = imgfs partition starting address / 20000 (see post #8 for screenshots, shown here is for Diamond)
e.g. (imgfs starting address) 340000 (shown above) / 20000 = 1A
Go to offset 0x81C and change the value to 1A
Save and close the os.nb.payload file in hex editor. Insert the new XIP into this file using this command
“insert.exe -i xip_out.bin -o OS.nb.payload -d 0x00020000 -s 0x00320000” (check your insert start address, shown here is for Diamond) (ignore this if the XIP is already inserted and shifted to this location with this size)
The value (02) seen at hex offset 0x1BF should not be changed or touched since that value tells the OS that first partition starts from the third Sector of the ROM (0x800 (sector size) + 0x800 = hex offset 0x1000) Currently the reduced ULDR partition starts from third sector
Now create the OS.nb from the edited OS.nb.payload to be used as template for cooking using this command
“nbmerge.exe –kaiser (check your device) OS.nb” (without -conservative switch)
NOTE
For best results directly use the OS.nb.payload as template for cooking without merging it into OS.nb. For this you will need to edit the CreateROM.bat
Note the change in red and delete the blue lines from this bat file
copy ROM\OS.nb.payload temp\OS.nb.payload
..\TOOLS\NBSplit -kaiser OS.nb
Rem rename os.nb.extra os-new.nb.extra
!Aman!'s awesome tutorial on removing ULDR partition from devices which don't have the FAT partition (part03) can be refered here: http://forum.xda-developers.com/showthread.php?t=446506

Screenshots of MBR and MSFLSH50 Regions
MBR Region
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
MSFLSH50 Region
Attached these images in Pictorial.zip with post #1 for offline reference

Gain more Storage Space with LZX compression
Thanks to:
spocky12 for cecompr_nt.dll (attached)
ivanmmj for cecompr.dll (attached) This module supports LZX compression as well as the default XPR algorithm
Replace the cecompr.dll found in the OEMXipKernel (or whichever folder you have your XIP modules) with the attached cecompr.dll module that supports LZX compression
The LZX compression takes a load of your RAM while cooking which makes the continuing IMGFSFromDump tool crash. To avoid that replace the attached cecompr_nt.dll file found in “Tools/IMGFS” folder of your kitchen
Pause your kitchen process right after it extracts the IMGFS.bin and before it inserts the files into it. (A simple “PAUSE” in the batch file will suffice). Then open up your IMGFS.bin in hex editor of your choice and search for the string "XPR". Replace the FIRST one, FIRST & ONLY ONE with "LZX". Close the hex editor, save the file and let your kitchen continue with the cooking
After flashing the ROM cooked with this module should give you approx. 10MB more space in the storage memory
Original Posts:
ivanmmj: http://forum.xda-developers.com/showpost.php?p=3678382&postcount=877
spocky12: http://forum.xda-developers.com/showpost.php?p=3690996&postcount=904

Space for Donors
High value real estate space for donors for all my work on XDA
Special Thanks
Piranha1 $5
Vippie $5
Steven Ellis $10
Guitarguy $5
Ckpv5 $5
Letama $10
!Aman! $5
LoriInWa $10
Noonski $15
Kevin $5
Dogmale $1
Nazaliyah $1​

Miscellanous uploads related to XIP porting
Kitchen Files
I use "DiamondKitchen_v0.4" kitchen to cook Diamond ROMs. The XIPInsert file is something that I made to automate the insertion and nbmerge process (well, something automatic is better than complete manual )
If you select to use the XIPInsert batch file then you must have DiamondKitchen_v0.4\XIP\xip_out.bin and DiamondKitchen_v0.4\XIP\OS.nb.payload to use this option, else the existing OS.nb file in \ROM folder will get deleted
Note: The insert values used in the batch file is for Diamond ROMs. Please check and edit these as per your devices
Code:
[B][U][I]!COOK.cmd[/I][/U][/B]
Modified to provide options to include the below batch file or to continue without it
Also included necessary warning
Code:
[B][U][I]8a.XIPInsert.bat[/I][/U][/B]
@echo off
cd [COLOR=green]XIP[/COLOR]
..\TOOLS\insert.exe -i [COLOR=green]xip_out.bin[/COLOR] -o [COLOR=green]OS.nb.payload[/COLOR] -d 0x00320000 -s 0x004C0000
..\TOOLS\nbmerge -kaiser OS.nb
copy OS.nb ..\ROM\OS.nb
del OS.nb
del *.payload
del *out.bin
exit
Deprotecting ROMs: My friend, Ervius has made a small tool to "deprotect" a protected ROM , here: http://forum.xda-developers.com/showthread.php?t=465642

First to say THANKS
Cheers

and second to say this is real hard work...but u have done it bro..
congrats

another amazing work by the XIP Master

Thanks for sharing the info, ababrekar! I'm sure this will help out many people (myself included ).

Bite Down and Don't Give Up.
Sounds like someone i know from.... hey, it is you!
Good Job,

ababrekar said:
Use this command for inserting the xip_out.bin into the .payload (presuming you already have the shipped OS.nb.payload file in the same working folder
insert.exe -i xip_out.bin -o OS.nb.payload -d 0x00320000 -s 0x004C0000
Check these values with your device imgfs since in Diamond the XIP starts at 0x00320000 and the imgfs starts at 0x007A0000, but for some reason the imgfs signature in Diamond is at 0x007E0000
Click to expand...
Click to collapse
just to make it more clear, the value for "-s"= (starting offset of imgfs - starting offset of XIP)
PS: wonderful job writing this guide brother

Ouch - too much like work, but it is nice to know how to do it.
Thanks for your effort!
Best regards,
-boggsie
So, how does one get hooked into the flow of new releases?

Perhaps would be a good idea to use one of your reserved posts as a repository of good XIP.BIN files with versions and info about the ROM extracted, so all we can use in our ROMs... only a tought...

jcespi2005 said:
Perhaps would be a good idea to use one of your reserved posts as a repository of good XIP.BIN files with versions and info about the ROM extracted, so all we can use in our ROMs... only a tought...
Click to expand...
Click to collapse
That is a good idea but I'll do that for posting the unedited XIP.bin files from dumped ROMs since the xip_out.bin I build would be for my device. People wont want to ruin their ROMs with someone else's ported XIP, right?

Re-signing the system

Various aspects of the Nook system are signed with a signature from Barnes & Noble.
There are a few places where signatures are compared.
Various system apps used a single "shared id" and they must all have the same signature.
/system/framework/framework-res.apk must have a correct signature with respect to AndroidManifest.xml.
In any case, it's your Nook, what are you going to do?
Re-signing the system
make a full backup and make sure that it is good
create your own signature http://developer.android.com/tools/publishing/app-signing.html
make a directory for your patch
create the subdirectory META-INF\com\google\android\
put a copy of update-binary in there
write a updater-script and put it in there
create the subdirectory system\app
create the subdirectory system\framework
For each of your APKs in /system/app and also /system/framework/framework-res.apk:
unzip them somewhere
delete the whole directory META-INF from them
zip the directory
jarsigner them with your own personal signature
zipalign the APK (optional if you are lazy and don't see the point)
put it in the appropriate patch directory
Then:
zip the patch directory
copy it to your SD card
make sure that your WiFi is turned on if you are using ADB over WiFi!
recovery boot using ClockworkMod
install the patch from SD card
reboot
updater-script
Code:
# Replace signed components
mount("ext2", "/dev/block/mmcblk0p5", "/system");
package_extract_dir("system/app", "/system/app");
package_extract_dir("system/framework", "/system/framework");
unmount("/system");
# Delete packages.xml
mount("ext3", "/dev/block/mmcblk0p8", "/data");
delete("/data/system/packages.xml");
unmount("/data");
Flies in the ointment, caveats, etc...
The packages.xml contains some form of certs that have all changed.
Right now, the simplest way I know to deal with this is just to delete packages.xml.
The problem is, this will break most user applications since the user IDs will no longer agree.
The easiest thing to do is just to reinstall them.
For applications with a lot of data, it would be best to back up the configs or data.
When you first boot up, you may think that you are in a "boot loop".
The boot animation will run continuously.
If you have ADB connect still (and you had better!) you can fix this.
Your launcher application is probably causing lots of error on startup.
There are two ways to fix the problem with the launcher (or any other app)
uninstall and reinstall it
go into /data/data/com.myapp.whatever and chown everything to the user id of the application.
Code:
busybox chown -R 10011: databases
Don't chown the lib directory if there is one.
Then you should have a device that boots up normally.
Good luck, Mr. Phelps.

Renate NST said:
create your own signature http://developer.android.com/tools/publishing/app-signing.html
Click to expand...
Click to collapse
Renate,
Won’t it be easier to use Andriod media key?
If we do, we can patch packages.xml, instead of deleting it, right?
Renate NST said:
For each of your APKs in /system/app and also /system/framework/framework-res.apk:
unzip them somewhere
delete the whole directory META-INF from them
zip the directory
jarsigner them with your own personal signature
zipalign the APK (optional if you are lazy and don't see the point)
put it in the appropriate patch directory
Click to expand...
Click to collapse
I wrote a script to do just that, can be adapted easily...
Code:
@set keystore=..\keys\media.jks
@set storepass=android
@set alias=media
@set resigned_dir=.\new
@for %%i in ( .\*.apk ) do @(
echo %%i
copy %%i %resigned_dir%\%%~ni_%%~xi
zip -d %resigned_dir%\%%~ni_%%~xi META-INF\*
jarsigner -keystore %keystore% -storepass %storepass% %resigned_dir%\%%~ni_%%~xi %alias%
zipalign -f 4 %resigned_dir%\%%~ni_%%~xi %resigned_dir%\%%~ni%%~xi
del %resigned_dir%\%%~ni_%%~xi
)
@goto :eof
Just my 2 cents…
---------- Post added at 04:31 PM ---------- Previous post was at 04:22 PM ----------
Renate NST said:
...
Then you should have a device that boots up normally.
Good luck, Mr. Phelps.
Click to expand...
Click to collapse
Guys,
If you run into a problem following Renate steps, it’ll be practically impossible to troubleshot without logcat log.
It might be a bit safer to use ADB over USB then over Wireless.
Even if you run into boot loop, ADB should work still.
I’m not 100% sure, if you need framework operational to establish wireless connection (for ADB to use).
ADB over USB definitely doesn't need framework running.

Yes, of course I used a script to resign the individual APKs.
Yours is nice though.
I'm not sure what you mean by "Android media key".
Do you mean the androiddebug key?
Did you re-sign framework-res.apk too?
Well, one advantage of deleting packages.xml is that it gets rid of the cruft.
I was thinking of just writing a little utility that resolved the renumbered user ids and fixed file ownership.
P.S. WiFi works fine when the boot animation is still looping.
The loop animation just runs until something wants to use the screen.
The system is actually 100% up at that point.
It's just that your home application (a launcher probably) can't run.
You can still start an application by am start intent.
That's also a warning to not presume that your Nook is dead just because the display loops.

Renate NST said:
Yes, of course I used a script to resign the individual APKs.
Yours is nice though.
Click to expand...
Click to collapse
Thank you!
Renate NST said:
I'm not sure what you mean by "Android media key".
Do you mean the androiddebug key?
Click to expand...
Click to collapse
I don’t remember now, it was long time ago.
AFAIR, it was 4 keys
testkey -- a generic key for packages that do not otherwise specify a key.
platform -- a test key for packages that are part of the core platform.
shared -- a test key for things that are shared in the home/contacts process.
media -- a test key for packages that are part of the media/download system.
You can download them still from Google repository
http://mirror.yongbok.net/pub/pub/linux/android/repository/build/target/product/security/
Most ppl call media key androiddebug key, don’t ask me why.
Renate NST said:
Did you re-sign framework-res.apk too?
Click to expand...
Click to collapse
Not as of now. Waiting for your Reader.apk...
Renate NST said:
Well, one advantage of deleting packages.xml is that it gets rid of the cruft.
I was thinking of just writing a little utility that resolved the renumbered user ids and fixed file ownership.
Click to expand...
Click to collapse
I dunno if it recreates UserID properly.
I.e. you have apps A, B, C installed they got UserIDs 10001, 10002 & 10003 respectively.
Then you uninstall A & B and delete delete packages.xml, would C get 10003 still?
Need to test it.
Renate NST said:
P.S. WiFi works fine when the boot animation is still looping.
The loop animation just runs until something wants to use the screen.
The system is actually 100% up at that point.
Click to expand...
Click to collapse
Yep. Thanks for confirming this!
Renate NST said:
It's just that your home application (a launcher probably) can't run.
You can still start an application by am start intent.
That's also a warning to not presume that your Nook is dead just because the display loops.
Click to expand...
Click to collapse
When I see Nook booting image "with running dots", ADB is up already.
I was under impressing that’s the image ppl see while in boot loop.
ps shows it as bootanimation process
I guess, we are NOT on the same page again…

The running dots (boot animation) gets started as the system starts.
It just runs until something takes over the screen.
If it runs continuously, it could mean that the system is in a boot loop or
simply that no application is rising to the challenge to do something.
On the other hand, if the dots are running, but it hiccups and starts over from the first dot, that's a real boot loop.

Renate NST said:
The running dots (boot animation) gets started as the system starts.
It just runs until something takes over the screen.
If it runs continuously, it could mean that the system is in a boot loop or
simply that no application is rising to the challenge to do something.
On the other hand, if the dots are running, but it hiccups and starts over from the first dot, that's a real boot loop.
Click to expand...
Click to collapse
Renate,
I neither completely agree with you on bootanimation app nor want to pollute this thread with useless (IMO) discussion about it. If you want discuss it further – could you open another thread?

Well, I proved that you can take Settings.apk and SettingsProvider.apk off the emulator, sign them and install them.
There are a number of problems with that, starting out that the opening screen is white on white.
Also, my Nook seems to think it's a phone now and the hack that I did for the "n" button is broken.
I switched back to the stock version.
On the plus side, my Nook now opens with just a button press and no swiping.
I remember some people were interested in that.
It's probably something in settings.db

Renate NST said:
Well, I proved that you can take Settings.apk and SettingsProvider.apk off the emulator, sign them and install them.
Click to expand...
Click to collapse
AFAIR, everything on emulator is signed with keys I posted and nothing with B&N key - you don't need to resign.

Renate NST said:
Various aspects of the Nook system are signed with a signature from Barnes & Noble.
Click to expand...
Click to collapse
Renate,
I can write a script (Win) to do:
Parse packages.xml to find APKs run as 'shared-user name="android.media" userId="10000"'
Pull (backup) them to PC
Resign
Stop framework
Push resigned APKs to NST
Replace B&N cert reference in packages.xml to the one we used
Start framework
It might be some manual steps...
Do you think it might be useful?
And another script to restore...

First thing, I think that doing a system update to replace (as recommended in my first post) is overkill.
I wasn't sure whether simply starting and stopping the framework from the shell would be sufficient.
Apparently it is.
My only excuse is that I've bricked my Nook about 20 times and was being conservative.
What you want to sign your Nook with is your choice.
I hadn't looked into using any common signatures.
Android only mentions the single debug key in their documentation.
The emulator apks are signed with an Android signature, but not the same as the debug key.
ApokrifX said:
I can write a script (Win) to do:
Parse packages.xml to find APKs run as 'shared-user name="android.media" userId="10000"'
Click to expand...
Click to collapse
Ok, but there is also all the other sharedUserId="1000"
I'm not sure how the cert references work in packages.xml
Does it work for framework-res.apk too?

Looks like I cannot answer your question.
I guess, we can create a table [sharedUserId] – [App], [sharedUserId] – [Cert] and [Cert] - [App]
It can shed some light on how it works.
I can see same sharedUserId used with different certs, so apps should use different android users...
See below:
Don’t know how to map sharedUserId to android user yet.
My [current] understanding is:
userId "10000" and above are apps generated.
Below – for system use.
I have now:
<package name="com.bn.nook.quickstart" codePath="/system/app/QuickStartActivity.apk" system="true" ts="1217592000000" version="7" sharedUserId="1000">
<cert index="4" />
<package name="com.google.android.server.checkin" codePath="/system/app/GoogleCheckin.apk" system="true" ts="1292347460000" version="7" sharedUserId="1000">
<sigs count="1">
<cert index="13" />
Obviously, due to cert mismatch, "com.bn.nook.quickstart" & "com.google.android.server.checkin" should use different users.
---------- Post added at 09:29 PM ---------- Previous post was at 09:04 PM ----------
ApokrifX said:
Looks like I cannot answer your question.
I guess, we can create a table [sharedUserId] – [App], [sharedUserId] – [Cert] and [Cert] - [App]
It can shed some light on how it works.
Click to expand...
Click to collapse
Here we go:
Code:
0 10019 com.google.android.apps.gtalkservice /system/app/gtalkservice.apk
0 10019 com.google.android.googleapps /system/app/GoogleApps.apk
0 10019 com.google.android.providers.gmail /system/app/GmailProvider.apk
0 10019 com.google.android.providers.talk /system/app/TalkProvider.apk
0 10021 com.google.android.gm /system/app/Gmail.apk
0 10022 com.android.vending /system/app/Vending.apk
1 10002 com.android.globalsearch /system/app/GlobalSearch.apk
1 10002 com.android.googlesearch /system/app/GoogleSearch.apk
1 10002 com.android.inputmethod.latin /system/app/LatinIME.apk
1 10002 com.android.launcher /system/app/Launcher.apk
1 10002 com.android.providers.applications /system/app/ApplicationsProvider.apk
1 10002 com.android.providers.contacts /system/app/ContactsProvider.apk
1 10002 com.android.providers.userdictionary /system/app/UserDictionaryProvider.apk
10 10001 com.adobe.air /system/app/AirRuntime.apk
10 10017 de.devmil.minimaltext /data/app/mt262.apk
10 10023 com.google.android.talk /system/app/Talk.apk
11 10013 com.ngc.fora /data/app/com.ngc.fora.apk
12 10015 siir.es.adbWireless /data/app/siir.es.adbWireless-1.apk
13 1000 com.google.android.providers.subscribedfeeds /system/app/GoogleSubscribedFeedsProvider.apk
13 1000 com.google.android.server.checkin /system/app/GoogleCheckin.apk
14 10018 com.david1171.minimalistblack /data/app/com.david1171.minimalistblack-1.apk
15 10014 com.smart.swkey /data/app/SWKey21.apk
16 10030 com.asksven.betterbatterystats /data/app/com.asksven.betterbatterystats.apk
17 10027 jackpal.androidterm /data/app/jackpal.androidterm.apk
18 10029 com.googlecode.droidwall.free /data/app/com.googlecode.droidwall.free.apk
19 10016 org.adw.launcher /data/app/org.adw.launcher-1.apk
2 10026 org.coolreader /data/app/org.coolreader.apk
20 10012 com.noshufou.android.su /data/app/Superuser.apk
3 10024 berserker.android.apps.sshdroid /data/app/berserker.android.apps.sshdroid.apk
4 1000 android /system/framework/framework-res.apk
4 1000 com.android.providers.settings /system/app/SettingsProvider.apk
4 1000 com.android.providers.subscribedfeeds /system/app/AccountAndSyncSettings.apk
4 1000 com.android.settings /system/app/Settings.apk
4 1000 com.bn.app.crypto.server /system/app/CryptoServer.apk
4 1000 com.bn.authentication.svc /system/app/BnAuthenticationService.apk
4 1000 com.bn.demomode /system/app/DemoMode.apk
4 1000 com.bn.devicemanager /system/app/DeviceManager.apk
4 1000 com.bn.nook.quickstart /system/app/QuickStartActivity.apk
4 1000 com.bn.syschecksum /system/app/SysChecksum.apk
4 1000 com.bn.waveformdownloader.svc /system/app/WaveformDownloader.apk
4 10005 com.android.certinstaller /system/app/CertInstaller.apk
4 10009 com.android.packageinstaller /system/app/PackageInstaller.apk
4 1001 com.android.phone /system/app/Phone.apk
4 1001 com.android.providers.telephony /system/app/TelephonyProvider.apk
4 10011 android.tts /system/app/TtsService.apk
5 10000 com.android.gallery /system/app/Gallery.apk
5 10000 com.android.providers.downloads /system/app/DownloadProvider.apk
5 10000 com.android.providers.drm /system/app/DrmProvider.apk
5 10000 com.android.providers.media /system/app/MediaProvider.apk
5 10000 com.bn.nook.accessories /system/app/Accessories.apk
5 10000 com.bn.nook.affiledownloadservice /system/app/AFfileDownloadService.apk
5 10000 com.bn.nook.cloud.service /system/app/CloudService.apk
5 10000 com.bn.nook.community /system/app/NookCommunity.apk
5 10000 com.bn.nook.dadmin /system/app/DownloadAdmin.apk
5 10000 com.bn.nook.home /system/app/Home.apk
5 10000 com.bn.nook.library /system/app/Library.apk
5 10000 com.bn.nook.reader.activities /system/app/Reader.apk
5 10000 com.bn.nook.shop /system/app/Shop.apk
5 10000 com.bn.nook.social /system/app/Social.apk
6 10025 com.andoku.two.free /data/app/com.andoku.two.free.apk
7 10028 org.connectbot /data/app/org.connectbot.apk
8 10003 com.bn.cloud.svc /system/app/BnCloudRequestSvc.apk
8 10004 com.android.browser /system/app/Browser.apk
8 10006 com.bn.deviceregistrator /system/app/DeviceRegistrator.apk
8 10007 com.android.htmlviewer /system/app/HTMLViewer.apk
8 10008 com.android.music /system/app/Music.apk
8 10010 com.svox.pico /system/app/PicoTts.apk
9 10020 com.benhirashima.nookcolorsettings /system/app/NookColorTools.apk
ApokrifX said:
Don’t know how to map sharedUserId to android user yet.
Click to expand...
Click to collapse
Need help with this one... :crying:

Yes, the cert indexes are the same for all the things that are signed with the same signature,
but they can even be different in some cases.
All my the apps I wrote and signed with my own key are 0.
All of the system that I signed with my own key are 1.
All of the other cert indexes go from 2 to 8.
The problem is, these are indexes into something and I don't know what/where that is.
When you change a signature, you have to change the something.
When you change signatures in most cases, the system shrugs and rebuilds packages,xml
It's mostly changing the signature on framework-res.apk (name="android") that causes the biggest problems.
For another perspective on the whole package management, try this:
Code:
dumpsys package > /sdcard/package.txt
(This generates a lot of text, hence the redirect.)

Renate NST said:
Android only mentions the single debug key in their documentation.
Click to expand...
Click to collapse
URL?
Renate NST said:
The emulator apks are signed with an Android signature, but not the same as the debug key.
Click to expand...
Click to collapse
Compared few keys (Subject Key Identifiers) out of curiosity:
Android keys
Code:
testkey 48:59:00:56:3D:27:2C:46:AE:11:86:05:A4:74:19:AC:09:CA:8C:11
shared CB:4C:7E:2C:DB:B3:F0:AD:A9:8D:AB:79:96:8D:17:2E:9D:BB:1E:D1
platform 4F:E4:A0:B3:DD:9C:BA:29:F7:1D:72:87:C4:E7:C3:8F:20:86:C2:99
media CA:29:3C:AA:8B:C0:ED:3E:54:2E:EF:42:05:A2:BF:F2:B5:7E:4D:75
NC2.1 EMU
Code:
Browser 48:59:00:56:3D:27:2C:46:AE:11:86:05:A4:74:19:AC:09:CA:8C:11
LatinIME CB:4C:7E:2C:DB:B3:F0:AD:A9:8D:AB:79:96:8D:17:2E:9D:BB:1E:D1
framework-res 4F:E4:A0:B3:DD:9C:BA:29:F7:1D:72:87:C4:E7:C3:8F:20:86:C2:99
MediaProvider CA:29:3C:AA:8B:C0:ED:3E:54:2E:EF:42:05:A2:BF:F2:B5:7E:4D:75
---------- Post added at 10:13 PM ---------- Previous post was at 10:07 PM ----------
Renate NST said:
Yes, the cert indexes are the same for all the things that are signed with the same signature,
but they can even be different in some cases.
All my the apps I wrote and signed with my own key are 0.
All of the system that I signed with my own key are 1.
All of the other cert indexes go from 2 to 8.
The problem is, these are indexes into something and I don't know what/where that is.
Click to expand...
Click to collapse
Ok. Just to make sure, we are on same page again:
When cert mentioned 1st time, it got encoded right into packages.xml key="3082...9308a"
Next time it used, it's referenced by index.
Code:
<package name="com.google.android.providers.talk" codePath="/system/app/TalkProvider.apk" system="true" ts="1292347460000" version="7" sharedUserId="10019">
<sigs count="1">
<cert index="0" key="3082...9308a" />
...
<package name="com.google.android.googleapps" codePath="/system/app/GoogleApps.apk" system="true" ts="1292347460000" version="130" sharedUserId="10019">
<sigs count="1">
<cert index="0" />
Do you mean something else?
---------- Post added at 10:16 PM ---------- Previous post was at 10:13 PM ----------
Renate NST said:
Yes, the cert indexes are the same for all the things that are signed with the same signature,
but they can even be different in some cases.
All my the apps I wrote and signed with my own key are 0.
All of the system that I signed with my own key are 1.
Click to expand...
Click to collapse
Interesting...
Could you extract CERT.RSA from "the app" and "the system app" and compare, please?
---------- Post added at 10:22 PM ---------- Previous post was at 10:16 PM ----------
[/COLOR]
Renate NST said:
For another perspective on the whole package management, try this:
Code:
dumpsys package > /sdcard/package.txt
Click to expand...
Click to collapse
How do we map:
Code:
Package [com.asksven.betterbatterystats] (49ea9250):
userId=10030 gids=[1015, 3003]
to names we see with ls -l

ApokrifX said:
URL?
When cert mentioned 1st time, it got encoded right into packages.xml key="3082...9308a"
Next time it used, it's referenced by index.
Click to expand...
Click to collapse
Yup, it looks like you are 100% correct.
FWR signed signed with my key is different than an app signed with my key.
They are the same, except for the last 256 bytes which are different.
As you can see, the keys in package.xml are of different lengths
and at least in the cases that I checked are shorter than length(cert)-256 even.
Moreover the end of the keys in packages.xml don't agree with the same position.
http://developer.android.com/tools/publishing/app-signing.html
The SDK tools create the debug keystore/key with predetermined names/passwords:
Keystore name: "debug.keystore"
Keystore password: "android"
Key alias: "androiddebugkey"
Key password: "android"
CN: "CN=Android Debug,O=Android,C=US"
Click to expand...
Click to collapse
The point is not that this single key is documented, the point is that the others are not.

Renate NST said:
FWR signed signed with my key is different than an app signed with my key.
They are the same, except for the last 256 bytes which are different.
As you can see, the keys in package.xml are of different lengths
and at least in the cases that I checked are shorter than length(cert)-256 even.
Click to expand...
Click to collapse
Right.
Could you compare certs "X509v3 Subject Key Identifier", please?
Renate NST said:
Moreover the end of the keys in packages.xml don't agree with the same position.
Click to expand...
Click to collapse
I’m not sure, I get this...
Renate NST said:
The point is not that this single key is documented, the point is that the others are not.
Click to expand...
Click to collapse
---------- Post added at 11:15 PM ---------- Previous post was at 11:01 PM ----------
Looks like in certs in packages.xml are stored in pkcs8 hex format:
shared.pk8
Code:
0000000000: 30 82 04 BE 02 01 00 30 │ 0D 06 09 2A 86 48 86 F7
0000000010: 0D 01 01 01 05 00 04 82 │ 04 A8 30 82 04 A4 02 01
0000000020: 00 02 82 01 01 00 C8 C2 │ DB FD 09 4A 2D F4 5C 3F
0000000030: F1 A3 2E D2 18 05 EC 72 │ FC 58 D0 17 97 1B D0 F6
packages.xml
Code:
<cert index="2" key="3082...b2db" />
They can be easily dumped from packages.xml right into pkcs8 format, no need to get them from packages.

I know practically nothing about signing and certs specifically.
Taking this as a black box question:
Given a signed package, extract the cert with a zip tool,
how do you convert that data into something to write into packages.xml?
Yes, all the ASCII text is in both of these but the cert in the apk and the cert in packages are wildly different.
Yes, you could make a project of this and delve into the Android code to see where it all comes from but the effort seems excessive.
We know that if you delete packages.xml entirely it will get rebuilt (although not with the same non-shared ids as before).
Why not try just deleting all the certs and leaving the rest of it alone?

Renate NST said:
I know practically nothing about signing and certs specifically.
Taking this as a black box question:
Given a signed package, extract the cert with a zip tool,
how do you convert that data into something to write into packages.xml?
Click to expand...
Click to collapse
I didn’t do this part yet.
I guess, a bit fiddling with openssl or keytool will do just fine.
Renate NST said:
Yes, all the ASCII text is in both of these but the cert in the apk and the cert in packages are wildly different.
Click to expand...
Click to collapse
If you post both (from packages.xml), I’ll decrypt them.
I’m pretty sure, they are different.
Renate NST said:
Yes, you could make a project of this and delve into the Android code to see where it all comes from but the effort seems excessive.
Click to expand...
Click to collapse
Yep. There is no point.
Renate NST said:
We know that if you delete packages.xml entirely it will get rebuilt (although not with the same non-shared ids as before).
Why not try just deleting all the certs and leaving the rest of it alone?
Click to expand...
Click to collapse
I wrote already, what might be different if you do it.
IMO, just patching it might be safer...
BTW: I decoded certs from packages.xml - there 4 different ones from B&N there.

ApokrifX said:
I decoded certs from packages.xml - there 4 different ones from B&N there.
Click to expand...
Click to collapse
I still don't know what the tool is or how it operates.
I'm not saying that what is packed in an APK is different in substance from the cert in packages.xml,
I'm just saying that they are not trivially binary convertible from one to another.
If you just delete packages.xml you can either fix the non-shared user ids in packages
or fix the owners for /data/data directories.
I already have an auditing tool for resolving such user id discrepancies
and finding orphaned /data/data directories for apps that were deleted and not uninstalled.
It doesn't do anything, but it reports it so that you can.

Renate NST said:
I still don't know what the tool is or how it operates.
I'm not saying that what is packed in an APK is different in substance from the cert in packages.xml,
I'm just saying that they are not trivially binary convertible from one to another.
Click to expand...
Click to collapse
I dunno, they are trivially convertible, try for yourself:
Unzip CERT.RSA from stock Reader.apk
Obviously (or not), CERT.RSA is pkcs7 and certs in packages.xml are hex strings x509
Let’s convert pkcs7 -> x509
Code:
openssl pkcs7 -inform DER -in CERT.RSA -out CERT.PEM -print_certs
openssl x509 -inform PEM -in CERT.PEM -outform DER -out CERT.x509.DER
Now open CERT.x509.DER is any hex editor:
Code:
0000000000: 30 82 04 96 30 82 03 7E │ A0 03 02 01 02 02 09 00
0000000010: CF 3F 93 2A 95 18 91 A5 │ 30 0D 06 09 2A 86 48 86
...
0000000480: BF 46 EB 99 2F F8 A8 9A │ 1F 66 2D 91 4F 0C 93 FE
0000000490: 44 7D 2F D0 C2 CC DC F7 │ 5E 84
And compare with packages.xml
Code:
<cert index="5" key="308204963082037ea003020102020900cf3f932a951891a5300d06092a864886
…
bf46eb992ff8a89a1f662d914f0c93fe447d2fd0c2ccdcf75e84" />
Renate NST said:
If you just delete packages.xml you can either fix the non-shared user ids in packages or fix the owners for /data/data directories.
I already have an auditing tool for resolving such user id discrepancies
and finding orphaned /data/data directories for apps that were deleted and not uninstalled.
It doesn't do anything, but it reports it so that you can.
Click to expand...
Click to collapse
What about this case:
ApokrifX said:
I dunno if it recreates UserID properly.
I.e. you have apps A, B, C installed they got UserIDs 10001, 10002 & 10003 respectively.
Then you uninstall A & B and delete delete packages.xml, would C get 10003 still?
Need to test it.
Click to expand...
Click to collapse
Do we need to do anything manually or deleting packages.xml will recreates everything properly?

Well, you seem to have a handle on all this.
I've never heard of pkcs7 or any of its friends.
Deleting packages.xml will result in the non-shared user ids to be assigned in order as the APKs are discovered by the PackageManager.
User ids are only used for file permissions on /data/data as far as I know.

Galaxy Watch Active2 SM-R820: Can't do OTAs after changing CSC

Hey guys, as the title says I successfully changed my GWA2 CSC from DBT to XAR, but ran into some problems. The watch boots up normally and I can use it, install apps from the Galaxy Store, etc. but I am stuck on version R820XXU1ASHF/Tizen 4.0.0.6. My phone shows a 30.68MB update to BTG1, and it can download it and start installing it but when it gets to 97%, the watch resets and boots up the old (ASHF) firmware. Moreover, Samsung Pay says that it can't start since I've "modified my watch," but I think this can be due to the very old firmware.
I've already tried changing to another CSC (both to AUT and the original EUR) and reflashing the ASHF firmware but to no avail. I originally came from R820XXU1BTA1 but I can't find that anywhere.
What can I do to fix this? I've also found the firmware files on some paid sites, and I'd pay since it's nothing significant, but I'm really not sure if those are real. Has anyone here bought firmware from them? Can anyone set me up with anything even one version newer than what I have? I've been searching for hours but I seem to have hit a dead end.

I bought a firmware on Fullstockfirmware and it works fine. I can host R820XXU1BTF3.zip if you need, but i'm new member, can't post link.
---------- Post added at 05:20 PM ---------- Previous post was at 05:17 PM ----------
https://drive.google.com/file/d/1LmJ9uJl644ePVnwabkGt_swDY961B_Ds/view?usp=drivesdk
Here is a link for the firmware

Noname761 said:
I bought a firmware on Fullstockfirmware and it works fine. I can host R820XXU1BTF3.zip if you need, but i'm new member, can't post link.
---------- Post added at 05:20 PM ---------- Previous post was at 05:17 PM ----------
https://drive.google.com/file/d/1LmJ9uJl644ePVnwabkGt_swDY961B_Ds/view?usp=drivesdk
Here is a link for the firmware
Click to expand...
Click to collapse
Thank you so much! You're a godsend! I flashed this in a heartbeat.
The update to BTG1 and SPay still don't work, but BTF1 is a way better point to be stuck on.
Plus, I checked my Knox bit, and it is not set. Maybe I messed something up (file permissions, line terminators, etc.) in /csa...

with this firmware i have samsung pay, but i can't test it because my bank is not supported. and the ecg and blood pressure works with 23.tpk and shm caranava. oh and nothing for the firmware. it's normal to share on a sharing forum ?

Noname761 said:
with this firmware i have samsung pay, but i can't test it because my bank is not supported. and the ecg and blood pressure works with 23.tpk and shm caranava. oh and nothing for the firmware. it's normal to share on a sharing forum
Click to expand...
Click to collapse
Could you by any chance give me the output of the following command? You don't need the combination firmware or root to run it.
Code:
sdb shell "ls -l /csa/csc/csc-active-customer.inf /csa/imei/prodcode.dat && hexdump -C /csa/csc/csc-active-customer.inf && hexdump -C /csa/imei/prodcode.dat"
My output looks like this:
Code:
-rwxrwxr-x 1 root system_share 3 Aug 12 10:27 /csa/csc/csc-active-customer.inf
-rw-rw-r-- 1 root system_share 14 Aug 11 15:23 /csa/imei/prodcode.dat
00000000 58 41 52 |XAR|
00000003
00000000 53 4d 2d 52 38 32 30 4e 5a 4b 41 58 41 52 |SM-R820NZKAXAR|
0000000e

here is mine
Code:
-rwxrwxr-x 1 root system_share 3 Aug 8 19:26 /csa/csc/csc-active-customer.inf
-rw-rw-r-- 1 root system_share 14 Aug 7 07:53 /csa/imei/prodcode.dat
00000000 58 45 46 |XEF|
00000003
00000000 53 4d 2d 52 38 32 30 4e 5a 53 41 58 45 46 |SM-R820NZSAXEF|
0000000e

Noname761 said:
here is mine
Code:
-rwxrwxr-x 1 root system_share 3 Aug 8 19:26 /csa/csc/csc-active-customer.inf
-rw-rw-r-- 1 root system_share 14 Aug 7 07:53 /csa/imei/prodcode.dat
00000000 58 45 46 |XEF|
00000003
00000000 53 4d 2d 52 38 32 30 4e 5a 53 41 58 45 46 |SM-R820NZSAXEF|
0000000e
Click to expand...
Click to collapse
Thanks for all your help, alas I can't find what's wrong with my watch...

before updating, i used the combination firmware to change my CSC and then i flash a stock firmware. I made the updates with wearable and I finally flash the version 4.0.0.8.

IMHO this is Rollback Prevention crap.. of Bootloader sboot.bin...
If Firmware is lower... Alphabet knowledge and count from 0 - 10 is enough skills...
Additional Infos can be taken from here:
http://fota-cloud-dn.ospserver.net/firmware/XAR/SM-R820/version.xml
I see ASHF... and you confirmed it fount FOTA delta package... :good: :good:
Now check Bootloader Version...
Code:
sdb shell
Code:
cat /proc/cmdline
To bypass you need same or higher Firmware...
Post result of Command...
And I could try to help you...
IMHO BTF3 is not valid FOTA base in XAR chain...
BTD3 or something like this was before on XAR...
BTG1 not leaked yet... otherwise we would do this.
Best Regards

adfree said:
IMHO this is Rollback Prevention crap.. of Bootloader sboot.bin...
If Firmware is lower... Alphabet knowledge and count from 0 - 10 is enough skills...
Additional Infos can be taken from here:
http://fota-cloud-dn.ospserver.net/firmware/XAR/SM-R820/version.xml
I see ASHF... and you confirmed it fount FOTA delta package... :good: :good:
Now check Bootloader Version...
Code:
sdb shell
Code:
cat /proc/cmdline
To bypass you need same or higher Firmware...
Post result of Command...
And I could try to help you...
IMHO BTF3 is not valid FOTA base in XAR chain...
BTD3 or something like this was before on XAR...
BTG1 not leaked yet... otherwise we would do this.
Best Regards
Click to expand...
Click to collapse
Code:
sh-3.2$ cat /proc/cmdline
console=ram loglevel=4 bootmode=ramdisk root=/dev/ram0 rw model=SM-R820 boot_ver=R820XXU1BTA1 hw_rev=05 sec_debug.enable=0 sec_debug.enable_user=0 tizenboot.sec_atd.tty=/dev/ttySAC0 tizenboot.emmc_checksum=0 tizenboot.dram_info=01,06,00,0.75G tizenboot.log=0x9b010000,0x200000,0x7f309,0x7ff90 tizenboot.boottime=1230ms tizenboot.sales_code=XAR warrantybit=0 sec_debug.bin=N lcdtype=0x402484 ess_setup=0x9b000000 [email protected] [email protected] DynSysLog=0 uart_sel=AP pmic_info=11 oops=panic [email protected] sec_debug.chipidfail_cnt=0 sec_debug.lpitimeout_cnt=0 sec_debug.cache_err_cnt=0 sec_debug.lpddr4_size=0.75 tizenboot.recovery_offset=1056512 tizenboot.carrierid_offset=1049156 tizenboot.carrierid= sec_debug.reset_reason=7 sec_debug.pwroffsrc=0x0 sec_debug.pwronsrc=0x8 sec_debug.rst_stat=0x20000000 tizenboot.verified_kern=1 tizenboot.fota_bl_status=none
I also found something interesting in /var/log/last_update.log which I will also attach to this post
Code:
UA/ERROR(SS_IMGVerfiyPartition) SS_IMGVerfiyPartition - SHA mismatch with SRC [/dev/mmcblk0p7] Expected [ffa4a910] Actual [ffa4a938]
UA/ERROR(SS_SetUpgradeState) FAILED to upgrade Cause:[0xd19]
I have pulled the delta.tar from the device and it seems that mmcblk0p7 is a ramdisk. I thought I'd replace the SHA value and pull the ole switcharoo but I can't find it anywhere

Code:
boot_ver=R820XXU1BTA1
This is the Knockout...
FOTA selfcheck detect that Bootloader not valid for ASHF Firmware...
Valid in case of FOTA crap...
BTA1 is inside FOTA chain of XAR CSC aka Sales Code:
http://fota-cloud-dn.ospserver.net/firmware/XAR/SM-R820/version.xml
Code:
R820XXU1BTA1/R820OXA1BTA1
Easiest way IMHO to flash whole BTA1 Firmware...
Best Regards

Thanks, I'll see if I can get my hands on that version...

@g511
Please check your Private Message... I sent you PM...
Best Regards

Hy guys.
Anyone can help me. I changed CSC and Samsung pay now is on the watch. The problem are two:
1- Samsung doesn't work because "the watch is modified"
2- doesn't work the upgrade. I download the update but doesn't install
Searching for a solution.
Thanks

@stampatori
Please, it is more helpfull if you give FULL details...
MINIMUM to know Model Name... Nobody here have Crystal Ball...
SM-R820?
Or LTE device like SM-R825F?
Or?
Best Regards

@adfree
Sorry.....?
My watch is a GWA2
SM-R820
Tizen 4.0.0.6
Firmware R820XXU1ASHF

@g511 search "techno proz change csc on watch active 2" on YOUTUBE and just follow. 100% works ! I did it 3 days ago and evrything is perfect !

Hello
I have the same problem with my NEW active 2 watch, I don´t know why it is happening, because my watch is NEW. I found this log in /opt/var/log/last_update.log
Code:
UA/(deleteNode): There is only one node. The list can't be made empty UA/ERROR(SS_FSVerifyNode) SS_FSVerifyNode - SHA mismatch with SRC - PATH [system/opt/system/csc/preconfig/TCE/usr/network/.delta_opername.db-journal] Expected [fff7d41c] Actual [fff7d430]
UA/ERROR(SS_SetUpgradeState) FAILED to upgrade Cause:[0xd15]
UA/ERROR(SS_AppendNode) Bad Nodes, Failed to pass verification - [Delta Path - /opt/usr/data/fota/save/delta.tar][OldPath - system/opt/system/csc/preconfig/TCE/usr/network/.delta_opername.db-journal] [NewPath - system/opt/system/csc/preconfig/TCE/usr/network/.delta_opername.db-journal]
UA/(tar_free_cfg_table): Free TAR CFG TABLE
UA/ERROR(SS_FSVerifyPartition) FS Verification Failed PartIndex: [4]
UA/(SS_FSClearNodes): Free Nodes idx=4
UA/(update_all): CSC verify failUA/(save_cause): save_cause entered, 0xd15
UA/(print_error_cause): The update failed because data was corrupted during update of device.UA/(save_cause): save_cause leaved!
UA/(main): [update_all ret=64537]
UA/(main): Result=64537
UA/(save_result): save_result entered, result=0xfc19
UA/(save_result): save_result leaved!
this is my /proc/cmdline
Code:
console=ram loglevel=4 bootmode=ramdisk root=/dev/ram0 rw model=SM-R825FS boot_ver=R825FXXU1ASJ3 hw_rev=05 sec_debug.enable=0 sec_debug.enable_user=0 tizenboot.sec_atd.tty=/dev/ttySAC0 tizenboot.emmc_checksum=0 tizenboot.dram_info=01,06,00,1.50G tizenboot.log=0x9b010000,0x200000,0x0,0xaba tizenboot.boottime=2140ms tizenboot.sales_code=COM warrantybit=0 sec_debug.bin=N lcdtype=0x402484 ess_setup=0x9b000000 [email protected] [email protected] DynSysLog=0 uart_sel=AP pmic_info=11 oops=panic [email protected] sec_debug.chipidfail_cnt=0 sec_debug.lpitimeout_cnt=0 sec_debug.cache_err_cnt=0 sec_debug.lpddr4_size=1.50 tizenboot.recovery_offset=1056512 tizenboot.carrierid_offset=1049156 tizenboot.carrierid= sec_debug.reset_reason=9 sec_debug.pwroffsrc=0x10 sec_debug.pwronsrc=0x1 sec_debug.rst_stat=0x10000 tizenboot.cp_reserved_mem=off tizenboot.verified_kern=1 tizenboot.fota_bl_status=none
this is my csc-active-customer.inf
Code:
sh-3.2$ hexdump -C /csa/csc/csc-active-customer.inf
00000000 43 4f 4d |COM|
00000003
this is my prodcode.dat
Code:
sh-3.2$ hexdump -C /csa/imei/prodcode.dat
00000000 53 4d 2d 52 38 32 35 46 5a 4b 41 43 4f 4d |SM-R825FZKACOM|
0000000e
Do you know why i can not update my watch ?
Thanks !

@andrs1294
All i can see for now is something mismatch with CSC... but not fully understand...
COM
http://fota-cloud-dn.ospserver.net/firmware/COM/SM-R825F/version.xml
TCE
http://fota-cloud-dn.ospserver.net/firmware/TCE/SM-R825F/version.xml
Both CSC / Sales Code are in same package... region Code:
OWO...
Code:
R825FXXU1ATA1/R825F[B]OWO[/B]1ATA1/R825FXXU1ATA1
I have only OXA and OLB package with ATA1 Firmware for netOdin...
Need some more time for investigation...
Found only 1 OWO package...:
Code:
R825FXXU1[B]ASI5[/B]
Best Regards

adfree said:
@andrs1294
All i can see for now is something mismatch with CSC... but not fully understand...
Both CSC / Sales Code are in same package... region Code:
OWO...
Code:
R825FXXU1ATA1/R825F[B]OWO[/B]1ATA1/R825FXXU1ATA1
I have only OXA and OLB package with ATA1 Firmware for netOdin...
Need some more time for investigation...
Found only 1 OWO package...:
Code:
R825FXXU1[B]ASI5[/B]
Best Regards
Click to expand...
Click to collapse
Thanks @adfree for your response. Here you can find my investigation:
The error message is:
Code:
There is only one node. The list can't be made empty UA/ERROR(SS_FSVerifyNode) SS_FSVerifyNode - SHA mismatch with SRC - PATH [system/opt/system/csc/preconfig/TCE/usr/network/.delta_opername.db-journal] Expected [ff9feddc] Actual [ff9fedf0]
So, I search about SS_FSVerifyNode code on internet, I found that that code is part of libtota-1.2.2-25.1.src.rpm.
Code:
...
if (SS_LoadFile(path, &source_file) == 0) {
if (memcmp(source_file.sha1, source_sha1, SHA_DIGEST_SIZE) != 0) {
SS_Free(source_file.data);
unsigned char actualShaBuffer[41] = { 0, };
hex_digest(source_file.sha1, actualShaBuffer, SHA_DIGEST_SIZE);
LOGE("SS_FSVerifyNode - SHA mismatch with SRC - PATH [%s] Expected [%s] Actual [%s]\n",
path, sha1src, actualShaBuffer);
SS_SetUpgradeState(E_SS_FSSRCCURRUPTED); // E_SS_FSSRCCURRUPTED (0xD15) /*Could NOT update FS as SRC seems to be corrupted */
return E_SS_FAILURE;
}
}
...
It is calculating SHA1 of the file system/opt/system/csc/preconfig/TCE/usr/network/.delta_opername.db-journal and then it compares with SHA inside the csc.img/CSC.txt inside the delta.tar file. Part of the content of the csc.img/CSC.txt is
Code:
DIFF:REG:system/opt/system/csc/preconfig/TCE/usr/network/.delta_opername.db-journal:system/opt/system/csc/preconfig/TCE/usr/network/.delta_opername.db-journal:[B]a4b298726c564ea01c9f21815c864e253493c269[/B]:f185bc963d1e61e372da5f1cda21e69a0cebf3ca:diff4_.delta_opername.db-journal_CSC.delta
PaTcHCoUnT:4 0 0 0 0 0
So I think my delta_opername.db-journal was edited in some moment, So the sha resumen doesnt match.