***THIS POST IS NOT COMPLETE, I WILL UPDATE MORE LATER***
First, an introduction:
The Image Update system allows the OEM (us! ) to issue updates to a "Live" filesystem - without disrupting user data. This allows, for example, a buggy driver to be updated after the phone has been shipped, or a software package to be updated to the latest version, with minimal knowledge on the user's part. The system validates all updates against an internal list of certificates, and refuses the update if a match is not obtained. This system can also be used to deploy entirely new software to the device (such as support for another locale, input method editor, application support for a new feature the carrier is rolling out, etc.)
Potential usage scenarios for this system
A central server could be maintained for all SYS/OEM updates - each ROM Chef would need to maintain a list of original packages, any updated package(s), and download URL's for each updated package. The user would then receive these updates through the built-in AutoUpdate facility in Windows Mobile, which can check periodically, or on-demand. Each Chef could maintain seperate download servers from the update server to minimize server load.
Alternatively, a chef could provide .cab.pkg updates in his or her ROM thread, on their own web site, etc., and the user could download these and install them at will. These packages can optionally be authenticated to be coming from the Chef, if the Chef wants to ensure updates are coming from him only. A public certificate could also be used to allow users to issue updates as well.
The more technical Summary
Image Update allows an OEM to issue updates to the OEM's, XIP, SYS, (possibly) Radio, or any combination of these. The update can be pushed to the user via a specially formatted SMS or by manual execution. There are at least 2 levels of certificate checking involved in the process, I believe against \SYS\Metadata\DefaultCerts.dat. The system reboots into the ULDR to apply the update, because the filesystem cannot be modified while actively mounted. The ULDR provides a minimal operating enviornment to facilitate this.
How does a Chef need to prepare a ROM for Image Updating?
The Chef would need to use a ROM Kitchen that leaves the .dsm and .rgu file structure intact (i.e. an "unprotected" ROM) - All .dsm's in this ROM would need to be properly formatted with Package Name, versioning info, etc. during the cooking process, in order to facilitate version checking, etc. Each .dsm would need to be signed with a certificate the Chef would use to validate the update as coming from him or her. (Alternatively a public certificate could be used like SDKCerts if the Chef chooses not to maintain direct control over updates, and allow other users to create updates as well)
The Chef also needs to ensure not to Reduce the ULDR partition or remove it; this would cripple the Update Loader and prevent the Image Update system from functioning.
The .cab.pkg format
At the root of a package, the .dsm defines the Package structure (all files, registry entries, etc.) It contains version info, certificates, and other data. A ROM consists of a number of these packages, in an area of flash memory that is not user-writable. When someone wants to issue an update using the ImageUpdate system, they create a matching .dsm, same guid, with a newer version number, same internal package name, same processor ID, os version, etc., there is also a flag that can be marked as an update package or a new package - in this .dsm they define the files that will make up the new, updated package. Here they can add or remove files. One of the files defined by the package is optionally an .rgu, and this defines the registry entries associated with the package. Also optionally included is a provxml to facilitate file operations (copy/replace/delete/rename/etc.) and further registry or metabase operations. This collection of files is rolled up into a ".cab.pkg" archive by a program like cabarc. Once in a pkg.cab format, the package is signed.
A .cab.pkg is referred to as a "Canonical Package" and multiple Canonical Packages can be rolled up into a single "Super Package" to facilitate updating multiple Packages at the same time.
Inside this .cab.pkg is where the "MNGE" file format comes in to play. Essentially, this format is Microsoft's way of storing an XIP Module in the filesystem. (Their equivalent of our imageinfo.bin + s000, s001, etc.) - The "MNGE" format is simply a 4-byte MNGE header, followed by the imageinfo.bin, followed by the s00x sections attached to the end. When the Image Update system processes a file in this MNGE format, it is added to the IMGFS as an XIP Module.
Deploying a .cab.pkg to a device
Now there are several ways to deploy this package to the device, the primary way is an OMA-DM SMS message, which triggers the Image Update system to initiate a download from a server defined in the SMS message. The server can be either a normal HTTP server or a secure HTTPS server. The Image Update system will prompt the user to plug in the usb cable to download over the users home connection if the user has not already authorized the Auto Update system to utilize their GPRS data connection.
Another way for a .cab.pkg update to be pushed to the system is simply through a file copy operation, be it ActiveSync, SD Card, Bluetooth, or otherwise. Once on the device, the .cab.pkg is executed by the user the same way a .cab would be.
The Update Agent
Initiated by either a completed download from push SMS, or user-executed, the "Update Agent" program (which is part of the \SYS\FWUPDATE Package) attempts to validate the Certificates, Package dependancies, and other info contained in the .cab.pkg and .dsm. Once validated, the "Update Agent" sets a flag that the bootloader reads, the flag is a boolean, off = boot into normal OS, on = boot into ULDR - so then the system reboots, the flag is read, and you load into...
The Update Loader
The "Update Loader" or "ULDR" which is a minimal kernel configuration, that provides just enough driver support to display info on screen, respond to user input, and read/write from the internal flash (NAND or NOR)
From here the ULDR does further validation on the .cab.pkg, and applies it to the filesystem. If there are any modules in the package it dynamically relocates the memory map to make sure there are no overlaps. This is why it's important that reloc's not be removed from your ROM - ULDR will fail in this case.
The End Result
Once the ULDR has completed updating, the device is again rebooted, back into the full system, where the now-updated packages are now a part of the IMGFS, any updated files are processed (.rgu, .provxml, etc.) - The package is now a full part of the ROM.
The new .dsm replaces the old .dsm (along with the other files in the package) and now a future update will be checked against this new package.
If the update was pushed via OMA-DM SMS, or AutoUpdate, the device Pushes a notification to the OMA-DM server notifying it of the update status.
What's missing right now to implent the ImageUpdate system?
We need a Kitchen that's properly configured to allow us to create versioning info, proper package names, and insert this along with a certificate (or multiple certificates) into the .dsm's.
We also need the Kitchen to be able to modify \SYS\Metadata\DefaultCerts.dat with the certificates used, so that it passes authentication. Alternatively the authentication checking could be patched out. (this one is easily doable at build-time)
We need a program that can convert from a standard file to an MNGE format, so we can implement modules in our .cab.pkg's. (done it seems, thanks ervius!)
We (optionally) need a properly configured web server that supports HTTP/HTTPS, can communicate the proper xml configuration data, and can be updated with new packages by Chefs. (this one's a ways off)
We (optionally) need a program to convert from MNGE format to a standard file to facilitate extracting modules from .cab.pkg's. (working hard on that)
I've attached a .cab.pkg for NetCF2. Open up mscoree.dll in a hex editor, and check out the MNGE header. This file becomes a module once processed by the ImageUpdate system. Note that all the executables (.exe/.dll/.mui) that become modules contain this MNGE header. All executables that are inserted into ROM as files keep their normal MZ file header. The first major step here will be in being able to convert between MZ<-->MNGE freely NetCF2 is a well known package that can be found in any stock ROM, so with this we have a good baseline to work with.
http://rapidshare.com/files/238295848/netcf.cab.pkg
--Reserved--
More Technical Specifications
The basic ImageUpdate Layout consists of:
[IPL] -- [MBR] -- [ULDR] -- [NK] -- [IMGFS] -- [TFAT]
[IPL] is the "Initial Program Loader" that handles basic init functions and determines if control should be handed over to ULDR, or NK through a flag set by UpdateBin.exe - the IPL is not contained within a partition. The IPL is copied entirely to RAM and executed from there. IPL loads NK into RAM, and also handles any decompression of NK if it's required - some SmartPhone's ive seen use SRPX compression for the NK partition. Once NK is copied to RAM it is then executed. The IPL is handled seperately from the other parts of the operating system, and is not flashed during a normal update.
[MBR] is the "Master Boot Record" and contains partition tables for the below components - it points to NK so when IPL loads the MBR, control is handed over to NK. The MBR contains information on where each partition is located on the flash (memory address), the size of the partition(s), and the type of each partition. The MBR is referenced from many components on the device such as IPL and ULDR in order to facilitate handoff of control between ULDR and NK. The MBR also serves as a boundry between the IPL which is not part of the regular partition structure, and the rest of the flash, which is part of the partitioning structure.
[ULDR] is the "Update Loader" and provides a basic WinMo system so that file operations can be done on the IMGFS partition while it's unmounted. The Update Loader is even able to update itself - during operation it is loaded entirely to RAM. On development workstations the ULDR supports a KITL connection, that can be used to load updates directly from the "Release" folder. It seems it may be possible through this method to flash a new image to the device, possibly opening up the ability to flash to devices that have not yet been flashed with "HardSPL"
[NK] is the "Kernel Partition" - or what we know as xip.bin - This component is updatable by ImageUpdate, and has a pre-defined "free space buffer" with room to grow, which defaults to 512KB. This partition holds only the kernel and drivers necessary to bring up the rest of the filesystem, from which the rest of WinMo is loaded. The Kernel Partition uses the same "Package" format as the IMGFS and is updatable in the same manner.
[IMGFS] is the "System Partition" - running the Image Update filesystem. This component is updatable by ImageUpdate, and has a pre-defined "free space buffer" with room to grow, which defaults to 9.5MB. The IMGFS uses the "Package" format to further split its components.
[TFAT] is the "Transaction Safe FAT File System" which is where all user-writable data goes.
In most Device Designs, there's a single NOR or NAND chip used for flash. This is important as due to the typical layout above, both NK and IMGFS must have a pre-defined amount of free space - because TFAT is the last partition on the drive, and cannot be shifted once flashed to the device. It's possible for the partition layout to be setup differently (Partitions in different order) to help alleviate that problem. The ImageUpdate system would really shine on a device with 2 flash chips, a NOR chip dedicated to the ImageUpdate partition and a NAND chip dedicated to the TFAT, but no OEM has created such a design yet.
Packages
Package Types
There are 3 different types of packages, Canonical, Update, and Super.
Canonical contains the entire contents of the package. It is used for a first-time package install, and if there are any major updates to be issued that would require the complete package. The file extension is .cab.pkg
Update contains a binary delta between a package already on the device, and the updated version of that package. In this manner the limited space is conserved (i.e. if a package change was a simple registry entry - no need to replace the 5mb of .dll and .exe in that package, just alter the .rgu with the new data. These packages are also referred to as "Delta" packages. The concept is similar to the unix implementation of Diff/Patch. The file extension is .cab.pku
Super contains a collection of update and/or canonical packages. This is very useful when you are attempting to bring in a new package that has dependencies on other packages - rather than reboot into ULDR for each individual package in the proper dependency order, they can all be introduced at once. Every package contained inside a super package is validated, and if one fails, the remaining valid updates may still be applied, as dependencies allow. A super package is simply an un-compressed .cab containing other packages, renamed to .cab.pks
The package layout itself is quite basic, it consists of a .dsm which contains all versioning info, association info, and dependency info. It also contains a list of all modules and files inside the package, and a certificate store of all approved certificates that will be allowed to update that package. Alongside the .dsm is an optional .rgu, which defines the registry settings associated with that package. Also optional is a .provxml file, which can be: mxip_[packagename]_[version].provxml, mxipcold_[packagename]_[version].provxml, or mxipupdate_[packagename]_[version].provxml. mxip and mxipcold are effectively treated the same, executed only on a cold boot. mxipupdate_ provxml's will be executed any time that package is updated, in addition to a cold boot - so if you are adding new .cab.pkgs and wish the .provxml to be executed immediately, it would need to be mxipupdate. This may not be desirable in some cases, such as when the provxml might override a user preference - in that case you would only want it to run on a cold boot, in order to avoid "strange" behavior on the user's side of things.
There is a "shadow order" defined in the .dsm as well - this controls what "priority" .rgu's are compiled together into the device registry hive - a package that shadows another package will override any .rgu entries that shadowed package may contain. This is important to consider when utilizing .cab.pkgs in order to obtain your desired end registry. This shadow order also applies to provxmls inside the package - a package that shadows another package will override its provxml settings as well.
The user registry hive is always top-most in the shadow order (except in the case of an mxipupdate_ provxml) - so any changes to .rgu registry settings will not override a user-changed registry setting. (example: You had foobar set to 5 in your initial deployment. At some point after flashing to his/her device, the user modified the registry, changing the value of foobar to 6. Your new .cab.pkg contains an .rgu changing foobar to 7 - on device, foobar will remain set to 6, as the user registry is higher in the shadow order than the .rgu) - in the case of an mxipupdate_ provxml these will override user settings.
So theoretically if we get this working we can apply updates to ANY portion of a rom via the .cab.pkg system. To XIP,SYS,or OEM without flashing a new rom. Essentially with this system working we would never need to flash again unless a major corruption occured?
Ive always been confused as to why autoupdate was included in roms still. I guess this sheds some light on it. I know that several individuals have toyed with OTA updates in the past. This could make that and a whole lot more a reality.
It seems to me silly that we aren't exploiting the MS autoupdate feature already...
I have no idea how to get it to work, but I'd love to have it working!
Yes, the .cab.pkg is the key to this whole thing, I already know how to redirect the Windows Mobile Auto Update client to look at another server, and i've studied the connection, it's a simple HTTPS connection, WinMo sends the server a manifest of all the .dsm's contained within your ROM, with version info, then the server checks against it's internal list of packages, if it finds an update, it pushes a URL to the device, which then triggers a download (it requests you to plug in the activesync cable if you've not checked the box to 'use my data connection for updates') - once the .cab.pkg is downloaded, it's checked against the signatures on the system, once verified the system reboots into the ULDR, and the update is applied.
I've attached a NetCF2 package to my first post, I can't get it to deploy on my ROM (fails during validation step) but it contains the modules in MNGE format, if we can decipher that format there's a whole bunch of goodies that will become available...
Also these packages can even be used to update the radio rom, it seems. So essentially everything but the Bootloader/ULDR can be updated with .cab.pkgs. It even looks like we can resize existing packages (i.e. remove files or modules from the ROM entirely) - this is something we can't do at all right now without a flash!
Da_G said:
Yes, the .cab.pkg is the key to this whole thing, I already know how to redirect the Windows Mobile Auto Update client to look at another server, and i've studied the connection, it's a simple HTTPS connection, WinMo sends the server a manifest of all the .dsm's contained within your ROM, with version info, then the server checks against it's internal list of packages, if it finds an update, it pushes a URL to the device, which then triggers a download (it requests you to plug in the activesync cable if you've not checked the box to 'use my data connection for updates') - once the .cab.pkg is downloaded, it's checked against the signatures on the system, once verified the system reboots into the ULDR, and the update is applied.
I've attached a NetCF2 package to my first post, I can't get it to deploy on my ROM (fails during validation step) but it contains the modules in MNGE format, if we can decipher that format there's a whole bunch of goodies that will become available...
Click to expand...
Click to collapse
So in theory, there could be a central place for SYS/XIP packages, where as and when new XIP/SYS updates come out, they can be uploaded and pushed to every device?
Have you worked out how to create the cab.pkg files, or is the one you've attached one taken from platform builder?
Just a thought:
Could the MNGE headered files not be replaced by files from a converted module, thus getting around the problem of what the hell the MNGE format does? Sure, we lose the ability to have modules instead of files, but it does bring more immediate benefits to your findings...
EDIT:
The MNGE headered files are smaller than the MZ equivalents... Are they simply a compressed version?
Yep, not pushed though as that needs to be triggered via an OMA-DM SMS message, and it's not practical for someone to maintain a database of all our numbers for such a purpose.. but easily though settings - autoupdate
I am able to extract files from .cab.pkg with winrar and 7zip, not able to create them just yet.. working on that. This one came from a blue birdy.
The MNGE headered files could indeed be replaced by a converted module, but in this case, there's a different reason for needing to convert from MNGE -> MZ,
It appears to me as though the file size difference had to do with the PE executable headers that are missing..
Da_G said:
Yep, not pushed though as that needs to be triggered via an OMA-DM SMS message, and it's not practical for someone to maintain a database of all our numbers for such a purpose.. but easily though settings - autoupdate
I am able to extract files from .cab.pkg with winrar and 7zip, not able to create them just yet.. working on that. This one came from a blue birdy.
The MNGE headered files could indeed be replaced by a converted module, but in this case, there's a different reason for needing to convert from MNGE -> MZ,
It appears to me as though the file size difference had to do with the PE executable headers that are missing..
Click to expand...
Click to collapse
Oh right. It's not hard to just check for updates every so often.
I just ran Cab2OEM on the cab.pkg files, and it extracts fine. So cab.pkg files are just cab files in terms of compression.
Is that because there are more up to date MNGE file versions than the MZ equivalents?
Is it just a case of replacing the file headers? *opens up hex edit*
Yep. the compression is your typical cab compression. That's why winrar and 7zip can open 'em and extract, but they don't support adding (i imagine cabarc would...
Yes, there are more up to date MNGE file versions than the MZ equivalents.
Unfortunately it doesn't look quite as simple as a simple hex copypasta, not terribly much more difficult though.
Da_G said:
Yep. the compression is your typical cab compression. That's why winrar and 7zip can open 'em and extract, but they don't support adding (i imagine cabarc would...
Yes, there are more up to date MNGE file versions than the MZ equivalents.
Unfortunately it doesn't look quite as simple as a simple hex copypasta, not terribly much more difficult though.
Click to expand...
Click to collapse
Where are you getting your MNGE files from? Or does your blue birdy wish to remain anonymous?
EDIT:
Looking at the hex, apart from the file header, the main difference seems to be that the whitespace has been removed in the MNGE version...
l3v5y said:
Where are you getting your MNGE files from? Or does your blue birdy wish to remain anonymous?
Click to expand...
Click to collapse
Do you know what a "hint" is ? I'd say quit asking... If he wanted to say who the birdy was, he would. Thank you!
Da_G said:
Yep. the compression is your typical cab compression. That's why winrar and 7zip can open 'em and extract, but they don't support adding (i imagine cabarc would...
Yes, there are more up to date MNGE file versions than the MZ equivalents.
Unfortunately it doesn't look quite as simple as a simple hex copypasta, not terribly much more difficult though.
Click to expand...
Click to collapse
put cabarc.exe intoa folder, then create a new subfolder called "package", put inside all files you need for package (dsm and rgu also!)
open in cabarc.exe root folder a dos prompt and write:
cabarc.exe N new_pack.cab.
pkg package\*.*
some seconds and you'll have the .pkg file ready, but nothing I know on how to install by the rom!
Thanks for the input ervius!
Installing into the ROM is simple, .cab.pkg is treated similar to a .cab by Windows Mobile, simply copying to device and clicking on it in file explorer allows you to install - device will authenticate signature, then do some further checking (i think checking on current packages in device by .dsm) - then once validated reboot into ULDR to apply update.
ervius said:
put cabarc.exe intoa folder, then create a new subfolder called "package", put inside all files you need for package (dsm and rgu also!)
open in cabarc.exe root folder a dos prompt and write:
cabarc.exe N new_pack.cab.
pkg package\*.*
some seconds and you'll have the .pkg file ready, but nothing I know on how to install by the rom!
Click to expand...
Click to collapse
for posted netcf example, the optimum is:
cabarc.exe - 20 N new_pack.cab.
pkg package\*.*
so, header (-s 20 reserve space for sign!), is same size, but how and with , sign it, I don't know!
Da_G said:
Thanks for the input ervius!
Installing into the ROM is simple, .cab.pkg is treated similar to a .cab by Windows Mobile, simply copying to device and clicking on it in file explorer allows you to install - device will authenticate signature, then do some further checking (i think checking on current packages in device by .dsm) - then once validated reboot into ULDR to apply update.
Click to expand...
Click to collapse
shure, but now maybe we have to fight against right sign code!?!?
bye!
Yes, I think the ROM will need be cooked with additional certs, in \SYS\Metadata\DefaultCerts.dat - these appear to be the certs that are being checked against. So we can replace with SDKCerts, sign .cab.pkgs with that, should be good!
Da_G said:
Yes, I think the ROM will need be cooked with additional certs, in \SYS\Metadata\DefaultCerts.dat - these appear to be the certs that are being checked against. So we can replace with SDKCerts, sign .cab.pkgs with that, should be good!
Click to expand...
Click to collapse
ok, go to work then, I ', yet thiniing about oldstyle buildos with this all new features of visualkitchen without platformrebuilder if someone wants use oldstyle (maybe I'm at my goal!)
ervius said:
ok, go to work then, I ', yet thiniing about oldstyle buildos with this all new features of visualkitchen without platformrebuilder if someone wants use oldstyle (maybe I'm at my goal!)
Click to expand...
Click to collapse
Can we not look at removing the signing check the same way cmonex did for the kernel? Or is that the same signing check?
Just so I am getting this right? I could cook a ROM with a custom DaveShaw https update server IP and then provide automatic updates to my ROM, bug fixes, new build release, normal CABs, etc. all using Windows Update from my website??
That would be damn useful, no more re-flashes
Dave
probably to create MNGE from module folder we just need
copy /b imageinfo.bin + S000 + S001 + ... module.dll
and add MNGE header to the beginning of the file.
but maybe i am wrong and this will not work in all cases.
at least when i converted in this way dll module from 21725 to MNGE and compared it to the same file which was originally in MNGE format, there were 0 differences.
I'm interested in removing the network lock from my SGA2 GT-I8610 (Samsung codename "Codina"), and while I'm a total noob regarding Android development, I do have some clue about coding and tried a bit of Googling and searching these forums here.
First of all, two useful threads as linked from this post (which was erroneously posted in a similar thread about unlocking the Ace II X GT-S7560M):
Good starting point: Backing up NV data
Partition table
My personal findings:
My serial number -- without the trailing X -- is found 4 times in cspsa.bin, as produced by following the instructions found in the first Backing up NV data post.
I stumbled across this -- especially the parameters on line 59 -- and wondered if this was any helpful. Curiously, I haven't been able to find that very same file in GitHub repos (most are forks of https://github.com/dh-harald/android_kernel_samsung_codina) based on Samsung's open source released i8610 "Codina" kernel.
There is a mention of "pff2cspsa 0.0.2 Thu Apr 12 10:19:02 2012" in cspsa.bin. Googling "pff2cspsa" yields, among few other results, http://pastebin.com/ku0WHZ1B (posted only yesterday!), which seems to be some file related to the NovaThor U8500 chipset, upon which the Ace 2 is based. That file also seems to list a couple of other cspsa related tools, such as cspsalist etc. I wonder if those tools and/or their source code can be found somewhere.
Another good starting point might be investigating if SIM unlock techniques have been found for any other NovaThor U8500 SoC-based phones, e.g. the SGS3mini. I haven't done that thoroughly yet.
Few people have reported success unlocking their SGS3minis using the Service Mode menu, where they were able to disable a network lock option called "Perso SHA256", so maybe this means that the CSPSA is somehow SHA-256 hashed or encrypted.
Feel free to comment your thoughts and/or if any of this might be worth further investigation.
One thing I haven't really found through searching: are you guys using just a hex editor to analyze cspsa.bin or are there any better tools? (E.g. is it possible to mount that thing as a loopback device on Linux?)
(Sorry for posting this to the user section -- as this is my first post, I'm not yet allowed to post to the dev section.)
discovered aswell
I found your post when I did a keyword search for 'pff2cspsa'. I'm trying to goof with my RIL at the moment.
I've found it in my CSPSA partition.
There's also 'nwm2gdf' mentioned aswell which looks like a utility.
Googling that found me some XML dump of someones packages in what seems like a Gentoo distro. It looks like theres a package called 'cspsa-tools', because the 'cspsa-tools-debuginfo' package contains nwm2gdf.
That package looks interesting & i'm going to see if I can find it now.
This is also my first post. SIgned up just to reply
The SHA-256 clue is very interesting thanks I'm gonna investigate further.
---------- Post added at 07:45 AM ---------- Previous post was at 07:11 AM ----------
Okay I've found some info:
Turns out Ericsson (ie the company that once joined up with Sony to be SonyEricsson) once upon a time released packages to manage CSPSA things. They make the chipset that the galaxy s2/3 is built on (one of them anyway).
<summary>Crash Safe Parameter Storage Area (CSPSA)</summary>
<description>
CSPSA (Crash Safe Parameter Storage Area) is a user space library developed by ST-Ericsson that provides access to the CSPSA at the end of the flash/eMMC used for storing, for example, calibrated radio and security parameters. It utilizes a raw MTD device or eMMC block device via libc to manipulate the contents in the flash/eMMC memory. All concurrent users except one are blocked until access to the underlying storage device is accessible. The CSPSA implementation provides no security concept for the data stored. Focus is on crash safeness and compatibility with production tools.
Click to expand...
Click to collapse
Now because I'm a new user I can't post external links. The captcha is F---ing me off so I'm going to obfuscate the link below:
http colon slash slash download.meego.com/live/home:/imppa/Trunk/i586/
Download all the files which mention CSPSA (theres a few down below, libcspsa ones so page down).
Also, they are rpm. If you are using a debian based system, use the 'alien' tool to convert to .deb packages with the -d switch.
Also, roll 32-bit ONLY. Forget about it if you're running 64-bit linux.
There's a bunch of handy tools there. I'm yet to goof with them but we'll see how we go.
edit: doesn't seem to work. All I'm getting is 'ERROR: CSPSA area could not be opened. Binary file not valid' when I try em on the image files I made of my CSPSA partitions.
---------- Post added at 08:08 AM ---------- Previous post was at 07:45 AM ----------
weblordpepe said:
I found your post when I did a keyword search for 'pff2cspsa'. I'm trying to goof with my RIL at the moment.
I've found it in my CSPSA partition.
There's also 'nwm2gdf' mentioned aswell which looks like a utility.
Googling that found me some XML dump of someones packages in what seems like a Gentoo distro. It looks like theres a package called 'cspsa-tools', because the 'cspsa-tools-debuginfo' package contains nwm2gdf.
That package looks interesting & i'm going to see if I can find it now.
This is also my first post. SIgned up just to reply
The SHA-256 clue is very interesting thanks I'm gonna investigate further.
---------- Post added at 07:45 AM ---------- Previous post was at 07:11 AM ----------
Okay I've found some info:
Turns out Ericsson (ie the company that once joined up with Sony to be SonyEricsson) once upon a time released packages to manage CSPSA things. They make the chipset that the galaxy s2/3 is built on (one of them anyway).
Now because I'm a new user I can't post external links. The captcha is F---ing me off so I'm going to obfuscate the link below:
http colon slash slash download.meego.com/live/home:/imppa/Trunk/i586/
Download all the files which mention CSPSA (theres a few down below, libcspsa ones so page down).
Also, they are rpm. If you are using a debian based system, use the 'alien' tool to convert to .deb packages with the -d switch.
Also, roll 32-bit ONLY. Forget about it if you're running 64-bit linux.
There's a bunch of handy tools there. I'm yet to goof with them but we'll see how we go.
edit: doesn't seem to work. All I'm getting is 'ERROR: CSPSA area could not be opened. Binary file not valid' when I try em on the image files I made of my CSPSA partitions.
Click to expand...
Click to collapse
Okay what you need to do is figure out which partition is your CSPSA partition. lets say its mmcblk009.
Install all those CSPA packages
check /etc/cspsa.conf. it will point to a file under /dev. copy your binary partition into /dev and edit the file to match the filename of your partition image.
then run: cspsa-server. It will say something like 'CSPSA0 started'.
then type cspa-cmd and you'll get a prompt
type 'help' to see stuff. but you'll want to type 'open CSPSA0'
then you can type 'ls' to see the key/value pairs for things stored in your CSPSA image file.
Bought an unlock today
I bought a network unlock today, did another NV backup, and compared the before and after states of cspsa.bin.
Some of it has changed; but around the first occurrence of the serial number, the file is surprisingly almost identical to its previous state. Now what has changed is the range from offset 0x457 (after "FF 24 00") to (excluding) 0x477. I'll take a closer look soon if anything else looks suspicious, but this might be our SIM lock...
A couple of short notes
A couple of short notes on the CSPSA tools you found:
There are not only binary RPMs, but also two RPM source packages from which they're built (which contain a spec file and an upstream tarball) -- they're at download.meego.com/live/home:/imppa/Trunk/src/ . I successfully built the upstream sources on Ubuntu 13.10 (amd64), but I had to remove the -Werror parameter from the Makefiles first. I didn't have to install any of those tools to my system (i.e. no sudo required), but I have to admit that I haven't yet built the binaries from cspsa-tools that require cspsa.h.
You don't need to copy cspsa.bin to /dev -- you can specify its location in cspsa.conf
You don't need to move cspsa.conf to /etc -- you can specify its location as a parameter to cspsa-server
The readme files provided with the sources also provide some information on the tools.
A bit of a glossary:
CSPSA -- crash safe parameter storage area
NWM -- Nokia Wireless Modem; cspsa-tools' readme.txt says
At startup of the U8500 there is a CSPSA client (cspsa2nwm) that reads
out the NWM files (if necessary) and stores them to a directory in the file
system. The original directory tree is fully re-created with original access
rights settings. These files can then be read by the modem (via MSA) to
configure itself.
Click to expand...
Click to collapse
PFF -- Parameter File Format; cspsa-tools' pff_readme.txt says
The parameter file format is a way to store parameters in a generic file
format. The advantage is that one can use common tools in Linux or windows
to parse the file structure and format. Also it is easy to use git or other
version control systems to keep track of changes in parameters.
Click to expand...
Click to collapse
GDF -- ?
The packages contain various tools to convert between those formats.
As you noted, cspa-cmd can be used to list the CSPSA's contents, and to store them to individual files, as well as exporting to a gdf file. That file can then be viewed by gdflist. That latter tool reveals that apparently my CSPSA's contents were created by nwm2gdf, and that apparently most its contents are supposed to be copied to /modem/RFHAL -- I wonder if that's really the sole purpose of the CSPSA, and if (on a functional phone) everything is also identically contained in that /modemfs/RFHAL. I'll need to do a diff of that file and that (possibly) corresponding record showing up in the gdflist output.
One more (near-trivial) note: the part of CSPSA that I had noticed in a hex editor to have changed after network unlocking is 0x477-0x457=0x20=32 bytes long. So that *could* be a SHA-256, as that would be 256 bits = 32 bytes long.
Hey, you must have done a mistake in last calculations. Actually, 32 bytes is 32 x 8 = 256 bits.
Wysłane z mojego GT-I8160 przy użyciu Tapatalka
This might be a mega necro, but is anyone by any chance even remotely interested in unlocking a phone this ancient?
I happened to have one at home which I use now as a replacement since I broke the touchscreen of my original device. In some of my free time I peeked around a bit, and managed to properly export the CSPSA entries. Sadly all this just results in a more human readable representation of hex values stored in the CSPSA, which means that without actually knowing what was stored there one must try a different approach
:INTRO:
OK: here's the thread we are talking about and working on getting VoLTE & VoWiFi to work :good:. For now this is for the H930 Open EU variant only, it worked for two peoples incl. me, but I think this is applicable on other variants of V30, with respective edits (/OP, you'll read later), too. Maybe even to other phones? I don't know, LG is sth (=something) special This thread will rise with time I think, with different edits or ways to reach the goal: we want VoLTE/VoWiFi, sth, LG imho should've taken more care of...
I hope, my english is sufficient so everyone understands what's to do And, btw.: you should have "Allow OEM Unlock" switched off.
User @Krekos/CZ/ did help me in a special way: he zipped his modem config files I by accident deleted from my phone and uploaded them :good: Without them I would've not being able to complete this work, and therefore I wanted to say thanks again to him at this point (I think you can't hear this anymore, hm? ). He's btw. the second person for whom this worked, he's got VoLTE&VoWiFi now.
These files should work on other SD835 phones too, I link them here for archival. Eventually we can get a collection from every variant, this way we might be able to activate VoLTE etc. for a carrier on a phone which was not intended to being used with other operators. So a call to other variant users: can you please tar and upload these files to XDA? Procedere is:
Connect your phone via ADB and open a command prompt
type:
Code:
adb shell tar -czf /sdcard/mbn-files.tar.gz /firmware/image/modem_pr/mcfg/configs
Upload this file to your post here in this thread. I'll add it to the list then.
Links to modem cfg files (mcfg_sw.mbn):
H930 EU Open
US998 Open
These files need special permissions, of course... Otherwise it would be boring
The path (on a LG phone) for oreo firmware is as follows:
/firmware/image/modem_pr/mcfg/configs
For pie it's not /firmware/, but /vendor/firmware_mnt/: pls remember to change path in the following process accordingly when using pie.
Permissions are:
files 0440 system:system
folders 0550 system:system
Apply them via terminal emulator or adb:
Code:
cd /firmware/image/modem_pr/mcfg/configs
chown -R system:system * {apply to everything in this folder and subfolders correct user and group}
chmod -R 0550 * {apply to everything in this folder and subfolders permissions}
find . -type f -print0 | xargs -0 chmod 0440 {apply to files only (not directories) in every (sub)folder permissions}
:MAIN PART:
Prerequisites:
- H930 (or variant, later)
- rooted
- File Manager with root privileges
- eventually a decent editor of your choice
- and a little bit knowledge about giving permissions and changing owners/groups, if needed
I won't give help about e.g. files not editable because of problems with filemanagers or root, do the search work for infos and parameters you need, or anything else, sry You have to prepare and maybe investigate some time to find some infos. You have to know, that this can lead to a brick, if you're doing weird stuff
Let's begin with backing up your complete ROM via TWRP, incl. efs, data and all you need. Backup your OP partition too:
Code:
dd if=/dev/block/bootdevice/by-name/OP of=/sdcard/OP.img
It's about 344mb in size and can be restored in recovery or ROM, adb or terminal emulator:
Code:
(adb shell) dd if=/sdcard/OP.img of=/dev/block/bootdevice/by-name/OP
Then gather some infos, for now it looks we only need MNC and MCC, which identifies your operator. Maybe it later turns out that you need parts of your IMSI too. You'll find it in LG's hiddenmenu:
*#546368#*930#
Field Test => SIM Info
You can check, if the IMS Settings are available already. If yes, then there's a high chance you don't need the "some more config" part.
Write it down and have it prepared. Now the editing session begins
First we will load our modem config for our operator. The corresponding script somehow seems broken as it should do its work with files in /data/shared folder, which isn't.
These configs can give some extra parameters and contain infos about your operator / SIM provider etc.: Open your filemanager (and/or maybe editor, if isn't integrated in fm) and open the file "/firmware/image/modem_pr/mcfg/configs/mcfg_sw/mbn_sw.txt". Here find the path for your provider. O2 Germany would be "mcfg_sw/generic_/joan_glo/eu/o2d/o2d/germany/mcfg_sw.mbn". Append these = the complete path for your config file.
Code:
/firmware/image/modem_pr/mcfg/configs/mcfg_sw/generic_/joan_glo/eu/o2d/o2d/germany
Copy your op config path, you need this for one of the following edits.
Now navigate to the folder "/data/property". There should be three files we need. If they are not there, create them with root:root, 0600 and following content pls:
persist.radio.buffet_mbn_file => the long path you copied before
persist.radio.buffet_enable => enable
persist.radio.sw_mbn_update => 1
Otherwise just edit and save them afterwards. Perform a reboot and then check in hiddenmenu => Field Test = MCFG Buffet, if this file is loaded. Should
Some more config is needed. We now need the before told MNC and MCC and maybe the IMSI. We need to point your ROM in the right direction (modem itself is done): localizing it and giving the exact operator infos. Have a check of following files and a look, if they are configured already correctly for your operator, or you have to edit them.
/OP/client_id.xml:
Is your MNC & MCC inside? If not, copy and paste an example line, edit this one so it suits both.
/OP/OPEN_EU/config/carrier_code.xml:
Same as above, but a little more complicated. Explanation with an example line:
Code:
<profile carrier="O2D" suffix="O2D" carrier_code_fast="" carrier_code_late="TLF" country="DE" mcc="262" mnc="07" mvno_type="imsi" mvno_match_data="26207200xxxxxxx" />
"Profile carrier": you find this in the long path to your .mbn file you copied before O2D is O2 Germany, edit this to fit your sim provider.
"suffix": same
"mvno_match_data": this is part of your IMSI. When your provider is already listed, compare the five numbers after your operatorcode: 26207200xx. This all together has to be the same as the first ten numbers of your IMSI. If not, edit it. This is one of the reasons for VoLTE chaos on O2D, but that's another story.
/OP/OPEN_EU/config/com.lge.ims.rcs.xml
Same as above: copy and paste a line of simoperator section and edit with your operator details. "prefix" is your country calling code.
/OP/OPEN_EU/config/featureset.xml:
Change every item which is "false" to "true"
/OP/OPEN_EU/config/vo_config.xml:
Most likely you need to create an entry for your operator with mnc and mcc. do this and change both variables to "1":
Code:
<info mcc="262" mnc="07"><!-- O2D (you can write anything here or leave it, it's commented out)-->
<prop
support_volte="1"
support_vt="1"
/>
</info>
[b][i]Maybe[/i][/b] for some rare cases this is the only edit you need to do to get VoLTE & VoWiFi running :D
Reboot now, then continue. If sth in custom.prop or build.prop is edited the wrong way, this can cause bootloops, so you don't have to do the beforedone work again. And: when something in general went wrong in /OP directories edits, you can dd your OP partition back.
/OP/OPEN_EU/cust.prop (they're all case sensitive):
ro.lge.capp_cupss.rootdir=/OP/OPEN_EU
ro.build.target_operator=OPEN
ro.build.target_country=EU
ro.build.target_region=EU
ro.product.locale.region=your region, e.g. "DE" for Germany
ro.product.locale.language=your language, e.g. "de" for Germany
ro.lge.cupssgroup=GLOBAL-COM
ro.lge.opensw=EUR-XX
ro.lge.radio_mcfg=1
persist.lg.ims.volte_open=1
Reboot.
/system/build.prop:
ro.product.locale: change to match your language
ro.lge.capp_cupss.rootdir=/OP
ro.lge.capp_cupss.op.dir=/OP
Reboot again .
:0UTR0:
The VoLTE and VoWiFi switches should be present now, a) in Phone Settings (both) and b) in Network Settings (VoLTE). We should now be able to hop into IMS Settings. You find it in LG's hiddenmenu => Field Test => IMS Setting.
First we let the IMS setting show an icon in statusbar, this way we don't need to dial the hiddenmenu code everytime we need the settings. A nice sideeffect: it shows if you're connected or not To do this tip on "Test" and activate "Show Icon". After that go back to the main screen and scroll down to "GPRI VoLTE/VoWiFi". Here you can activate VoLTE/ViLTE/VoWiFi/ViWiFi, depending on what your provider offers in your contract. Go back again to main screen, go into "Test" menu, then "Load Preset Configuration" => "Initialize configuration". Now you can do a reboot and press a thumb: maybe this was it and everything is up and running! Maybe...
For seeing if VoLTE is up you can check "Debug Screen" => "VoLTE Debug Screen" inside IMS setting.
You can also turn on VoLTE and VoWiFi indicators/symbols, which are then on the right side of your statusbar near the clock. In IMS settings: => "GPRI VoLTE/VoWiFi" => "Registration" => "VoLTE" => "Indicate VoLTE", and the same for VoWiFi. To be able to place a call in flight mode but with WiFi calling on, enable "Airplane Mode" in VoWiFi menu.
In case you experiment with IMS Setting and suddenly a com.android.phone (or other) fc occurs, you can boot into TWRP, mount data and delete the directory "/data/user_de/0/com.lge.ims" with its integrated filemanager.
K, you're done :cyclops: Pls give feedback how it went for you!
Wow. Thank you.
yeah, I hope it's written in a way everyone understands it atm it's more like a dev thread, it will be rewritten for newbies when everything is clear and fully pointed out
.. Uploaded stuff ..
Hi - thanks for this info. I'm trying to work with it, and added my files. + a few extras stuff> actually not really sure how to do this .... any help? ... Thanks I have a US998 phone.: but want to get the Canadian settings all working ; specifically for Freedom Mobile ( aka : Wind ) --- the files are from a saved backup ( dump ) {{ the uploaded files are from a backup of the H933 canada firmware dump }} ... I'm currently using the US998 kdz firmware on the phone ....
let's start with
- do you see "MCFG Buffet" in your hiddenmenu?
- can you list all the content (the main directories) in your /OP folder pls?
- does the firmware modem config folder exist and is your provider inside?
- and where does your ro.lge.cupssgroup point to? (adb shell getprop ro.lge.cupssgroup)
I don't know the specialities of other variants, especially the offered providers. I don't even know if this is possible and which files you uploaded to your phone. can you either list them or point to them with a link eventually?
OK thanks - I'll work on this some more tomorrow -- I don't know where the files I uploaded went to! > but I uploaded >Folders>>> Carrier Config, Modem_pr, qcril_database, vendor, wifi & build.prop. All files were dumped from a working Phone loaded with the Canadian .Kdz H933. ... > ok so I actually took all those files,and overwrote the files om my current phone setup which is US998 .Kdz -- didn't even crash the phone <<< Go figure. I just had to clear the cache and dalvik cache. VoLTE may be working, but my Carrier has a very limited VoLTE service... so I cant check right now. VoWifi is not working. Every setting and menu has it turned "on" but is no working. .... tomorrow I'll follow your instructions after I do a fresh install. of the .KDZ. ..... be well
be warned: this can be time consuming and causing headaches, too stock files would be the best to start with.
is it a native us998 phone? it's the one one can "frankensteining", isn't it? you've overwritten these files and partitions, when I understand you correctly?
you should get hands on following stock partition disk dumps and folders first (kdz won't help as all these files are preloaded and can't be flashed via kdz):
- canadian OP partition (the more important one) (canadian_OP.img <= example filename)
- US998 OP partition (for being able to revert back to stock and trying to get volte/vowifi running with only some edited files on stock us998 partitions, only a few added files or maybe folders from canadian OP partition and only the .mbn file for your provider, taken out of firmware modem config folder). this one shoukd be flashed and worked with.
- stock system image (kdz flash us998)
- clean unencrypted data partition (factory reset and then completely set up with your apps and settings, no db or ither files copied).
- you've got a backup of your stock efs partition?
try to keep your phone as clean as you can as this can only be of help about not being confused. are there gsm/(w)cdma/band differences between us998 and canadian 993?
this could go an interesting way let's see, maybe we get it is wifi calling enabled on your contract? I wasn't able to determine this out of your post.
there's the possibility to edit the thresholds when wifi calling aborts and switches to cell telephony or volte. it's located in /system/etc and inside a filename called "andsf_your_provider_code.xml", for o2 de it's named "andsf_26207.xml". I've set every threshold to -90, this way wifi calling is still working with only one or two bars wifi reception. otherwise it would abort just too early and switches to other telephony bands.
US998 mbn-files
Thanks for the detailed info.
Not working for me. I managed to add my operator and enable IMS but can't figure out how to make IMS register. Might be because my operator is a little more retarded and uses IPv4 only. No idea where to set that policy, it isn't in IMS settings anymore on the V30.
sounds like setup session in IMS setting. go into gpri volte/vowifi setting and try enabling volte and vowifi only, then tap on "call" menu entry. choose "volte" and tip "use sip preconditions" there, use the same setting for "vowifi". check "subscription" in main menu and "IMPI" there: is there a connection adress inside? example:
your_imsi@ims.mncXXX.mccXXX.3gppnetwork.org
replace mnc and mcc with yours but add a zero at the first place of your mnc. for o2 de's mnc which is 07 it looks like
your_imsi@ims.mnc007.mcc262.3gppnetwork.org
then try test menu and restart ims process. or perform a reboot.
but you'll have to investigate more, for yourself. what's your exact provider and mnc mcc?
edit: can be you need an IMS apn for your provider. in it you can set the protocol versions. what's logcat telling?
Hi. Thanks for this. Is there any way that this can be applied to aosp ROMs?
Those mcfg files are found inside the modem partition and you can pull them out from any kdz.
Do you have an idea where this is located? (/nv/)
Edit: Also, added some picture showing volte working... when it's not (switching back to something else)
you're really sure about this? since I've flashed a kdz and a modem and unfortunately this didn't bring back these files on my phone maybe I did sth. wrong, this is really interesting now. thought they were on efs partition; but these nv files should be on efs. I'm not really sure about that, maybe someone other can finally confirm :good:. edit: I remember I've got carrier services installed, would be interesting if this makes a difference. I hardly believe but try it:
https://play.google.com/store/apps/details?id=com.google.android.ims
about aosp: there's much more needed, proprietary files and more config inside sources etc pp. I evtl. wanted to do compilings again in winter, I wanted to try this but I need additional ssd space before
Oh sh*t...
I guess I lost my efs partition then...
I tried a bunch of stuff from your first post... sadly, I don't have an /OP/ directory at all... and when I try to force the mcfg mbn (from my sdcard lol, cauz my carrier is not provided in the us998 modem)
Edited a bunch of stuff in my build.prop to make it look more like the canadian one.. but it still doesn't work (even lost the volte option now in the mobile setting menu..)
Also, I tried to flash my canadian modem files... sadly.. the phone boot but the modem doesn't initialize at all... (no wifi and no mobile network)
aye, sh¡t... and there's no efs backup from your phone done / available as it sounds. you lost your IMEI too??
can you check the directory
/dev/block/bootdevices/by-name
and see if there's an OP "file"? eventually only the script which mounts it is broken. if I remember correctly there was a thread about regathering the IMEI or at least fixing some stuff on efs. maybe this helps you, unfort. I don't know where it was, you have to search yourself.
k, sleeping now, it's hard for me these days after my shoulder operation... hopefully we get your phone back running! I'm optimistic flashing a efs backup and changing the imei back to your original one for example. don't know if this is possible, but how often did I think "sh¡t, now it's over, phone's bricked" and then a solution appears out of nothing. you should've seen my face in the moment volte came.up on my phone xD xD xD
see you later be optimistic!
Oh sorry! I still got my efs partition.. I thought it would get overwritten when doing the frankenstein method.. But I guess not.. Is there any way I can debug why the modem doesnt work at all when flashing the canadian one while using the us998 system?
there is but I don't have it in mind atm it's explained here at xda somewhere, logcat and dmesg is used for that :good: can be that it is bands related? so you have to unlock bands in your modem settings? edit: most likely not as wifi is dead too...
seadersn said:
sounds like setup session in IMS setting. go into gpri volte/vowifi setting and try enabling volte and vowifi only, then tap on "call" menu entry. choose "volte" and tip "use sip preconditions" there, use the same setting for "vowifi". check "subscription" in main menu and "IMPI" there: is there a connection adress inside? example:
your_imsi@ims.mncXXX.mccXXX.3gppnetwork.org
replace mnc and mcc with yours but add a zero at the first place of your mnc. for o2 de's mnc which is 07 it looks like
your_imsi@ims.mnc007.mcc262.3gppnetwork.org
then try test menu and restart ims process. or perform a reboot.
but you'll have to investigate more, for yourself. what's your exact provider and mnc mcc?
edit: can be you need an IMS apn for your provider. in it you can set the protocol versions. what's logcat telling?
Click to expand...
Click to collapse
I already tried to set IPMI with my data like [email protected] but IMS still don't register and the setting doesn't stick. it always reverts to a default like [email protected].
I also created the ims APN but it doesn't help with anything.
I'm still digging but so far I have no clue what's going on.
hmm, did you try configuring sip telephony? for o2 germany the registration looks like
country code + phone number @ telefonica.de = (for germany) [email protected]. have a look at /data/user_de/0/com.lge.ims/shared_prefs/impu_list.xml
(user_de: I don't know if this is indicating the language the rom uses? so for an english language build it can look like user_en? I don't know...).