OK, time to give this subject its own thread. You can read about previous efforts here: http://forum.xda-developers.com/showthread.php?t=1113066. In particular, http://forum.xda-developers.com/showthread.php?t=1113066&page=11 is where I started.
Background: the policy database is essentially the Access Control List (ACL) store for WP7. ACLs are typically attached to objects (files/folders, registry keys/values, drivers/services, possibly even APIs). When a process tries to do something, the OS uses the process's security identifier (called a "Token", it identifies the account running the process and therefore the permissions that process has) and looks up the ACL specific to that operation. If the ACL authorizes that account to perform the operation, the kernel permits it. If not, it blocks the operation and indicates an error (most famously on WP7, 1260 or 0x4EC, meaning blocked by policy). For some OSes, like NT, that attachment is in the metadata which describes the object (for example, NTFS stores ACLs for each file and folder). Apparently, WP7 uses a centralized database of ACLs, stored as "policies", instead.
Why I'm doing this: the policy database is the key to fully unlocking the phone. I mean that literally; "full unlock" ROMs achieve that state by basically turning off policy enforcement. I don't necessarily want to do that - at least not phone-wide and constantly - but I want to be able to set my own policies, and possibly modify existing ones.
What can be done with it: well, one example is the subject of the thread I linked above: homebrew native EXEs require first being able to add policies for them. There are some other cool possibilities, like turning off ID_CAP_INTEROPSERVICES enforcement or allowing apps to write to the MaxUnsignedApp registry value directly. That gets around the risk of phones being re-locked and unable to interop-unlock again. Basically, it allows an app to do anything short of modify the ROM.
Purpose of this thread:
* Provide a central location of information about the policy system, policy database, and creation of custom policies.
* Collaborate on the project of understanding and modifying the policy database and policy system overall.
* Share interesting policies we've found in the database, or post custom policies that can be added to enable a cool hack.
* Discuss and share ways to preserve, going forward, our control over the policy system.
There has been concern raised that this work should not be mde public, because Microsoft will look at what we are doing and use that knowledge against us. There is some validity to that argument; if the work is done in secret, and any files posted that use the fruits of that work are heavily obfuscated, it would probably take Microsoft a little longer to block it if they decided to do so. Not terribly *much* longer though, I suspect - they have many tools at their disposal, full source code and documentation, and full understanding of the system in their engineer's minds. Any hack we find, they can reverse engineer or simply block access to whether or not they can read a thread about it here on XDA-Devs.
There's also the risk of malware. Malicious homebrew apps could abuse this knowledge to do serious damage to your phone, to steal info, and possibly even for direct financial effect (send premium SMS, for example). However, I see no real way around that problem; it's an inherent risk of unlocking a device. The simplest and best step to combat it is to not install untrasted apps, and the best way to be sure an app is trusted is to be able to analyze it. (This is one of the reasons I include the source for my apps, and encourage others to do the same.) Besides, it's already possible to do plenty of damage with existing homebrew hacks, yet somehow that problem hasn't materialized.
So, instead of secrecy, I propose openness. The best option we have to offset Microsoft's tools, knowledge, and source code is to collaborate, pooling the knowledge and effort of many hackers. If people want to keep certain things secret, by all means use email or PMs. In general, though, I think the failure to spread knowledge does more harm than good.
OK, that turned into a long enough intro that I'm going to post my first actual findings in a reply.
Policy-related files
There are actually two databases: one is for policies, and one is for accounts. They are located in \Windows\Security\ and are called policydb.vol and accountdb.vol. These files are locked (opened without sharing permitted) while the OS is running. There are two additional files in this folder: PolicyMeta.xml and PolicyCommit.xml. These files can be accessed using provxml, TouchXplorer, WP7 Root Tools, or HtcRoot Webserver.
The PolicyMeta XML file contains macros describing accounts, and metadata about the policies in the database. In particular, it contains a large number of bit masks that indicate different permissions. By itself, this file doesn't tell us much of use, but it will be a big help for understanding binary data in the the database. It's small and not commented, but easy to read.
The PolicyCommit.xml file contains the merged result of combining all the policy files on the phone. I don't know if anything actually reads this fine, but it's a nice human-readable (and searchable) view of the data that goes into the policy database. It contains a number of comments, but most are just where the various policies were merged from. It is the largest file.
The policy database file ("Volume" to use the term of the CEDB APIs) itself is large-ish (mine approaches a megabyte) and contains three CEDB databases. The first is a small single-record "database" (in SQL you'd call it a table) that appears to be used for transaction locking. The second is a single large record (several KB) that appears to be a bloom filter (Wikipedia has a pretty good article, the short version is that it is a quick and compact data structure for checking whether a given item is in a collection). The third database (named "PatternDBmultimap") is the real deal, containing thousands of policy records.
I haven't looked at the Accounts database much yet. It's smaller than the Policy database volume, but still a few hundred KB. A substantial portion of that is probably custom accounts created for each app that is installed (since each app has different permissions - specifically, each app has read and write access to a different set of folders - there must be a unique account for each).
The policies appear to come from a few sources. One of them is the many *.policy.xml files (the first part is usually a GUID) in the Windows folder. These files are locked in ROM, and define the core system policies (system accounts, permissions for system objects, etc.). The \Windows\Security\PolicyCommit.xml file (which is not in ROM, or even marked read-only) appears to be simply the result of merging all these files.
Another source of policies must be the application installer. Application-specific polices are not present in the PolicyCommit.xml merged file, but are in the database itself. It is reasonable to expect that they are created and removed by the package manager. This is a good sign for being able to modify policies ourselves.
The initial creation of the policy files appears to be up to a program, \Windows\PolicyLoader.exe. This program takes policy.xml files, merges them, and produces the merged result file and the policy database(s?). It's even possible to run it, given sufficient permissions. Unfortunately, it seems unable to modify the policies on a running device, and is believed to only run at first boot (or after a hard reset) or when an update CAB installs new policy XML files.
EDIT: Attaching the \Windows\Security\*.xml files from my phone, along with the decompiled source for PolicyLoader that was posted on the other thread.
The LG MFG app has a section for editing certain security policies. I can post the info from there, if it'd be of any help. By the way, it specifically says "Edit security policy through registry" so it might not be the same policies that you're talking about, I don't know.
EDIT: Actually, looks like those policies are a subset of the ones listed here: http://msdn.microsoft.com/en-us/library/bb416355.aspx
Analysis of the policy database
I wrote a function to dump the policy database to a text file (with inevitably some embedded binary). Each record in the database has four fields. I'll do my best to describe them below.
1) The first is a DATETIME struct (two 32-bit integers). This is the only 64-bit numerical type available except for a DOUBLE, so it might be selected just as a convenient way to store that many bits rather than because it's actually a date and time. In particular, when I converted them to actual dates and times, the years ranged from the 1970s well into future centuries... this seems an unlikely candidate for an actual set of dates.
What I think it actually is, is some kind of hash of the second field. It might be the index bits for the bloom filter, for example. The reason I think so is that, when there are multiple records with the same value in the second field, they also have the same value in this field, but even a slight difference in the value of the second field results in a very different first field.
This field is not unique, but it does appear to be the default sort order for the database. I don't know if that's ust because it's the first field, but it would make sense to have it be indexed using this field for fast lookup (binary search) after the bloom filter finds that the item is (probably) present.
2) This field is a binary BLOB struct (a size and a pointer). This field contains Unicode strings, sometimes with a bit of binary data (small, typically less than 20 bytes) tacked on the end. Strings plural; each one is NULL-character terminated.
This field appears to be the paths that indicate the object (or objects, since it can contain wildcards) that the policy applies to. If there is a policy in the XML for ResourceIri="/REGISTRY/HKLM/SOFTWARE/MICROSOFT/CAMERA/READWRITESETTINGS" then there will be a record in the database with the second field that would be written like this in C source code: L"REGISTRY\0HKLM\0SOFTWARE\0MICROSOFT\0CAMERA\0READWRITESETTINGS\0". I'm not sure what the occasional binary afterwards means, although there appears to be a specific value for a wildcard (represented in the source XML as ResourceIri=/PATH/WILDCARD/BASE/(*)", but the last part doesn't translate to Unicade the way you'd expect).
As mentioned above, I'm pretty sure that the first field is related to this one. Since the value of a bloom filter on this database would be to quickly establish "Is there a policy for this object?" it makes sense that the path (second field) is the data that gets hashed to produce the bits of the key. It's not really required to then store the key bits, but they make a reasoanble value to sort on.
3) The third field is also a binary BLOB, but the value of it is much more opaque. Typically in the range of 50-300 bytes in length, there are certain patterns that I've noticed within it (0x01 00 01/02 00 65 is a common prefix, and they typically end with 0x00 3X) but I have not yet determined what they actually represent.
Some logical possibilities are an account identifier (though that seems needlessly long for such a purpose) or possibly the permissions data directly. When the second field has a path to related objects (for example, the isolated storage of an application), the third field is often similar as well.
4) The fourth field is another DATETIME struct, but in this case is obviously not an actual date value. The high four bytes are (almost?) always 0xFFFFFFFC, and the low four bytes are typically 0x0000XXXX where the Xs can be anything. This value is not unique - there are numerous instances of 0xFFFFFFFC00000001, for example - but I'm not yet sure what it is.
The same guesses I offered for field 3 apply as well, with the caveat that it's probably not just a different representation of field 3 because two records can have the same value on field 4, and their field three values may not only differ, but be different sizes. I need to look at the XML files and see if there's a pattern between policies with the same field 4 and an equivalent data item in the XML.
I'm attaching the dump file I created of the policy database. It's best opened in a hex editor (Visual Studio does well enough) although you can also use Wordpad (Notepad won't respect the line endings). Wordpad can't show you the binary, of course, but it's a readable layout of the data.
The format is as follows:
ASCII string: "Index "
ASCII representation of an Integer for the index.
ASCII string: ": Prop0 (FILETIME): 0x"
ASCII representation of the DateTime, with a space between the high and low DWORDs.
ASCII string: " | Prop1 (BLOB, "
ASCII representation of the blob's integer size.
ASCII string: " bytes): "
Direct dump of the second field's BLOB buffer (multiple UNICODE strings).
ASCII string: " | Prop2 (BLOB, "
... and so on. I intentionally used ASCII to make the direct memory dumps, which are in UNICODE for the second field at least, stand out.
@Arktronic: Interesting. Those policies (in the registry) are a legacy holdover from WinMo, and at least some of them have been superceded by the new policy system, but the fact that LG gave them specific mention in their app suggests that they still have some relevance.
However, you're correct that those aren't the policies I was speaking of elsewhere in the thread. It may be a good idea to explore them both in parallel, though. Which ones does the LG app list?
Arktronic said:
The LG MFG app has a section for editing certain security policies. I can post the info from there, if it'd be of any help. By the way, it specifically says "Edit security policy through registry" so it might not be the same policies that you're talking about, I don't know.
EDIT: Actually, looks like those policies are a subset of the ones listed here: http://msdn.microsoft.com/en-us/library/bb416355.aspx
Click to expand...
Click to collapse
We already did some testing with those policy settings, but the ones granting more access were not available and the others could not get the app itself into an "unsafe" mode. But then again, I'm far from a professional when it comes down to these things, I just crossreferenced them all against the MSDN DB and looked for the ones that would make fileops possible, no luck.
I'm not sure if they added policies to the LG MFG app in the meanwhile (unlikely) but it might be worth it to investigate how the MFG app modifies those select policies.
GoodDayToDie said:
@Arktronic: Interesting. Those policies (in the registry) are a legacy holdover from WinMo, and at least some of them have been superceded by the new policy system, but the fact that LG gave them specific mention in their app suggests that they still have some relevance.
However, you're correct that those aren't the policies I was speaking of elsewhere in the thread. It may be a good idea to explore them both in parallel, though. Which ones does the LG app list?
Click to expand...
Click to collapse
The latest ROM's MFG app has the following policy IDs: 4104, 4105, 4108, 4109, 4110, 4111, 4113, 4119, 4120, 4121, 4124, 4131, 4132, 4141, 4142, 4143, and 4149.
The last one isn't in the MSDN doc; it calls itself "FIPS Self Test Policy" or SECPOLICY_FIPS_SELF_TESTS.
There are potentially useful things like SECPOLICY_OTAPROVISIONING (4111), which has the value of 3732 - no idea which flag(s) that represents - but if there's a way to send provisioning messages to WP7, that might open up quite a few possibilities.
I believe there's at least a chance for OTA provisioning. Sending custom SMS appears to be possible (click around from the link):
http://msdn.microsoft.com/en-us/library/ee498239.aspx
That said, it's almsot certainly either secured or disabled by default.
Hmm... does anybody want to take a shot at getting a decent decompile of lvmod.dll? I don't have the tools, though I probably should. Reading the disassembly is slow and painful.
I've found a few new things:
It's possible for two records to differ *only* on the third field, and even then the binary was more alike than not. Look at indexes 12 and 13 in the dump - they're really similar. They are built from the following policy rules (no promises on order):
Code:
<Rule PriorityCategoryId="PRIORITY_HIGH" ResourceIri="/REGISTRY/(*)" SpeakerAccountId="S-1-5-112-0-0-1" Description="TCB can do anything to all registry keys">
<Authorize>
<Match AccountId="S-1-5-112-0-0X02" AuthorizationIds="KEY_ALL_ACCESS, KEY_READ, KEY_WRITE, KEY_EXECUTE, GENERIC_READ, GENERIC_WRITE, GENERIC_EXECUTE, GENERIC_ALL, DELETE, READ_CONTROL, WRITE_DAC, WRITE_OWNER, SYNCHRONIZE, STANDARD_RIGHTS_REQUIRED, SPECIFIC_RIGHTS_ALL, ALL_ACCESS" />
</Authorize>
</Rule>
Code:
<Rule PriorityCategoryId="PRIORITY_LOW" ResourceIri="/REGISTRY/(*)" SpeakerAccountId="S-1-5-112-0-0-1" Description="Catch all rule to allow Normal and above apps to read/write to all unnamed keys">
<Authorize>
<Match AccountId="S-1-5-112-0-0X23" AuthorizationIds="KEY_ALL_ACCESS, KEY_READ, KEY_WRITE, KEY_EXECUTE" />
</Authorize>
</Rule>
I would have thought that either the different permissions being granted, or the different accounts they were granted to, would result in a different fourth field... but no such luck. Time to look into this further.
The accountdb.vol file has two databases in it, GroupMemberships (1105 records on my phone) and Accounts (291 records). The latter is actually much bigger in terms of data size, though - 70KB vs 31KB for GroupMemberships. The records in GM must be very small, probably just pair mappings.
Hey GoodDayToDie,
Awesome job on sharing all this low level findings from underneat the hood of my favourite mobile OS. While i'm not capable of researching this myself due to lack of knowledge I love to read about how you (and other well known WP7 hackers as well of course!!) tackle the security and are willing to share this with the community to combine power. I think threads such as these are really necessary to get to the finish. Keep up the good work, i've got a strong feeling we will get there eventually .
THANKS
Looks to me like this is the policy database.
Here is an example set of policies that enable/disable tethering on the Arrive.
Is shows the values needed to create/add a policy to the policy database. HTClv.dll shoudl be able do set/modify these values using "LVModProvisionSecurityForApplication"
You may already know this, but figured I would share.
Also, HTC has regedit.exe and HTC uses it to provision/make registry changes.
I will attach the regedit4 file HTC uses to configure the radios.
This also defines where the key UserProcGroup defines the TCB chamber a driver runs under. see... "UserProcGroup"=dword:5 ; TCB chamber
Seems with using the registry editor, we could elevate any driver to the Kernel chamber.
See attached....
Thanks for the info Paul!
I've heard of the "LVModProvisionSecurityForApplication" API before, yes, and it might be possible to use it here (*really* depends on how it works; if it just reads the app's manifest file like the normal XAP installer does, that's not very useful). LVModAuthenticateFile, LVModRouting, and LVModAuthorize may be extremely useful, though. It also might be helpful to try reverse-engineering how it interacts with the policy database.
The weird thing is, I don't have any htclv.dll or htcpl.dll on my phone, at least not in the \Windows folder. Perhaps they were removed in an older firmware update? It certainly sounds like they would provide the APIs I need - only for HTC phones, true, but they would provide.
The policy.xml file is the standard format read by PolicyLoader.exe, but that doesn't really help unless I can convince PolicyLoader to modify the ploicy database on a running phone.
Elevating an (already installed) driver to TCB might be useful (although I'm not certain that LVMod route-to-chamber rules wouldn't interfere) but all the useful HTC drivers are already in TCB, and installing any more drivers... well, I haven't been able to make that work yet, even old versions of official drivers with the necessary changes to the DllName in the registry.
It's really too bad you can't join in on hacking this stuff though, you've got the right ideas. Do you by any chance have a NoDo restore point you could downgrade to in order to try out some stuff on the old firmware?
Dumped the account database
Turns out the account info is quite straightforward. There are four fields per record.
0) String - the SID ("S-1-5-112-0-0X10-0X00000024").
1) Int32 - 0 for accounts, 1 for groups.
2) Int32 - always 0 on my phone.
3) String - account or group name ("TCB" or "ID_CAP_FILEVIEWER:Capability for hybrid file view app such as PDF reader etc." or "Settings3.exe Chamber" or "9BFACECD-C655-4E5B-B024-1E6C2A7456AC").
Not sure why the third field is there if it's always 0, but OK. The first and last were obvious, and the second was easy to infer. The last record has no fields, and the three immediately before it are without a fourth field; not sure why. All three are groups, and their SIDs are:
S-1-5-112-700-4160
S-1-5-112-700-5132A485-ADEE-5842-9490-856FFFFF2D6D
S-1-5-112-700-A22CF327-25C3-DB2A-A8DF-7BE586F11FBD
This database contained no binary blobs, so the dump file is plain ASCII text (the strings were originally Unicode but converted to ASCII gracefully). In the interest of making it easier to analyze, I ran a quick pass over the dump with sed and produced a CSV, which is attached.
Then, there's the GroupMemberships database. I think this one is probably less important for our concerns, but I wanted to take a look anyhow. It's the simplest so far, though that's not necessarily good. Each record has two fields, and both are just 32-bit ints.
0) Ranges from 0x30000006 to 0x3F0004A6, though the the third through fifth hex digits are always 0. Includes duplicates.
1) Ranges from 0x31000008 to 0x3100007A, then from 0x32000380 to 0x3200038C. Includes duplicates.
The mappings appear to be many-to-many (each account in multiple groups, each group holding multiple accounts) as expected. I'm guessing the first column is accounts and groups, and the second is the groups that the account or group belongs to. Given that some values appear in both columns (through in different records), I'm guessing nesting of groups is allowed.
I dumped and CSV-d this database, and it is attached as well. Ideas as to what's up with it are welcome too.