CE7/Windows Phone 7 - The Kernel, Memory Management, etc. - Windows Phone 7 Development and Hacking

Similar to the CE5 kernel that we're used to, the CE7 kernel is a 32 bit OS and runs a 4GB Virtual Address space. Similar to CE5, 2GB is reserved for the kernel and 2GB is reserved for user space.
This is where it begins to differ. CE5 handled the user space by splitting the process memory up into 32 slots of 32MB each. (This accounts for 1GB of user space, the rest is used for memory mapped files, fixed up modules, etc.) Any app currently in the foreground was swapped into slot 0 during execution, and swapped back into it's slot while it's backgrounded.
CE7 similarly uses 1GB for the process code, however now it's handled differently. Each running process is mapped to the entire 1GB space, allowing the full 1GB of VM available for use without the trickery needed to accomplish using more than 32MB of VM in CE5. This should allow for much more rich apps to be developed (whenever MSFT decides to allow us to write native code, which is coming at some point...)
This 1GB VM space will be dedicated to the process, and not accessible by any other process. The remaining 1GB of user space has to be utilized for inter-process memory sharing.
There are 256 priority levels as with CE5, for each individual thread within an app. Scheduling is handled on a round-robin basis for threads sharing the same priority level. Priority level 0 functions as a "real time" priority level and any thread running at this level will run through til completion before the scheduler runs another thread. True multitasking will not be accessible to programmers initially, though MSFT intends to unlock that later down the line (presumably around the time native code is allowed). There is no hard limit to the number of threads a program can utilize, it is only limited by the system resources available.
Regarding the limited APIs available to us "normal" programmers - OEMs and MOs will have access to an extended set of Managed APIs and a limited set of Native APIs that "normal" programmers won't get access to. This likely includes things like RIL.

Just give me an ARM build of it that is already ported over to run MSM7x00.
Thank you

Bump for visibility

good bump message too short grrr

Da_G said:
Similar to the CE5 kernel that we're used to, the CE7 kernel is a 32 bit OS and runs a 4GB Virtual Address space. Similar to CE5, 2GB is reserved for the kernel and 2GB is reserved for user space.
Click to expand...
Click to collapse
Are you sure it's CE7 and not CE6? Tweakers.net (which appears to have received the same docs you have) says it's 6 and 6 makes much more sense to me being that CE7 hasn't RTM'd yet.

Yes, i'm sure it's CE7. The docs do in fact say CE6 but the docs are dated, things move fast, and that's no longer the case
CE7 "Chelan" will likely RTM around the same time WP7 does.

Interesting. That's a lot of risk for the WP7 team to take on.

Windows 7, Windows Phone 7, so CE must be 7
What do u mean by "much more rich apps". 3D Games or what? I think that aplications on WM 6.5.5 are good and fast. Can u explain that rich applications?

where do you get this info from.. it does sound interesting
Keep it coming

CE7 supports directx 9 acceleration. XNA & SL 4. Do they sound any familiar?

Awesome information! Thanks and keep it coming; I can't wait for WP7 to be released!

Da_G said:
This should allow for much more rich apps to be developed (whenever MSFT decides to allow us to write native code, which is coming at some point...)
[...]
(presumably around the time native code is allowed).
Click to expand...
Click to collapse
what makes you think that microsoft is going to allow developpers to write native code on windows phone 7?
the whole wp7 sandbox architecture is based on .net, so it's hard to imagine that microsoft would allow native code...
aren't they going to allow it only on windows phone 7 "business edition", which much users won't have ? that would make sense...

if they do that then we are in luck the guys on here will eventually pull it apart and give it to us all, i serously doubt thats going to happen tho, i suspect 6.5 code base will continue side by side until devs support WP7 at which point there wont be such a big outcry.

FWIW I was told by members of the WP7 team that it's CE6. They weren't directly related with that aspect and weren't 100% certain (more like 99%) though.

RustyGrom said:
FWIW I was told by members of the WP7 team that it's CE6. They weren't directly related with that aspect and weren't 100% certain (more like 99%) though.
Click to expand...
Click to collapse
A MS spokesman has confirmed it is running on a hybrid CE6 R3 (meaning CE6 R3 + added features that are from CE7). No doubt in my mind that a update in a year or so after launch they will update it to the full CE7.

According to the OS Version that the device returns, its running CE 7. Check around the 6 minute mark here: http://live.visitmix.com/MIX10/Sessions/CL16

Sangheili said:
According to the OS Version that the device returns, its running CE 7. Check around the 6 minute mark here: http://live.visitmix.com/MIX10/Sessions/CL16
Click to expand...
Click to collapse
It says 7 as in Windows Phone 7.

While this is quite interesting, I have a couple of questions about this. First, since each process is now free to use the entire slice of memory assigned to userspace, does this mean that it now conducts multitasking similar to iPhoneOS and Android where it saves the application's state when backgrounded instead of the entire process itself? How does the OS handle background activities, such as background network transfers and job completions?
This leads me to my second question. Since memory allocation is done on a round-robin basis, does this mean that all backgorund activity (if backgrounded) must be completed within a certain amount of time? (For those unfamiliar with OS concepts, round-robin scheduling is a method some OSes employ where it gives a pool of jobs a set amount of time, a quantum, to complete their work. If some job in the pool doesn't finish by the end of that quantum and another job needs resources, that job is killed off in favor of the new one. More information here.) Or is the activity suspended until whatever process needs the memory is finished using it?
Thanks for the scrutinous research!

If you're referring to 3rd party apps, there is no multitasking for 3rd party apps. Depending on what the user does, the app will be "tombstoned" (application state saved).

RustyGrom said:
It says 7 as in Windows Phone 7.
Click to expand...
Click to collapse
Why would it say "Windows CE" and then append a version number that doesn't apply?
Anyway, even if it runs a WinCE 6 hybrid kernel, I hope there's SMP support there. I'd like to see a Tegra based phone. Such a device however would be a terrible waste of CPU power, if the system can't use both cores.

Related

WP7 -IS- Backwards compatible (well almost)

Applications that were made for Windows Mobile 6 are compatible with Windows Phone 7 Series. The interface of the new mobile operating system has been changed though, so the user interface for these applications will have to be changed as well.
"So there is no reason why programs written for Windows Mobile 6 cannot run on the new version of the OS", said Maarten Sonneveld of Microsoft Netherlands to Tweakers.net. "The interface is complete different though, so the applications will have to be changed somewhat before being ready for Windows Phone 7 Series".
It is still unclear how developers can port their user interfaces to the new version of Windows Mobile. Microsoft will only disclose how applications can be developed and distributed at their developer event Mix2010.
Microsoft announced it’s new OS on Monday afternoon at the Mobile World Congress in Barcelona. The OS is primarily aimed at synchronisation and integration with Microsft-services like Windows Live, Bing, Zune and Xbox Live. Aside from those Windows Phone 7 Series can also synchronise with Google-accounts and facebook.
Click to expand...
Click to collapse
Source
So in summary, while none of the current applications will run on it, the underlying non-UI APIs will be compatible. So if understand correctly, porting would just a case of redeveloping the UI then recompiling, rather than starting completely from scratch. This acts to filter out apps with no more developer support, and promote a consistent UI.
Doesn't sound too bad to me.
That might explain why TomTom was seen on that screenshot of WP7 running on the HD2 (although, it could be a fake!). TomTom takes control of the screen, so uses no WM interface elements. So, it might be able to run full-screen apps/games without changes.
But, who knows...
elyl said:
That might explain why TomTom was seen on that screenshot of WP7 running on the HD2 (although, it could be a fake!). TomTom takes control of the screen, so uses no WM interface elements. So, it might be able to run full-screen apps/games without changes.
But, who knows...
Click to expand...
Click to collapse
I was just thinking the same except if you use the included .net controls, there's no reason that the OS couldn't just reskin them automatically to be at least somewhat more in line with the WP7 styling.
This would be excellent if it's true - and I can't see why it wouldn't be. The UI may be new but why throw away a perfectly good underlying core.
What would also be ideal is if the "multi-tasking" involved an app being set to pause in the background by default, but with a "keep me running" API call available for apps that needed it. I'm sure most apps hog resourses not because they need to but because the developer hasn't really thought about how the rest of the device performs when his app has been left running.
Makes sense, WindowsCE core is still the same
Zaim2 said:
Applications that were made for Windows Mobile 6 are compatible with Windows Phone 7 Series
Click to expand...
Click to collapse
Absolutely wrong statement due to incorrect translation. Original: "De interface van Windows Phone 7 Series is totaal anders, waardoor er in elk geval iets aan de applicaties moet gebeuren voordat ze geschikt zijn voor Windows Phone 7 Series"
Even google translates it correctly:
"The interface of Windows 7 Phone Series is different, which in any case something should happen to the applications before they are suitable for Windows 7 Phone Series".
We have some "ms confidential" documentation dated January 2010 that proves that none of the existing apps would be compatible with WinPhone7. And the only programming suite that is available to "generic" application-writers is Silverlight+XNA. Native apps are prohibited. Only OEMs and MO are allowed to create them (and even they have a set of limitations).
We would not have even source code compatibility - as all our C++ apps have to be converted to .NET.
mamaich said:
We have some "ms confidential" documentation dated January 2010...
Click to expand...
Click to collapse
What the heck? And you say that only now? What else is in there? Any word about how background tasks are handled? Please give us some more information, or maybe, can you upload that documentation?
freyberry said:
maybe, can you upload that documentation?
Click to expand...
Click to collapse
Obviously I cannot. As it would reveal the person who provided it.
Just to prove that such info really exists - see attached screenshots.
I really hope that the community would force MS to change such a dumb idea to limit independent software vendors to create only managed apps. Prohibiting C++ as a developing language, and "hiding" Windows API from programmer would force lots of developers to abandon this platform. The main reason of success of old WinMobile OSes was the ability to recompile "desktop" apps to WinMobile with just a minor set of changes (ANSI->Unicode + some interface changes).
P.S. I don't read PMs.
Obviously I cannot. As it would reveal the person who provided it.
Just to prove that such info really exists - see attached screenshots.
Click to expand...
Click to collapse
Well, there's certainly a way to remove that information. But anyway, what about background tasks? Are third party applications allowed to run in the background?
mamaich said:
Obviously I cannot. As it would reveal the person who provided it.
Just to prove that such info really exists - see attached screenshots.
I really hope that the community would force MS to change such a dumb idea to limit independent software vendors to create only managed apps. Prohibiting C++ as a developing language, and "hiding" Windows API from programmer would force lots of developers to abandon this platform. The main reason of success of old WinMobile OSes was the ability to recompile "desktop" apps to WinMobile with just a minor set of changes (ANSI->Unicode + some interface changes).
P.S. I don't read PMs.
Click to expand...
Click to collapse
Wow, I can't believe noone has picked up on this
freyberry said:
Are third party applications allowed to run in the background?
Click to expand...
Click to collapse
OS itself supports multitasking, see attach. But "Windows Phone OS 7.0 Application Platform" that we'll be forced to use to create apps may force our application to be paused in background. I never programmed Silverlight and XNA and can't tell how multitatsking is made in them.
WinPhone 7 == Zune Phone. Both are based on CE kernel, and they should have lots of common in implementation of multitasking, clipboard, etc.
OS itself supports multitasking, see attach. But "Windows Phone OS 7.0 Application Platform" that we'll be forced to use to create apps may force our application to be paused in background.
Click to expand...
Click to collapse
The question is, can we write applications that are not automatically suspended when sent to the background? What are the policies on this?
It says multiple processes can run at the same time, but it does not say whether they get suspended automatically.
Is there any info on this? Maybe in the "Scheduling" section?
I’m not sure this is a big deal. I can see them saying a lot of native C++ apps may have compatibility issues. I could go either way on it with the limited amount of information I have on this. I’ll have a better opinion at and after MIX
Note that this could be old documentation, and it’s pretty annoying you're leaking confidential documentation. Personally, I hope you get into trouble for breaking your contract - they trust you and you're posting it? Yuck.
To be fair, though, every app we’ve written has been managed, and Microsoft hasn't t said P/Invoking is verboten, so what would be the problem?
There’s probably exceptions for games and the like, and the documents you've scanned even say a waiver is available to use the Native APIs. So I don’t know what you're complaining about…
Microsoft's dev teams have been listening to developers - why not get them to chime in and also give them a chance to hear you. Posting confidential Microsoft documents, assuming those are real, is not the way to get them to listen
Best,
-Auri
freyberry said:
The question is, can we write applications that are not automatically suspended when sent to the background? What are the policies on this?
It says multiple processes can run at the same time, but it does not say whether they get suspended automatically.
Is there any info on this? Maybe in the "Scheduling" section?
Click to expand...
Click to collapse
Personally, I like Android's approach to this, where Services can run in the background, but UI apps are allowed to be "put to sleep" while other apps run. But then again, we may see a lot of that come into play come MIX and "Answer Time"
Best,
-Auri
Well, I am now both excited and nervous -I think I will just cool my jets until MIX10 and just enjoy the eye candy for now. At worst - if the interface is nice, but the core is crap I am sure some of the boys here at xda will make us an inteface port for 6.5.x that acts and looks like the new hotness with the old compatibility. - lets see MIX
AuriRahimzadeh said:
Note that this could be old documentation, and it’s pretty annoying you're leaking confidential documentation.
Click to expand...
Click to collapse
Docs are dated 2010.
I'm not leaking the documentation. I'm sharing the information that anyway would be opened in some days, maybe weeks.
And screens are posted here just as a confirmation of my words. You may think that these pics come from my mind and are made with photoshop - it is your opinion.
I really think that WinPhone 7 would be a failure similar to desktop Vista. Of cause some people would like it, but most would stay on WM 6.x and wait for the next version.
Regarding P/Invoke. As far as I've seen, "unsafe" operations are prohibited in XNA and Silverlight. Otherwise we would be able to call coredll funcs and run native apps (and do everything else that is allowed in our chamber).
mamaich said:
Docs are dated 2010.
I'm not leaking the documentation. I'm sharing the information that anyway would be opened in some days, maybe weeks.
And screens are posted here just as a confirmation of my words. You may think that these pics come from my mind and are made with photoshop - it is your opinion.
I really think that WinPhone 7 would be a failure similar to desktop Vista. Of cause some people would like it, but most would stay on WM 6.x and wait for the next version.
Regarding P/Invoke. As far as I've seen, "unsafe" operations are prohibited in XNA and Silverlight. Otherwise we would be able to call coredll funcs and run native apps (and do everything else that is allowed in our chamber).
Click to expand...
Click to collapse
Mamaich any though of a WP7 ce6.0 bsp for all the current cortex A8 devices running on a ce5.2 bsp, will the new kernel support them natively or will extensive bsp/bootloader hacking be required?
P/invoke surely is a limitation of .NET CF, rather than Silverlight/XNA libraries?
I think it would be a bit stupid to remove P/Invoking, because it's just not realistic to rely on .NETCF alone which has soooo much stuff stripped out to minimize size.
Will we be seeing a whole new .NETCF so soon after 3.5? I highly doubt it...Unless MS have been working overtime the past year
Shame, time to stop mobile development altogether if this is true. When we developers are considered as dumb earning pipes for companies who in their arrogant big ways think they have all the wisdom, and app developers only make annoying software that makes their precious leaky OS'es crash, it's time to move on. i would have been talking about IPhone, Android etc, but sadly we must add Microsoft to the list also.
Then there's the $1195,- and airplane tickets we have to pay to get to the Mix2010 in oder to maybe maybe get to be a "partner" with access to limited native API's (probably only reserved for the big companies) and don't even bother talking about giving away 30% of our earnings to a company that last year made how many billions of profits was it ?
Time to start an XDA OS based on REAL Linux maybe ? NVidia have a nice dev-board available for $400,- with Tegra on it. That's what I call developer friendly.
Cheers !
Regardless of how this will play out, I'm pretty sure of two things:
1. Closing down the OS may be beneficial for the majority of users by bringing stability, ease of use, UI consistency, etc. Even though I don't like it.
2. Because the OS itself is multitasking, any and all restrictions may be hacked around, and a "jailbreak" will be possible.
Depending on how this whole thing will be implemented, jailbreaking and using "illegal" apps may be a major PITA (think iPhone 3GS/tethered jailbreak) or as easy as a few registry tweaks/installing additional certs/whatever. If Apple didn't chase JB with every update it would be a rather good platform for both mainstream "ordinary" users and those who want rather unusual things from their phones.
We'll have to wait and see how it evolves really to make a final judgment.

Advantages over iPhone

Seeing as WP7 will be almost as crippled as the iPhone, let's see ways in which it will be better, besides replaceable battery and memory card(and it's not certain every OEM will follow up on those either). So far it has two weaknesses that only the iPhone has: Lack of multitasking and apps must go through the marketplace.
In order to pick up iPhone users it will have to offer some advantage that the average iPhoner will notice.
Some advantages:
Information at a glance a la today screen with the hubs. iPhone has nothing like this.
It will (supposedly) have some degree of multitasking.
Two more hardware buttons.
Its funny since I've had my HD2 I've not really used multitasking and when I had my iPhone only not being able to use Spotify in the background bugged me so maybe certainly for me multitasking isn't a be all and end all.
Having read lots of stuff about WP7S, the conclusion I have come to is this...
There will be two types of apps
1. Apps with no need to run in the background
2. Apps that do need to run in the background
Examples of type 1 are games, spreadsheets and word processors.
Examples of type 2 are IM apps like palringo, and music streaming services such as pandora.
What will happen is that when you develop an app, by default it will not have the rights to use the background APIs. In order to gain access to them and have an app run in the background, you'll need to ask Microsoft to provide the access and make it a type 2 app. Microsoft will only allow this if you can convince them it is necessary for the functioning of your app.
Type 1 apps will simply pause when the user switches away from them. They will remain in memory but will be unable to execute any code until the user switches back to them, whence they will resume execution. This will ensure the app cannot hog any CPU and cause the UI to stutter or slow down. This is definitely a good thing.
Type 2 apps are given access to particular APIs to allow them to, for instance, download updates or postings on IM systems. This will be strictly controlled and priority will always be given to the UI, again to ensure it remains smooth and responsive.
That's my take on what's going to happen, and we'll see if I'm right at MIX 2010 next month.
So your answer is - yes it will multitask but only when it is truly needed. Which to me is the best of both worlds. It will ensure a smooth user experience whilst still allowing background operations.
Jim Coleman said:
In order to gain access to them and have an app run in the background, you'll need to ask Microsoft to provide the access and make it a type 2 app. Microsoft will only allow this if you can convince them it is necessary for the functioning of your app.
Click to expand...
Click to collapse
Lets hope theyre not too stingy with giving out access to these API's!
The hubs/services (I'm not sure what MS is calling these) system looks good; getting new relative options available on multiple hubs just from installing a single app (like they demo'd with Facebook) should make all the apps work together much better than on an iPhone. I already want to try to make one to generate a music playlist based on past plays, and another to find lyrics to the currently playing song; If I understand the system properly, these would automatically integrate into any 3rd party apps using the appropriate media API's.
Also the context-sensitive search looks to be awesome.
One disadvantage: possible lack of native code execution and probably no OpenGL support - making it harder for iPhone app developers to port their existing apps to Windows Phone.
weesals said:
One disadvantage: possible lack of native code execution and probably no OpenGL support - making it harder for iPhone app developers to port their existing apps to Windows Phone.
Click to expand...
Click to collapse
why the heck should iPhone devs have an easy migration to WP7 if WM 6 devs don't?
weesals said:
Lets hope theyre not too stingy with giving out access to these API's!
Click to expand...
Click to collapse
The impression I'm getting so far is that they will be very stingy indeed. The only people who will ever get access to non-standard API's will be phone manufacturers and networks, and even they usually won't get access to the native API's most of the time. Microsoft will not publish any documentation about native API's. To get access to them the manufacturers will have to apply to Microsoft on a case by case basis. If Microsoft judges that a native API is required (and if there actually is one that might help) then only at that point will they release any information, and a condition of this is that they will vet the resulting piece of software to verify that the native API is being used correctly, and forbid the release of the software if it isn't.
What we don't know yet is where multi-tasking sits within all this. Is it a standard managed API, an extended managed API, or a native API?
why must every phone be compared to an iphone...personally I never liked the iphone, never will...only good thing about iphone is the apps..otherwise it sucks..and high end smartphones should not be compared to it!
The only thing I like about iPhone is how I use the virtual keyboard to type text.
I have tested HD2 and iPhone in a store, and from my own perspective, iPhone is more responsive and accurate compared to HD2.
I hope WP7 can be better than those 2 platforms in this task.
giggles33 said:
why must every phone be compared to an iphone...personally I never liked the iphone, never will...only good thing about iphone is the apps..otherwise it sucks..and high end smartphones should not be compared to it!
Click to expand...
Click to collapse
gogol said:
The only thing I like about iPhone is how I use the virtual keyboard to type text.
I have tested HD2 and iPhone in a store, and from my own perspective, iPhone is more responsive and accurate compared to HD2.
I hope WP7 can be better than those 2 platforms in this task.
Click to expand...
Click to collapse
that isnt aways based on the OS or software, but the quality of the touch screen.
Jim Coleman said:
Having read lots of stuff about WP7S, the conclusion I have come to is this...
Click to expand...
Click to collapse
This definitely seems like the best thing to do for multitasking in WP7.
We are going to need a task manager though...
As for comparing to the iPhone:
-WP7 will be available in different hardware configurations, giving the consumer a choice in the style and capability of their device.
-Xbox integration, which will most likely include Arcade games (ported for playability of touchscreens)
-Better hardware standards
-Not quite as locked down (hopefully)
RAMMANN said:
why the heck should iPhone devs have an easy migration to WP7 if WM 6 devs don't?
Click to expand...
Click to collapse
Because that's where the money is.
Seems people are struggling to come up with any, maybe something magical will appear in the next few weeks, although I doubt it, the advantages of WM seem like they will be gone with WP7, people on forums like this and blogs have been asking for a windows mobile iphone [without being an iphone] and it looks like they're going to heed the demand.
The most important advantages are gone.
They've made an exact copy and think it is enough. But it's not. When you try to catch up, you have to be better.
There's almost nothing WP7 is better at. It's an exact copy of iPhone OS with a better UI on top, but lacking the thousands of applications. That's not going to be enough and I really can't think about a reason why consumers and developers would be excited about this.
(and don't get me wrong - I LOVE the UI - it's just not enough)
Free Microsoft Office (Document viewing, creation, downloading, and editing)
Abobe Flash Player 10.1 is coming
File downloads (possibly)
Apps like a Wi-Fi router and file manager will likely come and be allowed
XBOX LIVE! Enough said.
Zune integration and support (I'm a Zune user)
1GHz Snapdragon is the processor minimum (This will lead to awesome apps and games)
WVGA display minimum (You might not care too much about this one, but I've seen the difference, and it's AMAZING!)
Bing search (That's just my preference.)
Contextual search (A handy feature, I suppose.)
There is not an app collection of 100,000 with most of which being totally useless. This means that you"ll be able to find the good apps.
Even if Microsoft won't allow apps like a Wi-Fi router and file manager, all we would need to do would be to get all WinPhone7 users on XDA to install the XNA Game Studio (and possibly the Win Phone7 SDK) and we could simply upload .ccgame files to XDA instead of .cab files.
giggles33 said:
why must every phone be compared to an iphone...personally I never liked the iphone, never will...only good thing about iphone is the apps..otherwise it sucks..and high end smartphones should not be compared to it!
Click to expand...
Click to collapse
I know! Why must smartphones always be compared to a simple feature phone! I've tried the iPhone/ iPod touch (3rd generation) at Best Buy stores, and, let's just say, they froze more and gave out more errors in 5 sec. than 5 WinMo devices did combined over the course of 2 hours. The iPhone's keyboard isn't too great either. It's (the errors thing) 100% true.
Jim Coleman said:
There will be two types of apps
1. Apps with no need to run in the background
2. Apps that do need to run in the background
Examples of type 1 are games, spreadsheets and word processors.
Examples of type 2 are IM apps like palringo, and music streaming services such as pandora.
What will happen is that when you develop an app, by default it will not have the rights to use the background APIs. In order to gain access to them and have an app run in the background, you'll need to ask Microsoft to provide the access and make it a type 2 app. Microsoft will only allow this if you can convince them it is necessary for the functioning of your app.
Type 1 apps will simply pause when the user switches away from them. They will remain in memory but will be unable to execute any code until the user switches back to them, whence they will resume execution. This will ensure the app cannot hog any CPU and cause the UI to stutter or slow down. This is definitely a good thing.
Type 2 apps are given access to particular APIs to allow them to, for instance, download updates or postings on IM systems. This will be strictly controlled and priority will always be given to the UI, again to ensure it remains smooth and responsive..
Click to expand...
Click to collapse
This is the right answer. Anybody who calms down would see that this makes sense. More Apple-like approval process for Type 2, free reign for Type 1
Shasarak said:
The impression I'm getting so far is that they will be very stingy indeed. The only people who will ever get access to non-standard API's will be phone manufacturers and networks, and even they usually won't get access to the native API's most of the time.
What we don't know yet is where multi-tasking sits within all this. Is it a standard managed API, an extended managed API, or a native API?
Click to expand...
Click to collapse
Yeah, you're talking about native vs managed stuff, which is not the same as simply allowing an app to have a background process. True, AT&T and HTC will have to apply to for native API use for stuff relating to making calls, etc, but that was only about OEMS and network operators.
Regular 3rd party guys, of which there are many, will be expected to get a way to do what they need on the device. Pandora we've seen in Music, you can expect apps like Palringo showing up in People
burnblue said:
This is the right answer. Anybody who calms down would see that this makes sense. More Apple-like approval process for Type 2, free reign for Type 1
Click to expand...
Click to collapse
Just because it makes sense doesn't mean Microsoft will act like that. In fact, I'm sure they will not.
The mass market will not benefit from every joe having all the API's because it's going make programs that cause glitches/crashes/memory leaks, etc. They are doing what they think is best for mass market and that is make sure things work well on the device and everything is user friendly with the least amount of hiccups possible. So that means more restrictions on us.
^^^ +1
Jim Coleman said:
What will happen is that when you develop an app, by default it will not have the rights to use the background APIs. In order to gain access to them and have an app run in the background, you'll need to ask Microsoft to provide the access and make it a type 2 app. Microsoft will only allow this if you can convince them it is necessary for the functioning of your app.
Type 1 apps will simply pause when the user switches away from them. They will remain in memory but will be unable to execute any code until the user switches back to them, whence they will resume execution. This will ensure the app cannot hog any CPU and cause the UI to stutter or slow down. This is definitely a good thing.
Type 2 apps are given access to particular APIs to allow them to, for instance, download updates or postings on IM systems. This will be strictly controlled and priority will always be given to the UI, again to ensure it remains smooth and responsive.
That's my take on what's going to happen, and we'll see if I'm right at MIX 2010 next month.
So your answer is - yes it will multitask but only when it is truly needed. Which to me is the best of both worlds. It will ensure a smooth user experience whilst still allowing background operations.
Click to expand...
Click to collapse
This neither solves problems nor guarantees anything though. Poor code is still poor code. Too many apps running is STILL too many apps running (slows the UI). MS can police neither. So, your #2 solution really makes no sense and has no advantages. MS has no way of predicting who will run what app and when on their phones. What if a user chooses to run several "Type 2" apps? Will you get some sort of error message? Will the MS police arrest you for ruining the UI experience? What happens after several years of approved type 2 apps hitting the market? Now were back to the same problems of WM.
Dude, we're talking about 1Ghz+, 512MB+ RAM phones here! You can run lots of apps without slowing anything down. Really, the "multitasking slows down the UI" argument is utter bull****. A good OS handles multitasking in a way that doesn't slow down anything. Restrictions are only necessary if the OS itself sucks. A good OS doesn't need them.

Windows Phone 7 - Introduction to the .xap (replaces .cab)

So, with WP7, we lose all support for the .cab and associated API as it exists now. Replacing it is the .xab format.
What's a .xap?
A .xap is a simple, every day .zip file, renamed to .xap. Inside, it contains the app and all relevant dependencies. There are a number of possible .xml files that could be included inside the .xap to determine things like required security access level, to tell the system which .dll contains the main() for the application, etc.
I believe the .zip also provides a container for the virtual filesystem available to the app (not sure on that, it may be stored in a separate container, have to analyze more)
At least initially, .xaps will only be available for deployment through the Marketplace.
Regarding preloaded applications by OEM/MO: Requirements are much more strict in this regard now due to frequent end-user complaints about "slow, laggy, etc" Stock ROMs. I know every one of you reading this knows what I mean Preloaded App Requirements (which will be distributed as .xap) as follows:
Maximum of 6 preloaded applications on the device, not to exceed 60MB
All preloaded apps must pass Marketplace submission process (some extended APIs are available to OEM/MO so the process is slightly relaxed in that regard)
The application(s) and all future updates must be free of charge.
The apps must launch without dependency on network availability.
The apps must persist through a "hard reset".
The apps must be updatable and revocable (!!!!) through the Marketplace.
The apps must notify the user at first launch of any capabilities to be utilized and get user consent (to access compass, accelerometer, network, etc.)
I've attached a .xap to this post for your examination. It's renamed to .zip for the attachment system to allow it.
Hehe.. this reminds of the "widgets" for Vista and 7 or the "apk"s for Android. Same stuff it sounds like Thanks for the info master Da_G
Does this mean .cab.pkgs are being changed too?
The .cab.pkg format remains intact for imageupdate (actually I haven't examined it in depth just yet, but all indications are that they have not changed .cab.pkg format)
Bump for visibility
Interesting...Wonder if there will be a process to convert some cabs to xabs.
Highly unlikely. xab's are silverlight applications meaning you have to use xaml , c# code and libabries all in one small zipped file. Cab's are Cabinent files that has an inf file that specifes what libabries and files are going to be enclosed in the file. To put it simply a xab is a standalone application that does not require extraction or installation to run and a cab is an application which requires an extraction and for its contents to be placed in specific areas in order for the dependents to find and use them.
Also to clarify. Local storage for xab's are not defined or stored in the xab file. they are defined by the silverlight runtimes which is handled by the os. As of now since there is little information as to how the windows phone internal structure is (apart from us knowing that windows phone will utilised microsoft unified storage.). on windows 7 and windws vista after u install the silverliht runtimes all xab's that request local storage is stored in <SYSTEMDRIVE>\Users\<user>\AppData\LocalLow\Microsoft\Silverlight\is .. Just note silverlight local storage works just like flash local storage. the only exception so far for windows phone is that u will not be able to access a lot of local directories just predefined stuff like music, pictures and documents.
Just before people get into bad habits; they are xap, not xab files. No relationship to cabs whatsoever save as a container format.
Da_G said:
Regarding preloaded applications by OEM/MO: Requirements are much more strict in this regard now due to frequent end-user complaints about "slow, laggy, etc" Stock ROMs. I know every one of you reading this knows what I mean Preloaded App Requirements (which will be distributed as .xap) as follows:
[*]Maximum of 6 preloaded applications on the device, not to exceed 60MB
Click to expand...
Click to collapse
That is just brain damaged. Pre-loaded apps add clutter, but they also cut down on cost. Choose your poison. Pre-loading has little to do with with speed penalties, when done properly. Frankly, if roms have the same ancient architecture under WM7, then Microsoft really needs some technical leadership replaced.
[*]All preloaded apps must pass Marketplace submission process (some extended APIs are available to OEM/MO so the process is slightly relaxed in that regard)
Click to expand...
Click to collapse
Now this is where some quality review comes in. It all depends on how good the standards are, and I dare say they will seem lower and lower as time passes. Hell, they're already admitting that OEMs will have relaxed standards.
[*]The application(s) and all future updates must be free of charge.
Click to expand...
Click to collapse
That's just silly. You'll get a bunch of lite software versions with next to zero shelf life instead of upgradable versions with marginal shelf life.
[*]The apps must launch without dependency on network availability.
Click to expand...
Click to collapse
what does this even mean? Does that mean no internet based app can be installed? All it really means is you have to quit gracefully if the network isn't available.
[*]The apps must persist through a "hard reset".
Click to expand...
Click to collapse
This is a good thing, but primarily a reflection of back when flash memory was in short supply. Haven't run into it in forever.
[*]The apps must be updatable and revocable (!!!!) through the Marketplace.
Click to expand...
Click to collapse
Well, updateable is good...but revocable? Maybe removable would be more consumer friendly. Makes me think of the PS3.
[*]The apps must notify the user at first launch of any capabilities to be utilized and get user consent (to access compass, accelerometer, network, etc.)
Click to expand...
Click to collapse
What I take from all of this is that
a) they want to drive more traffic through the marketplace.
b) they want to drive more traffic through Windows Certification
Good for the average consumer, great for Microsoft. Personally, the only point that has any value to me at all is a central marketplace. The rest of the bullets are ways for Microsoft to drive seperation between their brand name and many software vendor's crappy products.
ahhhha , sound interesting .
gguruusa said:
That is just brain damaged. Pre-loaded apps add clutter, but they also cut down on cost. Choose your poison. Pre-loading has little to do with with speed penalties, when done properly. Frankly, if roms have the same ancient architecture under WM7, then Microsoft really needs some technical leadership replaced.
Click to expand...
Click to collapse
I don't know, I wish MS enforced that same restriction on the Desktops OSes too. Nothing worse than getting a Dell or Sony PC full of preloaded gunk.
gguruusa said:
That's just silly. You'll get a bunch of lite software versions with next to zero shelf life instead of upgradable versions with marginal shelf life.
Click to expand...
Click to collapse
It's a big leap to come to that conclusion seeing as most software that ships with phone doesn't have additional charges. The restriction as I read it really means you just won't get a tonne of unwanted trial-ware on you shiny new phone.
Eoinoc said:
I don't know, I wish MS enforced that same restriction on the Desktops OSes too. Nothing worse than getting a Dell or Sony PC full of preloaded gunk.
Click to expand...
Click to collapse
maybe, but that same preloaded gunk cut the price of your dell and sony. While I don't like preloaded gunk, I don't like expense either. What I do like is being able to make the decision myself of how much gunk vs expense I am willing to tolerate.
It's a big leap to come to that conclusion seeing as most software that ships with phone doesn't have additional charges. The restriction as I read it really means you just won't get a tonne of unwanted trial-ware on you shiny new phone.
Click to expand...
Click to collapse
I agree it is target at no trial-ware. Any idea how people in the business world get around that? Lite versions of software (aka cripple-ware). Pay per use software. I'm sure there are other strategies. Frankly, if they enforce the ability to remove, I'm not that particular on how much gets pre-loaded. The fact of the matter is that the problem isn't how much crap comes with your phone; it is that you don't get to pick whether it is installed.
great find Da_G, so its XAB no more cabs
the0ne said:
great find Da_G, so its XAB no more cabs
Click to expand...
Click to collapse
XaB no.
XaP
tighoor said:
XaB no.
XaP
Click to expand...
Click to collapse
oops ..
How bad this is for the guys that dev here?
or... how good?
guessing .xap is short for XNA Application Package ?
vladimir2989 said:
guessing .xap is short for XNA Application Package ?
Click to expand...
Click to collapse
close, but no. In fact, it's actually a silverlight application package - it's been used for web stuff since silverlight released.
how to convert XAP to OEM/EXT package ?
I'm not sure what you mean by "OEM/EXT" package, but it's probably not possible. If you want to include an app with the phone, that *is* possible but the only way I know of is to include the XAPs in the ROM and then install them on first bootup. Probably not the best approach.

Max 2GB storage access for app !

There you go!
http://wmpoweruser.com/?p=15451
Techticker.co.uk have published this summary of the recently held UK TechDays seminar held by Microsoft to introduce developers to their new mobile OS.
While the 9 minute video by and large cover information we have heard earlier, one issue of note is that apparently each application will only have access to 2 GB of storage on the device, irrespective of the actual size of storage.
Don't really see that as a problem; the main storage hogs are going to be music/videos/pictures, which are handled through accesible libraries without this 2GB limit. Maps for GPS apps could hit the limit, although I think I see the envisaged scenario being more on streamed data with caching.
Yeah I don't see what's the problem here. Just more BS drama from WMPoweruser to generate traffic.
Tito662 said:
Yeah I don't see what's the problem here. Just more BS drama from WMPoweruser to generate traffic.
Click to expand...
Click to collapse
The problem is that it's just heuristic restrictions on the usage of the operating system.
Why would you limit the size of the space an application can use? Is the underlying OS FAT32 and Microsoft is worried that stupid app developers will store everything as one monolithic file and therefore break their app when the store reaches 2GB?
It doesn't make sense, and restrictions that don't make sense -- regardless of what they are -- are a problem.
Yea, this is pretty restrictive, but you guys make it sound like 512MBs. How many apps do you have installed on your device as of now?
Spike15 said:
The problem is that it's just heuristic restrictions on the usage of the operating system.
Why would you limit the size of the space an application can use? Is the underlying OS FAT32 and Microsoft is worried that stupid app developers will store everything as one monolithic file and therefore break their app when the store reaches 2GB?
It doesn't make sense, and restrictions that don't make sense -- regardless of what they are -- are a problem.
Click to expand...
Click to collapse
No, it's Silverlight Isolated Storage for saving stuff 'on disk'. SIS stores have specificed, finite sizes. Who knows, it might be fat32, might be exfat.
Spike15 said:
The problem is that it's just heuristic restrictions on the usage of the operating system.
Why would you limit the size of the space an application can use? Is the underlying OS FAT32 and Microsoft is worried that stupid app developers will store everything as one monolithic file and therefore break their app when the store reaches 2GB?
It doesn't make sense, and restrictions that don't make sense -- regardless of what they are -- are a problem.
Click to expand...
Click to collapse
This is limited by Silvelight...
but i don't think this is a big problem,this limitation can easy change by update
This isn't really accurate. They haven't fully decided on a max space. WP7 uses exFAT and isn't limited to a single file size. I believe this can be configured per app and developers will be able to request more than 2GB if their app needs it for whatever reason. 2GB is just the 'default' and isn't even enforced in the CTP. http://msdn.microsoft.com/en-us/library/ff402541(v=VS.92).aspx
My bigger problem with this isolated storage is that for no reason at all other than bad engineering it's incredibly slow. sviluppomobile.blogspot.com/2010/04/wp7-isolatedstorage-speed.html about 127x slower.
When I posted about this on the msdn forum, some MVP just basically told me to "use a webservice" and if i don't than this is not the platform for me.
I had issues with that when i was making some silverlight apps some time ago. But i figured them out. What microsoft is doing here with windows phone 7 is that they are using the phone to marry apps with their cloud counterparts. Apps like netflix ap mobile nytimes will use if at most 10 mb of local storage on the device. anyone that need more than 512 mb of local storage for their app, is doing something really wrong. in regards to speed of the local storage i have realised that each local storage created is essential a true isolated storage area. meaning its essentially a small partiton formatted in some format (might be fat) and all the process of reading from that and storing to it really gets some painful io problems. But it seems with windows phone 7 the unified storage engine seems to fix that. Essential from my views of the way my programs are working. when an app is launched. all other apps are essentialy frozen in their last state, so only essential apps and stacks are running. during this process the local storage is put into ram and the application access the program at close to the native speed of the ram. when a user hits the back button it essential takes the local storage and saves back to the slower storage medium and saves the state of the application..
hope this makes sense.
What do you plan on installing on your WP7? World of Warcraft?
Imho, you don't need more than 2 gigs per app, hell, even my Tomtom with all Europe installed on it fits on my 2 gigs microSD card.
Remember, it's a freaking phone!

[singularity]

[SINGULARITY] -
Singularity
Singularity (and the language of such Sing#) is a Microsoft operating system currently on codeplex as RDK 2.0 which is now core to this project - getting Sing# and Singularity to run on ARM (hd2) then can easily boot NT or anything and everything - essentially, NT will happen, but is irrelevant, as need to here first give MAGLDR an d HD2 ability to run Common Language Runtime AND Singularity (.ARM ver of .X86) -
GOAL= make ARM Singularity Kernel run on HD2 then run apps using this core as native apps or strap out onto whatever...
See update on last page of this thread.
ntonhd2 said:
Cotulla: repsonse to your question along with basic test build, just for compile practice run (check for errors), was succesfull; this is for ARM low level bootloader (ARMLDR ) which runs on ARM (hd2, ultimately here) and then grabs LDR (ntldr) then all other files (see my reply) then NTOSKRNL.EXE -> its attached for you to download on next page - thanks again for your input .
NT on ARM:
http://www.microsoft.com/presspass/press/2011/jan11/01-05SOCsupport.mspx
http://www.microsoft.com/Presspass/Features/2011/jan11/01-05SinofskySOC.mspx
http://www.bloomberg.com/news/2010-...ion-of-windows-for-arm-chips-at-ces-show.html
http://thecoffeedesk.com/news/index.php/2009/04/23/net-could-be-key-in-windows-on-arm-netbooks/
http://www.osnews.com/story/24165/Windows_NT_on_ARM_It_s_a_Server_Thing
Please also read my last post regarding Xbox running NT.
And understand I AM TALKING ABOUT NTOSKRNL with Native CLI and not running full WindowsXP or 7 or watever! .
hi xda, put this in hd2 general as could be relevant to linux or wp7 or hd2. Thinking of starting project here of pretty grand scale if people are interested. Now that a lot of work has already been done i think it will not be as hard as it may appear or sound at first.
I am thinking about using new wp7 bldr +- oal +- nk.exe to set up emulation of bios expected on pc then trying to jump to 2003 server equiv ntoskrnl.exe. (and then probably just a native command line interface like alex ionescu tinykrnl project back in the day, a ncli for nt with usb keyboard and not much more to start with: Further dev much later).
Nk will handle underlying lack of pci, bios, ints, and addresses, (+is firmware) but actual switching to nt kernel is for real after that: To build a strapping kernel with ce7/wp7 architecture and initial drivers that goes on to then launch full nt kernel.
Yeah - i have \nt\private\ntos\ source code and no it is not the normal nt4 or other w2k leak- it is a complete and buildable kernel; pm me and i will give proof, or the code if you can build and want to work on this. This is not x86/x64 work obviously so is not for those without ability: Need to do some heavy lifting to get recompile build happening for arm, qualcomm ' snapdragon nt :d. Otherwise is only emulation and not a good idea. This is 2be real. As non-x86/x64 support for nt (nt4 did ppc, mips, and now ia64) this kinda porting is not a foreign concept: There is sufficient info out there with reference to everything from softpc.new (inside ms code) to wow64cpu.dll and other x86/x64 specific init routines, spinlock and interrupt handling, asm code samps, bochs methods, qemu methods, et.al. Which can be used in one way or another or taken over if required: If all taken into account to paint big picture: Use of emulation technology methods for non-emulation project just opens up underlying logic. That is it. This is also why i suggest using wp7/ce7 base 4 init. Do not want emulation. Real deal here only. I refer to all these items above as observations which could be taken into account if need be: From tinykrnl, reactos, bochs, wine, efi, and other such things can make porting over kernel easier: At the end of the day, ce7/wp7 ' bldr, oal, nk.exe (or whatever derivatives thereof) will be 'firmware' in big picture. Another reason i am considering wp7 as base to strap is drivers are there to make a ce+bios or efi-type (?) pre-loader that takes all ce7 initialization further and passes on to nt (nk.exe runs including all setup as would be done by ntldr, a fake or psuedo-real ntdetect.com, system.hiv then passes data structs to our ntoskrnl.exe) and do all that needs be done. I can handle pc side completely but need bit of help with someone who gets nkglobal and other structures and use of platform builder with experience prefered in creation of new bsp. Maybe other ways - instead of ce, ie- grub, linux, openbios, openefi, but either way just want to prove it could be done is all.
Click to expand...
Click to collapse
anybody here capable?
to quote Da_G:
Yup, RustyGrom pretty much has it covered. First, it's called "CE" for Compact Edition, and this is not a misnomer in any way. The system is designed to be as compact as possible (There are build-time switches for everything, so you can toggle off nearly all the components to acheive a very "light" image) obviously, including drivers for components not present would be a waste of space, as they would never get used. So there are none included. On the PC side of things the BIOS provides a basic level of functionality using a standard interface so generic drivers are created to bring the platform up to that level, and from there vendor-specific drivers can be loaded.
If you want to put an embedded device in terms of a desktop computer and loading Windows 7 on it, you start out with a fully assembled computer (video card, motherboard, cpu, ram, etc.) - power it on. It loads up the BIOS which initializes the basic hardware and begins to load the rest from the hard drive. The embedded device loads up the NAND XLDR, which provides only flash read/write support. The XLDR then loads the "EBOOT" or "IPL" into ram on typical devices. HTC doesn't use the EBOOT/IPL model as such (here already we're breaking away from the "standard" even further) and instead has that split out into mARM AMSS (a custom designed RtOS that loads and runs the Modem ARM CPU) and SPL. Once the AMSS loads the SPL into ram and executes it, the SPL initializes the aARM (apps ARM CPU), does various checks (are we in update mode? do we need to expose a flash interface to update the rest of the OS? do we just boot up the os and move aside?)
Then finally you get past the highly device-specific code and on to the (slightly) more generic CE Kernel/drivers which get copied into ram by the SPL and executed (Native Kernel/XIP partition)
So, how different is CE7/WP7 from that model? (Which is the model we have now in CE5.x/WM6.x) - The mARM AMSS provides a different interface and initialization proceedure. That means any of the WP7 drivers from a donor device we might port from would not work at all with our current AMSS. Which in turn means no boot without re-writing the drivers/kernel or AMSS.
So to compare it to a desktop PC once again, we need to write a BIOS, a Hardware Abstraction Layer, and a set of drivers for each component on the system (likely a good deal of the drivers would be usable once the rest is done)
Do I sound jaded yet? Yes, yes I am It's probably a factor of 10 more complicated than I thought it would be initially.
Here's the JTAG pinouts that need to be connected, btw. There are pins on both sides of the motherboard which also is truely a pain in my ****, as i originally intended to mount an external port on the HD2 so I could easily keep a JTAG connection with it, but you basically have to remove the entire motherboard to maintain a reliable connection, which really precludes running it on a live device.
Click to expand...
Click to collapse
JTAG working now .
Ummm expect to hear from Microsoft lawyers in 5....4....3....
RustyGrom said:
Ummm expect to hear from Microsoft lawyers in 5....4....3....
Click to expand...
Click to collapse
Yeah i would be in breach of the non-disclosure-agreement i signed so removed.
But i am in inner city cbd wifi hotspot area and jump around unsecured cafe signals and other businesses and also use proxy servers and..... on top of that..... my own added tweaks for safe measure!
so, cafe+wifi+proxy, +other_anon, means there is absolutely no chance.
RustyGrom said:
Ummm expect to hear from Microsoft lawyers in 5....4....3....
Click to expand...
Click to collapse
reading your stuff on ce7. is this a bad idea you think? or not possible? no interest? i think it can be done.
ntonhd2 said:
reading your stuff on ce7. is this a bad idea you think? or not possible? no interest? i think it can be done.
Click to expand...
Click to collapse
I just don't think it's possible or worth it to bother trying to port NT to ARM while Microsoft is doing the same already. You're not going to be able to put together the team required meanwhile hiding from MS. It's just a stupid idea imo and really has no benefit. I mean what's your end goal here? To run Win7 on our devices?
Judging from this and other posts you have made, I suspect the most "source" you have is the "Windows Research Kernel", which is the source for a portion of ntoskrnl.exe from Server 2003 SP1, approximately. That would be no-where near enough, and it's not even enough to compile "just a kernel". It actually has a number of pre-compiled parts that it just pulls in.
Not to mention such a project is just asking to get shot down in a legal firefight. The WRK is given to academic institutions for studying the world's most popular desktop kernel, and is done so under a strict NDA.
ntoskrnl.exe by itself isn't enough to produce a workable OS anyway, especially one from the Server 2003 era.
hounsell said:
Judging from this and other posts you have made, I suspect the most "source" you have is the "Windows Research Kernel", which is the source for a portion of ntoskrnl.exe from Server 2003 SP1, approximately. That would be no-where near enough, and it's not even enough to compile "just a kernel". It actually has a number of pre-compiled parts that it just pulls in.
Not to mention such a project is just asking to get shot down in a legal firefight. The WRK is given to academic institutions for studying the world's most popular desktop kernel, and is done so under a strict NDA.
ntoskrnl.exe by itself isn't enough to produce a workable OS anyway, especially one from the Server 2003 era.
Click to expand...
Click to collapse
Sigh.. why don't people read before they make these ridiculous and thoughtless posts? Realize that there are people from Microsoft ON these threads. Also, RESEARCH IN DEPTH BEFORE POSTING SUCH A THREAD.
snickler said:
Sigh.. why don't people read before they make these ridiculous and thoughtless posts? Realize that there are people from Microsoft ON these threads. Also, RESEARCH IN DEPTH BEFORE POSTING SUCH A THREAD.
Click to expand...
Click to collapse
There are more microsoft people on xda than most realize .
RustyGrom said:
I just don't think it's possible or worth it to bother trying to port NT to ARM while Microsoft is doing the same already. You're not going to be able to put together the team required meanwhile hiding from MS. It's just a stupid idea imo and really has no benefit. I mean what's your end goal here? To run Win7 on our devices?
Click to expand...
Click to collapse
sure, sourcecode factor (nda) and secrecy/MS are complexities: but not as hard as people think here: it is TWO COMPLETELY DIFFERENT THINGS TO TRY AND GET WINDOWS7-ON-ARM to what I suggested (NT-CONCEPT-ON-ARM-WITH-Native-CLI) and no I would not use WRK sourcecode (lol) as part of my daywork i have access to (not ce) full sourcecode.
see my last post here,
can be done .
hounsell said:
Judging from this and other posts you have made, I suspect the most "source" you have is the "Windows Research Kernel", which is the source for a portion of ntoskrnl.exe from Server 2003 SP1, approximately. That would be no-where near enough, and it's not even enough to compile "just a kernel". It actually has a number of pre-compiled parts that it just pulls in.
Not to mention such a project is just asking to get shot down in a legal firefight. The WRK is given to academic institutions for studying the world's most popular desktop kernel, and is done so under a strict NDA.
ntoskrnl.exe by itself isn't enough to produce a workable OS anyway, especially one from the Server 2003 era.
Click to expand...
Click to collapse
What does this statement really mean?
might be a bad idea on hd2, fine, accepted, but your comment at the end doesn't make sense to me. so, ntoskrnl.exe for wp7 or nt4 (another era than 2003 .net) would make a difference? that is silly. besides, i made it clear that a psuedo-firmware setup would be required to setup the datastructures that NTLDR would prepare (along with NTDETECT.COM, and bios+pci_bus+ACPI interaction, (plus system or setupreg.hiv)), etc: so what are you saying exactly? my point was to not run any win32 or win64 gui or subsystem. never even mention win32k, gdi, etc. I was very clearly talking about native cli (ntdll.dll) and a prompt- maybe usb keyboard- as ARM NT Conceptual. Please, enlighten me . PS> yeah, I know the wrk and am fully aware of \prebuilt\ libraries and obj code: but, no, I was not intending on using this as base. I admit, hd2 nt prob bad idea: btw was ARM NT concept more than anything! and yeah, with the secrecy and legal issues it would be too complex and overwhelming to do so, accepted, but if I were truly to do this NO i would not use WRK lol .
And regarding Microsoft, yes, I accept that there are a LOT of employees on xda and it is crawled and watched for obvious reasons: covered that.
PPS> re WRK, no, would (if i were to try doing this that is) use what I already have access to as part of my work> under full NDA I have full source to a few different bases including all of 2003 and even HyperVServer and AzureOS trees. .
unfortunately I do not have windows phone 7 code access though! Thanks.
RustyGrom said:
I just don't think it's possible or worth it to bother trying to port NT to ARM while Microsoft is doing the same already. You're not going to be able to put together the team required meanwhile hiding from MS. It's just a stupid idea imo and really has no benefit. I mean what's your end goal here? To run Win7 on our devices?
Click to expand...
Click to collapse
Yep...... but there is a LOT of portability in the original nt4 and even w2k trees with alpha, mips, ppc, os2+posix, original softpc.new+ntvdm, and even newer, that would let this be done a lot easier than most realize: remember here that:
I AM NOT SAYING LETS RUN WIN32 ON OUR HD2: I AM SAYING LETS TRY RUN NTOSKRNL ON ARM.
big difference guys.
RustyGrom, I assume your talking about ARM-Cortex etc (msnt-2-arm)..... THIS is what i wanted to do but a much more lightweight and ms-testing-protocol-free-process; homebrew version in experimental state would ensure much speedier development: it is not that hard a concept to attempt to port over an earlier (nt4 or w2k) kernel FIRST then look at better (2003 & 7) memory management etc: the point here is PROOF OF CONCEPT NT ON ARM: that is it, like what you refer to. Read my first post: any remember tinykrnl.org? Alex Ionescu ? Reactos? it could be done a LOT easier than you all think!
only NT on ARM official stuff i am aware of is this (rumour/talk/concept/theory/design atm):
http://www.microsoft.com/Presspass/Features/2011/jan11/01-05SinofskySOC.mspx
http://thecoffeedesk.com/news/index.php/2009/04/23/net-could-be-key-in-windows-on-arm-netbooks/
http://www.osnews.com/story/24165/Windows_NT_on_ARM_It_s_a_Server_Thing
If you know NT like i do- then you would see it could readily be done but yes, I admit I do not know enoug about 'phones'/ce-platform. That's why I started THIS THREAD HERE: to get some thought on the subject is all .
what then would be major problems to overcome then and this is assuming concept of say:
0). hd2 power on
1). ipl/equiv
2). hspl.
3). magldr
4). dft leo70 rom
5). bsp/oal, bldr/uldr, OS.NB ->(NK.EXE).
6). remap, reinit, load and place (prep) data structures expected by ntoskrnl.exe (osloader, detect, pci, bios, etc).
7). jump to ntoskrnl.exe
?
For the record, a few years ago i did this exact thing: ported nt kernel over to another platform. myself and others re-wrote ntoskrnl.exe (+hal+drivers) and integrated osloader.exe(ntldr), and all data structures as would be passed to kernel from ntldr, registry system hiv, ntdetect, missing bios, missing interrupt+dma+pci-bus+acpi+power, etc into one (debug/xdk) single DEFAULT.XBE.
it only worked on XDK debug kit xbox consoles with serial+scsi+128mbRAM (and a custom lpc debug mod) but it worked. using code from intel and tianocore EFI/UEFI toolkits (and bits and pieces from here and there) and concepts such as PALcode as used by non-x86 osloader (.exe not ntldr) for simulacrum bios/firmware you can pass a predefined set of structures to ntoskrnl and ensure processor regs etc ARE ALL GOOD AND SYSTEM IS READY then call into KiSystemStartup, ExpInitializeExecutive, and begin modified phase0 of NTOSKRNL.EXE.
similar thing was done with CE.NET for Xbox - a default.xbe with linux code b4 NK.NB0
worked and works .
anyway, how u wanna solve the next problems?
1)missing CL compiler for ARM with same set of features like CL for X86.
(CL version for ARM for WCE doesn't have all features supported and usually outdated)
2)this ARM compiler store exception info in other format (not SEH frames, but universal table for functions ".pdata")
3)which files u exactly wanna build for ARM? is it "ntoskrnl.exe bootvid.dll hal.dll"?
4)which final results u gotta got?
5)why u need touch WP7? u can just look to example code in Android kernel and implement something. so replace PC standard timer realization inside HAL.dll with QSD8250 specific timer code. it's much better to start.
how many ppl u have in ur team?
Cotulla said:
anyway, how u wanna solve the next problems?
1)missing CL compiler for ARM with same set of features like CL for X86.
(CL version for ARM for WCE doesn't have all features supported and usually outdated)
2)this ARM compiler store exception info in other format (not SEH frames, but universal table for functions ".pdata")
3)which files u exactly wanna build for ARM? is it "ntoskrnl.exe bootvid.dll hal.dll"?
4)which final results u gotta got?
5)why u need touch WP7? u can just look to example code in Android kernel and implement something. so replace PC standard timer realization inside HAL.dll with QSD8250 specific timer code. it's much better to start.
how many ppl u have in ur team?
Click to expand...
Click to collapse
************************************************************
update: Attached is ARM low level bootloader just built; this could be used to load LDR and then ntoskrnl.exe .
************************************************************
Please let me know your thoughts and please try to get this to run with debug if you can and pass results & thoughts back to me - Cheers. Hopefully it built ok. What do you think of using this method then? but with FULL & PROPER NTOSKRNL.EXE!
************************************************************
Hi Cotulla, thanks for your reply: appreciate it here.
[also much thanks for hspl, magldr, dft android, leo70ROM. .]
ok, sorry if this is a bit all over the place, i have cut and pasted my answers around to try clean it up but it is late and i think my brain is a bit dead sorry, but answers are here anyway . hope makes sense. firstly please have a look at this video and let me know what you think .
http://www.youtube.com/watch?v=RFNuY2OFRjU
that is ARM..... i am going through build environment and sourcecode now..... thoughts?
http://www.youtube.com/watch?v=n3v4YC9RT-g&feature=related
can learn a lot from wine. i agree with you on linux. same for virtualization, emulation, etc, like bochs qemu everything . sandboxing and hypervisor unveils a LOT . another thing i wanted to ask you was what do you think of FPGA technology for reverse engineering unknown systems? for example, if i were to start almost any project, like say leo70DFTrelease, or NT on Xbox, or whatever, doesnt matter, i think it is worth spending the time or money (for private company to do it for you) and have an FPGA version of the target device being hacked (hd2 in leo70rom case) and then undo the software problems from a hardware logic perspective. just the way i have worked on things many times and it works for me anyway. but I digress.......... . if i were to have done wp7hd2 (leo70rom) and magldr, then i would have had to have had (for me, not as good a dev as you) a FPGA based HD2 made up that ran in every way same but with which i could get right in there and do whatever i needed to do to see response& debug. let me know what you reckon... ok... digress now :
1)missing CL compiler for ARM with same set of features like CL for X86.
(CL version for ARM for WCE doesn't have all features supported and usually outdated)
what features specifically we need here?
what about tweaking this:
http://reactos.colinfinck.de/files/RosBE-Windows/RosBE-ARM-1.0.exe
2)this ARM compiler store exception info in other format (not SEH frames, but universal table for functions ".pdata")
http://www.reactos.org/wiki/PSEH
http://www.reactos.org/forum/viewtopic.php?f=9&t=5716
reading up on _IMAGE_CE_RUNTIME_FUNCTION_ENTRY. just going over stacks and frames and overall exception handling on ARM. are there any issues with reverse execute, virtual unwind? for this type of execution- how would you handle?
more to the point- how would you do this project lol.
problems with prolog/epi? what about moving over x86 asm code? i am right now typing this to you whilst getting updated on specifics on registerslooking at emulators to see this in action. i am reading these here. let me know if on right path and please put up links to whatev will make this project concept a reality . Cheers .
see here
http://www.cl.cam.ac.uk/~mwd24/phd/swarm.html
http://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html
http://www.codeproject.com/KB/threads/StackWalker.aspx?msg=2818356
can you recommend any compiler, emulator, os, setup, even equipment (JTAG etc etc) i should use, buy, try?
3)which files u exactly wanna build for ARM? is it "ntoskrnl.exe bootvid.dll hal.dll"?
depends on method: i agree (see below) that probably android or (htc-)linux is probably more likely to work but leo70_rom made me think maybe jump from (touch wp7) nk.exe? and are you saying use linux as in LinuxBios type setup?
would need emulated bios, pci bus fixed up (?), QSD timer HAL, ACPI (?), etc ,,, so probably would end up with the following:
a) BIOS (ce7 exe or linux ?): options here could be to make NT think it is running on PALcode, uEFI, or standard ACPI BIOS (your thoughts?). I think uEFI (tianocore/Intel) is best bet here perhaps. this would include MBR code (efi equiv or pal equiv depending) and any psuedo-real or "real" initialization i think.
b) mbr execution merged to and included in above, bootsect. in sim' 'firmware'.
c) $LDR$ @ OSLOADER.EXE (osloader.exe is non-x86 ntldr as im sure you know WITHOUT the code to run ntdetect.com and acts in PALcode architecture to pass on predefined data structues from firmware: tells NTOSKRNL.EXE where and what 2 execute).
d) HAL.DLL (timer, power/acpi, spinlocks, interrupts). another reason i leant towards WP7 as pre-NT launcher is because i assumed that something like BSP, OAL, etc, could be maybe used as base: if not for code, then logical base. what base(s) did you use to create WP7 if i may ask? ie: CE7? I have just installed Platform-Builder. but yeah, i here you regarding android/linux kernel example: ultimately are you saying better, easier, more logical, to go with android/linux you think Cotulla?
e) BOOTVID.DLL
f) KDCOM.DLL (if wp7 would make use of KITL?)
g) drivers as required including the following: ntbootdd.sys (?) might allow easier diversion from bios lack of INT13 and other support: remap to whatever can handle this properly. equivalents for ACPI.sys, filesystem drivers, other power, basics. how should i be looking at things from NT side of things, as in \ObjectTypes like \??, \Global?? etc .... and items like ROOT device in ARM (either CE or linux preloaded) context? any thoughts on how object manager would need to be brought up? for me, now, that is where it gets crucial and is core.
h)SMSS.EXE (NATIVE.EXE) but to begin with could just get drivers and all that working first and strap up into cmdcons (SPCMDCON.SYS). just blue-screen SMSS (windows setup) enough to prove kernel to run on ARM cpu. your thoughts?
i) SYSTEM reg key hive (setupreg.hiv etc?)
...
4)which final results u gotta got?
Tinykrnl type native CLI.
http://www.betaarchive.co.uk/imageupload/1193217573.or.99024.jpg
with USB keyboard support like htc-linux then go from there..... would love a prompt from which could just call any given call - be it CreateProcess or NtCreateProcess or ANYTHING: and it just does it (with debug/KITL) without question . but native NT command line is good for now. not going near win32.
5)why u need touch WP7? u can just look to example code in Android kernel and implement something. so replace PC standard timer realization inside HAL.dll with QSD8250 specific timer code. it's much better to start.
yeah....
I thought linux probably would end up being better: just liked symmetry of windowsCEx-strapping-windowsNTx: making a windowsCE-EFI/BIOS: but yeah, something like LinuxBios (android kernel etc) would be a lot easier in the end yeah? All this is overly simplified and very conceptual but there are basic answers. . once a solid idea has been formed then this could actually be done i think. and before Microsoft . Do you believe Reactos-ARM-build environment could be used? Am i missing anything? 9 people team+myself (+any help you can offer) would make 10 (+1). I think this is a good idea to at least try and i believe with your assistance, guidance, well, it would get done and then complete the HD2 line up fully. . In conclusion, right now, I need ARM emulator software, platform builder, and fully working Compact Edition 7 on HD2 to get some more thoughts and try few things out in platform builder debug then can get final decision, design, plan and start to get everything working. Even though will probably go with Linux/Android obviously as above, I still need 2see init on CE7 on HD2 and be able 2use this along with whatever else we can! have a look at all above links... thanks.
Cotulla, thanks again 4reply>please PM [email protected] something but not posting..... await your PM.
what about this ( http://research.microsoft.com/en-us/projects/singularity/ ) could be of use to NT port with respect to CLR ? haha, or just outright hd2 port Microsoft RDK OS ' singularity ' ? .
************************************************************
update: Attached is ARM low level bootloader just built; this could be used to load LDR and then ntoskrnl.exe .
************************************************************
Please let me know your thoughts and please try to get this to run with debug if you can and pass results & thoughts back to me - Cheers. Hopefully it built ok. What do you think of using this method then? but with FULL & PROPER NTOSKRNL.EXE!
************************************************************
I don't have big knowledge of Windows NT system, but I think it's must be enough to provide basic stuffs for kernel start up.
I guess NT using only int13 services for reading data from disk, int15 services used to detect memory configuration and int10 for initial boot mode.
Because it's embedded hardware, the devices in the system are fixed and limited. So it's enough to provide fixed values for kernel, like available ram memory range.
No need of using any complex systems with CE / Linux.
About CE, you can get almost full kernel sources in PB6.0, trial can be downloaded from MS site.
afaik it's enough to load kernel and dependent modules (drivers) to ram and then run them. after this action kernel drivers should able to run properly on hardware.
About Reactos, I appreciate work of involved people, but I doubt that it's stable
About this project, I don't know yet if I will contribute. I am looking how much it's interesting for me
I always have interesting different things in my hobby as well, so I have choose that to do As well, me is part of DFT team, I need discuss it with them
Now I am asking you to understand more details about your idea(s)
Cotulla said:
I don't have big knowledge of Windows NT system, but I think it's must be enough to provide basic stuffs for kernel start up.
I guess NT using only int13 services for reading data from disk, int15 services used to detect memory configuration and int10 for initial boot mode.
Because it's embedded hardware, the devices in the system are fixed and limited. So it's enough to provide fixed values for kernel, like available ram memory range.
No need of using any complex systems with CE / Linux.
About CE, you can get almost full kernel sources in PB6.0, trial can be downloaded from MS site.
afaik it's enough to load kernel and dependent modules (drivers) to ram and then run them. after this action kernel drivers should able to run properly on hardware.
About Reactos, I appreciate work of involved people, but I doubt that it's stable
About this project, I don't know yet if I will contribute. I am looking how much it's interesting for me
I always have interesting different things in my hobby as well, so I have choose that to do As well, me is part of DFT team, I need discuss it with them
Now I am asking you to understand more details about your idea(s)
Click to expand...
Click to collapse
sure....... . anything ReactOS -freeldr, any arm code, whatever, is just to get basic idea up- to see the actual jump whilst watching (be it by jtag, kitl, usb, or telepathy interface to QD) and go from there; although im sure you could use ReactOS arm code lowlevel bootloader to jump into EITHER "freeldr" or proper "ntldr" or "osloader.exe" (modified of course to have no pci bus scan and the rest.....) that is the dilemma: either jump COMPLETELY like winmo6-android with all structures setup DIRECTLY INTO KERNEL and avoid the whole LDR side of things in that sense anyway; or, well, totally from scratch rebuild loader and subsequently deal with 'firmware' issues... i really do not care in the end if its a jump from one kernel to another (one os to another) because project here is to RUN NT ON ARM/HD2 and not to necessarily have it homogenous down to LDR.
as long as thread, memory, native api, other calls, all that, is truly ntoskrnl = you are running nt on your arm hd2! .
LDR does not matter.... total new rebuild or jump.... whatever comes first .
Thanks Cotulla, yes, we understand where your coming from re do not need linux, ce, and complexities there and i agree: just want to use these for initial testing and deployment of early code with some kitl, debug.... on other notes, trying to put all into organized groups, slowly but surely yes, with bit of faith we will get there in the end .
if totally up to me i would probably take intel/tianocore EFI specification as the base if this could somehow be easily made to run on ARM in this particular context. ie EFI on a HD2!
look at this raw control power!>>> http://www.ami.com/support/doc/AMI_Debug_UEFI_Dsheet_PUB_2008-06-10.pdf
also along these lines, just briefly (is helpful in concept design):
http://x86asm.net/articles/uefi-hypervisors-winning-the-race-to-bare-metal/index.html
http://sourceforge.net/projects/gnu...orig.tar.gz/gnu-efi_3.0h.orig.tar.gz/download
http://x86asm.net/articles/introduction-to-uefi/
http://sourceforge.net/projects/efidevkit/
http://www.logic.nl/Products/Technology/BIOS-and-EFI.aspx
ok, summing up thoughts here>>>
0) object manager and objects; going over arm & ce7, as well as winmo6 and other ce, and comparing with nt and win32/64; just looking at how on final arm release, the \ObjectTypes will be different in the end. very interesting stuff.
1) LACK-OF. no pci bus which is highly expected by ldr/detect so make kernel prob see system in 'PALcode' or EFI mode. pass ldr data structs to kernel in that type of form. otherwise gets very messy and we are not going to hack around because you will end up with an emulator !. this will work but key is determing what 'firmware' passes this data to nt kernel - not from our perspective- but as NT.
2) BIOS. INT services are not used by kernel in that way after it becomes supervisor so will redo drivers unless preload remap somehow. INT only there during ntldr (or can load in ntbootdd.sys to supply these) and this is all pre-phase0 and is very early on.
3) HAL and clk
4) INT services are not used by kernel in that way after it becomes supervisor so will redo drivers unless preload remap somehow. INT only there during ntldr (or can load in ntbootdd.sys to supply these) and this is all pre-phase0 and is very early on.
5) kitl and kdcom
6) registry to pass on (setupreg).
8) filesystem, screen, other drivers
9) final native cli (ntdll.dll) or maybe initially just spcmdcon.sys.
above not in order ..... sorting it all out though .....
ok, looks daunting but like i said before you could get up an nt kernel in setup mode with setup ldr and drivers and old blue screen "dos" mode native subsystem which uses the SMSS.EXE and NTDLL.DLL that are seperately contained in \i386\system32\ or \cmdcons\system32\ - very limited subsystem but is full nt os at kernel . so........ if not ce and not linux preloading, WOW . it is quite an amazing project but doable; so basically just need to see how this armldr (low level strap - be it Reactos or my own clean job- will do both) code runs on the device itself and step by step add the rest in as required! but i still believe actual dev be better jumping from preexisting environment having kitl or some sort of serial or usb debug already there and then working way down to lowest possible level; so, basically, working backwards down to processor.
Doing it all from scratch and CLEAN . (in the end!). .
my brain just straight up exploded.
thanks a lot.
http://www.youtube.com/watch?v=xKc_XGuvNIk .
for the record:
so far without any errors have successfully been able to build the ntdll.dll, hal.dll, smss.exe, bootvid.dll, fastfat.sys, for ARM with no modifications at all, but not yet done a build on the LDR or NTOSKRNL.
just testing compiler here is all and not writing new: this is very early on and i have changed absolutely nothing.
once fill in gaps will give it a go on hd2.
attached.

Categories

Resources