I don't mean for this question to be a flame bait but I'll be using Microsoft and their win32 API as a example of a legacy API.
Now what I am wondering here is Microsoft is spending a lots of their money and energy in maintaining their legacy API, including all of the "glitches/bugs/workaround" that are needed to keep the API functioning the same. Now I'm aware that in Windows 7 they are providing a way for the customer to run their application in a "Windows XP" VM which would be one such way for them to start cleaning up their win32 API because they could then push all of the application into the "Windows XP" VM.
So now what I am wondering is, is it possible to virtualization a legacy API in such way that an customer/program can still access and use it, yet at the same time be able to take advantage of the newer version/API? Because as far as I understand it, if the application is ran in the "Windows XP" VM, it won't be able to access any of the newer API/feature of Windows 7.
The thing that puzzles me about this question when it comes up is that Windows has been doing this since NT came out in the mid nineties. This is how NT runs DOS and Win16 programs, and how it always has. The NTVDM virtualization layer runs 16-bit apps under Win32 with very little special support from the core OS. This is just one example - another is WINE, which as I understand it does a pretty reasonabl job of running windows apps on top of an API set which is very different from that of windows. So it is definitely possible.
The more pertinent question would be why Microsoft would consider it. In order for you to think it is necessary you have to think two things. 1) There is something better to replace the win32 API with and 2) Maintaining the Win32 API is a burden.
Both of these are questionable. In the case of kernel duties, such as accessing hardware and synchronizing and doing threads and processes and memory the Win32 API does a pretty good job, and is ultimately quite close to what the kernel really does. If you think there is a better API then that must mean there is also a better kernel. I personally don't think that NT needs replacing right now. For graphics and windowing, admitedly gdi32 is a bit long in the tooth. But Microsoft solved that problem by building WPF right alongside it. This then brings in the burden question. Well, sure there are two APIs to maintain, but if you virtualized GDI on top of WPF you'd still have to maintain both anyway so there is no benefit there. The advantage of running both in parallel is that GDI already exists and is already tested. All you have to do is to fix the occasional bug, whereas a new virtualization layer would have to be written and tested all over again, which takes time away from making WPF better.
In terms of maintaining back compat, that isn't as much of a burden as it sounds. It is mainly a test question - you have to test that the API behaviour doesn't change, but again - those tests have already been written, so it isn't really any extra work.
So, to answer a question with a question, why would they bother?
This is an interesting question, at least to me, here are some of my thoughts.
Your understanding is correct, an application running in the XP VM only has access to the Win32 APIs provided by XP in the VM. One of the many ways that I have seen Microsoft's approach to enhancing specific APIs is to create new functions with the enhanced/fixed functionality and name the new function by append Ex and even ExEx to the original name, for example
GetVersion
GetVersionEx
For functions that accept pointers to structures, the structures are 'versioned' by using the size of the structure to determine the functionality required, so older code would be passing a previous size of the structure while newer code would be passing in the newer larger strucure and the API functions accordingly.
I guess, the problem has become that it is no longer just differences in how an API works, but more integral to the functioning of the operating system and the internal structures which have changes significantly enough that arguably badly written code is effectively broken.
As to your actual question, I guess it would be quite tough. Even if one thought to let the OS adjust how it executes code based on a target OS version in the PE header of the executable, what would happen if a newer DLL was loaded into the process that targeted the latest OS, now how should the OS handle this when the code is executing? IMHO, I think this would be very challenging, one frought with pitfalls that would ultimately fail.
Of course that is just my raw thoughts on the topic so I might be 100% wrong and there is some simple approach that just did not come to mind.
Related
As far as I'm aware, Windows hasn't been decompiled by anyone yet. Obviously it's complicated, but surely it should've been done by now to some degree?
My thinking behind this is that if the end-user has access to the software, and the computer is able to run it, then even an obfuscated version of it must be obtainable?
I'm obviously missing something, I'm just not sure what.
There's nothing preventing Windows from being decompiled (apart from EULA and similar legal bindings, of course). As you noted, the code must run on the CPU at some time, and the CPU must read the code from memory, and you can read from memory too. Some parts of the system can be a bit trickier, since to run the OS you need to give the OS some exclusive priviledges (that's how most modern protected OSes work), but it's nothing that can't be worked around. In any case, there's not a lot of effort to prevent Windows decompilation - that would have barely any benefit, while making debugging, error reporting and similar harder. Microsoft even goes so far to provide a special debug version of Windows that's even more tailored for software development.
The main point is that there's little reason to decompile Windows. What practical use would such a massive effort have? And if you're a corporation that needs access to Windows source codes (for example, when developing embedded solutions), you can get them. Just because Windows isn't open source doesn't mean the sources aren't available.
If you're not someone who needs their own version of Windows (common in the times of Windows CE), there's even less of a reason to decompile Windows. You need to stick to the defined public APIs anyway - that's a good practice regardless of whether the software is open source or not. APIs are contracts - implementation details you'd get through decompiling aren't. They might very well change with the next security hotfix or such. This is especially important given how serious Windows is about compatibility - it's quite rare for an update (or even a new major release) to break compatibility with old software.
So, if you want to decompile Windows, there's nothing technical that's really preventing you from doing so. But you're looking at tens of millions of lines of source code that was compiled by very smart compilers, with bits of handwritten optimised assembly thrown around, tons of compatibility workarounds that might as well be outright obfuscation (remember, you don't get the comments - just the actually compiled code). Are you willing to spend a few hundred thousand hours to satisfy your idle curiosity? :P
I'm looking to develop a program that detects calls to certain Windows API functions and simply records the calling process, call count, and hopefully their arguments, to later mark them as benign or malicious.
The GUI program API monitor is a good example of the functionality I'm trying to achieve. Ideally I would like to track each desired API function individually and get the caller PID and parameters when or after it is used, without user input. The program should be able to run on any windows 7 machine, but can be limited to 32bit applications.
I understand there are several methods of hooking a function, and from my understanding Microsoft detours implements one of these, but I don't know if its the one best suited to what I want to do. I've seen detours, easyhook, deviare API hook, and others mentioned on very old posts, but I have a hard time getting my head around the differences and features of each.
So my question is, given what I'm trying to do, what do you recommend and why?
For reference I'm an intermediate level programmer, but a beginner at Windows programming.
Thanks for your help
I'm part of Nektra Deviare API Hook team. Our hooking engine is used by a large number of companies all over the world, in different types of end-user products (e.g.: Anti Virus, Data Loss Prevention, AI, handicapped software, Data Classification, App virtualization).
Deviare-InProc is the MS Detours replacement and Deviare2 has built in all the RPC you need to hook another process and get the calls in your own process.
We continuously fix all reported issues. You can verify it in our GitHub:
https://github.com/nektra/Deviare2
https://github.com/nektra/Deviare-InProc
You can see Deviare2 running in Nektra's SpyStudio API Monitor.
Detours is an excellent software but very expensive (USD 10k). In addition to this, it completely lacks of support. It can be compared to Deviare InProc,
EasyHook used to be a good start point because it was the only free option. But, now Deviare2 family is open source and EasyHook has a lot of stability issues for the real world.
Does Windows 10 support running older Win32 (MFC, ATL, Visual Basic 6) applications on ARM processors? Does it require some form of emulation or conversion?
There's no x86 Win32 emulation at all. You need to use a toolset designed for the platform.
As with 7/8.1 Windows has leaned further and further into the Net way of doing things. So many of the commandline functions are done through net calls.
Also note that Win10 is pretty much Win NT, it is basically what Win98 should have been, to save us the disasterous influx of virus's on what was an OS with a swing door and no form of protection.
That NT side of things will affect all programmers in time, particularly over the following,
The rights of your users. This is a good thing because we have all been frustrated at our users leaving the doors open for virus and hacking. NT at least helps elliminate a lot of that.
File handling. Win10 is a big step closer to an OS on demand (Which is Microsoft's current target), so we can not assume items that our software makes use of will always be locally present, so we must go through the .NET route ready for when ondemand comes in properly so that the OS will handle the demands for us. Though it does worry me that we currently have no real clues as to how that will be handled if the request can not be full filled.
But also we can not be lazy with file access rights. For example we tend to make assumptions in the user's area about access rights, then get bitten in the bum when we do a scan or search of all directories, only to find DirectoryInfo.GetDirectories is unuseable unless we make sure special folders will not stop it part way through.
Since all directories will in time be special folders, we need to be handling the access rights on the work we do now. More easily done in C++ than C# im my opinion.
So, if you have done it in 'Managed' code then it ought to go anywhere that C# and VB go, call my synical if you like, but I can not help but have doubts about that, I can not really see MS finding it desirable to have on-demand applications and OS on NET but also providing Win32 wrapped in MFC running as an alternative. You may find your code is trapped in a shrinking box.
I am developing socket reading on an ActiveX EXE (i.e on a seperate thread).
How many sockets i can safely read independently?
I am working on windows XP OS.
I think this might be an operating system limit - I think I remember running up against a limit of 80 sockets on the XP machine I was using several years ago.
I would recommend that you abandon your effort and go with a commercial solution. I remember going down this path back in the 90s and running into a brick wall with ActiveX EXEs as far as threading goes. The thing is that ActiveX EXEs are apartment threaded, not free-threaded, so you don't get completely independent threads.
And doing server side threads properly is hard enough in modern languages, let alone ones that weren't designed for this purpose.
I ended up purchasing Server Sockets from Dart. Easily the best investment for that project. The performance is truly great - you are only limited by the system resources.
MSWINSCK.OCX is a very old way of doing things; it came with Visual Basic 6.0 and i remember using it way back when. i'm not sure the licensing on it... apparently it registers fine under 32-bit win7, but not 64-bit; here's a link to how to get it to register on 64-bit systems: http://angrybyte.com/windows-hacks/mswinsck-ocx-for-64-bit-windows-7-vista/
if you have an MSDN subscription or similar that gives you the ability to download the developer tools (bizSpark, etc. will do it too) then i believe that will also give you a license to redistribute the .ocx.
(btw, i don't actually remember the interface, but i seem to remember it being at least slightly more intuitive than the berkeley socket() interfaces.)
however, personal recommendation given your requirements: learn the APIs, there are lots of examples out there, and just write yourself a class that encapsulates them in a similar way as, say, the .NET Socket class... the APIs aren't that hard and i'm sure there's lots of help to be had here as well, and that's probably better than relying on something that's out-of-date like the control...
The Free MS Windows replacement operating system ReactOS has just released a new version. They have a large and active development team.
Have you tried your software with it yet?
if so what is your recommendation?
Is it time to start investigating it as a serious Windows replacement?
Targeting ReactOS specifically is a bit too narrow IMO -- perhaps a better focus is to target compatibility with WINE. Because ReactOS shares so many of its usermode DLLs with WINE, targeting WINE should result in the app running just fine on ReactOS.
Of course, there will always be things that WINE can't emulate well (hence the need for ReactOS). In this way, it seems that if something runs in WINE, it will run in ReactOS, whereas the fact that something runs in ReactOS doesn't mean that it will necessarily run in WINE.
Targeting WINE is well documented, perhaps easier to test, and by definition, should make your app compatible with ReactOS as a matter of course. In this way, you're not only gathering the rather large user base of current WINE users, but you're future-proofing yourself for whenever anyone wants to use your app with ReactOS.
In their homepage, at the Tour you can see a partial list of office, tools and games that already run OK (or more or less) at ReactOS. If you subscribe to the newsletter, you'll receive info about much more - for instance, I was quite surprised when I read most SQL Server 2000 tools actually work on ReactOS!! Query Analyzer, OSQL and Books Online work fine, Enterprise Manager and Profiler are buggy and the DBMS won't work at all.
At a former workplace (an all MS shop) we investigated seriously into it as a way to reduce our expenditure in licenses whilst keeping our in-house developed apps. Since it couldn't run MSDE fine, we had to abandon the project - hope in the future this will be solved and my ex-coworkers can push it again.
These announcements might as well be also on their homepage - I couldn't find them after 5 mins. of searching, though. Probably the easiest way to know all these compatibility issues is to join the newsletter, or look for its archives.
I have been tracking this OS' progress for quite some time. I believe it has all the potential to really bring an OSS operating system to the masses for it breaks the "chicken and egg" problem: it has applications and drivers from the very beginning (since it aims to have full ABI compatibility with MS Windows).
Just wait for their first beta, I won't be surprised if they surpass Linux in popularity really soon after that...
Post Edit: Found it! Look at section Support Database, it's the web place to go look for whether a particular piece of hardware of some program works on ReactOS.
ReactOS has been under development for a long long time.
They were in some hot water earlier because some of their code appeared to be line by line dissasembly of some NT kernel code, I think they have replaced all of it.
I wouldn't bother with cross platform testing until they hit the same market penetration as Linux, which I would wager is never.
Until ReactOS doesn't randomly crash just sitting there within 5 minutes of booting, I won't worry about testing my code on it. Don't get me wrong, I like ReactOS, but it's just not stable enough for any meaningful testing yet!
No, I do not think it is time to start thinking of it as a Windows replacement.
As the site states, it's still in the Alpha stages. More importantly, whos Windows replacement? Yours? Your users? The former is one thing, the latter is categorically a no-go.
As an aside, I'm not really sure who this OS is targetting. It has to be people who rely on Windows software but don't want to pay, because people who simply don't want Windows can use MacOS / Linux, and the support (community or otherwise) for these choices is good.
Moreover, if you use Linux you already have some amounts of Windows software support via Wine.
Back to people who rely on Windows software but don't want to pay. If they are home users they can just simply pirate it, if they are large business users they already have support contracts and trained people etc. It's hard enough for large businesses to be OK to update to new versions of Windows, let alone an open source replacement.
So I suppose that leaves small businesses who don't want to obtain illegal copies of MS software, can't afford the OS licences and rely on software that only runs on Windows and has bad of non-existent Wine compatibility.
It is a useful replacement for Windows when it runs 'your' software without crashing. At the moment it is not a general purpose os as it is too unstable (being only alpha) but people have used ReactOS successfully in anger for specific tasks already. As a windows replacement it has multiple potential uses, sandbox systems, test and development systems, multiple virtual instances, embedded devices, even packaging/bundling legacy apps with their own compatible o/s. Driver and application compatibility, freed from Microsoft's policy of planned obsolescence and regular GUI renewal, what's not to like?