In C/C++, Windows executables are linked against static libraries that import DLL files containing Windows API procedures.
But how do we access those procedures from Forth code (e.g. GForth)? Is it possible at all?
I'm aware that there's Win32Forth capable of doing Win32 stuff, but I'm interested how (and if) this could be done in Forth implementations that lack this functionality from the box (yet do run on target OS and are potentially able to interact with it on a certain level).
What currently comes up to my mind is loading the DLL files in question and somehow locating the address of a procedure to execute - but then, execute how? (All I know is that Windows API uses the stdcall
convention). And how do we locate a procedure without a C header? (I'm very new to Forth and just a bit less new to C++. Please bear with me if my musings are nonsense).
In general case, to implement foreign functions interface (FFI) for dynamically loaded libraries in some Forth system as extension (i.e., without changing source code and recompilation), we need the dlopen and dlsym functions, Forth assembler, and intimate knowledge of the Forth-system organization and ABI.
Sometimes it could be done even without assembler. For example, though SP-Forth has FFI, foreign calls were also implemented in pure Forth as a result of native code generation and union of the return stack with the native hardware stack.
Regarding Gforth, it seems that in the version 0.7.9 (see releases) it doesn't have FFI for stdcall calling convention out of the box (it supports cdecl only), although it has dlopen and dlsym, and an assembler. So, it should be feasible to implement FFI for stdcall.
Yes, you could do this in Gforth according to its documentation. The biggest problem will be dealing with call backs, which the Windows API relies on rather heavily. There is an unsupported package to deal with this, see 5.25.6 Callbacks. I have not attempted this myself in Gforth, but the documentation looks adequate.
You might also want to check MPE's VFXForth. From their website:
Windows API Access
VFX Forth can access all the standard Windows API calls, as well as functions in any other DLLs. The function interface allows API calls to be defined by cut and paste from other language reference manuals, for example:
EXTERN: int PASCAL CreateDialogIndirectParam( HINSTANCE, void *,HWND, WNDPROC, LPARAM );
EXTERN: int PASCAL SetWindowText( HANDLE, LPSTR );
EXTERN: HANDLE PASCAL GetDlgItem( HANDLE, int );
This is down the page a bit at VFX Forth for Windows.
As I do my Forth on Mac and Linux, I can't work through the Windows for Gforth to provide more detail, sorry.
Gforth 0.7.9 provides Windows API calls generated by Swig from the Windows header files. The C interface uses a wrapper library, which is compiled by the C compiler, to pass parameters from the Forth stack to the system functions; as the C compiler understands stdcall, and the header files declare Windows API as stdcall, this "just works".
As all pre-generated C bindings live in the directory "unix" (for historical reasons), include unix/win32.fs gives you the win32 part of the Windows API.
Callbacks in the event loop are still a problem, as Gforth is a Cygwin program, and Cygwin has its special event loop task... but I hope that problem can be fixed.
Related
If this question deals with too basic a matter, please forgive me.
As a somewhat-close-to-beginner-level programmer, I really wonder about this--whether the underlying code of every win API function is compiled altogether at the time of writing an app, or whether the machine code for executing win APIs stays in the memory as part of the OS since the pc is booted up, and only the app uses them?
All the APIs for an OS are used by many apps by means of function call. So I thought that rather than making every individual app include the API machine code on their own, apps just contain the header or signature to call the APIs and the API machine code addresses are mapped when launching the app.
I am sorry that I failed to make this question succinct due to my poor English. I really would like to get your insights. Thank you.
The implementation for (most) API calls is provided by the system by way of compiled modules (Portable Executable images). Application code only contains enough information so that the system can identify and load the required modules, and resolve the respective imports.
As an example consider the following code that shows a message box, waits for it to close, and then exits the program:
#include <Windows.h>
int main()
{
::MessageBoxW(nullptr, L"Foo", L"Bar", MB_OK);
}
Given the function signature (declared in WinUser.h, which gets pulled in from Windows.h) the compiler can almost generate a call instruction. It knows the number of arguments, their expected types, and the order and location the callee expects them in. What's missing is the actual target address inside user32.dll, that's only known after a process was fully initialized, and had the user32.dll module mapped into its address space.
Clearly, the compiler cannot postpone code generation until after load time. It needs to generate a call instruction now. Since we know that "all problems in computer science can be solved by another level of indirection" that's what the compiler does, too: Instead of emitting a direct call instruction it generates an indirect call. The difference is that, while a direct call immediately needs to provide the target address, an indirect call can specify the address at which the target address is stored.
In x86 assembly, instead of having to say
call _MessageBoxW#16 ; uh-oh, not yet known
the compiler can conveniently delegate the call to the Import Address Table (IAT):
call dword ptr [__imp__MessageBoxW#16]
Disaster averted, we've bought us just enough time to fix things up before the code actually executes.
Once a process object is created the system hands over control to its primary thread to finish initialization. Part of that initialization is loading dependencies (such as user32.dll here). Once that has completed, the system finally knows the load address (and ultimately the address of imported symbols, such as _MessageBoxW#16), and can overwrite the IAT entry at address __imp__MessageBoxW#16 with the imported function address.
And that is approximately how the system provides implementations for system services without requiring client applications to know where (physically) they will find them.
I'm saying "approximately" because things are somewhat more involved in reality. If that is something you'll want to learn about, I'll leave it up to Raymond Chen. He has published a series of blog entries covering this topic in far more detail:
How were DLL functions exported in 16-bit Windows?
How were DLL functions imported in 16-bit Windows?
How are DLL functions exported in 32-bit Windows?
Exported functions that are really forwarders
Rethinking the way DLL exports are resolved for 32-bit Windows
Calling an imported function, the naive way
How a less naive compiler calls an imported function
Issues related to forcing a stub to be created for an imported function
What happens when you get dllimport wrong?
Names in the import library are decorated for a reason
Why can't I GetProcAddress a function I dllexport'ed?
I am not sure if StringCbPrintf and the include file strsafe.h where it is defined belong the the WinAPI. On one hand, Microsoft documents the function on its WinAPI sites and strsafe.h is under the Windows SDK directory structure which indicates (to me, at least) that it is indeed part of the WinAPI. On the other hand, strsafe.h includes stdio.h etc. which belong to the CRT. I was always under the impression that the WinAPI is completely independent from the CRT (but not vice versa). Possibly, my assumption about the relationship between WinAPI and CRT is wrong. Thus my question: is StringCbPrintf part of the WinAPI?
The StrSafe API is a bit strange because it does not have its own .DLL nor its own exported functions. I assume it was developed this way because it needed to support older versions of Windows that had already been released. It was created during the WinXP service pack security push:
During February and March 2002, all application development in
Microsoft stopped and developers took part in the Security Push
initiative. The goal was to check all code for possible security
vulnerabilities and fix those problems. One of the outcomes of the
Security Push was a library of safe string functions called
"strsafe.lib" with an associated header called "strsafe.h." This
library is available through the Platform SDK that can be downloaded
from the MSDN web site and is automatically installed as part of
Visual C++.NET 2003.
As far as I can tell, a copy of strsafe.h was also included with Writing Secure Code (Second Edition) by Michael Howard and David LeBlanc but I'm not sure if they are the original authors (David LeBlanc is the author of SafeInt):
You can find a copy of Strsafe.h in the companion content in the
folder Secureco2\Strsafe.
msvcrt.dll is basically a system file these days, only Windows 95 shipped without it. You are not supposed to use it as your C run-time but SDK code from Microsoft can probably use it without issues.
msvcrt.dll is now a "known DLL," meaning that it is a system
component owned and built by Windows. It is intended for future
use only by system-level components.
If you want to use msvcrt.dll as your C run-time as well then you must use the WDK for <= Windows 7 but when using the inline version of StrSafe.h, as long as you link to a .lib that contains the required vsnprintf type functions it should not really matter which CRT it comes from. There is also a StrSafe.lib file but Microsoft recommends that you use the inline version.
You are correct that the Windows API is supposed to be independent of the CRT but StrSafe also supports stdin functions like StringCbGetsA and they did not choose to separate those into a separate header for whatever reason. That combined with the need for a existing vsnprintf type function to do the actual work means that StrSafe is somewhat attached to the CRT even though it is meant to be used by all WinAPI developers.
There is probably no true answer to whether it is part of the WinAPI or not since it is a bit subjective. Since it is included with the SDK in the include folder one would assume that Microsoft believes it is a SDK/API component and not a CRT component.
If it's not implemented in Windows and exported from one of its DLLs (as e.g. CreateFile() or CloseHandle() from kernel32.dll), I'd say it's not part of the WinAPI, even if it ends up calling things that are implemented in Windows.
I have to deal with one ancient software module which only support its API being called from VC++6.0.
If I add an indirection layer, that is, wrapping the module in a dynamic library written with VC++6.0 and transferring API to the underlying module. The dll will be called from a VC++ tool chain provided in Visual Studio 2015.
My question is:
Is this doable in principle?
Are there any pitfalls one might want to may attention?
Can it be done in a better way?
Update: I guess that the C ABI is stable through different windows versions as well as corresponding supported VS version. If the dll is made with pure C, this might be done without too much trouble.
There are at least two issues preventing portability between versions/vendors: C++ object/class layout and the C/C++ memory allocators.
To avoid the first issue you should design the visible API as a set of C functions (like most of the Win32 API).
The memory issue can be solved by switching from new/malloc to one of the native Windows functions like LocalAlloc, HeapAlloc or CoTaskMemAlloc. Assuming your old .dll is not statically linked it would theoretically be possible to force newer code to use the malloc in msvcrt.dll but it makes your build setup a bit fragile and it goes against current best practices.
Another alternative is for the middleman .dll to implement everything as a COM object:
EXTERN_C HRESULT WINAPI CreateMyObject(IMyObject**ppv) ...
I've written some high level interpreters and a simple byte code compiler and interpreter and I want to start making a powerful intermediate language for my small operating system.
It has its own API just like windows does, and the only thing which prevents me of starting this project is to know how these specific API calls (for example the win32 forms api) are being made on the assembly level.
Is there a way to see the assembly output of not optimized c code for example and look how exatly the calls are being made? Or any sources on the WWW?
Thanks in advance
Having C documentation for the API, and knowing the calling convention / ABI, should be enough to create asm that uses it. There's no "magic" needed (no inline syscall instructions or anything like that).
Much of the Win32 API is implemented in user-space DLLs, so API calls are no different from other library function calls. (i.e. an indirect CALL with a function pointer, if I recall correctly).
Often the library function implementation will involve a syscall to interact with the kernel (or for 32-bit code, maybe an int or sysenter, I'm not sure), but this interface is not documented and is not stable across different Windows versions.
I want to use "printf" in driver code (DDK), therefore I've included stdio.h. But the compiler says:
error LNK2001: unresolved external symbol __imp__printf
Any ideas? I seen somewhere that it is not possible - but that's awful - I can't believe it. Why can't I use standard C routines in kernel code?
C functions like printf come from a static cstd.lib or something AFAIK don't they?
Why would WDK provide me with stdio.h then?
The Windows kernel only supports part of the standard C runtime. In particular, high-level functionality — like file streams, console I/O, and networking — is not supported. Instead, you need to use native kernel APIs for similar functionality.
The reason that stdio.h is included with the WDK is because some parts of the C runtime are provided for your convenience. For example, you can use memcmp (although the native RtlCompareMemory is preferred). Microsoft has not picked through the CRT headers to #ifdef out the bits and pieces that are not available in kernel mode. Once you develop some experience writing kernel drivers, you'll get the hang of what's possible in the kernel, and what probably won't work.
To address your high-level question: you're probably looking for some debug/logging mechanism. You really have two options:
DbgPrintEx is the easiest to use. It's basically a drop-in for printf (although you need to be careful about certain types of string inserts when running >=DISPATCH_LEVEL). Output goes to the debugger, or, if you like, to DbgView.
WPP is the industrial-strength option. The initial learning curve is pretty steep (although there are samples in the WDK). However, it is very flexible (e.g., you can create your own shrieks, like Print("My IP address is: %!IPV4!", ip);), and it is very fast (Microsoft ships WPP tracing in the non-debug builds of most Windows components).