I wonder why the WinAPI is so much different from "normal" C programming?
I mean, in school I learned that every C programm has a main() function (WinAPI uses WinMain with some special parameters), some variable types like int, long, char etc. (WinAPI uses things like LPCSTR, BOOL, etc.) so why did Microsoft decide to go such a different way with their OS API?
When I saw my first WinAPI program I it looks more like a new language to me... ;)
The original Windows API was designed in the 1984-85 time frame, over 25 years ago. Hungarian Notation was all the rage, so putting the type of a variable into the declaration was the thing to do. For example, in pure C, there is no way to indicate a 'far' pointer, which is what the LP in LPCSTR indicates, but in 1985, it was very important to distinguish between regular pointers and far pointers. (That importance went by the wayside when 32-bit windows took over in the mid-90s, but the syntax persists...)
Also, C doesn't really distinguish between just a pointer to a char and a pointer to a static string. Thus the lpsz types.
In the end, it's about bringing a stronger, consistent typing to the parameters than plain C allowed in 1984. As for the WinMain, it's because a Windows program is pretty fundamentally different from a command line program. If you look in the library, you'd probably find a main() function that sets up the parameters and then calls into an extern WinMain function (i.e. yours).
There are two major reasons:
Complexity. The C language is minimal, providing the building blocks on which more complex architectures can be constructed. LPCSTR, BOOL and other types that you find in Win32 are typedefs or structs built on top of C.
Event orientation. C is usually taught assuming that your program is proactive and in control of things. In an event-oriented environment, such as Windows (or any other GUI-based OS), your program is called by the OS, so it usually sits there in a loop waiting for messages to arrive.
The APIs of other GUI-based OSs may feel different to Win32, because there is no single solution, but the problem they are solving is the same one.
Microsoft's Raymond Chen writes in his blog:
Although the function WinMain is
documented in the Platform SDK, it's
not really part of the platform.
Rather, WinMain is the conventional
name for the user-provided entry point
to a Windows program.
The real entry point is in the C
runtime library, which initializes the
runtime, runs global constructors, and
then calls your WinMain function (or
wWinMain if you prefer a Unicode entry
point).
I'd say most of it is a question of style. The standards grew out of the Unix world, so for example the library functions have short names, and there aren't a whole lot of typedefs. I presume that reflects the choices of the designers of C and Unix. On the other hand Windows has LongFunctionNamesInMixedCase, and LOTSOFTYPEDEFS, *PTYPEDEFSFORPOINTERSTOO.
Some of it is also the perception of necessity. For example WinMain() has things like nCmdShow, because graphical apps will call ShowWindow() and I suppose they wanted to be able to pass the argument to that to a newly launched process. Whether or not that is actually needed might be another question.
And of course, some of the APIs do very different things. In Windows there's a lot of emphasis on passing messages, and processing messages on a per-thread basis. CreateFile() has a lot of flags that the Unix world doesn't have, including sharing modes which determine what another process can do while you have a file open.
They really didn't "go such a different way," as you put it.
WinMain() is simply the entry point looked for by the Windows OS. Conceptually, it's no different than main().
As for the symbol definitions (LPCSTR, BOOL, etc.), part of this is for ease of use. For example, it's shorter to write LPCSTR than const char *. Another example is the BOOL typedef which is not supported by the C language. The other reason is to insulate the developer from changes to the underlying hardware, e.g., the change from 16-bit to 32-bit to 64-bit architectures.
By no means should this answer be considered exhaustive. It's just a couple of things that I've noticed from the programming I've done with the Win32/MFC.
Windows API programming is event driven, while, up until that point, most C programming was linear. WinMain() is thus a shortcut into the libraries for writing using OS functionality - while main() is part of the C language.
While we're on the subject, C has few built in types, and, at the time, had few ways of indicating them. The windows "types" (HWND, LPSTR, BOOL, etc) reflect data types commonly used in windows programming, and make an attempt to indicate to the programmer what the data types will be.
The Hungarian notation is a bit of a mis-use of the original versions, in that there are an unnecessary number of qualifiers in many variables.
Related
There are two ways of procedure call, save address to register or save it in stack. I read that the way of procedure call is OS specific. I want to understand how OS restricts that. Can't compiler generate a code that saves address in register and load it later, or save it in stack and pop it when needed?
Just want to understand the role of OS here.
Thank you.
The operating system has no function in it whatsoever, except that the OS's own libraries may use a specific calling convention.The compiler determines the calling convention. Its' not OS specific but rather language and compiler specific.
Programming languages do things in different ways. For example, the nested procedures of Ada and Pascal need context passed to them behind the scenes that are not needed in C and C++.
In the old days there was pretty much chaos on this.
By the late 1970's the VMS operating system had a calling convention defined and all compilers made by the vendor complied with it. This made it possible for Fortran to call Pascal to call C to call Fortran. However, even there, things were not 100% transparent. In fact, the VMS compilers had extensions to languages to call function in other languages. In FORTRAN 77, everything was passed by reference. There had to extensions to call C function that expected everything to be passed by value.
I am implementing SMBIOS reading functionality for Windows systems. As API levels vary, there are several methods to support:
trouble-free GetSystemFirmwareTable('RSMB') available on Windows Server 2003 and later;
hardcore NtOpenSection(L"\\Device\\PhysicalMemory") for legacy systems prior to and including Windows XP;
essential WMI data in L"Win32_ComputerSystemProduct" path through cumbersome COM automation calls as a fallback.
Methods 1 and 3 are already implemented, but I am stuck with \Device\PhysicalMemory, as NtOpenSection always yields 0xC0000034 (STATUS_OBJECT_NAME_NOT_FOUND) — definitely not one of the possible result codes in the ZwOpenSection documentation. Of course, I am aware that accessing this section is prohibited starting from Windows Server 2003sp1 and perhaps Windows XP-64 as well, so I am trying this on a regular Windows XP-32 system — and the outcome is no different to that of a Windows 7-64, for example. I am also aware that administrator rights may be required even on legacy systems, but people on the internets having faced this issue reported more relevant error codes for such scenario, like 0xC0000022 (STATUS_ACCESS_DENIED) and 0xC0000005 (STATUS_ACCESS_VIOLATION).
My approach is based on the Libsmbios library by Dell, which I assume to be working.
UNICODE_STRING wsMemoryDevice;
OBJECT_ATTRIBUTES oObjAttrs;
HANDLE hMemory;
NTSTATUS ordStatus;
RtlInitUnicodeString(&wsMemoryDevice, L"\\Device\\PhysicalMemory");
InitializeObjectAttributes(&oObjAttrs, &wsMemoryDevice,
OBJ_CASE_INSENSITIVE, NULL, NULL);
ordStatus = NtOpenSection(&hMemory, SECTION_MAP_READ, &oObjAttrs);
if (!NT_SUCCESS(ordStatus)) goto Finish;
I thought it could be possible to debug this, but native API seems to be transparent to debuggers like OllyDbg: the execution immediately returns once SYSENTER instruction receives control. So I have no idea why Windows cannot find this object. I also tried changing the section name, as there are several variants in examples available online, but that always yields 0xC0000033 (STATUS_OBJECT_NAME_INVALID).
Finally, I found the cause of such a strange behavior, — thanks to you, people, confirming that my code snippet (it was an actual excerpt, not a forged example) really works. The problem was that I did not have Windows DDK installed initially (I do have now, but still cannot integrate it with Visual Studio in a way that Windows SDK integrates automatically), so there was a need to write definitions by hand. Particularly, when I realized that InitializeObjectAttributes is actually a preprocessor macro rather than a Win32 function, I defined RtlInitUnicodeString as a macro, too, since its effect is even simpler. However, I was not careful enough to notice that UNICODE_STRING.Length and .MaximumLength are in fact meant for content size and buffer size instead of length, i. e. number of bytes rather than number of characters. Consequently, my macro was setting the fields to a half of their expected value, thus making Windows see only the first half of the L"\\Device\\PhysicalMemory" string, — with obvious outcome.
Off late I'd been hearing that applications written in different languages can call each other's functions/subroutines. Now, till recently I felt that was very natural - since all, yes all - that's what I thought then, silly me! - languages are compiled into machine code and that should be same for all the languages. Only some time back did I realise that even languages compiled in 'higher machine code' - IL, byte code etc. can interact with each other, the applications actually. I tried to find the answer a lot of times, but failed - no answer satisfied me - either they assumed I knew a lot about compilers, or something that I totally didn't agree with, and other stuff...Please explain in an easy to understand way how this works out. Especially how languages compiled into 'pure' machine code have different something called 'calling conventions' is what is making me clutch my hair.
This is actually a very broad topic. Languages compiled to machine code can often call each others' routines, though usually not without effort; e.g., C++ code can call C routines when properly declared:
// declare the C function foo so it can be called by C++ code
extern "C" {
void foo(int, char *);
}
This is about as simple as it gets, because C++ was explicitly designed for compatibility with C (it includes support for calling C++ routines from C as well).
Calling conventions indeed complicate the picture in that C routines compiled by one compiler might not be callable from C compiled by another compiler, unless they share a common calling convention. For example, one compiler might compile
foo(i, j);
to (pseudo-assembly)
PUSH the value of i on the stack
PUSH the value of j on the stack
JUMP into foo
while another might push the values of i and j in reverse order, or place them in registers. If foo was compiled by a compiler following another convention, it might try to fetch its arguments off the stack in the wrong order, leading to unpredictable behavior (consider yourself lucky if it crashes immediately).
Some compilers support various calling conventions for this purpose. The Wikipedia article introduces calling conventions; for more details, consult your compiler's documentation.
Finally, mixing bytecode-compiled or interpreted languages and lower-level ones in the same address space is still more complicated. High-level language implementations commonly come with their own set of conventions to extend them with lower-level (C or C++) code. E.g., Java has JNI and JNA.
I am aware that this is nothing new and has been done several times. But I am looking for some reference implementation (or even just reference design) as a "best practices guide". We have a real-time embedded environment and the idea is to be able to use a "debug shell" in order to invoke some commands. Example: "SomeDevice print reg xyz" will request the SomeDevice sub-system to print the value of the register named xyz.
I have a small set of routines that is essentially made up of 3 functions and a lookup table:
a function that gathers a command line - it's simple; there's no command line history or anything, just the ability to backspace or press escape to discard the whole thing. But if I thought fancier editing capabilities were needed, it wouldn't be too hard to add them here.
a function that parses a line of text argc/argv style (see Parse string into argv/argc for some ideas on this)
a function that takes the first arg on the parsed command line and looks it up in a table of commands & function pointers to determine which function to call for the command, so the command handlers just need to match the prototype:
int command_handler( int argc, char* argv[]);
Then that function is called with the appropriate argc/argv parameters.
Actually, the lookup table also has pointers to basic help text for each command, and if the command is followed by '-?' or '/?' that bit of help text is displayed. Also, if 'help' is used for a command, the command table is dumped (possible only a subset if a parameter is passed to the 'help' command).
Sorry, I can't post the actual source - but it's pretty simple and straight forward to implement, and functional enough for pretty much all the command line handling needs I've had for embedded systems development.
You might bristle at this response, but many years ago we did something like this for a large-scale embedded telecom system using lex/yacc (nowadays I guess it would be flex/bison, this was literally 20 years ago).
Define your grammar, define ranges for parameters, etc... and then let lex/yacc generate the code.
There is a bit of a learning curve, as opposed to rolling a 1-off custom implementation, but then you can extend the grammar, add new commands & parameters, change ranges, etc... extremely quickly.
You could check out libcli. It emulates Cisco's CLI and apparently also includes a telnet server. That might be more than you are looking for, but it might still be useful as a reference.
If your needs are quite basic, a debug menu which accepts simple keystrokes, rather than a command shell, is one way of doing this.
For registers and RAM, you could have a sub-menu which just does a memory dump on demand.
Likewise, to enable or disable individual features, you can control them via keystrokes from the main menu or sub-menus.
One way of implementing this is via a simple state machine. Each screen has a corresponding state which waits for a keystroke, and then changes state and/or updates the screen as required.
vxWorks includes a command shell, that embeds the symbol table and implements a C expression evaluator so that you can call functions, evaluate expressions, and access global symbols at runtime. The expression evaluator supports integer and string constants.
When I worked on a project that migrated from vxWorks to embOS, I implemented the same functionality. Embedding the symbol table required a bit of gymnastics since it does not exist until after linking. I used a post-build step to parse the output of the GNU nm tool for create a symbol table as a separate load module. In an earlier version I did not embed the symbol table at all, but rather created a host-shell program that ran on the development host where the symbol table resided, and communicated with a debug stub on the target that could perform function calls to arbitrary addresses and read/write arbitrary memory. This approach is better suited to memory constrained devices, but you have to be careful that the symbol table you are using and the code on the target are for the same build. Again that was an idea I borrowed from vxWorks, which supports both teh target and host based shell with the same functionality. For the host shell vxWorks checksums the code to ensure the symbol table matches; in my case it was a manual (and error prone) process, which is why I implemented the embedded symbol table.
Although initially I only implemented memory read/write and function call capability I later added an expression evaluator based on the algorithm (but not the code) described here. Then after that I added simple scripting capabilities in the form of if-else, while, and procedure call constructs (using a very simple non-C syntax). So if you wanted new functionality or test, you could either write a new function, or create a script (if performance was not an issue), so the functions were rather like 'built-ins' to the scripting language.
To perform the arbitrary function calls, I used a function pointer typedef that took an arbitrarily large (24) number of arguments, then using the symbol table, you find the function address, cast it to the function pointer type, and pass it the real arguments, plus enough dummy arguments to make up the expected number and thus create a suitable (if wasteful) maintain stack frame.
On other systems I have implemented a Forth threaded interpreter, which is a very simple language to implement, but has a less than user friendly syntax perhaps. You could equally embed an existing solution such as Lua or Ch.
For a small lightweight thing you could use forth. Its easy to get going ( forth kernels are SMALL)
look at figForth, LINa and GnuForth.
Disclaimer: I don't Forth, but openboot and the PCI bus do, and I;ve used them and they work really well.
Alternative UI's
Deploy a web sever on your embedded device instead. Even serial will work with SLIP and the UI can be reasonably complex ( or even serve up a JAR and get really really complex.
If you really need a CLI, then you can point at a link and get a telnet.
One alternative is to use a very simple binary protocol to transfer the data you need, and then make a user interface on the PC, using e.g. Python or whatever is your favourite development tool.
The advantage is that it minimises the code in the embedded device, and shifts as much of it as possible to the PC side. That's good because:
It uses up less embedded code space—much of the code is on the PC instead.
In many cases it's easier to develop a given functionality on the PC, with the PC's greater tools and resources.
It gives you more interface options. You can use just a command line interface if you want. Or, you could go for a GUI, with graphs, data logging, whatever fancy stuff you might want.
It gives you flexibility. Embedded code is harder to upgrade than PC code. You can change and improve your PC-based tool whenever you want, without having to make any changes to the embedded device.
If you want to look at variables—If your PC tool is able to read the ELF file generated by the linker, then it can find out a variable's location from the symbol table. Even better, read the DWARF debug data and know the variable's type as well. Then all you need is a "read-memory" protocol message on the embedded device to get the data, and the PC does the decoding and displaying.
What is the best way to design a C API for dlls which deals with the problem of passing "objects" which are C runtime dependent (FILE*, pointer returned by malloc, etc...). For example, if two dlls are linked with a different version of the runtime, my understanding is that you cannot pass a FILE* from one dll to the other safely.
Is the only solution to use windows-dependent API (which are guaranteed to work across dlls) ? The C API already exists and is mature, but was designed from a unix POV, mostly (and still has to work on unix, of course).
You asked for a C, not a C++ solution.
The usual method(s) for doing this kind of thing in C are:
Design the modules API to simply not require CRT objects. Get stuff passed accross in raw C types - i.e. get the consumer to load the file and simply pass you the pointer. Or, get the consumer to pass a fully qualified file name, that is opened , read, and closed, internally.
An approach used by other c modules, the MS cabinet SD and parts of the OpenSSL library iirc come to mind, get the consuming application to pass in pointers to functions to the initialization function. So, any API you pass a FILE* to would at some point during initialization have taken a pointer to a struct with function pointers matching the signatures of fread, fopen etc. When dealing with the external FILE*s the dll always uses the passed in functions rather than the CRT functions.
With some simple tricks like this you can make your C DLLs interface entirely independent of the hosts CRT - or in fact require the host to be written in C or C++ at all.
Neither existing answer is correct: Given the following on Windows: you have two DLLs, each is statically linked with two different versions of the C/C++ standard libraries.
In this case, you should not pass pointers to structures created by the C/C++ standard library in one DLL to the other. The reason is that these structures may be different between the two C/C++ standard library implementations.
The other thing you should not do is free a pointer allocated by new or malloc from one DLL that was allocated in the other. The heap manger may be differently implemented as well.
Note, you can use the pointers between the DLLs - they just point to memory. It is the free that is the issue.
Now, you may find that this works, but if it does, then you are just luck. This is likely to cause you problems in the future.
One potential solution to your problem is dynamically linking to the CRT. For example,you could dynamically link to MSVCRT.DLL. That way your DLL's will always use the same CRT.
Note, I suggest that it is not a best practice to pass CRT data structures between DLLs. You might want to see if you can factor things better.
Note, I am not a Linux/Unix expert - but you will have the same issues on those OSes as well.
The problem with the different runtimes isn't solvable because the FILE* struct belongs
to one runtime on a windows system.
But if you write a small wrapper Interface your done and it does not really hurt.
stdcall IFile* IFileFactory(const char* filename, const char* mode);
class IFile {
virtual fwrite(...) = 0;
virtual fread(...) = 0;
virtual delete() = 0;
}
This is save to be passed accross dll boundaries everywhere and does not really hurt.
P.S.: Be careful if you start throwing exceptions across dll boundaries. This will work quiet well if you fulfill some design creterions on windows OS but will fail on some others.
If the C API exists and is mature, bypassing the CRT internally by using pure Win32 API stuff gets you half the way. The other half is making sure the DLL's user uses the corresponding Win32 API functions. This will make your API less portable, in both use and documentation. Also, even if you go this way with memory allocation, where both the CRT functions and the Win32 ones deal with void*, you're still in trouble with the file stuff - Win32 API uses handles, and knows nothing about the FILE structure.
I'm not quite sure what are the limitations of the FILE*, but I assume the problem is the same as with CRT allocations across modules. MSVCRT uses Win32 internally to handle the file operations, and the underlying file handle can be used from every module within the same process. What might not work is closing a file that was opened by another module, which involves freeing the FILE structure on a possibly different CRT.
What I would do, if changing the API is still an option, is export cleanup functions for any possible "object" created within the DLL. These cleanup functions will handle the disposal of the given object in the way that corresponds to the way it was created within that DLL. This will also make the DLL absolutely portable in terms of usage. The only worry you'll have then is making sure the DLL's user does indeed use your cleanup functions rather than the regular CRT ones. This can be done using several tricks, which deserve another question...