I will be generating RSA keys in a multi thread application. I use RtlGenRandom as RNG.
I know I can call in multi threads with hanging/crash or bugs.
My question is:
May calling RtlGenRandom in multiple threads give me duplicate bytes on each thread?
Please don't remind me to use CNG or cryptgenrandom as this answer says they all end up in bcryptPrimitives!AesRNG*.
Only Microsoft could tell you that for sure, or what exactly is the contract behind that function, but Microsoft basically says you should get away from this function.
The RtlGenRandom function is available for use in the operating
systems specified in the Requirements section. It may be altered or
unavailable in subsequent versions. Instead, use the CryptGenRandom
function
Anyway, if you disassemble it, you find the
ProcessPrng -> AesRNGState_generate -> SymCryptRngAesGenerate
function call chains and there are also EnterCriticalSection calls along the way, so I'd "imagine" it's thread safe.
PS: the last SymCryptRngAesGenerate function is open source.
See also this crypto/rand: Currently using deprecated API for random number generation on Windows github discussion about the subject.
Ok, it may be a bit difficult to explain:
Suppose someone creates a Windows application (using C# or any other language) that uses the GetDesktopWindow() function on the user32.dll to capture a Screenshot and then sends this image to any online service.
Since it's custom made application, no anti-virus software will be able to determine that it's a virus because it's still an unknown application for it. Also, there are legitimate uses for such API, so it's not necessarily a virus, it can be a harmless window capture tool or some kind of espionage tool.
What I want to know is: Is there any way to see what a specific EXE file does regarding the Windows functions? Can I know if "myapp.exe" uses GetDesktopWindow() of user32.dll?
This is only one example. There are plenty other Windows endpoints that I would like to know when they're used by any application.
Is there a way to do that?
It depends to what lengths you want to go doing that. It's essentially a game of cat and mouse - bad actors will attempt to find new ways to circumvent your detection by jumping through some obscure hoops, you will add more sophisticated detection methods for those tricks, they will think of new tricks, and so on.
Also, it depends on whether you want to statically or dynamically determine that, and whether you actually want to know if GetDesktopWindow is called or if "the program gets a handle to the desktop window" (which can be achieved in other ways as well).
Here is a non-exhaustive list of ideas:
You could statically determine whether the function is imported by looking at the import directory. Research the PE file structure to find out more. This article may help.
This method of detection can be easily circumvented by dynamically importing the function using LoadLibrary and GetProcAddress.
You could scan the file for the string GetDesktopWindow to detect possible usage for dynamic import.
This method of detection can be easily circumvented by packing, encrypting or otherwise obfuscating the name of the dynamically imported function.
You could dynamically observe whether the GetDesktopWindow function gets called by registering an AppInit_DLL or a global hook which is injected into every new process and hook the GetDesktopWindow function from inside the process by overwriting its first bytes with a jump to your own code, notifying your detection component somehow, executing the original bytes and jumping back. (Microsoft Detours can help there.)
This method of detection can be circumvented if the target notices the hook and removes it before calling, since its in its own process space. (You could also do some tricks with acting like a debugger and setting a hardware breakpoint on the first instruction of GetDesktopWindow, but yet again there would be ways to detect or circumvent that since the target could also modify the debug registers.)
You could build a driver that does this from kernel-mode instead, but now we are getting really deep.
Note that until now we focused on the actual GetDesktopWindow function from user32.dll. But what if the target will just use a different way to achieve its goal of getting a desktop window handle?
The desktop window handle for the current thread is stored in the TIB (thread information block) which is accessible via fs:[18] from user mode. You can see this in the GetDesktopWindow source code of ReactOS which is pretty accurate compared to Microsoft's actual implementation (which you can verify by looking at it in a debugger). The target could therefore just access the TIB and extract this value, without even calling GetDesktopWindow at all.
The target could just take a known top-level window such as the shell's hidden compatibility window which you'll get via GetShellWindow() or - to avoid detection of GetShellWindow too - for example FindWindow(NULL, "Program Manager") (or even a newly created window!) and call GetAncestor(hWnd, GA_PARENT) on it to get the desktop window handle.
I'm sure, with some creativity, your adversaries will come up with more clever ideas than these.
Also, if we take this one step further and take a look at the ultimate goal of taking a screenshot, there as well exist other ways to achieve that. First example coming to mind: They could use keybd_event to emulate pressing the PrnSc key and then read the screenshot out of the clipboard data.
So it's all a matter of how far you want to take this.
By the way, you may find the drltrace project interesting - it is a library call tracer.
Is it required for COM to be initialized to use SysAllocString, SysFreeString and other BSTR handling functions?
The MSDN says
You need to initialize the COM library on a thread before you call any of the library functions except CoGetMalloc
It appears to me that BSTR allocation functions use IMalloc and therefore COM init is not required?
I cannot find official documentation that says you can call these functions without initializing COM but I can take you on a trip down memory lane for some cute observations.
Because Windows 95 ran on machines with 4 MB of RAM, Explorer tries really hard not to load ole32.dll. So hard in fact that the shell team wrote their own mini version of COM and people writing shell extensions were supposed to only call SH* functions, not Co* functions to avoid loading the real COM/OLE stuff if possible.
This blog post says that mini-COM is 100% gone in Windows XP and later but I'm not sure how far back it is theoretically possible for the shell to be in mini-COM mode.
Things get interesting when you realize that Windows 2000 has a shell interface called IShellFolder2 and one of its methods (GetDetailsEx) returns a VARIANT. And VARIANTs can contain a BSTR! IColumnProvider::GetItemData also returns a VARIANT.
With this information you could infer that Windows 2000 could allocate a BSTR without you having to call CoInitialize, just call SHGetDesktopFolder to get a IShellFolder and ask for a property that is returned as a string.
There are however two unknowns here:
Is it possible for Windows 2000 to use mini-COM?
Will the IShellFolder2 implementation in shell32.dll initialize real COM if it has to return a string?
I know that IShellFolder2 and IColumnProvider are involved when a Explorer window is in details mode (or if it needs a tooltip?) so it seems unlikely that the shell would be willing to load real COM for this.
Even if I could confirm all of this it would not really help you because I assume your main target is not Windows 2000.
What I think I know so far is that the CORBA specification as such doesn't allow any differences between the IDL the server program uses and the IDL the client program uses.
However, in practice, certain differences are bound to work (pretty) universally, because the underlying communication mechanism is very probably GIOP (at least IIOP) and some differences are bound to not be detectable via IIOP.
What I like to establish is which differences are allowed between the server and client IDL universally between arbitrary ORBs as long as GIOP/IIOP is used.
For example: So far I assume it works to:
Add any type/interface to the server IDL as long as the types the client IDL knows about aren't touched or any unknown new types sent back to the client.
Add a method to an existing interface on the server side -- the client should be able to continue call objects with this interface, even though his IDL doesn't list said method. (This seems to be answered with yes here.)
Add a member to the end of an enum, as long as the client never sees this new value.
Add a member to a union, as long as the client never sees this Union type with the discriminator set to the new value.
My aim is to get to something like a short list of stuff one can do in an existing IDL to extend "the server" with new stuff without having to recompile exiting clients with the modified IDL.
Yes, the server and client method set need not match completely as the methods are accessed by name (the operation field in GIOP message) and independently. In other words, GIOP call includes the method name as string, later the parameters are encoded as they are expected by this parameter. See the example of the CORBA tie and CORBA stub.
Yes, if you create and export a new interface, it is just a new interface. It can be bound to any name service independently from others, and clients unaware of this new interface just will not be able to use it. The will be able to use the known types bound to the same name service.
Yes, GIOP writes enums as unsigned longs, first value is always encoded as zeros, successive identifiers take ascending numeric values, in order of declaration from left to right. Hence it is perfectly safe to append new enum identifies but not to remove and not to reorder.
Read GIOP specification, helps a lot. It is also very good to look into the code, generated by IDL compiler, and how does it change when something is changed in the IDL.
Surely it is not a good practice to use mismatching IDLs just because of the lack of care as it is also easy to introduce incompatible changes. This makes any sense probably only in cases if you cannot longer reach and update the client as it has been released to the user.
I am aware that this is nothing new and has been done several times. But I am looking for some reference implementation (or even just reference design) as a "best practices guide". We have a real-time embedded environment and the idea is to be able to use a "debug shell" in order to invoke some commands. Example: "SomeDevice print reg xyz" will request the SomeDevice sub-system to print the value of the register named xyz.
I have a small set of routines that is essentially made up of 3 functions and a lookup table:
a function that gathers a command line - it's simple; there's no command line history or anything, just the ability to backspace or press escape to discard the whole thing. But if I thought fancier editing capabilities were needed, it wouldn't be too hard to add them here.
a function that parses a line of text argc/argv style (see Parse string into argv/argc for some ideas on this)
a function that takes the first arg on the parsed command line and looks it up in a table of commands & function pointers to determine which function to call for the command, so the command handlers just need to match the prototype:
int command_handler( int argc, char* argv[]);
Then that function is called with the appropriate argc/argv parameters.
Actually, the lookup table also has pointers to basic help text for each command, and if the command is followed by '-?' or '/?' that bit of help text is displayed. Also, if 'help' is used for a command, the command table is dumped (possible only a subset if a parameter is passed to the 'help' command).
Sorry, I can't post the actual source - but it's pretty simple and straight forward to implement, and functional enough for pretty much all the command line handling needs I've had for embedded systems development.
You might bristle at this response, but many years ago we did something like this for a large-scale embedded telecom system using lex/yacc (nowadays I guess it would be flex/bison, this was literally 20 years ago).
Define your grammar, define ranges for parameters, etc... and then let lex/yacc generate the code.
There is a bit of a learning curve, as opposed to rolling a 1-off custom implementation, but then you can extend the grammar, add new commands & parameters, change ranges, etc... extremely quickly.
You could check out libcli. It emulates Cisco's CLI and apparently also includes a telnet server. That might be more than you are looking for, but it might still be useful as a reference.
If your needs are quite basic, a debug menu which accepts simple keystrokes, rather than a command shell, is one way of doing this.
For registers and RAM, you could have a sub-menu which just does a memory dump on demand.
Likewise, to enable or disable individual features, you can control them via keystrokes from the main menu or sub-menus.
One way of implementing this is via a simple state machine. Each screen has a corresponding state which waits for a keystroke, and then changes state and/or updates the screen as required.
vxWorks includes a command shell, that embeds the symbol table and implements a C expression evaluator so that you can call functions, evaluate expressions, and access global symbols at runtime. The expression evaluator supports integer and string constants.
When I worked on a project that migrated from vxWorks to embOS, I implemented the same functionality. Embedding the symbol table required a bit of gymnastics since it does not exist until after linking. I used a post-build step to parse the output of the GNU nm tool for create a symbol table as a separate load module. In an earlier version I did not embed the symbol table at all, but rather created a host-shell program that ran on the development host where the symbol table resided, and communicated with a debug stub on the target that could perform function calls to arbitrary addresses and read/write arbitrary memory. This approach is better suited to memory constrained devices, but you have to be careful that the symbol table you are using and the code on the target are for the same build. Again that was an idea I borrowed from vxWorks, which supports both teh target and host based shell with the same functionality. For the host shell vxWorks checksums the code to ensure the symbol table matches; in my case it was a manual (and error prone) process, which is why I implemented the embedded symbol table.
Although initially I only implemented memory read/write and function call capability I later added an expression evaluator based on the algorithm (but not the code) described here. Then after that I added simple scripting capabilities in the form of if-else, while, and procedure call constructs (using a very simple non-C syntax). So if you wanted new functionality or test, you could either write a new function, or create a script (if performance was not an issue), so the functions were rather like 'built-ins' to the scripting language.
To perform the arbitrary function calls, I used a function pointer typedef that took an arbitrarily large (24) number of arguments, then using the symbol table, you find the function address, cast it to the function pointer type, and pass it the real arguments, plus enough dummy arguments to make up the expected number and thus create a suitable (if wasteful) maintain stack frame.
On other systems I have implemented a Forth threaded interpreter, which is a very simple language to implement, but has a less than user friendly syntax perhaps. You could equally embed an existing solution such as Lua or Ch.
For a small lightweight thing you could use forth. Its easy to get going ( forth kernels are SMALL)
look at figForth, LINa and GnuForth.
Disclaimer: I don't Forth, but openboot and the PCI bus do, and I;ve used them and they work really well.
Alternative UI's
Deploy a web sever on your embedded device instead. Even serial will work with SLIP and the UI can be reasonably complex ( or even serve up a JAR and get really really complex.
If you really need a CLI, then you can point at a link and get a telnet.
One alternative is to use a very simple binary protocol to transfer the data you need, and then make a user interface on the PC, using e.g. Python or whatever is your favourite development tool.
The advantage is that it minimises the code in the embedded device, and shifts as much of it as possible to the PC side. That's good because:
It uses up less embedded code space—much of the code is on the PC instead.
In many cases it's easier to develop a given functionality on the PC, with the PC's greater tools and resources.
It gives you more interface options. You can use just a command line interface if you want. Or, you could go for a GUI, with graphs, data logging, whatever fancy stuff you might want.
It gives you flexibility. Embedded code is harder to upgrade than PC code. You can change and improve your PC-based tool whenever you want, without having to make any changes to the embedded device.
If you want to look at variables—If your PC tool is able to read the ELF file generated by the linker, then it can find out a variable's location from the symbol table. Even better, read the DWARF debug data and know the variable's type as well. Then all you need is a "read-memory" protocol message on the embedded device to get the data, and the PC does the decoding and displaying.