There is ReadDirectoryChangesW in Winapis. But there is no ReadDirectoryChangesA.
Curious.
Why is it treated specially?
Is there anyone who knows the reason or history?
It never existed in Win98/98/SE/Me. Many (most?) of the functions that have been NT-only from the beginning (e.g., the Net* API) are only available in "wide" versions (though in the case of Net*, they omitted the "W" suffix).
The kernel internally is using unicode encoding. Win32 APIs with W suffix usually translate to the internal NtXxx APIs. The A version of the Win32 API needs to translate from ASCII to Unicode, call the W-version, and then translate any output back to ASCII.
In the case of ReadDirectoryChangesW the last part is impossible to do because the data is read directly from the file system to the caller's buffer, and potentially after the API returns so it is impossible to implement ReadDirectoryChangesA.
Related
My understanding is that TranslateMessage collates a sequence of key events and adds a WM_CHAR message to the queue if it results in a character. So on my computer what happens when I press Alt+1234 (6 relevant events/messages, including releasing the Alt) is the single character "Ê" comes out (in certain places).
Let's say I have a sequence of virtual key codes and related keypress data generated from the LL keyboard hook. Is there some way of using the Windows OS logic to translate this sequence into a real character? For example, could I construct contrived MSG structures, call TranslateMessage on them and then catch the WM_CHAR ensuing events? That seems very outside of Windows' expectations; haven't tried it yet but it seems like it could cause all kinds of subtle problems.
The cleanest solution I can think of so far is just to re-implement the logic myself to figure out the characters from the virtual codes. This is unfortunate of course since Windows internals already seem to know how to do this! Is there a better way?
I am aware of the existence of MapVirtualKeyA but this does not seem to handle a sequence of virtual key codes.
I am also aware that it is possible to do a hook on all GetMessage calls which could be used just to grab the WM_CHAR messages from every process. However this seems an extremely heavy solution: I need to make a separate DLL unlike for the WH_KEYBOARD_LL hook and then use some sort of IPC to send the characters back to my host process. Also MSDN explicitly says that you should avoid doing global hooks this for anything outside debugging and I need this to work on production machines.
I am also also aware of KeysConverter in .NET (I am fine to use .NET for this) but again this does not seem to deal with sequences of virtual keys like given above.
I'm using Notepad++ as my test case, but I was also able to replicate the exact same behavior on a program I wrote that (indirectly) uses the Win32 API, so I'm trusting at this time that the behavior is specific to Windows, and not to the individual programs.
At any rate, I'm trying to use alt-codes to transmit unicode text to a program. Specifically, I'm using the "Alt Plus" method described in the first point here. For any and all alt-codes below 0xffff, this method works perfectly fine. But the moment I try to transmit an alt-code above this threshold, the input only retains the last four digits I typed.
From notepad++ using UCS-2 Big Endian:
To generate this output, I used the following keystrokes:
[alt↓][numpad+][numpad1][numpad2][numpad3][numpad4][alt↑]
[alt↓][numpad+][numpad1][numpad2][numpad3][numpad5][alt↑]
[alt↓][numpad+][numpad1][numpad2][numpad3][numpad4][numpad5][alt↑]
Knowing that the first two bytes are encoding specific data, it's clear that the first two characters transmitted were successful and correct, but the third was truncated to only the last 2 bytes. Specifically, it did not try to perform a codepoint→UTF16 conversion; it simply dropped the higher bits.
I observed identical behavior on the program I wrote.
So my questions then are in two parts:
Is this a hard constraint of Windows (or at least the Win32 API), or is it possible to alter this behavior to split higher codepoints into multiple UTF-16 (or UTF-8) characters?
Is there a better solution for transmitting unicode to the application that doesn't involve writing a manual parser in the software itself?
To expand on that second point: In the program I wrote, I was able implement the intended functionality by discarding the "Text Handling" code and writing, by hand, a unicode parser in the "Key Handling" code which would, if the [alt] key were held down, save numpad inputs to an internal buffer and convert them to a codepoint when [alt] was released. I can use this as a solution if I must, but I'd like to know if there's a better solution, especially on the OS side of things, rather than a software solution.
What are the relevant standards, man pages, RFCs or other pieces of documentation when implementing a POSIX-style unicode terminal emulator?
The scope of this question spans everything from handling multi-codepoint unicode characters and other unicode pitfalls, behaviour of the terminal when resizing, control sequences to RGB values associated with certain color codes.
While articles such as the Wikipedia page on ANSI escape sequences might suffice for using a terminal emulator, writing one that will behave correctly for all applications, which includes correctly handling invalid, unknown or user-defined inputs requires actual standard documentation.
My best source so far are ECMA-048, man 3 termios, and the source code of various other terminal emulators.
Obviously, you already added The Unicode Standard to your list of sources. :-)
By a POSIX-style unicode terminal emulator, do you mean a terminal emulator accepting the whole Unicode charset (or a large subset of it), and running on POSIX-compliant operating system? Then since POSIX restricts itself since 2001 on 8-bit chars, this pretty much means a UTF-8 terminal emulator, a restricted case of such emulators where you would not have to deal with various charsets and encodings (definitively a good thing) but where characters are basically multi-byte, which in turn may call for functions like wcwidth(3) (which is not strictly POSIX, only XPG by the way); more generally, rendering problems can be arbitrarily complex with Unicode, including BiDi, Indic scripts with reordering vowels, ...
If you mean something else, please elaborate.
Otherwise, since an emulator also relies on a keyboard, you may encounter interesting content on Wikipedia.
Another source of documentation with a lot of information you might use is the Microsoft Go Global site.
How can I find the address of a WndProc (of a window of another process). Even if I inject a DLL and try to find it with either GetClassInfoEx() or GetWindowLong() or GetWindowLongPtr() I always get values like 0xffff08ed, which is definitely not an executable address. It is according to MSDN: "... the address of the window procedure, or a handle representing the address of the window procedure."
Unfortunately that is not good enough for me I need the actual address. Spy++ does the job right most of the time (but even that sometimes fails). So it should be be possible. Thanx.
[EDIT:] Kudos to Chris Becke for providing a super fast, and correct solution to my little problem!
Perhaps you are being stymied because you are asking for the wrong version of the windowproc.
Window Procs, like applications, occur in two flavors: ansi and unicode. Windows cannot return a raw pointer to a ansi window to a unicode application, or visa versa, as they will attempt to call it with the wrong string type.
So, there is no GetWindowLongPtr function. Its a macro that resolves to two 'real' functions the windows api provides: GetWindowLongPtrA and GetWindowLongPtrW. If the window is a unicode window, and GetWindowLongPtrA is called windows will return a handle instead of the raw pointer, so that it can intercept calls (made via CallWindowProc) and marshal the string's from ansi to unicode. The opposite conversion holds the other way.
Even if you call the correct function, you still might get a handle back - its completely possible that ansi code has subclassed a unicode window. so the windowproc has been completely replaced by one of the callWindowProc handles.
In that case - tough luck I guess.
To extend Chris Becke's answer (which solved my problem, thanks!):
So, there is no GetWindowLongPtr function. Its a macro that resolves to two 'real' functions the windows api provides: GetWindowLongPtrA and GetWindowLongPtrW. If the window is a unicode window, and GetWindowLongPtrA is called windows will return a handle instead of the raw pointer, so that it can intercept calls (made via CallWindowProc) and marshal the string's from ansi to unicode. The opposite conversion holds the other way.
You can check whether the window in question is an unicode or ANSI window by calling the IsWindowUnicode function. Using this information you can determine which GetWindowLongPtr function needs to be called (at runtime),
On POSIX systems rename(2) provides for an atomic rename operation, including overwriting of the destination file if it exists and if permissions allow.
Is there any way to get the same semantics on Windows? I know about MoveFileTransacted() on Vista and Server 2008, but I need this to support Win2k and up.
The key word here is atomic... the solution must not be able to fail in any way that leaves the operation in an inconsistent state.
I've seen a lot of people say this is impossible on win32, but I ask you, is it really?
Please provide reliable citations if possible.
See ReplaceFile() in Win32 (http://research.microsoft.com/pubs/64525/tr-2006-45.pdf)
Win32 does not guarantee atomic file meta data operations. I'd provide a citation, but there is none - that fact that there's no written or documented guarantee means as much.
You're going to have to write your own routines to support this. It's unfortunate, but you can't expect win32 to provide this level of service - it simply wasn't designed for it.
In Windows Vista and Windows Server 2008 an atomic move function has been added - MoveFileTransacted()
Unfortunately this doesn't help with older versions of Windows.
Interesting article here on MSDN.
Starting with Windows 10 1607, NTFS does support an atomic superseding rename operation. To do this call NtSetInformationFile(..., FileRenameInformationEx, ...) and specify the FILE_RENAME_POSIX_SEMANTICS flag.
Or equivalently in Win32 call SetFileInformationByHandle(..., FileRenameInfoEx, ...) and specify the FILE_RENAME_FLAG_POSIX_SEMANTICS flag.
you still have the rename() call on Windows, though I imagine the guarantees you want cannot be made without knowing the filesystem you're using - no guarantees if you're using FAT for instance.
However, you can use MoveFileEx and use the MOVEFILE_REPLACE_EXISTING
and MOVEFILE_WRITE_THROUGH options. The latter has this description in MSDN:
Setting this value guarantees that a
move performed as a copy and delete
operation is flushed to disk before
the function returns. The flush occurs
at the end of the copy operation.
I know that's not necessarily the same as a rename operation, but I think it might be the best guarantee you'll get - if it does that for a file move, it should for a simpler rename.
The MSDN documentation avoids clearly stating which APIs are atomic and which are not, but Niall Douglas states in his Cppcon 2015 talk that the only atomic function is
SetFileInformationByHandle
with FILE_RENAME_INFO.ReplaceIfExists set to true. It's available starting with Windows Vista / 2008 Server.
Niall is the author of a highly complicated LLFIO library and is an expert in file system race conditions so I believe if you're writing an algorithm where atomicity is crucial, better be safe than sorry and use the suggested function even though nothing in ReplaceFile's description states it's not atomic.
A fair number of answers but not the one I was expecting... I had the understanding (perhaps incorrectly) that MoveFile could be atomic provided that the proper stars aligned, flags were used, and file system was the same on the source as target. Otherwise, the operation would fall back to a [Copy->Delete]File.
Given that; I was also had the understanding that MoveFile -- when it is atomic -- was just setting the file information which also could be done here: setfileinfobyhandle.
Someone gave a talk called "Racing the Filesystem" which goes into some more depth about this. (about 2/3rds down they talk about atomic rename)
There is std::rename and starting with C++17 std::filesystem::rename.
It's unspecified what happens if destination exists with std::rename:
If new_filename exists, the behavior is implementation-defined.
POSIX rename, however, is required to replace existing files atomically:
This rename() function is equivalent for regular files to that defined
by the ISO C standard. Its inclusion here expands that definition to
include actions on directories and specifies behavior when the new
parameter names a file that already exists. That specification
requires that the action of the function be atomic.
Thankfully, std::filesystem::rename requires that it behaves just like POSIX:
Moves or renames the filesystem object identified by old_p to new_p as
if by the POSIX rename
However, when I tried to debug, it appears that std::filesystem::rename as implemented by VS2019 (as of March 2020) simply calls MoveFileEx, which isn't atomic in some cases.
So, possibly, when all bugs in its implementation are fixed, we'll see portable atomic std::filesystem::rename.