i am coding for windows 7 and windows 10 and want to have only a single main binary for my application.
In this application i want to join a path and a file name so that all relative path specs are eliminated.
Now i found out that when using PathCombine() function i am portable but the function itself has the chance by design to create buffer overrun issues under certain input conditions.
For PathCchCombine() the security is improved by an extra parameter providing the results buffer size but its not available on the first mentioned platform (the function is only there on Win8 or higher - maybe provided with api-ms-win-core-path-l1-1-0.dll or just by the Windows KernelBase.dll).
How to solve that so that i can keep a single binary, don't need to provide extra DLLs and still stay save against buffer overruns?
Is there some alternate function for Windows 7 that will just serve me?
indicated solution:
i need to use PathCombine() becaue its the only option that works on Win7.
i have to supply a results buffer of MAX_PATH (smaller is risky, bigger is useless).
i have to accept that even if Win10 might support path length of 32kB or more there is no simple solution (like a single API call) that will work with any of the platform determined limits and/or content determined results length. - there are function variants (thinking of PathAllocCanonicalize) that do dynamic allocations and thus the caller does not need any pre-result knowledge, but all of these functions seem to be available only with Win8 or higher.
Related
I am implementing SMBIOS reading functionality for Windows systems. As API levels vary, there are several methods to support:
trouble-free GetSystemFirmwareTable('RSMB') available on Windows Server 2003 and later;
hardcore NtOpenSection(L"\\Device\\PhysicalMemory") for legacy systems prior to and including Windows XP;
essential WMI data in L"Win32_ComputerSystemProduct" path through cumbersome COM automation calls as a fallback.
Methods 1 and 3 are already implemented, but I am stuck with \Device\PhysicalMemory, as NtOpenSection always yields 0xC0000034 (STATUS_OBJECT_NAME_NOT_FOUND) — definitely not one of the possible result codes in the ZwOpenSection documentation. Of course, I am aware that accessing this section is prohibited starting from Windows Server 2003sp1 and perhaps Windows XP-64 as well, so I am trying this on a regular Windows XP-32 system — and the outcome is no different to that of a Windows 7-64, for example. I am also aware that administrator rights may be required even on legacy systems, but people on the internets having faced this issue reported more relevant error codes for such scenario, like 0xC0000022 (STATUS_ACCESS_DENIED) and 0xC0000005 (STATUS_ACCESS_VIOLATION).
My approach is based on the Libsmbios library by Dell, which I assume to be working.
UNICODE_STRING wsMemoryDevice;
OBJECT_ATTRIBUTES oObjAttrs;
HANDLE hMemory;
NTSTATUS ordStatus;
RtlInitUnicodeString(&wsMemoryDevice, L"\\Device\\PhysicalMemory");
InitializeObjectAttributes(&oObjAttrs, &wsMemoryDevice,
OBJ_CASE_INSENSITIVE, NULL, NULL);
ordStatus = NtOpenSection(&hMemory, SECTION_MAP_READ, &oObjAttrs);
if (!NT_SUCCESS(ordStatus)) goto Finish;
I thought it could be possible to debug this, but native API seems to be transparent to debuggers like OllyDbg: the execution immediately returns once SYSENTER instruction receives control. So I have no idea why Windows cannot find this object. I also tried changing the section name, as there are several variants in examples available online, but that always yields 0xC0000033 (STATUS_OBJECT_NAME_INVALID).
Finally, I found the cause of such a strange behavior, — thanks to you, people, confirming that my code snippet (it was an actual excerpt, not a forged example) really works. The problem was that I did not have Windows DDK installed initially (I do have now, but still cannot integrate it with Visual Studio in a way that Windows SDK integrates automatically), so there was a need to write definitions by hand. Particularly, when I realized that InitializeObjectAttributes is actually a preprocessor macro rather than a Win32 function, I defined RtlInitUnicodeString as a macro, too, since its effect is even simpler. However, I was not careful enough to notice that UNICODE_STRING.Length and .MaximumLength are in fact meant for content size and buffer size instead of length, i. e. number of bytes rather than number of characters. Consequently, my macro was setting the fields to a half of their expected value, thus making Windows see only the first half of the L"\\Device\\PhysicalMemory" string, — with obvious outcome.
This question already has answers here:
What is a Windows Handle?
(7 answers)
Closed 9 years ago.
How can i get HANDLE to the memory region or mapped file ?
What actually HANDLE is ?
Please do not answer like this :
A handle is an abstract reference to some resourc,e provided to you by
another party (usually the OS), that you can hand back to reference
that resource.
I'm interested in technical side
A HANDLE is just some arbitrary nugget to some data.
For example, it's returned by the following APIs: CreateFile() and OpenProcess() - as you can tell, these two are very different, yet return the exact same data-type.
Or for memory, you can get access to the heap (which is returned as a HANDLE from GetProcessHeap()) and then use HeapAlloc() against it.
As MSDN points out, it's used in many other contexts:
http://msdn.microsoft.com/en-us/library/windows/apps/ms724211%28v=vs.85%29.aspx
Access token
Communications device
Console input
Console screen buffer
Event
File
File mapping
I/O completion port
Job
Mailslot
Memory resource notification
Mutex
Named pipe
Pipe
Process
Semaphore
Thread
Transaction
Waitable timer
If you do just want to know what it is:
In simple operating systems such handles are really only pointers to a structure in Kernel space. The structure contain more information about the handle itself (e.g. pointer to a file structure). More complex operating systems like Windows will check the validity of a handle before trying to access this structure.
Because the HANDLE is opaque in Windows (this means: Microsoft says: Do not interpret it - it is just a 32-bit number) the actual meaning of a HANDLE may be different in different versions of Windows. Even within one Version (e.g. Vista 32-bit vs. Vista 64-bit or XP SP2 vs XP SP3) the (internal) meaning may be different.
Microsoft may change the meaning anytime they want to change it (it may be a pointer in Windows 95 but an index into an array in Windows 8 - who knows).
Handles in windows used to be pointer to a pointer to memory location of an object. In win 3.1 days some program cast some kind of handle to see what's behind. Today to my knowledge this is not possible anymore, they are simple unique identifier or token to communicate between you and the system. Some more information http://en.wikibooks.org/wiki/Windows_Programming/Handles_and_Data_Types some more from msdn read paragraph object in 16 bits windows http://msdn.microsoft.com/en-us/library/ms810501.aspx
When I thought about resizing images and saving the new sizes parallel on the server, I came to the following question:
// Original size
DSC_18342.jpg
// New size: Use an "x" for "times"
DSC_18342_640x480px.jpg
// New size: Use the real "×" for "times"
DSC_18342_640×480px.jpg
The point is, that it's slightly easier if you got a real × instead of an x in the file name, as the unit px already contains the x, which makes it a little bit harder to read.
Question: What problems could I get in, when using the Html entity in the filename?
Sidenotes: I'm writing an open source, publicly available script, so the targeted server can be anything - therefore I'm also interested (and will vote up) edge cases, that I'm not aware off.
Thank you all!
You may have noticed, that I'm aware, that I could simply avoid it (which I'll do anyway), but I'm interested in this issue and learning about it, so please just take above example as possible case.
There are file systems that simply don't support unicode. This may be less of a problem if you make unicode support a requirement of your application.
Some consideration about different unicode file system are given in File Systems, Unicode, and Normalization.
A concluding remark (from a viewpoint of solaris file systems) is:
Complete compatibility and seamless interoperability with
all other existing Unicode file systems appears not 100%
possible due to inherent differences.
I can imagine that there will be problems especially when migrating the application. Just storing files is probably no problem but if their names are stored in a database there might be a mismatch after migration.
Say there is a buggy program that contains a sprintf() and i want to change it to a snprintf so it doesn't have a buffer overflow.. how do I do that in IDA??
You really don't want to make that kind of change using information from IDA pro.
Although IDA's disassembly is relatively high quality, it's not high quality enough to support executable rewriting. Converting a call to sprintf to a call to snprintf requires pushing a new argument on to the stack. That requires the introduction of a new instruction, which impacts the EA of everything that follows it in the executable image. Updating those effective addresses requires extremely high quality disassembly. In particular, you need to be able to:
Identify which addresses in the executable are data, and which ones are code
Identify which instruction operands are symbolic (address references) and which instruction operands are numeric.
Ida can't (reliably) give you that information. Also, if the executable is statically linked against the crt, it may not contain snpritnf, which would make performing the rewriting by hand VERY difficult.
There are a few potential workarounds. If there is sufficient padding available in (or after) the function making the call, you might be able to get away with only rewriting a single function. Alternatively, if you have access to object files, and those object files were compiled with the /GY switch (assuming you are using Visual Studio) then you may be able to edit the object file. However, editing the object file may still require substantial fix ups.
Presumably, however, if you have access to the object files you probably also have access to the source. Changing the source is probably your best bet.
On POSIX systems rename(2) provides for an atomic rename operation, including overwriting of the destination file if it exists and if permissions allow.
Is there any way to get the same semantics on Windows? I know about MoveFileTransacted() on Vista and Server 2008, but I need this to support Win2k and up.
The key word here is atomic... the solution must not be able to fail in any way that leaves the operation in an inconsistent state.
I've seen a lot of people say this is impossible on win32, but I ask you, is it really?
Please provide reliable citations if possible.
See ReplaceFile() in Win32 (http://research.microsoft.com/pubs/64525/tr-2006-45.pdf)
Win32 does not guarantee atomic file meta data operations. I'd provide a citation, but there is none - that fact that there's no written or documented guarantee means as much.
You're going to have to write your own routines to support this. It's unfortunate, but you can't expect win32 to provide this level of service - it simply wasn't designed for it.
In Windows Vista and Windows Server 2008 an atomic move function has been added - MoveFileTransacted()
Unfortunately this doesn't help with older versions of Windows.
Interesting article here on MSDN.
Starting with Windows 10 1607, NTFS does support an atomic superseding rename operation. To do this call NtSetInformationFile(..., FileRenameInformationEx, ...) and specify the FILE_RENAME_POSIX_SEMANTICS flag.
Or equivalently in Win32 call SetFileInformationByHandle(..., FileRenameInfoEx, ...) and specify the FILE_RENAME_FLAG_POSIX_SEMANTICS flag.
you still have the rename() call on Windows, though I imagine the guarantees you want cannot be made without knowing the filesystem you're using - no guarantees if you're using FAT for instance.
However, you can use MoveFileEx and use the MOVEFILE_REPLACE_EXISTING
and MOVEFILE_WRITE_THROUGH options. The latter has this description in MSDN:
Setting this value guarantees that a
move performed as a copy and delete
operation is flushed to disk before
the function returns. The flush occurs
at the end of the copy operation.
I know that's not necessarily the same as a rename operation, but I think it might be the best guarantee you'll get - if it does that for a file move, it should for a simpler rename.
The MSDN documentation avoids clearly stating which APIs are atomic and which are not, but Niall Douglas states in his Cppcon 2015 talk that the only atomic function is
SetFileInformationByHandle
with FILE_RENAME_INFO.ReplaceIfExists set to true. It's available starting with Windows Vista / 2008 Server.
Niall is the author of a highly complicated LLFIO library and is an expert in file system race conditions so I believe if you're writing an algorithm where atomicity is crucial, better be safe than sorry and use the suggested function even though nothing in ReplaceFile's description states it's not atomic.
A fair number of answers but not the one I was expecting... I had the understanding (perhaps incorrectly) that MoveFile could be atomic provided that the proper stars aligned, flags were used, and file system was the same on the source as target. Otherwise, the operation would fall back to a [Copy->Delete]File.
Given that; I was also had the understanding that MoveFile -- when it is atomic -- was just setting the file information which also could be done here: setfileinfobyhandle.
Someone gave a talk called "Racing the Filesystem" which goes into some more depth about this. (about 2/3rds down they talk about atomic rename)
There is std::rename and starting with C++17 std::filesystem::rename.
It's unspecified what happens if destination exists with std::rename:
If new_filename exists, the behavior is implementation-defined.
POSIX rename, however, is required to replace existing files atomically:
This rename() function is equivalent for regular files to that defined
by the ISO C standard. Its inclusion here expands that definition to
include actions on directories and specifies behavior when the new
parameter names a file that already exists. That specification
requires that the action of the function be atomic.
Thankfully, std::filesystem::rename requires that it behaves just like POSIX:
Moves or renames the filesystem object identified by old_p to new_p as
if by the POSIX rename
However, when I tried to debug, it appears that std::filesystem::rename as implemented by VS2019 (as of March 2020) simply calls MoveFileEx, which isn't atomic in some cases.
So, possibly, when all bugs in its implementation are fixed, we'll see portable atomic std::filesystem::rename.