SetFilePointerEx fails to read physical disk beyond size of LONG - winapi

It's taken a few years, but I am finally taking the plunge into VC++. I need to be able to read x number of sectors of a physical device (namely a hard drive). I am using the CreateFile() and SetFilePointerEx() and ReadFile() APIs.
I have done a LOT of reading online in all the major forums about this topic. I have exhausted my research and now I feel it's time to ask the experts to weigh in on this dilemma. As this is my very first post ever on this topic, please go easy on my :)
I should also point out that this is a .DLL that I consume with a simple C# app. The plumbing all works fine. It's the SetFilePointer(Ex)() APIs that are causing me grief.
I can get the code to work up until about the size of a LONG (4,xxx,xxx) - I can't remember the exact value. It suffices to say that I can read everything up to and including sector # 4,000,000 but not 5,000,000 or above. The problem lies in the "size" of the parameters for the SetFilePointer() and SetFilePointerEx() APIs. I've tried both and so far, SetFilePointerEx() seems to be what I should use to work on 64-bit systems.
The 2nd and 3rd parameters of the SetFilePointer are defined as follows:
BOOL WINAPI SetFilePointerEx(
__in HANDLE hFile,
__in LARGE_INTEGER liDistanceToMove,
__out_opt PLARGE_INTEGER lpNewFilePointer,
__in DWORD dwMoveMethod
);
Please note that I have tried passing the LowPart and the HighPart as the 2nd and 3 parameters without any success as I get a CANNOT CONVERT LARGE_INTEGER TO PLARGE_INTEGER (for parameter 3).
HERE IS MY CODE. I USE A CODE-BREAK TO VIEW buff[0], etc. I would like to read past the 4,xxx,xxx limitation. Obviously I am doing something wrong. Each read past this limit resets my file pointer to sector 0.
#include "stdafx.h"
#include <windows.h>
#include <conio.h>
extern "C"
__declspec(dllexport) int ReadSectors(long startSector, long numSectors)
{
HANDLE hFile;
const int SECTOR_SIZE = 512;
const int BUFFER_SIZE = 512;
LARGE_INTEGER liDistanceToMove;
PLARGE_INTEGER newFilePtr = NULL; // not used in this context.
// just reading from START to END
liDistanceToMove.QuadPart = startSector * SECTOR_SIZE;
DWORD dwBytesRead, dwPos;
LPCWSTR fname = L"\\\\.\\PHYSICALDRIVE0";
char buff[BUFFER_SIZE];
// Open the PHYSICALDEVICE as a file.
hFile = CreateFile(fname,
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL);
// Here's the API definition
/*BOOL WINAPI SetFilePointerEx(
__in HANDLE hFile,
__in LARGE_INTEGER liDistanceToMove,
__out_opt PLARGE_INTEGER lpNewFilePointer,
__in DWORD dwMoveMethod
);*/
dwPos = SetFilePointerEx(hFile, liDistanceToMove, NULL, FILE_BEGIN);
if(ReadFile(hFile, buff, BUFFER_SIZE, &dwBytesRead, NULL))
{
if(dwBytesRead > 5)
{
BYTE x1 = buff[0];
BYTE x2 = buff[1];
BYTE x3 = buff[2];
BYTE x4 = buff[3];
BYTE x5 = buff[4];
}
}
// Close both files.
CloseHandle(hFile);
return 0;
}

startSector * SECTOR_SIZE;
startSector is a long (32bits), SECTOR_SIZE is a int (also 32bits), multiply these two guys and the intermediate result is going to be a long, which will overflow and you then stuff it into the __int64 of the LARGE_INTEGER, which is too late. You want to operate on __int64s, something like
liDistanceToMove.QuadPart = startSector;
liDistanceToMove.QuadPart *= SECTOR_SIZE;
for example.

Related

inet_ntoa UNICODE or similar (convert IN_ADDR to UNICODE)

I have an application server built as UNICODE running on Windows only. The application accepts lots of clients using multithreading.
Part of the application is responsible for logging connections, to log IPv4 addresses I convert them to UNICODE from ANSI. I achieve this using inet_ntoa + MultiByteToWideChar.
My frustration is that there is no UNICODE version of inet_ntoa and I am forced to convert each IPv4 to UNICODE. Although MultiByteToWideChar is relatively fast, it is still a performance loss and prone to failure, especially when you have 100,000 connections coming in.
Researching a bit I have found functions like RtlIpv4AddressToStringExW, however I do not have experience with these and applying it to real life applications can be fatal failure without proper testing.
Investigating the above function I have seen that there is no call to MultiByteToWideChar, but instead to RtlIpv4AddressToStringW and swprintf_s, which tells me I really don't need to convert anything (?).
Please advise what is the best way to obtain the readable UNICODE version of IN_ADDR.
Any advice is much appreciated.
Update:
I investigated inet_ntoa and there is quite a lot of code there, retrieving stuff from TLS among other stuff. Overkill from my opinion.
Also investigated WSAAddressToString as proposed by #Alex Guteniev. Again a ton of code, for what purpose...
Also investigated RtlIpv4AddressToStringExW, and my findings worry me again how Windows API's are designed and offered.
Windows Implementation:
LONG __stdcall RtlIpv4AddressToStringExW(const struct in_addr *Address, USHORT Port, PWSTR AddressString, PULONG AddressStringLength)
{
PULONG v4; // rdi
PWSTR v5; // rbp
USHORT v6; // si
wchar_t *v7; // rax
signed __int64 v8; // rbx
ULONG v9; // ebx
LONG result; // eax
WCHAR S; // [rsp+20h] [rbp-68h]
char v12[4]; // [rsp+4Ch] [rbp-3Ch]
v4 = AddressStringLength;
v5 = AddressString;
v6 = Port;
if ( Address && AddressStringLength && (AddressString || !*AddressStringLength) )
{
v7 = RtlIpv4AddressToStringW(Address, &S);
v8 = (signed __int64)v7;
if ( v6 )
v8 = (signed __int64)&v7[swprintf_s(v7, (v12 - (char *)v7) >> 1, L":%u", (unsigned __int16)__ROR2__(v6, 8))];
v9 = (unsigned __int64)((v8 - (signed __int64)&S) >> 1) + 1;
if ( *v4 >= v9 )
{
memmove(v5, &S, 2i64 * v9);
result = 0;
*v4 = v9;
return result;
}
*v4 = v9;
}
return -1073741811;
}
PWSTR __stdcall RtlIpv4AddressToStringW(const struct in_addr *Addr, PWSTR S)
{
__int64 v3; // [rsp+20h] [rbp-28h]
__int64 v4; // [rsp+28h] [rbp-20h]
__int64 v5; // [rsp+30h] [rbp-18h]
LODWORD(v5) = Addr->S_un.S_un_b.s_b4;
LODWORD(v4) = Addr->S_un.S_un_b.s_b3;
LODWORD(v3) = Addr->S_un.S_un_b.s_b2;
return &S[swprintf_s(S, 0x10ui64, L"%u.%u.%u.%u", Addr->S_un.S_un_b.s_b1, v3, v4, v5)];
}
RectOS Implementation:
NTSTATUS
NTAPI
RtlIpv4AddressToStringExW(
_In_ const struct in_addr *Address,
_In_ USHORT Port,
_Out_writes_to_(*AddressStringLength, *AddressStringLength) PWCHAR AddressString,
_Inout_ PULONG AddressStringLength)
{
WCHAR Buffer[IPV4_ADDR_STRING_MAX_LEN + IPV4_PORT_STRING_MAX_LEN];
NTSTATUS Status;
ULONG Length;
PWSTR End;
if (!Address || !AddressString || !AddressStringLength)
return STATUS_INVALID_PARAMETER;
Status = RtlStringCchPrintfExW(Buffer,
RTL_NUMBER_OF(Buffer),
&End,
NULL,
0,
Port ? L"%u.%u.%u.%u:%u"
: L"%u.%u.%u.%u",
Address->S_un.S_un_b.s_b1,
Address->S_un.S_un_b.s_b2,
Address->S_un.S_un_b.s_b3,
Address->S_un.S_un_b.s_b4,
WN2H(Port));
ASSERT(Status == STATUS_SUCCESS);
if (!NT_SUCCESS(Status))
return STATUS_INVALID_PARAMETER;
Length = End - AddressString;
if (*AddressStringLength > Length)
{
Status = RtlStringCchCopyW(AddressString,
*AddressStringLength,
Buffer);
ASSERT(Status == STATUS_SUCCESS);
*AddressStringLength = Length + 1;
return STATUS_SUCCESS;
}
*AddressStringLength = Length + 1;
return STATUS_INVALID_PARAMETER;
}
My opinion here is that the ReactOS implementation is FAR better than the one in Windows. I believe RtlStringCchPrintfExW is much safer than swprintf_s.
In the end, is it safe for me (and anyone else) to create their own implementation of formatting?
Use WSAAddressToStringW if you want a generic function to convert addresses.
These Windows API are not necessarily optimized for throughput, instead they probably handle different addresses type.
If you are limited to specific address types, like IPv4 only, you can probably create your own implementation, if performance that much matters.
Consider SIMD (SSE, AVX) if you need top performance. If you just need a simple implementation, consider just using swprintf (or calling _itow few times if you mind format specification parsing overhead)

Windows memory metric to detect memory leak

We have large old legacy server code running as a 64bit windows service.
The service has a memory leak which, at the moment, we do not have the resources to fix.
As the service is resilient to restart, a temporary terrible 'solution' we want is to detect when the service's memory exceeded, e.g., 5GB, and exit the service (which has auto restart for such cases).
My question is which metric should I go for? Is using GlobalMemoryStatusEx to get
MEMORYSTATUSEX.ullTotalVirtual- MEMORYSTATUSEX.ullAvailVirtual right?
GlobalMemoryStatusEx is wrong. You do not want to fill up the machine memory until 5 GB are left in total.
You need GetProcessMemoryInfo.
BOOL WINAPI GetProcessMemoryInfo(
__in HANDLE Process,
__out PPROCESS_MEMORY_COUNTERS ppsmemCounters,
__in DWORD cb
);
From an example using GetProcessMemoryInfo:
#include <windows.h>
#include <stdio.h>
#include <psapi.h>
// To ensure correct resolution of symbols, add Psapi.lib to TARGETLIBS
// and compile with -DPSAPI_VERSION=1
void PrintMemoryInfo( DWORD processID )
{
HANDLE hProcess;
PROCESS_MEMORY_COUNTERS pmc;
// Print the process identifier.
printf( "\nProcess ID: %u\n", processID );
// Print information about the memory usage of the process.
hProcess = OpenProcess( PROCESS_QUERY_INFORMATION |
PROCESS_VM_READ,
FALSE, processID );
if (NULL == hProcess)
return;
if ( GetProcessMemoryInfo( hProcess, &pmc, sizeof(pmc)) )
{
printf( "\tWorkingSetSize: 0x%08X\n", pmc.WorkingSetSize );
printf( "\tPagefileUsage: 0x%08X\n", pmc.PagefileUsage );
}
CloseHandle( hProcess );
}
int main( void )
{
// Get the list of process identifiers.
DWORD aProcesses[1024], cbNeeded, cProcesses;
unsigned int i;
if ( !EnumProcesses( aProcesses, sizeof(aProcesses), &cbNeeded ) )
{
return 1;
}
// Calculate how many process identifiers were returned.
cProcesses = cbNeeded / sizeof(DWORD);
// Print the memory usage for each process
for ( i = 0; i < cProcesses; i++ )
{
PrintMemoryInfo( aProcesses[i] );
}
return 0;
}
Although unintuitive you need to read PagefileUsage which gets you the committed memory which was allocated by your process. WorkingSetSize is unreliable because if the machine gets tight on memory the OS will write all data to the page file. That can cause WorkingSetSize to be small (e.g. 100 MB) but in reality you leaked already 20 GB of memory. This would result in a saw tooth pattern in memory consumption until the page file is full. Working set is only the actively used memory which might hide the multi GB memory leak if the machine is under memory pressure.

How to write memory of a dll?

I'm trying to replace a specific string in memory belongs to a dll. Here's the code.
I can read it, and it gives me correct result, but when writing VC++ shows 'Access violation writing location'.
HMODULE HMODULE1 = LoadLibrary(L"my.dll");
std::string x1(8, '\0');
std::string x2 = "CIFCDMEY";
auto startPos = (void*)((char*)(HMODULE1)+0x1158A0 + 9);
// Correct, I can read the memory
memcpy_s((void*)x1.data(), x1.size(), startPos, x1.size());
// Access violation writing location
memcpy_s(startPos, x2.size(), x2.data(), x2.size());
auto handle = GetCurrentProcess();
SIZE_T num;
auto ret = WriteProcessMemory(handle, startPos, x2.data(), x2.size(), &num);
auto lastError1 = GetLastError();
LPVOID lpMessageBuffer1 = NULL;
size_t size1 = ::FormatMessage(
FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS,
NULL,
lastError1,
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
(LPTSTR)&lpMessageBuffer1,
0,
NULL);
std::wstring errorMessage1;
if (size1 > 0) {
// errorMessage1: Invalid access to memory location.
errorMessage1 = std::wstring((LPCTSTR)lpMessageBuffer1, size1);
}
In 'Watch' window, the value of variable startPos is my.dll!0x0f2a58a9 (load symbols for additional information).
I know people use 'WriteProcessMemory' to write memory of a process, how about a dll?
If the target memory page does not have write permissions you need to take them using VirtualProtect()
I use a simple wrapper function for all my patching, it uses the PAGE_EXECUTE_READWRITE memory protection constant because if you are modifying a code page this will avoid crashes when the instruction pointer lands in the same memory page and you only used PAGE_READWRITE
void Patch(char* dst, char* src, const intptr_t size)
{
DWORD oldprotect;
VirtualProtect(dst, size, PAGE_EXECUTE_READWRITE, &oldprotect);
memcpy(dst, src, size);
VirtualProtect(dst, size, oldprotect, &oldprotect);
}

Segmentation fault when changing default gateway

I wrote a simple application on Qt4 that modifier network adapter parameters, for that I have a slot called setInterfaceParams, implemented as so:
DWORD WinNetInterface::setInterfaceParams(QString index, QString ip, QString netmask, QString gateway)
{
DWORD res = NULL;
HINSTANCE lib = (HINSTANCE) LoadLibrary((WCHAR *)"iphlpapi.dll");
_SetAdapterIpAddress SetAdapterIpAddress = (_SetAdapterIpAddress) GetProcAddress(lib, "SetAdapterIpAddress");
PWSTR pszGUID = NULL;
//char *szGUID = (char *)index.toStdString().c_str();
QByteArray a = index.toLocal8Bit();
char *szGUID = a.data();
WideCharToMultiByte(CP_ACP, 0, pszGUID, -1, szGUID, sizeof(szGUID), NULL, NULL);
// Method 01
res = SetAdapterIpAddress(szGUID,
0,
inet_addr(ip.toStdString().c_str()),
inet_addr(netmask.toStdString().c_str()),
inet_addr(gateway.toStdString().c_str()));
// End of method 01
// Method 02
/*res = SetAdapterIpAddress("{422C5689-A17B-402D-A6A2-22CE13E857B5}",
0,
inet_addr("192.168.1.10"),
inet_addr("255.255.255.0"),
inet_addr("192.168.1.1"));*/
// End of method 02
return res;
}
When I click on button that connected to slot setInterfaceParams, I get segmentation fault. If I comment method01, nothing happen, the some thing happen when I use method02.
I tried this function on a simple c++ application and it is work fine, test on Windows XP SP3.
#include <windows.h>
#include <winsock2.h>
#include <iphlpapi.h>
#include <stdio.h>
#include <iostream>
typedef DWORD (WINAPI *_SetAdapterIpAddress )(char *szAdapterGUID,
DWORD dwDHCP,
DWORD dwIP,
DWORD dwMask,
DWORD dwGateway);
int main()
{
HINSTANCE lib = (HINSTANCE) LoadLibrary("iphlpapi.dll");
_SetAdapterIpAddress SetAdapterIpAddress = (_SetAdapterIpAddress) GetProcAddress(lib, "SetAdapterIpAddress");
PWSTR pszGUID = NULL;
char szGUID[] = "{422C5689-A17B-402D-A6A2-22CE13E857B5}";
DWORD dwSize = 0;
WideCharToMultiByte(CP_ACP, 0, pszGUID, -1, szGUID, sizeof(szGUID), NULL, NULL);
DWORD res = SetAdapterIpAddress(szGUID,
0,
inet_addr("192.168.1.10"),
inet_addr("255.255.255.0"),
inet_addr("192.168.1.1"));
std::cout << res;
return 0;
}
LoadLibrary((WCHAR *)"iphlpapi.dll");
That can't work, the literal string is in 8-bits, casting it without real conversion doesn't make it wide, so the dll loading probably failed.
You should use the TEXT or _T macro around most of the literal strings passed to WinAPI functions to make them regular or wide depending on the compilation options:
LoadLibrary(_T("iphlpapi.dll"));
which will translate to either LoadLibrary("iphlpapi.dll"); or LoadLibrary(L"iphlpapi.dll");.
Also you should always check the value returned by the LoadLibrary and GetProcAddress functions, which return NULL if the call is unsuccessful.

Creating a file exceeding filepath limit

I have a test that creates a series of folders in a loop until it exceeds the MAX_PATH (260). This returns ERROR_PATH_NOT_FOUND(0x3). We have a build machine that runs this test but on the build machine it returns ERROR_FILENAME_EXCED_RANGE (0xce).
My machine is Windows 7 but the build machine is Vista. Could that be why they return different values? If not, does anyone know why this might happen?
EDIT: I am expecting to get an error, im testing a file system driver. I just do not understand why i am getting two different error codes from the same test on different machines.
Here is the code
homeDir << "C:\Users\me\TestFolder";
string childDir = "\\LongChildDirectoryName";
string dir = homeDir.str();
DWORD lastErr = ERROR_SUCCESS;
while(lastErr == ERROR_SUCCESS)
{
int len = dir.size();
if(len > (MAX_PATH - 12))
{
CuFail(tc, "Filepath greater than max allowed should be");
}
dir += childDir;
if(!CreateDirectory(dir.c_str(), NULL))
{
lastErr = GetLastError();
if (lastErr == ERROR_ALREADY_EXISTS)
lastErr = ERROR_SUCCESS;
}
}
CuAssert(tc, "Check error is ERROR_PATH_NOT_FOUND", lastErr == ERROR_PATH_NOT_FOUND);
The logic is flawed. If homeDir.str() returns a name that doesn't exist, the return value from CreateDirectory will be ERROR_PATH_NOT_FOUND. You can demonstrate the problem by simply doing this:
string childDir("\\LongChildDirectoryName");
string dir("foo");
The CreateDirectory call will then get the path foo\LongChildDirectoryName, and if foo doesn't exist, you get ERROR_PATH_NOT_FOUND. The fix is simply to add this before the while loop:
CreateDirectory(dir.c_str(), NULL);
You also need to move the length check after the strings have been concatenated, not before. Using the "\\?\" syntax Alex suggested would also be a good idea.
To use longer paths you need to use the "wide" version of CreateFile(), CreateFileW().
See this MSDN article on the topic:
HANDLE WINAPI CreateFile(
__in LPCTSTR lpFileName,
__in DWORD dwDesiredAccess,
__in DWORD dwShareMode,
__in_opt LPSECURITY_ATTRIBUTES lpSecurityAttributes,
__in DWORD dwCreationDisposition,
__in DWORD dwFlagsAndAttributes,
__in_opt HANDLE hTemplateFile
);
lpFileName [in]
The name of the file or device to be created or opened.
In the ANSI version of this function, the name is limited to MAX_PATH characters.
To extend this limit to 32,767 wide characters, call the Unicode version of the
function and prepend "\\?\" to the path. For more information, see Naming Files,
Paths, and Namespaces.

Resources