For 32-bit Windows, following declaration of _SYSTEM_PROCESSES structure ( System Information Class 5 ) with ZwQuerySystemInformation works fine for my purpose to construct process tree.
typedef struct _SYSTEM_PROCESSES
{ // System Information Class 5
ULONG NextEntryDelta;
ULONG ThreadCount;
ULONG Reserved1[6];
LARGE_INTEGER CreateTime;
LARGE_INTEGER UserTime;
LARGE_INTEGER KernelTime;
UNICODE_STRING ProcessName;
ULONG BasePriority;
ULONG ProcessId;
ULONG InheritedFromProcessId;
ULONG HandleCount;
ULONG Reserved2[2];
} SYSTEM_PROCESSES, * PSYSTEM_PROCESSES;
On the other hand, it doesn't function well in 64-bit Windows. If I declare the ProcessId as ULONG64, then the data for ProcessId comes right. Is the datatype defined for above structure is right for Windows-64?
For some weird reason, process and thread id's are 64bit in the kernel and 32bit in the documented windows api on x64
If you look at SYSTEM_PROCESS_INFORMATION # ntinternals you see that they have declared the PID's as HANDLE (pointer sized)
Related
I have an application server built as UNICODE running on Windows only. The application accepts lots of clients using multithreading.
Part of the application is responsible for logging connections, to log IPv4 addresses I convert them to UNICODE from ANSI. I achieve this using inet_ntoa + MultiByteToWideChar.
My frustration is that there is no UNICODE version of inet_ntoa and I am forced to convert each IPv4 to UNICODE. Although MultiByteToWideChar is relatively fast, it is still a performance loss and prone to failure, especially when you have 100,000 connections coming in.
Researching a bit I have found functions like RtlIpv4AddressToStringExW, however I do not have experience with these and applying it to real life applications can be fatal failure without proper testing.
Investigating the above function I have seen that there is no call to MultiByteToWideChar, but instead to RtlIpv4AddressToStringW and swprintf_s, which tells me I really don't need to convert anything (?).
Please advise what is the best way to obtain the readable UNICODE version of IN_ADDR.
Any advice is much appreciated.
Update:
I investigated inet_ntoa and there is quite a lot of code there, retrieving stuff from TLS among other stuff. Overkill from my opinion.
Also investigated WSAAddressToString as proposed by #Alex Guteniev. Again a ton of code, for what purpose...
Also investigated RtlIpv4AddressToStringExW, and my findings worry me again how Windows API's are designed and offered.
Windows Implementation:
LONG __stdcall RtlIpv4AddressToStringExW(const struct in_addr *Address, USHORT Port, PWSTR AddressString, PULONG AddressStringLength)
{
PULONG v4; // rdi
PWSTR v5; // rbp
USHORT v6; // si
wchar_t *v7; // rax
signed __int64 v8; // rbx
ULONG v9; // ebx
LONG result; // eax
WCHAR S; // [rsp+20h] [rbp-68h]
char v12[4]; // [rsp+4Ch] [rbp-3Ch]
v4 = AddressStringLength;
v5 = AddressString;
v6 = Port;
if ( Address && AddressStringLength && (AddressString || !*AddressStringLength) )
{
v7 = RtlIpv4AddressToStringW(Address, &S);
v8 = (signed __int64)v7;
if ( v6 )
v8 = (signed __int64)&v7[swprintf_s(v7, (v12 - (char *)v7) >> 1, L":%u", (unsigned __int16)__ROR2__(v6, 8))];
v9 = (unsigned __int64)((v8 - (signed __int64)&S) >> 1) + 1;
if ( *v4 >= v9 )
{
memmove(v5, &S, 2i64 * v9);
result = 0;
*v4 = v9;
return result;
}
*v4 = v9;
}
return -1073741811;
}
PWSTR __stdcall RtlIpv4AddressToStringW(const struct in_addr *Addr, PWSTR S)
{
__int64 v3; // [rsp+20h] [rbp-28h]
__int64 v4; // [rsp+28h] [rbp-20h]
__int64 v5; // [rsp+30h] [rbp-18h]
LODWORD(v5) = Addr->S_un.S_un_b.s_b4;
LODWORD(v4) = Addr->S_un.S_un_b.s_b3;
LODWORD(v3) = Addr->S_un.S_un_b.s_b2;
return &S[swprintf_s(S, 0x10ui64, L"%u.%u.%u.%u", Addr->S_un.S_un_b.s_b1, v3, v4, v5)];
}
RectOS Implementation:
NTSTATUS
NTAPI
RtlIpv4AddressToStringExW(
_In_ const struct in_addr *Address,
_In_ USHORT Port,
_Out_writes_to_(*AddressStringLength, *AddressStringLength) PWCHAR AddressString,
_Inout_ PULONG AddressStringLength)
{
WCHAR Buffer[IPV4_ADDR_STRING_MAX_LEN + IPV4_PORT_STRING_MAX_LEN];
NTSTATUS Status;
ULONG Length;
PWSTR End;
if (!Address || !AddressString || !AddressStringLength)
return STATUS_INVALID_PARAMETER;
Status = RtlStringCchPrintfExW(Buffer,
RTL_NUMBER_OF(Buffer),
&End,
NULL,
0,
Port ? L"%u.%u.%u.%u:%u"
: L"%u.%u.%u.%u",
Address->S_un.S_un_b.s_b1,
Address->S_un.S_un_b.s_b2,
Address->S_un.S_un_b.s_b3,
Address->S_un.S_un_b.s_b4,
WN2H(Port));
ASSERT(Status == STATUS_SUCCESS);
if (!NT_SUCCESS(Status))
return STATUS_INVALID_PARAMETER;
Length = End - AddressString;
if (*AddressStringLength > Length)
{
Status = RtlStringCchCopyW(AddressString,
*AddressStringLength,
Buffer);
ASSERT(Status == STATUS_SUCCESS);
*AddressStringLength = Length + 1;
return STATUS_SUCCESS;
}
*AddressStringLength = Length + 1;
return STATUS_INVALID_PARAMETER;
}
My opinion here is that the ReactOS implementation is FAR better than the one in Windows. I believe RtlStringCchPrintfExW is much safer than swprintf_s.
In the end, is it safe for me (and anyone else) to create their own implementation of formatting?
Use WSAAddressToStringW if you want a generic function to convert addresses.
These Windows API are not necessarily optimized for throughput, instead they probably handle different addresses type.
If you are limited to specific address types, like IPv4 only, you can probably create your own implementation, if performance that much matters.
Consider SIMD (SSE, AVX) if you need top performance. If you just need a simple implementation, consider just using swprintf (or calling _itow few times if you mind format specification parsing overhead)
The last member WCHAR Name[1] is assumed to be a string. It's a well-known technic.
// c:\Program Files (x86)\Windows Kits\8.0\Include\um\DbgHelp.h
typedef struct _SYMBOL_INFOW {
ULONG SizeOfStruct;
ULONG TypeIndex; // Type Index of symbol
ULONG64 Reserved[2];
ULONG Index;
ULONG Size;
ULONG64 ModBase; // Base Address of module comtaining this symbol
ULONG Flags;
ULONG64 Value; // Value of symbol, ValuePresent should be 1
ULONG64 Address; // Address of symbol including base address of module
ULONG Register; // register holding value or pointer to value
ULONG Scope; // scope of the symbol
ULONG Tag; // pdb classification
ULONG NameLen; // Actual length of name
ULONG MaxNameLen;
WCHAR Name[1]; // Name of symbol
} SYMBOL_INFOW, *PSYMBOL_INFOW;
But the following code generates V641. Why?
PSYMBOL_INFOW pSym = NULL;
pSym = (PSYMBOL_INFOW) malloc(sizeof(SYMBOL_INFOW) + MAX_SYM_NAME);
Thank you for the interest to our product and the issues you described. We will examine the provided code fragments and will try to fix these issues.
Is there a way to check how many views have been mapped on to a memory mapped file on Windows?
Something like the equivalent of shmctl(... ,IPC_STAT,...) on Linux?
I had the same need to access the number of shared views. So I created this question: Accessing the number of shared memory mapped file views (Windows)
You may find a solution that suits your needs there.
As per Scath comment, I am going to add here the proposed solution, although merit should go to eryksun and RbMm. Making use of NtQueryObject call one can access the HandleCount (although it may not be 100% reliable):
#include <stdio.h>
#include <windows.h>
#include <winternl.h>
typedef NTSTATUS (__stdcall *NtQueryObjectFuncPointer) (
HANDLE Handle,
OBJECT_INFORMATION_CLASS ObjectInformationClass,
PVOID ObjectInformation,
ULONG ObjectInformationLength,
PULONG ReturnLength);
int main(void)
{
_PUBLIC_OBJECT_BASIC_INFORMATION pobi;
ULONG rLen;
// Create the memory mapped file (in system pagefile) (better in global namespace
// but needs SeCreateGlobalPrivilege privilege)
HANDLE hMap = CreateFileMapping(INVALID_HANDLE_VALUE, NULL, PAGE_READWRITE|SEC_COMMIT,
0, 1, "Local\\UniqueShareName");
// Get the NtQUeryObject function pointer and then the handle basic information
NtQueryObjectFuncPointer _NtQueryObject = (NtQueryObjectFuncPointer)GetProcAddress(
GetModuleHandle("ntdll.dll"), "NtQueryObject");
_NtQueryObject(hMap, ObjectBasicInformation, (PVOID)&pobi, (ULONG)sizeof(pobi), &rLen);
// Check limit
if (pobi.HandleCount > 4) {
printf("Limit exceeded: %ld > 4\n", pobi.HandleCount);
exit(1);
}
//...
Sleep(30000);
}
I used the 'ntQuerySystemInformation' to get all the handle information like:
NtQuerySystemInformation(SystemHandleInformation, pHandleInfor, ulSize,NULL);//SystemHandleInformation = 16
struct of pHandleInfor is:
typedef struct _SYSTEM_HANDLE_INFORMATION
{
ULONG ProcessId;
UCHAR ObjectTypeNumber;
UCHAR Flags;
USHORT Handle;
PVOID Object;
ACCESS_MASK GrantedAccess;
} SYSTEM_HANDLE_INFORMATION, *PSYSTEM_HANDLE_INFORMATION;
It works well in xp 32bit, but in Win7 64bit can only get the right pid that less than 65535. The type of processId in this struct is ULONG, I think it can get more than 65535. What's wrong with it? Is there any other API instead?
There are two enum values for NtQuerySystemInformation to get handle info:
CNST_SYSTEM_HANDLE_INFORMATION = 16
CNST_SYSTEM_EXTENDED_HANDLE_INFORMATION = 64
And correspondingly two structs: SYSTEM_HANDLE_INFORMATION and SYSTEM_HANDLE_INFORMATION_EX.
The definitions for these structs are:
struct SYSTEM_HANDLE_INFORMATION
{
short UniqueProcessId;
short CreatorBackTraceIndex;
char ObjectTypeIndex;
char HandleAttributes; // 0x01 = PROTECT_FROM_CLOSE, 0x02 = INHERIT
short HandleValue;
size_t Object;
int GrantedAccess;
}
struct SYSTEM_HANDLE_INFORMATION_EX
{
size_t Object;
size_t UniqueProcessId;
size_t HandleValue;
int GrantedAccess;
short CreatorBackTraceIndex;
short ObjectTypeIndex;
int HandleAttributes;
int Reserved;
}
As You can see, the first struct really can only contain 16-bit process id-s...
See for example ProcessExplorer project's source file ntexapi.h for more information.
Note also that the field widths for SYSTEM_HANDLE_INFORMATION_EX in my struct definitions might be different from theirs (that is, in my definition some field widths vary depending on the bitness), but I think I tested the code both under 32-bit and 64-bit and found it to be correct.
Please recheck if necessary and let us know if You have additional info.
From Raymond Chen's article Processes, commit, RAM, threads, and how high can you go?:
I later learned that the Windows NT folks do try to keep the numerical values of process ID from getting too big. Earlier this century, the kernel team experimented with letting the numbers get really huge, in order to reduce the rate at which process IDs get reused, but they had to go back to small numbers, not for any technical reasons, but because people complained that the large process IDs looked ugly in Task Manager. (One customer even asked if something was wrong with his computer.)
typedef struct _FILE_BOTH_DIR_INFORMATION {
ULONG NextEntryOffset;
ULONG FileIndex;
LARGE_INTEGER CreationTime;
LARGE_INTEGER LastAccessTime;
LARGE_INTEGER LastWriteTime;
LARGE_INTEGER ChangeTime;
LARGE_INTEGER EndOfFile;
LARGE_INTEGER AllocationSize;
ULONG FileAttributes;
ULONG FileNameLength;
ULONG EaSize;
CCHAR ShortNameLength;
WCHAR ShortName[12];
WCHAR FileName[1];
} FILE_BOTH_DIR_INFORMATION, *PFILE_BOTH_DIR_INFORMATION;
FileNameLength is declared ULONG. I guessed this is byte-count because all(or most) string lengths are byte-count in the kernel.
Yesterday, I wrote wrong code, because I misunderstood that it means count of char when I see CCHAR ShortNameLength. Now, I know ShortNameLength requires bytes-count.
Then, what does C mean in CCHAR?
C means count in Hungarian Notation. A variable named cch would be a "count of chars" and you would expect it to contain a string length. So CCHAR is the type that can contain a count of chars.
It's a horrible abuse of Hungarian notation typical of the Windows team.