What is the win32 API function for private bytes (the ones you can see in perfmon).
I'd like to avoid the .NET API
BOOL WINAPI GetProcessMemoryInfo(
__in HANDLE Process,
__out PPROCESS_MEMORY_COUNTERS ppsmemCounters,
__in DWORD cb
);
Where ppsmemCounters parameter can be a PROCESS_MEMORY_COUNTERS or PROCESS_MEMORY_COUNTERS_EX structure. Just typecast PROCESS_MEMORY_COUNTERS_EX to PROCESS_MEMORY_COUNTERS.
PROCESS_MEMORY_COUNTERS_EX.PrivateUsage is what you're looking for.
More info here and here
You can collect the same data perfmon shows using the performance counters API
You need to clarify what you are trying to do. These are internal figures whose value is not really controlled by any API.
Technically Private Bytes is the commit charge, the amount of memory allocated in the swap file to hold the contents of the applications private memory should it be swapped out.
Generally private bytes = amount of dynamically allocated memory + some extra.
Related
I'm currently working on a lecture (and learning) about rootkits for Windows. I was able to hook the SSDT entry for NtWriteFile and display a simple message on WinDbg, but i'm now curious about what would be the best (and the safest) way of changing the parameters before passing them to the original function. In this example, i'm trying to change the buffer if it contains "My String". How could i swap the content of Buffer safely?
NTSTATUS ZwWriteFileHook(
IN HANDLE FileHandle,
IN HANDLE Event,
IN PIO_APC_ROUTINE ApcRoutine,
IN PVOID ApcContext,
IN PIO_STATUS_BLOCK IoStatusBlock,
IN PVOID Buffer,
IN ULONG Length,
IN PLARGE_INTEGER ByteOffset,
IN PULONG Key
)
{
...
if (!strncmp((PCHAR) Buffer, "My String", Length)) {
// Modify parameters here
}
ntStatus = ((PZwWriteFile) zwWriteFileOld)(FileHandle, Event,
ApcRoutine, ApcContext, IoStatusBlock, Buffer, Length, ByteOffset,
Key);
...
}
Thank you.
First of all, there is an issue with accessing (reading/writing) UserMode pointers directly like that, you first want to probe the pointer for READ, if you write to it, you also need to to probe for WRITE using the kernel APIs ProbeForRead and ProbeForWrite.
Second, I rather not write to the UserMode supplied buffer a modified value but allocate a new buffer and then free it afterwards, in my opinion it is the safest way.
I'm working on a benchmark tool which among other things measures the time and memory used by an external process performing an operation. I'm mostly interested in the peak pageable memory size (a.k.a. PageFileBytesPeak performance counter / Process.PeakPagedMemorySize64 / peak private bytes). This is a .NET project so a pure .NET solution would be preferable, however this is most likely not a possibility.
The problem here is that I won't know the peak memory usage before the process has exited. I can't read the performance counters for the process when it no longer exists. So I could instead poll it while the process is running.
However this is not preferable as if I poll too often I will interfere with the time it takes for the process to complete its work, and if poll too rarely the result won't be accurate (the process will most likely hit its peak memory usage right before it exits). So I'm hoping there is some way to do it reliably that is less hacky than the solutions I have come up with so far:
Inject DLL into process, report value via IPC mechanism on DLL_PROCESS_DETACH.
Patch/Hook ExitProcess in target process, report value via IPC mechanism before executing real ExitProcess.
Pretend to be a debugger, measure value on EXIT_PROCESS_DEBUG_EVENT (the process won't be cleaned up by the kernel before ContinueDebugEvent is called).
Reading extant PerfMon counters should be a very low overhead operation, esp. for system counters like the ones you want to work with, since the counters are typically (possibly always? not sure) implemented using a block of shared memory (mapped file).
I'd implement polling with a runtime configurable interval, and only resort to more complex techniques if you find this is affecting your application materially. If you want to sanity check this first, set up PerfMon to monitor the counter(s) of interest and see if that kills your application when running at a usable refresh interval.
Turns out that GetProcessMemoryInfo works even after the process has exited as long as you have an active handle to it. The virtual memory usage isn't available this way though if you happen to need that.
Only caveat is that the size of the values depends on the bitness of the process it's called from, so the values may overflow if a 32 bit process measures the memory usage of a 64 bit process.
Example:
[DllImport("psapi.dll", SetLastError=true)]
static extern bool GetProcessMemoryInfo(IntPtr hProcess, out PROCESS_MEMORY_COUNTERS counters, int size);
[StructLayout(LayoutKind.Sequential)]
private struct PROCESS_MEMORY_COUNTERS
{
public uint cb;
public uint PageFaultCount;
public UIntPtr PeakWorkingSetSize;
public UIntPtr WorkingSetSize;
public UIntPtr QuotaPeakPagedPoolUsage;
public UIntPtr QuotaPagedPoolUsage;
public UIntPtr QuotaPeakNonPagedPoolUsage;
public UIntPtr QuotaNonPagedPoolUsage;
public UIntPtr PagefileUsage;
public UIntPtr PeakPagefileUsage;
}
public long BenchmarkProcessMemoryUsage(string fileName, string arguments)
{
ProcessStartInfo startInfo = new ProcessStartInfo(fileName, arguments);
startInfo.UseShellExecute = false;
Process process = Process.Start();
process.WaitForExit();
PROCESS_MEMORY_COUNTERS counters;
if (!GetProcessMemoryInfo(process.Handle, out counters, Marshal.SizeOf(typeof(PROCESS_MEMORY_COUNTERS))))
throw new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error());
return (long)counters.PeakPagefileUsage;
}
I have a large binary file (about 1 GB) that I want to process sequentially. I'm using a memory mapped file to access the file. Is there a way to tell Windows to swap in the whole file as soon as possible? Currently, it looks as if windows is only loading a single page each time a page fault occurs which results in very slow processing.
I'm really surprised nobody's mentioned PrefetchVirtualMemory:
BOOL WINAPI PrefetchVirtualMemory(
_In_ HANDLE hProcess,
_In_ ULONG_PTR NumberOfEntries,
_In_ PWIN32_MEMORY_RANGE_ENTRY VirtualAddresses,
_In_ ULONG Flags
);
Here are some other links:
http://msdn.microsoft.com/en-us/library/ms810613.aspx
Performance of Win32 memory mapped files vs. CRT fopen/fread
http://duartes.org/gustavo/blog/post/page-cache-the-affair-between-memory-and-files
Personally, I have no idea whether any of this is a Good Idea or not for your application. But sure - try it, and see if it helps!
IMHO ..
When you wish to force a chunk of the file to be paged in to memory, call ReadFile using the file handle that you used to create the file mapping.
After quite a bit of googling and some hints given here, I finally managed to find a layout of the FS segment (used by windows to store TIB data). Of particular interest to me is the ArbitraryUserPointer member provided in the PSDK:
typedef struct _NT_TIB {
struct _EXCEPTION_REGISTRATION_RECORD *ExceptionList;
PVOID StackBase;
PVOID StackLimit;
PVOID SubSystemTib;
union {
PVOID FiberData;
DWORD Version;
};
PVOID ArbitraryUserPointer;
struct _NT_TIB *Self;
} NT_TIB;
How safe exactly is it to use this variable (under Vista and above)? and does it still exist on x64?
Secondary to that is the access of this variable. I'm using MSVC, and as such I
have access to the __readfsdword & __readgsqword intrinsics, however, MSDN for some reason marks these as privileged instructions:
These intrinsics are only available in kernel mode, and the routines are only available as intrinsics.
They are of course not kernel only, but why are they marked as such, just incorrect documentation? (my offline VS 2008 docs don't have this clause).
Finally, is it safe to access ArbitraryUserPointer directly via a single __readfsdword(0x14) or is it preferred to use it via the linear TIB address? (which will still require a read from FS).
ArbitraryUserPointer is an internal field not for general use. The operating system uses it internally, and if you overwrite it, you will corrupt stuff. I concede that it has a very poor name.
In case you're still for an answer, I've had the same problem too and posted my question, similar to yours:
Thread-local storage in kernel mode?
I need a TLS-equivalent in the kernel-mode driver. To be exact, I have a deep function call tree which originates at some point (driver's dispatch routine for instance), and I need to pass the context information.
In my specific case the catch is that I don't need a persistent storage, I just need a thread-specific placeholder for something for a single top-level function call. Hence I decided to use an arbitrary entry in the TLS array for the function call, and after it's done - restore its original value.
You get the TLS array by the following:
DWORD* get_Tls()
{
return (DWORD*) (__readfsdword(0x18) + 0xe10);
}
BTW I have no idea why the TIB is usually accessed by reading the contents of fs:[0x18]. It's just pointed by the fs selector. But this is how all the MS's code accesses it, hence I decided to do this as well.
Next, you choose an arbitrary TLS index, say 0.
const DWORD g_dwMyTlsIndex = 0;
void MyTopLevelFunc()
{
// prolog
DWORD dwOrgVal = get_Tls()[g_dwMyTlsIndex];
get_Tls()[g_dwMyTlsIndex] = dwMyContextValue;
DoSomething();
// epilog
get_Tls()[g_dwMyTlsIndex] = dwOrgVal;
}
void DoSomething()
{
DWORD dwMyContext = get_Tls()[g_dwMyTlsIndex];
}
I want to monitore swap space usage on windows 2003 server. If the usage is over 80% for 10 minutes, an alarm will be generated. There are lot of tools for RAM, but how about swap usage? How do I simulate that condition and do the test?
Use the built-in performance counters. You can fetch them via WMI/Win32_Perf:
http://msdn.microsoft.com/en-us/library/aa394270%28v=VS.85%29.aspx
or the raw Performance counter/registry interfaces:
http://msdn.microsoft.com/en-us/library/aa373083%28v=VS.85%29.aspx
To force the page file to be used. Start Commiting Memmory. Use the VirtualAlloc api call:
LPVOID WINAPI VirtualAlloc(
__in_opt LPVOID lpAddress,
__in SIZE_T dwSize,
__in DWORD flAllocationType,
__in DWORD flProtect
);
and set flAllocationType to MEM_COMMIT (0x1000), this should start memory being used. Once memory is suffcient exhausted, then the page file should be automatically employed. I suspect you'll have to start measuring usage and then determine heuristically as to when %usage you require happens.
To monitor it read the performance counters. The paging file set has a %usage counter you can read. Start here on how to consume them. All you need is to create a windows service that reads the info and then rings the appropriate alarms.
.Net : https://learn.microsoft.com/en-us/archive/blogs/bclteam/how-to-read-performance-counters-ryan-byington
C++ : http://msdn.microsoft.com/en-us/library/aa373219(v=VS.85).aspx or http://msdn.microsoft.com/en-us/library/aa373214(v=VS.85).aspx