Windows memory metric to detect memory leak - winapi

Question

We have large old legacy server code running as a 64bit windows service.
The service has a memory leak which, at the moment, we do not have the resources to fix.
As the service is resilient to restart, a temporary terrible 'solution' we want is to detect when the service's memory exceeded, e.g., 5GB, and exit the service (which has auto restart for such cases).
My question is which metric should I go for? Is using GlobalMemoryStatusEx to get
MEMORYSTATUSEX.ullTotalVirtual- MEMORYSTATUSEX.ullAvailVirtual right?

Answer 1

GlobalMemoryStatusEx is wrong. You do not want to fill up the machine memory until 5 GB are left in total.
You need GetProcessMemoryInfo.
BOOL WINAPI GetProcessMemoryInfo(
__in HANDLE Process,
__out PPROCESS_MEMORY_COUNTERS ppsmemCounters,
__in DWORD cb
);
From an example using GetProcessMemoryInfo:
#include <windows.h>
#include <stdio.h>
#include <psapi.h>
// To ensure correct resolution of symbols, add Psapi.lib to TARGETLIBS
// and compile with -DPSAPI_VERSION=1
void PrintMemoryInfo( DWORD processID )
{
HANDLE hProcess;
PROCESS_MEMORY_COUNTERS pmc;
// Print the process identifier.
printf( "\nProcess ID: %u\n", processID );
// Print information about the memory usage of the process.
hProcess = OpenProcess( PROCESS_QUERY_INFORMATION |
PROCESS_VM_READ,
FALSE, processID );
if (NULL == hProcess)
return;
if ( GetProcessMemoryInfo( hProcess, &pmc, sizeof(pmc)) )
{
printf( "\tWorkingSetSize: 0x%08X\n", pmc.WorkingSetSize );
printf( "\tPagefileUsage: 0x%08X\n", pmc.PagefileUsage );
}
CloseHandle( hProcess );
}
int main( void )
{
// Get the list of process identifiers.
DWORD aProcesses[1024], cbNeeded, cProcesses;
unsigned int i;
if ( !EnumProcesses( aProcesses, sizeof(aProcesses), &cbNeeded ) )
{
return 1;
}
// Calculate how many process identifiers were returned.
cProcesses = cbNeeded / sizeof(DWORD);
// Print the memory usage for each process
for ( i = 0; i < cProcesses; i++ )
{
PrintMemoryInfo( aProcesses[i] );
}
return 0;
}
Although unintuitive you need to read PagefileUsage which gets you the committed memory which was allocated by your process. WorkingSetSize is unreliable because if the machine gets tight on memory the OS will write all data to the page file. That can cause WorkingSetSize to be small (e.g. 100 MB) but in reality you leaked already 20 GB of memory. This would result in a saw tooth pattern in memory consumption until the page file is full. Working set is only the actively used memory which might hide the multi GB memory leak if the machine is under memory pressure.