My Malloc is failing in my project.
Malloc runs several times via a one of the functions but fails due to lack of memory.
I am trying to increase the heap size in my VC++ but it gives me the error as above in the subject.
Can someone please tell me what is wrong in this ?
Windows server 2003 R2 Enterprise edition
And i am using VC++ 98 edition.
I tried some search but could not get anything conclusive on how to use /HEAP OPTION.
should the numbers be in MB ?
message_t* Allocate_momory(MsgType_t msgType, UInt16 dataLength)
{
// TO DO: Allocate memenory and return the pointer
message_t* mes_t;
mes_t = (message_t*) malloc(sizeof (message_t));
mes_t->msgType = msgType;
mes_t->dataLength = 0;
mes_t->clientID = 0;
mes_t->usageCount = 0;
mes_t->dataBuf = malloc(sizeof (dataLength));
return mes_t;
}
Yes it worked... But it unfortunatly did not solve my problem with malloc :( !!
This is a huge project with too many files.
I can't post the code but can someone guide me how should i try to debug a problem where malloc is failing ?
/HEAP sets the heap size in bytes. Also the square brackets in the documentation denote an optional parameter - you don't actually type these in. So it would be e.g.
/HEAP:1073741824
for a 1 GB heap, or
/HEAP:1073741824,16777216
if you really do want to specify the "commit" parameter in addition to the heap size (you probably don't).
Unfortunately I don't think this will solve your real problem, which is that you are running out of memory. You may have memory leaks, which you can track down with a tool such as valgrind. If that's not the case then you have a bad design, which will be a lot harder to fix than memory leaks.
Related
In Little Kernel boot loader at many places no check for null after malloc or memalign.
For example : in
void flash_init(void) at flash_cmdlist = memalign(32, 1024);
char *target_cmdline(char *org_cmdline)
{
cmdline = (char *)malloc(MAX_CMDLINE_LEN);
memset(cmdline, 0, MAX_CMDLINE_LEN);
}
Is it assumed that malloc will not return null while booting?
shunty, you should probably do some code background research :)
The code you're referring to isn't actually contributed by the author of LK but from Google/Qcom, and that part (aboot) was created using some dump of the legacy android bootloader, I still have files which look way too similar but cannot publish them to public domain.
The reason they don't check for null is because in the legacy bootloader heap was implemented as VERY basic, there was no free, no chunks, just allocate, increment address and forget about it because heap wouldn't be used for very long anyway so that wasn't a problem there but in LK which in itself is very much capable of OS like tasks, it is just plain wrong since heap supports free and other standard heap functions.
You should always check for null, however how does the bootloader recover if it can't malloc? There is nowhere to recover to... It could reboot, but this can lead to non stop rebooting. Maybe that's why they didn't check for null.
How to find size of a semaphore object in windows?
I tried using sizeof() but we cannot give name of the sempahore object as an argument to sizeof. It has to be the handle. sizeof(HANDLE) gives us the size of handle and not semaphore.
This what is known as an "opaque handle.". There is no way to know how big it really is, what it contains or how any of the functions work internally. This gives Microsoft the ability to completely rewrite the implementation with each new version of Windows if they want to without worrying about breaking existing code. It's a similar concept to having a public and private interface to a class. Since we are not working on the Windows kernel, we only get to see the public interface.
Update:
It might be possible to get a rough idea of how big they are by creating a bunch and monitoring what happens to your memory usage in Process Explorer. However, since there is a good chance that they live in the kernel and not in user space, it might not show up at all. In any case, there are no guarantees about any other version of Windows, past or future, including patches/service packs.
It's something "hidden" from you. You can't say how big it is. And it's a kernel object, so it probably doesn't even live in your address space. It's like asking "how big is the Process Table?", or "how many MB is Windows wasting?".
I'll add that I have made a small test on my Windows 7 32 bits machine: 100000 kernel semaphores (with name X{number} with 0 <= number < 100000)) : 4 mb of kernel memory and 8 mb of user space (both measured with Task Manager). It's about 40 bytes/semaphore in kernel space and 80 bytes/semaphore in user space! (this in Win32... In 64 bits it'll probably double)
This question already has answers here:
malloc() vs. HeapAlloc()
(8 answers)
Closed 2 years ago.
I'm having code that, for various reasons, I'm trying to port from the C runtime to one that uses the Windows Heap API. I've encountered a problem: If I redirect the malloc/calloc/realloc/free calls to HeapAlloc/HeapReAlloc/HeapFree (with GetProcessHeap for the handle), the memory seems to be allocated correctly (no bad pointer returned, and no exceptions thrown), but the library I'm porting says "failed to allocate memory" for some reason.
I've tried this both with the Microsoft CRT (which uses the Heap API underneath) and with another company's run-time library (which uses the Global Memory API underneath); the malloc for both of those works well with the library, but for some reason, using the Heap API directly doesn't work.
I've checked that the allocations aren't too big (>= 0x7FFF8 bytes), and they're not.
The only problem I can think of is memory alignment; is that the case? Or other than that, is there a fundamental difference between the Heap API and the CRT memory API that I'm not aware of?
If so, what is it? And if not, then why does the static Microsoft CRT (included with Visual Studio) take some extra steps in malloc/calloc before calling HeapAlloc? I'm suspecting there's a difference but I can't think of what it might be.
Thank you!
As I found out the hard way...
The difference isn't fundamental, but HeapReAlloc (which uses RtlReAllocateHeap) does not automatically treat a null pointer as a hint to call HeapAlloc; it fails instead.
Another important difference:
void *ptr = NULL;
HeapFree(GetProcessHeap(), 0, ptr);
has undefined behavior, while
void *ptr = NULL;
free(ptr);
is well defined (no operation performed).
UPDATE 2021:
The HeapFree documentation has been updated in January 2021, and now states:
[in] lpMem
A pointer to the memory block to be freed. This pointer is returned by
the HeapAlloc or HeapReAlloc function. This pointer can be NULL.
Honestly, it is not clear whether this applies since a specific SDK release or since ever. For sure that parameter was marked with _Frees_ptr_opt_ already on 10.0.16299.0, so probably was just a documentation issue.
Windows HeapFree, msvcrt free: do they cause the memory being freed to be paged-in? I am trying to estimate if not freeing memory at exit would speed up application shutdown significantly.
NOTE: This is a very specific technical question. It's not about whether applications should or should not call free at exit.
If you don't cleanly deallocate all your resources at application shutdown it will make it nigh on impossible to detect if you have any really serious problems - like memory leaks - which would be more of a problem than a slow shut down. If the UI disappears quickly, then the user will think the it has shut down quickly even if it has a lot of work still to do. With UI, perception of speed is more important than actual speed. When the user selects the 'Exit Application' option, the main application window should immediately disappear. It doesn't matter if the application takes a few seconds after that to free up everything an exit gracefully, the user won't notice.
I ran a test for HeapFree. The following program has access violation inside HeapFree at i = 31999:
#include <windows.h>
int main() {
HANDLE heap = GetProcessHeap();
void * bufs[64000];
// populate heap
for (unsigned i = 0; i < _countof(bufs); ++i) {
bufs[i] = HeapAlloc(heap, 0, 4000);
}
// protect a block in the "middle"
DWORD dwOldProtect;
VirtualProtect(
bufs[_countof(bufs) / 2], 4000, PAGE_NOACCESS,
&dwOldProtect);
// free blocks
for (unsigned i = 0; i < _countof(bufs); ++i) {
HeapFree(heap, 0, bufs[i]);
}
}
The stack is
ntdll.dll!_RtlpCoalesceFreeBlocks#16() + 0x12b9 bytes
ntdll.dll!_RtlFreeHeap#12() + 0x91f bytes
shutfree.exe!main() Line 19 C++
So it looks like the answer is "Yes" (this applies to free as well, since it uses HeapFree internally)
I'm almost certain the answer to the speed improvement question would be "yes". Freeing a block may or may not touch the actual block in question, but it will certainly have to update other bookkeeping information in any case. If you have zillions of small objects allocated (it happens), then the effort required to free them all could have a significant impact.
If you can arrange it, you might try setting up your application such that if it knows it's going to quit, save any pending work (configuration, documents, whatever) and exit ungracefully.
I'm writing the memory manager for an application, as part of a team of twenty-odd coders. We're running out of memory quota and we need to be able to see what's going on, since we only appear to be using about 700Mb. I need to be able to report where it's all going - fragmentation etc. Any ideas?
You can use existing memory debugging tools for this, I found Memory Validator 1 quite useful, it is able to track both API level (heap, new...) and OS level (Virtual Memory) allocations and show virtual memory maps.
The other option which I also found very usefull is to be able to dump a map of the whole virtual space based on VirtualQuery function. My code for this looks like this:
void PrintVMMap()
{
size_t start = 0;
// TODO: make portable - not compatible with /3GB, 64b OS or 64b app
size_t end = 1U<<31; // map 32b user space only - kernel space not accessible
SYSTEM_INFO si;
GetSystemInfo(&si);
size_t pageSize = si.dwPageSize;
size_t longestFreeApp = 0;
int index=0;
for (size_t addr = start; addr<end; )
{
MEMORY_BASIC_INFORMATION buffer;
SIZE_T retSize = VirtualQuery((void *)addr,&buffer,sizeof(buffer));
if (retSize==sizeof(buffer) && buffer.RegionSize>0)
{
// dump information about this region
printf(.... some buffer information here ....);
// track longest feee region - usefull fragmentation indicator
if (buffer.State&MEM_FREE)
{
if (buffer.RegionSize>longestFreeApp) longestFreeApp = buffer.RegionSize;
}
addr += buffer.RegionSize;
index+= buffer.RegionSize/pageSize;
}
else
{
// always proceed
addr += pageSize;
index++;
}
}
printf("Longest free VM region: %d",longestFreeApp);
}
You can also find out information about the heaps in a process with Heap32ListFirst/Heap32ListNext, and about loaded modules with Module32First/Module32Next, from the Tool Help API.
'Tool Help' originated on Windows 9x. The original process information API on Windows NT was PSAPI, which offers functions which partially (but not completely) overlap with Tool Help.
Our (huge) application (a Win32 game) started throwing "Not enough quota" exceptions recently, and I was charged with finding out where all the memory was going. It is not a trivial job - this question and this one were my first attempts at finding out. Heap behaviour is unexpected, and accurately tracking how much quota you've used and how much is available has so far proved impossible. In fact, it's not particularly useful information anyway - "quota" and "somewhere to put things" are subtly and annoyingly different concepts. The accepted answer is as good as it gets, although enumerating heaps and modules is also handy. I used DebugDiag from MS to view the true horror of the situation, and understand how hard it is to actually thoroughly track everything.