I have a C++ project which has a extremely recursive function. This project when deployed and run on the dedicated machine it is eating upto 12 GB of memory. Previously, this process used to fail at 2GB limit. Hence, I converted this entire solution having 20 projects to 64 bit platform on visual studio 2010 platform. Now, it doesn't stop at 2 GB but eating up almost my whole RAM. Total RAM usage is 15.9 GB out of total 16 GB. I am using COM+ to run these application as dllhost. My questions are:
How to limit the memory being allocated fully causing failure of the process?
Does windows frees the memory after recursion is over? Because I am not seeing this process releasing memory until I kill this process.
Terminating case is not the problem, I guess, beacuse my job is succeeded by using Expiration Time option in Recycling in Component Services.Or is there any other problem?
Related
I am writing a real-time processing application and it allocates a lot of memory and release them properly.
memory consumption of my app is ok and it hasn't any leakage after a long period, but I encounter another problem.
my server has 64 Gb memory.
before my application starts to work, the task manager shows it has 5 GB memory 'In use', after 10 hours running my application, the task manager shows my app consumes 10 GB memory but the task manager shows the server has 60 GB memory in use.
when I terminate my application 'In use' memory goes down as 10 GB and shows 50 GB.
furthermore, when I see 'User' tab of task manager, it shows 12 GB memory usage.
Now I want to know the difference between task manager 'In use' memory usage(60 GB) and my process memory usage( user memory usage 12 GB) origins from what? and how can I confront it? is there any memory management point that I forget to apply?
because memory 'In use' growth until it reaches memory capacity, server performance will be reduced in a while.
what is this additional memory usage in the task manager 'in use' section?
I read below similar questions on StackOverflow but I didn't receive my answer clearly.
How does Windows Task Manager calculate "In Use" memory
Accessing CPU/RAM usage (like with Task Manager, but via API!)?
I watched this observation in Windows server 2016/2019 and Windows 10.
I can't take a screenshot from the server now, so I take a screenshot from my pc and show related part in the below images. as you could see I have similar problem in my pc and there is a difference in user memory usage(1105MB) and task manager 'In use' memory usage(2.6 GB).
So, yesterday I opened task manager in Win 8 (64 bit) and noticed that Chrome (32-bit for some reason) didn't use the whole power my PC has got. So I was running an AI JavaScript program and I noticed that my CPU was running at 1% and Memory was only runnning 120 MB, and that forced me to think why would I wait 5 minutes for it to run instead of somehow boosting it to at least 60%. As far as I know Windows automatically distributes the hardware usage to programs so I'm asking what's the problem:
Is it because it's x32?
Is it because I should manually configure windows to give it more power?
Note: I did search Google, but all I got is that people actually complain about High CPU usage and I've got the opposite.
32 bit doesn't make a difference here. Javascript is inherently single-threaded, so by default (not counting web workers) it won't use more than a single core on your machine. It just cannot. Also memory usage doesn't necessarily tell you how hard a program is working. Some need lots of memory, others only little.
It's up to programs to use the resources of the machine most efficiently; if they don't, there is nothing you can do with Windows to make them run better or faster.
I develop small projects, using ASP.NET MVC 3-5 and Visual Studio 2010/2013. I have a very slow process of debugging. I tried to accelerate the process of debugging using this statement. Unfortunately, the debugging process is still very slow.
Is there some way to put debugging symbols in RAM?
P.S. I have Core 2 duo e6600, 3 GB RAM, 256 GB HDD (90% full).
Essentially this is what's happening when you're attaching the debugger. It will load all the symbol files into memory so that you can debug the process. This takes time, the best way to improve that performance is to make sure your machine's drive is fast enough (SSD or 10k rpm drive) and that the drive is properly defragmented.
In an application I'm working on, under certain conditions the memory usage will shoot through the roof, effectively locking up my computer. I don't think it's a memory leak, and there are no errors, it just needs too much memory. The memory usage jumps to 99% in Task Manager and Windows stops working, forcing me to reboot.
Is it possible to set a maximum amount of memory VS can use while debugging? I'm not looking for a way to make it run out of memory faster, I just want to keep some memory free so Windows can keep working.
Visual Studio 2010
Windows 7 64b
8GB RAM
C# .NET
Edit:
I'm not asking how to fix a memory leak. I'm trying to limit the memory used by the VS debugger. For example, my PC has 8GB RAM, but my application has to run on a PC with 2GB RAM. So I want to configure VS to only use 2GB. If the application tries to allocate 2.0001GB I want VS to tell it there is no more memory (probably causing a crash).
This isn't exactly the answer you were looking for, but it might help others, so I'm posting:
I would try the following:
1) Download Oracle Virtualbox
2) Download Disk2VHD.exefrom Microsoft Sysinternals
3) Clone your system using Disk2VHD
4) Configure a VM with the memory restrictions you want.
In this way you can restrict the RAM and CPUs used by your task, and possibly recover easier from the case you describe.
We have some Win32 console applications running on Windows Server 2003 Service Pack 2 that regularly fail with this:
Error 1450 (ERROR_NO_SYSTEM_RESOURCES): "Insufficient system resources exist to complete the requested service."
All the documentation we've found suggests it is linked to the number of Free System Page Table Entries running out. We have 16GB RAM in these machines and use the /3GB Operating System switch to squeeze the Windows kernel into 1GB and allow our processes access to 3GB of address space. This drastically reduces the total number of Free System Page Table Entries, so combined with our heavy use of MapViewOfFile() it is perhaps not surprising that the kernel page table entries are running out.
However, when using Performance Monitor to view the Free System Page Table Entries counter, the value is around 36,000 on reboot and doesn't go down when our application starts. I find it hard to believe that our application, which opens many large memory-mapped files, doesn't have any effect on the kernel page table. If we can't believe the counter, it's much more difficult to test the effect of any system changes we make.
There is a promising Knowledge Base article, The Performance tool does not accurately show the available Free System Page Table entries in Windows Server 2003, but it says the problem has been fixed in Service Pack 1, and we are already on Service Pack 2.
Has anyone else struggled with or solved this issue?
Update: I have checked !sysptes in windbg (debugging the kernel) and the value matches the performance counter, around 36,000. I guess this is most likely to mean that there really are that many free page table entries and Windows is telling the truth. It does leave the question of why we're getting 1450 errors though, if the PTEs are not running out.
Further update: We never did get to the bottom of why the 1450 errors were occurring. However, instead we upgraded the OS on these servers to 64-bit Windows. This allows the existing 32-bit applications (without recompilation) to access a full 4GB of virtual address space, and lets the kernel memory area with those pesky Page Table Entries be as big as it likes too. I don't think we've had a 1450 error since.
Can you try the windbg command "!sysptes" to get System PTE Information? I'm not sure if you can do this with live kernel debug, you may have to get a memory dump.
I'm not sure why you assume that ERROR_NO_SYSTEM_RESOURCES is caused only by running out of free System Page Table Entries ? As far as I know, such generic error codes are used for more than one resource type. And in fact, the first Google hit suggests that running out of file cache memory may cause it too. (KB on an XP bug, which tripped this error mode).
In your case, I'd be checking the "Handle Count". Another possible problem is address space fragmentation. If you you want to create a 1GB file mapping view, you need 1GB of free address space, and it has to be contiguous. If you map a 1GB file, a 800 MB file, and a 1GB file, close the 800MB one and open a 900MB file, the 900MB file may not fit in the hole that's left.
MS has 2 ways to allow there 32 bit OS to "deal" with hardware that has 4 GB or more of RAM.
Option 1: is what you did with the /3GB Switch in the Boot.ini.
Option 1 Pros and Cons:
(CONS) This option sucks 1 GB from the normal 2 GB kernel area - hence making the OS struggle to meet the demands of both Paged Pool allocations and kernel stack allocations. So a person might think that using the /3GB Switch will help their, but really this option is screwing the 32 bit Window OS into a slow death.
(CONS) But, This gives my App 3GB.... WRONG (Hence this is a CON) The catch is that ONLY application that have been recompiled from the vendor to be "/3GB Switch aware" can really use the extra 1 GB. Hence the whole use of the /3GB Switch is a really BAD J.O.K.E on everyone.
Read this link for a much better write-up:
http://blogs.technet.com/askperf/archive/2007/03/23/memory-management-demystifying-3gb.aspx
Option 2: Use the /PAE switch in the Boot.ini.
Option 2 Pros and Cons:
(PROS) This really this only option if you have a more then 4GB of RAM. It tricks a application by placing the complete application memory footprint in RAM. Normally, only a application "Working Set" memory is in RAM and the remaining application memory requirements go into Windows Pagefile. What is a application total memory requirements?? - it called "Virtual Size".
In my world, I have a big fat Java based IBM Product that I deal with. The server that is running the "application" has 16 GB of RAM. I simply add the /PAE switch and watch (thanks to sysinternals Processes Explorer) application paging requests go from 200 KB per sec to up to 4MB per sec.
Question: "Why"?
Answer: The whole application is in RAM.
Question: "Does the application know that it is completely running in RAM?
Answer: No - It is running that same old way that it was always run, "THINKING" that it's has part of itself as the "Working Set" memory living in RAM and the remaining application memory requirements go into Windows Pagefile.
Yes, it is that flipping GOOD.
Please Note: Microsoft has done a poor job telling anyone about the great Windows OS option. Duh
Try it and report back to stackoverflow....