Is it possible to share a memory region between an application compiled with MinGW and one with Visual Studio ?
I am relying on boost interprocess:
shared_memory_object shm (create_only, "MySharedMemory", read_write);
shm.truncate(1000);
mapped_region region(shm, read_write);
int *pi = (int *)region.get_address();
I already realized that this is not possible via Cygwin, as the boost shared_memory_object is then created via the Cygwin posix layer.
A simple test program confirms that it is possible. One caveat is to use the exact same boost versions. Between boost 1.53 and 1.54 the unique id creation for the underlying memory mapped file has apparently changed.
I did however not check for memory alignment issues.
Related
I'm trying to build a program from its source code with VC 11. When the compiler is about to finish, it raises the error mentioned in title of this post.
As I've read here and in other forums, I tried to both close as many programs as possible and enlarge the size of the swap file in Windows... neither works.
I've read about a parameter called \Zm but I don't understand how to use it.
Can you please help me?
Take a look at this documentation which gives possible solutions:
I also had that problem and found the documentation useful. Main points:
If the compiler also issues errors C1076 and C3859, use the /Zm compiler option to lower the memory allocation limit. More heap space
is available to your application if you lower the remaining memory
allocation.
If the /Zm option is already set, try removing it. Heap space might be
exhausted because the memory allocation limit specified in the option
is too high. The compiler uses a default limit if you remove the /Zm
option.
If you are compiling on a 64-bit platform, use the 64-bit compiler toolset. For information, see How to: Enable a 64-Bit Visual C++
Toolset on the Command Line.
On 32-bit Windows, try using the /3GB boot.ini switch.
Increase the size of the Windows swap-file.
Close other running programs.
Eliminate unnecessary include files.
Eliminate unnecessary global variables, for example, by allocating memory dynamically instead of declaring a large array.
Eliminate unused declarations.
Split the current file into smaller files.
I can't tell much about the /Zm parameter, but I had the same issue (compiler is out of heap space).
What has helped me was the /m:4 (4 for the count of your CPUs) parameter so that you can use multiple CPUs for building.
Hope that helps you as well.
Also, if you are running on x64, be sure that the x64 version of "msbuild.exe" and "cl.exe" is beeing used. I had the issue that even when using e.g. the x64 ms powershell, the compiler would still choose the 32-bit version of msbuild.exe (in task manager "msbuild.exe*32", windows 7)
In addition to the other answers here (and in my case), fatal error C1060: compiler is out of heap space can be caused by a syntax error. The following code (in certain circumstances) can cause this error even with correct compiler options -- for instance if you've previously successfully compiled the same program.
r.push_back(e[1];
instead of
r.push_back(e[1]);
It seems to only cause this error rather than the standard error C2143: syntax error: missing ')' before ';' when r and e are of certain types, but it's worth checking any code you've edited recently if the program previously compiled without errors.
We had similar problem: a relativelly simple program (although, full of templates, using Eigen library) persistently failed to compile on one of the computers. All were using MSVC2013 x64, but only one was unable to compile the program due to C1060 error. We tried different compiler flags, setting/unsetting -Zm, but failed to resolve it without modifying code.
Some pointers were, however, given to us, when we switched from x64/x64 (64bit compiler for 64bit resulting executable) version of the compiler to the x86/x86 (32bit compiler for 32bit resulting executable). The x86 compiler gave us exact locations of the problematic parts of the program - calls to template functions receiving heavy templated objects. We have rewritten those to normal functions (build in different object file) and that solved the problem...
VS: Visual Studio 2015
OS: Windows10
If you are using VS2015 as your IDE, maybe there is another solution:
Go to update the VS2015 "Update3" package and everything will work smoothly.
In my case, a main program would not compile is VS 2022 Community Edition (free). It had many include files. By process of elimination, I managed to compile it once I removed any "volatile" modifiers in declarations that had this modifier.
A very strange bug, to say the least!
I got this error when compiling OnnxRuntime with MS Visual C++ 17 2022.
The solution for this issue was to close all other programs and compile using a single thread (in this case, removing the --parallel argument from the build.bat call).
I use a small in-house cluster (approx. 31 machines) to help complete parallel runs of a numerical groundwater model. After the groundwater model completes, a short post-processor manipulates some data for use by a code that is collecting output from the parallel runs. It used to be that the post-processor, written in fortran and compiled using intel's visual fortran from inside Vis. Studio 2010, was distribute-able, meaning I could place the .exe in amongst the other files that were distributed across the cluster and run on the various machines. Now, however, if I compile the very same fortran using visual studio 2012 with Intel Fortran XE 2013, I get an error on the cluster machines stating, "The program can't start because MSVCRR110.dll is missing from your computer. Try reinstalling the program to fix this probelm."
One of the reasons I like using Fortran is that it used to be self-contained. In other words, if I wrote small programs to do some short pithy task in another language, say R or Python, then I have to either install these programs on all the cluster machines or else "sandbox" the programs in with my distributed files. Forget about trying to use C# or VB etc., because then the cluster machines would need to have .NET framework installed. For the time being I can go back and use VS2010 to compile, but I don't anticipate having this option available to me much longer. Is there another alternative for keeping fortran programs "self-contained" in VS2012? As you can see in the code below, nothing complicated and no reason for it not be a self-contained executable:
program Calc_Seep
implicit none
! Variables
integer reason
real wp,time,ft,fts,fr,fo,fst,fro,loss1,loss2,loss3
character (len=120) line
character*50 txt
character*20 fmt
wp = 9.38 !a needed constant value
!read(*,*) txt
open(5,file='balance.out')
read(5,'(A)') line
do while (.NOT.line.EQ.'')
read(5,'(A)',IOSTAT=Reason) line
if (Reason < 0) exit
end do
read(line,*) time,ft,fts,fr,fo,fst,fro
!acre-ft/mi/yr
loss1 = (ft/time)*0.3048*5280*208*24/(0.3048**3)/43560.17
!ft/day
loss2 = (ft/time)*24/(0.3048**2)/wp
!cfs/mi
loss3 = (ft/time)*24/(0.3048**2)*5280/86400
close(5)
!now write the processed values to a file
open(5,file="Seepage.out")
write(5,'(A)') "acre-ft/mi/yr ft/day cfs/mi"
write(5,100) loss1, loss2, loss3
100 format(3f13.6)
close(5)
end program Calc_Seep
This is simply a change in default project properties for newly created projects. As of Intel Visual Fortran Composer XE 2013 SP1 (compiler version 14), the default is to link against the DLL libraries, matching what MS Visual C++ does. Existing projects are not affected and you can still change the libraries setting (Fortran > Libraries > Runtime Library) to "Multithreaded" from "Multithreaded DLL". If you do that, then it will again link to the static libraries, though there are some libraries provided only in DLL form (OpenMP and coarray support).
Alright, I found this guide and a few others on the internet which suggest running the following command from the VS 2010 IDE directory using the Visual Studio Command Prompt:
editbin /largeaddressaware devenv.exe
I've run this, and everything so far seems to work fine (I haven't run into any issues yet). But what I can't find information on is what negative implications, if any, there are by making Visual Studio 2010 use more than 2GB of RAM? Visual Studio was built to use a max of 2 GB of RAM. If VS was meant to use more than 2 GB of RAM, then I wouldn't have to hack the binary lol. While I love flying by the seat of my pants and trying new things without preparing for the worst (it's all I'm good at, haha), I'd at least like to know what issues I should be prepared to deal with should something go wrong.
TL;DR;: What negative implications are there, if any, by using the "editbin" command above to make Visual Studio 2010 aware of memory addresses greater than 2 GB?
The negative implications of enabling largeaddressaware is that the application could crash or corrupt memory in strange ways. The program was written assuming that no pointer value it had to deal with would be > 2GB. This can be done in subtle ways. The canonical example is probably calculating the midpoint address between to pointers.
ptrMid = (ptr1 + pt2) / 2;
That will work great if all of your pointers are < 2GB, but if they aren't you will get an incorrect result due to overflow.
ptrMid = (0x80000000 + 0x80000004) / 2 = 0x0000002, not 0x80000002
And not only do you have to worry about Visual Studio not being able to handle pointers > 2GB, any add-in would be affected by this as well.
See this question for some more things that have to be checked before enable largeaddressaware, see this question: What to do to make application Large Address Aware?
You really should never use editbin to change largeaddressaware on an application you don't control.
After reading this discussion and checking the existing headers, it looks like VS2010 already has this capability applied, at least for my installation anyway (64bit win7). If it was already compiled in I don't think you need to worry about bad side-effects.
This appears to be by design.
Recall that even when the /3GB switch is set, 32-bit programs receive
only 2GB of address space unless they indicate their willingness to
cope with addresses above 2GB by passing the /LARGEADDRESSAWARE flag.
This flag means the same thing on 64-bit Windows. But since 64-bit
Windows has a much larger address space available to it, it can afford
to give the 32-bit Windows program the entire 4GB of address space to
use. This is mentioned almost incidentally in Knowledge Base article Q889654 in the table "Comparison of memory and CPU limits in the
32-bit and 64-bit versions of Windows".
In other words, certain categories of 32-bit programs (namely, those
tight on address space) benefit from running on 64-bit Windows
machine, even though they aren't explicitly taking advantage of any
64-bit features.
http://blogs.msdn.com/b/oldnewthing/archive/2005/06/01/423817.aspx
Editbin is a Microsoft utility, so they're basically claiming that it works.
I posted this on SuperUser...but I was hoping the pros here at SO might have a good idea about how to fix this as well....
Normally we develop in VS 2005 Pro, but I wanted to give VS 2010 a spin. We have custom build tools based off of GNU make tools that are called when creating an executable.
This is the error that I see whenever I call my external tool:
...\gnu\make.exe): *** couldn't commit memory for cygwin heap, Win32 error 487
The caveat is that it still works perfectly fine in VS2005, as well as being called straight from the command line. Also, my external tool is setup exactly the same as in VS 2005.
Is there some setting somewhere that could cause this error to be thrown?
From problem with heap, win32 error 487 :
Each Cygwin app gets a special heap
area to hold stuff which is inherited
to child processes. Eg. all file
descriptor structures are stored in
that heap area (called the "cygheap").
The cygheap has room for at least 4000
file descriptor structures. But -
that's the clue - it's fixed size. The
cygheap can't grow. It's size is
reserved at the application's start
and it's blocks are commited on
demand.
For some reason your server
application needs all the cygheap
space when running under the described
conditions.
A possible solution might be found in Changing Cygwin's Maximum Memory:
Cygwin's heap is extensible. However,
it does start out at a fixed size and
attempts to extend it may run into
memory which has been previously
allocated by Windows. In some cases,
this problem can be solved by adding
an entry in the either the
HKEY_LOCAL_MACHINE (to change the
limit for all users) or
HKEY_CURRENT_USER (for just the
current user) section of the registry.
Add the DWORD value heap_chunk_in_mb
and set it to the desired memory limit
in decimal MB. It is preferred to do
this in Cygwin using the regtool
program included in the Cygwin
package. (For more information about
regtool or the other Cygwin utilities,
see the section called “Cygwin
Utilities” or use the --help option of
each util.) You should always be
careful when using regtool since
damaging your system registry can
result in an unusable system. This
example sets memory limit to 1024 MB:
regtool -i set /HKLM/Software/Cygwin/heap_chunk_in_mb 1024
regtool -v list /HKLM/Software/Cygwin
Exit all running Cygwin processes and
restart them. Memory can be allocated
up to the size of the system swap
space minus any the size of any
running processes. The system swap
should be at least as large as the
physically installed RAM and can be
modified under the System category of
the Control Panel.
It wouldn't hurt to ensure that the maximum size of your windows swap file is large enough.
To summerize : The environment doesn't allocate enough heap space for the cygwin executables. For some reason the problem is more acute with VS2010 Express. You need to either fix the environment, or use another Linux port than cygwin, or use Microsoft utilities.
From the cygwin email lists it looks like other people have run into similar situations, even when not running via Visual Studio, to which they've found that the solution is often to play with Cygwin's maximum memory settings:
http://www.cygwin.com/cygwin-ug-net/setup-maxmem.html
(note: it's worth reading this conversation, from above, about some values that did and didn't work).
Others have also reported issues with Anti-Virus software (recommendation is to unload from memory for some reason), and possibly also compatibility settings (try with it set to XP) which can affect cygwin in certain cases. See: http://www.avrfreaks.net/index.php?name=PNphpBB2&file=viewtopic&p=377066
As for Visual Studio: Are you on a 64bit machine and if so are you usually running the tool in a 64bit environment?
I've found that because Visual Studio 2010 runs in 32bit, tools launched from it are launched as 32bit processes (for a good illustration of this, add "cmd" as a tool). I'm not sure why this wouldn't be affected on 2005 (unless 2005 lets the system launch the process (64bit) and 2010 handles it itself (32bit)).
On Linux, FreeBSD and other systems I have valgrind for checking for memory errors like invalid reads and similar. I really love valgrind. Now I have to test code on Solaris/OpenSolaris and can't find a way to get information on invalid reads/writes in an as nice way (or better ;-)) as valgrind there.
When searching for this on the net I find references to libumem, but I get only reports about memory leaks there, not invalid access. What am I missing?
The dbx included with the Sun Studio compilers includes memory access checking support in its "Run Time Checking" feature (the check subcommand). See:
Solaris Studio 12.4 dbx manual: Chapter 9: Using Runtime Checking
Debugging Applications with Sun Studio dbx, dbxtool, and the Thread Analyzer
Leonard Li's Weblog: Runtime Memory Checking
The related "Sun Memory Error Discovery Tool" is also available from
http://cooltools.sunsource.net/discover/
Since version 3.11.0, Valgrind does run on Solaris.
See Release Notes and Supported Platforms.
More precisely, x86/Solaris and amd64/Solaris is now supported.
Support for sparc/Solaris is still in works.
watchmalloc is a quite useful library that can be dynamically loaded for your program (usually no need for recompiling) and then sets watchpoints at all the usually problematic memory locations, like freed areas or after an allocated memory block.
If your program accesses one of these invalid areas it gets a signal and you can inspect it in the debugger.
Depending on the configuration problematic areas can be watched for writes only, or also for reads.