I'm getting this error every time I try to run the application though it compile well:
pool allocator: Specified pool size too big for this device
Current file: /home/marco/Desktop/tools.c
function: PTC3D
line: 330
This file was compiled: -ta=tesla:cc35,cc50,cc60,cc70,cc70,cc75,cc80
The strange thing is that I get this error only since I restarted the PC, while before I've never get it.
I compile with:
CC = nvc
CFLAGS = -c -acc -ta=tesla:managed:cuda11.0 -Minfo=accel -w -O3 -DTEST_CASE=3
LDFLAGS = -lm -acc -ta=tesla:managed:cuda11.0
In the code nothing has been changed so maybe it is related to the compiler. I installed a new program today and I could have touched something I shouldn't have.
The message should just be a warning. The pool allocator will be by-passed and instead the CUDA Unified Memory API routines will be called directly for each allocation. You might see some performance degradation if you have a lot of small allocations since the API calls have a relatively high overhead, but shouldn't hurt functionality.
The default CUDA Unified Memory pool size is 1GB, though this is modifiable by setting the environment variable NVCOMPILER_ACC_POOL_SIZE. You might try setting the size to something smaller to see if it fixes the messages. Full details can be found at: https://docs.nvidia.com/hpc-sdk/compilers/hpc-compilers-user-guide/index.html#acc-mem-unified
Exactly why the message starting appearing is unclear, but it's most likely hardware related, or possibly a CUDA driver issue. What device and CUDA driver are you using? Has anything changed with hardware?
I solved going in Software & Updates, in Additional Drivers, setting the recommended driver: NVIDIA driver meta package.
Related
I'm trying to do some debugging on a server on an issue that I suspect is related to a buffer overflow, so I tried to compile my code with -fsanitize=address to enable address sanitizing.
It compiled, and the resulting software runs. However, I'm trying to get a core dump when the address sanitizer detects an error since that is pretty much the only way I can get information out of the system due to the setup.
I am calling the software with ASAN_OPTIONS=abort_on_error=1 prepended on the command line (using a shell script to do that), and have checked that ulimit -c gives unlimited as result, but it just won't produce a core dump.
What am I missing?
This is on an ubuntu 14.04 server with gcc version 4.8.4
EDIT: sysctl kernel.core_pattern gives back kernel.core_pattern = |/usr/share/apport/apport %p %s %c %P. This probably means that apport is enabled (at least in some form). However, I have been able to get proper core files on this system from asserts and SIGFPEs in the software (that is where the suspicion of array overruns comes from).
Let me guess, is this x64 target? Coredumps are disabled there to avoid dumping 16 TB shadow memory (see docs for disable_coredump here for details).
Newer versions of GCC/Clang remove shadow from core by default so that one could do something like
export ASAN_OPTIONS=abort_on_error=1:disable_coredump=0
but I'm afraid 4.8 is too old for this.
As an alternative suggestion, why backtraces are not enough for you? You could use log_path or log_to_syslog to preserve them if you do not have access to programs stderr.
NB: I posted suggestion to enable coredumps on all platforms.
I saw a few post here and there talking about this issue, and the only two options seemed to be to either rebuild kernel with appropriate memory split or buy a package from Eltechs.
Since were are talking about an open source software, I believe there should be people who got Wine (installed from jessie-backports) working on RPi3 without buying some extra patch.
However, every time I build a kernel with required memory split option and try executing winecfg, it gives me this kind off error asking to build kernel with a different memory split option. So I'm going in circles here.
Warning: memory above 0x80000000 doesn't seem to be accessible.
Wine requires a 3G/1G user/kernel memory split to work properly.
wine: failed to map the shared user data: c0000018
So first it asked to rebuild the kernel with 2G/2G memory split, then, after I've done that, it asked me to build the kernel with 1G/3G, and then with 3G/1G again.
Can we, the open source people, sort this issue out once and forever and run x86 apps on RPi3? :)
I am making a project that should compile on Windows and Linux. I have made the project in Visual Studio and then made a makefile for linux. I created all the files in Windows with VS.
It compiles and runs perfectly in VS but when I run the makefile and it runs g++ I get
$ g++ -c -I include -o obj/Linux_x86/Server.obj src/Server.cpp
cc1plus: error: include: Value too large for defined data type
cc1plus: error: src/Server.cpp: Value too large for defined data type
The code is nothing more than a Hello World atm. I just wanted to make sure that everything was working before I started development. I have tried searching but to no avail.
Any help would be appreciated.
I have found a solution on Ubuntu at least. I, like you have noticed that the error only occurs on mounted samba shares - it seems to come from g++ 'stat'ing the file, the inode returns a very large value.
When mounting the share add ,nounix,noserverino to the options, ie:
mount -t cifs -o user=me,pass=secret,nounix,noserverino //server/share /mount
I found the info at http://bbs.archlinux.org/viewtopic.php?id=85999
I had similar problem. I compiled a project in a CIFS mounted samba share. With one Linux kernel the compilation was done, but using an other Linux kernel (2.6.32.5), I got similar error message: "Value too large for defined data type". When I used the proposed "nounix,noserverino" CIFS mounting option, the problem was fixed. So in that case there is a problem with CIFS mounting, so the error message is misleading, as there are no big files.
GNU Core Utils:
27 Value too large for defined data type
It means that your version of the utilities were not compiled with
large file support enabled. The GNU utilities do support large files
if they are compiled to do so. You may want to compile them again and
make sure that large file support is enabled. This support is
automatically configured by autoconf on most systems. But it is
possible that on your particular system it could not determine how to
do that and therefore autoconf concluded that your system did not
support large files.
The message "Value too large for defined data type" is a system error
message reported when an operation on a large file is attempted using
a non-large file data type. Large files are defined as anything larger
than a signed 32-bit integer, or stated differently, larger than 2GB.
Many system calls that deal with files return values in a "long int"
data type. On 32-bit hardware a long int is 32-bits and therefore this
imposes a 2GB limit on the size of files. When this was invented that
was HUGE and it was hard to conceive of needing anything that large.
Time has passed and files can be much larger today. On native 64-bit
systems the file size limit is usually 2GB * 2GB. Which we will again
think is huge.
On a 32-bit system with a 32-bit "long int" you find that you can't
make it any bigger and also maintain compatibility with previous
programs. Changing that would break many things! But many systems make
it possible to switch into a new program mode which rewrites all of
the file operations into a 64-bit program model. Instead of "long"
they use a new data type called "off_t" which is constructed to be
64-bits in size. Program source code must be written to use the off_t
data type instead of the long data type. This is typically done by
defining -D_FILE_OFFSET_BITS=64 or some such. It is system dependent.
Once done and once switched into this new mode most programs will
support large files just fine.
See the next question if you have inadvertently created a large file
and now need some way to deal with it.
12 years after this question was posted, I get the same error, when building my c++ project in docker on ubuntu 20.04 in Windows 11 WSL2 using an old 32-bit compiler (gcc-linaro-arm-linux-gnueabi-2012.01-20120125_linux/bin/arm-linux-gnueabi-g++).
The issue is related to mounting of the windows filesystem into WSL, where we get 64 bit inodes, not supported by the old 32 bit toolchain, see also The 64 bit inode problem
As a quick workaround you can build if you move your project into the home folder of the ubuntu.
An alternative solution is to replace the stat() with LD_PRELOAD as described in Build on Windows 10 with WSL, section "Fix stat in 32-bit binaries". This shall be done in the docker image. To compile inode64.c I had to additionally add 32 bit header files with
sudo apt-get install gcc-multilib
If you are on mergerfs filesystem, removing use_ino option will solve the issue: https://github.com/trapexit/mergerfs/issues/485
I think your g++ parameters are a bit off, or conflicting.
-c compile only
-I directory of your includes (just plain include might be ambiguous. Try full path)
-o outfile (but -c says compile only)
I wonder if there is a good way to find the source code that causes a heap corruption error, given the memory address of the data that was written 'outside' the allocated heap block in Visual Studio;
Dedicated (0008) free list element 26F7F670 is wrong size (dead)
(Trying to write down some notes on how to find memory errors)
Begin with installing windbg:
http://www.microsoft.com/whdc/Devtools/Debugging/default.mspx
Then turn on the pageheap like this:
gflags.exe /p /enable yourexecutable.exe /full
This will insert a non writable page after each heap allocation.
After this launch the executable from inside windbg, any writes outside the heap will now be caught by this debugger. To turn of the pageheap afterwards use this:
gflags.exe /p /disable yourexecutable.exe
More info on how to use the pageheap here.
For Window 10 you could enable the PageHeap option in the GFlags Tool, this tool is included as part of the Debugging Tools for Windows.
The Page Heap options in GFlags lets you select standard heap verification or full-page heap verification. Beware, the full heap verification uses a full page of memory for each allocation so it can cause system memory shortages.
To enable the Page Heap in GFlags:
•To enable standard page heap verification, the standard version will write a pattern at the end of each heap allocation and then examine the pattern when the allocations are freed.
To verify all processes use:
gflags /r +hpa
gflags /k +hpa
for a single process use:
gflags /p /enable ImageFileName
•To enable full page heap verification for one process, this option places an inaccessible page at the end of each allocation so that the program stops immediately if it tries to accesses memory beyond the allocation, this should only be used on a single process due to the heavy memory consumption.
gflags /i ImageFileName +hpa
gflags /p /enable ImageFileName /full
The two commands above are interchangeable.
Note: All page heap settings mentioned above are system wide settings stored in the registry (except /k) and remain effective until you change them. The /k setting is a Kernel flag setting are set for this session and will be lost when Windows shuts down
Another helpful tool is the Application Verifier, but this is not part of the Debugging Tools for Windows, rather it is included in the Windows Software Development Kit (SDK).
Maybe you can try Microsoft's Application Verifier.
It solved a similar problem for me once,by turning on extra checks on heap operations.
In my opinion, the randomness of corrupted address is because the heap can be 'subtly' damaged, and the problem won't show up until something big happens to the heap (like massive allocation/free).
You could set a breakpoint on a write to the memory address. The debugger will then show you the code that writes to the location, but you still need to work out which of the writes are causing the problem.
It's probably too late but if it compiles with gcc and can run on linux you may use valgrind to find the source of the problem (i don't remember the flags, i only used it once with great success).
more info about Gflags and PageHeap(which helped a lot):
http://msdn.microsoft.com/en-us/library/windows/hardware/ff549561%28v=vs.85%29.aspx
I am assuming C++ as the language.
If the error is reproducible and the corrupted address is always the same, you can put a data breakpoint to stop the program when writing at this address.
Make sure all libraries you are linking to are compiled in the same CLR version as the application you are running - all in Release or all in Debug.
When you compile in either Debug and Release you are actually targeting two different versions of the C runtime library. These versions are quite different and they use different strategies for allocating memory and they use different heaps. But the most important thing to know is that they are NOT compatible with one another.
The Release C runtime library allocated memory as expected, whereas the Debug will add extra information, such as guard blocks to track buffer overflow and the location that called the allocation function, and in turn it allocates more memory than the Release.
If your are linking your application to a mix of DLLs which were built in Release and Debug, you will most likely end up trying to delete an object in one CLR which was created in another. This means you will be trying to free more or less memory than what was allocated to the object and this can corrupt the heap.
You should build your application, as well as attach to libraries which are built under the same configuration, either Release or Debug.
This problem can occur especially in modules that are being compiled with different compilers.
There is a way to work around, which I will mention but do not recommend. If for some reason you still need to build in different modes, this work around will allows for all memory to be allocated and freed from the same shared heap. The API GetProcessHeap will allow you to access the shared heap throughout the different modules. By using the HeapAlloc & HeapFree you can allocate and free memory in the shared heap. Note: the HeapAlloc and HeapFree should replace all calls to malloc and free in your application.
On Linux, FreeBSD and other systems I have valgrind for checking for memory errors like invalid reads and similar. I really love valgrind. Now I have to test code on Solaris/OpenSolaris and can't find a way to get information on invalid reads/writes in an as nice way (or better ;-)) as valgrind there.
When searching for this on the net I find references to libumem, but I get only reports about memory leaks there, not invalid access. What am I missing?
The dbx included with the Sun Studio compilers includes memory access checking support in its "Run Time Checking" feature (the check subcommand). See:
Solaris Studio 12.4 dbx manual: Chapter 9: Using Runtime Checking
Debugging Applications with Sun Studio dbx, dbxtool, and the Thread Analyzer
Leonard Li's Weblog: Runtime Memory Checking
The related "Sun Memory Error Discovery Tool" is also available from
http://cooltools.sunsource.net/discover/
Since version 3.11.0, Valgrind does run on Solaris.
See Release Notes and Supported Platforms.
More precisely, x86/Solaris and amd64/Solaris is now supported.
Support for sparc/Solaris is still in works.
watchmalloc is a quite useful library that can be dynamically loaded for your program (usually no need for recompiling) and then sets watchpoints at all the usually problematic memory locations, like freed areas or after an allocated memory block.
If your program accesses one of these invalid areas it gets a signal and you can inspect it in the debugger.
Depending on the configuration problematic areas can be watched for writes only, or also for reads.