Visual C++ debug runtime library features so-called allocation hooks. Works this way: you define a callback and call _CrtSetAllocHook() to set that callback. Now every time a memory allocation/deallocation/reallocation is done CRT calls that callback and passes a handful of parameters.
I successfully used an allocation hook to find a reproduceable memory leak - basically CRT reported that there was an unfreed block with allocation number N (N was the same on every program run) at program termination and so I wrote the following in my hook:
int MyAllocHook( int allocType, void* userData, size_t size, int blockType,
long requestNumber, const unsigned char* filename, int lineNumber)
{
if( requestNumber == TheNumberReported ) {
Sleep( 0 );// a line to put breakpoint on
}
return TRUE;
}
since the leak was reported with the very same allocation number every time I could just put a breakpoint inside the if-statement and wait until it was hit and then inspect the call stack.
What other useful things can I do using allocation hooks?
You could also use it to find unreproducible memory leaks:
Make a data structure where you map the allocated pointer to additional information
In the allocation hook you could query the current call stack (StackWalk function) and store the call stack in the data structure
In the de-allocation hook, remove the call stack information for that allocation
At the end of your application, loop over the data structure and report all call stacks. These are the places where memory was allocated but not freed.
The value "requestNumber" is not passed on to the function when deallocating (MS VS 2008). Without this number you cannot keep track of your allocation. However, you can peek into the heap header and extract that value from there:
Note: This is compiler dependent and may change without notice/ warning by the compiler.
// This struct is a copy of the heap header used by MS VS 2008.
// This information is prepending each allocated memory object in debug mode.
struct MsVS_CrtMemBlockHeader {
MsVS_CrtMemBlockHeader * _next;
MsVS_CrtMemBlockHeader * _prev;
char * _szFilename;
int _nLine;
int _nDataSize;
int _nBlockUse;
long _lRequest;
char _gap[4];
};
int MyAllocHook(..) { // same as in question
if(nAllocType == _HOOK_FREE) {
// requestNumber isn't passed on to the Hook on free.
// However in the heap header this value is stored.
size_t headerSize = sizeof(MsVS_CrtMemBlockHeader);
MsVS_CrtMemBlockHeader* pHead;
size_t ptr = (size_t) pvData - headerSize;
pHead = (MsVS_CrtMemBlockHeader*) (ptr);
long requestNumber = pHead->_lRequest;
// Do what you like to keep track of this allocation.
}
}
You could keep record of every allocation request then remove it once the deallocation is invoked, for instance: This could help you tracking memory leak problems that are way much worse than this to track down.
Just the first idea that comes to my mind...
Related
Given a shared array of integer counters, I am interested to know if a thread can atomically fetch and add an array element without locking the entire array?
Here's an illustration of working model that uses mutex to lock access to the entire array.
// thread-shared class members
std::mutex count_array_mutex_;
std::vector<int> counter_array_( 100ish );
// Thread critical section
int counter_index = ... // unpredictable index
int current_count;
{
std::lock_guard<std::mutex> lock(count_array_mutex_);
current_count = counter_array_[counter_index] ++;
}
// ... do stuff using current_count.
I'd like multiple threads to be able to fetch-add separate array elements simultaneously.
So far, in my research of std::atomic<int> I'm thrown off that constructing the atomic object also constructs the protected member. (And plenty of answers explaining why you can't make a std::vector<std::atomic<int> > )
C++20 / C++2a (or whatever you want to call it) will add std::atomic_ref<T> which lets you do atomic operations on an object that wasn't atomic<T> to start with.
It's not available yet as part of the standard library for most compilers, but there is a working implementation for gcc/clang/ICC / other compilers with GNU extensions.
Previously, atomic access to "plain" data was only available with some platform-specific functions like Microsoft's LONG InterlockedExchange(LONG volatile *Target, LONG Value); or GNU C / C++
type __atomic_add_fetch (type *ptr, type val, int memorder) (the same builtins that C++ libraries for GNU compilers use to implement std::atomic<T>.)
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0019r8.html includes some intro stuff about the motivation. CPUs can easily do this, compilers can already do this, and it's been annoying that C++ didn't expose this capability portably.
So instead of having to wrestle with C++ to get all the non-atomic allocation and init done in a constructor, you can just have every access create an atomic_ref to the element you want to access. (It's free to instantiate as a local, at least when it's lock-free, on any "normal" C++ implementations).
This will even let you do things like resize the std::vector<int> after you've ensured no other threads are accessing the vector elements or the vector control block itself. And then you can signal the other threads to resume.
It's not yet implemented in libstdc++ or libc++ for gcc/clang.
#include <vector>
#include <atomic>
#define Foo std // this atomic_ref.hpp puts it in namespace Foo, not std.
// current raw url for https://github.com/ORNL/cpp-proposals-pub/blob/master/P0019/atomic_ref.hpp
#include "https://raw.githubusercontent.com/ORNL/cpp-proposals-pub/580934e3b8cf886e09accedbb25e8be2d83304ae/P0019/atomic_ref.hpp"
void inc_element(std::vector<int> &v, size_t idx)
{
v[idx]++;
}
void atomic_inc_element(std::vector<int> &v, size_t idx)
{
std::atomic_ref<int> elem(v[idx]);
static_assert(decltype(elem)::is_always_lock_free,
"performance is going to suck without lock-free atomic_ref<T>");
elem.fetch_add(1, std::memory_order_relaxed); // take your pick of memory order here
}
For x86-64, these compile exactly the way we'd hope with GCC,
using the sample implementation (for compilers implementing GNU extensions) linked in the C++ working-group proposal. https://github.com/ORNL/cpp-proposals-pub/blob/master/P0019/atomic_ref.hpp
From the Godbolt compiler explorer with g++8.2 -Wall -O3 -std=gnu++2a:
inc_element(std::vector<int, std::allocator<int> >&, unsigned long):
mov rax, QWORD PTR [rdi] # load the pointer member of std::vector
add DWORD PTR [rax+rsi*4], 1 # and index it as a memory destination
ret
atomic_inc_element(std::vector<int, std::allocator<int> >&, unsigned long):
mov rax, QWORD PTR [rdi]
lock add DWORD PTR [rax+rsi*4], 1 # same but atomic RMW
ret
The atomic version is identical except it uses a lock prefix to make the read-modify-write atomic, by making sure no other core can read or write the cache line while this core is in the middle of atomically modifying it. Just in case you were curious how atomics work in asm.
Most non-x86 ISAs like AArch64 of course require a LL/SC retry loop to implement an atomic RMW, even with relaxed memory order.
The point here is that constructing / destructing the atomic_ref doesn't cost anything. Its member pointer fully optimizes away. So this is exactly as cheap as a vector<atomic<int>>, but without the headache.
As long as you're careful not to create data-race UB by resizing the vector, or accessing an element without going through atomic_ref. (It would potentially manifest as a use-after-free on many real implementations if std::vector reallocated the memory in parallel with another thread indexing into it, and of course you'd be atomically modifying a stale copy.)
This definitely gives you rope to hang yourself if you don't carefully respect the fact that the std::vector object itself is not atomic, and also that the compiler won't stop you from doing non-atomic access to the underlying v[idx] after other threads have started using it.
One way:
// Create.
std::vector<std::atomic<int>> v(100);
// Initialize.
for(auto& e : v)
e.store(0, std::memory_order_relaxed);
// Atomically increment.
auto unpredictable_index = std::rand() % v.size();
int old = v[unpredictable_index].fetch_add(1, std::memory_order_relaxed);
Note that std::atomic<> copy-constructor is deleted, so that the vector cannot be resized and needs to be initialized with the final count of elements.
Since resize functionality of std::vector is lost, instead of std::vector you may as well use std::unique_ptr<std::atomic<int>[]>, e.g.:
// Create.
unsigned const N = 100;
std::unique_ptr<std::atomic<int>[]> p(new std::atomic<int>[N]);
// Initialize.
for(unsigned i = 0; i < N; ++i)
p[i].store(0, std::memory_order_relaxed);
// Atomically increment.
auto unpredictable_index = std::rand() % N;
int old = p[unpredictable_index].fetch_add(1, std::memory_order_relaxed);
I am new to C++ coding and want to know if i am using a const char* ptr and and doing a step, will there be any memory leak. If yes, how to tackle that? I cannot show the exact code but the outline is
char letter[];
const char* lettrptr= *somefunc(argument)*;
strcpy(letter,lettrptr);
The whole code is woking fine. But my doubt is will there be any memory leak for const char* ptr and if yes how to solve that?
A memory leak doesn't come from using const char *, or any other type.
A memory leak comes from not freeing the memory after it is no longer needed.
In your case, you have the function somefunc. It is either written by you of by someone else. Anyway, you have to find out whether the function expects the caller to free the memory or not.
Because of the const, it is very likely that the function doesn't expect you to free the memory, but to be safe you should check anyway.
It depends.
Does somefunc() allocate memory for the string? Or does it return a pointer to something statically allocated?
In the first case, somefunc() will return the result of a call to malloc() (or calloc() or some similar function). If this is the case, that memory can be leaked, so you will need to free() it.
In the second case, somefunc() will return a char* that is either a global, or a local static like one of the following:
char global_str[] = "Hello";
const char *somefuncA()
{
return global_str;
}
const char *somefuncB()
{
static char static_str[] = "World";
return static_str;
}
If one of these is the case, the memory will not leak (and you must NOT try to free() it).
This issue is my misunderstanding of how the standard is using my custom allocator. I have a stateful allocator that keeps a vector of allocated blocks. This vector is pushed into when allocating and searched through during de-allocation.
From my debugging it appears that different instances of my object (this*'s differ) are being called on de-allocation. An example may be that MyAllocator (this* = 1) is called to allocate 20 bytes, then some time later MyAllocator (this* = 2) is called to de-allocate the 20 bytes allocated earlier. Abviously the vector in MyAllocator (this* = 2) doesn't contain the 20 byte block allocated by the other allocator so it fails to de-allocate. My understanding was that C++11 allows stateful allocators, what's going on and how do i fix this?
I already have my operator == set to only return true when this == &rhs
pseudo-code:
template<typename T>
class MyAllocator
{
ptr allocate(int n)
{
...make a block of size sizeof(T) * n
blocks.push_back(block);
return (ptr)block.start;
}
deallocate(ptr start, int n)
{
/*This fails because the the block array is not the
same and so doesn't find the block it wants*/
std::erase(std::remove_if(blocks.begin,blocks.end, []()
{
return block.start >= (uint64_t)ptr && block.end <= ((uint64_t)ptr + sizeof(T)*n);
}), blocks.end);
}
bool operator==(const MyAllocator& rhs)
{
//my attempt to make sure internal states are same
return this == &rhs;
}
private:
std::vector<MemoryBlocks> blocks;
}
Im using this allocator for an std::vector, on gcc. So as far as i know no weird rebind stuff is going on
As #Igor mentioned, allocators must be copyable. Importantly though they must share their state between copies, even AFTER they have been copied from. In this case the fix was easy, i made the blocks vector a shared_ptr as suggested and then now on copy all the updates to that vector occur to the same vector, since they all point to the same thing.
I am using BDS 2006 Turbo C++ for a long time now and some of my bigger projects (CAD/CAM,3D gfx engines and Astronomic computations) occasionally throw an exception (for example once in 3-12 months of 24/7 heavy duty usage). After extensive debugging I found this:
//code1:
struct _s { int i; } // any struct
_s *s=new _s[1024]; // dynamic allocation
delete[] s; // free up memory
this code is usually inside template where _s can be also class therefore delete[] this code should work properly, but the delete[] does not work properly for structs (classes looks OK). No exceptions is thrown, the memory is freed, but it somehow damages the memory manager allocation tables and after this any new allocation can be wrong (new can create overlapped allocations with already allocated space or even unallocated space hence the occasional exceptions)
I have found that if I add empty destructor to _s than suddenly seems everything OK
struct _s { int i; ~_s(){}; }
Well now comes the weird part. After I update this to my projects I have found that AnsiString class has also bad reallocations. For example:
//code2:
int i;
_s *dat=new _s[1024];
AnsiString txt="";
// setting of dat
for (i=0;i<1024;i++) txt+="bla bla bla\r\n";
// usage of dat
delete[] dat;
In this code dat contains some useful data, then later is some txt string created by adding lines so the txt must be reallocated few times and sometimes the dat data is overwritten by txt (even if they are not overlapped, I thing the temp AnsiString needed to reallocate txt is overlapped with dat)
So my questions are:
Am I doing something wrong in code1, code2 ?
Is there any way to avoid AnsiString (re)allocation errors ? (but still using it)
After extensive debugging (after posting question 2) I have found that AnsiString do not cause problems. They only occur while using them. The real problem is probably in switching between OpenGL clients. I have Open/Save dialogs with preview for vector graphics. If I disable OpenGL usage for these VCL sub-windows than AnsiString memory management errors disappears completely. I am not shore what is the problem (incompatibility between MFC/VCL windows or more likely I made some mistake in switching contexts, will further investigate). Concern OpenGL windows are:
main VCL Form + OpenGL inside Canvas client area
child of main MFC Open/Save dialog + docked preview VCL Form + OpenGL inside Canvas client area
P.S.
these errors depend on number of new/delete/delete[] usages not on the allocated sizes
both code1 and code2 errors are repetitive (for example have a parser to load complex ini file and the error occurs on the same line if the ini is not changed)
I detect these errors only on big projects (plain source code > 1MB) with combined usage of AnsiString and templates with internal dynamic allocations, but is possible that they are also in simpler projects but occurs so rarely that I miss it.
Infected projects specs:
win32 noinstall standalone (using Win7sp1 x64 but on XPsp3 x32 behaves the same)
does not meter if use GDI or OpenGl/GLSL
does not meter if use device driver DLLs or not
no OCX,or nonstandard VCL component
no DirectX
1 Byte aligned compilation/link
do not use RTL,packages or frameworks (standalone)
Sorry for bad English/grammar ...
any help / conclusion / suggestion appreciated.
After extensive debugging i finely isolated the problem.
Memory management of bds2006 Turbo C++ became corrupt after you try to call any delete for already deleted pointer. for example:
BYTE *dat=new BYTE[10],*tmp=dat;
delete[] dat;
delete[] tmp;
After this is memory management not reliable. ('new' can allocate already allocated space)
Of course deletion of the same pointer twice is bug on programmers side, but i have found the real cause of all my problems which generates this problem (without any obvious bug in source code) see this code:
//---------------------------------------------------------------------------
class test
{
public:
int siz;
BYTE *dat;
test()
{
siz=10;
dat=new BYTE[siz];
}
~test()
{
delete[] dat; // <- add breakpoint here
siz=0;
dat=NULL;
}
test& operator = (const test& x)
{
int i;
for (i=0;i<siz;i++) if (i<x.siz) dat[i]=x.dat[i];
for ( ;i<siz;i++) dat[i]=0;
return *this;
}
};
//---------------------------------------------------------------------------
test get()
{
test a;
return a; // here call a.~test();
} // here second call a.~test();
//---------------------------------------------------------------------------
void main()
{
get();
}
//---------------------------------------------------------------------------
In function get() is called destructor for class a twice. Once for real a and once for its copy because I forget to create constructor
test::test(test &x);
[Edit1] further upgrades of code
OK I have refined the initialization code for both class and struct even templates to fix even more bug-cases. Add this code to any struct/class/template and if needed than add functionality
T() {}
T(const T& a) { *this=a; }
~T() {}
T* operator = (const T *a) { *this=*a; return this; }
//T* operator = (const T &a) { ...copy... return this; }
T is the struct/class name
the last operator is needed only if T uses dynamic allocations inside it if no allocations are used you can leave it as is
This also resolves other compiler issues like this:
Too many initializers error for a simple array in bcc32
If anyone have similar problems hope this helps.
Also look at traceback a pointer in c++ code mmap if you need to debug your memory allocations...
I have a linked list structure like this
typedef struct list_node {
int data;
struct list_node *next;
}l_node;
void print_list(l_node *head) {
l_node *cur_node = head;
while(cur_node!=NULL) {
printf("%d\t", cur_node->data);
cur_node = cur_node->next;
}
}
void main() {
printf("List");
l_node *new_node = (l_node*)malloc(sizeof(l_node));
print_list(new_node);
}
When I compile gcc linkedlist.c and do ./a.out
I get output
List
0
But when I tried it in VC++, I got error (since I am trying to access invalid memory location in cur_node->next).
So, does malloc of gcc allocate 0 value by default to the integer variable inside the structure? Why I didn't get the same error while doing the same print_list in gcc?
The contents of the memory returned by malloc are not initialized. You cannot read from that memory before you initialize it (by writing to it at least once).
gcc may be "helpfully" zero-initializing the memory, but that behavior isn't required. The Visual C++ C Runtime (CRT) will give you uninitialized memory in a release build (for maximum performance) and memory initialized with the special fill byte 0xcd in a debug build (to help you find where you may be using uninitialized memory).
So, basically, you need to initialize the memory before you use it. If you want the runtime to zero-initialize the heap block before it gives it to you, you may use calloc.
you need to assign new_node->next = NULL because you check if the current node is NULL or not, and malloc does not initialize the allocated space with any value, so there is no gurantee that the value would be initialized. To be safe you need to assign NULL manually to the tail of the linked list, or an invalid pointer.