I'm slightly puzzled by the lack of documentation on the issue, so I may be completely off track here:
When I allocate memory in order to return an object through an unique pointer whose value I have modified, what allocater should I use?
The documentation says that I can provide MIDL_user_allocate() and MIDL_user_free() and the stub will use these -- however that does not make sense in CLSCTX_INPROC_SERVER, as the calling object would need to use (and hence resolve) my allocater.
So, how should I allocate memory here, so that the stub code can properly free the list if the DLL is loaded into SVCHOST, and applications can still use the DLL directly if they so desire.
idl:
HRESULT GetItems([out] DWORD *count, [out, size_is(,count)] ITEM **items);
cpp:
HRESULT STDMETHODCALLTYPE impl::GetBuffer(DWORD *count, ITEM **items)
{
*count = 0;
*items = reinterpret_cast<ITEM *>(/* ??? */);
if(!*items)
return E_OUTOFMEMORY;
*count = 5;
/* fill in items */
return S_OK;
}
From here:
Out-parameters must be allocated by the one called; they are freed by the caller using the standard COM task memory allocator.
where COM task memory allocator is either the set of IMalloc methods or the set of CoTaskMemAlloc()/CoTaskMemRealloc()/CoTaskMemFree() functions that provide the same functionality.
The midl_user-*() functions you mention are used for RPC memory management. You need them in case you deal with RPC interfaces, not COM interfaces.
Related
It is known that memory allocation with new calls respective type constructor and memory allocation with malloc does not. But what about kmalloc?
I am trying to develop some system calls and I need to assign memory to a structure below.
struct mailbox{
unsigned long existing_messages;
unsigned long mxid;
struct message *msg;
struct message *last_node;
mailbox(){
existing_messages = 0;
mxid = 0;
msg = NULL;
last_node = NULL;
}
};
If I allocate memory with kmalloc will it call constructor for struct mailbox at allocation time? if not what are the reasonable possible ways to get the constructor called except calling constructor explicitly. Is there any equivalent function as new for memory allocation in kernel?
kmalloc doesn't call constructor.
one way in C++ is to call "placement new".
example:
void* ptr = malloc( sizeof(T) );
T* p = new (ptr) T(); //construct object in memory
note:
you need to call destructor explicitly to avoid memory leaks in object itself,
and then call corresponding de-allocation routine for this memory.
p->~T(); //call destructor
free(ptr); //free memory
When I read asio source code, I am curious about how asio making data synchronized between threads even a implicit strand was made. These are code in asio:
io_service::run
mutex::scoped_lock lock(mutex_);
std::size_t n = 0;
for (; do_run_one(lock, this_thread, ec); lock.lock())
if (n != (std::numeric_limits<std::size_t>::max)())
++n;
return n;
io_service::do_run_one
while (!stopped_)
{
if (!op_queue_.empty())
{
// Prepare to execute first handler from queue.
operation* o = op_queue_.front();
op_queue_.pop();
bool more_handlers = (!op_queue_.empty());
if (o == &task_operation_)
{
task_interrupted_ = more_handlers;
if (more_handlers && !one_thread_)
{
if (!wake_one_idle_thread_and_unlock(lock))
lock.unlock();
}
else
lock.unlock();
task_cleanup on_exit = { this, &lock, &this_thread };
(void)on_exit;
// Run the task. May throw an exception. Only block if the operation
// queue is empty and we're not polling, otherwise we want to return
// as soon as possible.
task_->run(!more_handlers, this_thread.private_op_queue);
}
else
{
std::size_t task_result = o->task_result_;
if (more_handlers && !one_thread_)
wake_one_thread_and_unlock(lock);
else
lock.unlock();
// Ensure the count of outstanding work is decremented on block exit.
work_cleanup on_exit = { this, &lock, &this_thread };
(void)on_exit;
// Complete the operation. May throw an exception. Deletes the object.
o->complete(*this, ec, task_result);
return 1;
}
}
in its do_run_one, the unlock of mutex are all before execute handler. If there is a implicit strand, handler will not executed concurrent, but the problem is: thread A run a handler which modify data, and thread B run next handler which read the data which had been modified by thread A. Without the protect of mutex, how thread B seen the changes of data made by thread A? The mutex unlocking ahead of handler execution doesn't make a happen-before relationship between threads access the data which handler accessed.
When I go further, the handler execution use a thing called fenced_block:
completion_handler* h(static_cast<completion_handler*>(base));
ptr p = { boost::addressof(h->handler_), h, h };
BOOST_ASIO_HANDLER_COMPLETION((h));
// Make a copy of the handler so that the memory can be deallocated before
// the upcall is made. Even if we're not about to make an upcall, a
// sub-object of the handler may be the true owner of the memory associated
// with the handler. Consequently, a local copy of the handler is required
// to ensure that any owning sub-object remains valid until after we have
// deallocated the memory here.
Handler handler(BOOST_ASIO_MOVE_CAST(Handler)(h->handler_));
p.h = boost::addressof(handler);
p.reset();
// Make the upcall if required.
if (owner)
{
fenced_block b(fenced_block::half);
BOOST_ASIO_HANDLER_INVOCATION_BEGIN(());
boost_asio_handler_invoke_helpers::invoke(handler, handler);
BOOST_ASIO_HANDLER_INVOCATION_END;
}
what is this? I know fence seems a sync primitive which supported by C++11 but this fence is totally writen by asio itself. Does this fenced_block help to do the job of data synchronization?
UPDATED
After I google and read this and this, asio indeed use memory fence primitive to synchronize data in threads, that is more faster than unlock till the handler execute complete(speed difference on x86). In fact Java volatile keyword is implemented by insert memory barrier after write & before read this variable to make happen-before relationship.
If someone could simply describe asio memory fence implemenation or add something I missed or misunderstand, I will accept it.
Before the operation invokes the user handler, Boost.Asio uses a memory fence to provide the appropriate memory reordering without forcing mutual execution of handler execution. Thus, thread B would observe changes to memory that occurred within the context of thread A.
C++03 did not specify requirements for memory visibility with regards to multi-threaded execution. However, C++11 defines these requirements in ยง 1.10 Multi-threaded executions and data races, as well as the Atomic operations and Thread support library sections. Boost and C++11 mutexes do perform the appropriate memory reordering. For other implementations, it is worth checking the mutex library's documentation to verify memory reordering occurs.
Boost.Asio memory fences are an implementation detail, and thus always subject to change. Boost.Asio abstracts itself from the architecture/compiler specific implementations through a series of conditional defines within asio/detail/fenced_block.hpp where only a single memory barrier implementation is included. The underlying implementation is contained within a class, for which a fenced_block alias is created via a typedef.
Here is a relevant excerpt:
#elif defined(__GNUC__) && (defined(__hppa) || defined(__hppa__))
# include "asio/detail/gcc_hppa_fenced_block.hpp"
#elif defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))
# include "asio/detail/gcc_x86_fenced_block.hpp"
#elif ...
...
namespace asio {
namespace detail {
...
#elif defined(__GNUC__) && (defined(__hppa) || defined(__hppa__))
typedef gcc_hppa_fenced_block fenced_block;
#elif defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))
typedef gcc_x86_fenced_block fenced_block;
#elif ...
...
} // namespace detail
} // namespace asio
The implementations of the the memory barriers are specific to the architecture and compilers. Boost.Asio has a family of asio/detail/*_fenced_blocked.hpp header files. For example, the win_fenced_block uses InterlockedExchange for Borland; otherwise it uses the xchg assembly instruction, which has an implicit lock prefix when used with a memory address. For gcc_x86_fenced_block, Boost.Asio uses the memory assembly instruction.
If you find yourself needing to use a fence, then consider the Boost.Atomic library. Introduced in Boost 1.53, Boost.Atomic provides an implementation of thread and signal fences based the C++11 standard. Boost.Asio has been using its own implementation of memory fences prior to the Boost.Atomic being added to Boost. Also, the Boost.Asio fences are scoped based. fenced_block will perform an acquire in its constructor, and a release in its destructor.
Visual C++ debug runtime library features so-called allocation hooks. Works this way: you define a callback and call _CrtSetAllocHook() to set that callback. Now every time a memory allocation/deallocation/reallocation is done CRT calls that callback and passes a handful of parameters.
I successfully used an allocation hook to find a reproduceable memory leak - basically CRT reported that there was an unfreed block with allocation number N (N was the same on every program run) at program termination and so I wrote the following in my hook:
int MyAllocHook( int allocType, void* userData, size_t size, int blockType,
long requestNumber, const unsigned char* filename, int lineNumber)
{
if( requestNumber == TheNumberReported ) {
Sleep( 0 );// a line to put breakpoint on
}
return TRUE;
}
since the leak was reported with the very same allocation number every time I could just put a breakpoint inside the if-statement and wait until it was hit and then inspect the call stack.
What other useful things can I do using allocation hooks?
You could also use it to find unreproducible memory leaks:
Make a data structure where you map the allocated pointer to additional information
In the allocation hook you could query the current call stack (StackWalk function) and store the call stack in the data structure
In the de-allocation hook, remove the call stack information for that allocation
At the end of your application, loop over the data structure and report all call stacks. These are the places where memory was allocated but not freed.
The value "requestNumber" is not passed on to the function when deallocating (MS VS 2008). Without this number you cannot keep track of your allocation. However, you can peek into the heap header and extract that value from there:
Note: This is compiler dependent and may change without notice/ warning by the compiler.
// This struct is a copy of the heap header used by MS VS 2008.
// This information is prepending each allocated memory object in debug mode.
struct MsVS_CrtMemBlockHeader {
MsVS_CrtMemBlockHeader * _next;
MsVS_CrtMemBlockHeader * _prev;
char * _szFilename;
int _nLine;
int _nDataSize;
int _nBlockUse;
long _lRequest;
char _gap[4];
};
int MyAllocHook(..) { // same as in question
if(nAllocType == _HOOK_FREE) {
// requestNumber isn't passed on to the Hook on free.
// However in the heap header this value is stored.
size_t headerSize = sizeof(MsVS_CrtMemBlockHeader);
MsVS_CrtMemBlockHeader* pHead;
size_t ptr = (size_t) pvData - headerSize;
pHead = (MsVS_CrtMemBlockHeader*) (ptr);
long requestNumber = pHead->_lRequest;
// Do what you like to keep track of this allocation.
}
}
You could keep record of every allocation request then remove it once the deallocation is invoked, for instance: This could help you tracking memory leak problems that are way much worse than this to track down.
Just the first idea that comes to my mind...
I have a callback that is called through a delegate. Inside it I will need to treat the buffer data that arrive from a record procedure. Normally in a unmanaged context I could do a reinterpret_cast on dwParam1 to get the object reference.
But in a manged context how can I cast a DWORD_PTR to a managed object ref?
static void WaveInProc(HWAVEIN hwi, UINT uMsg, DWORD_PTR dwInstance, DWORD_PTR dwParam1, DWORD_PTR dwParam2)
{
ControlLib::SoundDevice^ soudDevice = ?cast_native2managed?(dwParam1);
Here ya go, more or less what gcroot (per my comment above) does:
using namespace System;
using namespace System::Runtime::InteropServices;
// track an object by a normal (not pinned) handle, encoded in a DWORD_PTR
DWORD_PTR ParamFromObject(Object^ obj)
{
return reinterpret_cast<DWORD_PTR>(GCHandle::ToIntPtr(GCHandle::Alloc(obj)).ToPointer());
}
static void WaveInProc(PVOID hwi, UINT uMsg, DWORD_PTR dwInstance, DWORD_PTR dwParam1, DWORD_PTR dwParam2)
{
// unwrap the encoded handle...
GCHandle h = GCHandle::FromIntPtr(IntPtr(reinterpret_cast<void*>(dwParam1)));
try
{
// and cast your object back out
SoundDevice^ x = safe_cast<SoundDevice^>(h.Target);
}
finally
{
// remember to free the handle when you're done, otherwise your object will never be collected
GCHandle::Free(h);
}
}
I'm not a C++/CLI expert but at a glance I don't think this is possible. If it were possible it would be very much unsafe. The basic problem here is that managed references and native pointers are not the same thing and don't support the same set of operations.
What makes me think this is not possible is that managed objects can move around in memory. Garbage collection operations for instance compact and move memory which correspondingly changes the addresses of objects. Hence it's not possible to have a raw pointer / address value of a managed object because any given GC could invalidate it.
This is not strictly true as you can pin an object in memory. But even then I don't think there is a way to get C++ to treat an arbitrary address as a managed handle.
I think a much safer approach would be the following.
Wrap the managed object inside of a native object
Pass the address of the native object throughout your API's
Use reinterpret_cast to get back to the native type and then safely access your managed handle.
I'm using IDebugSymbols::GetNameByOffset and I'm finding that I get the same symbol name for different functions that overload the same name.
E.g. The code I'm looking up the symbols for might be as follows:
void SomeFunction(int) {..}
void SomeFunction(float) {..}
At runtime, when I have an address of an instruction from each of these functions I'd like to use GetNameByOffset and tell the two apart somehow. I've experimented with calling SetSymbolOptions toggling the SYMOPT_UNDNAME and SYMOPT_NO_CPP flags as documented here, but this didn't work.
Does anyone know how to tell these to symbols apart in the debugger engine universe?
Edit: Please see me comment on the accepted answer for a minor amendment to the proposed solution.
Quote from dbgeng.h:
// A symbol name may not be unique, particularly
// when overloaded functions exist which all
// have the same name. If GetOffsetByName
// finds multiple matches for the name it
// can return any one of them. In that
// case it will return S_FALSE to indicate
// that ambiguity was arbitrarily resolved.
// A caller can then use SearchSymbols to
// find all of the matches if it wishes to
// perform different disambiguation.
STDMETHOD(GetOffsetByName)(
THIS_
__in PCSTR Symbol,
__out PULONG64 Offset
) PURE;
So, I would get the name with IDebugSymbols::GetNameByOffset() (it comes back like "module!name" I believe), make sure it is an overload (if you're not sure) using IDebugSymbols::GetOffsetByName() (which is supposed to return S_FALSE for multiple overloads), and look up all possibilities with this name using StartSymbolMatch()/EndSymbolMatch(). Not a one liner though (and not really helpful for that matter...)
Another option would be to go with
HRESULT
IDebugSymbols3::GetFunctionEntryByOffset(
IN ULONG64 Offset,
IN ULONG Flags,
OUT OPTIONAL PVOID Buffer,
IN ULONG BufferSize,
OUT OPTIONAL PULONG BufferNeeded
);
// It can be used to retrieve FPO data on a particular function:
FPO_DATA fpo;
HRESULT hres=m_Symbols3->GetFunctionEntryByOffset(
addr, // Offset
0, // Flags
&fpo, // Buffer
sizeof(fpo), // BufferSize
0 // BufferNeeded
));
and then use fpo.cdwParams for basic parameter size discrimination (cdwParams=size of parameters)