I asked a question about boost::asio here but some additional questions came up today. I have some very simple Server Client structure and at one point this async_write command:
ushort _nSetupReceiveBuffer[_nDynamicSize];
boost::asio::async_write(m_oSocket, boost::asio::buffer(&_nSetupReceiveBuffer, _nDynamicSize),
[this, self](boost::system::error_code _oError, std::size_t)
{
std::cout << _nSetupReceiveBuffer.size() << std::endl;
});
Unfortunately it results in error: ‘_nSetupReceiveBuffer’ is not captured error.
So my questions are:
How can I capture nSetupReceiveBuffer or better capture its reference? (Capturing its reference with [this, self, &_nSetupReceiveBuffer] builds but results in Segmentation fault (core dumped) error even before any data is received, since, I assume, it is executed as callback the original variable is already deleted.)
I use ushort because I want to transmit cv::Mat images with CV_16U setting and tried to follow this idea. Do you have other ideas how to transmit cv::Mat files via boost? I can only use lossless containers, but I want to avoid high CPU loads. In my case the bandwith shouldn't be the problem. However that is only half of the truth, I tried to use a serializer which serializes the image to a string, which worked fine but increased the size dramatically :-(
To avoid the problems from 1. I could make _nSetupReceiveBuffer a member variable, but I need to allocate it dynamically at runtime, which I don't know how to do? And a second drawback would be, that the variable type ushort is fixed, but I want to be flexible, e.g. if I have another video stream of CV_8U type, I need to change it to uchar.
My last approach was to reshape() the cv::Mat and use a std::vector<ushort> as member variabel which stores the single values. Is that reasonable or only generating large overhead? Can I define a vector as member variable without a type specified, or should I define one for each type and then just use the appropriate one?
Thank you for your help.
How can I capture nSetupReceiveBuffer or better capture its reference? (Capturing its reference with [this, self, &_nSetupReceiveBuffer] builds but results in Segmentation fault (core dumped) error even before any data is received, since, I assume, it is executed as callback the original variable is already deleted.)
Capturing the reference of a variable that will disappear is useless.
Capturing the variable means you can not pass it to the async call.
To avoid the problems from 1. I could make _nSetupReceiveBuffer a member variable, but I need to allocate it dynamically at runtime, which I don't know how to do?
Use std::vector. This should be lesson 1 in C++.
>And a second drawback would be, that the variable type ushort is fixed, but I want to be flexible, e.g. if I have another video stream of CV_8U type, I need to change it to uchar.
This indicates you should probably separate the IO operation from your class. If you make a type to represent the IO operation, you can make the buffer a member of that type.
My last approach was to reshape() the cv::Mat and use a std::vector as member variabel which stores the single values. Is that reasonable or only generating large overhead? Can I define a vector as member variable without a type specified, or should I define one for each type and then just use the appropriate one?
I'd suggest separating the serialization concerns, and use Boost Serialization, like e.g. given here: Serializing OpenCV Mat_<Vec3f>
If you can be sure your matrices are always continuous, you could use the buffer directly:
boost::asio::async_write(m_oSocket, boost::asio::buffer(mat.ptr(), mat.total()),
[this, self](boost::system::error_code ec, std::size_t tranferred)
{
std::cout << tranferred << std::endl;
});
Of course, you'll have the same lifetime considerations as with the nSetupReceiveBuffer described above. Also, keep in mind that other threads should not touch the data until the IO operation completes.
Related
I have a question regarding the thrust library when using CUDA.
I am using a thrust function, i.e. exclusive_scan, and I want to use raw pointers. I am using raw (device) pointers because I want to have full control of when the memory is allocated and deallocated.
After the function call, I will hand over the pointer to another data structure and then free the memory in either the destructor of this data structure, or in the next function call, when I recompute my (device) pointers. I came across for example this problem here now, which recommends to wrap the data structure in a device_vector. But then I run into the problem that the memory is freed once my device_vector goes out of scope, which I do not want. Having the device pointer globally is also not an option, since I am hacking code, i.e. it is used as a buffer and I would have to rewrite a lot if I wanted to do something like that.
Does anyone have a good workaround regarding this? The only chance I do see right now is to rewrite the thrust-function on my own, only using raw device-pointers.
EDIT: I misread, I can wrap it in a device_ptr instead of a device_vector.
Asking further though, how could I solve this if there wasn't the option of using a device_ptr?
There is no problem using plain pointers in thrust methods.
For data on the device do:
....
struct DoSomething {
__device__ int operator()(int item) { return 1; }
};
int* IntData;
cudaMalloc(&IntData, sizeof(int) * count);
auto dev_data = device_pointer_cast(IntData);
thrust::generate(dev_data, dev_data + count, DoSomething());
thrust::sort(dev_data, dev_data + count);
....
cudaFree(IntData);
For data on the host use plain malloc/free and raw_pointer_cast instead of device_pointer_cast.
See: thrust: Memory management
I wrote some code like this:
shared_ptr<int> r = make_shared<int>();
int *ar = r.get();
delete ar; // report double free or corruption
// still some code
When the code ran up to delete ar;, the program crashed, and reported "double free or corruption", I'm confused why double free? The "r" is still in the scope, and not popped-off from stack. Do the delete operator do something magic?? Does it know the raw pointer is handled by a smart pointer currently? and then counter in "r" be decremented to zero automatically?
I know the operations is not recommended， but I want to know why?
You are deleting a pointer that didn't come from new, so you have undefined behavior (anything can happen).
From cppreference on delete:
For the first (non-array) form, expression must be a pointer to an object type or a class type contextually implicitly convertible to such pointer, and its value must be either null or pointer to a non-array object created by a new-expression, or a pointer to a base subobject of a non-array object created by a new-expression. If expression is anything else, including if it is a pointer obtained by the array form of new-expression, the behavior is undefined.
If the allocation is done by new, we can be sure that the pointer we have is something we can use delete on. But in the case of shared_ptr.get(), we cannot be sure if we can use delete because it might not be the actual pointer returned by new.
shared_ptr<int> r = make_shared<int>();
There is no guarantee that this will call new int (which isn't strictly observable by the user anyway) or more generally new T (which is observable with a user defined, class specific operator new); in practice, it won't (there is no guarantee that it won't).
The discussion that follows isn't just about shared_ptr, but about "smart pointers" with ownership semantics. For any owning smart pointer smart_owning:
The primary motivation for make_owning instead of smart_owning<T>(new T) is to avoid having a memory allocation without owner at any time; that was essential in C++ when order of evaluation of expressions didn't provide the guarantee that evaluation of the sub-expressions in the argument list was immediately before call of that function; historically in C++:
f (smart_owning<T>(new T), smart_owning<U>(new U));
could be evaluated as:
T *temp1 = new T;
U *temp2 = new U;
auto &&temp3 = smart_owning<T>(temp1);
auto &&temp4 = smart_owning<U>(temp2);
This way temp1 and temp2 are not managed by any owning object for a non trivial time:
obviously new U can throw an exception
constructing an owning smart pointer usually requires the allocation of (small) ressources and can throw
So either temp1 or temp2 could be leaked (but not both) if an exception was thrown, which was the exact problem we were trying to avoid in the first place. This means composite expressions involving construction of owning smart pointers was a bad idea; this is fine:
auto &&temp_t = smart_owning<T>(new T);
auto &&temp_u = smart_owning<U>(new U);
f (temp_t, temp_u);
Usually expression involving as many sub-expression with function calls as f (smart_owning<T>(new T), smart_owning<U>(new U)) are considered reasonable (it's a pretty simple expression in term of number of sub-expressions). Disallowing such expressions is quite annoying and very difficult to justify.
[This is one reason, and in my opinion the most compelling reason, why the non determinism of the order of evaluation was removed by the C++ standardisation committee so that such code is not safe. (This was an issue not just for memory allocated, but for any managed allocation, like file descriptors, database handles...)]
Because code frequently needed to do things such as smart_owning<T>(allocate_T()) in sub-expressions, and because telling programmers to decompose moderately complex expressions involving allocation in many simple lines wasn't appealing (more lines of code doesn't mean easier to read), the library writers provided a simple fix: a function to do the creation of an object with dynamic lifetime and the creation of its owning object together. That solved the order of evaluation problem (but was complicated at first because it needed perfect forwarding of the arguments of the constructor).
Giving two tasks to a function (allocate an instance of T and a instance of smart_owning) gives the freedom to do an interesting optimization: you can avoid one dynamic allocation by putting both the managed object and its owner next to each others.
But once again, that was not the primary purpose of functions like make_shared.
Because exclusive ownership smart pointers by definition don't need to keep a reference count, and by definition don't need to share the data needed for the deleter either between instances, and so can keep that data in the "smart pointer"(*), no additional allocation is needed for the construction of unique_ptr; yet a make_unique function template was added, to avoid the dangling pointer issue, not to optimize a non-thing (an allocation that isn't done in the fist place).
(*) which BTW means unique owner "smart pointers" do not have pointer semantic, as pointer semantic implies that you can makes copies of the "pointer", and you can't have two copies of a unique owner pointing to the same instance; "smart pointers" were never pointers anyway, the term is misleading.
Summary:
make_shared<T> does an optional optimization where there is no separate dynamic memory allocation for T: there is no operator new(sizeof (T)). There is obviously still the creation of an instance with dynamic lifetime with another operator new: placement new.
If you replace the explicit memory deallocation with an explicit destruction and add a pause immediately after that point:
class C {
public:
~C();
};
shared_ptr<C> r = make_shared<C>();
C *ar = r.get();
ar->~C();
pause(); // stops the program forever
The program will probably run fine; it is still illogical, indefensible, incorrect to explicitly destroy an object managed by a smart pointer. It isn't "your" resource. If pause() could exit with an exception, the owning smart pointer would try to destroy the managed object which doesn't even exist anymore.
It of course depends on how library implements make_shared, however most probable implementation is that:
std::make_shared allocates one block for two things:
shared pointer control block
contained object
std::make_shared() will invoke memory allocator once and then it will call placement new twice to initialize (call constructors) of mentioned two things.
| block requested from allocator |
| shared_ptr control block | X object |
#1 #2 #3
That means that memory allocator has provided one big block, which address is #1.
Shared pointer then uses it for control block (#1) and actual contained object (#2).
When you invoke delete with actual object kept by shred_ptr ( .get() ) you call delete(#2).
Because #2 is not known by allocator you get an corruption error.
See here. I quot:
std::shared_ptr is a smart pointer that retains shared ownership of an object through a pointer. Several shared_ptr objects may own the same object. The object is destroyed and its memory deallocated when either of the following happens:
the last remaining shared_ptr owning the object is destroyed;
the last remaining shared_ptr owning the object is assigned another pointer via operator= or reset().
The object is destroyed using delete-expression or a custom deleter that is supplied to shared_ptr during construction.
So the pointer is deleted by shared_ptr. You're not suppose to delete the stored pointer yourself
UPDATE:
I didn't realize that there were more statements and the pointer was not out of scope, I'm sorry.
I was reading more and the standard doesn't say much about the behavior of get() but here is a note, I quote:
A shared_ptr may share ownership of an object while storing a pointer to another object. get() returns the stored pointer, not the managed pointer.
So it looks that it is allowed that the pointer returned by get() is not necessarily the same pointer allocated by the shared_ptr (presumably using new). So delete that pointer is undefined behavior. I will be looking a little more into the details.
UPDATE 2:
The standard says at § 20.7.2.2.6 (about make_shared):
6 Remarks: Implementations are encouraged, but not required, to perform no more than one memory allocation. [ Note: This provides efficiency equivalent to an intrusive smart pointer. — end note ]
7 [ Note: These functions will typically allocate more memory than sizeof(T) to allow for internal bookkeeping structures such as the reference counts. — end note ]
So an specific implementation of make_shared could allocate a single chunk of memory (or more) and use part of that memory to initialize the stored pointer (but maybe not all the memory allocated). get() must return a pointer to the stored object, but there is no requirement by the standard, as previously said, that the pointer returned by get() has to be the one allocated by new. So delete that pointer is undefined behavior, you got a signal raised but anything can happen.
Raw pointers can point to objects allocated on the stack or on the heap.
Heap allocation example:
// heap allocation
int* rawPtr = new int(100);
std::cout << *rawPtr << std::endl; // 100
Stack allocation example:
int i = 100;
int* rawPtr = &i;
std::cout << *rawPtr << std::endl; // 100
Heap allocation using auto_ptr example:
int* rawPtr = new int(100);
std::unique_ptr<int> uPtr(rawPtr);
std::cout << *uPtr << std::endl; // 100
Stack allocation using auto_ptr example:
int i = 100;
int* rawPtr = &i;
std::unique_ptr<int> uPtr(rawPtr); // runtime error
Are 'smart pointers' intended to be used to point to dynamically created objects on the heap? For C++11, are we supposed to continue using raw pointers for pointing to stack allocated objects? Thank you.
Smart pointers are usually used to point to objects allocated with new and deleted with delete. They don't have to be used this way, but that would seem to be the intent, if we want to guess the intended use of the language constructs.
The reason your code crashes in the last example is because of the "deleted with delete" part. When it goes out of scope, the unique_ptr will try to delete the object it has a pointer to. Since it was allocated on the stack, this fails. Just as if you had written, delete rawPtr;
Since one usually uses smart pointers with heap objects, there is a function to allocate on the heap and convert to a smart pointer all in one go. std::unique_ptr<int> uPtr = make_unique<int>(100); will perform the actions of the first two lines of your third example. There is also a matching make_shared for shared pointers.
It is possible to use smart pointers with stack objects. What you do is specify the deleter used by the smart pointer, providing one that does not call delete. Since it's a stack variable and nothing need be done to delete it, the deleter could do nothing. Which makes one ask, what's the point of the smart pointer then, if all it does is call a function that does nothing? Which is why you don't commonly see smart pointers used with stack objects. But here's an example that shows some usefulness.
{
char buf[32];
auto erase_buf = [](char *p) { memset(p, 0, sizeof(buf)); };
std::unique_ptr<char, decltype(erase_buf)> passwd(buf, erase_buf);
get_password(passwd.get());
check_password(passwd.get());
}
// The deleter will get called since passwd has gone out of scope.
// This will erase the memory in buf so that the password doesn't live
// on the stack any longer than it needs to. This also works for
// exceptions! Placing memset() at the end wouldn't catch that.
The runtime error is due to the fact that delete was called on a memory location that was never allocated with new.
If an object has already been created with dynamic storage duration (typically implemented as creation on a 'heap') then a 'smart pointer' will not behave correctly as demonstrated by the runtime error.
Are 'smart pointers' intended to be used to point to dynamically
created objects on the heap? For C++11, are we supposed to continue
using raw pointers for pointing to stack allocated objects?
As for what one is supposed to do, well, it helps to think of the storage duration and specifically how the object was created.
If the object has automatic storage duration (stack) then avoid taking the address and use references. The ownership does not belong with the pointer and a reference makes the ownership clearer.
If the object has dynamic storage duration (heap) then a smart pointer is the way to go as it can then manage the ownership.
So for the last example, the following would be better (pointer owns the int):
auto uPtr = std::make_unique<int>(100);
The uPtr will have automatic storage duration and will call the destructor when it goes out of scope. The int will have dynamic storage duration (heap) and will be deleteed by the smart pointer.
One could generally avoid using new and delete and avoid using raw pointers. With make_unique and make_shared, new isn't required.
Are 'smart pointers' intended to be used to point to dynamically created objects on the heap?
They are intended for heap-allocated objects to prevent leaks.
The guideline for C++ is to use plain pointers to refer to a single object (but not own it). The owner of the object holds it by value, in a container or via a smart pointer.
Are 'smart pointers' intended to be used to point to dynamically created objects on the heap?
Yes, but that's just the default. Notice that std::unique_ptr has a constructor (no. (3)/(4) on that page) which takes a pointer that you have obtained somehow, and a "deleter" that you provide. In this case the unique pointer will not do anything with the heap (unless your deleter does so).
For C++11, are we supposed to continue using raw pointers for pointing to stack allocated objects? Thank you.
You should use raw pointers in code that does not "own" the pointer - does not need to concern itself with allocation or deallocation; and that is regardless of whether you're pointing into the heap or the stack or elsewhere.
Another place to use it is when you're implementing some class that has a complex ownership pattern, for protected/private members.
PS: Please, forget about std::auto_ptr... pretend it never existed :-)
How do i allocate memory using new at a fixed location? My book says to do this:
char *buf=new char[sizeof(sample)];
sample *p=new(buf)sample(10,20);
Here new is allocating memory at buf's address, and (10,20) are values being passed. But what is sample? is it an address or a data type?
let me explain this code to you...
char *buf=new char[sizeof(sample)];
sample *p=new(buf)sample(10,20);
This is really four lines of code, written as two for your convenience. Let me just expand them
char *buf; // 1
buf = new char[sizeof(sample)]; // 2
sample *p; // 3
p = new(buf)sample(10,20); // 4
Line 1 and 3 are simple to explain, they are both declaring pointers. buf is a pointer to a char, p is a pointer to a sample. Now, we can not see what sample is, but we can assume that it is either a class defined else where, or some of data type that has been typedefed (more or less just given a new name) but either way, sample can be thought as a data type just link int or string
Line 2 is a allocating a block of memory and assigning it our char pointer called buf. Lets say sample was a class that contains 2 ints, this means it is (under most compilers) going to be 8 bytes (4 per int). And so buf points to the start of a block of memory that has been set aside to hold chars.
Line 4 is where it gets a big complex. if it where just p = new sample(10,20) it would be a simple case of creating a new object of type sample, passing it the two ints and storing the address of this new object in the pointer p. The addition of the (buf) is basically telling new to make use of the memory pointed to by buf.
The end effect is, you have one block of memory allocated (more or less 8 bytes) and it has two pointers pointing to it. One of the points, buf, is looking at that memory as 8 chars, that other, p, is looking at is a single sample.
Why would you do this?
Normally, you wouldn't. Modern C++ has made the sue of new rather redundant, there are many better ways to deal with objects. I guess that main reason for using this method, is if for some reason you want to keep a pool of memory allocated, as it can take time to get large blocks of memory and you might be able to save your self some time.
For the most part, if you think you need to do something like this, you are trying to solve the wrong thing
A Bit Extra
I do not have much experience with embedded or mobile devices, but I have never seen this used.
The code you posted is basically the same as just doing sample *p = new sample(10,20) neither method is controlling where the sample object is created.
Also consider that you do not always need to create objects dynamically using new.
void myFunction(){
sample p = sample(10,20);
}
This automatically creates a sample object for you. This method is much more preferable as it is easier to read and understand and you do not need to worry about deleting the object, it will be cleaned up for you when the function returns.
If you really do need to make use of dynamic objects, consider use of smart pointers, something like unique_ptr<sample> This will give you the ability to use dynamic object creation but save you the hassle of manual deleting the object of type sample (I can point you towards more info on this if you life)
It is a datatype or a typedef.
What I am trying to do is have application A send application B a pointer to an object which A has allocated on shared memory ( using boost::interprocess ). For that pointer transfer I intend to use boost::interprocess::message_queue. Obviously a direct raw pointer from A is not valid in B so I try to transfer an offset_ptr allocated on the shared memory. However that also does not seem to work.
Process A does this:
typedef offset_ptr<MyVector> MyVectorPtr;
MyVectorPtr * myvector;
myvector = segment->construct<MyVectorPtr>( boost::interprocess::anonymous_instance )();
*myvector = segment->construct<MyVector>( boost::interprocess::anonymous_instance )
(*alloc_inst_vec); ;
// myvector gets filled with data here
//Send on the message queue
mq->send(myvector, sizeof(MyVectorPtr), 0);
Process B does this:
// Create a "buffer" on this side of the queue
MyVectorPtr * myvector;
myvector = segment->construct<MyVectorPtr>( boost::interprocess::anonymous_instance )();
mq->receive( myvector, sizeof(MyVectorPtr), recvd_size, priority);
As I see it, in this way a do a bit copy of the offset pointer which invalidates him in process B. How do I do this right?
It seems you can address it as described in this post on the boost mailing list.
I agree there is some awkwardness here and offset_ptr doesn't really work for what you are trying to do. offset_ptr is useful if the pointer itself is stored inside of another class/struct which also is allocated in your shared memory segment, but generally you have some top-level item which is not a member of some object allocated in shared memory.
You'll notice the offset_ptr example kindof glosses over this - it just has a comment "Communicate list to other processes" with no details. In some cases you may have a single named top-level object and that name can be how you communicate it, but if you have an arbitrary number of top-level objects to communicate, it seems like just sending the offset from the shared memory's base address is the best you can do.
You calculate the offset on the sending in, send it, and then add to the base adddress on the receiving end. If you want to be able to send nullptr as well, you could do like offset_ptr does and agree that 1 is an offset that is sufficiently unlikely to be used, or pick another unlikely sentinel value.