Put c++ iterator into queue - c++11

Is there any problem to put c++ iterator into queue? For example:
vector<vector<int>> vecs;
queue<pair<vector<int>::iterator, vector<int>::iterator>> mq;
for (auto &e : vecs)
{
mq.push(make_pair(e.begin(), e.end()));
}

Iterators may be invalidated if you modify the variable vecs.
For more details, refer to iterator invalidation rules.

#include<iostream>
#include<vector>
#include<queue>
#include<algorithm>
#include<utility>
using namespace std;
vector<vector<int>> vecs;
queue<pair<vector<int>::iterator, vector<int>::iterator>> mq;
void populate_vector()
{
for(int i = 0;i<6;++i)
{
for(int j =0;j<6;++j)
{
vecs[i].push_back(i+j);
}
}
}
void print_queue()
{
queue<pair<vector<int>::iterator, vector<int>::iterator>> :: iterator qiter = begin(mq);qiter != end(mq);++qiter)
cout<<"*qiter.first"<<*qiter.first<<"*qiter.second"<<*qiter.second<<endl;
}
void print_vector_run_time_error(vector<vector<int>> cont){
try{
for(int i =0;i<6;++i)
{
for(int j =0;j<6;++j)
{
cout<<cont[i][j]<<" ";
}
cout<<endl;
}
}
catch(exception &e){
cout<<e.what();
}
}
int main()
{
for (auto &e : vecs)
{
mq.push(make_pair(e.begin(), e.end()));
}
/*
if the below 3 function calls were not present,compilation and running happens successfully .
This code creates problems at compile or run time as described below (whenever memory is accessed and iterators are invalidated).
*/
print_queue();
/*
gives a compile time error as the compiler does not recognize a call to vector<int>::iterator as iterator is not a container
*/
populate_vector();
/*
the above function compiles successfully but terminates at run time , because memory is accessed during run time to fill the
elements with a value(C++11 vector is populated always at run time).
*/
print_vector_run_time_error(vecs);
/* above function call to print vector compiles successfully , but terminates at run time
because memory is accessed at run time to print the value of the elements.
*/
return 0;
}
This code could give an explanation for the problem.

Related

Implement Stack using Queues

I am tried to solve this problem but can't rectify number of times. I replace what I know
class MyStack {
public:
int *arr;
int tops;
MyStack() {
arr=new int(25);
int tops=-1;
}
void push(int x) {
arr[++tops]=x;
}
int pop() {
tops--;
return arr[tops];
}
int top() {
return arr[tops];
}
bool empty() {
bool a;
a= tops==-1?true:false;
return a;
}
};
When I'm using some other complier it works and I got perfect.
Why the problem repit again? I got two problem has same error.
All your functions in MyStack (except empty()) have chances of causing illegal memory access. Here's some simple driver code for your class that would cause crashes.
int main()
{
MyStack stack;
int x = stack.pop(); //will crash
int x = stack.top();//will crash
for(int i = 0; i < 26; ++i) {
stack.push(i); //will crash at the last push()
}
return 0;
}
You need to protect:
reads to arr[] member via top() and pop() by always validating if tops is pointing to a valid value. If it's -1 it would cause a crash while accessing arr[-1] due to a heap-buffer underflow.
writes to arr[] via push() by always verifying tops and whether it's within the bounds of the allocated memory for arr[]. In the code, arr[] is backed by 25 integer elements. What if there are 26 elements pushed to the stack? The last would be a heap-buffer overflow.

std::atomic on struct bit-fields

I'm modifying some existing open source library and there is a struct (say named as Node) containing bit-fields, e.g.
struct Node {
std::atomic<uint32_t> size:30;
std::atomic<uint32_t> isnull:1;
};
To fit my needs, these fields need to be atomic so I was expecting to use std::atomic for this and faced compile time error:
bit-field 'size' has non-integral type 'std::atomic<uint32_t>'
According to documentation, there is a restricted set of types which can be used for std::atomic
Can anyone advise/have idea on how to get functionality of atomic fields with the minimum impact to the existing source code?
Thanks in advance!
I used an unsigned short as an example below.
This is less ideal, but you could sacrifice 8 bits and insert a std::atomic_flag in the bit field with a union. Unfortunately, std::atomic_flag type is a std::atomic_bool type.
This structure can be spin locked manually every time you access it. However, the code should have minimal performance degradation (unlike creating, locking, unlocking, destroying with a std::mutex and std::unique_lock).
This code may waste about 10-30 clock cycles to enable low cost multi-threading.
PS. Make sure the reserved 8 bits below are not messed up by the endian structure of the processor. You may have to define at the end for big-endian processors. I only tested this code on an Intel CPU (always little-endian).
#include <iostream>
#include <atomic>
#include <thread>
union Data
{
std::atomic_flag access = ATOMIC_FLAG_INIT; // one byte
struct
{
typedef unsigned short ushort;
ushort reserved : 8;
ushort count : 4;
ushort ready : 1;
ushort unused : 3;
} bits;
};
class SpinLock
{
public:
inline SpinLock(std::atomic_flag &access, bool locked=true)
: mAccess(access)
{
if(locked) lock();
}
inline ~SpinLock()
{
unlock();
}
inline void lock()
{
while (mAccess.test_and_set(std::memory_order_acquire))
{
}
}
// each attempt will take about 10-30 clock cycles
inline bool try_lock(unsigned int attempts=0)
{
while(mAccess.test_and_set(std::memory_order_acquire))
{
if (! attempts) return false;
-- attempts;
}
return true;
}
inline void unlock()
{
mAccess.clear(std::memory_order_release);
}
private:
std::atomic_flag &mAccess;
};
void aFn(int &i, Data &d)
{
SpinLock lock(d.access, false);
// manually locking/unlocking can be tighter
lock.lock();
if (d.bits.ready)
{
++d.bits.count;
}
d.bits.ready ^= true; // alternate each time
lock.unlock();
}
int main(void)
{
Data f;
f.bits.count = 0;
f.bits.ready = true;
std::thread *p[8];
for (int i = 0; i < 8; ++ i)
{
p[i] = new std::thread([&f] (int i) { aFn(i, f); }, i);
}
for (int i = 0; i < 8; ++i)
{
p[i]->join();
delete p[i];
}
std::cout << "size: " << sizeof(f) << std::endl;
std::cout << "count: " << f.bits.count << std::endl;
}
The result is as expected...
size: 2
count: 4

libtorrent - storage_interface readv explanation

I have implemented a custom storage interface in libtorrent as described in the help section here.
The storage_interface is working fine, although I can't figure out why readv is only called randomly while downloading a torrent. From my view the overriden virtual function readv should get called each time I call handle->read_piece in piece_finished_alert. It should read the piece for read_piece_alert?
The buffer is provided in read_piece_alert without getting notified in readv.
So the question is why it is called only randomly and why it's not called on a read_piece() call? Is my storage_interface maybe wrong?
The code looks like this:
struct temp_storage : storage_interface
{
virtual int readv(file::iovec_t const* bufs, int num_bufs
, int piece, int offset, int flags, storage_error& ec)
{
// Only called on random pieces while downloading a larger torrent
std::map<int, std::vector<char> >::const_iterator i = m_file_data.find(piece);
if (i == m_file_data.end()) return 0;
int available = i->second.size() - offset;
if (available <= 0) return 0;
if (available > num_bufs) available = num_bufs;
memcpy(&bufs, &i->second[offset], available);
return available;
}
virtual int writev(file::iovec_t const* bufs, int num_bufs
, int piece, int offset, int flags, storage_error& ec)
{
std::vector<char>& data = m_file_data[piece];
if (data.size() < offset + num_bufs) data.resize(offset + num_bufs);
std::memcpy(&data[offset], bufs, num_bufs);
return num_bufs;
}
virtual bool has_any_file(storage_error& ec) { return false; }
virtual ...
virtual ...
}
Intialized with
storage_interface* temp_storage_constructor(storage_params const& params)
{
printf("NEW INTERFACE\n");
return new temp_storage(*params.files);
}
p.storage = &temp_storage_constructor;
The function below sets up alerts and invokes read_piece on each completed piece.
while(true) {
std::vector<alert*> alerts;
s.pop_alerts(&alerts);
for (alert* i : alerts)
{
switch (i->type()) {
case read_piece_alert::alert_type:
{
read_piece_alert* p = (read_piece_alert*)i;
if (p->ec) {
// read_piece failed
break;
}
// piece buffer, size is provided without readv
// notification after invoking read_piece in piece_finished_alert
break;
}
case piece_finished_alert::alert_type: {
piece_finished_alert* p = (piece_finished_alert*)i;
p->handle.read_piece(p->piece_index);
// Once the piece is finished, we read it to obtain the buffer in read_piece_alert.
break;
}
default:
break;
}
}
Sleep(100);
}
I will answer my own question. As Arvid said in the comments: readv was not invoked because of caching. Setting settings_pack::use_read_cache to false will invoke readv always.

Stored lambda function calls are very slow - fix or workaround?

In an attempt to make a more usable version of the code I wrote for an answer to another question, I used a lambda function to process an individual unit. This is a work in progress. I've got the "client" syntax looking pretty nice:
// for loop split into 4 threads, calling doThing for each index
parloop(4, 0, 100000000, [](int i) { doThing(i); });
However, I have an issue. Whenever I call the saved lambda, it takes up a ton of CPU time. doThing itself is an empty stub. If I just comment out the internal call to the lambda, then the speed returns to normal (4 times speedup for 4 threads). I'm using std::function to save the reference to the lambda.
My question is - Is there some better way that the stl library internally manages lambdas for large sets of data, that I haven't come across?
struct parloop
{
public:
std::vector<std::thread> myThreads;
int numThreads, rangeStart, rangeEnd;
std::function<void (int)> lambda;
parloop(int _numThreads, int _rangeStart, int _rangeEnd, std::function<void(int)> _lambda) //
: numThreads(_numThreads), rangeStart(_rangeStart), rangeEnd(_rangeEnd), lambda(_lambda) //
{
init();
exit();
}
void init()
{
myThreads.resize(numThreads);
for (int i = 0; i < numThreads; ++i)
{
myThreads[i] = std::thread(myThreadFunction, this, chunkStart(i), chunkEnd(i));
}
}
void exit()
{
for (int i = 0; i < numThreads; ++i)
{
myThreads[i].join();
}
}
int rangeJump()
{
return ceil(float(rangeEnd - rangeStart) / float(numThreads));
}
int chunkStart(int i)
{
return rangeJump() * i;
}
int chunkEnd(int i)
{
return std::min(rangeJump() * (i + 1) - 1, rangeEnd);
}
static void myThreadFunction(parloop *self, int start, int end) //
{
std::function<void(int)> lambda = self->lambda;
// we're just going to loop through the numbers and print them out
for (int i = start; i <= end; ++i)
{
lambda(i); // commenting this out speeds things up back to normal
}
}
};
void doThing(int i) // "payload" of the lambda function
{
}
int main()
{
auto start = timer.now();
auto stop = timer.now();
// run 4 trials of each number of threads
for (int x = 1; x <= 4; ++x)
{
// test between 1-8 threads
for (int numThreads = 1; numThreads <= 8; ++numThreads)
{
start = timer.now();
// this is the line of code which calls doThing in the loop
parloop(numThreads, 0, 100000000, [](int i) { doThing(i); });
stop = timer.now();
cout << numThreads << " Time = " << std::chrono::duration_cast<std::chrono::nanoseconds>(stop - start).count() / 1000000.0f << " ms\n";
//cout << "\t\tsimple list, time was " << deltaTime2 / 1000000.0f << " ms\n";
}
}
cin.ignore();
cin.get();
return 0;
}
I'm using std::function to save the reference to the lambda.
That's one possible problem, as std::function is not a zero-runtime-cost abstraction. It is a type-erased wrapper that has a virtual-call like cost when invoking operator() and could also potentially heap-allocate (which could mean a cache-miss per call).
If you want to store your lambda in such a way that does not introduce additional overhead and that allows the compiler to inline it, you should use a template parameter. This is not always possible, but might fit your use case. Example:
template <typename TFunction>
struct parloop
{
public:
std::thread **myThreads;
int numThreads, rangeStart, rangeEnd;
TFunction lambda;
parloop(TFunction&& _lambda,
int _numThreads, int _rangeStart, int _rangeEnd)
: lambda(std::move(_lambda)),
numThreads(_numThreads), rangeStart(_rangeStart),
rangeEnd(_rangeEnd)
{
init();
exit();
}
// ...
To deduce the type of the lambda, you can use an helper function:
template <typename TF, typename... TArgs>
auto make_parloop(TF&& lambda, TArgs&&... xs)
{
return parloop<std::decay_t<TF>>(
std::forward<TF>(lambda), std::forward<TArgs>(xs)...);
}
Usage:
auto p = make_parloop([](int i) { doThing(i); },
numThreads, 0, 100000000);
I wrote an article that's related to the subject:
"Passing functions to functions"
It contains some benchmarks that show how much assembly is generated for std::function compared to a template parameter and other solutions.

Why doesn't my freopen function work?

#include <stdio.h>
#include <string.h>
#define MAXN 15
char forbid[MAXN][MAXN];
int dp[2][1<<MAXN],c[1<<MAXN],*dp1,*dp2;
int cnt_one(int x)
{
int s=0;
while(x)
{
s++;
x&=x-1;
}
return s;
}
int main()
{
int t,n,s,a,b,i,j,k;
/*This is my use of freopen function*
************************************/
freopen("datain.txt","r",stdin);
freopen("dataout.txt","w",stdout);
/*This is just a dynamic program to solve a mathematical problem*
****************************************************************/
for(i=0;i<(1<<MAXN);i++) c[i]=cnt_one(i);
scanf("%d",&t);
while(t--)
{
memset(forbid,0,sizeof(forbid));
memset(dp[0],0,sizeof(int)*(1<<MAXN));
dp[0][0]=1;
scanf("%d%d",&n,&s);
while(s--)
{
scanf("%d%d",&a,&b);
forbid[a][b]=1;
}
for(i=1;i<=n;i++)
{
if(i%2)
{
dp1=dp[0];
dp2=dp[1];
}
else
{
dp1=dp[1];
dp2=dp[0];
}
memset(dp2,0,sizeof(int)*(1<<MAXN));
for(j=0;j<(1<<n);j++)
{
if(c[j]!=i-1) continue;
for(k=0;k<n;k++)
{
if(!(j>>k&1)&&!forbid[i][n-k]) dp2[j^(1<<k)]+=dp1[j];
}
}
}
printf("%d\n",dp2[(1<<n)-1]);
}
return 0;
}
This is my program.I used a dynamic program method to solve a mathematical problem.
But when I used the "freopen" function to redirect the "stdout" stream to the file of "dataout.txt",it failed and the file had no data in it.
Could you tell me why I can get the data from "datain.txt" but I can't output data into "dataout.txt"?Is there something wrong with my "freopen" function for "stdout" stream?
I think the problem is that you have a segmentation fault that you will not see. If you place a printf and a return after the freopen I think it will work (works for me).
I would recommend you to use "valgrind" on the resulting executable compiled for debug to see at what line the program crashes. If you want someone to help you further, you need to provide an example input.

Resources