How to use OpenMP to deal with two for loops with - openmp

I am new to OpenMP... Please help me with this dumb question. Thank you :)
Basically, I want to use OpenMP to speed up two for loops. But I do not know why it keeps saying: invalid controlling predicate for the for loop.
By the way, my GCC version is gcc (Ubuntu 6.2.0-5ubuntu12) 6.2.0 20161005, and OS I am using is Ubuntu 16.10.
Basically, I generate a toy data that has a typical Key-Value style, like this:
Data = {
"0": ["100","99","98","97",..."1"];
"1": ["100","99","98","97",..."1"];
...
"999":["100","99","98","97",..."1"];
}
Then, for each key, I want to compare its value with the rest of the keys. Here, I sum them up through "user1_list.size()+user2_list.size();". As for each key, the sum-up process is totally independent of other keys, which means this works for parallelism.
Here is my toy example code.
#include <map>
#include <vector>
#include <string>
#include <iostream>
#include "omp.h"
using namespace std;
int main(){
// Create Data
map<string, vector<string>> data;
for(int i=0; i != 1000; i++){
vector<string> list;
for (int j=100; j!=0; j--){
list.push_back(to_string(j));
}
data[to_string(i)]=list;
}
cout << "Data Total size: " << data.size() << endl;
int count = 1;
#pragma omp parallel for private(count)
for (auto it=data.begin(); it!=data.end(); it++){
//cout << "Evoke Thread: " << omp_get_thread_num();
cout << " count: " << count << " / " << data.size() << endl;
count ++;
string user1 = it->first;
vector<string> user1_list = it->second;
for (auto it2=data.begin(); it2!=data.end(); it2++){
string user2 = it2->first;
vector<string> user2_list = it2->second;
cout << "u1:" << user1 << " u2:" << user2;
int total_size = user1_list.size()+user2_list.size();
cout << " total size: " << total_size << endl;
}
}
return 0;
}

Related

How to read chunk of the data from a hdf5 file in c++?

I want to read a chunk of data which is just one frame of many frames stored in one dataset. The shape of the whole dataset is (10, 11214,3), 10 frames each frame has 11214 rows and 4 columns. Here is the file. The chunk I want to read would have the shape (11214,3). I can print the predefined array using, but I'm not sure how can I read data from a hdf5 file. Here is my code,
#include <h5xx/h5xx.hpp>
#include <boost/multi_array.hpp>
#include <iostream>
#include <vector>
#include <cstdio>
typedef boost::multi_array<int, 2> array_2d_t;
const int NI=10;
const int NJ=NI;
void print_array(array_2d_t const& array)
{
for (unsigned int j = 0; j < array.shape()[1]; j++)
{
for (unsigned int i = 0; i < array.shape()[0]; i++)
{
printf("%2d ", array[j][i]);
}
printf("\n");
}
}
void write_int_data(std::string const& filename, array_2d_t const& array)
{
h5xx::file file(filename, h5xx::file::trunc);
std::string name;
{
// --- create dataset and fill it with the default array data (positive values)
name = "integer array";
h5xx::create_dataset(file, name, array);
h5xx::write_dataset(file, name, array);
// --- create a slice object (aka hyperslab) to specify the location in the dataset to be overwritten
std::vector<int> offset; int offset_raw[2] = {4,4}; offset.assign(offset_raw, offset_raw + 2);
std::vector<int> count; int count_raw[2] = {2,2}; count.assign(count_raw, count_raw + 2);
h5xx::slice slice(offset, count);
}
}
void read_int_data(std::string const& filename)
{
h5xx::file file(filename, h5xx::file::in);
std::string name = "integer array";
// read and print the full dataset
{
array_2d_t array;
// --- read the complete dataset into array, the array is resized and overwritten internally
h5xx::read_dataset(file, name, array);
printf("original integer array read from file, negative number patch was written using a slice\n");
print_array(array);
printf("\n");
}
}
int main(int argc, char** argv)
{
std::string filename = argv[0];
filename.append(".h5");
// --- do a few demos/tests using integers
{
array_2d_t array(boost::extents[NJ][NI]);
{
const int nelem = NI*NJ;
int data[nelem];
for (int i = 0; i < nelem; i++)
data[i] = i;
array.assign(data, data + nelem);
}
write_int_data(filename, array);
read_int_data(filename);
}
return 0;
}
I'm using the h5xx — a template-based C++ wrapper for the HDF5 library link and boost library.
The datasets are stored in particles/lipids/box/positions path. The dataset name value holds the frames.
argv[0] is not what you want (arguments start at 1, 0 is the program name). Consider bounds checking as well:
std::vector<std::string> const args(argv, argv + argc);
std::string const filename = args.at(1) + ".h5";
the initialization can be done directly, without a temporary array (what is multi_array for, otherwise?)
for (size_t i = 0; i < array.num_elements(); i++)
array.data()[i] = i;
Or indeed, make it an algorithm:
std::iota(array.data(), array.data() + array.num_elements(), 0);
same with vectors:
std::vector<int> offset; int offset_raw[2] = {4,4}; offset.assign(offset_raw, offset_raw + 2);
std::vector<int> count; int count_raw[2] = {2,2}; count.assign(count_raw, count_raw + 2);
besides being a formatting mess can be simply
std::vector offset{4,4}, count{2,2};
h5xx::slice slice(offset, count);
On To The Real Question
The code has no relevance to the file. At all. I created some debug/tracing code to dump the file contents:
void dump(h5xx::group const& g, std::string indent = "") {
auto dd = g.datasets();
auto gg = g.groups();
for (auto it = dd.begin(); it != dd.end(); ++it) {
std::cout << indent << " ds:" << it.get_name() << "\n";
}
for (auto it = gg.begin(); it != gg.end(); ++it) {
dump(*it, indent + "/" + it.get_name());
}
}
int main()
{
h5xx::file xaa("xaa.h5", h5xx::file::mode::in);
dump(xaa);
}
Prints
/particles/lipids/box/edges ds:box_size
/particles/lipids/box/edges ds:step
/particles/lipids/box/edges ds:time
/particles/lipids/box/edges ds:value
/particles/lipids/box/positions ds:step
/particles/lipids/box/positions ds:time
/particles/lipids/box/positions ds:value
Now we can drill down to the dataset. Let's see whether we can figure out the correct type. It certainly is NOT array_2d_t:
h5xx::dataset ds(xaa, "particles/lipids/box/positions/value");
array_2d_t a;
h5xx::datatype detect(a);
std::cout << "type: " << std::hex << ds.get_type() << std::dec << "\n";
std::cout << "detect: " << std::hex << detect.get_type_id() << std::dec << "\n";
Prints
type: 30000000000013b
detect: 30000000000000c
That's a type mismatch. I guess I'll have to learn to read that gibberish as well...
Let's add some diagnostics:
void diag_type(hid_t type)
{
std::cout << " Class " << ::H5Tget_class(type) << std::endl;
std::cout << " Size " << ::H5Tget_size(type) << std::endl;
std::cout << " Sign " << ::H5Tget_sign(type) << std::endl;
std::cout << " Order " << ::H5Tget_order(type) << std::endl;
std::cout << " Precision " << ::H5Tget_precision(type) << std::endl;
std::cout << " NDims " << ::H5Tget_array_ndims(type) << std::endl;
std::cout << " NMembers " << ::H5Tget_nmembers(type) << std::endl;
}
int main()
{
h5xx::file xaa("xaa.h5", h5xx::file::mode::in);
// dump(xaa);
{
h5xx::group g(xaa, "particles/lipids/box/positions");
h5xx::dataset ds(g, "value");
std::cout << "dataset: " << std::hex << ds.get_type() << std::dec << std::endl;
diag_type(ds.get_type());
}
{
array_2d_t a(boost::extents[NJ][NI]);
h5xx::datatype detect(a);
std::cout << "detect: " << std::hex << detect.get_type_id() << std::dec << std::endl;
diag_type(detect.get_type_id());
}
}
Prints
dataset: 30000000000013b
Class 1
Size 4
Sign -1
Order 0
Precision 32
NDims -1
NMembers -1
detect: 30000000000000c
Class 0
Size 4
Sign 1
Order 0
Precision 32
NDims -1
NMembers -1
At least we know that HST_FLOAT (class 1) is required. Let's modify array_2d_t:
using array_2d_t = boost::multi_array<float, 2>;
array_2d_t a(boost::extents[11214][3]);
This at least makes the data appear similarly. Let's ... naively try to read:
h5xx::read_dataset(ds, a);
Oops, that predictably throws
terminate called after throwing an instance of 'h5xx::error'
what(): /home/sehe/Projects/stackoverflow/deps/h5xx/h5xx/dataset/boost_multi_array.hpp:176:read_dataset(): dataset "/particles/lipi
ds/box/positions/value" and target array have mismatching dimensions
No worries, we can guess:
using array_3d_t = boost::multi_array<float, 3>;
array_3d_t a(boost::extents[10][11214][3]);
h5xx::read_dataset(ds, a);
At least this does work. Adapting the print function:
template <typename T> void print_array(T const& array) {
for (auto const& row : array) {
for (auto v : row) printf("%5f ", v);
printf("\n");
}
}
Now we can print the first frame:
h5xx::read_dataset(ds, a);
print_array(*a.begin()); // print the first frame
This prints:
80.480003 35.360001 4.250000
37.450001 3.920000 3.960000
18.530001 -9.690000 4.680000
55.389999 74.339996 4.600000
22.110001 68.709999 3.850000
-4.130000 24.040001 3.730000
40.160000 6.390000 4.730000
-5.400000 35.730000 4.850000
36.669998 22.450001 4.080000
-3.680000 -10.660000 4.180000
(...)
That checks out with h5ls -r -d xaa.h5/particles/lipids/box/positions/value:
particles/lipids/box/positions/value Dataset {75/Inf, 11214, 3}
Data:
(0,0,0) 80.48, 35.36, 4.25, 37.45, 3.92, 3.96, 18.53, -9.69, 4.68,
(0,3,0) 55.39, 74.34, 4.6, 22.11, 68.71, 3.85, -4.13, 24.04, 3.73,
(0,6,0) 40.16, 6.39, 4.73, -5.4, 35.73, 4.85, 36.67, 22.45, 4.08, -3.68,
(0,9,1) -10.66, 4.18, 35.95, 36.43, 5.15, 57.17, 3.88, 5.08, -23.64,
(0,12,1) 50.44, 4.32, 6.78, 8.24, 4.36, 21.34, 50.63, 5.21, 16.29,
(0,15,1) -1.34, 5.28, 22.26, 71.25, 5.4, 19.76, 10.38, 5.34, 78.62,
(0,18,1) 11.13, 5.69, 22.14, 59.7, 4.92, 15.65, 47.28, 5.22, 82.41,
(0,21,1) 2.09, 5.24, 16.87, -11.68, 5.35, 15.54, -0.63, 5.2, 81.25,
(...)
The Home Stretch: Adding The Slice
array_2d_t read_frame(int frame_no) {
h5xx::file xaa("xaa.h5", h5xx::file::mode::in);
h5xx::group g(xaa, "particles/lipids/box/positions");
h5xx::dataset ds(g, "value");
array_2d_t a(boost::extents[11214][3]);
std::vector offsets{frame_no, 0, 0}, counts{1, 11214, 3};
h5xx::slice slice(offsets, counts);
h5xx::read_dataset(ds, a, slice);
return a;
}
There you have it. Now we can print any frame:
print_array(read_frame(0));
Printing the same as before. Let's try the last frame:
print_array(read_frame(9));
Prints
79.040001 36.349998 3.990000
37.250000 3.470000 4.140000
18.600000 -9.270000 4.900000
55.669998 75.070000 5.370000
21.920000 67.709999 3.790000
-4.670000 24.770000 3.690000
40.000000 6.060000 5.240000
-5.340000 36.320000 5.410000
36.369999 22.490000 4.130000
-3.520000 -10.430000 4.280000
(...)
Checking again with h5ls -r -d xaa.h5/particles/lipids/box/positions/value |& grep '(9' | head confirms:
(9,0,0) 79.04, 36.35, 3.99, 37.25, 3.47, 4.14, 18.6, -9.27, 4.9, 55.67,
(9,3,1) 75.07, 5.37, 21.92, 67.71, 3.79, -4.67, 24.77, 3.69, 40, 6.06,
(9,6,2) 5.24, -5.34, 36.32, 5.41, 36.37, 22.49, 4.13, -3.52, -10.43,
(9,9,2) 4.28, 35.8, 36.43, 4.99, 56.6, 4.09, 5.04, -23.37, 49.42, 3.81,
(9,13,0) 6.31, 8.83, 4.56, 22.01, 50.38, 5.43, 16.3, -2.92, 5.4, 22.02,
(9,16,1) 70.09, 5.36, 20.23, 11.12, 5.66, 78.48, 11.34, 6.09, 20.26,
(9,19,1) 61.45, 5.35, 14.25, 48.32, 5.35, 79.95, 1.71, 5.38, 17.56,
(9,22,1) -11.61, 5.39, 15.64, -0.19, 5.06, 80.43, 71.77, 5.29, 75.54,
(9,25,1) 35.14, 5.26, 22.45, 56.86, 5.56, 16.47, 52.97, 6.16, 20.62,
(9,28,1) 65.12, 5.26, 19.68, 71.2, 5.52, 23.39, 49.84, 5.28, 22.7,
Full Listing
#include <boost/multi_array.hpp>
#include <h5xx/h5xx.hpp>
#include <iostream>
using array_2d_t = boost::multi_array<float, 2>;
template <typename T> void print_array(T const& array)
{
for (auto const& row : array) { for (auto v : row)
printf("%5f ", v);
printf("\n");
}
}
void dump(h5xx::group const& g, std::string indent = "") {
auto dd = g.datasets();
auto gg = g.groups();
for (auto it = dd.begin(); it != dd.end(); ++it) {
std::cout << indent << " ds:" << it.get_name() << std::endl;
}
for (auto it = gg.begin(); it != gg.end(); ++it) {
dump(*it, indent + "/" + it.get_name());
}
}
array_2d_t read_frame(int frame_no) {
h5xx::file xaa("xaa.h5", h5xx::file::mode::in);
h5xx::group g(xaa, "particles/lipids/box/positions");
h5xx::dataset ds(g, "value");
array_2d_t arr(boost::extents[11214][3]);
std::vector offsets{frame_no, 0, 0}, counts{1, 11214, 3};
h5xx::slice slice(offsets, counts);
h5xx::read_dataset(ds, arr, slice);
return arr;
}
int main()
{
print_array(read_frame(9));
}

Segmentation Fault 11 with Maps and vector c++

I am trying to get a frequency table from a given text.
But the output is showing segmentation fault 11. I don't know why.
I am a newbie. Your help with the code will be very well appreciated.
You can edit my code to make me learn a better method of writing code as well. Thanks a lot.
#include<iostream>
#include<map>
#include<string>
#include<vector>
void make_table(vector<pair<char, int> > &table , string path){
string text = "Hello thusnvkj.ernbuilvgqboipghq3pojavnaj.,fbvlkarebihfg094why091[3tugjvlksbdfv ajklvrpt-30qjhrgiaoehk.BL;H]IH;LGBJSFDNOWI;HBPWRHGB;ORTWIHGOQHRWI0TUGJRLKEWHUGIH49P0-IT302-UR9GM,NXM,BNX,MNMB/E/RGP'KGP34OR[2=O-O-=0-3-1I0-439890375892R0U;L.GNLS.N.SVMS/FS/FKWEP[IF0W))_*(&*(^^&$%#^%$&%*(^*&)(*)_*_(()())))]]'";
map<char, int> m;
for(int i=0; text[i]!='\0'; i++){
m[text[i]]++;
}
map<char,int>::iterator it;
int j=0;
for(it=m.begin();it!=m.end(); it++){
table[j].first=it->first;
table[j].second=it->second;
j++;
//cout << it->first << " " << it->second << endl;
}
return;
}
int main(){
vector<pair<char , int> > table;
string path;
//cin >> path;
path= "Hi";
make_table(table, path);
// make_table function will give us the sorted table(vector of pair) in the decreasing order of frequency of a character.
/*for(int i =0; i<table.size(); i++){
cout << table[i].first << " " << table[i].second << endl;
}
*/
return 0;
}
You get an error because you try to access elements in an empty vector
table[j].first=it->first;
table[j].second=it->second;
Here you can add one line outside the for loop
table.resize(text.size());

Strange behaviour of for_each and push_back()

I was doing some testing with for_each and the use of lambda functions and I'm stuck on this (compiled with g++ -std=c++11, gcc version 5.3.1)
#include <iostream>
#include <vector>
#include <algorithm>
using namespace std;
int main() {
vector<int> vi = {1,1,1,1};
int end =0;
cout << "vi contains: ";
for_each(vi.begin(), vi.end(),[](int i){
cout << i << " ";
});
cout << endl;
for_each(vi.begin(),vi.end(),[&](int i){
cout << "i="<<i<<" ";
if(i==1){
vi.push_back(1);
end++;
}
});
cout << endl;
cout << "end=" << end << endl;
cout << "now vi contains: ";
for_each(vi.begin(), vi.end(),[](int i){
cout << i << " ";
});
cout << endl;
return 0;
}
and this is the output of this code
vi contains: 1 1 1 1
i=1 **i=0** i=1 i=1
end=3
now vi contains: 1 1 1 1 1 1 1
why is, at the first iteration of the loop, i equal to 0?

OpenCV perspectiveTransform broken function

Im trying to use perspectiveTransform but I keep getting error. I tried to follow the solution from this thread http://answers.opencv.org/question/18252/opencv-assertion-failed-for-perspective-transform/
_players[i].getCoordinates() is of type Point
_homography_matrix is a 3 x 3 Mat
Mat temp_Mat = Mat::zeros(2, 1, CV_32FC2);
for (int i = 0; i < _players.size(); i++)
{
cout << Mat(_players[i].get_Coordinates()) << endl;
perspectiveTransform(Mat(_players[i].get_Coordinates()), temp_Mat, _homography_matrix);
}
Also, how do I convert temp_Mat into type Point ?
OpenCV Error: Assertion failed (scn + 1 == m.cols) in cv::perspectiveTransform
Basically you just need to correct from
Mat(_players[i].get_Coordinates()) ...
to
Mat2f(_players[i].get_Coordinates()) ...
In the first case you are creating a 2x1, 1 channel float matrix, in the second case (correct) you create a 1x1, 2 channel float matrix.
You also don't need to initialize temp_Mat.
You can also use template Mat_ to better control the types of your Mats. E.g. creating a Mat of type CV_32FC2 is equivalent to create a Mat2f.
This sample code will show you also how to convert back and forth between Mat and Point:
#include <opencv2\opencv.hpp>
#include <vector>
using namespace std;
using namespace cv;
int main()
{
// Some random points
vector<Point2f> pts = {Point2f(1,2), Point2f(5,10)};
// Some random transform matrix
Mat1f m(3,3, float(0.1));
for (int i = 0; i < pts.size(); ++i)
{
cout << "Point: " << pts[i] << endl;
Mat2f dst;
perspectiveTransform(Mat2f(pts[i]), dst, m);
cout << "Dst mat: " << dst << endl;
Point2f p(dst(0));
cout << "Dst point: " << p << endl;
}
return 0;
}

intrusive_ptr, shared_ptr performance tests

class X {
public:
std::string name;
int age;
long references;
X(string n, int a) : references(0), name(n), age(a) {}
};
inline void intrusive_ptr_add_ref(X* x){
++x->references;
}
inline void intrusive_ptr_release(X* x){
if(--x->references == 0)
delete x;
}
int _tmain(int argc, _TCHAR* argv[])
{
time_t t=clock();
size_t rounds=1000000;
for(size_t i=0; i<rounds; i++)
{
intrusive_ptr<X> myX(new X("Michael",40));
myX->age++;
}
cout << "Time taken to generate " << rounds << " of intrusive_ptr is "
<< clock()-t << endl;
t=clock();
for(size_t i=0; i<rounds; i++)
{
boost::shared_ptr<X> myX(new X("Michael",40));
myX->age++;
}
cout << "Time taken to generate " << rounds << " of shared_ptr is "
<< clock()-t << endl;
t=clock();
for(size_t i=0; i<rounds; i++)
{
std::shared_ptr<X> myX(new X("Michael",40));
myX->age++;
}
cout << "Time taken to generate " << rounds << " of Microsoft shared_ptr is "
<< clock()-t << endl;
t=clock();
for(size_t i=0; i<rounds; i++)
{
boost::shared_ptr<X> myX=boost::make_shared<X>("Michael",40);
myX->age++;
}
cout << "Time taken to generate " << rounds << " of shared_ptr using make_shared is "
<< clock()-t << endl;
t=clock();
for(size_t i=0; i<rounds; i++)
{
std::shared_ptr<X> myX=std::make_shared<X>("Michael",40);
myX->age++;
}
cout << "Time taken to generate " << rounds << " of Microsoft shared_ptr using make_shared is "
<< clock()-t << endl;
_getche();
return 0;
}
I got below results using vs2010 for release mode.
Time taken to generate 1000000 of intrusive_ptr is 116
Time taken to generate 1000000 of shared_ptr is 175
Time taken to generate 1000000 of Microsoft shared_ptr is 182
Time taken to generate 1000000 of shared_ptr using make_shared is 176
Time taken to generate 1000000 of Microsoft shared_ptr using make_shared is 120
Seems intrusive_ptr is the fastest, but seems MS is also doig well with shared_ptr using make_shared function. But why is boost make_shared performing not as well as MS version? Anybody did a similiar test? Anything that is wrong with my test or there is something that I didn't consider?

Resources