Not getting 100% utilization out of CPU when using OpenMP - parallel-processing

I am using OpenMP to parallelize a for loop in my code by adding "#pragma omp parallel for" before the for loop. The code runs without producing any errors, but I have noticed that the core utilization is not constant throughout the execution. At the beginning, I get 100% utilization on all threads, but as the execution continues, this number drops to ~70% for all threads. I monitored this behavior using htop (I am running on a Linux machine). At the beginning, all of the threads show green bars with 100% on the right. After sometimes, the green starts to decrease and some red starts to creep up like this:
I have looked and saw that the red shows up whenever kernel threads are used. However, I still do not understand why it shows up in the first place. This behavior only occurs when the for loop is parallelized. When I take out the #pragma statement, the code runs at 100% throughout. My for loop has the following format:
double xe1, ye1, ze1, te1, e1;
double xe2, ye2, ze2, te2, e2;
double xi1, yi1, zi1, ti1;
double xi2, yi2, zi2, ti2;
bool calculate_signal = true;
time_t loop_start, loop_end;
int max = omp_get_max_threads();
omp_set_dynamic(0);
omp_set_num_threads(max);
#pragma omp parallel for
for (int k = 0; k < index; k++) {
time(&loop_start);
AvalancheMicroscopic aval;
aval.SetSensor(&sensor);
aval.EnableSignalCalculation(calculate_signal);
aval.AvalancheElectron(electrons.at(k*4 + 0),electrons.at(k*4 + 1),electrons.at(k*4 + 2),electrons.at(k*4 + 3), 0.1, 0,0,0);
int np = aval.GetNumberOfElectronEndpoints();
DriftLineRKF drift;
drift.SetSensor(&sensor);
drift.EnableSignalCalculation(calculate_signal);
for (int j = np; j--;) {
aval.GetElectronEndpoint(j, xe1, ye1, ze1, te1, e1,
xe2, ye2, ze2, te2, e2, status);
drift.DriftIon(xe1, ye1, ze1, te1);
}
time(&loop_end);
std::cout << "Time for " << np << " electrons: " << double(loop_end - loop_start) << std::endl;
}
This code is written using a toolkit called Garfield++ (https://garfieldpp.web.cern.ch/garfieldpp/; https://gitlab.cern.ch/garfield/garfieldpp/-/tree/master/). Here, electrons() is a 1d vector that holds coordinates. The index variable that can range from 1 to 7000 depending on the user. I have ran with index = 1000 and index = 200 and gotten the same behavior. Also, np can range from 1 to ~2000 and is something that is determined by the outer loop, so user has no control over this.
I have tried schedule(dynamic), but that didn't seem to help. Any insight into what caused this kind of behavior and how to fix it is greatly appreciated. Thank you.
Edit 1: The code is edited to provide more details.
Edit 2: I ran the code again to find the average time it takes for each loop. The answer, as expected, is not very straightforward (at least not for me). The code were run twice: one without parallelization (#pragma statement is commented out) and one with parallelization. Both executions were run by using the same electrons vector. Below, I have printed the time (in unit of seconds) each loop takes. The amount of time it takes to finished the inner loop is on the right side of the semicolon, while the number on the left of the semicolon np. I calculated the time by using time_t. I edited the above code to show what I did. Also, another thing to note is that the first case (w/o parallel) took 6065 sec to run while the second case took 2183 sec.
Using 1 core Using 10 cores
Time for 1 electrons: 7 Time for 1 electrons: 11
Time for 1565 electrons: 91 Time for 1 electrons: 0
Time for 1 electrons: 8 Time for 1 electrons: 10
Time for 1 electrons: 7 Time for 92 electrons: 12
Time for 3229 electrons: 162 Time for 1 electrons: 12
Time for 1 electrons: 7 Time for 1 electrons: 12
Time for 1 electrons: 7 Time for 397 electrons: 21
Time for 1 electrons: 6 Time for 563 electrons: 41
Time for 1 electrons: 8 Time for 902 electrons: 79
Time for 1 electrons: 7 Time for 572 electrons: 18
Time for 1 electrons: 7 Time for 1 electrons: 14
Time for 2028 electrons: 103 Time for 1207 electrons: 80
Time for 898 electrons: 57 Time for 391 electrons: 4
Time for 1 electrons: 7 Time for 1392 electrons: 1
Time for 1 electrons: 8 Time for 297 electrons: 9
Time for 1 electrons: 7 Time for 1439 electrons: 39
Time for 1 electrons: 8 Time for 1 electrons: 14
Time for 1739 electrons: 91 Time for 1 electrons: 13
Time for 1977 electrons: 99 Time for 1 electrons: 14
Time for 3053 electrons: 152 Time for 393 electrons: 21
Time for 166 electrons: 20 Time for 543 electrons: 31
Time for 1 electrons: 8 Time for 1767 electrons: 38
Time for 1 electrons: 7 Time for 744 electrons: 5
Time for 573 electrons: 39 Time for 436 electrons: 44
Time for 31 electrons: 16 Time for 2444 electrons: 19
Time for 774 electrons: 51 Time for 2469 electrons: 4
Time for 680 electrons: 44 Time for 368 electrons: 30
Time for 514 electrons: 36 Time for 895 electrons: 29
Time for 1281 electrons: 69 Time for 2223 electrons: 6
Time for 2847 electrons: 148 Time for 1 electrons: 8
Time for 1004 electrons: 57 Time for 1394 electrons: 9
Time for 1422 electrons: 76 Time for 1 electrons: 4
Time for 762 electrons: 48 Time for 3283 electrons: 9
Time for 1817 electrons: 96 Time for 1 electrons: 3
Time for 2133 electrons: 110 Time for 1 electrons: 11
Time for 895 electrons: 52 Time for 1010 electrons: 5
Time for 939 electrons: 55 Time for 1 electrons: 12
Time for 743 electrons: 44 Time for 484 electrons: 5
Time for 1121 electrons: 63 Time for 112 electrons: 19
Time for 2482 electrons: 126 Time for 733 electrons: 0
Time for 1449 electrons: 79 Time for 1 electrons: 14
Time for 2037 electrons: 106 Time for 143 electrons: 9
Time for 4227 electrons: 204 Time for 1 electrons: 11
Time for 611 electrons: 39 Time for 394 electrons: 36
Time for 738 electrons: 44 Time for 1489 electrons: 3
Time for 1416 electrons: 75 Time for 540 electrons: 66
Time for 519 electrons: 39 Time for 1 electrons: 13
Time for 676 electrons: 45 Time for 1835 electrons: 27
Time for 497 electrons: 36 Time for 837 electrons: 5
Time for 303 electrons: 27 Time for 559 electrons: 18
Time for 2695 electrons: 144 Time for 828 electrons: 8
Time for 120 electrons: 18 Time for 811 electrons: 28
Time for 142 electrons: 19 Time for 136 electrons: 49
Time for 809 electrons: 48 Time for 1881 electrons: 9
Time for 27 electrons: 14 Time for 1 electrons: 12
Time for 343 electrons: 26 Time for 816 electrons: 13
Time for 168 electrons: 19 Time for 651 electrons: 2
Time for 78 electrons: 15 Time for 1 electrons: 11
Time for 883 electrons: 50 Time for 878 electrons: 18
Time for 563 electrons: 36 Time for 361 electrons: 14
Time for 418 electrons: 30 Time for 2581 electrons: 26
Time for 808 electrons: 46 Time for 155 electrons: 10
Time for 1977 electrons: 96 Time for 518 electrons: 50
Time for 610 electrons: 38 Time for 1241 electrons: 8
Time for 326 electrons: 26 Time for 722 electrons: 4
Time for 84 electrons: 16 Time for 223 electrons: 6
Time for 3116 electrons: 145 Time for 362 electrons: 10
Time for 979 electrons: 53 Time for 865 electrons: 15
Time for 259 electrons: 24 Time for 1 electrons: 10
Time for 761 electrons: 44 Time for 791 electrons: 3
Time for 2690 electrons: 128 Time for 1 electrons: 11
Time for 795 electrons: 48 Time for 1 electrons: 10
Time for 1 electrons: 8 Time for 826 electrons: 83
Time for 1762 electrons: 80 Time for 2428 electrons: 2
Time for 2410 electrons: 124 Time for 1159 electrons: 17
Time for 454 electrons: 33 Time for 149 electrons: 10
Time for 1 electrons: 7 Time for 1355 electrons: 4
Time for 2912 electrons: 146 Time for 457 electrons: 25
Time for 1624 electrons: 85 Time for 523 electrons: 26
Time for 857 electrons: 54 Time for 561 electrons: 16
Time for 4517 electrons: 221 Time for 235 electrons: 43
Time for 2083 electrons: 113 Time for 708 electrons: 4
Time for 1268 electrons: 75 Time for 1388 electrons: 51
Time for 7548 electrons: 358 Time for 369 electrons: 3
Time for 557 electrons: 40 Time for 403 electrons: 7
Time for 1 electrons: 7 Time for 1234 electrons: 13
Time for 1 electrons: 7 Time for 3203 electrons: 13
Time for 1 electrons: 8 Time for 262 electrons: 27
Time for 1167 electrons: 65 Time for 444 electrons: 43
Time for 1410 electrons: 77 Time for 686 electrons: 7
Time for 1 electrons: 7 Time for 7654 electrons: 26
Time for 4490 electrons: 412 Time for 731 electrons: 23
Time for 3093 electrons: 146 Time for 1336 electrons: 77
Time for 1170 electrons: 67 Time for 2117 electrons: 111
Time for 1 electrons: 7 Time for 1750 electrons: 96
Time for 1 electrons: 8 Time for 1089 electrons: 58
Time for 1 electrons: 7 Time for 3823 electrons: 185
Time for 1 electrons: 7 Time for 996 electrons: 54
Time for 713 electrons: 45 Time for 1 electrons: 7
Time for 973 electrons: 57 Time for 1 electrons: 8

Related

cumsum with more than 1 variable using ddply

I'm trying to get cumsum for more than one variable using ddply, but it's not working.
I'm using this code:
ddply(.data=Summaryday, .variables=('DaysToClose_'),.fun=transform,
cumsumPosit=cumsum(PositCount),
cumsumNegat=cumsum(NegatCount))
but the result isn't correct:
DaysToClose_ PositCount NegatCount cumsumPosit cumsumNegat
1 1 7340 27256 7340 27256
2 2 2243 7597 2243 7597
3 3 1526 4545 1526 4545
4 4 1315 3756 1315 3756
5 5 1142 3320 1142 3320
6 6 1216 3118 1216 3118
7 7 1252 3324 1252 3324
8 8 1180 3077 1180 3077
9 9 975 2053 975 2053
10 10 684 1429 684 1429
11 11 613 1244 613 1244
12 12 596 1199 596 1199
13 13 542 1218 542 1218
14 14 711 1434 711 1434
15 15 645 1333 645 1333
16 16 577 899 577 899
17 17 373 667 373 667
18 18 369 656 369 656
19 19 340 624 340 624
If someone can help me on this, I appreciate that.
I am not sure why you would use ddply here. You can't really subset by DaysToClose because each row is then a unique subset, and so you always get cumsum of a single value. Maybe you'd want to use mutate instead
library(tidyverse)
data %>% mutate(cumsumPosit = cumsum(PositCount),
cumsumNegat = cumsum(NegatCount))

Keep lines based on ratio between lines

I have a sort -g k9 command on a file that gives me this in the bash standard output:
55.19 645 156 15 9 520 58 702 0.0 661
55.50 636 159 16 9 520 58 693 0.0 654
55.19 645 156 15 9 520 58 702 0.0 658
56.52 644 147 16 9 520 59 701 0.0 669
55.97 645 151 15 9 520 65 709 0.0 672
55.97 645 151 15 9 520 65 709 4e-124 674
28.32 671 301 32 1 507 48 702 3e-49 183
28.32 671 301 32 1 507 47 701 3e-49 183
31.40 516 247 24 86 507 196 698 1e-46 176
31.41 519 243 25 86 507 196 698 5e-46 175
27.72 588 290 26 19 481 98 675 2e-39 154
30.56 337 170 17 101 413 302 598 5e-20 96.3
30.56 337 170 17 101 413 302 598 8e-20 95.5
I would like to cut my data based on the 9th column. The idea would be to compare the value of the 9th column on line i, divide it by the value of the 9th column on line i+1, and if the ratio is 0 OR 0/0 OR > 1e-50, line i and i+1 are kept. As soon as one of these conditions is not filled, stop reading. The desired output would be:
55.19 645 156 15 9 520 58 702 0.0 661
55.50 636 159 16 9 520 58 693 0.0 654
55.19 645 156 15 9 520 58 702 0.0 658
56.52 644 147 16 9 520 59 701 0.0 669
55.97 645 151 15 9 520 65 709 0.0 672
55.97 645 151 15 9 520 65 709 4e-124 674
I can obtain this output with head -n 6 but this is obviously not based on the condition on values in the 9th column. Please note that the values are in 'scientific' format.
I know how to do this in Python (write the standard output to a file, calculate ratios, etc.) but for commodity reasons I'd prefer a shell-based solution (with awk or sort for instance) although I don't know if that's possible. Thanks for your help!
Just exit the script when the condition is not accomplished; otherwise, print the previous line and store the 9th field to compare on the next loop:
$ awk '($9 && prev/$9>1e-50) {exit} {print stored; prev=$9; stored=$0}' file
55.19 645 156 15 9 520 58 702 0.0 661
55.50 636 159 16 9 520 58 693 0.0 654
55.19 645 156 15 9 520 58 702 0.0 658
56.52 644 147 16 9 520 59 701 0.0 669
55.97 645 151 15 9 520 65 709 0.0 672
55.97 645 151 15 9 520 65 709 4e-124 674

Nurse scheduling model formulation in AMPL

I have been working on a Nurse scheduling problem in AMPL for the following conditions:
Total no. of Nurses=20
Total no. of shits= 3 #morning,day,night
Planning Horizon 7 days: let's say M T W R F Sa Su
Along with following constraints:
Max no. of working days in a week: 5
A rest days after 4 continuous
night shifts.
Consecutive night and morning shifts are not allowed.
Demand per shift is 7 nurses.
A nurse can only work in one shift per day, i.e. morning, night, day
Cost scenarios:
Morning shift: $12
Day shift: $13
Night shift : $15
Objective function is to minimize the cost of operation as per Nurse preferences.
Can anyone give me an idea of how this problem can be formulated ?
So at first some things unusual in your problem definition:
This is not a real optimization problem, since your objective function is fixed per definition (every shift has 7 nurses, and every nurse has an equal price per shift)
In your Problem you defined 7 nurses per shift with a maimum of 5 working days. So you need 7 nurses on three shifts on seven days. This equals 147 nurse/shifts. But with the cap of five working days and only one shift per day, you just have 20 Nurses on 5 shifts, which equals to 100 nurse/shifts.
I've built the problem in Mathprog but the code should be more or less equal to AMPL. I've started with three sets for the nurses, days and shifts.
set shifts := {1,2,3};
set days := {1,2,3,4,5,6,7};
set nurses := {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20};
The shedule is defined as a set of binary variables:
var schedule{nurses, days, shifts}, binary;
The simple objective contains the sum of all nurse/shifts in this week with the related prices:
minimize cost: sum{i in nurses, j in days}(schedule[i,j,1]*c_morning+schedule[i,j,2]*c_day+schedule[i,j,3]*c_night);
To your first constraint one can limit the sum of all shifts per nurse to five, since there is only one shift per day possible:
s.t. working_days{n in nurses}:
sum{i in days, j in shifts}(schedule[n,i,j]) <= 5;
The restday is the hardest part of the problem. For simplicity I've created another set which just contains the days, where a nurse could have achived four night-shifts in a row. You can also formulate the constraint with the original set of days and exclude the first four days.
set nigth_days := {5,6,7};
s.t. rest{n in nurses,i in nigth_days}:
(schedule[n,i-4,3]+schedule[n,i-3,3]+schedule[n,i-2,3]+schedule[n,i-1,3]+sum{j in shifts}(schedule[n,i,j])) <= 4;
For not having a morning-shift after a night-shift I used the same attempt like for the rest days. The seventh day is excluded, since there is no eigth day where we can look for a morning-shift.
set yester_days := {1,2,3,4,5,6};
s.t. night_morning{i in yester_days, n in nurses}:
(schedule[n,i,3]+schedule[n,i+1,1]) <= 1;
The demand of four nurses per shift should be met (I've reduced the number since more then 4 nurses are infeasible, due to the 5 shift limit)
s.t. demand_shift{i in days, j in shifts}:
sum{n in nurses}(schedule[n,i,j]) = 4;
The fifth constraint is to limit the shifts per day to a max of one.
s.t. one_shift{n in nurses, i in days}:
sum{ j in shifts}(schedule[n,i,j]) <= 1;
set nurse; #no. of full time employees working in the facility
set days; #planning horizon
set shift; #no. of shift in a day
set S; #shift correseponding to the outsourced nurses
set D;#day corresponding to the outsourced nurses
set N;#
# ith nurse working on day j
# j starts from Monday (j=1), Tuesday( j=2), Wednesday (j=3), Thursday(j=4), Friday(j=5), Saturday(j=6), Sunday(j=7)
#s be the shift as morning, day and night
param availability{i in nurse, j in days};
param costpershift{i in nurse, j in days, s in shift};
param outcost{n in N, l in D, m in S};
var nurseavailability{i in nurse,j in days,s in shift} binary; # = 1 if nurse i is available on jth day working on sth shift, 0 otherwise
var outsourced{n in N, l in D, m in S} integer;
#Objective function
minimize Cost: sum{i in nurse, j in days, s in shift} costpershift[i,j,s]*nurseavailability[i,j,s]+ sum{ n in N, l in D, m in S}outcost[n,l,m]*outsourced[n,l,m];
#constraints
#maximum no. of shifts per day
subject to maximum_shifts_perday {i in nurse,j in days}:
sum{s in shift} nurseavailability[i,j,s]*availability[i,j] <= 1;
#maximum no. of working says a week
subject to maximum_days_of_work {i in nurse}:
sum{j in days,s in shift} availability[i,j]*nurseavailability[i,j,s]<=5; #maximum working days irrespective of shifts
# rest days after night shifts
subject to rest_days_after_night_shift{i in nurse}:
sum{j in days} availability[i,j]*nurseavailability[i,j,3]<=4;
#demand per shift
subject to supply{j in days, s in shift, l in D, m in S}:
sum{i in nurse} availability[i,j]*nurseavailability[i,j,s] + sum{n in N} outsourced[n,l,m]=7;
#outsourcing only works well when there is more variability in supply.
#increasing the staff no. would be effective for reducing the cost variability in demand.
#considering a budget of $16,000 per week
#outsourcing constraints: a maximum of 20 nurses can be outsourced per shift
# no. of fulltime employees=30
#demand is 7 nurses per shift
#the average variability
#all nurses are paid equally # $12 per hour.
#cost of an outsourced shift is $144.
#cost of morning shift is $96.
#cost of day shift is $104.
#cost of night shift is $120.
data;
#set nurse ordered:= nurse1 nurse2 nurse3 nurse4 nurse5 nurse6 nurse7 nurse8
#nurse9 nurse10 nurse11 nurse12 nurse13 nurse14 nurse15 nurse16 nurse17
#nurse18 nurse19 nurse20;
set nurse:= 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30;
#set days ordered:= Monday Tuesday Wednesday Thursday Friday Saturday Sunday;
set days:= 1 2 3 4 5 6 7;
#set shift ordered:= Morning Day Night;
set shift:= 1 2 3;
set D:= 1 2 3 4 5 6 7; #outsourced days
set S:=1 2 3; #outshit
set N := 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20;
param outcost
[*,*,1]:
1 2 3 4 5 6 7:=
1 144 144 144 144 144 144 144
2 144 144 144 144 144 144 144
3 144 144 144 144 144 144 144
4 144 144 144 144 144 144 144
5 144 144 144 144 144 144 144
6 144 144 144 144 144 144 144
7 144 144 144 144 144 144 144
8 144 144 144 144 144 144 144
9 144 144 144 144 144 144 144
10 144 144 144 144 144 144 144
11 144 144 144 144 144 144 144
12 144 144 144 144 144 144 144
13 144 144 144 144 144 144 144
14 144 144 144 144 144 144 144
15 144 144 144 144 144 144 144
16 144 144 144 144 144 144 144
17 144 144 144 144 144 144 144
18 144 144 144 144 144 144 144
19 144 144 144 144 144 144 144
20 144 144 144 144 144 144 144
[*,*,2]:
1 2 3 4 5 6 7:=
1 144 144 144 144 144 144 144
2 144 144 144 144 144 144 144
3 144 144 144 144 144 144 144
4 144 144 144 144 144 144 144
5 144 144 144 144 144 144 144
6 144 144 144 144 144 144 144
7 144 144 144 144 144 144 144
8 144 144 144 144 144 144 144
9 144 144 144 144 144 144 144
10 144 144 144 144 144 144 144
11 144 144 144 144 144 144 144
12 144 144 144 144 144 144 144
13 144 144 144 144 144 144 144
14 144 144 144 144 144 144 144
15 144 144 144 144 144 144 144
16 144 144 144 144 144 144 144
17 144 144 144 144 144 144 144
18 144 144 144 144 144 144 144
19 144 144 144 144 144 144 144
20 144 144 144 144 144 144 144
[*,*,3]:
1 2 3 4 5 6 7:=
1 144 144 144 144 144 144 144
2 144 144 144 144 144 144 144
3 144 144 144 144 144 144 144
4 144 144 144 144 144 144 144
5 144 144 144 144 144 144 144
6 144 144 144 144 144 144 144
7 144 144 144 144 144 144 144
8 144 144 144 144 144 144 144
9 144 144 144 144 144 144 144
10 144 144 144 144 144 144 144
11 144 144 144 144 144 144 144
12 144 144 144 144 144 144 144
13 144 144 144 144 144 144 144
14 144 144 144 144 144 144 144
15 144 144 144 144 144 144 144
16 144 144 144 144 144 144 144
17 144 144 144 144 144 144 144
18 144 144 144 144 144 144 144
19 144 144 144 144 144 144 144
20 144 144 144 144 144 144 144;
param availability:
1 2 3 4 5 6 7 :=
1 0 0 0 0 0 0 0
2 1 1 1 1 1 1 1
3 1 1 1 1 1 1 1
4 1 1 1 1 1 1 1
5 1 1 1 1 1 1 1
6 1 1 1 1 1 1 1
7 1 0 1 1 1 1 1
8 1 1 1 1 1 1 1
9 1 1 1 1 1 1 1
10 1 1 1 1 1 1 1
11 1 1 1 1 1 1 1
12 1 1 1 1 1 1 1
13 1 1 1 1 1 1 1
14 1 1 1 1 1 1 1
15 1 1 1 1 1 1 1
16 1 1 1 1 1 1 1
17 0 1 1 1 1 1 1
18 1 1 1 1 1 1 1
19 1 1 1 1 1 1 1
20 1 1 1 1 1 1 1
21 1 1 1 1 1 1 1
22 1 1 1 1 1 1 1
23 1 1 1 1 1 1 1
24 1 1 1 1 1 1 1
25 1 1 1 1 1 1 1
26 1 1 1 1 1 1 1
27 1 1 1 1 1 1 1
28 1 1 1 1 1 1 1
29 1 1 1 1 1 1 1
30 1 1 1 1 1 1 1;
param costpershift:=
[*,*,1]: 1 2 3 4 5 6 7 :=
1 96 96 96 96 96 96 96
2 96 96 96 96 96 96 96
3 96 96 96 96 96 96 96
4 96 96 96 96 96 96 96
5 96 96 96 96 96 96 96
6 96 96 96 96 96 96 96
7 96 96 96 96 96 96 96
8 96 96 96 96 96 96 96
9 96 96 96 96 96 96 96
10 96 96 96 96 96 96 96
11 96 96 96 96 96 96 96
12 96 96 96 96 96 96 96
13 96 96 96 96 96 96 96
14 96 96 96 96 96 96 96
15 96 96 96 96 96 96 96
16 96 96 96 96 96 96 96
17 96 96 96 96 96 96 96
18 96 96 96 96 96 96 96
19 96 96 96 96 96 96 96
20 96 96 96 96 96 96 96
21 96 96 96 96 96 96 96
22 96 96 96 96 96 96 96
23 96 96 96 96 96 96 96
24 96 96 96 96 96 96 96
25 96 96 96 96 96 96 96
26 96 96 96 96 96 96 96
27 96 96 96 96 96 96 96
28 96 96 96 96 96 96 96
29 96 96 96 96 96 96 96
30 96 96 96 96 96 96 96
[*,*,2] : 1 2 3 4 5 6 7 :=
1 104 104 104 104 104 104 104
2 104 104 104 104 104 104 104
3 104 104 104 104 104 104 104
4 104 104 104 104 104 104 104
5 104 104 104 104 104 104 104
6 104 104 104 104 104 104 104
7 104 104 104 104 104 104 104
8 104 104 104 104 104 104 104
9 104 104 104 104 104 104 104
10 104 104 104 104 104 104 104
11 104 104 104 104 104 104 104
12 104 104 104 104 104 104 104
13 104 104 104 104 104 104 104
14 104 104 104 104 104 104 104
15 104 104 104 104 104 104 104
16 104 104 104 104 104 104 104
17 104 104 104 104 104 104 104
18 104 104 104 104 104 104 104
19 104 104 104 104 104 104 104
20 104 104 104 104 104 104 104
21 104 104 104 104 104 104 104
22 104 104 104 104 104 104 104
23 104 104 104 104 104 104 104
24 104 104 104 104 104 104 104
25 104 104 104 104 104 104 104
26 104 104 104 104 104 104 104
27 104 104 104 104 104 104 104
28 104 104 104 104 104 104 104
29 104 104 104 104 104 104 104
30 104 104 104 104 104 104 104
[*,*,3] : 1 2 3 4 5 6 7 :=
1 120 120 120 120 120 120 120
2 120 120 120 120 120 120 120
3 120 120 120 120 120 120 120
4 120 120 120 120 120 120 120
5 120 120 120 120 120 120 120
6 120 120 120 120 120 120 120
7 120 120 120 120 120 120 120
8 120 120 120 120 120 120 120
9 120 120 120 120 120 120 120
10 120 120 120 120 120 120 120
11 120 120 120 120 120 120 120
12 120 120 120 120 120 120 120
13 120 120 120 120 120 120 120
14 120 120 120 120 120 120 120
15 120 120 120 120 120 120 120
16 120 120 120 120 120 120 120
17 120 120 120 120 120 120 120
18 120 120 120 120 120 120 120
19 120 120 120 120 120 120 120
20 120 120 120 120 120 120 120
21 120 120 120 120 120 120 120
22 120 120 120 120 120 120 120
23 120 120 120 120 120 120 120
24 120 120 120 120 120 120 120
25 120 120 120 120 120 120 120
26 120 120 120 120 120 120 120
27 120 120 120 120 120 120 120
28 120 120 120 120 120 120 120
29 120 120 120 120 120 120 120
30 120 120 120 120 120 120 120;

GCD and AppleEvents / NSOperationQueue and AppleScript

As i understood, the threads provided by GCD do have a runloop but no source/port. Now i use some of methods that call AppleScripts thru AppleEvents inside an NSOperationQueue. And sometimes my app crashes with following stacktrace.
my questions:
Usage of AppleScript inside a NSInvocationOperation or NSBlockOperation
Usage of AppleEvents inside GCD-Threads
do i have to add a source / port to a runloop of a gcd-thread before using AppleEvents?
do i have to add a source / port to a runloop of a gcd-thread before using [CATransaction commit]?
do i have to add a source / port to a runloop of a gcd-thread before using [NSObject performSelectorAfterDelay...] ?
if yes, how?
can i simply call CFRunLoopRunInMode(kCFRunLoopDefaultMode, 5.0, false); /// or 30 or the defined timeout of appleevent?
..
Thread 9: Dispatch queue: com.apple.root.default-priority
0 libSystem.B.dylib 0x00007fff88276e82 semaphore_wait_signal_trap 10
1 libSystem.B.dylib 0x00007fff8827c3cd pthread_mutex_lock 469
2 com.apple.applescript 0x000000011d3d461f AppleScriptComponent 50
3 com.apple.applescript 0x000000011d3edbcc AGenericCall::Delegate(ComponentInstanceRecord*) 46
4 com.apple.applescript 0x000000011d3ed520 AGenericManager::HandleOSACall(ComponentParameters*) 54
5 com.apple.applescript 0x000000011d3ed4b0 GenericComponent 219
6 com.apple.openscripting 0x00007fff8381c6da OSAExecuteEvent 63
7 com.apple.Foundation 0x00007fff86f320d8 -[NSAppleScript(NSPrivate) _executeAppleEvent:withMode:error:] 161
8 xxx 0x0000000100048af0 -[TCallScript callScript:withArrayOfParameters:] 480
9 xxx 0x0000000100048ffd -[TCallScript callHandler:withParameters:] 477
10 xxx 0x0000000100036032 -[ZFOpenWindowController getSafariItemForCurrentTabForWindow:] 66
11 xxx 0x00000001000346af -[ZFOpenWindowController refreshWindowList:] 1679
12 xxx 0x000000010003541c -[ZFOpenWindowController bringZFToForegroundZoomOut:orJustLinkFrontMost:toItem:] 2988
13 xxx 0x0000000100053556 __-[ZFSelectionTool openWithPreferredApplication:]_block_invoke_516 54
14 com.apple.Foundation 0x00007fff86ed87d9 -[NSBlockOperation main] 140
15 com.apple.Foundation 0x00007fff86ec906d -[__NSOperationInternal start] 681
16 com.apple.Foundation 0x00007fff86ec8d23 ____startOperations_block_invoke_2 99
17 libSystem.B.dylib 0x00007fff882b2ce8 _dispatch_call_block_and_release 15
18 libSystem.B.dylib 0x00007fff88291279 _dispatch_worker_thread2 231
19 libSystem.B.dylib 0x00007fff88290bb8 _pthread_wqthread 353
20 libSystem.B.dylib 0x00007fff88290a55 start_wqthread 13
...
Thread 11 Crashed: Dispatch queue: com.apple.root.default-priority
0 com.apple.applescript 0x000000011d40f658 BCHandleError() 182
1 com.apple.applescript 0x000000011d4059ca UASExecute1() 2546
2 com.apple.applescript 0x000000011d3dd1d5 ASExecuteEvent(AEDesc const*, unsigned int, int, unsigned int*) 695
3 ...ple.CoreServices.CarbonCore 0x00007fff85388e04 CallComponentFunction 28
4 com.apple.applescript 0x000000011d3d4cad AppleScriptComponent 1728
5 com.apple.applescript 0x000000011d3edbcc AGenericCall::Delegate(ComponentInstanceRecord*) 46
6 com.apple.applescript 0x000000011d3ed520 AGenericManager::HandleOSACall(ComponentParameters*) 54
7 com.apple.applescript 0x000000011d3ed4b0 GenericComponent 219
8 com.apple.openscripting 0x00007fff8381c6da OSAExecuteEvent 63
9 com.apple.Foundation 0x00007fff86f320d8 -[NSAppleScript(NSPrivate) _executeAppleEvent:withMode:error:] 161
10 xxx 0x0000000100048af0 -[TCallScript callScript:withArrayOfParameters:] 480
11 xxx 0x0000000100048ffd -[TCallScript callHandler:withParameters:] 477
12 xxx 0x0000000100035c67 -[ZFOpenWindowController getItemForAXDocumentOfFrontMostWindow] 119
13 xxx 0x00000001000359f1 -[ZFOpenWindowController getItemForFrontMostWindow:] 721
14 xxx 0x00000001000348ac -[ZFOpenWindowController bringZFToForegroundZoomOut:orJustLinkFrontMost:toItem:] 60
15 xxx 0x0000000100053556 __-[ZFSelectionTool openWithPreferredApplication:]_block_invoke_516 54
16 com.apple.Foundation 0x00007fff86ed87d9 -[NSBlockOperation main] 140
17 com.apple.Foundation 0x00007fff86ec906d -[__NSOperationInternal start] 681
18 com.apple.Foundation 0x00007fff86ec8d23 ____startOperations_block_invoke_2 99
19 libSystem.B.dylib 0x00007fff882b2ce8 _dispatch_call_block_and_release 15
20 libSystem.B.dylib 0x00007fff88291279 _dispatch_worker_thread2 231
21 libSystem.B.dylib 0x00007fff88290bb8 _pthread_wqthread 353
22 libSystem.B.dylib 0x00007fff88290a55 start_wqthread 13
Note that the AppleScript component has limited thread safety. As of 10.6, you can use it on non-main threads, but you will need to create a new ComponentInstance for each thread. I think you can do this with OSAKit, otherwise you'll need to resort to the gnarly Carbon API. What's best really depends on what you're trying to achieve (e.g. are you running user-supplied scripts, or scripts hardcoded into your application, and what's the motivation for running them via NSOperationQueue).

data.frame to spatial polygone data frame

I have this data.frame
data <- read.table(text="Id x y valecolo valecono
1 1 12.18255221 29.406365240 4 990
2 2 9.05893970 20.923087170 4 1090
3 3 1.11192442 2.460411416 0 420
4 4 15.51290096 27.185287490 16 1320
5 5 20.41913438 32.166268590 13 1050
6 6 12.75939095 17.552435030 60 1010
7 7 28.06853355 30.839057830 12 1030
8 8 6.96288868 7.177616682 33 1010
9 9 30.60527190 20.792242110 23 640
10 10 12.07646283 7.658266843 19 810
11 11 10.42878294 5.520913954 0 700
12 12 23.61674977 11.111217320 0 838
13 13 27.16148898 12.259423750 11 1330
14 14 28.00931750 6.258448426 20 777
15 15 20.79999922 -0.000877298 4 630
16 16 21.59999968 -0.005502197 38 830
17 17 19.46122172 -1.229166015 7 740
18 18 28.20370719 -6.305622777 12 660
19 19 29.94840042 -7.192584050 0 1030
20 20 29.28601258 -12.133404940 10 870
21 21 5.88104817 -3.608777319 0 1050
22 22 30.37845976 -26.784308510 0 900
23 23 13.68270042 -12.451253320 0 300
24 24 26.01871530 -26.024342420 22 1330
25 25 20.17735764 -20.829648070 21 1190
26 26 5.04404016 -5.550464740 7 1030
27 27 17.98312114 -26.468988540 0 1200
28 28 8.50660753 -12.957145840 9 850
29 29 10.79633248 -18.938827100 36 1200
30 30 13.36599497 -28.413203870 7 1240
31 31 10.77987946 -28.531459810 0 350
32 32 8.35194396 -24.410755680 28 910
33 33 1.55014408 -12.302725060 10 980
34 34 -0.00388992 -17.899999200 12 1120
35 35 -2.82062504 -16.155620130 12 450
36 36 -4.75903628 -22.962014490 20 920
37 37 -6.07839546 -15.339592840 28 840
38 38 -11.32647798 -24.068047630 0 665
39 39 -11.88138209 -24.245262620 12 1180
40 40 -14.06823800 -25.587589260 36 350
41 41 -10.92180227 -18.461223360 7 1180
42 42 -12.48843186 -20.377660600 0 400
43 43 -18.63696964 -27.415068190 18 1220
44 44 -16.73351789 -23.807549250 0 500
45 45 -22.49024869 -29.944803740 7 1040
46 46 -22.66130064 -27.391018580 0 500
47 47 -15.26565038 -17.866446720 16 1060
48 48 -24.20192852 -23.451155780 0 600
49 49 -21.39663774 -20.089958090 0 750
50 50 -12.33344998 -9.875526199 16 980
51 51 -30.94772590 -22.478895910 0 790
52 52 -24.85783868 -15.225318840 25 720
53 53 -2.44485324 -1.145728097 54 970
54 54 -24.67985433 -7.169018707 4 500
55 55 -30.82457650 -7.398346555 4 750
56 56 -23.56898920 -5.265475270 4 760
57 57 -3.91708603 -0.810208045 0 350
58 58 -26.86563675 -4.251776497 0 440
59 59 -26.64738877 -1.675324623 8 450
60 60 -8.79897138 -0.134558536 11 830
61 61 -21.78250663 1.716077388 0 920
62 62 -28.98396759 6.007465815 24 980
63 63 -34.61607994 8.311853049 8 500
64 64 -25.63850107 7.453677191 15 880
65 65 -22.98762116 11.266290120 11 830
66 66 -33.48522130 19.100848030 0 350
67 67 -25.53096486 16.777135830 21 740
68 68 -18.95412327 15.681238150 0 300
69 69 -8.94874230 8.144324435 0 500
70 70 -10.91433241 10.579099310 4 750
71 71 -13.44807236 14.327310800 0 1090
72 72 -16.24086139 20.940019610 0 500
73 73 -17.51162097 24.111886810 0 940
74 74 -12.47496424 18.363422910 0 1020
75 75 -17.76118016 27.990410510 0 660
76 76 -5.54534556 9.730834410 0 850
77 77 -11.30971858 29.934766840 0 950
78 78 -10.38743785 27.493148220 0 740
79 79 -8.61491396 25.166312360 0 950
80 80 -3.40550077 14.197273530 0 710
81 81 -0.77957621 3.770246702 0 750
82 82 -3.01234325 21.186924550 0 1200
83 83 -2.05241931 32.685624900 0 1200
84 84 -2.26900366 36.128820600 0 970
85 85 0.82954518 5.790885396 0 850
86 86 22.08151130 19.671119440 19 870
87 87 12.60107972 23.864904860 0 1260
88 88 9.78406607 26.163968270 0 600
89 89 11.69995152 33.091322170 0 1090
90 90 20.64705880 -16.439632140 0 840
91 91 24.68314851 -21.314655730 0 1561
92 92 30.33133300 -27.235396100 0 1117
93 93 -26.24691654 -22.405635470 0 1040
94 94 -21.68016500 -24.458519270 10 1000
95 95 -1.57455856 -30.874986140 0 500
96 96 -29.75642086 -5.610894981 0 350
97 97 -3.66771076 26.448084810 0 900
98 98 -26.54457307 29.824419350 0 1050
99 99 -17.90426678 18.751297440 0 200
100 100 10.22894253 -6.274450952 0 880")
And I would like to create a visualization with the polygons of thiessen, then colorize the polygons according to their "valecono" value.
I tried this:
> library(deldir)
> z <- deldir(x,y,rw=c(-34.51608,30.7052719,-30.774986,36.2288206))
> w <- tile.list(z)
> plot(w, fillcol=data$valecono, close=TRUE)
Which seems weird to me, and I'm not sure how R attributed these colors.
Do you have any other suggestions for this case?
I also tried to convert my data.frame in SpatialPolygonsDataFrame, what I did not manage. I tried to convert my data.frame into SpatialPointsDataFrame, which was not a problem, but was not very useful, because I did not find how to convert it then to a SpatialPointsDataFrame.
spdf <- SpatialPointsDataFrame(coords = coords, data = data,
proj4string = CRS("+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0"))
I try all this because I think that with a SpatialPointsDataFrame, it would be easier to have this visualization of polygons with colors according to the valecono of the points.
You can do
library(dismo)
coordinates(data) <- ~x + y
v <- voronoi(data)
spplot(v, "valecolo")
With base plot
s <- (floor(sort(v$valecono)/400) + 1)
plot(v, col=rainbow(60)[v$valecolo+1])
points(data, cex=s/2, col=gray((1:4)/4)[s])

Resources