Observing invalid value inside the loop - go

I've stumbled on a buggy Golang code that was trying to use mutex to prevent changes to the variables printed in a goroutine:
runtime.GOMAXPROCS(1)
mutex := new(sync.Mutex)
for i := 0; i < 10; i++ {
for j := 0; j < 10; j++ {
mutex.Lock()
go func() {
fmt.Println(i, j, i + j);
mutex.Unlock()
}()
}
}
It is clear to me that mutex doesn't lock directly, but on the next iteration, when the value has been incremented already. What is not clear is why j variable reaches 10 according to the output:
...
0 7 7
0 8 8
0 9 9
1 10 11 <--- isn't supposed to be here
...
1 9 10
2 10 12
...
I tried to debug the code and j = 10 is printed when the outer loop for i increments its value. It looks as if the outer loop was releasing the thread allowing goroutine to execute and see invalid value of 10. Could someone clarify this behavior?

You have data races. The results are undefined.
$ go run -race racer.go
==================
WARNING: DATA RACE
Read at 0x00c000016110 by goroutine 7:
main.main.func1()
/home/peter/gopath/racer.go:17 +0x7f
Previous write at 0x00c000016110 by main goroutine:
main.main()
/home/peter/gopath/racer.go:14 +0xf1
Goroutine 7 (running) created at:
main.main()
/home/peter/gopath/racer.go:16 +0xcd
==================
0 1 1
0 2 2
0 3 3
0 4 4
0 5 5
0 6 6
0 7 7
0 8 8
0 9 9
==================
WARNING: DATA RACE
Read at 0x00c000016108 by goroutine 16:
main.main.func1()
/home/peter/gopath/racer.go:17 +0x50
Previous write at 0x00c000016108 by main goroutine:
main.main()
/home/peter/gopath/racer.go:13 +0x140
Goroutine 16 (running) created at:
main.main()
/home/peter/gopath/racer.go:16 +0xcd
==================
1 10 11
1 1 2
1 2 3
1 3 4
1 4 5
1 5 6
1 6 7
1 7 8
1 8 9
1 9 10
2 10 12
2 1 3
2 2 4
2 3 5
2 4 6
2 5 7
2 6 8
2 7 9
2 8 10
2 9 11
3 10 13
3 1 4
3 2 5
3 3 6
3 4 7
3 5 8
3 6 9
3 7 10
3 8 11
3 9 12
4 10 14
4 1 5
4 2 6
4 3 7
4 4 8
4 5 9
4 6 10
4 7 11
4 8 12
4 9 13
5 10 15
5 1 6
5 2 7
5 3 8
5 4 9
5 5 10
5 6 11
5 7 12
5 8 13
5 9 14
6 10 16
6 1 7
6 2 8
6 3 9
6 4 10
6 5 11
6 6 12
6 7 13
6 8 14
6 9 15
7 10 17
7 1 8
7 2 9
7 3 10
7 4 11
7 5 12
7 6 13
7 7 14
7 8 15
7 9 16
8 10 18
8 1 9
8 2 10
8 3 11
8 4 12
8 5 13
8 6 14
8 7 15
8 8 16
8 9 17
9 10 19
9 1 10
9 2 11
9 3 12
9 4 13
9 5 14
9 6 15
9 7 16
9 8 17
9 9 18
Found 2 data race(s)
exit status 66
$
racer.go:
package main
import (
"fmt"
"runtime"
"sync"
)
func main() {
runtime.GOMAXPROCS(1)
mutex := new(sync.Mutex)
for i := 0; i < 10; i++ {
for j := 0; j < 10; j++ {
mutex.Lock()
go func() {
fmt.Println(i, j, i+j)
mutex.Unlock()
}()
}
}
}
Go: Data Race Detector

You have data race, so the results are undefined. Run it with the -race option to see.
When you call mutex.Lock() inside the loop body first, that doesn't block. Then you launch a goroutine that reads i and j and the main goroutine continues to the next iteration of the inner loop, and increments j. Then calls lock again, which will block until the previous goroutine finishes.
But you already have an unsynchronized access (read and write) to j.

Let me to answer why you can get the impossible 10 when print j.
Because when you use goroutine in a loop, fmt.Println(i, j, i+j) race with i++/j++, you cannot determine what value exactly when you print out, and if j increase to the bound, it's possible to print out 10.
if you want to prevent from this race, you can transfer i, j as argument values, e.g.
runtime.GOMAXPROCS(1)
for i := 0; i < 10; i++ {
for j := 0; j < 10; j++ {
go func(a, b, c int) {
fmt.Println(a, b, c);
}(i, j, i+j)
}
}
Hope this helps.

Related

R Margins outcome non-compliant with excpected raw data

We've run an Interrupted Time Series analysis on some aggregate count data using a Poisson regression. Code shown below - where Subject Total is the count, Quarter is time, int2 is the dummy variable for the intervention [0 pre, 1 post] and time_since_intervention2 the dummy variable for time since intervention [0 pre, 1:N post].
fit1a <- glm(`Subject Total` ~ Quarter + int2 + time_since_intervention2 , df, family = "poisson")
Quarter Subject Total int2 time_since_intervention2 subjectfit subcounter
1 1 34 0 0 34.20968 34.20968
2 2 32 0 0 33.39850 33.39850
3 3 36 0 0 32.60656 32.60656
4 4 34 0 0 31.83339 31.83339
5 5 23 0 0 31.07856 31.07856
6 6 34 0 0 30.34163 30.34163
7 7 33 0 0 29.62217 29.62217
8 8 24 0 0 28.91977 28.91977
9 9 31 0 0 28.23402 28.23402
10 10 32 0 0 27.56454 27.56454
11 11 21 0 0 26.91093 26.91093
12 12 26 0 0 26.27282 26.27282
13 13 22 0 0 25.64984 25.64984
14 14 28 0 0 25.04163 25.04163
15 15 28 0 0 24.44784 24.44784
16 16 22 0 0 23.86814 23.86814
17 17 14 1 1 17.88365 23.30218
18 18 16 1 2 17.01622 22.74964
19 19 20 1 3 16.19087 22.21020
20 20 19 1 4 15.40556 21.68355
21 21 13 1 5 14.65833 21.16939
22 22 15 1 6 13.94735 20.66743
23 23 16 1 7 13.27085 20.17736
24 24 8 1 8 12.62717 19.69892
Due to the need to exponentiate the outcome the summary is currently being derived using the margins package.
> summary(margins(fit1a))
factor AME SE z p lower upper
int2 -5.7843 5.1734 -1.1181 0.2635 -15.9241 4.3555
Quarter -0.5809 0.2469 -2.3526 0.0186 -1.0649 -0.0970
time_since_intervention2 -0.6227 0.9955 -0.6255 0.5316 -2.5738 1.3285
If reading the outcome correctly it would suggest that the level change between the final quarter in the pre-intervention period and first in the post-intervention period is -5.7843.
I've tried inputting coefficient values into my model [initial intercept = 35.0405575], but they don't appear to correlate at all with the subjectfit data, which I believed it would. Should the level change reported by the margins package replicate the difference in the full data.....?

Why does printing out local data mess with MPI_GATHERV results?

The following code needs 12 processors to run properly.
program GameOfLife
use mpi
implicit none
integer ierr, myid, numprocs
integer send, recv, count, tag
parameter (tag=111)
integer recv_buff, send_buff, request
integer stat(MPI_STATUS_SIZE)
integer N, m, i, j, sum
parameter (N=3) !# of squares per a processors
integer W, H
parameter (W=4,H=3) !# of processors up and across
integer A(N,N), buff(0:N+1,0:N+1), G(N*H, N*W)
! real r
integer sizes(2), subsizes(2), starts(2), recvcount(N*N)
integer newtype, intsize, resizedtype
integer(kind=MPI_ADDRESS_KIND) extent, begin
integer disp(W*H)
call MPI_INIT(ierr)
call MPI_COMM_RANK(MPI_COMM_WORLD, myid, ierr)
call MPI_COMM_SIZE(MPI_COMM_WORLD, numprocs, ierr)
! fill up subgrid
do i = 1, N
do j = 1, N
! call random_number(r)
A(i,j) = myid! floor(r*2)
end do
end do
do i = 1, N
print *, A(i, :)
end do
starts = [0,0]
sizes = [N*H, N*W]
subsizes = [N, N]
call MPI_Type_create_subarray(2, sizes, subsizes, starts, &
MPI_ORDER_FORTRAN, MPI_INTEGER, newtype, ierr)
call MPI_Type_size(MPI_INTEGER, intsize, ierr)
extent = intsize*N
begin = 0
call MPI_Type_create_resized(newtype, begin, extent, resizedtype, ierr)
call MPI_Type_commit(resizedtype, ierr)
disp = [0, 1, 2, 9, 10, 11, 18, 19, 20, 27, 28, 29]
recvcount = 1
call MPI_GATHERV(A,N*N,MPI_INTEGER,G,recvcount,disp,resizedtype,0,MPI_COMM_WORLD,ierr)
call MPI_WAIT(request, stat, ierr)
if ( myid == 0 ) then
do i = 1, N*H
print *, G(i,:)
end do
endif
call MPI_FINALIZE(ierr)
end program GameOfLife
When ran without printing out the matrix A, everything works mostly okay. But If I try to print out A before feeding it to the gather statement, I get a jumbled mess.
What's going on here? I assume memory is trying to be accessed at the same time or something along those lines.
Output of G looks like
0 0 0 4 4 4 0 -1302241474 1 13 13 13
0 0 0 4 4 4 0 0 0 13 13 13
0 0 0 4 4 4 -10349344 -12542198 -10350200 13 13 13
1 1 1 5 5 5 59 59 59 14 14 14
1 1 1 5 5 5 -1342953792 0 0 14 14 14
1 1 1 5 5 5 32767 0 0 14 14 14
2 2 2 6 6 6 -1342953752 1451441280 0 15 15 15
2 2 2 6 6 6 32767 10985 0 15 15 15
2 2 2 6 6 6 -10350200 1 0 15 15 15
3 3 3 7 7 8 8 8 12 12 12 0
3 3 3 7 7 8 8 8 12 12 12 0
3 3 3 7 7 8 8 8 12 12 12 0

How to make a N×N matrix each of i-th row and i-th column having all the elements 1 to 2N-1?

This question was asked as a puzzle in one Book of Puzzles by RS AGGARWAL, which stated the problem as to build an order N matrix where each i'th row and i'th column combined have all the elements from 1 to 2N-1.
For instance, for N=2
[3,2]
[1,3]
I want to know when is an answer possible for it for which values of N it is possible to make a matrix and how to make it? and write code for it
this has simple solution for square matrices where n is power of 2 so n=1,2,4,8,16,... do not ask me why there surely is some math proof for it ...
The algorithm to create such matrix is easy:
clear matrix (with 0)
loop i through all values i=1,2,3...2n-1
for each i find all locations where i matrix is not yet filled (0) and there is not i present in row and column
fill the position with i and repeat until no such location found.
In C++ something like this:
//---------------------------------------------------------------------------
const int n=8;
int m[n][n];
//---------------------------------------------------------------------------
// compute histogram u[n+n] of values per ith row,col of m[n][n]
void hist_rst(int *u ) { for (int j=0;j<n+n;j++) u[j]=0; }
void hist_row(int *u,int m[n][n],int i) { for (int j=0;j<n;j++) u[m[j][i]]=1; }
void hist_col(int *u,int m[n][n],int i) { for (int j=0;j<n;j++) u[m[i][j]]=1; }
//---------------------------------------------------------------------------
void matrix_init(int m[n][n])
{
int i,x,y,h[n][n+n];
// clear matrix (unused cells)
for (x=0;x<n;x++)
for (y=0;y<n;y++)
m[x][y]=0;
// clear histograms
for (i=0;i<n;i++) hist_rst(h[i]);
// try to fill values 1..2n-1
for (i=1;i<n+n;i++)
{
// find free position
for (x=0;x<n;x++) if (!h[x][i])
for (y=0;y<n;y++) if (!h[y][i])
if (!m[x][y])
{
// set cell
m[x][y]=i;
h[x][i]=1;
h[y][i]=1;
break;
}
}
}
//---------------------------------------------------------------------------
here few outputs:
1
1 3
2 1
1 5 6 7
2 1 7 6
3 4 1 5
4 3 2 1
1 9 10 11 12 13 14 15
2 1 11 10 13 12 15 14
3 4 1 9 14 15 12 13
4 3 2 1 15 14 13 12
5 6 7 8 1 9 10 11
6 5 8 7 2 1 11 10
7 8 5 6 3 4 1 9
8 7 6 5 4 3 2 1
for non power of 2 matrices you could use backtracking but take in mind even 4x4 matrix will have many iterations to check ... so some heuristics would need to be in place to make it possible in finite time... as brute force is (n+n)^(n*n) so for n=4 there are 281474976710656 combinations to check ...
[edit1] genere&test solution for even n
//---------------------------------------------------------------------------
const int n=6;
int m[n][n];
//---------------------------------------------------------------------------
// compute histogram u[n+n] of values per ith row,col of m[n][n]
void hist_rst(int *u ) { for (int j=0;j<n+n;j++) u[j]=0; }
void hist_row(int *u,int m[n][n],int i) { for (int j=0;j<n;j++) u[m[j][i]]=1; }
void hist_col(int *u,int m[n][n],int i) { for (int j=0;j<n;j++) u[m[i][j]]=1; }
//---------------------------------------------------------------------------
void matrix_init2(int m[n][n]) // brute force
{
int x,y,a,ax[(n*n)>>1],ay[(n*n)>>1],an,u[n+n];
// clear matrix (unused cells)
for (x=0;x<n;x++)
for (y=0;y<n;y++)
m[x][y]=0;
// main diagonal 1,1,1,1...
for (x=0;x<n;x++) m[x][x]=1;
// 1st row 1,2,3...n
for (x=1;x<n;x++) m[x][0]=x+1;
// cells for brute force
for (an=0,x=0;x<n;x++)
for (y=0;y<x;y++)
if (!m[x][y])
{
ax[an]=x;
ay[an]=y;
an++;
m[x][y]=2;
}
// brute force attack values 2,3,4,5,...,n-1
for (;;)
{
// increment solution
for (a=0;a<an;a++)
{
x=ax[a];
y=ay[a];
m[x][y]++;
if (m[x][y]<=n) break;
m[x][y]=2;
}
if (a>=an) break; // no solution
// test
for (x=0;x<n;x++)
{
hist_rst(u);
hist_col(u,m,x);
hist_row(u,m,x);
for (y=1;y<=n;y++) if (!u[y]) { y=0; x=n; break; }
}
if (y) break; // solution found
}
// mirror other triangle
for (x=0;x<n;x++)
for (y=0;y<x;y++)
m[y][x]=m[x][y]+n-1;
}
//---------------------------------------------------------------------------
however its slow so do not try to go with n>6 without more optimizations/better heuristics... for now it is using triangle+mirror and diagonal + first row hard-coded heuristics.
maybe somehow exploit the fact that each iterated value will be placed n/2 times could speed this up more but too lazy to implement it ...
Here output for n=6:
[ 52.609 ms]
1 2 3 4 5 6
7 1 6 5 3 4
8 11 1 2 4 5
9 10 7 1 6 3
10 8 9 11 1 2
11 9 10 8 7 1
iterating through 5^10 cases ...
As requested by Spektre, here is the 6x6 matrix.
I an interesting property that may be used as heuristic. We need only to solve a triangular matrix because the other half can be easily deduced. We fill the upper (or lower) half of the matrix by values from 1 to n only. We can then complete the matrix by using the property that a[j][i] = 2n + 1 - a[i][j].
Another property I found is that there is a trivial way to place 1, 2 and N in the matrix. The values 1 are all on the diagonal, the values 2 and N are next to the diagonal at a step 2.
Finally, another thing I found is that matrix with odd N have no solutions. It is because the value in a[i][j] belongs to row and column i and row and column j. We thus need an even number of row and columns to store all values.
Here is the 6x6 matrix I found manually.
1 2 3 4 5 6
11 1 6 5 3 4
10 7 1 2 4 5
9 8 11 1 6 3
8 10 9 7 1 2
7 9 8 10 11 1
As we can see 2 + 11 = 6 + 7 = 3 + 10 = 13 = 2*6+1.
Here is a 4x4 matrix
1 2 3 4
7 1 4 3
6 5 1 2
5 6 7 1
Here again 2 + 7 = 4 + 5 = 3 + 6 = 9 = 2*4+1
It is possible to have other permutations of values >N, but with the 2N+1 property we can trivially deduce one triangular matrix from the other.
EDIT
Here is a solution for power two sized matrix. The matrix of size 2048x2048 is generated in 57ms (without printing).
#include <stdio.h>
int **newMatrix(int n) {
int **m = calloc(n, sizeof(int*));
m[0] = calloc(n*n, sizeof(int));
for (int i = 1; i < n; i++)
m[i] = m[0]+i*n;
return m;
}
void printMatrix(int **m, int n) {
for (int i = 0; i < n; i++) {
for (int j = 0; j < n; j++)
printf("%3d ", m[i][j]);
printf("\n");
}
}
void fillPowerTwoMatrix(int **m, int n) {
// return if n is not power two
if (n < 0 || n&(n-1) != 0)
return;
for (int i = 0; i < n; i++)
m[0][i] = i+1;
for (int w = 1; w < n; w *= 2)
for (int k = 0; k < n; k += 2*w)
for (int i = 0; i < w; i++)
for (int j = k; j < k+w; j++) {
m[i+w][j] = m[i][j+w];
m[i+w][j+w] = m[i][j];
}
int k = 2*n+1;
for (int i = 1; i < n; i++)
for (int j = 0; j < i; j++)
m[i][j] = k - m[j][i];
}
int main() {
int n = 16;
int **m = newMatrix(n);
fillPowerTwoMatrix(m, n);
printMatrix(m, n);
return 0;
}
Here is the matrix 16x16. As can be seen there is a symmetry that is exploited to efficiently generate the matrix.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
31 1 4 3 6 5 8 7 10 9 12 11 14 13 16 15
30 29 1 2 7 8 5 6 11 12 9 10 15 16 13 14
29 30 31 1 8 7 6 5 12 11 10 9 16 15 14 13
28 27 26 25 1 2 3 4 13 14 15 16 9 10 11 12
27 28 25 26 31 1 4 3 14 13 16 15 10 9 12 11
26 25 28 27 30 29 1 2 15 16 13 14 11 12 9 10
25 26 27 28 29 30 31 1 16 15 14 13 12 11 10 9
24 23 22 21 20 19 18 17 1 2 3 4 5 6 7 8
23 24 21 22 19 20 17 18 31 1 4 3 6 5 8 7
22 21 24 23 18 17 20 19 30 29 1 2 7 8 5 6
21 22 23 24 17 18 19 20 29 30 31 1 8 7 6 5
20 19 18 17 24 23 22 21 28 27 26 25 1 2 3 4
19 20 17 18 23 24 21 22 27 28 25 26 31 1 4 3
18 17 20 19 22 21 24 23 26 25 28 27 30 29 1 2
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1

Number Pyramid Algorithm

I was searching around the internet trying to find the algorithm of the following pyramid:
1
2 3 2
3 4 5 4 3
4 5 6 7 6 5 4
5 6 7 8 9 8 7 6 5
6 7 8 9 10 11 10 9 8 7 6
7 8 9 10 11 12 13 12 11 10 9 8 7
8 9 10 11 12 13 14 15 14 13 12 11 10 9 8
9 10 11 12 13 14 15 16 17 16 15 14 13 12 11 10 9
10 11 12 13 14 15 16 17 18 19 18 17 16 15 14 13 12 11 10
I wasn't able to find the algorithm, my question is: Does anyone know the algorithm and/or name for this type of pyramid?
Any help is greatly appreciated. I'm sorry if anything in the post is wrong in some way, new posting here.
Here's one solution .... but you should not be lazy, this isn't so hard :)
It is written in Java ....
What you see in the System.out.print() is "Ternary operator". You need to check if it is two digit number, to reduce the number of empty spaces.
public static void Pyramid(int rows) {
int r = 1; // r is current row
int emptySpaces = rows;
while(r <= rows) {
//print the empty spaces (3 empty spaces in one iteration)
for(int i = 1; i < emptySpaces; i++) {
System.out.print(" ");
}
//print the numbers to the middle including the middle number
for(int i = 0; i < r; i++) {
System.out.print((r+i)/10 == 0 ? (r+i + " ") : (r+i + " "));
}
//print the numbers from the middle to the end
for(int i = r-2; i >= 0; i--) {
System.out.print((r+i)/10 == 0 ? (r+i + " ") : (r+i + " "));
}
//print new line, reduce empty spaces
System.out.println("");
emptySpaces--;
r++;
}
}

sort vs sortrows in Matlab

Is it possible to achieve the same functionality with sort function than using sortrows. My matrix has over 4million+ rows and sortrows is bringing in a lot of latency because of iterations. (a vectorizated approach would be appreciated)
%Col1 -> date, Col2 -> id, Col3 -> ranking within each date-group (to help you debug)
data = [ ...
734614 5 3; 734615 6 5; 734622 1 1; 734615 1 1; 734615 4 3;
734622 2 2; 734622 4 3; 734615 3 2; 734615 5 4; 734614 3 2;
734614 1 1; 734622 8 4; 734622 9 5;] ;
sortedanswer =
734614 1 1
734614 3 2
734614 5 3
734615 1 1
734615 3 2
734615 4 3
734615 5 4
734615 6 5
734622 1 1
734622 2 2
734622 4 3
734622 8 4
734622 9 5
Thanks!
You could do it as
[~,indx]=sort(data(:,1));
sortedanswer=data(indx,:)
sortedanswer =
734614 5 3
734614 3 2
734614 1 1
734615 6 5
734615 1 1
734615 4 3
734615 3 2
734615 5 4
734622 1 1
734622 2 2
734622 4 3
734622 8 4
734622 9 5
Note that it is sorted by the rows in the first column. The order of the rows is the same as that in the original data, which is why you see 5 3 in the second and third columns in the first row in mine.

Resources