I have a data file containing values of longitudes and latitudes (displayed on two columns) measured by a GPS along a profile at regular intervals. at a certain point on my profile, the GPS stopped working, hence in my data i have zeros instead of values of longitudes and latitudes. I want to interpolate between this fields to get values of longitudes and latitudes instead of zeros.
to be more clear here is a simple example of how my file looks like.
[12 7] ;
[14 8 ];
[0 0];
[0 0];
[20 11];
[22 12]
NB: the number are on two columns, it's the editor's problem
i want to interpolate where i got zeros. i am working on bash and i have no idea on how to do it
You might have luck with a linear regression done once for the first column and once for the second column.
Assume we're working on the first column. The input / x axis is the index of the measurement, and the output / y axis is the measurement itself. So your data can become ((1, 12), (2, 14), (3, 0), (4, 0), (5, 20), (6, 22)). Based on the known input-output relationships, for indices (1, 2, 5, 6), you need to deduce a formula of the form y = ax + b. So you basically need to find a and b. Once you have those you can find the y for input 3 as 3a + b and the y for input 4 as 4a + b.
You'll find a different a and b for the second column.
How to find a and b is a little bit complicated. You can look at this article for a nice introduction as well as the formulas for computing a and b (named b0 and b1 inside).
One last thing - I would not attempt doing such a thing in Bash. While it is possible, as Bash has support for arrays and math, it is not what it was designed for. Call out to Matlab/Octave or R, and use their results. Those tools have builtin support for reading files such as yours as well as for doing the regression.
You can do that with awk, here is a script:
script.awk
BEGIN { FS="[ [\\];]+"
# decide on the number of floating digits for the interpolated figures
FMTStr= "[%.1f, %.1f];\n"
}
{ if( ($2== 0) && ( $3 == 0) ) { zeroLines++ }
else {
for( i = 1; i <= zeroLines; i++ ) {
t1 = prev1 + (($2 - prev1) / (zeroLines + 1 )) * i
t2 = prev2 + (($3 - prev2) / (zeroLines + 1 ) ) * i
printf(FMTStr, t1, t2)
}
# either printf(FMTStr, $2, $3) #or
print $0
prev1 = $2
prev2 = $3
zeroLines = 0
}
}
Use it so: awk -f script.awk yourfile, it gives you
[12 7] ;
[14 8 ];
[16.0, 9.0];
[18.0, 10.0];
[20 11];
[22 12];
Related
I found this problem in a programming forum Ohjelmointiputka:
https://www.ohjelmointiputka.net/postit/tehtava.php?tunnus=ahdruu and
https://www.ohjelmointiputka.net/postit/tehtava.php?tunnus=ahdruu2
Somebody said that there is a solution found by a computer, but I was unable to find a proof.
Prove that there is a matrix with 117 elements containing the digits such that one can read the squares of the numbers 1, 2, ..., 100.
Here read means that you fix the starting position and direction (8 possibilities) and then go in that direction, concatenating the numbers. For example, if you can find for example the digits 1,0,0,0,0,4 consecutively, you have found the integer 100004, which contains the square numbers of 1, 2, 10, 100 and 20, since you can read off 1, 4, 100, 10000, and 400 (reversed) from that sequence.
But there are so many numbers to be found (100 square numbers, to be precise, or 81 if you remove those that are contained in another square number with total 312 digits) and so few integers in a matrix that you have to put all those square numbers so densely that finding such a matrix is difficult, at least for me.
I found that if there is such a matrix mxn, we may assume without loss of generalty that m<=n. Therefore, the matrix must be of the type 1x117, 3x39 or 9x13. But what kind of algorithm will find the matrix?
I have managed to do the program that checks if numbers to be added can be put on the board. But how can I implemented the searching algorithm?
# -*- coding: utf-8 -*-
# Returns -1 if can not put and value how good a solution is if can be put. Bigger value of x is better.
def can_put_on_grid(grid, number, start_x, start_y, direction):
# Check that the new number lies inside the grid.
x = 0
if start_x < 0 or start_x > len(grid[0]) - 1 or start_y < 0 or start_y > len(grid) - 1:
return -1
end = end_coordinates(number, start_x, start_y, direction)
if end[0] < 0 or end[0] > len(grid[0]) - 1 or end[1] < 0 or end[1] > len(grid) - 1:
return -1
# Test if new number does not intersect any previous number.
A = [-1,-1,-1,0,0,1,1,1]
B = [-1,0,1,-1,1,-1,0,1]
for i in range(0,len(number)):
if grid[start_x + A[direction] * i][start_y + B[direction] * i] not in ("X", number[i]):
return -1
else:
if grid[start_x + A[direction] * i][start_y + B[direction] * i] == number[i]:
x += 1
return x
def end_coordinates(number, start_x, start_y, direction):
end_x = None
end_y = None
l = len(number)
if direction in (1, 4, 7):
end_x = start_x - l + 1
if direction in (3, 6, 5):
end_x = start_x + l - 1
if direction in (2, 0):
end_x = start_x
if direction in (1, 2, 3):
end_y = start_y - l + 1
if direction in (7, 0, 5):
end_y = start_y + l - 1
if direction in (4, 6):
end_y = start_y
return (end_x, end_y)
if __name__ == "__main__":
A = [['X' for x in range(13)] for y in range(9)]
numbers = [str(i*i) for i in range(1, 101)]
directions = [0, 1, 2, 3, 4, 5, 6, 7]
for i in directions:
C = can_put_on_grid(A, "10000", 3, 5, i)
if C > -1:
print("One can put the number to the grid!")
exit(0)
I also found think that brute force search or best first search is too slow. I think there might be a solution using simulated annealing, genetic algorithm or bin packing algorithm. I also wondered if one can apply Markov chains somehow to find the grid. Unfortunately those seems to be too hard for me to implemented at current skills.
There is a program for that in https://github.com/minkkilaukku/square-packing/blob/master/sqPackMB.py . Just change M=9, N=13 from the lines 20 and 21.
I need help optimizing this loop. matrix_1 is a (nx 2) int matrix and matrix_2 is a (m x 2), m & n very.
index_j = 1;
for index_k = 1:size(Matrix_1,1)
for index_l = 1:size(Matrix_2,1)
M2_Index_Dist(index_j,:) = [index_l, sqrt(bsxfun(#plus,sum(Matrix_1(index_k,:).^2,2),sum(Matrix_2(index_l,:).^2,2)')-2*(Matrix_1(index_k,:)*Matrix_2(index_l,:)'))];
index_j = index_j + 1;
end
end
I need M2_Index_Dist to provide a ((n*m) x 2) matrix with the index of matrix_2 in the first column and the distance in the second column.
Output example:
M2_Index_Dist = [ 1, 5.465
2, 56.52
3, 6.21
1, 35.3
2, 56.52
3, 0
1, 43.5
2, 9.3
3, 236.1
1, 8.2
2, 56.52
3, 5.582]
Here's how to apply bsxfun with your formula (||A-B|| = sqrt(||A||^2 + ||B||^2 - 2*A*B)):
d = real(sqrt(bsxfun(#plus, dot(Matrix_1,Matrix_1,2), ...
bsxfun(#minus, dot(Matrix_2,Matrix_2,2).', 2 * Matrix_1*Matrix_2.')))).';
You can avoid the final transpose if you change your interpretation of the matrix.
Note: There shouldn't be any complex values to handle with real but it's there in case of very small differences that may lead to tiny negative numbers.
Edit: It may be faster without dot:
d = sqrt(bsxfun(#plus, sum(Matrix_1.*Matrix_1,2), ...
bsxfun(#minus, sum(Matrix_2.*Matrix_2,2)', 2 * Matrix_1*Matrix_2.'))).';
Or with just one call to bsxfun:
d = sqrt(bsxfun(#plus, sum(Matrix_1.*Matrix_1,2), sum(Matrix_2.*Matrix_2,2)') ...
- 2 * Matrix_1*Matrix_2.').';
Note: This last order of operations gives identical results to you, rather than with an error ~1e-14.
Edit 2: To replicate M2_Index_Dist:
II = ndgrid(1:size(Matrix_2,1),1:size(Matrix_2,1));
M2_Index_Dist = [II(:) d(:)];
If I understand correctly, this does what you want:
ind = repmat((1:size(Matrix_2,1)).',size(Matrix_1,1),1); %'// first column: index
d = pdist2(Matrix_2,Matrix_1); %// compute distance between each pair of rows
d = d(:); %// second column: distance
result = [ind d]; %// build result from first column and second column
As you see, this code calls pdist2 to compute the distance between every pair of rows of your matrices. By default this function uses Euclidean distance.
If you don't have pdist2 (which is part of the the Statistics Toolbox), you can replace line 2 above with bsxfun:
d = squeeze(sqrt(sum(bsxfun(#minus,Matrix_2,permute(Matrix_1, [3 2 1])).^2,2)));
I have a system of inequalities and constraints:
Let A=[F1,F2,F3,F4,F5,F6] where F1 through F6 are given.
Let B=[a,b,c,d,e,f] where a<=b<=c<=d<=e<=f.
Let C=[u,v,w,x,y,z] where u<=v<=w<=x<=y<=z.
Equation 1: if(a>F1, 1, 0) + if(a>F2, 1, 0) + ... + if(f>F6, 1, 0) > 18
Equation 2: if(u>a, 1, 0) + if(u>b, 1, 0) + ... + if (z>f, 1, 0) > 18
Equation 3: if(F1>u, 1, 0) + if(F1>v, 1, 0) + ... + if(F6>z, 1, 0) > 18
Other constraints: All variables must be integers between 1 and N (N is given).
I wish to merely count the number of integer solutions to my variables (I do not wish to actually solve them). I know how to use solvers to calculate systems of equations in matrices but this usually assumes those equations use = as opposed to >=, >, <, or <=.
Here's a stab at it.
This is horribly inefficient, as I compute the Cartesian product of the two vectors, then compare each tuple combination. This also won't scale past 2 dimensions.
Also, I'm worried this isn't exactly what you are looking for, because I'm solving each equation independently. If you're looking for all the integer values that satisfy a 3-dimensional space bound by the system of inequalities, well, that's a bit of a brain bender for me, albeit very interesting.
Python anyone?
#sample data
A =[12,2,15,104,54,20]
B =[10,20,30,40,50,60]
C =[100,200,300,400,500,600]
import itertools
def eq1():
product = itertools.product(B,A) #construct Cartesian product of 2 lists
#list(product) returns a Cartesian product of tuples
# [(12, 10), (12, 20), (12, 30)... (2, 10), (2, 20)... (20, 60)]
#now, use a list comprehension to compare the values in each tuple,
# generating a list of only those that satisfy the inequality...
# then return the length of that list - which is the count
return len([ Bval for Bval, Aval in list(product) if Bval > Aval])
def eq2():
product = itertools.product(C,B)
return len([ Cval for Cval, Bval in list(product) if Cval>Bval])
def eq3():
product = itertools.product(A,C)
return len([ Aval for Aval, Cval in list(product) if Aval>Cval])
print eq1()
print eq2()
print eq3()
This sample data returns:
eq1 : 21
eq2 : 36
eq3 : 1
But doesn't know how to combine these answers into a single integer count of all 3 - there's some kind of union that's going to happen between the lists.
My sanity test is in equation 3, which returns '1' - because only when Aval = 104 does it satisfy Aval > Cval for Cval only at 100.
I have one problem with exportation results from Mathematica. Two matrices A and B have to be exported in special form.
These two codes make a list of data exported from Maple.
It is important that exported file opened with wordpad looks like column (File attached).
Please, just if you already checked that it is working, write me answer, thank you! You can check your answer comparing with files down.
Codes are here
Matrices A and B with code in Maple and exported file
http://www.2shared.com/file/49wW8Z0-/EXAMPLE_EXPORT_MAPLE_FINAL.html
And also I will present it here to everybody can see easy
Code 1)
A := Matrix(2, 2, {(1, 1) = (455200000000/6133413)*w(1), (1, 2) = -(1792000000000/116534847)*w(1), (2, 1) = (455200000000/6133413)*w(2), (2, 2) = -(1792000000000/116534847)*w(2)})
precision := double: writeto(`Aexport.for`):
for i from 1 to 2 do:for j from 1 to 2 do:
if A[i,j]<>0 then codegen[fortran]([A00[i,j]=A[i,j]],optimized):
fi:od:od:writeto(terminal):
Code 2)
B := Matrix(2, 2, {(1, 1) = 6436781.609, (1, 2) = 0, (2, 1) = 0, (2, 2) = 3862068.966})
writeto(Bexport);
for i to 2 do
for j to 2 do
printf("%016.15E\n", B[i, j])
end do:
end do:
writeto(terminal)
This is a translation of the (B) part only:
matrix = {{6436781.609, 0}, {0, 3862068.966}}
Export["Bexport", Map[FortranForm, N#Flatten[matrix]], "Table"]
Please test it and let me know if it works for you.
Differences compared to the Maple version: the E is written as lowercase and the number of digits that are output is not fixed (but, as you can see, all significant digits are preserved). Will these differences cause problems in your application?
I believe this does what you want for matrix B:
b = {{6436781.609, 0}, {0, 3862068.966}}
bformatted =
NumberForm[
Flatten#b,
{16, 15},
NumberFormat -> (Row[{#, "E+", StringTake["00" <> #3, -2]}] &)
];
bstring =
StringReplace[
ToString#bformatted,
{"{"|"}"|" " -> "", "," -> "\n"}
];
WriteString["Bexport.dat", bstring, "\n"]
Close["Bexport.dat"]
I have a sorted list of overlapping intervals, intervals are never contained in each other, e.g.,
[(7, 11), (9, 14), (12, 17)]
The constraint for the output is to keep every element as close as possible to its
origin (the middle of the interval), preserve the order of the input, and remove all overlap. Only an
approximate solution is necessary. The expected result for the example input would be:
[(5,9), (9, 14), (14, 19)]
I'm only aware of solutions that go about this in some simulation
style: shift each element by some value in a free direction and
iterate until all overlap has been removed.
Is there an existing algorithm to solve this?
find the overall average:
in our example:
(7 + 11 + 9 + 14 + 12 + 17)/6 = 11.667
find the total length:
(11-7) + (14-9) + (17-12) = 4 + 5 + 5 = 14;
find the new min/max;
14/2 = 7
11.667 - 7 = 4.667
11.667 + 7 = 18.667
you can round 'em
4.667 ~ 5
18.667 ~ 19
start from the min, creating the sections by the intervals
(5, (11-7)+5) = (5,9)
(9, (14-9)+9) = (9,14)
(14, (17-12)+14) = (14,19)
NOTE:
this method will not keep the elements as equal as possible to the originals, but will keep them as close as possible to the original considering their relative values (preserving the center)
EDIT:
if you want to keep the averages of all intervals as close as possible to the original, you can implement a mathematical solution.
our problem's input is:
a1=(a1,1, a1,2) , ... , an=(an,1,an,2)
we will define:
ai1 = a1,2-a1,1 // define the intervals
b1 = (d, d+ai1)
bn = (d + sum(ai1..ain-1), d + sum(ai1..ain) )
bi1 = b1,2-b1,1 // define the intervals
we need to find a 'd' such as:
s = sum( abs((a1,1+a1,2)/2 - (b1,1+b1,2)/2) )
min(s) is what we want
in our example:
a1 = (7,11), ai1 = 4, Aavg1 = 9
a2 = (9,14), ai2 = 5, Aavg2 = 11.5
a3 = (12,7), ai3 = 5, Aavg3 = 14.5
b1 = (d, d+4) Bavg1 = d+2
b2 = (d+4, d+9) Bavg2 = d+6.5
b3 = (d+9, d+14) Bavg3 = d+11.5
s = abs(9-(d+2)) + abs(11.5-(d+6.5)) + abs(14.5-(d+11.5)) = abs(7-d) + abs(5-d) + abs(3-d)
now calculcate the derivative to find min/max OR iterate over d to get a result. in our case you will need to iterate from 3 to 7
that should do the trick
Given that the solution must be order-preserving, we can formulate this problem as a linear program. Let [ai, bi] be the ith interval. Let variables xi be the left shift of the ith interval and yi be the right shift of the ith interval.
minimize sumi (xi + yi)
subject to
(*) for all i: bi - xi + yi ≤ ai+1 - xi+1 + yi+1
for all i: xi, yi ≥ 0
Rewrite constraint (*) by introducing a variable zi.
for all i: xi - yi - xi+1 + yi+1 - zi = 0
for all i: zi ≥ bi - ai+1
Now the problem is reduced to computing a minimum-cost circulation, which can be done in poly-time. I have a feeling, however, that there's a more direct solution to this problem.
The graph looks something like
(*)
---- | ----
/ z| \
/ i| \
/ xi | xi+1 \
|/ <---- v <---- \|
... (*) ...
----> ---->
yi yi+1