The blue points are a scatterplot (x-y points), not "lines"
The question is, how can I detect the x coordinates at which the "lines" end (marked with green arrows)
I think that the coordinates are proportional to powers of 2 (here is the same chart in logarithmic scale)
The red line shows the accumulated number of points. There is not a fall on the point density where the lines end.
The "lines" do not end all at the same x coordinate, and the y coordinates depend on the dataset (not all are at the same scales, so I cannot just hardcode one y coordinate)
Maybe if I could detect the periodicity of the y coordinate, I could group the points by their approximate maximum x value. (note that the minimum y separation is also unknown)
I'm not asking about a particular language but a suggestion for an algorithm. (although I prefer R or python)
I don't even know what tag to use for this question, I welcome comments on the appropriate tags.
Here is a sample file, and here is a snippet from it:
0.000035048,-0.000012875
0.000035107,0.000000060
0.000039995,0.000004888
0.000036120,-0.000003874
0.000036716,0.000000596
0.000037014,0.000000298
0.000038803,0.000001788
0.000039697,0.000000894
0.000039995,0.000000298
0.000041783,0.000001788
0.000039101,-0.000002682
0.000041485,0.000002384
0.000041008,-0.000000477
0.000039995,-0.000001013
0.000040054,0.000000060
0.000040889,0.000000834
0.000041485,0.000000596
0.000044882,0.000003397
0.000044882,0.000000000
0.000047863,0.000002980
0.000045002,-0.000002861
0.000045002,0.000000000
0.000047684,0.000002682
0.000047326,-0.000000358
0.000047207,-0.000000119
0.000047207,0.000000000
0.000047386,0.000000179
0.000047386,0.000000000
0.000047445,0.000000060
0.000047982,0.000000536
0.000048995,0.000001013
0.000050008,0.000001013
0.000053227,0.000003219
0.000054479,0.000001252
0.000054777,0.000000298
0.000053108,-0.000001669
0.000050008,-0.000003099
0.000051796,0.000001788
0.000051618,-0.000000179
0.000052810,0.000001192
0.000052810,0.000000000
0.000051975,-0.000000834
0.000051558,-0.000000417
0.000053465,0.000001907
0.000053406,-0.000000060
0.000053406,0.000000000
0.000053406,0.000000000
0.000052810,-0.000000596
0.000052810,0.000000000
0.000052571,-0.000000238
0.000052571,0.000000000
0.000051975,-0.000000596
0.000050008,-0.000001967
0.000048280,-0.000001729
0.000047445,-0.000000834
0.000047445,0.000000000
0.000047445,0.000000000
0.000047445,0.000000000
0.000049531,0.000002086
0.000049353,-0.000000179
0.000049412,0.000000060
0.000049412,0.000000000
0.000048876,-0.000000536
0.000048518,-0.000000358
0.000047982,-0.000000536
0.000047803,-0.000000179
0.000047803,0.000000000
0.000048935,0.000001132
0.000047743,-0.000001192
0.000047624,-0.000000119
0.000047445,-0.000000179
0.000047147,-0.000000298
0.000046015,-0.000001132
0.000044882,-0.000001132
0.000043213,-0.000001669
0.000039995,-0.000003219
0.000039995,0.000000000
0.000039279,-0.000000715
0.000039995,0.000000715
0.000039995,0.000000000
0.000041008,0.000001013
0.000041068,0.000000060
0.000044882,0.000003815
0.000041544,-0.000003338
0.000041544,0.000000000
0.000043392,0.000001848
0.000042021,-0.000001371
0.000042021,0.000000000
0.000043929,0.000001907
0.000044763,0.000000834
0.000042200,-0.000002563
0.000044465,0.000002265
0.000044465,0.000000000
0.000044465,0.000000000
0.000043511,-0.000000954
0.000043988,0.000000477
0.000043929,-0.000000060
0.000043929,0.000000000
0.000043929,0.000000000
0.000043929,0.000000000
0.000043869,-0.000000060
0.000043809,-0.000000060
0.000043809,0.000000000
0.000045300,0.000001490
0.000045300,0.000000000
0.000045717,0.000000417
0.000045478,-0.000000238
0.000045478,0.000000000
0.000045478,0.000000000
0.000044763,-0.000000715
0.000044763,0.000000000
0.000044465,-0.000000298
0.000042796,-0.000001669
0.000042856,0.000000060
0.000042856,0.000000000
0.000044405,0.000001550
0.000044286,-0.000000119
0.000044286,0.000000000
0.000044286,0.000000000
0.000044286,0.000000000
0.000044405,0.000000119
0.000044465,0.000000060
0.000045478,0.000001013
0.000045478,0.000000000
0.000045419,-0.000000060
0.000045002,-0.000000417
0.000043631,-0.000001371
0.000043631,0.000000000
0.000044882,0.000001252
0.000044882,0.000000000
0.000044823,-0.000000060
0.000044703,-0.000000119
0.000044644,-0.000000060
0.000044644,0.000000000
0.000048399,0.000003755
0.000048161,-0.000000238
0.000048101,-0.000000060
0.000048161,0.000000060
0.000048161,0.000000000
0.000048161,0.000000000
0.000048459,0.000000298
0.000048459,0.000000000
0.000048459,0.000000000
0.000048280,-0.000000179
0.000048280,0.000000000
0.000048459,0.000000179
0.000048280,-0.000000179
0.000047982,-0.000000298
0.000048280,0.000000298
0.000048220,-0.000000060
0.000047982,-0.000000238
0.000047982,0.000000000
0.000047982,0.000000000
0.000048399,0.000000417
0.000048399,0.000000000
0.000048459,0.000000060
0.000048399,-0.000000060
0.000048399,0.000000000
0.000046253,-0.000002146
0.000046194,-0.000000060
0.000047982,0.000001788
0.000048161,0.000000179
0.000048161,0.000000000
0.000048161,0.000000000
0.000047982,-0.000000179
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047088,-0.000000894
0.000047982,0.000000894
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047266,-0.000000715
0.000047982,0.000000715
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000048399,0.000000417
0.000048399,0.000000000
0.000048399,0.000000000
0.000048399,0.000000000
0.000047982,-0.000000417
0.000047982,0.000000000
0.000047266,-0.000000715
0.000047982,0.000000715
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047505,-0.000000477
0.000047505,0.000000000
0.000047982,0.000000477
0.000047982,0.000000000
0.000047982,0.000000000
0.000048339,0.000000358
0.000048339,0.000000000
0.000048399,0.000000060
0.000047803,-0.000000596
0.000048280,0.000000477
0.000048280,0.000000000
0.000048280,0.000000000
0.000048280,0.000000000
0.000048280,0.000000000
0.000048280,0.000000000
0.000048161,-0.000000119
0.000048161,0.000000000
0.000048101,-0.000000060
0.000048101,0.000000000
0.000047922,-0.000000179
0.000047863,-0.000000060
0.000047028,-0.000000834
0.000047028,0.000000000
0.000047982,0.000000954
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047088,-0.000000894
0.000047982,0.000000894
0.000047982,0.000000000
0.000047505,-0.000000477
0.000047922,0.000000417
0.000047505,-0.000000417
0.000047207,-0.000000298
0.000047147,-0.000000060
0.000047207,0.000000060
0.000047088,-0.000000119
0.000047088,0.000000000
0.000047028,-0.000000060
0.000047028,0.000000000
0.000047028,0.000000000
0.000047028,0.000000000
0.000047803,0.000000775
0.000047803,0.000000000
0.000047803,0.000000000
0.000047803,0.000000000
0.000047803,0.000000000
0.000047803,0.000000000
0.000047922,0.000000119
0.000047863,-0.000000060
0.000047863,0.000000000
0.000047863,0.000000000
0.000047922,0.000000060
0.000047922,0.000000000
0.000047922,0.000000000
0.000047863,-0.000000060
0.000047863,0.000000000
0.000047028,-0.000000834
0.000046611,-0.000000417
0.000046611,0.000000000
0.000046611,0.000000000
0.000046611,0.000000000
0.000046611,0.000000000
0.000046611,0.000000000
0.000046015,-0.000000596
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000045002,-0.000001013
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000043869,-0.000001132
0.000043690,-0.000000179
0.000043690,0.000000000
0.000045776,0.000002086
0.000045776,0.000000000
0.000045776,0.000000000
0.000045776,0.000000000
0.000043690,-0.000002086
0.000043690,0.000000000
0.000043690,0.000000000
0.000043750,0.000000060
0.000043750,0.000000000
0.000043690,-0.000000060
0.000045776,0.000002086
0.000045776,0.000000000
0.000045776,0.000000000
0.000045776,0.000000000
0.000045776,0.000000000
0.000045776,0.000000000
0.000045776,0.000000000
0.000045717,-0.000000060
0.000045717,0.000000000
0.000045717,0.000000000
0.000044107,-0.000001609
0.000044107,0.000000000
0.000044107,0.000000000
0.000044107,0.000000000
0.000044107,0.000000000
0.000045776,0.000001669
0.000046015,0.000000238
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000046015,0.000000000
0.000045002,-0.000001013
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000044167,-0.000000834
0.000044167,0.000000000
0.000044167,0.000000000
0.000045300,0.000001132
0.000045300,0.000000000
0.000045300,0.000000000
0.000046790,0.000001490
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046790,0.000000000
0.000046730,-0.000000060
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000046730,0.000000000
0.000045121,-0.000001609
0.000045121,0.000000000
0.000045121,0.000000000
0.000045121,0.000000000
0.000045121,0.000000000
0.000045002,-0.000000119
0.000045002,0.000000000
0.000045002,0.000000000
0.000044644,-0.000000358
0.000043750,-0.000000894
0.000042975,-0.000000775
0.000042796,-0.000000179
0.000042796,0.000000000
0.000042796,0.000000000
0.000042140,-0.000000656
0.000042140,0.000000000
0.000042140,0.000000000
0.000042200,0.000000060
0.000042200,0.000000000
0.000045002,0.000002801
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000042379,-0.000002623
0.000042379,0.000000000
0.000044882,0.000002503
0.000044882,0.000000000
0.000044882,0.000000000
0.000044823,-0.000000060
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044882,0.000000060
0.000044882,0.000000000
0.000044882,0.000000000
0.000044823,-0.000000060
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044823,0.000000000
0.000044882,0.000000060
0.000042677,-0.000002205
0.000042737,0.000000060
0.000044763,0.000002027
0.000044763,0.000000000
0.000044763,0.000000000
0.000043511,-0.000001252
0.000043511,0.000000000
0.000044763,0.000001252
0.000044763,0.000000000
0.000044763,0.000000000
0.000044763,0.000000000
0.000044763,0.000000000
0.000044763,0.000000000
0.000044763,0.000000000
0.000044763,0.000000000
0.000044763,0.000000000
0.000044882,0.000000119
0.000044882,0.000000000
0.000044882,0.000000000
0.000044882,0.000000000
0.000044882,0.000000000
0.000044882,0.000000000
0.000044882,0.000000000
0.000044882,0.000000000
0.000045002,0.000000119
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000046492,0.000001490
0.000046492,0.000000000
0.000047028,0.000000536
0.000047505,0.000000477
0.000047803,0.000000298
0.000047863,0.000000060
0.000047922,0.000000060
0.000047803,-0.000000119
0.000047803,0.000000000
0.000047803,0.000000000
0.000047803,0.000000000
0.000047922,0.000000119
0.000047982,0.000000060
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047982,0.000000000
0.000047803,-0.000000179
0.000047803,0.000000000
0.000047803,0.000000000
0.000047803,0.000000000
0.000047803,0.000000000
0.000045359,-0.000002444
0.000045359,0.000000000
0.000045300,-0.000000060
0.000045359,0.000000060
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000044525,-0.000000834
0.000044525,0.000000000
0.000044525,0.000000000
0.000044525,0.000000000
0.000044525,0.000000000
0.000044525,0.000000000
0.000044525,0.000000000
0.000044525,0.000000000
0.000045359,0.000000834
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000043988,-0.000001371
0.000043988,0.000000000
0.000043988,0.000000000
0.000045359,0.000001371
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000044048,-0.000001311
0.000045359,0.000001311
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000045359,0.000000000
0.000044525,-0.000000834
0.000045121,0.000000596
0.000045121,0.000000000
0.000045121,0.000000000
0.000045121,0.000000000
0.000045121,0.000000000
0.000045121,0.000000000
0.000044048,-0.000001073
0.000043988,-0.000000060
0.000043750,-0.000000238
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043809,0.000000060
0.000044882,0.000001073
0.000044882,0.000000000
0.000044882,0.000000000
0.000044882,0.000000000
0.000043809,-0.000001073
0.000043750,-0.000000060
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043750,0.000000000
0.000043631,-0.000000119
0.000043631,0.000000000
0.000043631,0.000000000
0.000043571,-0.000000060
0.000043631,0.000000060
0.000043631,0.000000000
0.000045359,0.000001729
0.000047028,0.000001669
0.000047028,0.000000000
0.000047028,0.000000000
0.000047028,0.000000000
0.000043988,-0.000003040
0.000045955,0.000001967
0.000045955,0.000000000
0.000045955,0.000000000
0.000045955,0.000000000
0.000045955,0.000000000
0.000045955,0.000000000
0.000045002,-0.000000954
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000045002,0.000000000
0.000044107,-0.000000894
0.000044107,0.000000000
0.000044107,0.000000000
0.000044107,0.000000000
0.000044107,0.000000000
0.000043690,-0.000000417
0.000043690,0.000000000
0.000043690,0.000000000
0.000043690,0.000000000
0.000043690,0.000000000
0.000043690,0.000000000
0.000043690,0.000000000
0.000043690,0.000000000
0.000043690,0.000000000
0.000043988,0.000000298
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
0.000043988,0.000000000
I had a go on analysing the numbers, starting with this Python script:
file = open(r"C:\Users\timok\Downloads\sample_.csv")
d = {}
for line in file:
x, y = map(float, line.split(","))
d[y] = max(x, d.get(y, x))
for y, x in sorted(d.items()):
print(y, x)
This aggregates the data by Y-coordinate, keeping the maximum X-coordinate. This seemed already a good push in the right direction.
Looking at these results, I could see that these coordinates were multiples of a common (small) factor. After playing around, I found that factor to be near 1/16777215, so that allowed me to map the coordinates to integers:
d = {}
c = 16777215
for line in file:
x, y = map(float, line.split(","))
y = round(y*c)
x = round(x*c)
d[y] = max(x, d.get(y, x))
for y, x in sorted(d.items()):
print(y, x)
Looking at the Y-values around 0, the output is:
...
1 2044
2 4096
3 2050
4 8192
5 2052
6 4100
7 2039
8 16384
9 1991
10 4104
11 2041
12 8200
13 1896
14 4108
15 2011
16 32768
...
And here we see the pattern arise: where Y is a power of 2, the corresponding maximum X coordinate is also a power of 2 and also doubling each time. We see a little noise on the first value (2044 instead of 2048), but the next few values are powers of 2:
...
1 2044
2 4096
4 8192
8 16384
16 32768
...
Looking further, we see this pattern:
The (mapped) X-coordinate is 211+z, where z represents the number of trailing zeroes that the (mapped) Y-coordinate has in its binary representation. So:
y
binary
z
x = 211+z
1
00001
0
2048
2
00010
1
4096
3
00011
0
2048
4
00100
2
8192
5
00101
0
2048
6
00110
1
4096
7
00111
0
2048
8
01000
3
16384
9
01001
0
2048
10
01010
1
4096
11
01011
0
2048
12
01100
2
8192
13
01101
0
2048
14
01110
1
4096
15
01111
0
2048
16
10000
4
32768
So the boundary values you were looking for are 211+z for any natural number z. But this is after we did the mapping. So without the mapping, we get:
211+z / 16777215