I'm trying to figure out which of a bunch of lat/lon points fall within a certain region of the ocean by using a geopandas sjoin. But what I'm finding is that if I use an sjoin by 'within' or, equivalently, 'contains' - I end up with this weird set of points that are all bounded by the top and bottom latitudes of the region of interest, but have longitudes that that range all over.
Here's the code:
pmnm # this is the region of the ocean I'm interested in
geopoints # this is the set of all points
# sjoin to find geopoints that are "within" pmnm
pmnm_points = geopandas.sjoin(geopoints, pmnm, op='within', how='inner')
But it's giving me way too many points - the area I'm interested in is in red, and all the points that are supposedly "within" it are in blue:
If I the sjoin using 'intersects' instead, i.e.:
pmnm_points = geopandas.sjoin(geopoints, pmnm, op='intersects', how='inner')
then I get only the following 723 points that are actually where they should be:
And now here's the really weird part. If I take the larger set of points (that seem to all be bounded by the latitudes of the area, but not the actual contour) and basically manually check whether they're contained/within, I only get the 723 points that I thought ought to be showing up anyways:
num_contained = 0
for i in range(pmnm_points.shape[0]):
if pmnm.contains(pmnm_points.iloc[i]['geometry']).iloc[0]:
num_contained += 1
print(num_contained)
# prints 723 -- the correct number! what the heck!
So my question is: why are these points that aren't "within" the area still being picked up in the sjoin operation? What am I doing wrong?
Well, it looks like this may actually be an issue in pygeos. I uninstalled pygeos and the regular geopandas is able to run the sjoin correctly.
Related
Could you please help me understand the issue with H3 geospatial indexing?
import h3
geo_antarctic = {"type":"Polygon","coordinates":[[[-170.63764683701507,-85.05113000000047],[-170.63764683701507,-77.89462449499929],[-63.82520589349025,-66.39564184408599],[-49.69216225292467,-77.30460454007437],[-35.16653406678777,-77.89462449499929],[-9.255954059083527,-70.29658532122083],[40.994867774038596,-68.50197979740217],[89.56411960844528,-64.94027568501143],[163.48124599227498,-67.77106116580279],[172.90327508598565,-72.42721956336818],[165.83675326570284,-77.7288586062699],[178.18462781512582,-77.47601087207454],[178.57721236069702,-85.0171471646522],[-178.63764683701507,-85.05113000000047]]]}
idx = h3.polyfill(geo_antarctic, 3)
I'm expecting to get indices like these ones, which are located inside of the geojson polygon above:
83ef9efffffffff
83eea4fffffffff
83f125fffffffff
83f2a4fffffffff
But instead, h3.polyfill returns indices that are “flipped” by 90 degrees like these:
836682fffffffff
830e59fffffffff
83b294fffffffff
836733fffffffff
838f0bfffffffff
830372fffffffff
All works fine for other geojsons that don't span Antarctica..
I’m using Python 3.10.7 and H3 3.7.4.
I would appreciate any hints.
Upd.
I used geo_json_conformant=True parameter and it flipped indices back. But it seems not all resolution 3 indices were generated and my expected indices are not in the list. On the image generated indices are in blue and expected are in red.
Upd 2
Following the suggestion from #nrabinowitz, I triangulated the original polygon from Pole and then polyfilled resulting "slices". Works perfectly fine, all missing indices are in place.
result
import h3
import geojson
geoj = {"type":"Polygon","coordinates":[[[-178.34111242523068,-85.0207089708011],[-178.69267492523034,-77.91567194747755],[-162.52079992523068,-78.4905544838336],[-140.02079992523,-73.8248242864237],[-126.66142492523065,-73.12494935304983],[-103.10673742523004,-74.59011176731619],[-103.45829992523063,-71.07406105104535],[-83.06767492523001,-73.52840349816283],[-61.97392492523001,-64.32087770911836],[-57.052049925230655,-62.43108077917767],[-59.86454992522999,-74.77584672076205],[-39.12236242523063,-77.8418507294947],[-12.052049925230301,-70.61261893331015],[35.05732507477002,-68.52824009786191],[53.33857507476973,-65.51296841598038],[76.54170007476968,-68.39918525054024],[93.06513757477003,-64.77413134111099],[143.69013757477003,-66.08937000336596],[173.22138757477006,-70.72898413027124],[167.94795007477003,-76.26869800825351],[177.79170007476975,-77.23507678015689],[178.60169170931843,-84.94715491814792],[-178.34111242523068,-85.0207089708011]]]}
polygon_coords = geoj["coordinates"][0]
pole_coord = (0.0, -89.999)
all_indexes = set()
for i in range(len(polygon_coords)-1):
polygon = geojson.Polygon([[pole_coord, tuple(polygon_coords[i]), tuple(polygon_coords[i+1])]])
idxes = h3.polyfill(dict(polygon), 5, geo_json_conformant=True)
all_indexes.update(idxes)
with open(f"./absent_polr.csv", "w") as out:
out.write("h3_idx\r\n")
out.write("\r\n".join(all_indexes))
H3 uses a Cartesian model for polygons used in polyfill, not a spherical model, so poles are slightly challenging. The other issue is that we assume the smaller polygon when some of the arcs are greater than 180 degrees, which is frequently the case very near the poles.
See the suggested workaround in this related issue:
I think the best workaround here is to slice up polygons that contain a pole. I think the simplest version of this, which ought to work, is to make triangles, one for each pair of vertexes, with the third vertex being the pole itself.
The other participant in that discussion glossed this as "slicing the polygon up like a pizza," which I thought was very descriptive.
There's a demo of this approach in this Observable notebook.
I am working in Arcgis 10.3.1 and I am trying to add some random points into a polygon shapefile as the center for circular sampling plots, however, every time I use the minimum allowed distance (I use 15 m because the plot radius will be 10 m) I get an error.
Without using the minimum allowed distance it works fine but usually the samples spacing is not right.
I was thinking maybe there was not space in the area to have such spacing but the area i 4000 m2 and a 10m radius circular plot is only around 300m2 (I need only 4). Even if I use 2m distance I get the error.
I am not sure if I am using it right. Is there any other way to add random points?
Any imput is welcome
Sorry I just figure out what was happening.
Apparently there was a file with the same name as my output and that is why I got the error codes.
On a shape from a logical image, I am trying to extract the field of view from any point inside the shape on matlab :
I tried something involving to test each line going through the point but it is really really long.(I hope to do it for each points of the shape or at least each point of it's contour wich is quite a few times)
I think a faster method would be working iteratively by the expansion of a disk from the considered point but I am not sure how to do it.
How can I find this field of view in an efficient way?
Any ideas or solution would be appreciated, thanks.
Here is a possible approach (the principle behind the function I wrote, available on Matlab Central):
I created this test image and an arbitrary point of view:
testscene=zeros(500);
testscene(80:120,80:120)=1;
testscene(200:250,400:450)=1;
testscene(380:450,200:270)=1;
viewpoint=[250, 300];
imsize=size(testscene); % checks the size of the image
It looks like this (the circle marks the view point I chose):
The next line computes the longest distance to the edge of the image from the viewpoint:
maxdist=max([norm(viewpoint), norm(viewpoint-[1 imsize(2)]), norm(viewpoint-[imsize(1) 1]), norm(viewpoint-imsize)]);
angles=1:360; % use smaller increment to increase resolution
Then generate a set of points uniformly distributed around the viewpoint.:
endpoints=bsxfun(#plus, maxdist*[cosd(angles)' sind(angles)'], viewpoint);
for k=1:numel(angles)
[CX,CY,C] = improfile(testscene,[viewpoint(1), endpoints(k,1)],[viewpoint(2), endpoints(k,2)]);
idx=find(C);
intersec(k,:)=[CX(idx(1)), CY(idx(1))];
end
What this does is drawing lines from the view point to each directions specified in the array angles and look for the position of the intersection with an obstacle or the edge of the image.
This should help visualizing the process:
Finally, let's use the built-in roipoly function to create a binary mask from a set of coordinates:
FieldofView = roipoly(testscene,intersec(:,1),intersec(:,2));
Here is how it looks like (obstacles in white, visible field in gray, viewpoint in red):
So I’m trying to find the rotational angle for stripe lines in images like the attached photo.
The only assumption is that the lines are parallel, and their orientation is about 90 degrees approximately more or less [say 5 degrees tolerance].
I have to make sure the stripe lines in the result image will be %100 vertical. The quality of the images varies as well as their histogram/greyscale values. So methods based on non-adaptive thresholding already failed for my cases [I’m not interested in thresholding based methods if I cannot make it adaptive]. Also, there are some random black clusters on top of the stripe lines sometimes.
What I did so far:
1) Of course HoughLines is the first option, but I couldn’t make it work for all my images, I had some partial success though following this great article:
http://felix.abecassis.me/2011/09/opencv-detect-skew-angle/.
The main reason of failure to my understanding was that, I needed to fine tune the parameters for different images. Parameters such as Canny/BW/Morphological edge detection (If needed) | parameters for minLinelength/maxLineGap/etc. For sure there’s a way to hack into this and make it work, but, to me this is a fragile solution!
2) What I’m working on right now, is to divide the image to a top slice and a bottom slice, then find the peaks and valleys of each slice. Then basically find the angle using the width of the image and translation of peaks. I’m currently working on finding which peak of the top slice belongs to which of the bottom slice, since there will be some false positive peaks in my computation due to existence of black/white clusters on top of the strip lines.
Example: Location of peaks for slices:
Top Slice = { 1, 33,67,90,110}
BottomSlice = { 3, 14, 35,63,90,104}
I am actually getting similar vectors when extracting peaks. So as can be seen, the length of vector might vary, any idea how can I get a group like:
{{1,3},{33,35},{67,63},{90,90},{110,104}}
I’m open to any idea about improving any of these algorithms or a completely new approach. If needed, I can upload more images.
If you can get a list of points for a single line, a linear regression will give you a formula for the straight line that best fits the points. A simple trig operation will convert the line formula to an angle.
You can probably use some line thinning operation to turn the stripes into a list of points.
You can run an accumulator of spatial derivatives along different angles. If you want a half-degree precision and a sample of 5 lines, you have a maximum 10*5*1500 = 7.5m iterations. You can safely reduce the sampling rate along the line tenfold, which will give you a sample size of 150 points per sample, reducing the number of iterations to less than a million. Somewhere around that point the operation of straightening the image ought to become the bottleneck.
I'm trying to find an algorithm (or algorithm ideas) for following a ridge on a 3D image, derived from a digital elevation model (DEM). I've managed to get very basic program working which just iterates across each row of the image marking a ridge line wherever it finds a large change in aspect (ie. from < 180 degrees to > 180 degrees).
However, the lines this produces aren't brilliant, there are often gaps and various strange artefacts. I'm hoping to try and extend this by using some sort of algorithm to follow the ridge lines, thus producing lines that are complete (that is, no gaps) and more accurate.
A number of people have mentioned snake algorithms to me, but they don't seem to be quite what I'm looking for. I've also done a lot of searching about path-finding algorithms, but again, they don't seem to be quite the right thing.
Does anyone have any suggestions for types or algorithms or specific algorithms I should look at?
Update: I've been asked to add some more detail on the exact area I'll be applying this to. It's working with gridded elevation data of sand dunes. I'm trying to extract the crests if these sand dunes, which look similar to the boundaries between drainage basins, but can be far more complex (for example, there can be multiple sand dunes very close to each other with gradually merging crests)
You can get a good estimate of the ridges using sign changes of the curvature. Note that the curvature will be near infinity at flat regions. Hence possible psuedo-code for a ridge detection algorithm could be:
for each face in the mesh
compute 1/curvature
if abs(1/curvature) != zeroTolerance
flag face as ridge
else
continue
(zeroTolerance is a number near but not equal to zero e.g. 0.003 etc)
Also Meshlab provides a module for normal & curvature estimation on most formats. You can test the idea using it, before you code it up.
I don't know how what your data is like or how much automation you need. This won't work if if consists of peaks without clear ridges (but then you probably wouldn't be asking the question.)
startPoint = highest point in DEM (or on ridge)
curPoint = startPoint;
line += curPoint;
Loop
curPoint = highest point adjacent to curPoint not in line; // (Don't backtrack)
line += point;
Repeat
Curious what the real solution turns out to be.
Edited to add: depending on the coarseness of your data set, 'point' can be a single point or a smoothed average of a local region of points.
http://en.wikipedia.org/wiki/Ridge_detection
You can treat the elevation as you would a grayscale color, then use a 2D edge recognition filter. There are lots of edge recognition methods available. The best would depend on your specific needs.