Summary: I need a tool that can put 60m+ points on a map image. I'm trying to show density map and would like to plot a dot for each point (lat/long) on the map.
Hi I'm working on project that requires a density map. I have latitude and longitude and all the tools that I have seen (Ammap, FusionCharts maps, google charts/map) requires either XML or JSON or some other data type with the data points. Problem here is that, I have 60 million + data points and transferring any type of object with that many data point is not feasible.
One solution I can think of is mapping latitude and longitude to pixels of the map image. That requires a lot of time and work. I was wondering if you guys have done something similar and know of tools that can do this for me. It doesn't have to be free.
You'd be much better off creating a heatmap instead of passing all of the individual points to the map, which would get very busy, very quickly.
There are already a few discussions over at GIS.StackExchange about this exact topic. Search the the [heatmap] tag.
Since you mentioned FusionCharts, assuming you can load all of your data into a Google Fusion Table, it looks like you should be able to make a heat map in Google Fusion Tables).
Related
Hello I am making a animation time series map in CARTO builder. The animation works fine, but I would like my points to continuously move from point A to B, not just populate from point A to B like its doing now if theres a vast distance for the particular ID for that point. I know this would work if I had many more coordinates filling in from point A to B like in this example, but I don't. So is there a way to do this in cartodb?
CARTO BUILDER allows you to analyze and visualize your data according to your values. So as you pointed out, you would need to enrich your dataset with data among those dates and locations. Regarding dates, you can use generate_series PostgreSQL function as explained here. In relation to generating points between two coordinates, you would need to create a line (via Connect with lines analysis or ST_MakeLine) and then apply ST_DumpPoints).
I am trying to create a map layer using D3 and leaflet for displaying a large number of unique GPS data points. I created it using geoJSON and Leaflet but the performance was poor. I finally got Topojson installed and working, but I cannot get it to produce a Multipoint geometry, only Point geometry which does not shrink the file much. I have passed in a CSV with all the points and used to geoJson file but only get 70,000 point geometries instead of one Multipoint. What am I missing? Do I need to write the Topojson myself? Want to avoid this if possible.
TopoJSON won't help you in this case. To quote the website:
Rather than representing geometries discretely, geometries in TopoJSON files are stitched together from shared line segments called arcs.
As you have no line segments, there's no point in using TopoJSON -- it won't reduce the size of the file.
+1 What Lars said. Your best bet is probably to load the point data as a CSV using d3.csv()instead of a GeoJSON or TopoJSON, as it is much more compact. You can then loop over the data, adding each point to a layer group.
That said, 70,000 is a lot and your map is still probably going to be very slow. You might want to think about using something like PostGIS (or CartoDB for that matter) and request only those points that are visible in a given map state.
I followed this example : http://bost.ocks.org/mike/map/
to display this topojson file :
https://github.com/max-l/topojson-experiments/blob/master/qc-map.json
the map is always empty, I tried setting the proper center
d3.geo.albers().center(-71,45) and many other things without success.
I noticed that Github is able to render it (with leaflet)
https://github.com/max-l/topojson-experiments/blob/master/qc-map.json
which makes me assume it is valid topojson..
I'd guess its a projection issue. The co-ordinates in the topojson and your center look like they are WSG84 or Mercator co-ordinates (the America's are expressed as negative longitudes) and they are similar to the coordinates in this question. WSG84 is generally a safe guess for projections particularity for things like Google Maps, although Albers is commonly used for North America.
I've made a gist using Mercator which displays the map and you can see the block here.
I have created a simple line graph with data from a mySQL database using PHP to return the data in JSON format.
https://gist.github.com/5fc4cd5f41a6ddf2df23
I would like to simulate "live" updating something similar to this example, but less complicated:
http://bl.ocks.org/2657838
I've been searching for examples on how to achieve this simply as new to D3 - to no avail.
I've looked at Mike Bostock's http://bost.ocks.org/mike/path/ path transitions, but not sure how to implement this using json data.
Can anyone help with either an example or some direction on how I could accomplish this?
Doing that kind of line transformations is tricky in SVG because moving large number of points just a little and rerendering the complete line can hurt performance.
For the case when interactivity with each data point is not paramount and the time series can grow to contain arbitrary number of points, consider using Cubism. It is a library based on d3 but meant specially for visualizing time-series data efficiently. To prevent rerendings of SVG, it draws the points on a canvas, allowing for cheap pixel by pixel transitions as new data arrives.
I am trying to visualize a river flow- basically, should be able to visualize river current direction and speed based on an user-defined external parameter. This is required to demonstrate vectors in two dimensions- given education needs, animation quality can be minimal- 'tolerable enough'.
I tried a simplistic approach by a blue background with lines indicating currents- comes out very weak and below my low standards!!
Can someone point out a good example/ approach for achieving the same? Thanks.
You can create an image filter that looks like water. Look at Jerry's image filters. Specifically look at the the caustic filter. You could animate it moving from one end of the river to the other end. You can also experiment with varying the time parameter. Since it's open source, you can translate it to other languages.
Here are some links to 3d visualizations.