"Live" graph d3.js with simulated data - d3.js

I have created a simple line graph with data from a mySQL database using PHP to return the data in JSON format.
https://gist.github.com/5fc4cd5f41a6ddf2df23
I would like to simulate "live" updating something similar to this example, but less complicated:
http://bl.ocks.org/2657838
I've been searching for examples on how to achieve this simply as new to D3 - to no avail.
I've looked at Mike Bostock's http://bost.ocks.org/mike/path/ path transitions, but not sure how to implement this using json data.
Can anyone help with either an example or some direction on how I could accomplish this?

Doing that kind of line transformations is tricky in SVG because moving large number of points just a little and rerendering the complete line can hurt performance.
For the case when interactivity with each data point is not paramount and the time series can grow to contain arbitrary number of points, consider using Cubism. It is a library based on d3 but meant specially for visualizing time-series data efficiently. To prevent rerendings of SVG, it draws the points on a canvas, allowing for cheap pixel by pixel transitions as new data arrives.

Related

Which D3 visualization/layout is suitable for Data Lineage?

As a newbie loving D3, I am trying to visualize data movement from source to destination that goes through multiple systems. But I am having hard time identifying suitable layout. Tried using Chord and Tree but don't know how to represent intermediate systems visually. Are there any recommendations/rule of thump on which layout/diagram is more appropriate for a given purpose (e.g.data lineage)?

Three.js tubeGeometry not coloring properly

I have a code that lets users enter data and plots it with a tube geometry. The code seems to work fine most of the time, however, one of the test data sets is not coloring properly.
Here is an example page for a site that I am building that solves for the position and velocity of a bungee jumper. Scroll to the bottom of the page and you will see a three js environment with a sin wave and a plot of the position of the jumper. These two items are charted with separate color maps and you can see that the sin wave is colored properly but the data is not.
At first I thought that maybe the data was too sparsely populated, but that was not the problem.
The code for this is too long to really paste here, but the fact that it charts right for all other data sets makes me think that I am missing something inherent to the tubeGeometry function.
Any ideas as to why the one tube is miscolored?
UPDATE: When I add additional interpolated points between each existing point in the data set, the error lessons. The more the padding, the less the error. This leads me to think that the error is due to the difference between the interpolation of the spline function from Three.SplineCurve3 and the true data. This would also explain why my other examples work fine since they are all sinusoid data.
How can I prevent SplineCurve3 from doing this, or what else can I use to create the Tube geometry?
I guess it is the mesh length counting problem (three did not count length on vector+vector+vector but by mesh.position+bounding radius)
Maybe you can separate curve to parts and color each part independent on their lenght.
There are some working approaches:
https://stemkoski.github.io/Three.js/Graphulus-Curve.html
https://stemkoski.github.io/Three.js/Graphulus-Surface.html
https://stemkoski.github.io/Three.js/Graphulus-Function.html

DataStructure to cache a large set of Chart DataPoints

I have a scenario like I am plotting a Chart using API like JFreeChart or SWT Chart or BIRT anything is fine.
The data for plotting the chart is bit high like 10GB. So how chart works is it just keep the latest data points like X,Y and discard other data for efficient memory utilization.
Like I got a scenario that once that is done a user comes and try to
zoom the chart or wanna see the certain specific DataPoints , so solve
this scenario I need to cache all the data points in the chart that
will again take the memory on toss as if I need to save entire data
points it may lead to huge memory.
So what is the most efficient algorithm or precisely any DataStructure to sort this problem.
It is nothing to do with java but I am programming in Java , so I mentioned Java here.

Generating a multipoint topojson file

I am trying to create a map layer using D3 and leaflet for displaying a large number of unique GPS data points. I created it using geoJSON and Leaflet but the performance was poor. I finally got Topojson installed and working, but I cannot get it to produce a Multipoint geometry, only Point geometry which does not shrink the file much. I have passed in a CSV with all the points and used to geoJson file but only get 70,000 point geometries instead of one Multipoint. What am I missing? Do I need to write the Topojson myself? Want to avoid this if possible.
TopoJSON won't help you in this case. To quote the website:
Rather than representing geometries discretely, geometries in TopoJSON files are stitched together from shared line segments called arcs.
As you have no line segments, there's no point in using TopoJSON -- it won't reduce the size of the file.
+1 What Lars said. Your best bet is probably to load the point data as a CSV using d3.csv()instead of a GeoJSON or TopoJSON, as it is much more compact. You can then loop over the data, adding each point to a layer group.
That said, 70,000 is a lot and your map is still probably going to be very slow. You might want to think about using something like PostGIS (or CartoDB for that matter) and request only those points that are visible in a given map state.

Visualization of river- animation via code

I am trying to visualize a river flow- basically, should be able to visualize river current direction and speed based on an user-defined external parameter. This is required to demonstrate vectors in two dimensions- given education needs, animation quality can be minimal- 'tolerable enough'.
I tried a simplistic approach by a blue background with lines indicating currents- comes out very weak and below my low standards!!
Can someone point out a good example/ approach for achieving the same? Thanks.
You can create an image filter that looks like water. Look at Jerry's image filters. Specifically look at the the caustic filter. You could animate it moving from one end of the river to the other end. You can also experiment with varying the time parameter. Since it's open source, you can translate it to other languages.
Here are some links to 3d visualizations.

Resources