d3 got slow all of a sudden - performance

Unfortunately, by 'all of a sudden', I mean between versions 3.1.1.10 and 3.2.0. I have a scatterplot with about 3000 points in it, and we are trying to make a mouse click based tooltip work since tooltips based on mouseover events don't really work for us.
So, while testing our initial attempt, we saw that it took a full 2 or 3 seconds before anything happened. I thought that was a little odd. I mean sure, we're doing a nearest-neighbor calculation, but it's only 3000 points, so what gives? I profiled it and found that once we got the event trigger, our calculations were only taking about 22 msec, so that's good. So what is causing d3 to take so much time in getting us active on mouse clicks? I started dropping in different versions of d3 to see if it was the culprit and sure enough, things were slow until I got back to version 3.1.0. That one have us the mouse event pretty much immediately and our tooltip popups were now working well.
Only problem is I can't use 3.1.10 because I need the zoom().center() functionality that was added more recently.
So any ideas? Anybody know why the slowdown occurred there? Can it be patched? Can someone suggest a clever workaround? I can't be the only one who has seen this problem.
I brought it up in the d3 forum, topic number qfz11UYgMC8
I understand why I cannot post codepen demos here without code, but the 2 demos I have demonstrate my point above by differing only in what version of d3 is included in them. I will include the code here if people really want to see it, but it's pretty much standard d3.js where I create my scatterplot from a json object, and it's thousands of lines of code.
And the problem is definitely based on the size of the scatterplot. A small plot of a dozen or so points pops up the tooltip right away, and the one with 3000 points is slow (on all d3.js libraries 3.2 and later).
Edit: adding code snippets, and codepen links
The code itself is thousands of lines long because the data itself gets embedded in the html. Here is what I hope is the relevant code...
<html>
<head>
<title>Rolling D3 speed test</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.2.0/d3.min.js"></script>
<!-- <script src="file://C:/d3.v3.5.16.min.js"></script> -->
<script type="text/javascript" charset="utf8" src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<script type="text/javascript" language="javascript" class="init">
// variable init left out for brevity
...
var svg = d3.select("body").append("svg")
.attr("width",viewportWidth)
.attr("height",viewportHeight)
.call(zoom)
.append("g")
.attr("class", "main")
.attr("transform","translate("+(left_shift)+","+(top_shift)+")");
...
//mouse click on svg (to display tooltip on nearest point)
d3.select('svg').on("click", function() {
var startTime = new Date();
var coords = [0, 0];
coords = d3.mouse(this);
var isClicked = isPointClicked(point_clicked);
if(!isClicked) clicked(coords[0], coords[1]);
point_clicked = false;
var finalEndTime = new Date();
var finalTimeDiff = finalEndTime - startTime;
console.log('all click actions completed in '+finalTimeDiff+' milliseconds.');
});
//mouse-right click event to remove tooltip
d3.select('body').on("contextmenu", function() {
d3.event.preventDefault();
var ele = d3.select('div.tooltip');
//if(ele!=="undefined" && ele!==null)
ele.style("opacity",0);
});
...
// x and y axis rendering removed for brevity
...
//draw all data
//var radius = 2.5; // 2.5 normally, 8 is good for testing small sets
var side = 2; // takes place of radius
// shiftrect needs to be half the square size to center square on data point
var shiftrect = 1;
for(var k=0;k<plot_data.series.length;++k) {
if(plot_data.series[k].visible === true) {
var y_list = plot_data.series[k].ydata;
var x_list = plot_data.series[k].xdata;
svg.selectAll("dot")
.data(x_list)
.enter().append("rect")
.attr("id", function(d, i) { return "rect"+k+i;})
.attr("x", function(d) {return x(parseFloat(d))-shiftrect;})
.attr("y", function(d,i) {return y(parseFloat(y_list[i]))-shiftrect;})
.attr("height", side)
.attr("width", side)
.attr("fill",function() { return color(k);})
.attr("clip-path", "url(#plot-clip)");
}
}
Codepen that is slow
http://codepen.io/pkrouse/details/ONwQmz/
Codepen that is fast. Only diff is version of D3 used
http://codepen.io/pkrouse/details/JXBpKo/

Related

D3 Selection Highlight (efficiency?)

I have a simple visual of many rects, over 100 I'd say. For aesthetic purposes I want to create a high light effect on mouse click. I also wanted to make this effect somewhat intuitive by removing that effect once the user clicks on a new rect. However I couldn't get this to work without resorting to a d3.selectAll() call, so I'm thinking this approach might not be ideal if this project gets any bigger. Here is the code:
.on('click.highlight', function() {
//set any previously highlighted rects back to normal color/brightness
d3.selectAll('.highlight').transition().duration(250)
.style('fill', function(d) { return d3.rgb(d.color)})
d3.select(this).classed('highlight',true);
//now it's safe to assign the current highlighted rect a brighter hue... i think
d3.select(this).transition().duration(250)
.style('fill', function(d) { return d3.rgb(d.color).brighter(.5)})
})
Though this code does what I wanted it to do, but presumably there could only ever be 1 other highlight rect to worry about at any give time. So again, I'm not sure that using d3.selectAll() is warranted here.
So anyway, is there a more efficient way? I'd like to keep it all within one .on('click') function if possible.
If you are looking to avoid use of .selectAll, you could create a selection of one rect that contains the last clicked rectangle. Each time you click on a rectangle:
unhighlight the previously selected highlighted rect
update that selection to reflect the most recently clicked rectangle
highlight the newly selected rect
I use the variable highlightedRect to hold the selection that will allow the above workflow:
var svg = d3.select("body").append("svg")
.attr("width",600)
.attr("height",400);
var highlightedRect = d3.select(null);
var rects = svg.selectAll("rect")
.data(d3.range(1600))
.enter()
.append("rect")
.attr("y",function(d) { return Math.floor(d/50)*12; })
.attr("x",function(d) { return d%50 * 12 })
.attr("width",11)
.attr("height",11)
.attr("stroke","white")
.on("click",function(d) {
// Recolor the last clicked rect.
highlightedRect.attr("fill","black");
// Color the new one:
highlightedRect = d3.select(this);
highlightedRect.attr("fill","steelblue");
})
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/4.10.0/d3.min.js"></script>

D3 v4: element 'snaps' to previous translate after programmatic transform

function zoomed() { svg.attr("transform", d3.event.transform); }
var zoom = d3.zoom().on("zoom", zoomed);
var svgMain = d3.select('body').append('svg').call(zoom);
var svg = svgMain.append('g') // All the drawing done here
When I translate svg programmatically with svg.call(zoom.translateBy, 100, 100) then drag the element with the mouse, svg transform attribute snaps back to the value from before the drag.
It is almost as if the transform effected by svg.call is not stored or saved, and reverts to the transform stored in d3.event.transform.
This question seems to be hitting on the same issue, though for v3.
Seems like you are applying the zoom behavior to two different nodes - svgMain and svg.
Try running svgMain.call(zoom.translateBy, 100, 100) instead of svg.call(...) and see if it solves the problem.

Making an button-activated animated tour in d3

I'm trying to make a button-activated d3 zoom tour through three Northeastern US points, but am having a hard time getting the data to show up visually (it shows in the console, though). I'm a beginning user and can usually solve things, but this is over my head.
Here are the an example that comes close to what I'm trying to do:
Zooms between different spots in US:
http://bl.ocks.org/mbostock/6242308
The example doesn't style the data, uses TopoJSON and also uses canvas to do the zooming calls. I'm trying to do the zoom with GeoJSON(so I can link to a CartoDB table), and style it.
I've gone through a lot to make both of those things happen and am running out of successes. Right now it comes up blank and has been. I can see the data live, but can't change the styling.
What am I doing wrong here? I'm sure it's something simple, but need a nudge.
<!DOCTYPE html>
<html lang ="en">
<head>
<meta charset="utf-8">
<body>
<script src="http://d3js.org/d3.v3.min.js"></script>
<style type="text/css">
canvas{
color: 'blue';
}
path.state {
color: 'red';
}
</style>
</head>
<body>
<script type="text/javascript">
var width = 960,
height = 500,
stateMap;
var sf = [-122.417, 37.775],
ny = [-74.0064, 40.7142];
var scale,
translate,
visibleArea, // minimum area threshold for points inside viewport
invisibleArea; // minimum area threshold for points outside viewport
var zoom = d3.behavior.zoom()
.size([width, height])
.on("zoom", zoomed);
var projection = d3.geo.mercator()
.translate([width/2, height/2])
.scale(500);
var canvas = d3.select("body").append("canvas")
.attr("width", width)
.attr("height", height);
var svg = d3.select("body")
.append("svg")
.attr("width", width)
.attr("height", height);
var context = canvas.node().getContext("2d");
var path = d3.geo.path()
.projection(simplify)
.context(context);
stateMap = d3.json("http://linepointpath.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM GRAPHstates", function(error, stateMap) {
console.log(stateMap)
canvas
svg.selectAll("path")
.data(stateMap.feature)
.enter().append("path")
.attr("class", "state")
.attr("d", path)
.call(zoomTo(sf, 4).event)
.transition()
.duration(60 * 1000 / 89 * 2)
.each(jump);
});
var simplify = d3.geo.transform({
point: function(x, y, z) {
if (z < visibleArea) return;
x = x * scale + translate[0];
y = y * scale + translate[1];
if (x >= -10 && x <= width + 10 && y >= -10 && y <= height + 10 || z >= invisibleArea) this.stream.point(x, y);
}
});
function zoomTo(location, scale) {
var point = projection(location);
return zoom
.translate([width / 2 - point[0] * scale, height / 2 - point[1] * scale])
.scale(scale);
}
function zoomed(d) {
translate = zoom.translate();
scale = zoom.scale();
visibleArea = 1 / scale / scale;
invisibleArea = 200 * visibleArea;
context.clearRect(0, 0, width, height);
context.beginPath();
path(d);
context.stroke();
}
function jump() {
var t = d3.select(this);
(function repeat() {
t = t.transition()
.call(zoomTo(ny, 6).event)
.transition()
.call(zoomTo(sf, 4).event)
.each("end", repeat);
})();
}
</script>
</body>
</html>
I suspect the larger issue is that you're using a GeoJSON when the example you're going by is using a TopoJSON. The differences between the two are likely causing problems with how the paths are being rendered.
Another problem you're running into is that with a canvas, the function calls are different. You're using the normal svg function calls to append an SVG, bind the data, and they style it. With canvas, you interact with the elements through the canvas context object. This has a different syntax and usage than the standard SVG object you're trying to use in your code. If you follow this tutorial you'll notice she doesn't call any of the SVG functions but instead uses the context API to draw and style the canvas elements. Be sure to look at the working example of the code.
In your case, this means your code here:
svg.selectAll("path")
.data(stateMap.feature)
.enter().append("path")
.attr("class", "state")
.attr("d", path)
.call(zoomTo(sf, 4).event)
.transition()
.duration(60 * 1000 / 89 * 2)
Is unnecessary and not doing anything for you. The code that generates the path is embedded in the zoomed() function using the context object:
context.clearRect(0, 0, width, height);
context.beginPath();
path(d);
context.stroke();
It's using the context functions to create the objects you're trying to show. There are a chain of function calls that generate this behavior and you'll need to break down the chain to make sure you're getting what you want at each step.
If you want to use the GeoJSON start with just getting the map to display and then applying the zoom functionality. It'll probably make your life a lot easier in the end to iteratively build the visualization you want.
For more information on the difference between canvas and svg with D3, including examples of doing the same operation with each, checkout this blogpost and good luck with the project.

When a d3.behavior.zoom event is triggered programatically, how do you set inital values for translate and scale?

The squares in the example below are part of an SVG group that has an initial translate and scale set.
Clicking on a square initiates a zoom transition. But the intial values set by the transition are different from my defaults, as made obvious by the jarring start to this transition.
How can I set initial values for translate and scale on a zoom transition that I initiate programatically?
var svg = d3.select("#main");
svg.append("rect").attr({"x":0,"y":0,"height":100,"width":100,"fill":"red"})
svg.append("rect").attr({"x":100,"y":100,"height":100,"width":100,"fill":"blue"})
svg.append("rect").attr({"x":0,"y":100,"height":100,"width":100,"fill":"green"})
svg.append("rect").attr({"x":100,"y":0,"height":100,"width":100,"fill":"yellow"})
var zoom = d3.behavior.zoom().on("zoom",function(){
var t = d3.event.translate;
var s = d3.event.scale;
console.log(s)
svg.attr("transform","translate("+t[0]+","+t[1]+") scale("+s+")")
}).scaleExtent([1,10]).scale(1).translate([0,0])
d3.select("svg").call(zoom)
d3.selectAll("rect").on("mousedown",function(){
var scale = Math.random()*3;
var translate = [Math.random()*200,Math.random()*200]
zoom.scale(scale);
zoom.translate(translate);
//new transition
var T = svg.transition().duration(5000)
zoom.event(T);
})
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.4.11/d3.min.js"></script>
<p style="font-weight:bold">When a zoom is triggered programatically, how do you set inital values for translate and scale?</p>
<p>Click on one of the squares</p>
<svg height="600px" width="600px">
<g id="main" transform="translate(25,25) scale(0.25)"></g>
</svg>
That is a problem with the zoom function itself. I would suggest zooming the children as opposed to the parent if that would work
var zoom = d3.behavior.zoom().on("zoom",function(){
var t = d3.event.translate;
var s = d3.event.scale;
svg.selectAll("rect").attr("transform","translate("+t[0]+","+t[1]+") scale("+s+")")
}).scaleExtent([1,10]);
EDIT
The problem with the above code is that d3.js does not register the transformation or initial state of the SVG. This problem runs deeper. As d3 does not keep track of the SVG transformations and just executes them. It only keeps track of transformations you've run on the library in a variable called __chart__.
So when the zoom function is run it just interpolates the variables and gives the output. As no functions have been run on this yet the __chart__ variable has not been set and causing the jerky start from (x=0, y=0, k=1).
Solution:
Run this code before the zoom transformation to set the initial chart manually
svg.transition().each(function(){
this.__chart__={x:25,y:25,k:0.25}; //or you can pick those values using attr
});
Zoom the svg programmatically to 25,25,0.25 first before any other function. (this is why your workaround works as the __chart__ variable gets set)
To set the initial value of the zoom, try something like this:
// Init zoom
var zoom = d3.behavior.zoom().on("zoom", function () {
svg.attr("transform", "translate(" + d3.event.translate + ")" + " scale(" + d3.event.scale + ")");
});
// Get SVG element
var svg = d3.select("svg")
.call(zoom)
.append("g");
// Create circle
svg.append("circle")
.attr("cx",0)
.attr("cy",0)
.attr("r", 5);
// Create init value
var scale = 5;
var translate = [50, 50];
// Set init value
zoom.scale(scale);
zoom.translate(translate);
// Call zoom event
svg.call(zoom.event);
// or svg.transition().call(zoom.event);
<script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.4.11/d3.min.js"></script>
<svg height="100px" width="100px"></svg>
I was looking for the answer to this, but it seems D3 has already evolved a couple of versions.
Although Majkl and cjds's answers helped me solve my problem, I thought it would help to leave more up to date information, since it was hard finding v5.4 examples out there, until I found Observable at least.
// Applies event transformation to the Group element's attribute
const zoom_action = () => g.attr("transform", d3.event.transform)
// Create the zoom handler
const zoom = d3
.zoom()
.on("zoom", zoom_action)
// Get SVG element and apply zoom behaviour
var svg = d3
.select("svg")
.call(zoom)
// Create Group that will be zoomed
var g = svg.append("g")
// Create circle
g.append("circle")
.attr("cx",0)
.attr("cy",0)
.attr("r", 5)
// Set initial scale and translation
zoom.scaleBy(svg, 5)
zoom.translateBy(svg, 50, 50)
<script src="https://d3js.org/d3.v5.js"></script>
<svg height="100px" width="100px"></svg>

Zooming and Panning a Mercator Map centered on the Pacific using d3.js

Apologies if this is a simple case of me being blind to the obvious, but I am trying to put together a page that shows a map of the world (data sourced from a TopoJSON file) in Mercator projection centered on the Pacific. I.e. Europe on the left, America on the right and Australia in the middle. A bit like this...
From this point I want to be able to zoom and pan the map to my hearts desire, but when I pan east or west, I want the map to scroll 'around' and not come to the end of the World (I hope that makes sense).
The code I am currently working on is here (or at the following Gist (https://gist.github.com/d3noob/4966228) or block (http://bl.ocks.org/d3noob/4966228));
<!DOCTYPE html>
<meta charset="utf-8">
<style>
body {font-size:11px;}
path {
stroke: black;
stroke-width: 0.25px;
}
</style>
<body>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://d3js.org/topojson.v0.min.js"></script>
<script>
var width = 960,
velocity = .005,
then = Date.now()
height = 475;
var projection = d3.geo.mercator()
.center([0, 0 ])
.scale(1000);
var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height);
var path = d3.geo.path()
.projection(projection);
var g = svg.append("g");
d3.json("world-110m.json", function(error, topology) {
g.selectAll("path")
.data(topojson.object(topology, topology.objects.countries).geometries)
.enter()
.append("path")
.attr("d", path)
.style("fill","black")
d3.timer(function() {
var angle = velocity * (Date.now() - then);
projection.rotate([angle,0,0]);
svg.selectAll("path")
.attr("d", path.projection(projection));
});
var zoom = d3.behavior.zoom()
.on("zoom",function() {
g.attr("transform","translate("+d3.event.translate.join(",")+")scale("+d3.event.scale+")")
});
svg.call(zoom)
});
</script>
</body>
</html>
The code is an amalgam of examples and as a result I can see a map that can rotate west to east automatically, and I can pan and zoom using the mouse, but when panning and zooming, from what I can tell, I am affecting the internal "g" element and not the map within the "svg" element.
There are plenty of good examples of being able to pan and zoom a map centered on the meridian. But none on the anti-meridian that I have discovered.
Any help would be greatly appreciated.
I ended up working on the same problem. Here's an example (see code) where you pan left/right to rotate the projection (with wraparound), and up/down to translate (clamped by max absolute latitude), with zoom as well. Ensures that projection always fits within viewbox.
I learned a lot about zoom behavior, and projection center() and rotate() interaction.
hope this code can solve your problem
var projection = d3.geo.equirectangular()
.center([0, 5])
.scale(90)
.translate([width / 2, height / 2])
.rotate([0, 0])
.precision(9);
Google maps on apple products work like this. Scrol left, and you will leave one Australia, then find another and another and another

Resources