I have a rowchart in DCjs that plots the top N values of a given parameter. However, for the unfiltered data these differ from each other by a very small number.
I've had to label each row with it's unique identifier, as my random generator produced two identical names, meaning that if I use name as the dimension, I end up with one of ten thousand data points having a value greater than 100%.
However, the main problem here is that the difference between each row is tiny, and can be around 0.0001.
However if I zoom in on that part of the x-axis using
var max = dim.top[1][0].value;
var min = dim.top(10)[9].value;
chart
.dimension(dim)
.group(group)
.x(d3.scaleLinear()
.range([-100, chart.width()])
.domain([min-(max-min)*0.1,max])
)
.cap(10)
.othersGrouper(null)
.colors(['#ff0000']);
Firstly I loose the ID label on the left. Secondly as I also have to use .elasticX(false) for the zooming to work, it means that when I add filters, the range of the x-axis doesn't change with the values e.g.
Is there a way to get dynamic zooming such that the range of the x-axis depends on the range of values presented?
elasticX is a really simple feature which does pretty much what your code does, although it locks the min or max to zero depending if the data is positive or negative:
var extent = d3.extent(_rowData, _chart.cappedValueAccessor);
if (extent[0] > 0) {
extent[0] = 0;
}
if (extent[1] < 0) {
extent[1] = 0;
}
(calculateAxisScale source)
This code gets called (indirectly) before each render and redraw.
Here's some general purpose advice for when elasticX or elasticY doesn't do exactly what you want. I've never seen it fail! (Which is saying something in such a quirky codebase as dc.js.)
First, disable elasticX. Then create a function which calculates the X domain and sets it:
function custom_elastic(chart) {
var max = chart.dimension().top[1][0].value;
var min = chart.dimension().top(10)[9].value;
chart.x().domain([min,max]);
}
I've parameterized it on the chart for generality.
Now we can have this function called on the preRender and preRedraw events. These events will pass the chart when they fire:
chart.on('preRender', custom_elastic)
.on('preRedraw', custom_elastic);
And that should do it!
BTW, you probably don't want to set the range of the scale - this is set automatically by the chart, and it's a little more complicated than you have it since it takes margins into account.
Debugging the min and max
Looking at your fiddle I realized that I hadn't given a second look to how you are calculating the min and max.
I also hadn't noticed that you had the range start at -100.
Good first step logging it; it reports
min: 0.81, max: 0.82
which is incorrect. The top ten are from 0.96 to 1.
The issue is that the dimension's key is the id, so the rows returned by .top() are in reverse alphabetical order (the "largest" strings).
Again you're on the right track with
console.log(Group.top(Infinity))
Yes! The group will give you the top 10 by value.
var top10 = thischart.group().top(10);
var max = top10[0].value;
var min = top10[9].value;
Nice!
fiddle
But wait, doesn't it look like the bars are stretching off the left side of the chart?
Hacking the row chart with a renderlet to draw bars starting at the left edge
Okay now it's clear that the row chart doesn't support this. Here is a hack to resize the bars to the left edge:
chart.on('renderlet', function(chart) {
var transform = chart.select('g.row rect').attr('transform');
var tx = +transform.split('(')[1].split(',')[0];
chart.selectAll('g.row rect')
.attr('transform', null)
.attr('width', function(d) {
return +d3.select(this).attr('width') + tx;
})
chart.selectAll('g.row text.row')
.attr('transform', null);
})
All the row rects are going to be offset by a large negative number, which we grab first in tx. Then we remove the transform from both the rects and the text, and add tx to the width of the row rects.
fiddle
Great! But where's the last bar? Well, we took the top ten values for the min and max, so the tenth bar is the minimum value.
You'll have to figure out what works for you, but I found that looking at the top 20 values left the top 10 at good sizes:
var N = 20;
var topN = thischart.group().top(N);
var max = topN[0].value;
var min = topN[N-1].value;
final fiddle
This hack does not play well with the built-in transitions, so I turned them off for this chart:
chart.transitionDuration(0)
It would be a lot more work to hack that part, and better to fix it in the chart itself.
Related
I have a dc.js heatmap working:
But I want to add grid lines to it, like so:
You can see that the lines to not match up with the bottom edges of the rects. Inserting the lines themselves is easy, you just start at zero and add 11 lines based on the height of the rects, which in this case will always be 11 / chart.effectiveHeight().
The reason they do not match up, seems to be that the top rect row does not always start at 0, instead, there seems to be a random(?) y position that the chart starts at, this will change with the height of the chart container, eg this y position starts at 5:
If it was consistent, then I could just start appending lines from that number instead of 0, but it is not. I have tried a couple of hacky work arounds, however I am unsure as to how to get the y position of all the rects after they are available in the DOM.
Interestingly the demo heatmap does not have this issue:
Here is the code for the heatmap:
const heat_map = dc.heatMap('#heatmap');
heat_map
.width(0)
.height(0)
.margins(margins)
.dimension(hm_dim)
.group(hm_group)
.keyAccessor(function(d) { return +d.key[0]; })
.valueAccessor(function(d) { return +d.key[1]; })
.colorAccessor(function(d) { return +d.value; })
.colors(color_scale)
.calculateColorDomain()
.yBorderRadius(0)
.xBorderRadius(0)
heat_map.render();
Is there a way to force the rects to begin at 0? Or get the random y position for the top rows? I did have a look at the source code but got a bit lost. Also I thought about creating a false group that would include each rect in the grid, and the grid lines could then be rect borders, but I thought that was a bit heavy handed.
Outlining the cells using CSS
It's easy to outline the cells using CSS:
rect.heat-box {
stroke-width: 1;
stroke: black;
}
Example fiddle.
However, as you point out, this only works if all the cells have values; crossfilter will not create the empty ones and I agree it would be absurd fill them in using a fake group just for some lines.
So, to answer your original question...
Why is there a gap at the top of the chart?
The heatmap calculates an integer size for the cells, and there may be space left over (since the space doesn't divide perfectly).
It's kind of nasty but the heatmap example avoids having extra space by calculating the width and height for the chart using the count of cells in each dimension:
chart
.width(45 * 20 + 80)
.height(45 * 5 + 40)
The default margins are {top: 10, right: 50, bottom: 30, left: 30} so this allocates 45x45 pixels for each cell and adds on the margins to get the right chart size.
Since the heatmap in this example draws 20 columns by 5 rows, it will calculate the cell width and height as 45.
Alternative Answer for Responsive/Resizable Charts
I am revisiting this question after rewriting my heatmap chart to be responsive - using the "ResizeObserver" method outlined in the dc.js resizing examples and Gordon's answer to this question
While specifying the chart width and height for the heatmap in Gordon's answer still works, it does not combine well with the resizing method because resized charts will have their .width and .height set to 'null'. Which means that this rounding issue will reoccur and the heat boxes will be again be offset by a random integer x or y value of anywhere between 0 and 5 (unless you want to write a custom resizing function for heatmaps).
The alternative answer is relatively simple and can be determined by selecting just one heat-box element in the heatmap.
The vertical offset value for the heat boxes is the remainder value when the heat-box y attribute is divided by the heat-box height attribute.
const heatbox_y = heat_map.select('.heat-box').attr('y);
const heatbox_height = heat_map.select('.heat-box').attr('height')
const vertical_offset = heatbox_y % heatbox_height
The modulus % will return the remainder.
The horizontal offset can be determined in the same way.
Thus you can append lines to the chart at regular intervals determined by the heatbox_height + the vertical_offset values.
This will work if you pick any heat-box in the chart, and so it is suitable for instances like this where you cannot guarantee that there will be a heat-box at each x or y level. And it means that you are free to set your chart height and width to 'null' if needed.
I'm looking for a way to plug in groups to my force-directed graph visualization. I've found three related examples so far:
Cola.js which would require adding another library and possibly retro-fitting my code to fit this different library.
This block, which is pretty hard to untangle.
This slide from mbostock's slide deck, which isn't what I want but on the right path...
What I'd like most is a simple way of adding something very close to the structure from the first link, but without too much overhead.
Right now I have a pretty standard setup:
var link = g.selectAll(".link")
.data(graph.links)
.enter().append("line")
.attr("class", "link")
.style(...
var node = g.selectAll(".node")
.data(graph.nodes)
.enter().append("g")
.attr("class", "node")
.attr("id", function(d) { return d.id; })
I was hoping to just grab the d3 code out of cola.js and mess with it, but that library seems fairly complicated so it wouldn't be too easy. I'm hoping it isn't too hard to get something kind of like this in straight d3:
Thanks!
I'm following the title "visualize groups of nodes" more than the suggested picture, but I think it wouldn't be that hard to tweak my answer to show bounding boxes as in the image
There's probably a few d3 only solutions, all of them almost certainly require tweaking the node positions manually to keep nodes grouped properly. The end result won't strictly be typical of a force-layout because links and node positions must be manipulated to show grouping in addition to connectivity - consquently, the end result will be a compromise between each force - node charge, length strength and length, and group.
The easiest way to accomplish your goal may be to:
Weaken link strength when links link different groups
On each tick, calculate each group's centroid
Adjust each node's position to move it closer to the group's centroid
Use a voronoi diagram to show the groupings
For my example here, I'll use Mike's canonical force layout.
Weaken links when links link different groups
Using the linked example, we can dampen the link strength when link target and link source have different groups. The specified strength will likely need to be altered depending on the nature of the force layout - more inter-connected groups will likely need to have weaker intergroup link strength.
To change the link strength depending on if we have an intergroup link or not, we might use:
var simulation = d3.forceSimulation()
.force("link", d3.forceLink().id(function(d) { return d.id; }).strength(function(link) {
if (link.source.group == link.source.target) {
return 1; // stronger link for links within a group
}
else {
return 0.1; // weaker links for links across groups
}
}) )
.force("charge", d3.forceManyBody().strength(-20))
.force("center", d3.forceCenter(width / 2, height / 2));
On Each Tick, Calculate Group Centroids
We want to force group nodes together, to do so we need to know the centroid of the group. The data structure of simulation.nodes() isn't the most amenable to calculating centroids, so we need to do a bit of work:
var nodes = this.nodes();
var coords ={};
var groups = [];
// sort the nodes into groups:
node.each(function(d) {
if (groups.indexOf(d.group) == -1 ) {
groups.push(d.group);
coords[d.group] = [];
}
coords[d.group].push({x:d.x,y:d.y});
})
// get the centroid of each group:
var centroids = {};
for (var group in coords) {
var groupNodes = coords[group];
var n = groupNodes.length;
var cx = 0;
var tx = 0;
var cy = 0;
var ty = 0;
groupNodes.forEach(function(d) {
tx += d.x;
ty += d.y;
})
cx = tx/n;
cy = ty/n;
centroids[group] = {x: cx, y: cy}
}
Adjust each node's position to move it closer to its group's centroid:
We don't need to adjust every node - just those that are straying fairly far from their centroids. For those that are sufficiently far we can nudge them closer using a weighted average of the centroid and the node's current position.
I modify the minimum distance used to determine if a node should be adjusted as the visualization cools. For the majority of the time when the visualization is active, when alpha is high, the priority is grouping, so most nodes will be forced towards the grouping centroid. As alpha drops towards zero, nodes should be grouped already, and the need to coerce their position is less important:
// don't modify points close the the group centroid:
var minDistance = 10;
// modify the min distance as the force cools:
if (alpha < 0.1) {
minDistance = 10 + (1000 * (0.1-alpha))
}
// adjust each point if needed towards group centroid:
node.each(function(d) {
var cx = centroids[d.group].x;
var cy = centroids[d.group].y;
var x = d.x;
var y = d.y;
var dx = cx - x;
var dy = cy - y;
var r = Math.sqrt(dx*dx+dy*dy)
if (r>minDistance) {
d.x = x * 0.9 + cx * 0.1;
d.y = y * 0.9 + cy * 0.1;
}
})
Use a Voronoi Diagram
This allows the easiest grouping of nodes - it ensures that there is no overlap between group shells. I haven't built in any verification to ensure that a node or set of node's aren't isolated from the rest of their group - depending on the visualization's complexity you might need this.
My initial thought was using a hidden canvas to calculate if shells overlapped, but with a Voronoi you could probably calculate if each group is consolidated using neighboring cells. In the event of non-consolidated groups you could use a stronger coercion on stray nodes.
To apply the voronoi is fairly straightforward:
// append voronoi
var cells = svg.selectAll()
.data(simulation.nodes())
.enter().append("g")
.attr("fill",function(d) { return color(d.group); })
.attr("class",function(d) { return d.group })
var cell = cells.append("path")
.data(voronoi.polygons(simulation.nodes()))
And update on each tick:
// update voronoi:
cell = cell.data(voronoi.polygons(simulation.nodes())).attr("d", renderCell);
Results
Altogether, this looks like this during the grouping phase:
And as the visualization finally stops:
If the first image is preferable, then remove the part the changes the minDistance as alpha cools down.
Here's a block using the above method.
Further Modification
Rather than using the centroid of each group's nodes, we could use another force diagram to position the ideal centroid of each group. This force diagram would have a node for each group, the strength of links between each group would correspond to te number of links between the nodes of the groups. Using this force diagram, we could coerce the original nodes towards our idealized centroids - the nodes of the second force layout.
This approach may have advantages in certain situations, such as by separating groups by greater amounts. This approach might give you something like:
I've included an example here, but hope that the code is commented sufficiently to understand without a breakdown like the above code.
Block of second example.
The voronoi is easy, but not always the most aesthetic, you could use a clip path to keep clip the polygons to some sort of oval, or use a gradient overlay to fade the polygons out as they reach the edges. One option that is likely possible depending on graph complexity is using a minimum convex polygon instead, though this won't work well with groups with less than three nodes. Bounding box's probably won't work in most instances, unless you really keep the coercion factor high (eg: keep minDistance very low the entire time). The trade off will always be what do you want to show more: connections or grouping.
I would like to load in additional data points for the graph only if it scales or translates. Suppose I have a graph for the specific time range (http://codepen.io/jayarjo/pen/gzfyj), now if the user pans or zooms it to a wider range I want to load a wider data corresponding to that range and plot additional data points. I'm not sure how to get the currently visible date range from the D3...?
Simply retrieve inverted values of the beginning and ending points of your scale, which apparently are 0 and width of the scale:
var startDate = x.invert(0);
var endDate = x.invert(width); // where width is the visible width of the x scale
Well, I'm starting with D3 and I'm trying to create a vertical bar chart.
I really don't know what's happening but some things are not working as expected for me (maybe because I'm just a noob on the matter).
I'm using line scales, works pretty well with axes, but it's miscalculating the height of the bars, for instance the higher values are not displayed (because of the low value of the result).
I've used the d3.max to determine the range. I really don't get what's happening.
var yScaleLeft = d3.scale.linear()
.domain([0, d3.max(stats)])
.range([realHeight, 0]);
.attr("height", function(d) {
return yScaleLeft(d);
});
Here is the code: http://jsfiddle.net/aKhhb/ Look at * Scales * and // Stats bars
(Just forget about the x-alignement of the bars, I will see that later, I want to set its height properly)
Thanks a lot! Y saludos desde Chile :)
The issue is that your input and output ranges are mirrored -- that is, the largest input value maps to the smallest output value. That is fine, but you need to take it into account when calculating the y and height attributes. Essentially, you had the calculations for both reversed.
Fixed fiddle here. I've also fixed the x axis by adding your margin and half of the bar width to the computed x positions. Oh and you don't need parseInt() when doing calculations, only when you actually want to parse an integer from a string.
Following is the stripped down version is what I'm using to generate histograms using d3 and a bit of jQuery.http://bl.ocks.org/4611158
While most of it might seem right, I'm still confused regarding
Why there is no '14' in the x-axis as should have been for the given input in the above example? Instead 13 gets the ordinate of what should have been 14's
In my trials d3.layout.histogram() assigned negative(and hence non-plot table) widths when I try altering the output range of scale to some non-zero value. why is it so? what is the possible workaround?
My main motive to use ordinal scale was to make ticks centrally aligned below the bars, unlike what Mike used in his demo for histograms. I've also made the number of bins equal to the number of ticks in d3.layout.histogram() for the very same purpose. I'm sure there might be a better way around to code what I'm looking for
Also any ideas how to add a 'graph' of indicator lines like its been done in nvd3 visualization (light gray in background )that will make it more pleasing?
There is no 14 and there are two 8s on the x-axis. This is because the bins function will diligently divide the range = 14 - 1 = 13 into 14 intervals as per the API reference:
The bins may be specified as a number, in which case the range of values will be split
uniformly into the given number of bins. Or, bins may be an array of threshold values,
defining the bins; the specified array must contain the rightmost (upper) value, thus
specifying n + 1 values for n bins. ...
Before solving this issue, I am guessing that the second problem you are facing is that if the rangeDefault has negative values, then some values are not plotted. To fix that problem, being unaware of the exact need of it, I will start by removing the following:
rangeDefault[0] = 0; //All histograms start from 0 <-- REMOVED
Then to fix the first problem, use the second form of arguments for binsas shown here:
var bins = [];
for(var ii = settings.range[0], jj = 0; ii <= settings.range[1] + 1; ii++, jj++)
bins[jj] = ii;
var data = d3.layout.histogram()
.bins(bins)(settings.data);
I hope this addresses the primary queries.
Adding the light grey indicator lines is fairly easy, as shown here. The changes were:
vis.css
.y.axis line.tick { opacity: .3; }
vis.js
Moving the axis before the chart in the DOM because of how SVG is laid out affects its z-index:
var gEnter = svg.enter().append("svg").append("g");
gEnter.append("g").attr("class", "x axis");
gEnter.append("g").attr("class","y axis");
gEnter.append("g").attr("class", "bars");
And finally making the major tickSize on the y-axis -(width - margin.right - margin.left):
yAxis = d3.svg.axis()
.scale(y)
.orient("left")
.tickSubdivide(true)
.tickPadding(5)
.ticks(10)
.tickSize(-(width - margin.right - margin.left), 2, 8);