editable scatter chart using dc.js - dc.js

What is a good approach to have a scatter plot in which the data can be edited in the plot itself with a click action?
The idea is to spot outliers in the data in the plot and filter the values in the plot itself, rather than having to change the source data.
Even better would be to remove the data from the crossfilter, but a solution that just filters is acceptable.

I've come up with a solution using the current dc.js (beta 32).
It does not support the brush (needs to have .brushOn(false)) - I'll explain in the enhancement request why this would need some changes to dc.js.
But it does support clicking on points to toggle them, and the reset link. (Clicking on the background to reset is also possible but not implemented here.)
What we'll do is define our own ExcludePointsFilter with the standard dc.js filter signature:
function compare_point(p1, p2) {
return p1[0] === p2[0] && p1[1] === p2[1];
}
function has_point(points, point) {
return points.some(function(p) {
return compare_point(point, p);
});
}
function ExcludePointsFilter(points) {
var points2 = points.slice(0);
points2.filterType = 'ExcludePointsFilter';
points2.isFiltered = function(k) {
return !has_point(points2, k);
};
return points2;
}
We'll calculate a new set of points each time one is clicked, and replace the filter:
scatterPlot.on('pretransition.exclude-dots', function() { #1
// toggle exclusion on click
scatterPlot.selectAll('path.symbol') #2
.style('cursor', 'pointer') // #3
.on('click.exclude-dots', function(d) { // #4
var p = [d.key[0],d.key[1]];
// rebuild the filter #5
var points = scatterPlot.filter() || [];
if(has_point(points, p))
points = points.filter(function(p2) {
return !compare_point(p2, p);
});
else
points.push(p);
// bypass scatterPlot.filter, which will try to change
// it into a RangedTwoDimensionalFilter #6
scatterPlot.__filter(null)
.__filter(ExcludePointsFilter(points));
scatterPlot.redrawGroup();
});
});
Explanation:
Every time the chart is rendered or redrawn, we'll annotate it before any transitions start
Select all the dots, which are path elements with the symbol class
Set an appropriate cursor (pointer-hand may not be ideal but there aren't too many to choose from)
Set up a click handler for each point - use the exclude-dots event namespace to make sure we're not interfering with anyone else.
Get the current filter or start a new one. Look to see if the current point being clicked on (passed as d) is in that array, and either add it or remove it depending.
Replace the current filters for the scatterplot. Since the scatter plot is deeply wed to the RangedTwoDimensionalFilter, we need to bypass its filter override (and also the coordinateGridMixin override!) and go all the way to baseMixin.filter(). Yes this is weird.
For good measure, we'll also replace the filter printer, which normally doesn't know how to deal with an array of points:
scatterPlot.filterPrinter(function(filters) {
// filters will contain 1 or 0 elements (top map/join is just for safety)
return filters.map(function(filter) {
// filter is itself an array of points
return filter.map(function(p) {
return '[' + p.map(dc.utils.printSingleValue).join(',') + ']';
}).join(',');
}).join(',');
});
Here is a working example in a fiddle: http://jsfiddle.net/gordonwoodhull/3y72o0g8/16/
Note, if you then want to do something with the excluded points, you can read them from scatterPlot.filter() - the filter is the array of points with some annotation. You may even be able to reverse the filter and then call crossfilter.remove() but I'll leave that as an exercise.

Related

Unable to filter individual stacks using dc.js with multiple X keys

Stacked Bar chart not able to filter on click of any Stack
I need to filter all the charts when clicking on any stack, which is not happening and struggling for a few days.
I've created a fiddle with link
http://jsfiddle.net/praveenNbd/09t5fd7v/13/
I feel am messing up with keys creation as suggested by gordonwoodhull.
function stack_second(group) {
return {
all: function () {
var all = group.all(),
m = {};
// build matrix from multikey/value pairs
all.forEach(function (kv) {
var ks = kv.key;
m[ks] = kv.value;
});
// then produce multivalue key/value pairs
return Object.keys(m).map(function (k) {
return {
key: k,
value: m[k]
};
});
}
};
}
I tried to follow this example https://dc-js.github.io/dc.js/examples/filter-stacks.html
Not able to figure out how below code works:
barChart.on('pretransition', function (chart) {
chart.selectAll('rect.bar')
.classed('stack-deselected', function (d) {
// display stack faded if the chart has filters AND
// the current stack is not one of them
var key = multikey(d.x, d.layer);
//var key = [d.x, d.layer];
return chart.filter() && chart.filters().indexOf(key) === -1;
})
.on('click', function (d) {
chart.filter(multikey(d.x, d.layer));
dc.redrawAll();
});
});
Can someone please point me out in the right direction.
Thanks for stopping by.
You usually don't want to use multiple keys for the X axis unless you have a really, really good reason. It is just going to make things difficult
Here, the filter-stacks example is already using multiple keys, and your data also has multiple keys. If you want to use your data with this example, I would suggest crunching together the two keys, since it looks like you are really using the two together as an ordinal key. We'll see one way to do that below.
You were also trying to combine two different techniques for stacking the bars, stack_second() and your own custom reducer. I don't think your custom reducer will be compatible with filtering by stacks, so I will drop it in this answer.
You'll have to use the multikey() function, and crunch together your two X keys:
dim = ndx.dimension(function (d) {
return multikey(d[0] + ',' + d[1], d[2]);
});
Messy, as this will create keys that look like 0,0xRejected... not so human-readable, but the filter-stacks hack relies on being able to split the key into two parts and this will let it do that.
I didn't see any good reason to use a custom reduction for the row chart, so I just used reduceCount:
var barGrp = barDim.group();
I found a couple of new problems when working on this.
First, your data doesn't have every stack for every X value. So I added a parameter to stack_second() include all the "needed" stacks:
function stack_second(group, needed) {
return {
all: function() {
var all = group.all(),
m = {};
// build matrix from multikey/value pairs
all.forEach(function(kv) {
var ks = splitkey(kv.key);
m[ks[0]] = m[ks[0]] || Object.fromEntries(needed.map(n => [n,0]));
m[ks[0]][ks[1]] = kv.value;
});
// then produce multivalue key/value pairs
return Object.entries(m).map(([key,value]) => ({key,value}));
}
};
}
Probably the example should incorporate this change, although the data it uses doesn't need it.
Second, I found that the ordinal X scale was interfering, because there is no way to disable the selection greying behavior for bar charts with ordinal scales. (Maybe .brushOn(false) is completely ignored? I'm not sure.)
I fixed it in the pretransition handler by explicitly removing the built-in deselected class, so that our custom click handler and stack-deselected class can do their work:
chart.selectAll('rect.bar')
.classed('deselected', false)
All in all, I think this is way too complicated and I would advise not to use multiple keys for the X axis. But, as always, there is a way to make it work.
Here is a working fork of your fiddle.

Stackbar chart in dcjs split by dimension

I am trying to create a stack bar chart in dcjs. The dcjs stack bar examples are quite clear the huge difference from the barchart with that of stack is that the stack function. The stack function takes the same group as input and it can take third parameter as function which decides by which value it has to split. I rather want a dimension to be split the entire bar chart.
Lets say the following data point is something like this
data = [
{activity:"A1",time_taken:10,activity_group:"Master A"},
{activity:"A2",time_taken:20,activity_group:"Master B"},
{activity:"A1",time_taken:30,activity_group:"Master C"},
{activity:"A2",time_taken:15,activity_group:"Master D"}
]
I want to have activity group in x-axis split by its activity representing time taken on y-axis, like this:
How do I achieve this ?
Your fiddle is on dc.js version 1.7, which is more than five years old and not something I can wrap my head around. :-/
I converted it to dc.js version 2, which also uses D3 v3.
dc.js is not great at showing the raw data, it's more about showing aggregated data. But in this case it could make sense to create a stack for every activity_group; that way it will automatically be assigned its own color.
Using ES6 we can get a list of all activity_group like this:
const stacks = [...new Set(data.map(row => row.activity_group))];
Now let's aggregate the data by stack:
var groupActivity = dimByActivity.group().reduce(
function reduceAdd(p, v) {
p[v.activity_group] += v.time_taken;
return p;
},
function reduceRemove(p, v) {
p[v.activity_group] -= v.time_taken;
return p;
},
function reduceInitial() {
return Object.fromEntries(stacks.map(stack => [stack,0]));
});
This is substantially the same as the stacked bar example except that we have a stack per activity_group.
Note that we are creating all the stacks in every bin, just leaving them zero where they don't exist. This is because dc.js expects the same stacks for every X value - it won't work otherwise.
As in the stacked bar example, we'll programmatically add the stacks to the chart, keeping in mind that we need to use .group() for the first stack:
function sel_stack(valueKey) {
return function(d) {
return d.value[valueKey];
};
}
// ...
stacks.forEach(function(stack, i) {
if(i===0)
chanUtil.group(groupActivity, stack, sel_stack(stack));
else
chanUtil.stack(groupActivity, stack, sel_stack(stack));
})
Here's the output. I messed a little with the margins and height in order to get the legend not to overlap and there are probably smarter ways to deal with this:
Fork of your fiddle.
As I said, this is making dc.js do something it doesn't want to do, so YMMV!

dc.js Weird mouse zooming for seriesChart

I want to use the mouse zoom functionality on seriesChart and have it filter for other charts of the same group.
When I enable the zoom with .mouseZoomable(true) on seriesChart, and zoom the chart, the other charts become empty.
This doesn't happen when I enable it on a LineChart.
Here is a simple example: https://codepen.io/udeste/pen/ZKeXmX
(Zoom the second chart with the mouse. All is working. But when you zoom the first chart the other charts go blank.)
What am I doing wrong? Is it a dc.seriesChart bug?
It's because dc.seriesChart required you to supply that strange dimension, but it didn't change the filter function accordingly.
You specified seriesDimension like so:
var seriesDimension = ndx.dimension(function(d) {
return [+d.Expt, +d.Hours];
});
But when you zoom, the dc.coordinateGridMixin just filters using a regular dc.filters.RangedFilter, which does not know about these kinds of two-dimensional "multikeys".
Probably since the series chart requires this kind of input, it should redefine the filter handler to also deal with multikeys. But until then, you can work around it by providing your own filterHandler:
seriesChart.filterHandler(function(dimension, filters) {
if(filters.length === 0) // 1
dimension.filter(null);
else {
console.assert(filters.length===1); // 2
console.assert(filters[0].filterType==='RangedFilter');
dimension.filter(function(d) { // 3
return filters[0][0] <= d[1] && d[1] < filters[0][1];
})
}
});
What this does:
Checks if this event is because the filters have been cleared, and clears the dimension's filter if so.
Asserts that the filter is what is expected. coordinateGridMixin will always supply a single dc.filters.RangedFilter but who knows what else could happen.
Supplies a substitute filter function that checks if the part of the key used by the keyAccessor falls within the range (instead of comparing the array with the range, which will always return false).
Here's a working fork of your codepen.
(Incidentally, it looks like this examples slams into a known issue where line segments off the edge of the chart are dropped instead of clipping the segments. It won't be quite as bad if there are more points. I don't think I've found a good workaround. Hopefully we'll fix this soon.)

Trouble with filters and triggers in dc.js - re-drawing is out of sync

I have two plots: a line plot and a bubble plot. When I click on the bubble plot, I want the line plot to be updated so that it is drawn with only the data related to that 'bubble'. This is different from the standard implementation whereby clicking would add or remove the data from the existing filter.
If you look at the image you can see that although 'model 0' is selected the plotted hazard (y-scale in plot 1) does not correspond.
And now when I click on 'model 5', I get the opposite.
My current implementation is posted as a jsfiddle here. I can see from the attached data table that I am achieving what I want, but the line plot does not re-draw correctly. In fact, it seems to re-draw with the last filter, not the new one.
This implementation is hacked from here: in particular, the renderlet and on("filtered", function (chart) { lines. However, to make this work, I have had to comment out the plot1.filter(chart.filter()); line for the second plot.
I don't really understand why a renderlet and the on("filtered" ... or on("postRedraw" ... listeners are needed together.
I have been round the houses on this one, so any suggestions would be very gratefully received.
I tried to simplify the jsfiddle to isolate the problem. Here is the adapted jsfiddle: http://jsfiddle.net/djmartin_umich/mKz7A/
Your plot2 keyAccessor accessed the df value from the p.value.df rather than using a dimension on df. My guess is that this is what was causing problems. Here is the adapted code:
dfDim = ndx.dimension(function (d) {return d.df;});
...
plot2.width(300)
.height(250)
.dimension(dfDim)
I also noticed that your plot2 valueAccessor and radiusAccessor were not using a computed average. Your code would overwrite est and estse for each record added or removed from the group. Here is the adapted code that computes the average:
dfGroup = dfDim.group().reduce(
//add
function (p, v) {
++p.count;
p.est += v.est;
p.avg_est = p.est / p.count;
p.estse += v.estse;
p.avg_estse = p.estse / p.count;
return p;
},
//remove
function (p, v) {
--p.count;
p.est -= v.est;
p.avg_est = p.est / p.count;
p.estse -= v.estse;
p.avg_estse = p.estse / p.count;
return p;
},
//init
function (p, v) {
return {
count: 0,
est: 0,
estse: 0,
avg_est: 0,
avg_estse: 0
};
});
After these changes, I believe the code behaves as you wanted.

Is it possible to create pie charts with object consistency?

the pie chart update example on the bl.ocks site doesn't update the elements 'in place':
http://bl.ocks.org/j0hnsmith/5591116
function change() {
clearTimeout(timeout);
path = path.data(pie(dataset[this.value])); // update the data
// set the start and end angles to Math.PI * 2 so we can transition
// anticlockwise to the actual values later
path.enter().append("path")
.attr("fill", function (d, i) {
return color(i);
})
.attr("d", arc(enterAntiClockwise))
.each(function (d) {
this._current = {
data: d.data,
value: d.value,
startAngle: enterAntiClockwise.startAngle,
endAngle: enterAntiClockwise.endAngle
};
}); // store the initial values
path.exit()
.transition()
.duration(750)
.attrTween('d', arcTweenOut)
.remove() // now remove the exiting arcs
path.transition().duration(750).attrTween("d", arcTween); // redraw the arcs
}
Instead, it just treats the new array of value as brand new data and resizes the chart accordingly.
I've created a fiddle demonstrating the issue very simply:
http://jsfiddle.net/u9GBq/23/
If you press 'add', it add a random int to the array: this works as intended.
If you press 'remove', the only element getting transitioned out is always the last element to have entered the pie. In short, it behaves like a LIFO stack.
The expected behaviour is for the relevant pie arc to get transitioned out instead.
Is it possible to apply object consistency to pies? I've also tried adding a key function (not demonstrated on the fiddle) but that just breaks (oddly enough it works fine with my stacked graphs).
Thank you.
The easiest solution to this problem is to set missing values to zero, rather than removing them entirely, as in Part III of the Pie Chart Update series of examples. Then you get object constancy for free: you have the same number of elements, in the same order, across updates.
Alternatively, if you want a data join as in Part IV, you have to tell D3 where the entering arcs should enter from, and where the exiting arcs should exit to. A reasonable strategy is to find the closest neighboring arc from the opposite data: for a given entering arc, find the closest neighboring arc in the old data (pre-transition); likewise for a given exiting arc, find the closest neighboring arc in the new data (post-transition).
To continue the example, say you’re showing sales of apples in different regions, and want to switch to show oranges. You could use the following key function to maintain object constancy:
function key(d) {
return d.data.region;
}
(This assumes you’re using d3.layout.pie, which wraps your original data and exposes it as d.data.)
Now say when you transition to oranges, you have the following old data and new data:
var data0 = path.data(), // retrieve the old data
data1 = pie(region.values); // compute the new data
For each entering arc at index i (where d is data1[i]), you can step sequentially through preceding data in data1, and see if you can find a match in data0:
var m = data0.length;
while (--i >= 0) {
var k = key(data1[i]);
for (var j = 0; j < m; ++j) {
if (key(data0[j]) === k) return data0[j]; // a match!
}
}
If you find a match, your entering arcs can start from the matching arc’s end angle. If you don’t find a preceding match, you can then look for a following matching arc instead. If there are no matches, then there’s no overlap between the two datasets, so you might enter the arcs from angle 0°, or do a crossfade. You can likewise apply this technique to exiting arcs.
Putting it all together, here’s Part V:
Ok, found the solution.
The trick was to pass the key this way:
path = path.data(pie(dataset), function (d) {return d.data}); // this is good
as opposed to not passing it, or passing it the wrong way:
path = path.data(pie(dataset, function (d) {return d.data})); // this is bad
And here's an updated fiddle with a working transition on the right arc! :)
http://jsfiddle.net/StephanTual/PA7WD/1/

Resources