I'm using code similar to that in the dc.js annotated example:
var ndx = crossfilter(data);
...
var dayName=["0.Sun","1.Mon","2.Tue","3.Wed","4.Thu","5.Fri","6.Sat"];
var dayOfWeek = ndx.dimension(function (d) {
var day = d.dd.getDay();
return dayName[day];
});
var dayOfWeekGroup = dayOfWeek.group();
var dayOfWeekChart = dc.rowChart("#day-of-week-chart");
dayOfWeekChart.width(180)
.height(180)
.group(dayOfWeekGroup)
.label(function(d){return d.key.substr(2);})
.dimension(dayOfWeek);
The issue I've got is that only days of the week present in the data are displayed in my rowChart, and there's no guarantee every day will be represented in all of my data sets.
This is desirable behaviour for many types of categories, but it's a bit disconcerting to omit them for short and well-known lists like day and month names and I'd rather an empty row was included instead.
For a barChart, I can use .xUnits(dc.units.ordinal) and something like .x(d3.scale.ordinal.domain(dayName)).
Is there some way to do the same thing for a rowChart so that all days of the week are displayed, whether present in data or not?
From my understanding of the crossfilter library, I need to do this at the chart level, and the dimension is OK as is. I've been digging around in the dc.js 1.6.0 api reference, and the d3 scales documentation but haven't had any luck finding what I'm looking for.
Solution
Based on #Gordon's answer, I've added the following function:
function ordinal_groups(keys, group) {
return {
all: function () {
var values = {};
group.all().forEach(function(d, i) {
values[d.key] = d.value;
});
var g = [];
keys.forEach(function(key) {
g.push({key: key,
value: values[key] || 0});
});
return g;
}
};
}
Calling this as follows will fill in any missing rows with 0s:
.group(ordinal_groups(dayNames, dayOfWeekGroup))
Actually, I think you are better off making sure that the groups exist before passing them off to dc.js.
One way to do this is the "fake group" pattern described here:
https://github.com/dc-js/dc.js/wiki/FAQ#filter-the-data-before-its-charted
This way you can make sure the extra entries are created every time the data changes.
Are you saying that you tried adding the extra entries to the ordinal domain and they still weren't represented in the row chart, whereas this did work for bar charts? That sounds like a bug to me. Specifically, it looks like support for ordinal domains needs to be added to the row chart.
Related
Stacked Bar chart not able to filter on click of any Stack
I need to filter all the charts when clicking on any stack, which is not happening and struggling for a few days.
I've created a fiddle with link
http://jsfiddle.net/praveenNbd/09t5fd7v/13/
I feel am messing up with keys creation as suggested by gordonwoodhull.
function stack_second(group) {
return {
all: function () {
var all = group.all(),
m = {};
// build matrix from multikey/value pairs
all.forEach(function (kv) {
var ks = kv.key;
m[ks] = kv.value;
});
// then produce multivalue key/value pairs
return Object.keys(m).map(function (k) {
return {
key: k,
value: m[k]
};
});
}
};
}
I tried to follow this example https://dc-js.github.io/dc.js/examples/filter-stacks.html
Not able to figure out how below code works:
barChart.on('pretransition', function (chart) {
chart.selectAll('rect.bar')
.classed('stack-deselected', function (d) {
// display stack faded if the chart has filters AND
// the current stack is not one of them
var key = multikey(d.x, d.layer);
//var key = [d.x, d.layer];
return chart.filter() && chart.filters().indexOf(key) === -1;
})
.on('click', function (d) {
chart.filter(multikey(d.x, d.layer));
dc.redrawAll();
});
});
Can someone please point me out in the right direction.
Thanks for stopping by.
You usually don't want to use multiple keys for the X axis unless you have a really, really good reason. It is just going to make things difficult
Here, the filter-stacks example is already using multiple keys, and your data also has multiple keys. If you want to use your data with this example, I would suggest crunching together the two keys, since it looks like you are really using the two together as an ordinal key. We'll see one way to do that below.
You were also trying to combine two different techniques for stacking the bars, stack_second() and your own custom reducer. I don't think your custom reducer will be compatible with filtering by stacks, so I will drop it in this answer.
You'll have to use the multikey() function, and crunch together your two X keys:
dim = ndx.dimension(function (d) {
return multikey(d[0] + ',' + d[1], d[2]);
});
Messy, as this will create keys that look like 0,0xRejected... not so human-readable, but the filter-stacks hack relies on being able to split the key into two parts and this will let it do that.
I didn't see any good reason to use a custom reduction for the row chart, so I just used reduceCount:
var barGrp = barDim.group();
I found a couple of new problems when working on this.
First, your data doesn't have every stack for every X value. So I added a parameter to stack_second() include all the "needed" stacks:
function stack_second(group, needed) {
return {
all: function() {
var all = group.all(),
m = {};
// build matrix from multikey/value pairs
all.forEach(function(kv) {
var ks = splitkey(kv.key);
m[ks[0]] = m[ks[0]] || Object.fromEntries(needed.map(n => [n,0]));
m[ks[0]][ks[1]] = kv.value;
});
// then produce multivalue key/value pairs
return Object.entries(m).map(([key,value]) => ({key,value}));
}
};
}
Probably the example should incorporate this change, although the data it uses doesn't need it.
Second, I found that the ordinal X scale was interfering, because there is no way to disable the selection greying behavior for bar charts with ordinal scales. (Maybe .brushOn(false) is completely ignored? I'm not sure.)
I fixed it in the pretransition handler by explicitly removing the built-in deselected class, so that our custom click handler and stack-deselected class can do their work:
chart.selectAll('rect.bar')
.classed('deselected', false)
All in all, I think this is way too complicated and I would advise not to use multiple keys for the X axis. But, as always, there is a way to make it work.
Here is a working fork of your fiddle.
I'm trying to implement a live data visualization (i.e. with new data arriving periodically) using dc.js. The problem I'm having is the following - when new data is added to the plot, already existing points often start to "dance around", even though they were not changed. Can this be avoided?
The following fiddle illustrates this.
My guess is that crossfilter sorts data internally, which results in points moving on the chart for data items that changed their position (index) in the internal storage. Data is added in the following way:
var data = [];
var ndx = crossfilter(data)
setInterval(function() {
var value = ndx.size() + 1;
if (value > 50) {
return;
}
var newElement = {
x: myRandom(),
y: myRandom()
};
ndx.add([newElement]);
dc.redrawAll();
}, 1000);
Any ideas?
I stand by my comments above. dc.js should be fixed by binding the data using a key function, and probably the best way to deal with the problem is just to disable transitions on the scatterplot using .transitionDuration(0)
However, I was curious if it was possible to work around the current problems by keeping the group in a set order using a fake group. And it is indeed, at least for this example where there is no aggregation and we just want to display the original data points.
First, we add a third field, index, to the data. This has to order the data in the same order in which it comes in. As noted in the discussion above, the scatter plot is currently binding data by its index, so we need to keep the points in a set order; nothing should be inserted.
var newElement = {
index: value,
x: myRandom(),
y: myRandom()
};
Next, we have to preserve this index through the binning and aggregation. We could keep it either in the key or in the value, but keeping it in the key seems more fitting:
xyiDimension = ndx.dimension(function(d) {
return [+d.x, +d.y, d.index];
}),
xyiGroup = xyiDimension.group();
The original reduction didn't make sense to me, so I dropped it. We'll just use the default behavior, which counts the number of rows which fall into each bin. The counts should be 1 if included, or 0 if filtered out. Including the index in the key also ensures uniqueness, which the original keys were not guaranteed to have.
Now we can create a fake group that keeps everything sorted by index:
var xyiGroupSorted = {
all: function() {
var ret = xyiGroup.all().slice().sort((a,b) => a.key[2] - b.key[2]);
return ret;
}
}
This will fetch the original data whenever it's requested by the chart, create a copy of the array (because the original is owned by crossfilter), and sort it to return it to the correct order.
And voila, we have a scatter plot that behaves the way it should, even though the data has gone through crossfilter.
Fork of your fiddle: https://jsfiddle.net/gordonwoodhull/mj81m42v/13/
[After all this, maybe we shouldn't have given the data to crossfilter in the first place! We could have just created a fake group which exposes the original data. But maybe there's some use to this technique. At least it proves that there's almost always a way to work around any problems in dc.js & crossfilter.]
I've been trying to use dc.js and crossfilter to both build charts and tables from a certain dataset.
So far building charts works fine, but I want to use the datatable functionality to build a small html table to summarize the data as follows:
|Year|TotalEmployees|
|2015|555|
|2016|666|
|2017|777|
My dataset has around 20 000 rows, here's a sample of the data:
var data = [
{"Year":"2015","Category":"1","NbEmployee":"51"},
{"Year":"2015","Category":"2","NbEmployee":"31"},
{"Year":"2015","Category":"3","NbEmployee":"14"}
{"Year":"2016","Category":"1","NbEmployee":"51"},
{"Year":"2016","Category":"2","NbEmployee":"55"},
{"Year":"2016","Category":"3","NbEmployee":"65"},
{"Year":"2017","Category":"1","NbEmployee":"76"},
{"Year":"2017","Category":"2","NbEmployee":"98"},
];
So far this piece of code returns one row of result per row of data, and although it feels like it should be a simple manipulation, I can't figure out the right syntax to build a summarized table with one row per year:
var ndx = crossfilter(data);
var tableDim = ndx.dimension(function(d) {
return d.Year;
});
var datatable = dc.dataTable("#dc-data-table");
datatable
.dimension(tableDim)
.group(function(d) {
d.NbEmployee += d.NbEmployee;
return d.Year;
})
.columns([
function(d) {return d.Year;},
function(d) {return d.NbEmployee;},
]);
I've tried countless times to apply the
.group().reduceSum()
functions to the dimension into a variable and then passing it to the .group() parameter, but I always end up with a compilation error, I'm pretty clueless right now.
The SQL translation of what I'm looking for is this:
SELECT
Year,
NbEmp = SUM(NbEmploye)
FROM DB
GROUP BY
Year
ORDER BY
Year
Thanks in advance for your help!
The dataTable's group is not a group - yes, pretty confusing to use this method to mean something completely different from what it means in all the other charts. Here, it's a function, everywhere else it's a crossfilter object.
The dataTable is unique out of the dc.js charts in that it reads its data from the .dimension() object. This is because it displays the raw rows of data, rather than aggregated data, by default.
However, it can be used to display a group instead. This works because the only method it actually calls on the dimension is .top(), if you choose to display in descending order.
If you want to display in ascending order, you can use a fake group to produce an object which supports the .bottom() method.
I'm using the grouped bar PR of dc.js and the corresponding grouped bar chart example as a baseline.
For some reason, I have to use numbers in my data as opposed to strings. (Convert "male" and "female" to 1/0). I'm guessing it has to do with the reduce functions I'm using. This also effects my x-axis labels, of course. I'd rather they show the text variations.
ndx = crossfilter(eData),
groupDim = ndx.dimension(function(d) {return d.service;}),
qtySumGroup = groupDim.group().reduce(
function(p,v) { p[v.component] = (p[v.component] || 0) + v.qty; return p; },
function(p,v) { p[v.component] = (p[v.component] || 0) - v.qty; return p; },
function() { return{}; });
I'm also noticing that it doesn't seem to crossfilter the data. When I click one of the bars in a group, it doesn't filter my other charts on the page. What am I missing?
Here's the first part of the answer. In order to use string components/genders for grouping, you'll need to adjust the way data is selected for "stacking" (actually grouping when this version of dc.js is used).
So, you can grab the component names by first walking the data and grabbing the components:
var components = Object.keys(etsData.reduce(function(p, v) {
p[v.component] = 1;
return p;
}, {}));
This builds an object where the keys are the component names, and then pulls just the keys as an array.
Then we use components to select the categories like so:
grpChart
.group(qtySumGroup, components[0], sel_stack(components[0]));
for(var i=1; i<components.length; ++i)
grpChart.stack(qtySumGroup, components[i], sel_stack(components[i]));
This is just the same as the original
grpChart
.group(qtySumGroup, "1", sel_stack('1'));
for(var i=2; i<6; ++i)
grpChart.stack(qtySumGroup, ''+i, sel_stack(i));
except that it is indexing by string instead of integer.
I realize this is not the important part of your question, but unfortunately filtering by stack segments is not currently supported in dc.js. I'll try to return to that part later today if I have time - it should be possible to hack it in using a dimension with compound keys (or using two dimensions) and a custom click event, but I haven't seen anyone try this yet.
It would no doubt be a helpful feature to add to dc.js, even if just as an external customization.
EDIT: I've added an example of filtering the segments of a stack, which should apply equally well for grouped bars (although I haven't tried it with your code). The technique is explained in the relevant dc.js issue.
Good Evening Everyone,
I'm trying to take the data from a database full of hour reports (name, timestamp, hours worked, etc.) and create a plot using dc.js to visualize the data. I would like the timestamp to be on the x-axis, the sum of hours for the particular timestamp on the y-axis, and a new bar graph for each unique name all on the same chart.
It appears based on my objectives that using crossfilter.js the timestamp should be my 'dimension' and then the sum of hours should be my 'group'.
Question 1, how would I then use the dimension and group to further split the data based on the person's name and then create a bar graph to add to my composite graph? I would like for the crossfilter.js functionality to remain intact so that if I add a date range tool or some other user controllable filter, everything updates accordingly.
Question 2, my timestamps are in MySQL datetime format: YYYY-mm-dd HH:MM:SS so how would I go about dropping precision? For instance, if I want to combine all entries from the same day into one entry (day precision) or combine all entries in one month into a single entry (month precision).
Thanks in advance!
---- Added on 2017/01/28 16:06
To further clarify, I'm referencing the Crossfilter & DC APIs alongside the DC NASDAQ and Composite examples. The Composite example has shown me how to place multiple line/bar charts on a single graph. On the composite chart I've created, each of the bar charts I've added a dimension based off of the timestamps in the data-set. Now I'm trying to figure out how to define the groups for each. I want each bar chart to represent the total time worked per timestamp.
For example, I have five people in my database, so I want there to be five bar charts within the single composite chart. Today all five submitted reports saying they worked 8 hours, so now all five bar charts should show a mark at 01/28/2017 on the x-axis and 8 hours on the y-axis.
var parseDate = d3.time.format('%Y-%m-%d %H:%M:%S').parse;
data.forEach(function(d) {
d.timestamp = parseDate(d.timestamp);
});
var ndx = crossfilter(data);
var writtenDimension = ndx.dimension(function(d) {
return d.timestamp;
});
var hoursSumGroup = writtenDimension.group().reduceSum(function(d) {
return d.time_total;
});
var minDate = parseDate('2017-01-01 00:00:00');
var maxDate = parseDate('2017-01-31 23:59:59');
var mybarChart = dc.compositeChart("#my_chart");
mybarChart
.width(window.innerWidth)
.height(480)
.x(d3.time.scale().domain([minDate,maxDate]))
.brushOn(false)
.clipPadding(10)
.yAxisLabel("This is the Y Axis!")
.compose([
dc.barChart(mybarChart)
.dimension(writtenDimension)
.colors('red')
.group(hoursSumGroup, "Top Line")
]);
So based on what I have right now and the example I've provided, in the compose section I should have 5 charts because there are 5 people (obviously this needs to be dynamic in the end) and each of those charts should only show the timestamp: total_time data for that person.
At this point I don't know how to further breakup the group hoursSumGroup based on each person and this is where my Question #1 comes in and I need help figuring out.
Question #2 above is that I want to make sure that the code is both dynamic (more people can be handled without code change), when minDate and maxDate are later tied to user input fields, the charts update automatically (I assume through adjusting the dimension variable in some way), and if I add a names filter that if I unselect names that the chart will update by removing the data for that person.
A Question #3 that I'm now realizing I'll want to figure out is how to get the person's name to show up in the pointer tooltip (the title) along with timestamp and total_time values.
There are a number of ways to go about this, but I think the easiest thing to do is to create a custom reduction which reduces each person into a sub-bin.
First off, addressing question #2, you'll want to set up your dimension based on the time interval you're interested in. For instance, if you're looking at days:
var writtenDimension = ndx.dimension(function(d) {
return d3.time.hour(d.timestamp);
});
chart.xUnits(d3.time.hours);
This will cause each timestamp to be rounded down to the nearest hour, and tell the chart to calculate the bar width accordingly.
Next, here's a custom reduction (from the FAQ) which will create an object for each reduced value, with values for each person's name:
var hoursSumGroup = writtenDimension.group().reduce(
function(p, v) { // add
p[v.name] = (p[v.name] || 0) + d.time_total;
return p;
},
function(p, v) { // remove
p[v.name] -= d.time_total;
return p;
},
function() { // init
return {};
});
I did not go with the series example I mentioned in the comments, because I think composite keys can be difficult to deal with. That's another option, and I'll expand my answer if that's necessary.
Next, we can feed the composite line charts with value accessors that can fetch the value by name.
Assume we have an array names.
compositeChart.shareTitle(false);
compositeChart.compose(
names.map(function(name) {
return dc.lineChart(compositeChart)
.dimension(writtenDimension)
.colors('red')
.group(hoursSumGroup)
.valueAccessor(function(kv) {
return kv.value[name];
})
.title(function(kv) {
return name + ' ' + kv.key + ': ' + kv.value;
});
}));
Again, it wouldn't make sense to use bar charts here, because they would obscure each other.
If you filter a name elsewhere, it will cause the line for the name to drop to zero. Having the line disappear entirely would probably not be so simple.
The above shareTitle(false) ensures that the child charts will draw their own titles; the title functions just add the current name to those titles (which would usually just be key:value).