Building a map with coloring condition - d3.js

I am trying to build a choropleth that is not exactly a choropleth in dc.js. What I am trying to do is color the map base on coloring condition and ultimately this will interact with other charts and filters as well. My csv looks like this:
country,id,condition,value
AU,1,yes,19
US,2,no,23
US,2,no,30
US,2,no,4
IN,3,yes,14
SG,4,yes,2
NZ,5,no,6
NZ,5,no,20
and this is my approach so far, producing the count of occurrences.
var ndx = crossfilter(data)
var countryDimension = ndx.dimension(function (d){
return d.country
});
var colors = d3.scale.ordinal().domain(['yes','no']).range(["green","blue"])
worldMap.width(mapWidth)
.height(mapHeight)
.dimension(countryDimension)
.group(countryDimension.group())
.projection(project)
.colors(colors)
.colorCalculator(function(d){
return d ? worldMap.colors()(d) : '#d8d8d8';
})
.overlayGeoJson(geoJson.features, "id", function(d){
return d.id;
})
.title(function(d){
return 'Country: ' + d.key + '\nCondition: ' + d.value;
});
I am quite new to this amazing world of d3 and dc.js. Although I have been reading the documentation and forums I cannot figure out how I could make it so that a map is drawn, and the countries with the condition 'yes' is colored green and countries with the condition 'no' is colored blue. So pretty much if i do console.log(d.value) it should return either 'yes' or 'no'. I don't get what I have to do with my 'group'.

If every country has the same value for condition every time it is listed in the data, then in some sense the data is denormalized. That's fine, because crossfilter works best with a single array of data.
Of course it means that the choropleth won't respond to brushing on other charts, since the value is not affected by how many rows are currently filtered. But it will be able to filter other charts.
Count yesses
There are a couple of ways to do this. One way to do it is to count the number of yesses, and set the value according the count:
var yesnoGroup = countryDimension.group().reduceSum(function(d) {
return d.condition === 'yes' ? 1 : 0;
});
worldMap.valueAccessor(function(kv) {
return kv.value ? 'yes' : 'no';
})
Grab first value
However this would probably cause countries to turn blue when they are filtered out by the other charts. So you could also use a "grab first value and hold onto it" strategy like this:
var yesnoGroup = countryDimension.group().reduce(
function(p, v) { // add
return v.condition;
},
function(p, v) { // remove
return p; // ignore remove event
},
function() { // initialize
return null; // no value
});
A little bit ugly and a weird way to use crossfilter, but that's just because crossfilter expects the data to have some effect on the reduced value, and it doesn't here.
EDIT: Three states
Based on the conversation below, I understand you're actually looking for three states: no, zero, and yes. (This makes more sense than the solutions above, but I'll leave those for posterity.) Here are two completely different ways to solve the no/zero/yes problem.
Both of these solutions use the following three-way color scale:
var colors = d3.scale.ordinal().domain(['no', 'zero', 'yes']).range(["blue", "grey", "green"])
No/zero/yes as negative/positive numbers
This is clever and simple: we'll just count each no as -1 and each yes as +1. If the sum is zero, we'll draw in grey. The only caveat here is if there are contradictions in the data, you could get a false zero. But that might be better than a false no or yes (?)
var nozeroyesGroup = countryDimension.group().reduceSum(function(d) {
return d.condition === 'no' ? -1 : d.condition === 'yes' : +1 : 0;
});
worldMap.valueAccessor(function(kv) {
return kv.value < 0 ? 'no' : kv.value > 0 ? 'yes' : 'zero';
})
No/yes polarity
We could also remember a count and polarity separately. This is maybe safer but also maybe slower. (Not that you'd notice unless your data is huge.) It's a bit more complicated. Kind of a matter of preference.
var nozeroyesGroup = countryDimension.group().reduce(
function(p, v) { // add
if(p.polarity && p.polarity != v.condition)
console.warn('inconsistent');
p.polarity = v.condition;
++p.count;
return p;
},
function(p, v) { // remove
if(p.polarity != v.condition || p.count <= 0)
console.warn('inconsistent');
--p.count;
return p;
},
function() { // initialize
return {count: 0, polarity: null}; // no value
});
worldMap.valueAccessor(function(kv) {
return kv.value.count ? kv.value.polarity : 'zero';
})

Related

"Zooming" a brushX domain?

I've made a timeline tool based off of Mike Bostock's old Brush & Zoom example. It works great when the date range is fairly simple but becomes unworkable when there are clusters of events (e.g. hourly) within a longer time range (e.g. days or weeks). The brush becomes too thin to be usable and the user is left trying to fiddle with zooming and panning in order to see the data (as in the example below).
As a first attempt at a solution I created a context menu for the brush and use the brush extent to redefine/filter the data based on the brush range (I may be using the wrong terms here). It 'sort of' works though it is a clunky and imprecise "one shot" method. Results and code below.
I am thinking that if I could "zoom" the brush (or "brush the brush") that would be a more interactive and user friendly way of working with this type of data situation. I've searched around for d3 examples and haven't found any. I am also concerned that my "subtimeline" approach won't be performative interactively since it redefines the date set and rebuilds the timeline.
I am interested in any ideas about how to handle this sort of data situation and/or if this "brushing the brush" is a dead end. Is there a better d3 way to handle this?
(edit: the display date for the last event above reads 10:50 – that is wrong, it should be 11:50 which is what is in the data)
// code edited for clarity
function createSubtimeline() {
subtimelineDates.push(moment(x.domain()[0], "L LT"));
subtimelineDates.push(moment(x.domain()[1], "L LT"));
updateData()
}
function updateData() {
var activeData
if (subtimelineDates.length != 0) {
var firstDate = subtimelineDates[0];
var lastDate = subtimelineDates[1];
activeData = timelineJson.events.filter(function (e) {
var startDate = moment(e.startDate, "L LT");
if (startDate.isAfter(firstDate) && startDate.isBefore(lastDate)) {
if (e.eventHidden == false) {
return true
} else {
return false
}
} else {
return false
}
});
} else {
activeData = timelineJson.events.filter(event => event.eventHidden == false);
}
var tStart = moment(activeData[0].startDate, "MM/D/YYYY h:mm:ss a");
var tEnd = moment(activeData[activeData.length - 1].startDate, "MM/D/YYYY h:mm:ss a");
// update timeline range
x.domain([tStart, tEnd]);
x2.domain([tStart, tEnd]);
}

avoid enumerating group names in d3.layout.stack()["group 1", "group 2"]

I am having a really hard time understanding d3.layout.stack() when groups are not listed manually. In the below example, similar to what I've found in other questions, groups are listed in [] as "Apple", etc., but as far as I understand this has to be inputted manually. I am seeking a way to not have to manually input "Apple", "Blueberry", etc.
var dataset = d3.layout.stack()(["Apple", "Blueberry", "Lettuce", "Orange"].map(function(fruit) {
return data.map(function(d) {
return {x: d.orchard, y: +d[fruit]};
});
}));
I've tried inserting a line in my data object as below, called 'names':
[{names='Apple','Blueberry','Lettuce','Orange'}, {Apple=1.0, Orange=2.0, Lettuce=1.0, orchard=小明, Blueberry=1.0}, {Apple=1.0, Orange=1.0, Lettuce=1.0, orchard=小陈, Blueberry=1.0}, {Apple=1.0, Orange=1.0, Lettuce=1.0, orchard=小虎, Blueberry=1.0}, {Orange=1.0, Lettuce=1.0, orchard=小桃, Blueberry=1.0, Apple=1.0}]
Is there a way to code something similar to below?
var dataset = d3.layout.stack()([d3.keys(names)].map(function(fruit) {
Should I be focused more on inserting a unique list of names into my data object, or do so by parsing my data in my d3 code itself to accumulate a list of unique group names?
I am wondering, if the d3.keys logic makes sense, if it can be applied to the below context too, instead of enumerating each case:
legend.append("text")
.attr("x", width + 5)
.attr("y", 9)
.attr("dy", ".35em")
.style("text-anchor", "start")
.text(function(d, i) {
for(var j =0; j<4; j++){
switch (i) {
case j: return d3.keys[j]
// switch (i) {
//
// case 0: return "orange"
// case 1: return "apple"
// case 2: return "blueberry"
// case 3: return "lettuce"
}
}
});
I ended up just converting the entire graph to d3 v5. Below is some notes based off a lot of sources I looked at mixed with my own work:
Better practice for stacked bar is to use
.data(d3.stack().keys(keys)(data))
where
var keys = d3.keys(data[0]).filter(function(d){
return d != "orchard";
});
or in other words:
var keys = d3.keys(data[0]).filter(d => d != "orchard")
This is useful for data that is pre-parsed in javascript. Say you have just columns in a csv:
var keys = csv.columns.slice(0);
is useful, but same philosophy for stacking applies.
Slight issue: if you have new categories arising later on in the data, i.e. a new fruit pineapple is part of data[1] but not data[0], key will not identify pineapple. It only responds to the data object of the first entry.
To not rely on data[1], data[0], etc., and "accumulate" keys for data[0], data[1], etc. while maintaining the same filter:
var key = [];
for(var i =0; i < d3.keys(data).length; i++){
var joinin = d3.keys(data[i]).filter(d => d != "orchard")
var key = key.concat(joinin)
// console.log(key)
}
There's most likely a better way of writing that code, but the explanation is:
If you wrote something like this you'd get keys for only one set of data:
var key = d3.keys(data[2]).filter(function(d){
return d != "orchard";
});
If you wrote this you get the keys for each iteration of data:
var key = [];
for(var i =0; i < d3.keys(data).length; i++){
var key = d3.keys(data[i]).filter(d => d != "orchard")
console.log(key)
key.push(key);
}
So the trick is to use a for loop to get each iteration of data but concat that into one singular list, which has no repeats.
[EDIT] What if you wanted the value given to each key? Again, this is for a data structure like this, which is a little unconventional:
data = [
{0:
{"Apple": 1}
{"orchard": xx}
}
{1:
{"Apple": 2}
{"orchard": xx}
}
]
You can use the below, where key will return ["Apple"], and key_values will return [1, 2]. Basically the filter d > 0 prevents any strings, so like names of orchards "xx". Does the same as filtering out orchards.
var key = [];
var key_values = [];
for(var i =0; i < d3.keys(data).length; i++){
var key_value = d3.entries(data[i]).map(d => d.value).filter(d => d > 0)
var key_values = key_values.concat(key_value)
var joinin = d3.keys(data[i]).filter(d => d != "orchard")
var key = key.concat(joinin)
}
(EDIT2) About the legend..
I just replaced my code with, and I know d3v5 can simplify the below(?),
var legend = svg.selectAll(".legend")
.data(color.domain())
.enter()
.append("g")
.attr("class","legend")
.attr("transform",function(d,i) {
return "translate(1300," + i * 15 + ")";
});
In my case input variables are discrete, like "Apple" etc., not integers, so we use scaleOrdinal and just use "keys" for the domain. You don't really need to write that though, I think it defaults to the input list of discrete variables? Not sure why it works.
var color = d3.scaleOrdinal()
.domain(keys)
// .range (whatever you want)

Kendo NumericTextBox with step and rounding

I want to setup kendoNumericTextBox to allow user input any integer number and set step to 1000. But when user enters any value and use spinner, it should update to next multiple of step.
For example:
enter 123, press spin up, value will be 1000
enter 1234, press spin up, value vill be 2000
Is it possible or only way is to handle spin event and modify value from there?
UPDATE:
Ok, guys, thnx for help.
I have this spin handler now and it seems to be working as expected.
function onSpin(e)
{
var currentvalue = kendo.parseInt(this.value());
if (currentvalue > 0)
{
this.value(Math.floor(currentvalue / this.step()) * this.step());
}
if (currentvalue < 0)
{
this.value(Math.ceil(currentvalue / this.step()) * this.step());
}
}
I have provided a dojo below with a potential solution for you:
https://dojo.telerik.com/UQohUjaP/2
I have created a function that will work on the spin and on change value so that it will step the value up/down on the value that you set e.g. 1000
the function is fairly simple and for brevity I have taken out the log statements here:
function onChange() {
var currentvalue = kendo.parseInt(this.value());
if (currentvalue > 0) {
currentvalue = Math.ceil(currentvalue / this.step()) * this.step();
} else if (currentvalue < 0) {
currentvalue = Math.floor(currentvalue / this.step()) * this.step();
} else {
currentvalue = 0;
}
this.value(currentvalue);
}
Unfortunately there doesn't seem to be a simple way of figuring out if the value has gone up or down so I am basically checking to see if the value is greater than 1 or less than 1 and then calculating the ceiling or the floor for the value and then stepping it in the right direction. in order to cater for zero we have a special condition which just sets the value to 0 assuming that is a valid value in your scenario
As you say, it's possible by listening to the spin event:
$("#numerictextbox").kendoNumericTextBox({
min: 0,
spin: function(e) {
var isUp = e.sender._key === 38,
isDown = e.sender._key === 40;
var m = Math.trunc(this.value()/1000),
value = isUp ? m + 1 : m;
this.value(value * 1000);
}
});
I doubt there's something out of the box, because your needs seem somewhat unusual.

Drawing gaps for missing dates in dc.js time series line graph

Okay, so I've seen this ticket and this question and have tried several examples already. Maybe I'm just dense, but I really haven't been able to crack this one.
I have a time series of events that has gaps in it. By default, dc.js connects a straight line over the gap (making it look like things are represented there when they really shouldn't be). For example, in this graph we have data as follows:
{"time":"2014-06-09T18:45:00.000Z","input":17755156,"output":250613233.333333},
{"time":"2014-06-09T18:46:00.000Z","input":18780286.6666667,"output":134619822.666667},
{"time":"2014-06-09T18:47:00.000Z","input":20074614.6666667,"output":203239834.666667},
{"time":"2014-06-09T18:48:00.000Z","input":22955373.3333333,"output":348996205.333333},
{"time":"2014-06-09T18:49:00.000Z","input":19119089.3333333,"output":562631022.666667},
{"time":"2014-06-09T18:50:00.000Z","input":15404272,"output":389916332},
{"time":"2014-06-09T18:51:00.000Z","input":null,"output":null},
{"time":"2014-06-09T21:25:20.000Z","input":5266038.66666667,"output":62598396},
{"time":"2014-06-09T21:26:20.000Z","input":6367678.66666667,"output":84494096},
{"time":"2014-06-09T21:27:20.000Z","input":5051610.66666667,"output":88812540},
{"time":"2014-06-09T21:28:20.000Z","input":5761069.33333333,"output":79098036},
{"time":"2014-06-09T21:29:20.000Z", "input":5110277.33333333,"output":45816729.3333333}
Even though there's only two actual groups of data, there's a line on that graph connecting them. How do I make dc.js line graphs draw 0 where there is no data at all. I've tried using .defined(function(d) { return !isNaN(d.x);}) and .defined(function(d) { return d.y != null; }) and such, but this is just iterating through data which isn't there.
It's tricky trying to preserve nulls when using crossfilter, because crossfilter is all about aggregation.
Remember that reduceSum will add any values it finds, starting from zero, and 0 + null === 0.
In your case, it looks like you're not actually aggregating, since your timestamps are unique, so you could do something like this:
var input = time.group().reduce(
function(p, d) {
if(d.input !== null)
p += d.input;
else p = null;
return p;
},
function(p, d) {
if(d.input !== null)
p -= d.input;
else p = null;
return p;
},
function(){ return 0; }
);
Yeah, that's a lot more complicated than reduceSum, and it may get even more complicated if more than one datum falls into a bucket. (Not sure what you'd want to do there - is it possible for a data point to be partly defined?)
With the reduction defined this way, null reduces to null and dc.js is able to find the gaps:
Fork of your fiddle (thanks!): http://jsfiddle.net/gordonwoodhull/omLko77k/3/
Edit: counting nulls
If you're doing a "real" reduction where there is more than one value in a bin, I think you'll need to count the number of non-null values as well as keeping a running sum.
When there are no non-null values, the sum should be null.
Reusing our code a bit better this time:
function null_counter(field) {
return {
add: function(p, d) {
if(d[field] !== null) {
p.nvalues++;
p.sum += d[field];
}
return p;
},
remove: function(p, d) {
if(d[field] !== null) {
p.nvalues--;
p.sum -= d[field];
if(!p.nvalues)
p.sum = null;
}
return p;
},
init: function() {
return {nvalues: 0, sum: null};
}
}
}
Applied like this (and getting the fields right this time):
var input_reducer = null_counter('input');
var input = time.group().reduce(
input_reducer.add,
input_reducer.remove,
input_reducer.init
);
var output_reducer = null_counter('output');
var output = time.group().reduce(
output_reducer.add,
output_reducer.remove,
output_reducer.init
);
Since we're reducing to an object with two values {nvalues, sum}, we need to make all our accessors a little more complicated:
.valueAccessor(function(kv) { return kv.value.sum; })
.defined(function(d){
return (d.data.value.sum !== null);
})
chart.stack(output, "Output bits",
function(kv) { return kv.value.sum; });
Updated fork: http://jsfiddle.net/gordonwoodhull/omLko77k/9/

dc.js - Creating a row chart from multiple columns and enabling filtering

This is similar to dc.js - how to create a row chart from multiple columns but I want to take it a step further and enable filtering when the rows are clicked.
To answer the question "What is it supposed to filter?" - Only show records with value > 0. For example when Row 'a' is clicked it will only show records with value for a > 0. Hence, the Type pie chart will change to foo:1, bar:2
I guess I have to overwrite onClick method? But I am not sure how.
chart.onClick = function(d) {}
jsfiddle from the answer to the above question - http://jsfiddle.net/gordonwoodhull/37uET/6/
Any suggestions?
Thanks!
Okay, here's a solution where if a record has values > 0 for any of the selected rows, that record is included. As #Ethan said, it's a matter of defining a filter handler:
sidewaysRow.filterHandler(function(dim, filters) {
if(filters && filters.length)
dim.filterFunction(function(r) {
return filters.some(function(c) {
return r[c] > 0;
});
})
else dim.filterAll();
return filters;
});
Also, since the filterFunction only has access to the key, we pass the entire record through as the key. This doesn't make a whole lot of sense in the "real world" but since we're already using crossfilter sideways, it is probably fine:
var dim = ndx.dimension(function(r) { return r; });
New version of the fiddle: https://jsfiddle.net/gordonwoodhull/b7cak6xj/
BTW it sounds like you want to only select one row at a time. Here's how to do that:
sidewaysRow.addFilterHandler(function(filters, filter) {
filters.length = 0;
filters[0] = filter;
return filters;
})
(This will be simpler in dc 2.1 on the develop branch, where the charts use the result of the filter handlers instead of requiring you to modify the filters in place; the body becomes just return [filter];)

Resources