"Zooming" a brushX domain? - d3.js

I've made a timeline tool based off of Mike Bostock's old Brush & Zoom example. It works great when the date range is fairly simple but becomes unworkable when there are clusters of events (e.g. hourly) within a longer time range (e.g. days or weeks). The brush becomes too thin to be usable and the user is left trying to fiddle with zooming and panning in order to see the data (as in the example below).
As a first attempt at a solution I created a context menu for the brush and use the brush extent to redefine/filter the data based on the brush range (I may be using the wrong terms here). It 'sort of' works though it is a clunky and imprecise "one shot" method. Results and code below.
I am thinking that if I could "zoom" the brush (or "brush the brush") that would be a more interactive and user friendly way of working with this type of data situation. I've searched around for d3 examples and haven't found any. I am also concerned that my "subtimeline" approach won't be performative interactively since it redefines the date set and rebuilds the timeline.
I am interested in any ideas about how to handle this sort of data situation and/or if this "brushing the brush" is a dead end. Is there a better d3 way to handle this?
(edit: the display date for the last event above reads 10:50 – that is wrong, it should be 11:50 which is what is in the data)
// code edited for clarity
function createSubtimeline() {
subtimelineDates.push(moment(x.domain()[0], "L LT"));
subtimelineDates.push(moment(x.domain()[1], "L LT"));
updateData()
}
function updateData() {
var activeData
if (subtimelineDates.length != 0) {
var firstDate = subtimelineDates[0];
var lastDate = subtimelineDates[1];
activeData = timelineJson.events.filter(function (e) {
var startDate = moment(e.startDate, "L LT");
if (startDate.isAfter(firstDate) && startDate.isBefore(lastDate)) {
if (e.eventHidden == false) {
return true
} else {
return false
}
} else {
return false
}
});
} else {
activeData = timelineJson.events.filter(event => event.eventHidden == false);
}
var tStart = moment(activeData[0].startDate, "MM/D/YYYY h:mm:ss a");
var tEnd = moment(activeData[activeData.length - 1].startDate, "MM/D/YYYY h:mm:ss a");
// update timeline range
x.domain([tStart, tEnd]);
x2.domain([tStart, tEnd]);
}

Related

D3.js Visualizations from Local Files

I have a mostly working code that can load one record, but I'm not sure how to expand it. The closest that I've been able to get is to pull 1 record column names from a csv based on the logic from this bl.ocks code snippet.
How can I modify the function below to pull an entire dataset rather than just one record?
By modifying code from the bl.ocks code, below is what I've been able to achieve. But I'm still having trouble looping the second function into pulling every row. In R, I could do something like rowtoHTML(0:length(data))
var rowToHtml = function( row ) {
var result = "";
for (key in row) {
result += key + ": " + row[key] + "<br/>"
}
return result;
}
var previewCsvUrl = function( csvUrl ) {
d3.csv( csvUrl, function( rows ) {
d3.select("div#preview").html(
"<b>First row:</b><br/>" + rowToHtml( rows[0] ));
})
}
d3.select("body")
.append("div")
.attr("id", "preview")
.style("margin", "5px")
previewCsvUrl("Test.csv")
This question has been asked before. I've tried to do my due diligence by reading through similar questions, but was not able to use the recommended solution based off of my technology constraints or due to inexperience with Javascript. The sample code I've used comes from a trimmed version of this bl.ocks code: http://bl.ocks.org/hlvoorhees/9d58e173825aed1e0218
Edit - I understand that hosting a file locally is the right move, but its not an option for me (hence this approach). Python has been recommended for local hosting, but I'm unable to install it on my machine at this time. I'm working with Edge and IE as my two browser options. I really appreciate the help!

Drawing gaps for missing dates in dc.js time series line graph

Okay, so I've seen this ticket and this question and have tried several examples already. Maybe I'm just dense, but I really haven't been able to crack this one.
I have a time series of events that has gaps in it. By default, dc.js connects a straight line over the gap (making it look like things are represented there when they really shouldn't be). For example, in this graph we have data as follows:
{"time":"2014-06-09T18:45:00.000Z","input":17755156,"output":250613233.333333},
{"time":"2014-06-09T18:46:00.000Z","input":18780286.6666667,"output":134619822.666667},
{"time":"2014-06-09T18:47:00.000Z","input":20074614.6666667,"output":203239834.666667},
{"time":"2014-06-09T18:48:00.000Z","input":22955373.3333333,"output":348996205.333333},
{"time":"2014-06-09T18:49:00.000Z","input":19119089.3333333,"output":562631022.666667},
{"time":"2014-06-09T18:50:00.000Z","input":15404272,"output":389916332},
{"time":"2014-06-09T18:51:00.000Z","input":null,"output":null},
{"time":"2014-06-09T21:25:20.000Z","input":5266038.66666667,"output":62598396},
{"time":"2014-06-09T21:26:20.000Z","input":6367678.66666667,"output":84494096},
{"time":"2014-06-09T21:27:20.000Z","input":5051610.66666667,"output":88812540},
{"time":"2014-06-09T21:28:20.000Z","input":5761069.33333333,"output":79098036},
{"time":"2014-06-09T21:29:20.000Z", "input":5110277.33333333,"output":45816729.3333333}
Even though there's only two actual groups of data, there's a line on that graph connecting them. How do I make dc.js line graphs draw 0 where there is no data at all. I've tried using .defined(function(d) { return !isNaN(d.x);}) and .defined(function(d) { return d.y != null; }) and such, but this is just iterating through data which isn't there.
It's tricky trying to preserve nulls when using crossfilter, because crossfilter is all about aggregation.
Remember that reduceSum will add any values it finds, starting from zero, and 0 + null === 0.
In your case, it looks like you're not actually aggregating, since your timestamps are unique, so you could do something like this:
var input = time.group().reduce(
function(p, d) {
if(d.input !== null)
p += d.input;
else p = null;
return p;
},
function(p, d) {
if(d.input !== null)
p -= d.input;
else p = null;
return p;
},
function(){ return 0; }
);
Yeah, that's a lot more complicated than reduceSum, and it may get even more complicated if more than one datum falls into a bucket. (Not sure what you'd want to do there - is it possible for a data point to be partly defined?)
With the reduction defined this way, null reduces to null and dc.js is able to find the gaps:
Fork of your fiddle (thanks!): http://jsfiddle.net/gordonwoodhull/omLko77k/3/
Edit: counting nulls
If you're doing a "real" reduction where there is more than one value in a bin, I think you'll need to count the number of non-null values as well as keeping a running sum.
When there are no non-null values, the sum should be null.
Reusing our code a bit better this time:
function null_counter(field) {
return {
add: function(p, d) {
if(d[field] !== null) {
p.nvalues++;
p.sum += d[field];
}
return p;
},
remove: function(p, d) {
if(d[field] !== null) {
p.nvalues--;
p.sum -= d[field];
if(!p.nvalues)
p.sum = null;
}
return p;
},
init: function() {
return {nvalues: 0, sum: null};
}
}
}
Applied like this (and getting the fields right this time):
var input_reducer = null_counter('input');
var input = time.group().reduce(
input_reducer.add,
input_reducer.remove,
input_reducer.init
);
var output_reducer = null_counter('output');
var output = time.group().reduce(
output_reducer.add,
output_reducer.remove,
output_reducer.init
);
Since we're reducing to an object with two values {nvalues, sum}, we need to make all our accessors a little more complicated:
.valueAccessor(function(kv) { return kv.value.sum; })
.defined(function(d){
return (d.data.value.sum !== null);
})
chart.stack(output, "Output bits",
function(kv) { return kv.value.sum; });
Updated fork: http://jsfiddle.net/gordonwoodhull/omLko77k/9/

Building a map with coloring condition

I am trying to build a choropleth that is not exactly a choropleth in dc.js. What I am trying to do is color the map base on coloring condition and ultimately this will interact with other charts and filters as well. My csv looks like this:
country,id,condition,value
AU,1,yes,19
US,2,no,23
US,2,no,30
US,2,no,4
IN,3,yes,14
SG,4,yes,2
NZ,5,no,6
NZ,5,no,20
and this is my approach so far, producing the count of occurrences.
var ndx = crossfilter(data)
var countryDimension = ndx.dimension(function (d){
return d.country
});
var colors = d3.scale.ordinal().domain(['yes','no']).range(["green","blue"])
worldMap.width(mapWidth)
.height(mapHeight)
.dimension(countryDimension)
.group(countryDimension.group())
.projection(project)
.colors(colors)
.colorCalculator(function(d){
return d ? worldMap.colors()(d) : '#d8d8d8';
})
.overlayGeoJson(geoJson.features, "id", function(d){
return d.id;
})
.title(function(d){
return 'Country: ' + d.key + '\nCondition: ' + d.value;
});
I am quite new to this amazing world of d3 and dc.js. Although I have been reading the documentation and forums I cannot figure out how I could make it so that a map is drawn, and the countries with the condition 'yes' is colored green and countries with the condition 'no' is colored blue. So pretty much if i do console.log(d.value) it should return either 'yes' or 'no'. I don't get what I have to do with my 'group'.
If every country has the same value for condition every time it is listed in the data, then in some sense the data is denormalized. That's fine, because crossfilter works best with a single array of data.
Of course it means that the choropleth won't respond to brushing on other charts, since the value is not affected by how many rows are currently filtered. But it will be able to filter other charts.
Count yesses
There are a couple of ways to do this. One way to do it is to count the number of yesses, and set the value according the count:
var yesnoGroup = countryDimension.group().reduceSum(function(d) {
return d.condition === 'yes' ? 1 : 0;
});
worldMap.valueAccessor(function(kv) {
return kv.value ? 'yes' : 'no';
})
Grab first value
However this would probably cause countries to turn blue when they are filtered out by the other charts. So you could also use a "grab first value and hold onto it" strategy like this:
var yesnoGroup = countryDimension.group().reduce(
function(p, v) { // add
return v.condition;
},
function(p, v) { // remove
return p; // ignore remove event
},
function() { // initialize
return null; // no value
});
A little bit ugly and a weird way to use crossfilter, but that's just because crossfilter expects the data to have some effect on the reduced value, and it doesn't here.
EDIT: Three states
Based on the conversation below, I understand you're actually looking for three states: no, zero, and yes. (This makes more sense than the solutions above, but I'll leave those for posterity.) Here are two completely different ways to solve the no/zero/yes problem.
Both of these solutions use the following three-way color scale:
var colors = d3.scale.ordinal().domain(['no', 'zero', 'yes']).range(["blue", "grey", "green"])
No/zero/yes as negative/positive numbers
This is clever and simple: we'll just count each no as -1 and each yes as +1. If the sum is zero, we'll draw in grey. The only caveat here is if there are contradictions in the data, you could get a false zero. But that might be better than a false no or yes (?)
var nozeroyesGroup = countryDimension.group().reduceSum(function(d) {
return d.condition === 'no' ? -1 : d.condition === 'yes' : +1 : 0;
});
worldMap.valueAccessor(function(kv) {
return kv.value < 0 ? 'no' : kv.value > 0 ? 'yes' : 'zero';
})
No/yes polarity
We could also remember a count and polarity separately. This is maybe safer but also maybe slower. (Not that you'd notice unless your data is huge.) It's a bit more complicated. Kind of a matter of preference.
var nozeroyesGroup = countryDimension.group().reduce(
function(p, v) { // add
if(p.polarity && p.polarity != v.condition)
console.warn('inconsistent');
p.polarity = v.condition;
++p.count;
return p;
},
function(p, v) { // remove
if(p.polarity != v.condition || p.count <= 0)
console.warn('inconsistent');
--p.count;
return p;
},
function() { // initialize
return {count: 0, polarity: null}; // no value
});
worldMap.valueAccessor(function(kv) {
return kv.value.count ? kv.value.polarity : 'zero';
})

How to speed up heavy conditional formatting rules

At our marketing company/agency, we're using a master tracker in Google Sheets to keep track of all paid advertising campaigns that we are handling for our clients. The document is getting longer and longer, and the variety of conditional formatting rules we are using is getting heavy and slow upon any change made to the document.
Five employees are using the document at any given time, making changes to the "STATUS" column upon any change to the campaign – if it is ready to upload, if it is LIVE, if it is paused etc. The conditional formatting simply changes the color of each line based on the value in the "STATUS" column. It also looks at the start/end dates and marks the line red if there is an issue. Etc.
How can I speed up processing using this document? I have created a minified version of our tracker with one line for each conditional formatting rule to make it easy for you to have a look.
I'm sure there are smarter ways to consolidate the rules and/or build a script that can handle the task more easily and more efficiently.
This answer uses a script to change the background color of a row whenever the Status is changed (works for "READY", "LIVE" and "DONE").
Live demo:
https://docs.google.com/spreadsheets/d/1bVwM1wSBVlZTmz5S95RXSrRQxlTKWWN_Hl4PZ81sbGI/edit?usp=sharing
The script is viewable under the "Tools - Script Editor..." menu. It is activated by an "onEdit" trigger (see Is it possible to automate Google Spreadsheets Scripts (e.g. without an event to trigger them)?).
Here is the script itself:
function onEdit(e) {
var STATUS_COL = 18;
var MAX_COLS = 18;
var COLOR_READY = "grey";
var COLOR_LIVE = "#512da8";
var COLOR_DONE = "green";
var activeSheet = SpreadsheetApp.getActiveSheet();
var cell = activeSheet.getActiveSelection();
var val = cell.getValues()[0][0];
var color = null;
switch (val) {
case "READY":
color = COLOR_READY;
break;
case "LIVE":
color = COLOR_LIVE;
break;
case "DONE":
color = COLOR_DONE;
break;
}
if (color != null) {
var row = activeSheet.getRange(cell.getRow(), 1, 1, MAX_COLS);
row.setBackgroundColor(color);
}
}
I had whole rows changing colors depending on some conditions. So I extracted complex formulas from conditional formatting panel into columns on sheet (I got "TRUE" or "FALSE") and referenced those columns in conditional formatting rules. For some reason calculation of conditional formatting formulas is much slower than same calculation inside cells.

dc.js - Creating a row chart from multiple columns and enabling filtering

This is similar to dc.js - how to create a row chart from multiple columns but I want to take it a step further and enable filtering when the rows are clicked.
To answer the question "What is it supposed to filter?" - Only show records with value > 0. For example when Row 'a' is clicked it will only show records with value for a > 0. Hence, the Type pie chart will change to foo:1, bar:2
I guess I have to overwrite onClick method? But I am not sure how.
chart.onClick = function(d) {}
jsfiddle from the answer to the above question - http://jsfiddle.net/gordonwoodhull/37uET/6/
Any suggestions?
Thanks!
Okay, here's a solution where if a record has values > 0 for any of the selected rows, that record is included. As #Ethan said, it's a matter of defining a filter handler:
sidewaysRow.filterHandler(function(dim, filters) {
if(filters && filters.length)
dim.filterFunction(function(r) {
return filters.some(function(c) {
return r[c] > 0;
});
})
else dim.filterAll();
return filters;
});
Also, since the filterFunction only has access to the key, we pass the entire record through as the key. This doesn't make a whole lot of sense in the "real world" but since we're already using crossfilter sideways, it is probably fine:
var dim = ndx.dimension(function(r) { return r; });
New version of the fiddle: https://jsfiddle.net/gordonwoodhull/b7cak6xj/
BTW it sounds like you want to only select one row at a time. Here's how to do that:
sidewaysRow.addFilterHandler(function(filters, filter) {
filters.length = 0;
filters[0] = filter;
return filters;
})
(This will be simpler in dc 2.1 on the develop branch, where the charts use the result of the filter handlers instead of requiring you to modify the filters in place; the body becomes just return [filter];)

Resources