D3 - dealing with JSON nested data structures - d3.js

My post is somehow a follow-up to this question : D3 - how to deal with JSON data structures?
In short, I'm in trouble with handling complex nested JSON structures.
Let me give a simple sample (that mirrors the above link) to illustrate:
var regularData = [[40,20,30,24,18,40],
[24,20,30,41,12,34]];
var myTable = d3.select("body").append("table")
.selectAll("tr")
.data(regularData, function(d) {
return d;
})
.enter()
.append("tr")
.selectAll("td")
.data(function(d) {
return d;
})
.enter()
.append("td")
.text(function(d) {
return d;
});
As already shown through D3's doc, this produces two lines of numbers, as expected.
But if I replace regularData by this:
var newData = [{"user":"jim","scores":[40,20,30,24,18,40]},
{"user":"ray","scores":[24,20,30,41,12,34]}];
and adapt the myTable's construction lines 3 to 5 with this:
.data(newData, function(d) {
return d.scores;
})
I would have expected to loop in the scores for each user. Instead, the second data clause is still bound to top-level objects (i.e. with properties "user" and "scores").
A brute-force approach would be to cast my data in a plain "array of arrays" adapted to each purpose. But maybe I missed something, and a more elegant way is possible ?
Thanks by advance for your help,
P.

You have slightly misunderstood what the second argument to d3.data is meant for.
That argument is used for object constancy by providing an id for the data and not as an accessor function, as in most other functions in the d3 API.
However, when a function is passed as the first argument, as is here:
.data(function(d) {
return d;
})
then it does behave as an accessor function.
In your case, what you want is something along these lines:
var newData = [{"user":"jim","scores":[40,20,30,24,18,40]},
{"user":"ray","scores":[24,20,30,41,12,34]}];
var myTable = d3.select("body").append("table")
.selectAll("tr")
.data(regularData, function(d) {
return d.user;
})
.enter()
.append("tr")
.selectAll("td")
.data(function(d) {
return d.scores;
})
.enter()
.append("td")
.text(function(d) {
return d;
});
I have used the user field as the id of the data and have returned the scores in the second call to data to create the td elements.

Related

attr() won't change an attribute unless I return it via an anonymous function

var k = -1;
someCircle.attr("cy", someScaleObject(++k));
All circles will have the same y alignment whereas expected behavior is to have each circle drawn below the other. Oddly enough, if I return someScaleObject(++k) via an anonymous function, this has the desired effect. Why? The author also declares a d variable within the scope of the anonymous function despite having no use for it. Can anyone explain why?
// Works as expected
someCircle.attr("cy", function () {
return someScaleObject(++k);
});
The full code can be found below and is drawn from D3 for the Impatient
// keys.js
function makeKeys() {
var ds1 = [["Mary", 1], ["Jane", 4], ["Anne", 2]];
var ds2 = [["Anne", 5], ["Jane", 3]];
var scX = d3.scaleLinear().domain([0, 6]).range([50, 300]),
scY = d3.scaleLinear().domain([0, 3]).range([50, 150]);
var j = -1, k = -1;
var svg = d3.select("#keys");
svg.selectAll("text")
.data(ds1).enter().append("text")
.attr("x", 20).attr("y", function (d) {
return scY(++j);
}).text(function (d) {
return d[0];
});
svg.selectAll("circle").data(ds1).enter().append("circle")
.attr("r", 5).attr("fill", "red")
.attr("cx", function (d) {
return scX(d[1]);
})
.attr("cy", function (d) {
return scY(++k); // this is what concerns me
});
svg.on("click", function () {
var cs = svg.selectAll("circle").data(ds2, function (d) {
return d[0];
});
cs.transition().duration(1000).attr("cx", function (d) {
return scX(d[1]);
})
cs.exit().attr("fill", "blue");
})
}
And the html file
<!DOCTYPE html>
<html>
<head>
<title></title>
<script src="d3.js"></script>
<script src="keys.js"></script>
</head>
<body onload="makeKeys()">
<svg id="keys" width="300" height="150"></svg>
</body>
</html>
In the following:
someCircle.attr("cy", someScaleObject(++k));
You aren't passing a function to selection.attr(), you execute someScaleObject(++k) and pass its return value instead. If you pass something other than a function to selection.attr() for the second parameter, you use the same value for all elements - this is why your circles are all positioned the same.
You could pass a function without executing it:
someCircle.attr("cy", someScaleObject);
However, you want to pass some integer to this function, D3 passes three variables to the function passed to selection.attr() - the datum, the current index, and the group of DOM nodes in the selection. These are passed in this order. In your case you want to pass some other variable to the function, so this method won't work.
You could build someScaleObject to track or access k itself from within the function - such that it doesn't need to be passed as a parameter. Or, we can nest the function inside an anonymous function:
someCircle.attr("cy", function() { return someScaleObject(++k) });
This allows us to pass parameters other than the datum, index, and group of nodes to the function we want to execute. This way selection.attr() is passed a function, that when executed for each element in the selection, execute someScaleObject the intended number of times.
However....
While the above is useful in terms of D3, in your specific situation, the need for passing k to the scale object is unclear, as k represents the index of each element. So instead of:
var k = -1;
selection.attr("cy", function() { return scY(++k); })
We can simply use:
selection.attr("cy", function(d,i) { return scY(i); })
Because the second parameter of a function passed to .attr() is the index of the element in the selection. D3 already tracks this. I'm actually not aware of a use case where the index is something that needs to be tracked externally.

Crossfilter / dc.table sorting by Timestamp

I want to sort my table by the timestamp descending but in this field my testcases ascending.
How can I do the oposite?
I hope i can explain my problem.
//------------------------table------------------------------------
var TestCaseDim2 = perfData.dimension(function (d) {
return d.TestCase;
});
dc.dataTable('#data-table')
.dimension(TestCaseDim2)
.group(function (d) {
return d.Timestamp.bold().fontcolor("darkblue");
})
.columns([
function (d) { return d.SerialNumber;},
function (d) { return d.Result;},
])
.width(get2ndWindowSize())
.order(d3.descending )
.size(700)
.renderlet(function (table) {
table.selectAll('.dc-table-group').classed('info', true);
});
I hope someone can help me.
In your JSFiddle example, I believe but am not sure that within your groups your test scripts are ordered by test case because that is the dimension you have used on your data table.
The real issue is that in a dc.js data table, you can't control the order of groups and the ordering within groups separately. The ordering within groups is based on the order returned by the dimension and the order of groups is determined by the text of the group. Both are either ascending or descending depending on what you set in dataTable.order.
So, what do we do? I'd recommend defining your dimension on the property you want to sort by within the groups, and then if necessary reversing the direction of the dimension sort by flipping the top/bottom functions on the dimension:
var perfData = crossfilter(data);
var testScriptDim = perfData.dimension(function (d) {
return d.testscript;
});
testScriptDim.top = testScriptDim.bottom;
dc.dataTable('#data-table')
.dimension(testScriptDim)
.group(function (d) {
return d.date.bold().fontcolor("darkblue");
})
.columns([
function (d) { return d.testscript; },
function (d) { return d.duration; },
function (d) { return d.clickcount; }
])
.size(100)
.order(d3.descending)
.renderlet(function (table) {
table.selectAll('.dc-table-group').classed('info', true);
});
Here's an updated version of your JSFiddle that I think does what you want: https://jsfiddle.net/esjewett/xzrp0ggv/1/

Update domain of color scale only once in d3 reusable charts

I am building a reusable chart following this tutorial: https://bost.ocks.org/mike/chart/. The full code is at the end of the question. I have the following problem:
As you can see the 'click' event on a specific component triggers a query that updates the whole chart retrieving new data. I am referring to this line:
selection.datum(relatedConcepts).call(chart); // Update this vis
Now this update works great, but of course given that in the function "chart" I also have
color.domain(data.map(function(d){ return d[0]}));
the domain of the color scale will be also updated and I don't want that.
So the question is: how do I set the color scale domain ONLY the first time the chart gets created?
d3.custom = d3.custom || {};
d3.custom.conceptsVis = function () {
var color = d3.scale.category20();
// To get events out of the module we use d3.dispatch, declaring a "conceptClicked" event
var dispatch = d3.dispatch('conceptClicked');
function chart(selection) {
selection.each(function (data) {
//TODO: This should be executed only the first time
color.domain(data.map(function(d){ return d[0]}));
// Data binding
var concepts = selection.selectAll(".progress").data(data, function (d) {return d[0]});
// Enter
concepts.enter()
.append("div")
.classed("progress", true)
.append("div")
.classed("progress-bar", true)
.classed("progress-bar-success", true)
.style("background-color", function (d) {
return color(d[0])
})
.attr("role", "progressbar")
.attr("aria-valuenow", "40")
.attr("aria-valuemin", "0")
.attr("aria-valuemax", "100")
.append("span") // (http://stackoverflow.com/questions/12937470/twitter-bootstrap-center-text-on-progress-bar)
.text(function (d) {
return d[0]
})
.on("click", function (d) {
// Update the concepts vis
d3.json("api/concepts" + "?concept=" + d[0], function (error, relatedConcepts) {
if (error) throw error;
selection.datum(relatedConcepts).call(chart); // Update this vis
dispatch.conceptClicked(relatedConcepts, color); // Push the event outside
});
});
// Enter + Update
concepts.select(".progress-bar").transition().duration(500)
.style("width", function (d) {
return (d[1] * 100) + "%"
});
// Exit
concepts.exit().select(".progress-bar").transition().duration(500)
.style("width", "0%");
});
}
d3.rebind(chart, dispatch, "on");
return chart;
};
ANSWER
I ended up doing what meetamit suggested and I added this:
// Getter/setter
chart.colorDomain = function(_) {
if (!arguments.length) return color.domain();
color.domain(_);
return chart;
};
to my conceptsVis function, so that from the outside I can do:
.... = d3.custom.conceptsVis().colorDomain(concepts);
Of course I deleted the line:
color.domain(data.map(function(d){ return d[0]}));
You can check if the domain is an empty array and only populate it if it is:
if(color.domain().length == 0) {
color.domain(data.map(function(d){ return d[0]}));
}
That being said, this behavior seems fundamentally wrong, or at least bug-prone. It means that the populating of the domain is a side-effect of the first render. But what is it about that first render that makes it different than subsequent calls and therefore worthy of setting the domain? What happens if later, as your app evolves, you decide to render a different dataset first and afterwards render what is currently the first dataset? Then you might end up with a different domain. It seems more sane to compute the domain explicitly, outside of the chart's code, and then pass the domain into the chart via a setter. Something like:
chart.colorDomain(someArrayOfValuesThatYouPreComputeOrHardCode)

D3 sort() with CSV data

I am trying all kinds of ways to make .sort() work on my csv dataset. No luck.
I'd just like to sort my data by a "value" column.
This is the function I'm running inside my d3.csv api call and before I select the dom and append my divs:
dataset = dataset.sort(function (a,b) {return d3.ascending(a.value, b.value); });
Before I get to the .sort, I clean the data:
dataset.forEach(function(d) {
d.funded_month = parseDate(d.funded_month);
d.value = +d.value;
});
};
Everything seems in order. When I console.log(d3.ascending(a.value, b.value)), I get the right outputs:
-1 d32.html:138
1 d32.html:138
-1 d32.html:138
1 d32.html:138
etc..
Yet the bars data doesn't sort.
It is not clear from the provided code but I will hazard a guess you are not handling async nature of d3.csv.
This plunkr shows your sort code working fine. Note where the data object is declared, populated, and used.
here is a partial listing. I have added buttons that re-order data. To achieve this we need to put the ordering logic inside render rather than inside the d3.csv callback.
<script type="text/javascript">
var data = [];
d3.csv("data.csv",
function(error, rows) {
rows.forEach(function(r) {
data.push({
expense: +r.expense,
category: r.category
})
});
render();
});
function render(d3Comparator) {
if(d3Comparator) data = data.sort(function(a, b) {
return d3[d3Comparator](a.expense, b.expense);
});
d3.select("body").selectAll("div.h-bar") // <-B
.data(data)
.enter().append("div")
.attr("class", "h-bar")
.append("span");
d3.select("body").selectAll("div.h-bar") // <-C
.data(data)
.exit().remove();
d3.select("body").selectAll("div.h-bar") // <-D
.style("width", function(d) {
return (d.expense * 5) + "px";
})
.select("span")
.text(function(d) {
return d.category;
});
}
</script>
<button onclick="render('ascending')">Sort ascending!</button>
<button onclick="render('descending')">Sort descending!</button>

Format for D3's data() binding

What is the required format for things passed to d3's .data()?
In this jsfiddle, I try to create several <div> elements for each metric. Unfortunately, nothing happens. I'm assuming this is related to an incorrect data structure?
http://jsfiddle.net/GppWz/
The main issue here is that you are trying to use a hash as a data source, while d3 wants your data in array format.
If you can, modify your data source so that you are receiving data in array format. If this is not possible, you can use the d3.entries function to convert the object into an array:
var listContainers = d3.select('#lists').selectAll('div')
.data(d3.entries(data))
.enter().append('div')
.attr('class', 'listContainer');
listContainers.append('h5')
.text(function(d) {
return d.key;
});
var item = listContainers.selectAll('.item').data(function(d) {
return d.value;
}).enter()
.append('div')
.attr('class', 'item')
.text(function(d) {
return 'average_dif = ' + d.average_dif;
});
// ...

Resources