I have a barChart with a d3.time.scale x-axis. I am displaying some data per hour, but the first and last data point bars are always cut in half when using centerBar(true).
(When using centerBar(false) the last bar disappears completely.)
The time window is based upon the data itself and is calculated as follows:
var minDate = dateDim.bottom(1)[0]["timestamp"];
var maxDate = dateDim.top(1)[0]["timestamp"];
.x(d3.time.scale().domain([minDate, maxDate]));
The last line sets the time scale domain to use min and maxDate.
This is how it looks:
I have increased the bar width slightly using .xUnits(function(){return 60;}) as the default is so thin that the first bar disappears within the y-axis.
Also I already tried to change the domain by substracting/adding one hour to min/maxDate, but this results in unexpected behaviour of the first bar.
I used the following to calculate the offset:
minDate.setHours(minDate.getHours() - 1);
maxDate.setHours(maxDate.getHours() + 1);
Is there a fix or workaround for this to add padding before the first and after the last bar?
Subtract an hour from the minDate and add an hour to the maxDate to get an hour worth of padding on each side of your min and max data.
The trick here is to use d3.time.hour.offset and play with offsets until it looks nice.
.x(d3.time.scale().domain([d3.time.hour.offset(minDate, -1), d3.time.hour.offset(maxDate, 2)])); `
See this fiddle http://jsfiddle.net/austinlyons/ujdxhd27/3/
The mistake was not realising JavaScript's passing-by-reference when using objects such as Date objects.
In addition to Austin's answer, which did solve the problem by using d3 functionality, I investigated why my initial attempt by modifying the minDate and maxDate variables failed.
The problem is that when creating the variables
var minDate = dateDim.bottom(1)[0]["timestamp"];
var maxDate = dateDim.top(1)[0]["timestamp"];
I created pointers to the actual data instead of creating new objects with the same value as the minDate and maxDate objects. The line
minDate.setHours(minDate.getHours() - 1);
therefore then manipulated the actual underlying data within the date dimension dateDim, which then led to the peculiar behaviour.
The obvious solution would have been to create new Date() objects like this:
var minDate = new Date(dateDim.bottom(1)[0]["timestamp"]);
var maxDate = new Date(dateDim.top(1)[0]["timestamp"]);
and then do the desired manipulations:
minDate.setHours(minDate.getHours() - 1);
maxDate.setHours(maxDate.getHours() + 1);
Related
I am building a timeline chart - that will change its date scale at the top when the brush becomes small to the scope of 1 day -- but when it hits this mode -- the labels overlap and it looks messy until you get to a 12 hour spread.
What is the best way of cleaning this functionality up so it doesn't overlap. I thought about having 1 line that shows date -- and another line under it that shows the hours at that level.
https://jsfiddle.net/aLh9d51t/
var tFormat = '%Y-%m';
var tTick = 'timeMonth';
if (days < 40) {
tFormat = '%Y-%m-%d';
tTick = 'timeWeek';
}
if (days <= 7) {
tFormat = '%Y-%m-%d';
tTick = 'timeDay';
}
if (days <= 1) {
tFormat = '%Y-%m-%d %H%p';
tTick = 'timeHour';
}
First, you can hide redundant parts of date when possible: show years, months, days only if there are more than one visible. So you definitely do not need years and months when you show hours and minutes.
Just look how default d3 axis handles this (e.g. https://bl.ocks.org/mbostock/1166403).
Second, considering your chart has fixed width, you can fine-tune different formats for different zoom levels (you already do this in your code snippet).
Take a look at this example: http://bl.ocks.org/oluckyman/6199145
It has similar logic as in your code snippet:
https://gist.github.com/oluckyman/6199145#file-axisdaysview-js-L33-L58
But the decision which format to choose depends on chart width:
https://gist.github.com/oluckyman/6199145#file-axisdaysview-js-L72-L75
And third, if you restricted to long labels for some reason, you can rotate them to 30°-45°
Also this could be useful: https://bl.ocks.org/mbostock/4149176
Summary
I have a large collection of data with various datetimes. Currently I have been able to group all my data to display properly in a local timezone; however, when trying to display this data in a different timezone the lines on a lineChart get choppy and the connection between them is not as smooth as when in a local timezone.
I had found this link here detailing a possible solution, but sadly this won't work without me duplicating my entire dataset with times offset from utc and then once as normal. The data I have is not only used in several charts to gain insights about trends and statistics, but there is a raw table used for display/editing/reviewing specific data members. Thus translating one into a utc time and calculating the time change in utc time would throw the other off.
https://groups.google.com/forum/#!msg/d3-js/iWmP9Npv2Go/xyypdLjWu2QJ
My question is: Is there a way to translate datetime data across timezones and have dc.js respect the timezone you would like to display in. I would like the adjusted graphs to look the same way the local graph looks where the lines are not one-sided based on the timezone.
code and photos
fiddle: https://jsfiddle.net/spacarar/j0urt9sy/49/
this is the correctly displaying image for my local timezone. The lines are smooth transitions between dates.
This is the incorrectly displaying image for any other timezone (depending on which side of my local changes orientation from leaning left to leaning right)
A simplified version of my data looks something like this:
var data = [
{
value: 42,
datetime: '2019-10-24T07:18:00.000000'
},
{
value: 10,
datetime: '2019-10-24T07:19:12.000000'
},
{
value: 12,
datetime: '2019-10-29T04:18:00.000000'
},
{
value: 8,
datetime: '2019-10-29T09:18:00.000000'
}
]
which I then fake group to fill in any dates that may not be present in the data and translate them using moment-timezone to end up with a data structure similar to this
{
value: 0,
datetime: moment timezone object with full datetime,
date: moment timezone object representing only date (0 hours, minutes, seconds, ms)
}
this fake grouped/fixed data is then used to create the chart with the following code
var ndx = dc.crossfilter(fakeGroupedData)
var dateDim = ndx.dimension(dc.pluck('date'))
var top = dateDim.top(1)[0] ? dateDim.top(1)[0].date : null
var bottom = dateDim.bottom(1)[0] ? dateDim.bottom(1)[0].date : null
var chart = dc.lineChart('#date-chart')
chart.yAxis().tickFormat(dc.d3.format(',.0f'))
chart.xAxis().ticks(10).tickFormat(d => moment(d).format('M/D'))
chart.dimension(dateDim)
.group(dateDim.group().reduceSum(dc.pluck('value')))
.x(dc.d3.scaleTime().domain([bottom, top]).nice())
.elasticY(true)
.renderArea(true)
.render()
A couple of points about your date-filling.
This is not what's normally meant by a "fake group". You're filling in the source data and all of your crossfilter groups are completely "real" :)
There isn't any point in filling at a higher resolution than you intend to show. To simplify the problem, I changed your code to fill by days, and it worked exactly the same:
let start = moment.tz(startDate, tzSelection).startOf('day')
let end = moment.tz(endDate, tzSelection).endOf('day')
let hours = end.diff(start, 'days')
for (let i = 0; i < hours; i++) {
let fakeTime = moment(start).add(i, 'days')
let date = moment(fakeTime.format('YYYY-MM-DD'))
fakeGroupedData.push({
value: 0,
datetime: fakeTime,
date
})
}
It might be easier to use d3-time, since that integrates tighter with dc.js, but I didn't want to make big changes to your code.
However, you are essentially quantizing by day, so you can set up your dimension to quantize to the beginning of the day in the current timezone, and that will fix your chart:
var dateDim = ndx.dimension(d => d3.timeDay(dc.pluck('date')(d)))
If you do this, you don't need to modify your input dates:
el.date = el.datetime //.clone().startOf('day')
D3 will truncate to the current day, and then crossfilter will bin at that resolution.
https://jsfiddle.net/gordonwoodhull/Lxvcoq3h/19/
Note that it's binning both of the 10/29 entries into one.
In my timezone UTC-5, the moment startOf('day') rounding was causing the first of those entries to land on the 28th, which matches what you said you wanted:
https://jsfiddle.net/gordonwoodhull/Lxvcoq3h/21/
You'll have to decide which one is correct for your application. The main point is that if you're displaying your charts in the local timezone, the data should be quantized to local days.
Good Evening Everyone,
I'm trying to take the data from a database full of hour reports (name, timestamp, hours worked, etc.) and create a plot using dc.js to visualize the data. I would like the timestamp to be on the x-axis, the sum of hours for the particular timestamp on the y-axis, and a new bar graph for each unique name all on the same chart.
It appears based on my objectives that using crossfilter.js the timestamp should be my 'dimension' and then the sum of hours should be my 'group'.
Question 1, how would I then use the dimension and group to further split the data based on the person's name and then create a bar graph to add to my composite graph? I would like for the crossfilter.js functionality to remain intact so that if I add a date range tool or some other user controllable filter, everything updates accordingly.
Question 2, my timestamps are in MySQL datetime format: YYYY-mm-dd HH:MM:SS so how would I go about dropping precision? For instance, if I want to combine all entries from the same day into one entry (day precision) or combine all entries in one month into a single entry (month precision).
Thanks in advance!
---- Added on 2017/01/28 16:06
To further clarify, I'm referencing the Crossfilter & DC APIs alongside the DC NASDAQ and Composite examples. The Composite example has shown me how to place multiple line/bar charts on a single graph. On the composite chart I've created, each of the bar charts I've added a dimension based off of the timestamps in the data-set. Now I'm trying to figure out how to define the groups for each. I want each bar chart to represent the total time worked per timestamp.
For example, I have five people in my database, so I want there to be five bar charts within the single composite chart. Today all five submitted reports saying they worked 8 hours, so now all five bar charts should show a mark at 01/28/2017 on the x-axis and 8 hours on the y-axis.
var parseDate = d3.time.format('%Y-%m-%d %H:%M:%S').parse;
data.forEach(function(d) {
d.timestamp = parseDate(d.timestamp);
});
var ndx = crossfilter(data);
var writtenDimension = ndx.dimension(function(d) {
return d.timestamp;
});
var hoursSumGroup = writtenDimension.group().reduceSum(function(d) {
return d.time_total;
});
var minDate = parseDate('2017-01-01 00:00:00');
var maxDate = parseDate('2017-01-31 23:59:59');
var mybarChart = dc.compositeChart("#my_chart");
mybarChart
.width(window.innerWidth)
.height(480)
.x(d3.time.scale().domain([minDate,maxDate]))
.brushOn(false)
.clipPadding(10)
.yAxisLabel("This is the Y Axis!")
.compose([
dc.barChart(mybarChart)
.dimension(writtenDimension)
.colors('red')
.group(hoursSumGroup, "Top Line")
]);
So based on what I have right now and the example I've provided, in the compose section I should have 5 charts because there are 5 people (obviously this needs to be dynamic in the end) and each of those charts should only show the timestamp: total_time data for that person.
At this point I don't know how to further breakup the group hoursSumGroup based on each person and this is where my Question #1 comes in and I need help figuring out.
Question #2 above is that I want to make sure that the code is both dynamic (more people can be handled without code change), when minDate and maxDate are later tied to user input fields, the charts update automatically (I assume through adjusting the dimension variable in some way), and if I add a names filter that if I unselect names that the chart will update by removing the data for that person.
A Question #3 that I'm now realizing I'll want to figure out is how to get the person's name to show up in the pointer tooltip (the title) along with timestamp and total_time values.
There are a number of ways to go about this, but I think the easiest thing to do is to create a custom reduction which reduces each person into a sub-bin.
First off, addressing question #2, you'll want to set up your dimension based on the time interval you're interested in. For instance, if you're looking at days:
var writtenDimension = ndx.dimension(function(d) {
return d3.time.hour(d.timestamp);
});
chart.xUnits(d3.time.hours);
This will cause each timestamp to be rounded down to the nearest hour, and tell the chart to calculate the bar width accordingly.
Next, here's a custom reduction (from the FAQ) which will create an object for each reduced value, with values for each person's name:
var hoursSumGroup = writtenDimension.group().reduce(
function(p, v) { // add
p[v.name] = (p[v.name] || 0) + d.time_total;
return p;
},
function(p, v) { // remove
p[v.name] -= d.time_total;
return p;
},
function() { // init
return {};
});
I did not go with the series example I mentioned in the comments, because I think composite keys can be difficult to deal with. That's another option, and I'll expand my answer if that's necessary.
Next, we can feed the composite line charts with value accessors that can fetch the value by name.
Assume we have an array names.
compositeChart.shareTitle(false);
compositeChart.compose(
names.map(function(name) {
return dc.lineChart(compositeChart)
.dimension(writtenDimension)
.colors('red')
.group(hoursSumGroup)
.valueAccessor(function(kv) {
return kv.value[name];
})
.title(function(kv) {
return name + ' ' + kv.key + ': ' + kv.value;
});
}));
Again, it wouldn't make sense to use bar charts here, because they would obscure each other.
If you filter a name elsewhere, it will cause the line for the name to drop to zero. Having the line disappear entirely would probably not be so simple.
The above shareTitle(false) ensures that the child charts will draw their own titles; the title functions just add the current name to those titles (which would usually just be key:value).
So this is a question regarding a rather specific problem. As I know from Gordon, main contributor of dc.js, there is no support for elasticY(true) function for logarithmic scales.
So, after knowing this, I tried to implement my own solution, by building a workaround, inside dc.js's renderlet event. This event is always triggered by a click of the user onto the barchart. What I wanted to do is this:
let groupSize = this.getGroupSize(fakeGroup, this.yValue);
let maximum = group.top(1)[0].value;
let minimum = group.top(groupSize)[groupSize-1].value;
console.log(minimum, maximum);
chart.y(d3.scale.log().domain([minimum, maximum])
.range(this.height, 0)
.nice()
.clamp(true));
I thought, that at this point the "fakeGroup" (which is just group.top(50)) contains only the data points that are NOT filtered out after the user clicked somewhere. However, this group always contains all data points that are in the top 50 and doesn't change on filter events.
What I really wanted is get all data points that are NOT filtered out, to get a new maximum and minimum for the yScale and rescale the yAxis accordingly by calling chart.y(...) again.
Is there any way to get only data rows that are still in the chart and not filtered out. I also tried using remove_empty_bins(group) but didn't have any luck with that. Somewhere is always all() or top() missing, even after giving remove_empty_bins both functions.
This is how i solved it:
I made a function called rescale(), which looks like this:
rescale(chart, group, fakeGroup) {
let groupSize = this.getGroupSize(fakeGroup, this.yValue);
let minTop = group.top(groupSize)[groupSize-1].value;
let minimum = minTop > 0 ? minTop : 0.0001;
let maximum = group.top(1)[0].value;
chart.y(d3.scale.log().domain([minimum, maximum])
.range(this.height, 0)
.nice()
.clamp(true));}
I think the parameters are pretty self-explanatory, I just get my chart, the whole group as set by dimension.group.reduceSum and a fake group I created, which contains the top 50 elements, to reduce bar count of my chart.
The rescale() method is called in the event listener
chart.on('preRedraw', (chart) => {
this.rescale(chart, group, fakeGroup);
}
So what I do is re-defining (re-setting min and max values regarding filtered data) the charts yAxis everytime the chart gets redrawn, which happens to also be every time one of my charts is filtered. So now, the scale always fits the filtered data the chart contains after filtering another chart.
I'm using code similar to that in the dc.js annotated example:
var ndx = crossfilter(data);
...
var dayName=["0.Sun","1.Mon","2.Tue","3.Wed","4.Thu","5.Fri","6.Sat"];
var dayOfWeek = ndx.dimension(function (d) {
var day = d.dd.getDay();
return dayName[day];
});
var dayOfWeekGroup = dayOfWeek.group();
var dayOfWeekChart = dc.rowChart("#day-of-week-chart");
dayOfWeekChart.width(180)
.height(180)
.group(dayOfWeekGroup)
.label(function(d){return d.key.substr(2);})
.dimension(dayOfWeek);
The issue I've got is that only days of the week present in the data are displayed in my rowChart, and there's no guarantee every day will be represented in all of my data sets.
This is desirable behaviour for many types of categories, but it's a bit disconcerting to omit them for short and well-known lists like day and month names and I'd rather an empty row was included instead.
For a barChart, I can use .xUnits(dc.units.ordinal) and something like .x(d3.scale.ordinal.domain(dayName)).
Is there some way to do the same thing for a rowChart so that all days of the week are displayed, whether present in data or not?
From my understanding of the crossfilter library, I need to do this at the chart level, and the dimension is OK as is. I've been digging around in the dc.js 1.6.0 api reference, and the d3 scales documentation but haven't had any luck finding what I'm looking for.
Solution
Based on #Gordon's answer, I've added the following function:
function ordinal_groups(keys, group) {
return {
all: function () {
var values = {};
group.all().forEach(function(d, i) {
values[d.key] = d.value;
});
var g = [];
keys.forEach(function(key) {
g.push({key: key,
value: values[key] || 0});
});
return g;
}
};
}
Calling this as follows will fill in any missing rows with 0s:
.group(ordinal_groups(dayNames, dayOfWeekGroup))
Actually, I think you are better off making sure that the groups exist before passing them off to dc.js.
One way to do this is the "fake group" pattern described here:
https://github.com/dc-js/dc.js/wiki/FAQ#filter-the-data-before-its-charted
This way you can make sure the extra entries are created every time the data changes.
Are you saying that you tried adding the extra entries to the ordinal domain and they still weren't represented in the row chart, whereas this did work for bar charts? That sounds like a bug to me. Specifically, it looks like support for ordinal domains needs to be added to the row chart.