My data look like this:
{
"raw_data": [
{
"agebracket": "",
"currentstatus": "Hospitalized",
"dateannounced": "05/06/2020",
"gender": "",
....
},
{
"agebracket": "",
"currentstatus": "Recovered",
"dateannounced": "05/06/2020",
"gender": "",
.........
},
{
"agebracket": "",
"currentstatus": "Hospitalized",
"dateannounced": "05/06/2020",
"gender": "",
.......
},
I am able to plot dc graph for the whole dataset. But now I want to filter it by "currentstatus" --> "Recovered", "Hospitalized", "Deceased".
Right now it looks like this:
https://blockbuilder.org/ninjakx/3699d4c0efb0ac1d81636cf0e05eda2d
I am trying to integrate it with https://blockbuilder.org/ninjakx/fbbae54c3f4d8b2df8f9b981d46857b4.
When I will click on confirmed box then pie and histogram will show results related to hospitalized. In that above (dashboard) Those three graphs didn't have to cross filter, So I was able to write the logic. But I am confused about this one. How should I go about filtering it by "currentstatus" ("Hospitalized, "Recovered" etc)
I want this to take a "currentstatus" variable and return results related to it.
var group = dim.group(function(d) {
return binwidth * Math.floor(d/binwidth); });
So that I can do :
barChart
.height(300)
.width(500) //give it a width
.dimension(dim)
.group(group, currentstatus) //<------------------ Here
.......
I am thinking about creating three arrays for Hospitalized, Recovered, and Deceased Resp. But I think there might be some shorter solution than going this lengthy way.
EDIT:
I tried that too but I don't know how to count.
The result should be(calculated using pandas) :
gender currentstatus
F Deceased 31
Hospitalized 4225
Recovered 33
M Deceased 60
Hospitalized 7570
Recovered 50
But I get this:
0:
key: "M"
value: {Hospitalized: 7549, Deceased: 51, Recovered: 13}
__proto__: Object
1: {key: "NA", value: {…}}
2:
key: "F"
value: {Hospitalized: 4200, Deceased: 25, Recovered: 7}
__proto__: Object
length: 3
__proto__: Array(0)
How do I count? I know this below logic is wrong:
var group1 = pieTypeDimension.group().reduce(
function(p, v) { // add
p[v.currentstatus] = (p[v.currentstatus] || 0) + 1;
return p;
},
function(p, v) { // remove
p[v.currentstatus] -= 1;
return p;
},
function() { // initial
return {};
});
log("group1:::", group1.top(Infinity));
Tried this too:
var group1 = pieTypeDimension.group().reduce(
function(p, v) { // add
++p.count;
log("count:::", p.count);
p[v.currentstatus] = (p[v.currentstatus] || 0) + p.count;
return p;
},
function(p, v) { // remove
--p.count;
p[v.currentstatus] -= p.count;
return p;
},
function(p, v) { // initial
return {count:0};
});
and get this:
0:
key: "M"
value:
count: 7613
Hospitalized: 28769566
Deceased: 173237
Recovered: 39888
__proto__: Object
__proto__: Object
1:
key: "NA"
value: {count: 3, Hospitalized: 6}
__proto__: Object
2:
key: "F"
value: {count: 4232, Hospitalized: 8903341, Deceased: 43001, Recovered: 10686}
__proto__: Object
length: 3
__proto__: Array(0)
Edit:
I didn't take account of age bracket thing. Considering that Now it matches with the dc.js solution. So that logic is correct.
gender currentstatus
F Deceased 25
Hospitalized 4200
Recovered 7
M Deceased 51
Hospitalized 7549
Recovered 13
Name: currentstatus, dtype: int64
I will answer in a way that works with filtering, because that's the primary use case for dc.js.
I'd suggest going with the idiomatic crossfilter reduction for stacked charts, just without actually stacking anything.
From the FAQ:
var group = dimension.group().reduce(
function(p, v) { // add
p[v.type] = (p[v.type] || 0) + v.value;
return p;
},
function(p, v) { // remove
p[v.type] -= v.value;
return p;
},
function() { // initial
return {};
});
where type in your case is currentstatus.
This will give you a group where the values are objects keyed on status.
If you have every status for every X value, then each value object will have all status as keys; if not, some will be undefined.
Use valueAccessor to pull the field that you want for your chart, defaulting to 0 if undefined:
chart.valueAccessor(kv => kv.value[currentstatus] || 0)
I am listening to an observable an after the first emit with all the objects, I would to get only the object that changed. So if I have:
[{name: 'Mark'},{name: 'Joe'}]
and then a name change I only get the object that changed. So if the object becomes:
[{name: 'Jean Mark'},{name: 'Joe'}]
I only get
[{name: 'Jean Mark'}]
Your Observable emits arrays and you want to know the difference between the currently emitted array and the previous one. Tracking array state changes has more to do with how to compare arrays or objects than with Observables.
If you want to track changes within an Observable it really comes down to comparing a previous with a current value. The logic you want to use here is up to you. e.g. you have to think about how to distinguish between a 'modified' value and newly 'added' value in an array?
Check out these questions to get you started:
How to get the difference between two arrays in JavaScript?
Comparing Arrays of Objects in JavaScript
How to determine equality for two JavaScript objects?
You can compare the current value cv to the previous one pv in an Observable by using pairwise. Here is a how it could look like.
const source = of(
[{ name: "Mark", p: 2 }, { name: "Joe", p: 3 }],
[{ name: "Jean Mark", p: 2 }, { name: "Joe", p: 3 }],
[{ name: "Jean Mark", p: 1 }, { name: "Joe", p: 3 }, { name: 'Alice' }],
[{ name: "Jean Mark", p: 1 }, { name: "Joe", p: 3 }],
[{ name: "Jean Mark", p: 1 }, { name: "Joe", p: 4 }],
[{ name: "Jean Mark", p: 1 }, { name: "Joe", p: 4 }]
);
// compare two objects
const objectsEqual = (o1, o2) =>
typeof o1 === "object" && Object.keys(o1).length > 0
? Object.keys(o1).length === Object.keys(o2).length &&
Object.keys(o1).every(p => objectsEqual(o1[p], o2[p]))
: o1 === o2;
// compare two arrays
// REPLACE this function with YOUR OWN LOGIC to get your desired output !!!
const difference = (prev, curr) => ({
added: curr.filter(o1 => !prev.some(o2 => objectsEqual(o1, o2))),
removed: prev.filter(o1 => !curr.some(o2 => objectsEqual(o1, o2)))
})
source.pipe(
startWith([]), // used so that pairwise emits the first value immediately
pairwise(), // emit previous and current value
map(([pv, cv]) => difference(pv, cv)) // map to difference between pv and cv
).subscribe(console.log);
https://stackblitz.com/edit/rxjs-m9ngjy?file=index.ts
You can watch an array (index value/add/remove) with javascript proxy, but that doesn't watch for object change in the array.
const handler = {
set: function(target, property, value, receiver){
console.log('setting ' + property + ' for ' + target + ' with value ' + value);
target[property] = value;
return true;
}
}
const arr=[{name: 'Mark'},{name: 'Joe'}];
const proxy = new Proxy(arr, handler);
// will log
proxy[0]="hello"
// won't log
proxy[0].name="ben"
if you also want to watch for object change then you need to either use proxy for every object added, or create your to be added object with Object.defineProperty()
and add your setter
There is also an existing library that watch for both object and array change, and it also use proxy
https://github.com/ElliotNB/observable-slim/
I need to search in a big json nested collection which have unique IDs recursively. The collection contains key values or nested arrays which contains keys. Keys can be anywhere in the object. Keys can be number or string.
Please note: Key values are unique if they are not in array. If they are in array, the key duplicates per items in array. For example,
"WebData": {
WA1: 3, //not in array so unique
WA3: 2, so unique
WA3: "NEO",
WebGroup : [
{ Web1: 1, //duplicate Web1
Web2: 2
},
{ Web1: 2, //duplicate Web2
Web2: 2
}]
}
What I want:
I will pass an array of keys in different variations for example
Not in Arrays: I will pass key return either their values or sum for example:
function(["WA1",""WA3", "RAE1"],"notsum")
If I pass (not sum)
["WA1",""WA3", "RAE1"]
and the operation is not "sum", it should return an array of their values from the collection
[3,2,1]
If I pass the same but operation is sum)
function(["WA1",""WA3", "RAE1"],"sum")
["WA1",""WA3", "RAE1"]
it should return sum from the collection
return 6
If in Array: If the value to search are in the array means they duplicate, then it should return me sum or their individual values again For example
["WEB1","Web2"]
. It could either return me,
[7,1] //Again total of 3+4, 0+1 //see in example
or
[[3,4],[0,1]] //Because values are duplicate and in array, just collect them
I need to do in an elegant way:
Full example of JSON:
{
version: "1.0"
submission : "editing"
"WebData": {
WA1: 3,
WA3: 2,
WA3: "NEO",
WebGroup : [
{ Web1: 3,
Web2: 0
},
{ Web1: 4,
Web2: 1
}]
},
"NonWebData": {
NWA1: 3,
NWA2: "INP",
NWA3: 2,
},
"FormInputs": {
FM11: 3,
FM12: 1,
FM13: 2,
"RawData" : {
"RawOverview": {
"RAE1" : 1,
"RAE2" : 1,
},
"RawGroups":[
{
"name": "A1",
"id": "1",
"data":{
"AD1": 'period',
"AD2": 2,
"AD3": 2,
"transfers": [
{
"type": "in",
"TT1": 1,
"TT2": 2,
},
{
"type": "out",
"TT1": 1,
"TT2": 2,
}
]
}
},
{
"name": "A2",
"id": "2",
"data":{
"AD1": 'period',
"AD2": 2,
"AD3": 2,
"transfers": [
{
"type": "in",
"TT1": 1,
"TT2": 2,
},
{
"type": "out",
"TT1": 1,
"TT2": 2,
}
]
}
}
]
},
"Other":
{ O1: 1,
O2: 2,
O3: "hello"
},
"AddedBy": "name"
"AddedDate": "11/02/2019"
}
I am not able to write a function here, which can do this for me, my code is simply searching in this array, and I loop through to find it, which is I am sure not the correct way.
My code is not elegant, and I am using somehow repetitive functions. This is just one snippet, to find out the keys in one level. I want only 1 or 2 functions to do all this
function Search(paramKey, formDataArray) {
var varParams = [];
for (var key in formDataArray) {
if (formDataArray.hasOwnProperty(key)) {
var val = formDataArray[key];
for (var ikey in val) {
if (val.hasOwnProperty(ikey)) {
if (ikey == paramKey)
varParams.push(val[ikey]);
}
}
}
}
return varParams;
}
One more test case if in Array: to Return only single array of values, without adding. (Update - I achieved this through editing the code following part)
notsumsingle: function (target, key, value) {
if (target[key] === undefined) {
target[key] = value;
return;
}
target.push(value);
},
"groupData": [
{
"A1G1": 1,
"A1G2": 22,
"AIG3": 4,
"AIG4": "Rob"
},
{
"A1G1": 1,
"A1G2": 41,
"AIG3": 3,
"AIG4": "John"
},
{
"A1G1": 1,
"A1G2": 3,
"AIG3": 1,
"AIG4": "Andy"
}
],
perform(["AIG2",""AIG4"], "notsum")
It is returning me
[
[
22,
41,
3
]
],
[
[
"",
"Ron",
"Andy"
]
]
Instead, can I add one more variation "SingleArray" like "sum" and "notsum" and get the result as single Array.
[
22,
41,
3
]
[
"",
"Ron",
"Andy"
]
4th one, I asked, is it possible the function intelligent enough to pick up the sum of arrays or sum of individual fields automatically. for example, in your example, you have used "sum" and "total" to identify that.
console.log(perform(["WA1", "WA3", "RAE1"], "total")); // 6
console.log(perform(["Web1", "Web2"], "sum")); // [7, 1]
Can the function, just use "sum" and returns single or array based on if it finds array, return [7,1] if not return 6
5th : I found an issue in the code, if the json collection is added this way
perform(["RAE1"], "notsum") //[[1,1]]
perform(["RAE1"], "sum") //2
It returns [1, 1], or 2 although there is only one RAE1 defined and please note it is not an array [] so it should not be encoded into [[]] array, just the object key
"RawData" : {
"RawOverview": {
"RAE1" : 1,
"RAE2" : 1,
}
For making it easier, and to take the same interface for getting sums or not sums and a total, without any array, you could introduce another operation string total for getting the sum of all keys.
This approach takes an object for getting a function which either add an value to an array at the same index or stores the value at an specified index, which match the given keys array of the function.
For iterating the object, you could take the key/value pairs and iterate until no more object is found.
As result, you get an array, or the total sum of all items.
BTW, the keys of an object are case sensitive, for example 'WEB1' does not match 'Web1'.
function perform(keys, operation) {
function visit(object) {
Object
.entries(object)
.forEach(([k, v]) => {
if (k in indices) return fn(result, indices[k], v);
if (v && typeof v === 'object') visit(v);
});
}
var result = [],
indices = Object.assign({}, ...keys.map((k, i) => ({ [k]: i }))),
fn = {
notsum: function (target, key, value) {
if (target[key] === undefined) {
target[key] = value;
return;
}
if (!Array.isArray(target[key])) {
target[key] = [target[key]];
}
target[key].push(value);
},
sum: function (target, key, value) {
target[key] = (target[key] || 0) + value;
}
}[operation === 'total' ? 'sum' : operation];
visit(data);
return operation === 'total'
? result.reduce((a, b) => a + b)
: result;
}
var data = { version: "1.0", submission: "editing", WebData: { WA1: 3, WA3: 2, WAX: "NEO", WebGroup: [{ Web1: 3, Web2: 0 }, { Web1: 4, Web2: 1 }] }, NonWebData: { NWA1: 3, NWA2: "INP", NWA3: 2 }, FormInputs: { FM11: 3, FM12: 1, FM13: 2 }, RawData: { RawOverview: { RAE1: 1, RAE2: 1 }, RawGroups: [{ name: "A1", id: "1", data: { AD1: 'period', AD2: 2, AD3: 2, transfers: [{ type: "in", TT1: 1, TT2: 2 }, { type: "out", TT1: 1, TT2: 2 }] } }, { name: "A2", id: "2", data: { AD1: 'period', AD2: 2, AD3: 2, transfers: [{ type: "in", TT1: 1, TT2: 2 }, { type: "out", TT1: 1, TT2: 2 }] } }] }, Other: { O1: 1, O2: 2, O3: "hello" }, AddedBy: "name", AddedDate: "11/02/2019" };
console.log(perform(["WA1", "WA3", "RAE1"], "notsum")); // [3, 2, 1]
console.log(perform(["WA1", "WA3", "RAE1"], "total")); // 6
console.log(perform(["Web1", "Web2"], "sum")); // [7, 1]
console.log(perform(["Web1", "Web2"], "notsum")); // [[3, 4], [0, 1]]
.as-console-wrapper { max-height: 100% !important; top: 0; }
I would like to show sum of multiple values as one chart output with amCharts. I am using dataLoader with JSON to get the data. I know I have to create a function for but I couldn't understand how to get the data from the dataLoader to calculate
{
"balloonText": "[[title]] of [[valueAxis]]:[[value]]",
"lineThickness": 3,
"id": "sumValue",
"title": "sum Value",
"valueField": (function() {
var sumValues = "calculation";
return sumValues
}
this attempt is probably not correct but this is how I started
{
"balloonText": "[[title]] of [[valueAxis]]:[[value]]",
"lineThickness": 3,
"id": "LoadigTime",
"title": "Loadig Time",
"valueField": (function() {
var sumValues = (HomePageLoad + LoginToParametersLoad + ParametersLoad + AlarmsLoad + SwitchSideLoad + LoginToAdminLoad + AdminLoad) / 7;
return sumValues
})
}
valueField cannot be a function, only a string reference to a field in your data.
If the chart is meant to be displaying the sum of all of those fields in your data as a chart, simply add logic to your postProcess callback to create a new dataset containing your sums, e.g.
postProcess: function(data) {
var newData = [];
data.forEach(function(dataItem) {
var item = {
YOUR_CATEGORY_FIELD: dataItem.YOUR_CATEGORY_FIELD, //replace with your category field name
sum: 0
};
//loop through your item's keys and sum everything up, filtering out
//your category property
item.sum = Object.keys(dataItem).reduce(function(sum, key) {
if (key !== "YOUR_CATEGORY_FIELD") {
sum += dataItem[key]
}
return sum;
}, 0);
newData.push(item);
});
return newData;
},
// ...
graphs: [{
valueField: "sum",
// other props here
}]
Is it possible to filter a crossfilter dataset which has an array as the value?
For example, say I have the following dataset:
var data = [
{
bookname: "the joy of clojure",
authors: ["Michael Fogus", "Chris Houser"],
tags: ["clojure", "lisp"]
},
{
bookname: "Eloquent Ruby",
authors: ["Russ Olsen"],
tags: ["ruby"]
},
{
bookname: "Design Patterns in Ruby",
authors: ["Russ Olsen"],
tags: ["design patterns", "ruby"]
}
];
Is there an easy way to access the books which are tagged by an particular tag? And also the books which have a particular author? The way I understand how to use crossfilter so far has me doing something like this:
var filtered_data = crossfilter(data);
var tags = filtered_data.dimension(function(d) {return d.tags});
var tag = tags.group();
And then when I access the grouping (like so):
tag.all()
I get this:
[{key: ["clojure", "lisp"], value: 1},
{key: ["design patterns", "ruby"], value: 1},
{key: ["ruby"], value: 1}]
When I would rather have this:
[{key: "ruby", value: 2},
{key: "clojure", value: 1},
{key: "lisp", value: 1},
{key: "design patterns", value: 1}]
I've added comments to the code below. Big picture: use reduce function.
var data = ...
var filtered_data = crossfilter(data);
var tags = filtered_data.dimension(function(d) {return d.tags});
tags.groupAll().reduce(reduceAdd, reduceRemove, reduceInitial).value()
Notice how I've used groupAll() instead of group() b/c we want our reduce functions (defined below) to operate on one group rather than 3 groups.
Now the reduce functions should look like this:
/*
v is the row in the dataset
p is {} for the first execution (passed from reduceInitial).
For every subsequent execution it is the value returned from reduceAdd of the prev row
*/
function reduceAdd(p, v) {
v.tags.forEach (function(val, idx) {
p[val] = (p[val] || 0) + 1; //increment counts
});
return p;
}
function reduceRemove(p, v) {
//omitted. not useful for demonstration
}
function reduceInitial() {
/* this is how our reduce function is seeded. similar to how inject or fold
works in functional languages. this map will contain the final counts
by the time we are done reducing our entire data set.*/
return {};
}
I've never used "crossfilter" (I'm assuming this is a JS library). Here are some pure JS methods though.
This...
data.filter(function(d) {
return d.authors.indexOf("Michael Fogus") !== -1;
})
returns this:
[{bookname:"the joy of clojure", authors:["Michael Fogus", "Chris Houser"], tags:["clojure", "lisp"]}]
This...
var res = {};
data.forEach(function(d) {
d.tags.forEach(function(tag) {
res.hasOwnProperty(tag) ? res[tag]++ : res[tag] = 1
});
})
returns this:
({clojure:1, lisp:1, ruby:2, 'design patterns':1})
To either of these, you can apply d3.entries to get your {key:"ruby", value: 2} format.