My data look like this:
{
"raw_data": [
{
"agebracket": "",
"currentstatus": "Hospitalized",
"dateannounced": "05/06/2020",
"gender": "",
....
},
{
"agebracket": "",
"currentstatus": "Recovered",
"dateannounced": "05/06/2020",
"gender": "",
.........
},
{
"agebracket": "",
"currentstatus": "Hospitalized",
"dateannounced": "05/06/2020",
"gender": "",
.......
},
I am able to plot dc graph for the whole dataset. But now I want to filter it by "currentstatus" --> "Recovered", "Hospitalized", "Deceased".
Right now it looks like this:
https://blockbuilder.org/ninjakx/3699d4c0efb0ac1d81636cf0e05eda2d
I am trying to integrate it with https://blockbuilder.org/ninjakx/fbbae54c3f4d8b2df8f9b981d46857b4.
When I will click on confirmed box then pie and histogram will show results related to hospitalized. In that above (dashboard) Those three graphs didn't have to cross filter, So I was able to write the logic. But I am confused about this one. How should I go about filtering it by "currentstatus" ("Hospitalized, "Recovered" etc)
I want this to take a "currentstatus" variable and return results related to it.
var group = dim.group(function(d) {
return binwidth * Math.floor(d/binwidth); });
So that I can do :
barChart
.height(300)
.width(500) //give it a width
.dimension(dim)
.group(group, currentstatus) //<------------------ Here
.......
I am thinking about creating three arrays for Hospitalized, Recovered, and Deceased Resp. But I think there might be some shorter solution than going this lengthy way.
EDIT:
I tried that too but I don't know how to count.
The result should be(calculated using pandas) :
gender currentstatus
F Deceased 31
Hospitalized 4225
Recovered 33
M Deceased 60
Hospitalized 7570
Recovered 50
But I get this:
0:
key: "M"
value: {Hospitalized: 7549, Deceased: 51, Recovered: 13}
__proto__: Object
1: {key: "NA", value: {…}}
2:
key: "F"
value: {Hospitalized: 4200, Deceased: 25, Recovered: 7}
__proto__: Object
length: 3
__proto__: Array(0)
How do I count? I know this below logic is wrong:
var group1 = pieTypeDimension.group().reduce(
function(p, v) { // add
p[v.currentstatus] = (p[v.currentstatus] || 0) + 1;
return p;
},
function(p, v) { // remove
p[v.currentstatus] -= 1;
return p;
},
function() { // initial
return {};
});
log("group1:::", group1.top(Infinity));
Tried this too:
var group1 = pieTypeDimension.group().reduce(
function(p, v) { // add
++p.count;
log("count:::", p.count);
p[v.currentstatus] = (p[v.currentstatus] || 0) + p.count;
return p;
},
function(p, v) { // remove
--p.count;
p[v.currentstatus] -= p.count;
return p;
},
function(p, v) { // initial
return {count:0};
});
and get this:
0:
key: "M"
value:
count: 7613
Hospitalized: 28769566
Deceased: 173237
Recovered: 39888
__proto__: Object
__proto__: Object
1:
key: "NA"
value: {count: 3, Hospitalized: 6}
__proto__: Object
2:
key: "F"
value: {count: 4232, Hospitalized: 8903341, Deceased: 43001, Recovered: 10686}
__proto__: Object
length: 3
__proto__: Array(0)
Edit:
I didn't take account of age bracket thing. Considering that Now it matches with the dc.js solution. So that logic is correct.
gender currentstatus
F Deceased 25
Hospitalized 4200
Recovered 7
M Deceased 51
Hospitalized 7549
Recovered 13
Name: currentstatus, dtype: int64
I will answer in a way that works with filtering, because that's the primary use case for dc.js.
I'd suggest going with the idiomatic crossfilter reduction for stacked charts, just without actually stacking anything.
From the FAQ:
var group = dimension.group().reduce(
function(p, v) { // add
p[v.type] = (p[v.type] || 0) + v.value;
return p;
},
function(p, v) { // remove
p[v.type] -= v.value;
return p;
},
function() { // initial
return {};
});
where type in your case is currentstatus.
This will give you a group where the values are objects keyed on status.
If you have every status for every X value, then each value object will have all status as keys; if not, some will be undefined.
Use valueAccessor to pull the field that you want for your chart, defaulting to 0 if undefined:
chart.valueAccessor(kv => kv.value[currentstatus] || 0)
Most examples I have found are using data that has time and number
var data = [
{
"Time": "19-Jan-2018 11:24:49.000 UTC",
"Speed": 1.885
},
{
"Time": "19-Jan-2018 11:24:59.000 UTC",
"Speed": 1.875
},
{
"Time": "19-Jan-2018 11:25:00.000 UTC",
"Speed": 1.878
},
{
"Time": "19-Jan-2018 11:25:01.000 UTC",
"Speed": 1.876
}
]
I am looking to stack type
var data = [
{
"Time": "19-Jan-2018 11:24:49.000 UTC",
"type": "CAT"
},
{
"Time": "19-Jan-2018 11:24:59.000 UTC",
"type": "DOG"
},
{
"Time": "19-Jan-2018 11:25:00.000 UTC",
"type": "CAT"
},
{
"Time": "19-Jan-2018 11:25:01.000 UTC",
"Type": "BAT"
}
]
How can I stack categorical data, while allowing the user to select time/category pairs, as in the following Example?
I adapted the example to time/category data in this fiddle.
Those dates would only parse in Chrome, so
const parseDate = d3.utcParse("%d-%b-%Y %H:%M:%S.%L UTC");
data.forEach(d => {
d.Time = parseDate(d.Time);
})
I changed the key functions to use ,
function multikey(x,y) {
return x + ',' + y;
}
function splitkey(k) {
return k.split(',');
}
I also changed fake group stack_second to convert string-dates from the multikeys back into Dates, and to initialize categories to 0 (since every stack has to be present for every X).
function stack_second(group, categories) {
return {
all: function() {
var all = group.all(),
m = {};
// build matrix from multikey/value pairs
all.forEach(function(kv) {
var ks = splitkey(kv.key);
m[ks[0]] = m[ks[0]] || Object.fromEntries(categories.map(c=>[c,0]));
m[ks[0]][ks[1]] = kv.value;
});
// then produce multivalue key/value pairs
return Object.keys(m).map(function(k) {
return {key: new Date(k), value: m[k]};
});
}
};
}
Get the array of categories from the source data:
const categories = Array.from(new Set(data.map(d => d.Type)).values());
When dealing with date/time, you have to choose a d3 time interval appropriate for your data. Here minutes looked right. Using UTC d3-time methods everywhere because your source data is UTC.
const interval = d3.utcMinute;
Calculate xscale domain and apply:
let extent = d3.extent(data, d=>d.Time);
extent[1] = interval.offset(extent[1], 1)
chart
.x(d3.scaleTime().domain(extent))
.xUnits(interval.range)
Right number of ticks, also formatted in UTC:
chart.xAxis().ticks(d3.utcMinute).tickFormat(d3.utcFormat('%H:%M'))
Match colors between stacks and wedges with
.colors(d3.scaleOrdinal().domain(categories).range(d3.schemeCategory10))
Crossfilter initialization, using the interval and categories:
const interval = d3.utcMinute; // choose appropriate to your data
var cf = crossfilter(data),
timeTypeDim = cf.dimension(function(d) { return multikey(interval(d.Time), d.Type); }),
timeTypeGroup = timeTypeDim.group(), // reduceCount by default
stackedGroup = stack_second(timeTypeGroup, categories);
And here's the chart code for completeness, although we've already discussed the relevant parts:
function sel_stack(i) {
return function(d) {
return d.value[i];
};
}
chart
.width(600)
.height(400)
.colors(d3.scaleOrdinal().domain(categories).range(d3.schemeCategory10))
.controlsUseVisibility(true)
.x(d3.scaleTime().domain(extent))
.xUnits(interval.range)
.margins({left: 80, top: 20, right: 10, bottom: 20})
.brushOn(false)
.clipPadding(10)
.title(function(d) {
return d.key + '[' + this.layer + ']: ' + d.value[this.layer];
})
.legend(dc.legend().x(540).y(50))
.dimension(timeTypeDim)
.group(stackedGroup, categories[0], sel_stack(categories[0]))
.renderLabel(true);
I need to search in a big json nested collection which have unique IDs recursively. The collection contains key values or nested arrays which contains keys. Keys can be anywhere in the object. Keys can be number or string.
Please note: Key values are unique if they are not in array. If they are in array, the key duplicates per items in array. For example,
"WebData": {
WA1: 3, //not in array so unique
WA3: 2, so unique
WA3: "NEO",
WebGroup : [
{ Web1: 1, //duplicate Web1
Web2: 2
},
{ Web1: 2, //duplicate Web2
Web2: 2
}]
}
What I want:
I will pass an array of keys in different variations for example
Not in Arrays: I will pass key return either their values or sum for example:
function(["WA1",""WA3", "RAE1"],"notsum")
If I pass (not sum)
["WA1",""WA3", "RAE1"]
and the operation is not "sum", it should return an array of their values from the collection
[3,2,1]
If I pass the same but operation is sum)
function(["WA1",""WA3", "RAE1"],"sum")
["WA1",""WA3", "RAE1"]
it should return sum from the collection
return 6
If in Array: If the value to search are in the array means they duplicate, then it should return me sum or their individual values again For example
["WEB1","Web2"]
. It could either return me,
[7,1] //Again total of 3+4, 0+1 //see in example
or
[[3,4],[0,1]] //Because values are duplicate and in array, just collect them
I need to do in an elegant way:
Full example of JSON:
{
version: "1.0"
submission : "editing"
"WebData": {
WA1: 3,
WA3: 2,
WA3: "NEO",
WebGroup : [
{ Web1: 3,
Web2: 0
},
{ Web1: 4,
Web2: 1
}]
},
"NonWebData": {
NWA1: 3,
NWA2: "INP",
NWA3: 2,
},
"FormInputs": {
FM11: 3,
FM12: 1,
FM13: 2,
"RawData" : {
"RawOverview": {
"RAE1" : 1,
"RAE2" : 1,
},
"RawGroups":[
{
"name": "A1",
"id": "1",
"data":{
"AD1": 'period',
"AD2": 2,
"AD3": 2,
"transfers": [
{
"type": "in",
"TT1": 1,
"TT2": 2,
},
{
"type": "out",
"TT1": 1,
"TT2": 2,
}
]
}
},
{
"name": "A2",
"id": "2",
"data":{
"AD1": 'period',
"AD2": 2,
"AD3": 2,
"transfers": [
{
"type": "in",
"TT1": 1,
"TT2": 2,
},
{
"type": "out",
"TT1": 1,
"TT2": 2,
}
]
}
}
]
},
"Other":
{ O1: 1,
O2: 2,
O3: "hello"
},
"AddedBy": "name"
"AddedDate": "11/02/2019"
}
I am not able to write a function here, which can do this for me, my code is simply searching in this array, and I loop through to find it, which is I am sure not the correct way.
My code is not elegant, and I am using somehow repetitive functions. This is just one snippet, to find out the keys in one level. I want only 1 or 2 functions to do all this
function Search(paramKey, formDataArray) {
var varParams = [];
for (var key in formDataArray) {
if (formDataArray.hasOwnProperty(key)) {
var val = formDataArray[key];
for (var ikey in val) {
if (val.hasOwnProperty(ikey)) {
if (ikey == paramKey)
varParams.push(val[ikey]);
}
}
}
}
return varParams;
}
One more test case if in Array: to Return only single array of values, without adding. (Update - I achieved this through editing the code following part)
notsumsingle: function (target, key, value) {
if (target[key] === undefined) {
target[key] = value;
return;
}
target.push(value);
},
"groupData": [
{
"A1G1": 1,
"A1G2": 22,
"AIG3": 4,
"AIG4": "Rob"
},
{
"A1G1": 1,
"A1G2": 41,
"AIG3": 3,
"AIG4": "John"
},
{
"A1G1": 1,
"A1G2": 3,
"AIG3": 1,
"AIG4": "Andy"
}
],
perform(["AIG2",""AIG4"], "notsum")
It is returning me
[
[
22,
41,
3
]
],
[
[
"",
"Ron",
"Andy"
]
]
Instead, can I add one more variation "SingleArray" like "sum" and "notsum" and get the result as single Array.
[
22,
41,
3
]
[
"",
"Ron",
"Andy"
]
4th one, I asked, is it possible the function intelligent enough to pick up the sum of arrays or sum of individual fields automatically. for example, in your example, you have used "sum" and "total" to identify that.
console.log(perform(["WA1", "WA3", "RAE1"], "total")); // 6
console.log(perform(["Web1", "Web2"], "sum")); // [7, 1]
Can the function, just use "sum" and returns single or array based on if it finds array, return [7,1] if not return 6
5th : I found an issue in the code, if the json collection is added this way
perform(["RAE1"], "notsum") //[[1,1]]
perform(["RAE1"], "sum") //2
It returns [1, 1], or 2 although there is only one RAE1 defined and please note it is not an array [] so it should not be encoded into [[]] array, just the object key
"RawData" : {
"RawOverview": {
"RAE1" : 1,
"RAE2" : 1,
}
For making it easier, and to take the same interface for getting sums or not sums and a total, without any array, you could introduce another operation string total for getting the sum of all keys.
This approach takes an object for getting a function which either add an value to an array at the same index or stores the value at an specified index, which match the given keys array of the function.
For iterating the object, you could take the key/value pairs and iterate until no more object is found.
As result, you get an array, or the total sum of all items.
BTW, the keys of an object are case sensitive, for example 'WEB1' does not match 'Web1'.
function perform(keys, operation) {
function visit(object) {
Object
.entries(object)
.forEach(([k, v]) => {
if (k in indices) return fn(result, indices[k], v);
if (v && typeof v === 'object') visit(v);
});
}
var result = [],
indices = Object.assign({}, ...keys.map((k, i) => ({ [k]: i }))),
fn = {
notsum: function (target, key, value) {
if (target[key] === undefined) {
target[key] = value;
return;
}
if (!Array.isArray(target[key])) {
target[key] = [target[key]];
}
target[key].push(value);
},
sum: function (target, key, value) {
target[key] = (target[key] || 0) + value;
}
}[operation === 'total' ? 'sum' : operation];
visit(data);
return operation === 'total'
? result.reduce((a, b) => a + b)
: result;
}
var data = { version: "1.0", submission: "editing", WebData: { WA1: 3, WA3: 2, WAX: "NEO", WebGroup: [{ Web1: 3, Web2: 0 }, { Web1: 4, Web2: 1 }] }, NonWebData: { NWA1: 3, NWA2: "INP", NWA3: 2 }, FormInputs: { FM11: 3, FM12: 1, FM13: 2 }, RawData: { RawOverview: { RAE1: 1, RAE2: 1 }, RawGroups: [{ name: "A1", id: "1", data: { AD1: 'period', AD2: 2, AD3: 2, transfers: [{ type: "in", TT1: 1, TT2: 2 }, { type: "out", TT1: 1, TT2: 2 }] } }, { name: "A2", id: "2", data: { AD1: 'period', AD2: 2, AD3: 2, transfers: [{ type: "in", TT1: 1, TT2: 2 }, { type: "out", TT1: 1, TT2: 2 }] } }] }, Other: { O1: 1, O2: 2, O3: "hello" }, AddedBy: "name", AddedDate: "11/02/2019" };
console.log(perform(["WA1", "WA3", "RAE1"], "notsum")); // [3, 2, 1]
console.log(perform(["WA1", "WA3", "RAE1"], "total")); // 6
console.log(perform(["Web1", "Web2"], "sum")); // [7, 1]
console.log(perform(["Web1", "Web2"], "notsum")); // [[3, 4], [0, 1]]
.as-console-wrapper { max-height: 100% !important; top: 0; }
I would like to filter from REST API by nested data. For example this object:
[
{
"name": "Handmade Soft Fish",
"tags": "Rubber, Rubber, Salad",
"categories": [
{
"name": "women",
"id": 2,
"parent_id": 0,
"permalink": "/women"
},
{
"name": "kids",
"id": 3,
"parent_id": 0,
"permalink": "/kids"
}
]
},
{
"name": "Tasty Rubber Soap",
"tags": "Granite, Granite, Chair",
"categories": [
{
"name": "kids",
"id": 3,
"parent_id": 0,
"permalink": "/kids"
}
]
}
]
is comming by GET /api/products?filter[include]=categories
and i would like to get only products which has category name "women". How do this?
LoopBack does not support filters based on related models.
This is a limitation that we have never had bandwidth to solve, unfortunately :(
For more details, see the discussion and linked issues here:
Filter on level 2 properties: https://github.com/strongloop/loopback/issues/517
Filter by properties of related models (use SQL JOIN in queries): https://github.com/strongloop/loopback/issues/683
Maybe you want to get this data by the Category REST API. For example:
GET /api/categories?filter[include]=products&filter[where][name]=woman
The result will be a category object with all products related. To this, will be necessary declare this relation on the models.
Try like this.It has worked for me.
const filter = {
where: {
'categories.name': {
inq: ['women']**strong text**
}
}
};
Pass this filter to request as path parameters and the request would be like bellow
GET /api/categoriesfilter=%7B%22where%22:%7B%categories.name%22:%7B%22inq%22:%5B%women%22%5D%7D%7D%7D
Can you share how it looks like without filter[include]=categorie, please ?
[edit]
after a few questions in comment, I'd build a remote method : in common/models/myModel.js (inside the function) :
function getItems(filter, categorieIds = []) {
return new Promise((resolve, reject) => {
let newInclude;
if (filter.hasOwnProperty(include)){
if (Array.isArray(filter.include)) {
newInclude = [].concat(filter.include, "categories")
}else{
if (filter.include.length > 0) {
newInclude = [].concat(filter.include, "categories");
}else{
newInclude = "categories";
}
}
}else{
newInclude = "categories";
}
myModel.find(Object.assign({}, filter, {include: newInclude}))
.then(data => {
if (data.length <= 0) return resolve(data);
if (categoriesIds.length <= 0) return resolve(data);
// there goes your specific filter on categories
const tmp = data.filter(
item => item.categories.findIndex(
categorie => categorieIds.indexOf(categorie.id) > -1
) > -1
);
return resolve(tmp);
})
}
}
myModel.remoteMethod('getItems', {
accepts: [{
arg: "filter",
type: "object",
required: true
}, {
arg: "categorieIds",
type: "array",
required: true
}],
returns: {arg: 'getItems', type: 'array'}
});
I hope it answers your question...
I'm trying to insert the results of a query from one table into another table. However, when I attempt to run the query I am receiving an error.
{
"deleted": 0 ,
"errors": 1 ,
"first_error": "Expected type OBJECT but found ARRAY." ,
"inserted": 0 ,
"replaced": 0 ,
"skipped": 0 ,
"unchanged": 0
}
Here is the the insert and query:
r.db('test').table('destination').insert(
r.db('test').table('source').map(function(doc) {
var result = doc('result');
return result('section_list').concatMap(function(section) {
return section('section_content').map(function(item) {
return {
"code": item("code"),
"name": item("name"),
"foo": result("foo"),
"bar": result("bar"),
"baz": section("baz"),
"average": item("average"),
"lowerBound": item("from"),
"upperBound": item("to")
};
});
});
});
);
Is there a special syntax for this, or do I have to retrieve the results and then run a separate insert?
The problem is that your inner query is returning a stream of arrays. You can't insert arrays into a table (only objects), so the query fails. If you change the outermost map into a concatMap it should work.
The problem here was that the result was a sequence of an array of objects. i.e
[ [ { a:1, b:2 }, { a:1, b:2 } ], [ { a:2, b:3 } ] ]
Therefore, I had to change the outer map call to a concatMap call. The query then becomes:
r.db('test').table('destination').insert(
r.db('test').table('source').concatMap(function(doc) {
var result = doc('result');
return result('section_list').concatMap(function(section) {
return section('section_content').map(function(item) {
return {
"code": item("code"),
"name": item("name"),
"foo": result("foo"),
"bar": result("bar"),
"baz": section("baz"),
"average": item("average"),
"lowerBound": item("from"),
"upperBound": item("to")
};
)});
});
});
}
Thanks goes to #AtnNn on the #rethinkdb freenode for pointing me in the right direction.