I have a data set like
{"parent":"/home","inside":"/files","filename":"type.jar",
"extension":"jar","type":"modified","archive"}
Likewise many there are many rows in the json array. I am using crossfilter to read the data and plot graphs and datatables. the Type in the data set has values "added", "modified" and "deleted".
I want to create a data table like
Extension | Added | Modified | Deleted
where added, modified and deleted will hold the count of the files with the specific extension. Can anyone suggest me a way to do so?
So far I have created a dimension like this:
var extensionType = facts.dimension(function(d) {
return d.extension; });
var extensionTypeGroup=extensionType.group();
and I get a grouped output like this,
{"key":"class","value":424},
{"key":"js","value":176},
{"key":"properties","value":26},
{"key":"jar","value":10},
{"key":"css","value":8},
{"key":"txt","value":6},
{"key":"war","value":4},
{"key":"png","value":4},
{"key":"handlebars","value":4},
{"key":"jar_local","value":2},
{"key":"aar","value":2}
How do I get the separate count of added deleted and modified?
Probably the easiest way to do this is to reduce to an object rather than a single value.
This is covered in the FAQ: How do I reduce multiple values at once? What if rows contain a single value but a different value per row? You probably just needed the right search terms to find it.
Actually it looks like the code from the FAQ will work for you unmodified:
var extensionTypeGroup = extensionType.group().reduce(
function(p, v) { // add
p[v.type] = (p[v.type] || 0) + v.value;
return p;
},
function(p, v) { // remove
p[v.type] -= v.value;
return p;
},
function() { // initial
return {};
});
Related
I have a script running on Google Sheets, which brings data from another spreadsheet/file as an array and sets one of its column's data as a data validation into a cell. Then, as the user picks one option of this data validation, the script goes back to that file and brings its related data and sets it in an adjacent column and this repeats about 3 times, making the process slow.
I was wondering if that would be possible to store the first data collection into the document property and set the data validations by grabbing related information from that data set, instead of going to the other file everytime.
Here's an update, with a working version:
function listaCategorias() {
let listaGeral = sheetBDCadProd.getRange(2, 1, sheetBDCadProd.getLastRow(), 45).getValues();//Gets all values
//Extracts a column of interest for this first data validation setting
let categorias = [];
for (let a = 0; a < listaGeral.length; a++) {
categorias.push(listaGeral[a][17])
}
let uniqueCat = [...new Set(categorias)]; //Gets a list of unique values. Not sure how I'd do that within new Set, so I did a for loop before
//Sets the data validation
const cell = sheetVendSobEnc.getRange('B5');
const validationCat = SpreadsheetApp.newDataValidation().requireValueInList(uniqueCat).setAllowInvalid(false).build();
cell.clearContent();
cell.clearDataValidations();
cell.setDataValidation(validationCat);
//Saves the data into the document property for usage in the next script/data validation
listaGeral = JSON.stringify(listaGeral)
PropertiesService.getDocumentProperties().setProperty('listaGeral', listaGeral);
}
//This is getting one of the columns, based on the option picked..the one generated by the data validation above.
function listaDescricao() {
const categoria = sheetVendSobEnc.getRange('B5').getValue();
const dadosCadProd = PropertiesService.getDocumentProperties().getProperty('listaGeral')
let cadGeral = JSON.parse(dadosCadProd);
//Filters the elements matching the option picked
let filteredNomeSobEnc = cadGeral.filter(function (o) { return o[17] === categoria });
//Filters unique values
let listToApply = filteredNomeSobEnc.map(function (o) { return o[7] }).sort().reverse();
let descUnica = listToApply.filter((v, i, a) => a.indexOf(v) === i);
Logger.log('Descrição Única: ' + descUnica)
}
It's working, but I'd like to know the rooms for improvement here.
Thanks.
I'm trying to create a generic composite chart that would take any csv file, read the columns and create a composite line chart with one line per column.
In this use case, the first column is always TimeStamp in all csv's and this will not be changed, while the rest of the columns may vary.
To get the column names from any csv, I am using the following line of code.
var mappedArray = d3.entries(data[0]);
Say, my mappedArray is like this :
["Date", "c1", "c2", "c3"]
I have created the Dimension for timestamp as follows:
var DateDim = ndx.dimension(function(d){return timeFmt(d[mappedArray[0].key]); });
For a single valued group, i would use something like this:
var testGrp = DateDim.group().reduceSum(function(d){return +d[mappedArray[1].key]; });
For a multi valued group, I tried something like this:
var testGrp1 = testDim.group().reduce(
function(p,v){
++p.count;
p.col1 = +v[mappedArray[1].key];
p.col2 = +v[mappedArray[2].key];
...
p.coln = +v[mappedArray[n].key];
return p;
},...
But, I cannot hard code like this, as the 'n' in mappedArray[n] may keep changing.
How can I recreate this group by using mappedArray.length?
You recreate by using the index field operator []. Use a for loop to create as many p.colX fields as needed.
var testGrp1 = testDim.group().reduce(
function(p,v){
++p.count;
for (let i = 1; i < mappedArray.length; i++) {
p["col" + i] = +v[mappedArray[i].key];
}
return p;
},...
Edit
The answer to your question in the comment about the chart.compose.
A possible way of doing is to create a function that constructs the array needed.
function makeComposeArray() {
var composeArr = [];
for(let i=1; i<=mappedArray.length; i++) {
composeArr.push(
dc.lineChart(chart)
.group(testGrp1, function(d){return d['col'+i]})
.valueAccessor(function(d){return d.value["col" + i]})
);
}
return composeArr;
}
chart.compose(makeComposeArray());
So I've encountered a weird issue when dealing with making Groups based on a variable when the crossfilter is using an array, instead of a literal number.
I currently have an output array of a date, then 4 values, that I then map into a composite graph. The problem is that the 4 values can fluctuate depending on the input given to the page. What I mean is that based on what it receives, I can have 3 values, or 10, and there's no way to know in advance. They're placed into an array which is then given to a crossfilter. When in testing, I was accessing using
dimension.group.reduceSum(function(d) { return d[0]; });
Where 0 was changed to whatever I needed. But I've finished testing, for the most part, and began to adapt it into a dynamic system where it can change, but there's always at least the first two. To do this I created an integer that keeps track of what index I'm at, and then increases it after the group has been created. The following code is being used:
var range = crossfilter(results);
var dLen = 0;
var curIndex = 0;
var dateDimension = range.dimension(function(d) { dLen = d.length; return d[curIndex]; });
curIndex++;
var aGroup = dateDimension.group().reduceSum(function(d) { return d[curIndex]; });
curIndex++;
var bGroup = dateDimension.group().reduceSum(function(d) { return d[curIndex]; });
curIndex++;
var otherGroups = [];
for(var h = 0; h < dLen-3; h++) {
otherGroups[h] = dateDimension.group().reduceSum(function(d) { return d[curIndex]; });
curIndex++;
}
var charts = [];
for(var x = 0; x < dLen - 3; x++) {
charts[x] = dc.barChart(dataGraph)
.group(otherGroups[x], "Extra Group " + (x+1))
.hidableStacks(true)
}
charts[charts.length] = dc.lineChart(dataGraph)
.group(aGroup, "Group A")
.hidableStacks(true)
charts[charts.length] = dc.lineChart(dataGraph)
.group(aGroup, "Group B")
.hidableStacks(true)
The issue is this:
The graph gets built empty. I checked the curIndex variable multiple times and it was always correct. I finally decided to instead check the actual group's resulting data using the .all() method.
The weird thing is that AFTER I used .all(), now the data works. Without a .all() call, the graph cannot determine the data and outputs absolutely nothing, however if I call .all() immediately after the group has been created, it populates correctly.
Each Group needs to call .all(), or only the ones that do will work. For example, when I first was debugging, I used .all() only on aGroup, and only aGroup populated into the graph. When I added it to bGroup, then both aGroup and bGroup populated. So in the current build, every group has .all() called directly after it is created.
Technically there's no issue, but I'm really confused on why this is required. I have absolutely no idea what the cause of this is, and I was wondering if there was any insight into it. When I was using literals, there was no issue, it only happens when I'm using a variable to create the groups. I tried to get output later, and when I do I received NaN for all the values. I'm not really sure why .all() is changing values into what they should be especially when it only occurs if I do it immediately after the group has been created.
Below is a screenshot of the graph. The top is when everything has a .all() call after being created, while the bottom is when the Extra Groups (the ones defined in the for loop) do not have the .all() call anymore. The data is just not there at all, I'm not really sure why. Any thoughts would be great.
http://i.stack.imgur.com/0j1ey.jpg
It looks like you may have run into the classic "generating lambdas from loops" JavaScript problem.
You are creating a whole bunch of functions that reference curIndex but unless you call those functions immediately, they will refer to the same instance of curIndex in the global environment. So if you call them after initialization, they will probably all try to use a value which is past the end.
Instead, you might create a function which generates your lambdas, like so:
function accessor(curIndex) {
return function(d) { return d[curIndex]; };
}
And then each time call .reduceSum(accessor(curIndex))
This will cause the value of curIndex to get copied each time you call the accessor function (or you can think of each generated function as having its own environment with its own curIndex).
I have a spreadsheet that I update on a regular basis. I also have to re-sort the spreadsheet when finished because of the changes made. I need to sort with multiple criteria like the below settings. I have searched for examples but my Google search skills have failed me.
Sort range from A1:E59
[x] Data has header rows
sort by "Priority" A > Z
then by "Open" Z > A
then by "Project" A > Z
Mogsdad's answer works fine if none of your cells have values automatically calculated via a formula. If you do use formulas, though, then that solution will erase all of them and replace them with static values. And even so, it is more complicated than it needs to be, as there's now a built-in method for sorting based on multiple columns. Try this instead:
function onEdit(e) {
var priorityCol = 1;
var openCol = 2;
var projectCol = 3;
var sheet = SpreadsheetApp.getActiveSheet();
var dataRange = sheet.getDataRange();
dataRange.sort([
{column: priorityCol, ascending: true},
{column: openCol, ascending: false},
{column: projectCol, ascending: true}
]);
}
Instead of making a separate function, you can use the built-in onEdit() function, and your data will automatically sort itself when you change any of the values. The sort() function accepts an array of criteria, which it applies one after the other, in order.
Note that with this solution, the first column in your spreadsheet is column 1, whereas if you're doing direct array accesses like in Mogsdad's answer, the first column is column 0. So your numbers will be different.
That is a nice specification, a great place to start!
Remember that Google Apps Script is, to a large extent, JavaScript. If you extend your searching into JavaScript solutions, you'll find plenty of examples of array sorts here on SO.
As it happens, much of what you need is in Script to copy and sort form submission data. You don't need the trigger part, but the approach to sorting can be easily adapted to handle multiple columns.
The workhorse here is the comparison function-parameter, which is used by the JavaScript Array.sort() method. It works through the three columns you've indicated, with ascending or descending comparisons. The comparisons used here are OK for Strings, Numbers and Dates. It could be improved with some cleaning up, or even generalized, but it should be pretty fast as-is.
function sortMySheet() {
var sheet = SpreadsheetApp.getActiveSheet();
var dataRange = sourceSheet.getDataRange();
var data = dataRange.getValues();
var headers = data.splice(0,1)[0]; // remove headers from data
data.sort(compare); // Sort 2d array
data.splice(0,0,headers); // replace headers
// Replace with sorted values
dataRange.setValues(data);
};
// Comparison function for sorting two rows
// Returns -1 if 'a' comes before 'b',
// +1 if 'b' before 'a',
// 0 if they match.
function compare(a,b) {
var priorityCol = 0; // Column containing "Priority", 0 is A
var openCol = 1;
var projectCol = 2;
// First, compare "Priority" A > Z
var result = (a[priorityCol] > b[priorityCol] ) ?
(a[priorityCol] < b[priorityCol] ? -1 : 0) : 1;
if (result == 0) {
// "Priority" matched. Then compare "Open" Z > A
result = (b[openCol] > a[openCol] ) ?
(b[openCol] < a[openCol] ? -1 : 0) : 1;
}
if (result == 0) {
// "Open" matched. Finally, compare "Project" A > Z
result = (a[projectCol] > b[projectCol] ) ?
(a[projectCol] < b[projectCol] ? -1 : 0) : 1;
}
return result;
}
Try this using the Apps Script sort instead of the native JavaScript. I had the same issue with sorting the header row(s) and this solved the issue.
So I think something like this should work:
function onOpen() {
SpreadsheetApp.getActiveSpreadsheet()
.getSheetByName("Form Responses 1").sort(2);
}
Regarding sorting by multiple columns, you can chain that sort() method, with the final sort() having the highest priority, and the first sort() the lowest. So something like this should sort by Start date, then by End date:
function onOpen() {
SpreadsheetApp.getActiveSpreadsheet()
.getSheetByName("Form Responses 1").sort(3).sort(2);
}
Reference link:-
https://support.google.com/docs/thread/16556745/google-spreadsheet-script-how-to-sort-a-range-of-data?hl=en
Not sure if this is still relevant, but you can use the sort() function to define another tab as a sorted version of the original data.
Say your original data is in a tab named Sheet1; I'm also going to act as though your Priority, Open, and Project columns are A, B, and C, respectively.
Create a new tab, and in cell A1 type:
=sort(Sheet1!A1:E59, 1, TRUE, 2, FALSE, 3, TRUE)
The first argument specifies the sheet and range to be sorted, followed by three pairs: the first of each pair specifies the column (A=1, B=2, etc.), and the second specifies ascending (TRUE) or descending (FALSE).
I have local data in a grid. How can I get all of the rows or IDs that are not removed after a user uses the filter toolbar? I need to get all filtered rows, regardless of pagination.
For example, say I begin with 50 rows in the grid. The user uses the filter toolbar and the set of rows decreases to 10 rows. How can I get those ten rows?
There are no direct way to get the information which you need. Internally jqGrid uses $.jgrid.from to filter local data. The main code which uses $.jgrid.from in inside of addLocalData. To get results which you need without studying all the code I suggest to use the fact that all filtered data will be returned by select method of $.jgrid.from (see the line of code). My suggestion is to catch the data before the data will be cut to the page size.
To do this I suggest to use sub-classing: overwriting of the method select method of $.jgrid.from. I demonstrate the technique in the examples created for the answer and this one.
In your case the code will be
var oldFrom = $.jgrid.from,
lastSelected;
$.jgrid.from = function (source, initalQuery) {
var result = oldFrom.call(this, source, initalQuery),
old_select = result.select;
result.select = function (f) {
lastSelected = old_select.call(this, f);
return lastSelected;
};
return result;
};
Now the variable lastSelected will save the array of elements which are results of the last sorting or filtering operation. Because $.jgrid.from is global the data are not connected to the grid. If you have more as one grid on the page it will be uncomfortable. One can fix the small disadvantage with the following line in the code of loadComplate of every grid:
loadComplete: function () {
this.p.lastSelected = lastSelected; // set this.p.lastSelected
}
In the way we introduce new jqGrid parameter lastSelected which will have close structure as data parameter, but will hold only last filtered data.
The following code will display the ids of filtered data in alert message
$("#getIds").click(function () {
var filteredData = $grid.jqGrid('getGridParam', 'lastSelected'), i, n, ids = [],
idName = $grid.jqGrid('getGridParam', 'localReader').id;
if (filteredData) {
for (i = 0, n = filteredData.length; i < n; i++) {
ids.push(filteredData[i][idName]);
}
alert("tolal number of filtered data: " + n + "\n" +
"ids of filtered data:\n" + ids.join(', '));
}
});
I used localReader.id parameter because property name used for local data are typically id or _id_. The _id_ will be used in case of data loaded from the server if one uses loadonce: true option.
The demo demonstrate the approach. If one filter for example only the data from FedEx and then clicks on "Show Ids" button one will see information about all filtered and not only about the data displayed on the current page:
UPDATED: free jqGrid provides new lastSelectedData option. See the demo in the list of demos.
You colud use afterSearch option of the search toolbar:
var filteredIDs = new Array(); //Global variable
$("#"+gridId).jqGrid("filterToolbar", { stringResult:true, searchOnEnter:false,
afterSearch:function(){
filteredIDs = $("#"+gridId).getDataIDs();
}
});
If you want to get the filtered rows instead the filtered IDs, use getRowData() instead of getDataIDs().
All, I found another answer which is far easier to include
loadComplete: function (gridData) {
var isSearchPerformed = $grid.getGridParam("postData")._search;
if (isSearchPerformed) {
$("#spanFilterTotal").text(gridData.records);
}
All you want is below:
$.each($grid.getRowData(), function( index, value ) {
a.push(value["COLUMN_NAME"]); //Get the selected data you want
});