group.all() call required for data to populate correctly - dc.js

So I've encountered a weird issue when dealing with making Groups based on a variable when the crossfilter is using an array, instead of a literal number.
I currently have an output array of a date, then 4 values, that I then map into a composite graph. The problem is that the 4 values can fluctuate depending on the input given to the page. What I mean is that based on what it receives, I can have 3 values, or 10, and there's no way to know in advance. They're placed into an array which is then given to a crossfilter. When in testing, I was accessing using
dimension.group.reduceSum(function(d) { return d[0]; });
Where 0 was changed to whatever I needed. But I've finished testing, for the most part, and began to adapt it into a dynamic system where it can change, but there's always at least the first two. To do this I created an integer that keeps track of what index I'm at, and then increases it after the group has been created. The following code is being used:
var range = crossfilter(results);
var dLen = 0;
var curIndex = 0;
var dateDimension = range.dimension(function(d) { dLen = d.length; return d[curIndex]; });
curIndex++;
var aGroup = dateDimension.group().reduceSum(function(d) { return d[curIndex]; });
curIndex++;
var bGroup = dateDimension.group().reduceSum(function(d) { return d[curIndex]; });
curIndex++;
var otherGroups = [];
for(var h = 0; h < dLen-3; h++) {
otherGroups[h] = dateDimension.group().reduceSum(function(d) { return d[curIndex]; });
curIndex++;
}
var charts = [];
for(var x = 0; x < dLen - 3; x++) {
charts[x] = dc.barChart(dataGraph)
.group(otherGroups[x], "Extra Group " + (x+1))
.hidableStacks(true)
}
charts[charts.length] = dc.lineChart(dataGraph)
.group(aGroup, "Group A")
.hidableStacks(true)
charts[charts.length] = dc.lineChart(dataGraph)
.group(aGroup, "Group B")
.hidableStacks(true)
The issue is this:
The graph gets built empty. I checked the curIndex variable multiple times and it was always correct. I finally decided to instead check the actual group's resulting data using the .all() method.
The weird thing is that AFTER I used .all(), now the data works. Without a .all() call, the graph cannot determine the data and outputs absolutely nothing, however if I call .all() immediately after the group has been created, it populates correctly.
Each Group needs to call .all(), or only the ones that do will work. For example, when I first was debugging, I used .all() only on aGroup, and only aGroup populated into the graph. When I added it to bGroup, then both aGroup and bGroup populated. So in the current build, every group has .all() called directly after it is created.
Technically there's no issue, but I'm really confused on why this is required. I have absolutely no idea what the cause of this is, and I was wondering if there was any insight into it. When I was using literals, there was no issue, it only happens when I'm using a variable to create the groups. I tried to get output later, and when I do I received NaN for all the values. I'm not really sure why .all() is changing values into what they should be especially when it only occurs if I do it immediately after the group has been created.
Below is a screenshot of the graph. The top is when everything has a .all() call after being created, while the bottom is when the Extra Groups (the ones defined in the for loop) do not have the .all() call anymore. The data is just not there at all, I'm not really sure why. Any thoughts would be great.
http://i.stack.imgur.com/0j1ey.jpg

It looks like you may have run into the classic "generating lambdas from loops" JavaScript problem.
You are creating a whole bunch of functions that reference curIndex but unless you call those functions immediately, they will refer to the same instance of curIndex in the global environment. So if you call them after initialization, they will probably all try to use a value which is past the end.
Instead, you might create a function which generates your lambdas, like so:
function accessor(curIndex) {
return function(d) { return d[curIndex]; };
}
And then each time call .reduceSum(accessor(curIndex))
This will cause the value of curIndex to get copied each time you call the accessor function (or you can think of each generated function as having its own environment with its own curIndex).

Related

Cypress: Get JQuery value without needing `then` or `each`

I'm hoping someone can help, but I've posted this as a Cypress discussion as well, although it might just be my understanding that's wrong
I need to get the Cypress.Chainable<JQuery<HTMLElement>> of the cell of a table using the column header and another cell's value in the same row.
Here's a working example JQuery TS Fiddle: https://jsfiddle.net/6w1r7ha9/
My current implementation looks as follows:
static findCellByRowTextColumnHeaderText(
rowText: string,
columnName: string,
) {
const row = cy.get(`tr:contains(${rowText})`);
const column = cy.get(`th:contains(${columnName})`)
const columnIndex = ???
return row.find(`td:eq(${columnIndex})`)
}
This function is required because I want to write DRY code to find cells easily for content verification, clicking elements inside of it etc.
The only example I've seen is this https://stackoverflow.com/a/70686525/1321908, but the following doesn't work:
const columns = cy.get('th')
let columnIndex = -1
columns.each((el, index) => {
if (el.text().includes(columnName) {
columnIndex = index
}
cy.log('columnIndex', columnIndex) // Outputs 2 as expected
})
cy.log('finalColumnIndex', columnIndex) // Outputs -1
My current thinking is something like:
const columnIndex: number = column.then((el) => el.index())
This however returns a Chainable<number> How to turn it into number, I have no idea. I'm using this answer to guide my thinking in this regard.
Using .then() in a Cypress test is almost mandatory to avoid flaky tests.
To avoid problems with test code getting ahead of web page updating, Cypress uses Chainable to retry the DOM query until success, or time out.
But the Chainable interface isn't a promise, so you can't await it. You can only then() it.
It would be nice if you could substitute another keyword like unchain
const column = unchain cy.get(`th:contains(${columnName})`)
but unfortunately Javascript can't be extended with new keywords. You can only add methods like .then() onto objects like Chainable.
Having said that, there are code patterns that allow extracting a Chainable value and using it like a plain Javascript variable.
But they are limited to specific scenarios, for example assigning to a global in a before() and using it in an it().
If you give up the core feature of Cypress, the automatic retry feature, then it's just jQuery exactly as you have in the fiddle (but using Cypress.$() instead of $()).
But even Mikhail's thenify relys on the structure of the test when you add a small amount of asynchronicity
Example app
<foo>abc</foo>
<script>
setTimeout(() => {
const foo = document.querySelector('foo')
foo.innerText = 'def'
}, 1000)
</script>
Test
let a = cy.get("foo").thenify()
// expect(a.text()).to.eq('def') // fails
// cy.wrap(a.text()).should('eq', 'def') // fails
cy.wrap(a).should('have.text', 'def') // passes
let b = cy.get("foo") // no thenify
b.should('have.text', 'def') // passes
Based on your working example, you will need to get the headers first, map out the text, then find the index of the column (I've chosen 'Col B'). Afterwards you will find the row containing the other cell value, then get all the cells in row and use .eq() with the column index found earlier.
// get headers, map text, filter to Col B index
cy.get("th")
.then(($headers) => Cypress._.map($headers, "innerText"))
.then(cy.log)
.invoke("indexOf", "Col B")
.then((headerIndex) => {
// find row containing Val A
cy.contains("tbody tr", "Val A")
.find("td")
// get cell containing Val B
.eq(headerIndex)
.should("have.text", "Val B");
});
Here is the example.

How to store data for use in multiple data validation using GAS?

I have a script running on Google Sheets, which brings data from another spreadsheet/file as an array and sets one of its column's data as a data validation into a cell. Then, as the user picks one option of this data validation, the script goes back to that file and brings its related data and sets it in an adjacent column and this repeats about 3 times, making the process slow.
I was wondering if that would be possible to store the first data collection into the document property and set the data validations by grabbing related information from that data set, instead of going to the other file everytime.
Here's an update, with a working version:
function listaCategorias() {
let listaGeral = sheetBDCadProd.getRange(2, 1, sheetBDCadProd.getLastRow(), 45).getValues();//Gets all values
//Extracts a column of interest for this first data validation setting
let categorias = [];
for (let a = 0; a < listaGeral.length; a++) {
categorias.push(listaGeral[a][17])
}
let uniqueCat = [...new Set(categorias)]; //Gets a list of unique values. Not sure how I'd do that within new Set, so I did a for loop before
//Sets the data validation
const cell = sheetVendSobEnc.getRange('B5');
const validationCat = SpreadsheetApp.newDataValidation().requireValueInList(uniqueCat).setAllowInvalid(false).build();
cell.clearContent();
cell.clearDataValidations();
cell.setDataValidation(validationCat);
//Saves the data into the document property for usage in the next script/data validation
listaGeral = JSON.stringify(listaGeral)
PropertiesService.getDocumentProperties().setProperty('listaGeral', listaGeral);
}
//This is getting one of the columns, based on the option picked..the one generated by the data validation above.
function listaDescricao() {
const categoria = sheetVendSobEnc.getRange('B5').getValue();
const dadosCadProd = PropertiesService.getDocumentProperties().getProperty('listaGeral')
let cadGeral = JSON.parse(dadosCadProd);
//Filters the elements matching the option picked
let filteredNomeSobEnc = cadGeral.filter(function (o) { return o[17] === categoria });
//Filters unique values
let listToApply = filteredNomeSobEnc.map(function (o) { return o[7] }).sort().reverse();
let descUnica = listToApply.filter((v, i, a) => a.indexOf(v) === i);
Logger.log('Descrição Única: ' + descUnica)
}
It's working, but I'd like to know the rooms for improvement here.
Thanks.

Google AppMaker: Fetch a MAX value

I am not able to fetch a max value from a number field in AppMaker. The field is filled with unique integers from 1 and up. In SQL I would have asked like this:
SET #tKey = (SELECT MAX(ID) FROM GiftCard);
In AppMaker I have done the following (with a bit help from other contributors in this forum) until now, and it returns tKey = "NaN":
var tKey = google.script.run.MaxID();
function MaxID() {
var ID_START_FROM = 11000;
var lock = LockService.getScriptLock();
lock.waitLock(3000);
var query = app.models.GiftCard.newQuery();
query.sorting.ID._descending();
query.limit = 1;
var records = query.run();
var next_id = records.length > 0 ? records[0].ID : ID_START_FROM;
lock.releaseLock();
return next_id;
}
There is also a maxValue() function in AppMaker. However, it seems not to work in that way I use it. If maxvalue() is better to use, please show :-)
It seems that you are looking in direction of auto incremented fields. The right way to achieve it would be using Cloud SQL database. MySQL will give you more flexibility with configuring your ids:
ALTER TABLE GiftCard AUTO_INCREMENT = 11000;
In case you strongly want to stick to Drive Tables you can try to fix your script as follow:
google.script.run
.withSuccessHandler(function(maxId) {
var tKey = maxId;
})
.withFailureHandler(function(error) {
// TODO: handle error
})
.MaxID();
As a side note I would also recommend to set your ID in onBeforeCreate model event as an extra security layer instead of passing it to client and reading back since it can be modified by malicious user.
You can try using Math.max(). Take into consideration the example below:
function getMax() {
var query = app.models.GiftCard.newQuery();
var allRecords = query.run();
allIds = [];
for( var i=0; i<allRecords.length;i++){
allIds.push(allRecords[i].ID);
}
var maxId = Math.max.apply(null, allIds);
return maxId;
}
Hope it helps!
Thank you for examples! The Math.max returned an undefined value. Since this simple case is a "big" issue, I will solve this in another way. This value is meant as a starting value for a sequence only. An SQL base is better yes!

crossfilter: obtain the count of values falling into the product of two columns

I have a data set like
{"parent":"/home","inside":"/files","filename":"type.jar",
"extension":"jar","type":"modified","archive"}
Likewise many there are many rows in the json array. I am using crossfilter to read the data and plot graphs and datatables. the Type in the data set has values "added", "modified" and "deleted".
I want to create a data table like
Extension | Added | Modified | Deleted
where added, modified and deleted will hold the count of the files with the specific extension. Can anyone suggest me a way to do so?
So far I have created a dimension like this:
var extensionType = facts.dimension(function(d) {
return d.extension; });
var extensionTypeGroup=extensionType.group();
and I get a grouped output like this,
{"key":"class","value":424},
{"key":"js","value":176},
{"key":"properties","value":26},
{"key":"jar","value":10},
{"key":"css","value":8},
{"key":"txt","value":6},
{"key":"war","value":4},
{"key":"png","value":4},
{"key":"handlebars","value":4},
{"key":"jar_local","value":2},
{"key":"aar","value":2}
How do I get the separate count of added deleted and modified?
Probably the easiest way to do this is to reduce to an object rather than a single value.
This is covered in the FAQ: How do I reduce multiple values at once? What if rows contain a single value but a different value per row? You probably just needed the right search terms to find it.
Actually it looks like the code from the FAQ will work for you unmodified:
var extensionTypeGroup = extensionType.group().reduce(
function(p, v) { // add
p[v.type] = (p[v.type] || 0) + v.value;
return p;
},
function(p, v) { // remove
p[v.type] -= v.value;
return p;
},
function() { // initial
return {};
});

Angular.js - Data from AJAX request as a ng-repeat collection

In my web app i'm reciving data every 3-4 seconds from an AJAX call to API like this:
$http.get('api/invoice/collecting').success(function(data) {
$scope.invoices = data
}
Then displaying the data, like this: http://jsfiddle.net/geUe2/1/
The problem is that every time i do $scope.invoices = data ng-repeat rebuilds the DOM area which is presented in the jsfiddle, and i lose all <input> values.
I've tried to do:
angular.extend()
deep version of jQuery.extend
some other merging\extending\deep copying functions
but they can't handle the situation like this:
On my client a have [invoice1, invoice2, invoice3] and server sends me [invoice1, invoice3]. So i need invoice2 to be deleted from the view.
What are the ways to solve this problem?
Check the ng-repeat docs Angular.js - Data from AJAX request as a ng-repeat collection
You could use track by option:
variable in expression track by tracking_expression – You can also provide an optional tracking function which can be used to associate the objects in the collection with the DOM elements. If no tracking function is specified the ng-repeat associates elements by identity in the collection. It is an error to have more than one tracking function to resolve to the same key. (This would mean that two distinct objects are mapped to the same DOM element, which is not possible.) Filters should be applied to the expression, before specifying a tracking expression.
For example: item in items track by item.id is a typical pattern when the items come from the database. In this case the object identity does not matter. Two objects are considered equivalent as long as their id property is same.
You need to collect data from DOM when an update from the server arrives. Save whatever data is relevant (it could be only the input values) and don't forget to include the identifier for the data object, such as data._id. All of this should be saved in a temporary object such as $scope.oldInvoices.
Then after collecting it from DOM, re-update the DOM with the new data (the way you are doing right now) $scope.invoices = data.
Now, use underscore.js _.findWhere to locate if your data._id is present in the new data update, and if so - re-assign (you can use Angular.extend here) the data-value that you saved to the relevant invoice.
Came out, that #luacassus 's answer about track by option of ng-repeat directive was very helpful but didn't solve my problem. track by function was adding new invoices coming from server, but some problem with clearing inactive invoices occured.
So, this my solution of the problem:
function change(scope, newData) {
if (!scope.invoices) {
scope.invoices = [];
jQuery.extend(true, scope.invoices, newData)
}
// Search and update from server invoices that are presented in scope.invoices
for( var i = 0; i < scope.invoices.length; i++){
var isInvoiceFound = false;
for( var j = 0; j < newData.length; j++) {
if( scope.invoices[i] && scope.invoices[i].id && scope.invoices[i].id == newData[j].id ) {
isInvoiceFound = true;
jQuery.extend(true, scope.invoices[i], newData[j])
}
}
if( !isInvoiceFound ) scope.invoices.splice(i, 1);
}
// Search and add invoices that came form server, but are nor presented in scope.invoices
for( var j = 0; j < newData.length; j++){
var isInvoiceFound = false;
for( var i = 0; i < scope.invoices.length; i++) {
if( scope.invoices[i] && scope.invoices[i].id && scope.invoices[i].id == newData[j].id ) {
isInvoiceFound = true;
}
}
if( !isInvoiceFound ) scope.invoices.push(newData[j]);
}
}
In my web app i'm using jQuery's .extend() . There's some good alternative in lo-dash library.

Resources