Coming from AngularJS I'm struggling trying to solve the next problem. I need a function that returns an object (lets call it A, but this object cannot be returned till all the requests that are contained in that function are resolved. The process should be like:
The object A is downloaded from a remote server
Using A, we do operations over another object (B)
B is downloaded from the server
B is patched using some attributes from A
Using A and the result of B we do operations over a third object, C
C is downloaded from the server
C is patched using some attributes from A and B
After B and C are processed, the function must return A
I'd like to understand how to do something like this using rxjs, but with Angular 6 most of the examples around the internet seem to be deprecated, and the tutorials out there are not really helping me. And I cannot modify the backend to make this a bit more elegant. Thanks a lot.
Consider the following Observables:
const sourceA = httpClient.get(/*...*/);
const sourceB = httpClient.get(/*...*/);
const sourceC = httpClient.get(/*...*/);
Where httpClient is Angular's HTTPClient.
The sequence of the operations you described may look as follows:
const A = sourceA.pipe(
switchMap(a => sourceB.pipe(
map(b => {
// do some operation using a and b.
// Return both a and b in an array, but you can
// also return them in an object if you wish.
return [a,b];
})
)),
switchMap(ab => sourceC.pipe(
map(c => {
// do some operations using a, b, and/or c.
return a;
})
))
);
Now you just need to subscribe to A:
A.subscribe(a => console.log(a));
You can read about RxJs operators here.
Well, first of all, it appears to me that this function-call, as described, would be somehow expected to block the calling process until all of the specified events have occurred – which of course is unreasonable in JavaScript.
Therefore, first of all, I believe that your function should require, as its perhaps-only parameter, a callback that will be invoked when everything has finally taken place.
Now – as to "how to handle steps 1, 2, and 3 elegantly" ... what immediately comes to mind is the notion of a finite-state machine (FSM) algorithm.
Let's say that your function-call causes a new "request" to be placed on some request-table queue, and, if necessary, a timer-request (set to go off in 1 millisecond) to service that queue. (This entry will contain, among other things, a reference to your callback.) Let's assume also that the request is given a random-string "nonce" that will serve to uniquely identify it: this will be passed to the various external requests and must be included in their corresponding replies.
The FSM idea is that the request will have a state, (attribute), such as: DOWNLOADING_FROM_B, B_DOWNLOADS_COMPLETE, DOWNLOADING_FROM_C, C_REQUESTS_COMPLETE, and so on. Such that each and every callback that will play a part in this fully-asynchronous process will (1) be able to locate a request-entry by its nonce, and then (2) unambiguously "know what to do next," and "what new-state (if any) to assign," based solely upon examination of the entry's state.
For instance, when the state reaches C_REQUESTS_COMPLETE, it would be time to invoke the callback that you originally provided, and to delete the request-table entry.
You can easily map-out all of the "state transitions" that might occur in an arbitrarily-complex scenario (what states can lead to what states, and what to do when they do), whether or not you actually create a data-structure to represent that so-called "state table," although sometimes it is even-more elegant(!) when you do. (Possibly-messy decision logic is simply pushed to a simple table-lookup.)
This is, of course, a classic algorithm that is applicable to – and, has been used in – "every programming language under the sun." (Lots of hardware devices use it, too.)
Related
I have two observables
baseObs$ = serviceCall(); // this return Observable<Foo>
secondObs$ = serviceCall(args); // this return Observable<Bar>
args in this example is public variable defined somewhere else. It doesn't need to be though if that makes this easier.
baseObs I can call whenever I want, but secondObs I can only call after baseObs is succesfully called and handled (don't know the right words so example follows):
I have now something like
baseObs$.subscribe(x => {
const args = x.args; // just example. Point is, I need x to build args.
serviceCall(args).subscribe(y => {
console.log(y); // This is fine
});
});
This suits my needs but I got feedback that no subscribe should live inside another subscribe. How would you achieve same thing using baseObs$ and secondObs$ defined above?
PS. All is pseudo code but hopefully I didn't do too much typos. I think the idea should be clear though.
In the simple case that the first observable only emits once (like a typical HTTP request), any one of switchMap, mergeMap etc. will do:
serviceCall().pipe(
switchMap(x => serviceCall(x.args))
).subscribe(console.log);
If that assumption is not true, you're going to want to read their respective documentation to understand how behavior will change between them. In fact, I'd recommend reading up on them to know their differences even just in general, as it's very valuable information when dealing with reactive code.
The Q&A is currently a subject of meta discussion, do participate. The current plan is to split where possible into Q&As. Answers to the A&A are community wiki and the question should become one when the status is resolved.
Preface
This Q&A strives to become a collection and a reference target for common errors encountered during development in Google Apps Script language in hopes to improve long-term maintainability of google-apps-script tag.
There are several similar and successful undergoings in other languages and general-purpose tags (see c++, android, php, php again), and this one follows suit.
Why it exists?
The amount of questions from both new and experienced developers regarding the meaning and solutions to errors encountered during development and production that can be effectively reduced to a single answer is substantial. At the time of writing, even running a query only by language tag yields:
"Cannot find method" 8 pages
"Cannot read property" 9 pages
"Cannot call ... in this context" 5 pages
"You do not have permission" 11 pages
Linking to a most relevant duplicate is hard and time-consuming for volunteers due to the need to consider nuances and often poorly-worded titles.
What it consists of?
Entries in this Q&A contain are designed to provide info on how to:
parse the error message structure
understand what the error entails
consistently reproduce (where applicable)
resolve the issue
provide a link to canonical Q&A (where possible)
Table of Contents
To help you navigate the growing reference please use the TOC below:
General errors
Service-specific errors
What this is not?
The scope of the Q&A is limited to common (not trivial). This is not:
a catch-all guide or "best practices" collection
a reference for general ECMAScript errors
GAS documentation
a resources list (we have a tag wiki for that)
What to add?
When adding an entry, please, consider the following:
is the error common enough (see "why" section for examples)?
can the solution be described concisely and be applicable for most cases?
Preface
The answer provides a guide on general errors that can be encountered when working with any Google service (both built-in and advanced) or API. For errors specific to certain services, see the other answer.
Back to reference
General errors
Message
TypeError: Cannot read property 'property name here' from undefined (or null)
Description
The error message indicates that you are trying to access a property on an Object instance, but during runtime the value actually held by a variable is a special data type undefined. Typically, the error occurs when accessing nested properties of an object.
A variation of this error with a numeric value in place of property name indicates that an instance of Array was expected. As arrays in JavaScript are objects, everything mentioned here is true about them as well.
There is a special case of dynamically constructed objects such as event objects that are only available in specific contexts like making an HTTP request to the app or invoking a function via time or event-based trigger.
The error is a TypeError because an "object" is expected, but "undefined" is received
How to fix
Using default values
Logical OR || operator in JavaScript has an intersting property of evaluating the right-hand side iff the left-hand is falsy. Since objects in JS are truthy, and undefined and null are falsy, an expression like (myVar || {}).myProp [(myVar || [])[index] for arrays] will guarantee that no error is thrown and the property is at least undefined.
One can also provide default values: (myVar || { myProp : 2 }) guarantees accessing myProp to return 2 by default. Same goes for arrays: (myVar || [1,2,3]).
Checking for type
Especially true for the special case, typeof operator combined with an if statement and a comparison operator will either allow a function to run outside of its designated context (i.e. for debugging purposes) or introduce branching logic depending on whether the object is present or not.
One can control how strict the check should be:
lax ("not undefined"): if(typeof myVar !== "undefined") { //do something; }
strict ("proper objects only"): if(typeof myVar === "object" && myVar) { //do stuff }
Related Q&As
Parsing order of the GAS project as the source of the issue
Message
Cannot convert some value to data type
Description
The error is thrown due to passing an argument of different type than a method expects. A common mistake that causes the error is accidental coercion of a number to string.
How to reproduce
function testConversionError() {
const ss = SpreadsheetApp.getActiveSheet();
ss.getRange("42.0",1);
}
How to fix
Make sure that the value referenced in the error message is of data type required by documentation and convert as needed.
Message
Cannot call Service and method name from this context
Description
This error happens on a context mismatch and is specific to container-bound scripts.
The primary use case that results in the error is trying to call a method only available in one document type (usually, getUi() as it is shared by several services) from another (i.e. DocumentApp.getUi() from a spreadsheet).
A secondary, but also prominent case is a result of calling a service not explicitly allowed to be called from a custom function (usually a function marked by special JSDoc-style comment #customfunction and used as a formula).
How to reproduce
For bound script context mismatch, declare and run this function in a script project tied to Google Sheets (or anything other than Google Docs):
function testContextMismatch() {
const doc = DocumentApp.getUi();
}
Note that calling a DocumentApp.getActiveDocument() will simply result in null on mismatch, and the execution will succeed.
For custom functions, use the function declared below in any cell as a formula:
/**
* #customfunction
*/
function testConversionError() {
const ui = SpreadsheetApp.getUi();
ui.alert(`UI is out of scope of custom function`);
}
How to fix
Context mismatch is easily fixed by changing the service on which the method is called.
Custom functions cannot be made to call these services, use custom menus or dialogs.
Message
Cannot find method Method name here
The parameters param names do not match the method signature for method name
Description
This error has a notoriously confusing message for newcomers. What it says is that a type mismatch occurred in one or more of the arguments passed when the method in question was called.
There is no method with the signature that corresponds to how you called it, hence "not found"
How to fix
The only fix here is to read the documentation carefully and check if order and inferred type of parameters are correct (using a good IDE with autocomplete will help). Sometimes, though, the issue happens because one expects the value to be of a certain type while at runtime it is of another. There are several tips for preventing such issues:
Setting up type guards (typeof myVar === "string" and similar).
Adding a validator to fix the type dynamically thanks to JavaScript being dynamically typed.
Sample
/**
* #summary pure arg validator boilerplate
* #param {function (any) : any}
* #param {...any} args
* #returns {any[]}
*/
const validate = (guard, ...args) => args.map(guard);
const functionWithValidator = (...args) => {
const guard = (arg) => typeof arg !== "number" ? parseInt(arg) : arg;
const [a,b,c] = validate(guard, ...args);
const asObject = { a, b, c };
console.log(asObject);
return asObject;
};
//driver IIFE
(() => {
functionWithValidator("1 apple",2,"0x5");
})()
Messages
You do not have permission to perform that action
The script does not have permission to perform that action
Description
The error indicates that one of the APIs or services accessed lacks sufficient permissions from the user. Every service method that has an authorization section in its documentation requires at least one of the scopes to be authorized.
As GAS essentially wraps around Google APIs for development convenience, most of the scopes listed in OAuth 2.0 scopes for APIs reference can be used, although if one is listed in the corresponding docs it may be better to use it as there are some inconsistencies.
Note that custom functions run without authorization. Calling a function from a Google sheet cell is the most common cause of this error.
How to fix
If a function calling the service is ran from the script editor, you are automatically prompted to authorize it with relevant scopes. Albeit useful for quick manual tests, it is best practice to set scopes explicitly in application manifest (appscript.json). Besides, automatic scopes are usually too broad to pass the review if one intends to publish the app.
The field oauthScopes in manifest file (View -> Show manifest file if in code editor) should look something like this:
"oauthScopes": [
"https://www.googleapis.com/auth/script.container.ui",
"https://www.googleapis.com/auth/userinfo.email",
//etc
]
For custom functions, you can fix it by switching to calling the function from a menu or a button as custom functions cannot be authorized.
For those developing editor Add-ons, this error means an unhandled authorization lifecycle mode: one has to abort before calls to services that require authorization in case auth mode is AuthMode.NONE.
Related causes and solutions
#OnlyCurrentDoc limiting script access scope
Scopes autodetection
Message
ReferenceError: service name is not defined
Description
The most common cause is using an advanced service without enabling it. When such a service is enabled, a variable under the specified identifier is attached to global scope that the developer can reference directly. Thus, when a disabled service is referenced, a ReferenceError is thrown.
How to fix
Go to "Resources -> Advanced Google Services" menu and enable the service referenced. Note that the identifier should equal the global variable referenced.
For a more detailed explanation, read the official guide.
If one hasn't referenced any advanced services then the error points to an undeclared variable being referenced.
Message
The script completed but did not return anything.
Script function not found: doGet or doPost
Description
This is not an error per se (as the HTTP response code returned is 200 and the execution is marked as successful, but is commonly regarded as one. The message appears when trying to make a request/access from browser a script deployed as a Web App.
There are two primary reasons why this would happen:
There is no doGet or doPost trigger function
Triggers above do not return an HtmlOutput or TextOutput instance
How to fix
For the first reason, simply provide a doGet or doPost trigger (or both) function. For the second, make sure that all routes of your app end with creation of TextOutput or HtmlOutput:
//doGet returning HTML
function doGet(e) {
return HtmlService.createHtmlOutput("<p>Some text</p>");
}
//doPost returning text
function doPost(e) {
const { parameters } = e;
const echoed = JSON.stringify(parameters);
return ContentService.createTextOutput(echoed);
}
Note that there should be only one trigger function declared - treat them as entry points to your application.
If the trigger relies on parameter / parameters to route responses, make sure that the request URL is structured as "baseURL/exec?query" or "baseURL/dev?query" where query contains parameters to pass.
Related Q&As
Redeploying after declaring triggers
Message
We're sorry, a server error occurred. Please wait a bit and try again.
Description
This one is the most cryptic error and can occur at any point with nearly any service (although DriveApp usage is particularly susceptible to it). The error usually indicates a problem on Google's side that either goes away in a couple of hours/days or gets fixed in the process.
How to fix
There is no silver bullet for that one and usually, there is nothing you can do apart from filing an issue on the issue tracker or contacting support if you have a GSuite account. Before doing that one can try the following common remedies:
For bound scripts - creating a new document and copying over the existing project and data.
Switch to using an advanced Drive service (always remember to enable it first).
There might be a problem with a regular expression if the error points to a line with one.
Don't bash your head against this error - try locating affected code, file or star an issue and move on
Syntax error without apparent issues
This error is likely to be caused by using an ES6 syntax (for example, arrow functions) while using the deprecated Rhino runtime (at the time of writing the GAS platform uses V8).
How to fix
Open "appscript.json" manifest file and check if runtimeVersion is set to "V8", change it if not, or remove any ES6 features otherwise.
Quota-related errors
There are several errors related to quotas imposed on service usage. Google has a comprehensive list of those, but as a general rule of thumb, if a message matches "too many" pattern, you are likely to have exceeded the respective quota.
Most likely errors encountered:
Service invoked too many times: service name
There are too many scripts running
Service using too much computer time for one day
This script has too many triggers
How to fix
In most cases, the only fix is to wait until the quota is refreshed or switch to another account (unless the script is deployed as a Web App with permission to "run as me", in which case owner's quotas will be shared across all users).
To quote documentation at the time:
Daily quotas are refreshed at the end of a 24-hour window; the exact time of this refresh, however, varies between users.
Note that some services such as MailApp have methods like getRemainingDailyQuota that can check the remaining quota.
In the case of exceeding the maximum number of triggers one can check how many are installed via getProjectTriggers() (or check "My triggers" tab) and act accordingly to reduce the number (for example, by using deleteTrigger(trigger) to get rid of some).
Related canonical Q&As
How are daily limitations being applied and refreshed?
"Maximum execution time exceeded" problem
Optimizing service calls to reduce execution time
References
How to make error messages more meaningful
Debugging custom functions
Service-specific errors
The answer concerns built-in service-related errors. For general reference see the other answer. Entries addressing issues with services listed in official reference are welcome.
Back to reference
SpreadsheetApp
The number of rows in the range must be at least 1
This error is usually caused by calling the getRange method where the parameter that sets the number of rows happens to equal to 0. Be careful if you depend on getLastRow() call return value - only use it on non-empty sheets (getDataRange will be safer).
How to reproduce
sh.getRange(1, 1, 0, sh.getLastColumn()); //third param is the number of rows
How to fix
Adding a guard that prevents the value from ever becoming 0 should suffice. The pattern below defaults to the last row with data (optional if you only need a certain number of rows) and to 1 if that also fails:
//willFail is defined elsewhere
sh.getRange(1, 1, willFail || sh.getLastRow() || 1, sh.getLastColumn());
Error: “Reference does not exist”
The error happens when calling a custom function in a spreadsheet cell that does not return a value. The docs do mention only that one "must return a value to display", but the catch here is that an empty array is also not a valid return value (no elements to display).
How to reproduce
Call the custom function below in any Google Sheets spreadsheet cell:
/**
* #customfunction
*/
const testReferenceError = () => [];
How to fix
No specific handling is required, just make sure that length > 0.
The number of rows or cells in the data does not match the number of rows or cells in the range. The data has N but the range has M.
Description
The error points to a mismatch in dimensions of range in relation to values. Usually, the issue arises when using setValues() method when the matrix of values is smaller or bigger than the range.
How to reproduce
function testOutOfRange() {
const ss = SpreadsheetApp.getActiveSpreadsheet();
const sh = ss.getActiveSheet();
const rng = sh.getActiveRange();
const vals = rng.getValues();
try {
vals.push([]);
rng.setValues(vals);
} catch (error) {
const ui = SpreadsheetApp.getUi();
ui.alert(error.message);
}
}
How to fix
If it is routinely expected for values to get out of bounds, implement a guard that catches such states, for example:
const checkBounds = (rng, values) => {
const targetRows = rng.getHeight();
const targetCols = rng.getWidth();
const { length } = values;
const [firstRow] = values;
return length === targetRows &&
firstRow.length === targetCols;
};
The coordinates of the range are outside the dimensions of the sheet.
Description
The error is a result of a collision between two issues:
The Range is out of bounds (getRange() does not throw on requesting a non-existent range)
Trying to call a method on a Range instance referring to a non-existent dimension of the sheet.
How to reproduce
function testOB() {
const ss = SpreadsheetApp.getActiveSpreadsheet();
const sh = ss.getActiveSheet();
const rng = sh.getRange(sh.getMaxRows() + 1, 1);
rng.insertCheckboxes();
}
How to fix
Check that number of rows (getMaxRow()) and columns (getMaxColumns()) are both greater or equal to the parameters passed to getRange() method call and change them accordingly.
Exception: You can't create a filter in a sheet that already has a filter.
Description
The message means that you are trying to call a createFilter method on a Range in a Sheet that already has a filter set (either via UI or script), thus violating the restriction on 1 filter per Sheet, to quote the documentation:
There can be at most one filter in a sheet.
How to reproduce
const testFilterExistsError = () => {
const sh = SpreadsheetApp.getActiveSheet();
const rng = sh.getDataRange();
const filter1 = rng.createFilter();
const filter2 = rng.createFilter();
};
How to fix
Add a guard that checks for the existence of the filter first. getFilter returns either a filter or null if called on a Range instance and is perfect for the job:
const testFilterGuard = () => {
const sh = SpreadsheetApp.getActiveSheet();
const rng = sh.getDataRange();
const filter = rng.getFilter() || rng.createFilter();
//do something useful;
};
UrlFetchApp
Attribute provided with no value: url
Description
The error is specific to UrlFetchApp service and happens when fetch or fetchAll method gets called with an empty string or non-string value.
How to reproduce
const response = UrlFetchApp.fetch("", {});
How to fix
Make sure that a string containing a URI (not necessarily valid) is passed to the method as its first argument. As its common root cause is accessing a non-existent property on an object or array, check whether your accessors return an actual value.
What I am really interested in with data-loader is the per request caching. For example say my graphql query needs to call getUser("id1") 3x. I would like something to dedupe that call.
However it seems like with data-loader I need to pass in an array of keys into my batch function, and multiple requests will be batched into one api calls.
This has me making a few assumptions that i dont like:
1.) That each service I am calling has a batch api (some of the ones im dealing with do not).
2.) What if multiple calls get batched into 1 api call, and that call fails because 1 of the items was not found. Normally I could handle this by returning null for that field, and that could be a valid case. Now however my entire call may fail, if the batch API decides to throw an error since 1 item was not found.
Is there anyway to use dataloader with single-key requests.
Both assumptions are wrong because the implementation of the batch function is ultimately left up to you. As indicated in the documentation, the only requirements when writing your batch function are as follows:
A batch loading function accepts an Array of keys, and returns a Promise which resolves to an Array of values or Error instances.
So there's no need for the underlying data source to also accept an array of IDs. And there's no need for one or more failed calls to cause the whole function to throw since you can return either null or an error instance for any particular ID in the array you return. In fact, your batch function should never throw and instead should always return an array of one or more errors.
In other words, your implementation of the batch function might look something like:
async function batchFn (ids) {
const result = await Promise.all(ids.map(async (id) => {
try {
const foo = await getFooById(id)
return foo
} catch (e) {
// either return null or the error
}
}))
}
It's worth noting that it's also possible to set the maxBatchSize to 1 to effectively disable batching. However, this doesn't change the requirements for how your batch function is implemented -- it always needs to take an array of IDs and always needs to return an array of values/errors of the same length as the array of IDs.
Daniel's solution is perfectly fine and is in fact what I've used so far, after extracting it into a helper function.
But I just found another solution that does not require that much boilerplate code.
new DataLoader<string, string>(async ([key]) => [await getEntityById(key)], {batch: false});
When we set batch: false then we should always get a key-array of size one passed as argument. We can therefore simply destructure it and return a one-sized array with the data. Notice the brackets arround the return value! If you omit those, this could go horribly wrong e.g. for string values.
I am building an SNMP Agent for a Windows application using the Microsoft WinSNMP API. Currently everything is working for single-item get and set-request, and also for get-next to allow walking the defined tree (albeit with some caveats that are not relevant to this question).
I am now looking at multi-item get and also get-bulk.
My current procedure is to iterate through the list of requested items (the varbindlist within the PDU), treating each one individually, effectively causing an internal get. The result is added to the VBL, set into the PDU, and then sent back to the SNMP Manager, taking into account invalid requests, etc.
My question is how should I handle "too much" data (data that cannot fit into a single transport layer message)? Or more accurately, is there a way to test whether data is "too big" without actually attempting to transmit? The only way I can see in the API is to try sending, check the error, and try again.
In the case of a get-request this isn't a problem - if you can't return all of the requested data, you fail: so attempt sending, and if the error report is SNMPAPI_TL_PDU_TOO_BIG, send a default "error" PDU.
However, it is allowable for a response to bulk-get to return partial results.
The only way I can see to handle this is a tedious (?) loop of removing an item and trying again. Something similar to the following (some detail removed for brevity):
// Create an empty varbindlist
vbl = SnmpCreateVbl(session, NULL, NULL);
// Add all items to the list
SnmpSetVb(vbl, &oid, &value); // for each OID/Value pair
// Create the PDU
pdu = SnmpCreatePdu(session, SNMP_PDU_RESPONSE, ..., vbl);
bool retry;
do {
retry = false;
smiINT failed = SnmpSendMsg(session, ..., pdu);
if (failed && SNMPAPI_TL_PDU_TOO_BIG == SnmpGetLastError()) {
// too much data, delete the last vb
SnmpDeleteVb(vbl, SnmpCountVbl(vbl));
SnmpSetPduData(pdu, ..., vbl);
retry = true;
};
} while(retry);
This doesn't seem like an optimal approach - so is there another way that I've missed?
As a side-note, I know about libraries such as net-snmp, but my question is specific to the Microsoft API.
The RFC does require you to do what you pasted,
https://www.rfc-editor.org/rfc/rfc3416
Read page 16.
There does not seem to be a function exposed by WinSNMP API that can do this for you, so you have to write your own logic to handle it.
A ZVAL is typically created with emalloc so it is destroyed at the end of a page request. Is there a way to take an existing ZVAL and make it persist in the SAPI (equivalent of pemalloc)? What about creating a ZVAL with pemalloc?
Ideally what I'd like to do (in PHP code) is this:
class Object
{
public $foo;
}
if(!($object = persist("object")))
{
$object = persist("object", new Object());
}
$object->foo[] = "bar";
print count($object->foo);
On each request count would return +1 (assuming the same PHP "worker" is used every time - I'm using PHP-FPM).
You're basically describing http://lxr.php.net/opengrok/xref/PHP_5_3/ext/sysvshm/sysvshm.c#242
The closest you can get without duplicating functionality which is already in shm is https://github.com/flavius/php-persist. The catch: in a prefork/multiprocess SAPI (like apache's), different requests from the same client may end up in different processes, and as such, you'll see different data (try it on Linux + Firefox with a hard refresh every time in the browser).
Note: It's a work-in-progress, currently it persists only an integer. Adding an array should be trivial. Patches welcome. It still needs the deserialization part, and to actually use persist()'s first parameter.
zend_object can't be persisent therefore you can't do this. extensions like APC serializes the object into memory.