Prevent custom functions from executing in Google Spreadsheets Google Apps Script - caching

When writing custom functions to be used in spreadsheet cells, the default behavior for a sheet is to recalculate on edits, i.e. adding column or rows will cause a custom function to update.
This is a problem if the custom function calls a paid API and uses credits, the user will consuming API credits automatically.
I couldn't figure out a way to prevent this, so I decided to use the UserCache to cache the results for an arbitrary 25 minutes, and serve it back to the user should they happen to repeat the same function call. It's definitely not bulletproof but it's better than nothing I suppose. Apparently the cache can hold 10mb, but is this the right approach? Could I be doing something smarter?
var _ROOT = {
cache : CacheService.getUserCache(),
cacheDefaultTime: 1500,
// Step 1 -- Construct a unique name for function call storage using the
// function name and arguments passed to the function
// example: function getPaidApi(1,2,3) becomes "getPaidApi123"
stringifyFunctionArguments : function(functionName,argumentsPassed) {
var argstring = ''
for (var i = 0; i < argumentsPassed.length; i++) {
argstring += argumentsPassed[i]
}
return functionName+argstring
},
//Step 2 -- when a user calls a function that uses a paid api, we want to
//cache the results for 25 minutes
addToCache : function (encoded, returnedValues) {
var values = {
returnValues : returnedValues
}
Logger.log(encoded)
this.cache.put(encoded, JSON.stringify(values), this.cacheDefaultTime)
}
//Step 3 -- if the user repeats the exact same function call with the same
//arguments, we give them the cached result
//this way, we don't consume API credits as easily.
checkCache : function(encoded) {
var cached = this.cache.get(encoded);
try {
cached = JSON.parse(cached)
return cached.returnValues
} catch (e) {
return false;
}
}
}

Google Sheets already caches the values of custom functions, and will only run them again when either a) the inputs to the function have changed or b) the spreadsheet is being opened after being closed for a long time. I'm not able to replicate the recalculation you mentioned when adding and removing columns. Here's a simple example function I used to test that:
function rng() {
return Math.random();
}
Your approach of using an additional cache for expensive queries looks fine in general. I'd recommend using the DocumentCache instead of the UserCache, since all users of the document can and should see the same cell values.
I'd also recommend a more robust encoding of function signatures, since your current implementation is able to distinguish between the arguments [1, 2] and [12]. You could stringify the inputs and then base64 encode it for compactness:
function encode(functionName, argumentsPassed) {
var data = [functionName].concat(argumentsPassed);
var json = JSON.stringify(data);
return Utilities.base64Encode(json);
}

Related

how the nativescript radlist view load on demand works

This might not be the question but it was the list of doubts which comes when learning native script from scratch.
I had a 1000 or more list of data stored in data table. know i want to display it on a list view but i don't want to read all the data at once. because i have images stored in other directory and want to read that also. So, for 20 to 30 data's the performance is quite good. but for 1000 data it is taking more than 15 minutes to read the data as well as images associated with it. since i'm storing some high quality images.
Therefore i decided to read only 20 data's with their respective images. and display it on list. know when user reaches the 15th data of the list. i decided to read 10 more data from the server.
know when i search this i came across "RadListView Load on Demand".
then i just looked at the code below.
public addMoreItemsFromSource(chunkSize: number) {
let newItems = this._sourceDataItems.splice(0, chunkSize);
this.dataItems.push(newItems);
}
public onLoadMoreItemsRequested(args: LoadOnDemandListViewEventData) {
const that = new WeakRef(this);
const listView: RadListView = args.object;
if (this._sourceDataItems.length > 0) {
setTimeout(function () {
that.get().addMoreItemsFromSource(2);
listView.notifyLoadOnDemandFinished();
}, 1500);
args.returnValue = true;
} else {
args.returnValue = false;
listView.notifyLoadOnDemandFinished(true);
}
}
In nativescript if i want to access binding element xml element. i must use observables in viewmodel or exports.com_name on associated js file.
but in this example it is started with public..! how to use this in javascript.
what is new WeakRef(this) ?
why it is needed ?
how to identify user has scrolled to 15 data, as i want to load more data when he came at 15th data.
after getting data how to update array of list and show it in listview ?
Finally i just want to know how to use load on demand
i tried to create a playground sample of what i have tried but it is giving error. it cannot found module of radlistview.
Remember i'm a fresher So, kindly keep this in mind when answering. thank you,
please modify the question if you feel it is not upto standards.
you can check the updated answer here
https://play.nativescript.org/?template=play-js&id=1Xireo
TypeScript to JavaScript
You may use any TypeScript compiler to convert the source code to JavaScript. There are even online compilers like the official TypeScript Playground for instance.
In my opinion, it's hard to expect ES5 examples any more. ES6-9 introduced a lot of new features that makes JavaScript development much more easier and TypeScript takes JavaScript to next level, interpreter to compiler.
To answer your question, you will use the prototype chain to define methods on your class in ES5.
YourClass.prototype.addMoreItemsFromSource = function (chunkSize) {
var newItems = this._sourceDataItems.splice(0, chunkSize);
this.dataItems.push(newItems);
};
YourClass.prototype.onLoadMoreItemsRequested = (args) {
var that = new WeakRef(this);
var listView = args.object;
if (this._sourceDataItems.length > 0) {
setTimeout(function () {
that.get().addMoreItemsFromSource(2);
listView.notifyLoadOnDemandFinished();
}, 1500);
args.returnValue = true;
} else {
args.returnValue = false;
listView.notifyLoadOnDemandFinished(true);
}
}
If you are using fromObject syntax for your Observable, then these functions can be passed inside
addMoreItemsFromSource: function (chunkSize) {
....
};
WeakRef: It helps managing your memory effiencetly by keeping a loose reference to the target, read more on docs.
How to load more:
If you set loadOnDemandMode to Auto then loadMoreDataRequested event will be triggered whenever user reaches the end of scrolling.
loadOnDemandBufferSize decides how many items before the end of scroll the event should be triggered.
Read more on docs.
How to update the array:
That's exactly what showcased in addMoreItemsFromSource function. Use .push(item) on the ObservableArray that is linked to your list view.

Time-based cache for REST client using RxJs 5 in Angular2

I'm new to ReactiveX/RxJs and I'm wondering if my use-case is feasible smoothly with RxJs, preferably with a combination of built-in operators. Here's what I want to achieve:
I have an Angular2 application that communicates with a REST API. Different parts of the application need to access the same information at different times. To avoid hammering the servers by firing the same request over and over, I'd like to add client-side caching. The caching should happen in a service layer, where the network calls are actually made. This service layer then just hands out Observables. The caching must be transparent to the rest of the application: it should only be aware of Observables, not the caching.
So initially, a particular piece of information from the REST API should be retrieved only once per, let's say, 60 seconds, even if there's a dozen components requesting this information from the service within those 60 seconds. Each subscriber must be given the (single) last value from the Observable upon subscription.
Currently, I managed to achieve exactly that with an approach like this:
public getInformation(): Observable<Information> {
if (!this.information) {
this.information = this.restService.get('/information/')
.cache(1, 60000);
}
return this.information;
}
In this example, restService.get(...) performs the actual network call and returns an Observable, much like Angular's http Service.
The problem with this approach is refreshing the cache: While it makes sure the network call is executed exactly once, and that the cached value will no longer be pushed to new subscribers after 60 seconds, it doesn't re-execute the initial request after the cache expires. So subscriptions that occur after the 60sec cache will not be given any value from the Observable.
Would it be possible to re-execute the initial request if a new subscription happens after the cache timed out, and to re-cache the new value for 60sec again?
As a bonus: it would be even cooler if existing subscriptions (e.g. those who initiated the first network call) would get the refreshed value whose fetching had been initiated by the newer subscription, so that once the information is refreshed, it is immediately passed through the whole Observable-aware application.
I figured out a solution to achieve exactly what I was looking for. It might go against ReactiveX nomenclature and best practices, but technically, it does exactly what I want it to. That being said, if someone still finds a way to achieve the same with just built-in operators, I'll be happy to accept a better answer.
So basically since I need a way to re-trigger the network call upon subscription (no polling, no timer), I looked at how the ReplaySubject is implemented and even used it as my base class. I then created a callback-based class RefreshingReplaySubject (naming improvements welcome!). Here it is:
export class RefreshingReplaySubject<T> extends ReplaySubject<T> {
private providerCallback: () => Observable<T>;
private lastProviderTrigger: number;
private windowTime;
constructor(providerCallback: () => Observable<T>, windowTime?: number) {
// Cache exactly 1 item forever in the ReplaySubject
super(1);
this.windowTime = windowTime || 60000;
this.lastProviderTrigger = 0;
this.providerCallback = providerCallback;
}
protected _subscribe(subscriber: Subscriber<T>): Subscription {
// Hook into the subscribe method to trigger refreshing
this._triggerProviderIfRequired();
return super._subscribe(subscriber);
}
protected _triggerProviderIfRequired() {
let now = this._getNow();
if ((now - this.lastProviderTrigger) > this.windowTime) {
// Data considered stale, provider triggering required...
this.lastProviderTrigger = now;
this.providerCallback().first().subscribe((t: T) => this.next(t));
}
}
}
And here is the resulting usage:
public getInformation(): Observable<Information> {
if (!this.information) {
this.information = new RefreshingReplaySubject(
() => this.restService.get('/information/'),
60000
);
}
return this.information;
}
To implement this, you will need to create your own observable with custom logic on subscribtion:
function createTimedCache(doRequest, expireTime) {
let lastCallTime = 0;
let lastResult = null;
const result$ = new Rx.Subject();
return Rx.Observable.create(observer => {
const time = Date.now();
if (time - lastCallTime < expireTime) {
return (lastResult
// when result already received
? result$.startWith(lastResult)
// still waiting for result
: result$
).subscribe(observer);
}
const disposable = result$.subscribe(observer);
lastCallTime = time;
lastResult = null;
doRequest()
.do(result => {
lastResult = result;
})
.subscribe(v => result$.next(v), e => result$.error(e));
return disposable;
});
}
and resulting usage would be following:
this.information = createTimedCache(
() => this.restService.get('/information/'),
60000
);
usage example: https://jsbin.com/hutikesoqa/edit?js,console

Fabric.js - Sync object:modified event to another client

Collaboration Mode:
What is the best way to propagate changes from Client #1's canvas to client #2's canvas? Here's how I capture and send events to Socket.io.
$scope.canvas.on('object:modified',function(e) {
Socket.whiteboardMessage({
eventId:'object:modified',
event:e.target.toJSON()
});
});
On the receiver side, this code works splendidly for adding new objects to the screen, but I could not find documentation on how to select and update an existing object in the canvas.
fabric.util.enlivenObjects([e.event], function(objects) {
objects.forEach(function(o) {
$scope.canvas.add(o);
});
});
I did see that Objects have individual setters and one bulk setter, but I could not figure out how to select an existing object based on the event data.
Ideally, the flow would be:
Receive event with targeted object data.
Select the existing object in the canvas.
Perform bulk update.
Refresh canvas.
Hopefully someone with Fabric.JS experience can help me figure this out. Thanks!
UPDATED ANSWER - Thanks AJM!
AJM was correct in suggesting a unique ID for every newly created element. I was also able to create a new ID for all newly created drawing paths as well. Here's how it worked:
var t = new fabric.IText('Edit me...', {
left: $scope.width/2-100,
top: $scope.height/2-50
});
t.set('id',randomHash());
$scope.canvas.add(t);
I also captured newly created paths and added an id:
$scope.canvas.on('path:created',function(e) {
if (e.target.id === undefined) {
e.target.set('id',randomHash());
}
});
However, I encountered an issue where my ID was visible in console log, but it was not present after executing object.toJSON(). This is because Fabric has its own serialization method which trims down the data to a standardized list of properties. To include additional properties, I had to serialize the data for transport like so:
$scope.canvas.on('object:modified',function(e) {
Socket.whiteboardMessage({
object:e.target.toJSON(['id']) // includes "id" in output.
})
});
Now each object has a unique ID with which to perform updates. On the receiver's side of my code, I added AJM's object-lookup function. I placed this code in the "startup" section of my application so it would only run once (after Fabric.js is loaded, of course!)
fabric.Canvas.prototype.getObjectById = function (id) {
var objs = this.getObjects();
for (var i = 0, len = objs.length; i < len; i++) {
if (objs[i].id == id) {
return objs[i];
}
}
return 0;
};
Now, whenever a new socket.io message is received with whiteboard data, I am able to find it in the canvas via this line:
var obj = $scope.canvas.getObjectById(e.object.id);
Inserting and removing are easy, but for updating, this final piece of code did the trick:
obj.set(e.object); // Updates properties
$scope.canvas.renderAll(); // Redraws canvas
$scope.canvas.calcOffset(); // Updates offsets
All of this required me to handle the following events. Paths are treated as objects once they're created.
$scope.canvas.on('object:added',function(e) { });
$scope.canvas.on('object:modified',function(e) { });
$scope.canvas.on('object:moving',function(e) { });
$scope.canvas.on('object:removed',function(e) { });
$scope.canvas.on('path:created',function(e) { });
I did something similar involving a single shared canvas between multiple users and ran into this exact issue.
To solve this problem, I added unique IDs (using a javascript UUID generator) to each object added to the canvas (in my case, there could be many users working on a canvas at a time, thus I needed to avoid collisions; in your case, something simpler could work).
Fabric objects' set method will let you add an arbitrary property, like an id: o.set('id', yourid). Before you add() a new Fabric object to your canvas (and send that across the wire), tack on an ID property. Now, you'll have a unique key by which you can pick out individual objects.
From there, you'd need a method to retrieve an object by ID. Here's what I used:
fabric.Canvas.prototype.getObjectById = function (id) {
var objs = this.getObjects();
for (var i = 0, len = objs.length; i < len; i++) {
if (objs[i].id == id) {
return objs[i];
}
}
return null;
};
When you receive data from your socket, grab that object from the canvas by ID and mutate it using the appropriate set methods or copying properties wholesale (or, if getObjectById returns null, create it).

Sequelize correctly executing multiple creates + updates

I have a cron job that scrapes a list of items on a website and then inserts or updates records in a database. When I scrape the page, I want to create records for new ones that haven't been created yet, otherwise update any existing ones. Currently I'm doing something like this:
// pretend there is a "Widget" model defined
function createOrUpdateWidget(widgetConfig) {
return Widget.find(widgetConfig.id)
.then(function(widget) {
if (widget === null) {
return Widget.create(widgetConfig);
}
else {
widget.updateAttributes(widgetConfig);
}
});
}
function createOrUpdateWidgets(widgetConfigObjects) {
var promises = [];
widgetConfigObjects.forEach(function(widgetConfig) {
promises.push(createOrUpdateWidget(widgetConfig));
});
return Sequelize.Promise.all(promises);
}
createOrUpdateWidgets([...])
.done(function() {
console.log('Done!');
});
This seems to work fine, but I'm not sure if I'm doing this "correctly" or not. Do all promises that perform DB interactions need to run serially, or is how I have them defined ok? Is there a better way to do this kind of thing?
What you're doing is pretty idiomatic and perfectly fine, the only room for improvement is to utilize the fact Sequelize uses Bluebird for promises so you get .map for free, which lets you convert:
function createOrUpdateWidgets(widgetConfigObjects) {
var promises = [];
widgetConfigObjects.forEach(function(widgetConfig) {
promises.push(createOrUpdateWidget(widgetConfig));
});
return Sequelize.Promise.all(promises);
}
Into:
function createOrUpdateWidgets(widgetConfigObjects) {
return Sequelize.Promise.map(widgetConfig, createOrUpdateWidget)
}
Other than that minor improvement - you're chaining promises correctly and seem to have the correct hang of it.

How to efficiently remove first element from a selection?

I have a page that displays some data using d3.js. Due to the heavy processing load, when the page load it freezes the browser for a few seconds.
I have determined that this "browser locking" behavior is due mostly to a line of the form:
selection.attr('d', linefn);
...where selection contains around 10K items.
I would like to replace this line with something like
function process_rest () {
if (selection.size() > 0) {
var next_item = first(selection); // function first() is hypothetical!
next_item.attr('d', linefn);
selection = rest(selection); // function rest() is hypothetical!
setTimeout(process_rest, 100);
return;
}
finish_up();
}
setTimeout(process_rest, 100);
I'm looking for an efficient way to implement either first and rest. My very naive guess would be something like:
function first(selection) {
return d3.select(selection[0][0]);
}
function rest(selection) {
selection[0] = selection[0].slice(1);
return selection;
}
...but, AFAIK, this is going "behind the API", or at least feels like it. Is there an "official" (i.e. documented) way to achieve the same result?
EDIT: deleted the shift variant (it's safer not to update selection until after the processing of the first element has been successfully completed).
You can simply use .each():
selection.each(function(d, i) {
setTimeout(function() { d3.select(this).attr("d", linefn); }, i * 100);
});

Resources