As you might know, Google gives only 1 hour of execution time for triggered scripts (web-apps and manual calls are not in this hour).
I hadn't found a good explanation of how to boost script perfomance, so I had to find out it myself. Here's what helped me:
10 tips & tricks to get the best perfomance in Google Script
There is an official Google's best practices documentation on how to write scripts. However it's not that comprehensive and misses some important tips and explanations, which you will additionally find here.
Use as less as possible time consuming methods.
Any method which is considered as time-consuming by Google you will always find in the execution report .
Some examples of time-consuming methods:
Methods, which read/save information to the Sheet: getValue, getValues, getDisplayValue, getRangeByName, getSheet, setValue, deleteRow etc. Some consume more, some less.
Methods, using other external services, like GmailApp, Utilities etc.
Document/Script properties (but they are a somewhat faster than getting data from sheet).
If you use Libraries, don't forget to create a new version of the library, when you finish coding, and switch off developer mode in you calling scripts (sorry for non-English screenshots). In my case it speeded up the dummy script launches (when the script got a signal, that it should stop) by 3 times.
The parts of code, which are non included into any function, and can be executed conditionally must be grouped into initialization functions and called when a condition is met. This actually can be applied to any part of code. The thing is that the code, which is located outside of any function, is always executed, whether it's in your basic project or in a library.
Use Batch operations when possible. One getValues() on 100 cells is much faster then 100 getValue() on each cell. Same for deleteRows() and setValues() etc.
If you use many script/document properties, use batch methods also.
If you use many static Named Ranges, create a cache for all of them and make an object (hash array) from that. Make a refreshing procedure and use it when required. My project has 137 named ranges: this point had a huge effect.
Avoid sleep method, if possible.
Google advises to use cache for web pages with fetchUrl (if applicable).
Google advises to avoid Library usage in UI heavy scripts (e.g. when you use triggers based on Google Sheet actions).
If your triggered script isn't supposed to work 24 hours make a work schedule for it and route your script to a lightweight procedure.
Example:
if ((new Date()).getHours() < 9) {
var TriggeredEveryMinuteFunction = function() {
//...do some lightweight stuff or nothing...
}
} else {
function TriggeredEveryMinuteFunction() {
// ...do some heavy stuff...
func2();
}
function func2() { /*some more stuff*/ }
var func3 = function() { /*some more stuff*/ }
var etc() { /*some more stuff*/ }
}
In this example functions func2,func3,etc are not compiled when it's less than 9 o'clock.
If you try to call them, you'll get "not found" message.
Related
Problem
I have 6 OnEdit Functions, which work as intended individually, but when together they don't work as intended. By this I mean some simply don't trigger.
Properties of the Script
They have different names - function onEdit(e) {code}, function onEdit1(e1) {code}, function onEdit2(e2) {code}, function onEdit3(e3) {code}, function onEdit4(e4) {code}, function onEdit5(e5) {code}
They are all in the same .gs tab
Some of them have the same variables. For example OnEdit has var range = e.range; and OnEdit5 has var range = e5.range;
My Understanding
I believe that you can run multiple OnEdit functions within the same .gs tab. Is this correct? Or do I need to somehow create new .gs tabs?
I believe that my onEdit functions should be named differently, so they are called correctly. Is this correct, or should I be getting rid of the different functions and putting them into one massive function? (I imagine this would lead to slower execution and more cases of not being able to isolate incorrect code).
I believe that the variables that are created within each function are specific to that function. Is this true? Or are they impacting each other?
Why I'm asking this
Iterations of this question seem to have been asked before. But people generally give advice on integrating two functions into one big one, rather than preparing someone to integrate 10-20 different OnEdit functions. Nor do they give a clear indication of best coding practices.
I've spent hours reading through this subject and feel that people new to scripts, like me, would greatly benefit from knowing this.
Thank you in advance for any contributions!
Notes:
There can only be one function with a same name. If there are two, the latter will overwrite the former. It's like the former never existed.
A function named onEdit is triggered automatically on (You guessed it!)edit
There's no simple trigger for other names like onEdit1 or onEdit2....
Simple triggers are limited to 30 seconds of execution
So, in a single code.gs file or even in a single project, there can only be one function named onEdit and trigger successfully.
If you create multiple projects, onEdit will trigger in each project asynchronously. But there are limits to number of projects that can be created and other quotas will apply.
Alternatively, you can use installed triggers: which doesn't have limit of 30s. You can also use any name for your function.
The best way to optimize functions is to never touch the spreadsheet unless it is absolutely necessary. For example, sorting various values inside the script is better than repeatedly calling .sort on the multiple ranges multiple times. The lesser the interaction between sheets and scripts, the better. A highly optimized script will only require two calls to spreadsheet: one to get the data and the other to set the data.
After optimizing the number of calls to sheet, you can optimize the script itself: Control the logic such that only the necessary amount of operations are done for each edit. For example, if the edit is in A1(A1,B1 are checkboxes, if clicked clears A2:A10,B2:B10 respectively), then you should check if A1 is clicked and If clicked, clear the range and exit and not to check for B1 again. Script optimization requires atleast a basic knowledge of JavaScript objects. Nevertheless, this isn't as effective as reducing the number of calls-which is the slowest part of any apps script.
References:
Best practices
I have a function that does a find/replace on text files, and it has worked well for some time. Until I needed to process a 12 million line file.
My initial code used Get-Content and Write-Content, and with the massive file it was going to take hours to process, not to mention the memory implications of loading 12 million lines into RAM.
So, I wrote a little test script to compare that approach vs Stream Reader/Writer. And Streaming looked like it was going to be a massive performance improvement, dropping processing to 30 seconds. I then added a .Replace() on each line, and total processing time only went up to maybe a minute. All good. So then I went to implement it in my real code, and performance has tanked again. That code is a PS1 that loads a number of PSM1 files. The function to do the find replace is in one of those PSM1 files, and that code calls functions in another PSM1. The test script was everything in a single small PS1.
Given that my test script didn't use a function call at all, I tested that first, so there is a function in the PS1 that gets called 12 million times from the loop in the same PS1. No real performance impact.
So, my thought then was that calling a function in one PSM1 that then calls a function in another PSM1 (12 million times) might be the issue. So I made a dummy function (which just returns the passed string, as if no replacement was needed) in the same PSM1 as the loop. And that is orders of magnitude slower.
I have not tested this with everything in the PS1, mostly because these functions are needed in three different scripts with very different argument requirements, so implementing it with Modules really made a lot of sense logistically, and changing that would be a massive undertaking.
That said, is there a known performance hit when calling a function that lives in a Module? I was under the impression that once the Modules are loaded, it's basically the same as if it was all in a single PS1, but perhaps not? FWIW, I am not using NameSpaces. All of my functions just have function name prefix on the noun side to avoid conflicts.
I also can't really post minimally functional code very easily since that's in a single file that doesn't exhibit the behavior. If there is no obvious answer to someone I guess my next step is to implement the test script with some modules, but that's not really apples to apples either, since my real modules are rather large.
To add a little context: When the function (in a PSM1) does not call a function and simply sets $writeLine = $originalLine total time is 15 seconds.
When doing an actual find and replace inline (no call to a function) like this $writeLine = $originalLine.Replace($replace, $with) total processing time is 16 seconds.
When calling a function in the same PSM1 that just returns the original string total time is 17 minutes.
But again, when it's all in a PS1 file with no modules, calling a function has minimal impact. So it certainly seems like calling a function in a PSM1, even from a function in that same PSM1, has a massive performance overhead.
And more context:
I moved the replace function in the test script into a Module. No appreciable change. So I moved the main code, including the loop, into a function in that module, and called it from the main script. Again, no real change. Both took around 15 seconds.
So, it's not something innate in Modules. That then begs the question, what could I be doing in my other modules that would trigger this behavior? This modules are 3000-10,000 lines of code, so there is a lot going on. Hopefully someone has some insight as to best practices with modules to mitigate this. And hopefully it's not "Don't use big modules". ;)
Final update:
It seems it IS a function of how big the module is. I deleted all the other functions in the Module that contains the loop, and performance is fine, 17 seconds. So, basically even as of PS5.0, the implementation of modules is pretty useless for anything large. Rather disconcerting. I wonder if the same would be true if all the functions where in a single file, and PowerShell performance with large files with lots of functions is just bad? Anyone have any experience down this road?
i have an assignment that is 3 weeks overdue, however my teacher still wants me to hand it up because its worth 15 percent. mine is to use an ultrasonic ranger and an LED bar to measure how close an object gets but i have no idea where to start. please help me!!!
It depends on the parts you have. Go through the specification sheets for each device and consider how you will communicate with them (such as reading a analogue input or communicating using SPI etc). A quick Google with the device part number should also come up with libraries and example code if it's a popular device.
Also, the complexity of the code depends on the deliverables of the project (i.e. is your code marked or is the functionality only checked)
A simple solution for your code would be a loop and some functions with the following structure:
// declare any variables you need for your processing below
void setup() {
// Set up your pins here
}
void loop() {
ReadRange();
// process ultrasonic ranger input here
OutputLED(/*either pass a variable here or leave the variable global - Loads of ways to run this*/);
//Add a delay here to slow down the response (if it's very noisy (very dodgy low pass filter)
}
void ReadRange(){
// code that reads the range using methods that you will research
}
void OutputLED(/*depends if you want to pass variables*/){
// code that outputs to the LEDs using methods that you will research
}
Edit: So I just thought I would add if you're being marked on coding as well, you could make the variables local to the function and pass them through etc. I won't go into detail as the information is easily available online and I would be teaching you basic coding as a result.
I've been diagnosing a performance issue with generating a CSV containing around 50,000 lines and I've narrowed it down to a single function that is used once per line.
After a lot of messing about, I've discovered that there is an overhead in using the function, rather than placing the logic directly in the loop - my question is: Why?!
The function in question is very simple, it accepts a string argument and passes that to a switch/case block containing around 15 options - returning the resulting string.
I've put a bunch of timers all over the place and discovered that a lot (not all) of the time this function call takes between 0 and 200 ms to run... however if I put the exact same code inline, it sits at 0 on every iteration.
All this points to a fundamental issue in my understanding of object instantiation and I'd appreciate some clarification.
I was always under the impression that if I instantiate a component at the top of a page, or indeed if I instantiate it in a persistent scope like Application or Session, then it would be placed into memory and subsequent calls to functions within that component would be lightning fast.
It seems however, that there is an overhead to calling these functions and while we're only talking a few milliseconds, when you have to do that 50,000 times it quickly adds up.
Furthermore, it seems that doing this consumes resources. I'm not particularly well versed in the way the JVM uses memory, I've read up on it and played with settings and such, but it's an overwhelming topic - especially for those of us with no Java development experience. It seems that when calling the method over inline code, sometimes the ColdFusion service just collapses and the request never ends. Other times it does indeed complete, although way too slowly. This suggests that the request can complete only when the server has the resources to handle it - and thus that the method call itself is consuming memory... (?)
If indeed the calling of a method has an overhead attached, I have a big problem. It's not really feasible to move all of this code inline, (while the function in question is simple, there are plenty of other functions that I will need to make use of) and doing so goes against everything I believe as a developer!!
So, any help would be appreciated.
Just for clarity and because I'm sure someone will ask for it, here's the code in question:
EDIT: As suggested, I've changed the code to use a struct lookup rather than CFSwitch - below is amended code for reference, however there's also a test app in pastebin links at the bottom.
Inside the init method:
<cfset Variables.VehicleCategories = {
'T1' : 'Beetle'
, 'T1C' : 'Beetle Cabrio'
, 'T2' : 'Type 2 Split'
, 'T2B' : 'Type 2 Bay'
, 'T25' : 'Type 25'
, 'Ghia' : 'Karmann Ghia'
, 'T3' : 'Type 3'
, 'G1' : 'MK1 Golf'
, 'G1C' : 'MK1 Golf Cabriolet'
, 'CADDY' : 'MK1 Caddy'
, 'G2' : 'MK2 Golf'
, 'SC1' : 'MK1/2 Scirocco'
, 'T4' : 'T4'
, 'CO' : 'Corrado'
, 'MISC' : 'MISC'
} />
Function being called:
<cffunction name="getCategory" returntype="string" output="false">
<cfargument name="vehicleID" required="true" type="string" hint="Vehicle type" />
<cfscript>
if (structKeyExists(Variables.VehicleCategories, Arguments.VehicleID)) {
return Variables.VehicleCategories[Arguments.VehicleID];
}
else {
return 'Base SKUs';
}
</cfscript>
</cffunction>
As requested, I've created a test application to replicate this issue:
http://pastebin.com/KE2kUwEf - Application.cfc
http://pastebin.com/X8ZjL7D7 - TestCom.cfc (Place in 'com' folder outside webroot)
http://pastebin.com/n8hBLrfd - index.cfm
Function call will always be slower than inline code in Any language. That's why there's inline keyword in C++, and in JVM land there is JIT optimizer that will inline functions for you if it deems necessary.
Now ColdFusion is yet another layer on top of JVM. Therefore a function in CF is not a function in JVM, so things don't translate 1:1 at the JIT optimizer standpoint. A CFML function is actually compiled down to a Java class. Also, scopes like arguments, local (Java hashtables) are created on every invocation. Those takes time and memory and therefore overhead.
...if I instantiate it in a persistent scope like Application or
Session, then it would be placed into memory and subsequent calls to
functions within that component would be lightning fast
It'd be faster than instantiating a new instance for sure, but it's not going to be "lightning fast" especially when you call it in a tight loop.
In conclusion, inline the function and if it's still not fast enough, locate the slowest part of the code and write it in Java.
Just a side note here, since Railo uses inner classes instead of complete independent classes, it is faster if you write in such a style as to have many small functions. In my experiments, both engines perform similarly with basic inline code. Adobe ColdFusion lends itself to large god functions if you need to squeak out performance under load. With the JVM being unable to inline ColdFusion functions during compilation, you'll never get the benefit of the compiler being smart with your code.
This is especially important if you created an application that uses a ton of explicit getters/setters and you find your traffic increasing from small volume to high volume. All those little functions will bring you to your knees vs. having fewer large "god" functions.
Slowest to fastest with one basic test we ran of 100,000 iterations:
Adobe ColdFusion (many small functions) ( 200X slower than Java)
Railo (many small functions) (60X slower)
ColdFusion / Railo (all code inline in one giant function) (10X slower)
Native Java Class (fastest)
I have a chunk of lua code that I'd like to be able to (selectively) ignore. I don't have the option of not reading it in and sometimes I'd like it to be processed, sometimes not, so I can't just comment it out (that is, there's a whole bunch of blocks of code and I either have the option of reading none of them or reading all of them). I came up with two ways to implement this (there may well be more - I'm very much a beginner): either enclose the code in a function and then call or not call the function (and once I'm sure I'm passed the point where I would call the function, I can set it to nil to free up the memory) or enclose the code in an if ... end block. The former has slight advantages in that there are several of these blocks and using the former method makes it easier for one block to load another even if the main program didn't request it, but the latter seems the more efficient. However, not knowing much, I don't know if the efficiency saving is worth it.
So how much more efficient is:
if false then
-- a few hundred lines
end
than
throwaway = function ()
-- a few hundred lines
end
throwaway = nil -- to ensure that both methods leave me in the same state after garbage collection
?
If it depends a lot on the lua implementation, how big would the "few hundred lines" need to be to reliably spot the difference, and what sort of stuff should it include to best test (the main use of the blocks is to define a load of possibly useful functions)?
Lua's not smart enough to dump the code for the function, so you're not going to save any memory.
In terms of speed, you're talking about a different of nanoseconds which happens once per program execution. It's harming your efficiency to worry about this, which has virtually no relevance to actual performance. Write the code that you feel expresses your intent most clearly, without trying to be clever. If you run into performance issues, it's going to be a million miles away from this decision.
If you want to save memory, which is understandable on a mobile platform, you could put your conditional code in it's own module and never load it at all of not needed (if your framework supports it; e.g. MOAI does, Corona doesn't).
If there is really a lot of unused code, you can define it as a collection of Strings and loadstring() it when needed. Storing functions as strings will reduce the initial compile time, however of most functions the string representation probably takes up more memory than it's compiled form and what you save when compiling is probably not significant before a few thousand lines... Just saying.
If you put this code in a table, you could compile it transparently through a metatable for minimal performance impact on repeated calls.
Example code
local code_uncompiled = {
f = [=[
local x, y = ...;
return x+y;
]=]
}
code = setmetatable({}, {
__index = function(self, k)
self[k] = assert(loadstring(code_uncompiled[k]));
return self[k];
end
});
local ff = code.f; -- code of x gets compiled here
ff = code.f; -- no compilation here
for i=1, 1000 do
print( ff(2*i, -i) ); -- no compilation here either
print( code.f(2*i, -i) ); -- no compile either, but table access (slower)
end
The beauty of it is that this compiles as needed and you don't really have to waste another thought on it, it's just like storing a function in a table and allows for a lot of flexibility.
Another advantage of this solution is that when the amount of dynamically loaded code gets out of hand, you could transparently change it to load code from external files on demand through the __index function of the metatable. Also, you can mix compiled and uncompiled code by populating the "code" table with "real" functions.
Try the one that makes the code more legible to you first. If it runs fast enough on your target machine, use that.
If it doesn't run fast enough, try the other one.
lua can ignore multiple lines by:
function dostuff()
blabla
faaaaa
--[[
ignore this
and this
maybe this
this as well
]]--
end