I need an output as follows for 1200+ accounts:
accounts_count = {
"a": 120,
"b": 45,
"z": 220
}
So it will take the first Letter of an account name and make count for all account with their initial letter.
How can I solve it using Zoho Deluge script in a custom function?
The hard part is getting all the contacts due to the 200 record limit in zoho
I'm not going to provide the full solution here, just the harder parts.
// this is the maximum number of records that you can query at once
max = 200;
// set to a value slightly higher than you need
max_contacts = 1500;
// round up one
n = max_contacts / max + 1;
// This is how many times you will loop (convert to int)
iterations = n.toNumber();
// this is a zoho hack to create a range of the length we want
counter = leftpad("1",iterations).replaceAll(" ","1,").toList();
// set paging
page = 1;
// TODO: initialize an alphabet map
//result = Map( {a: 0, b:0....});
// set to true when done so you're not wasting API Calls
done = false;
for each i in counter
{
// prevent extra crm calls
if(!done)
{
response = zoho.crm.getRecords("Contacts",page,max);
if(!isEmpty(response))
{
if(isNull(response.get(0)) && !isNull(response.get("code")))
{
info "Error:";
info response;
}
else
{
for each user in response
{
// this is the easy part: get the first letter (lowercase)
// get the current value and add one
// set the value again
// set in result map
}
}
}
else
{
done = true;
}
}
// increment the page
page = page + 1;
}
Related
I am working on a compiler and one aspect currently is how to wait for interpolated variable names to be resolved. So I am wondering how to take a nested interpolated variable string and build some sort of simple data model/schema for unwrapping the evaluated string so to speak. Let me demonstrate.
Say we have a string like this:
foo{a{x}-{y}}-{baz{one}-{two}}-foo{c}
That has 1, 2, and 3 levels of nested interpolations in it. So essentially it should resolve something like this:
wait for x, y, one, two, and c to resolve.
when both x and y resolve, then resolve a{x}-{y} immediately.
when both one and two resolve, resolve baz{one}-{two}.
when a{x}-{y}, baz{one}-{two}, and c all resolve, then finally resolve the whole expression.
I am shaky on my understanding of the logic flow for handling something like this, wondering if you could help solidify/clarify the general algorithm (high level pseudocode or something like that). Mainly just looking for how I would structure the data model and algorithm so as to progressively evaluate when the pieces are ready.
I'm starting out trying and it's not clear what to do next:
{
dependencies: [
{
path: [x]
},
{
path: [y]
}
],
parent: {
dependency: a{x}-{y} // interpolated term
parent: {
dependencies: [
{
}
]
}
}
}
Some sort of tree is probably necessary, but I am having trouble figuring out what it might look like, wondering if you could shed some light on that with some pseudocode (or JavaScript even).
watch the leaf nodes at first
then, when the children of a node are completed, propagate upward to resolving the next parent node. This would mean once x and y are done, it could resolve a{x}-{y}, but then wait until the other nodes are ready before doing the final top-level evaluation.
You can just simulate it by sending "events" to the system theoretically, like:
ready('y')
ready('c')
ready('x')
ready('a{x}-{y}')
function ready(variable) {
if ()
}
...actually that may not work, not sure how to handle the interpolated nodes in a hacky way like that. But even a high level description of how to solve this would be helpful.
export type SiteDependencyObserverParentType = {
observer: SiteDependencyObserverType
remaining: number
}
export type SiteDependencyObserverType = {
children: Array<SiteDependencyObserverType>
node: LinkNodeType
parent?: SiteDependencyObserverParentType
path: Array<string>
}
(What I'm currently thinking, some TypeScript)
Here is an approach in JavaScript:
Parse the input string to create a Node instance for each {} term, and create parent-child dependencies between the nodes.
Collect the leaf Nodes of this tree as the tree is being constructed: group these leaf nodes by their identifier. Note that the same identifier could occur multiple times in the input string, leading to multiple Nodes. If a variable x is resolved, then all Nodes with that name (the group) will be resolved.
Each node has a resolve method to set its final value
Each node has a notify method that any of its child nodes can call in order to notify it that the child has been resolved with a value. This may (or may not yet) lead to a cascading call of resolve.
In a demo, a timer is set up that at every tick will resolve a randomly picked variable to some number
I think that in your example, foo, and a might be functions that need to be called, but I didn't elaborate on that, and just considered them as literal text that does not need further treatment. It should not be difficult to extend the algorithm with such function-calling features.
class Node {
constructor(parent) {
this.source = ""; // The slice of the input string that maps to this node
this.texts = []; // Literal text that's not part of interpolation
this.children = []; // Node instances corresponding to interpolation
this.parent = parent; // Link to parent that should get notified when this node resolves
this.value = undefined; // Not yet resolved
}
isResolved() {
return this.value !== undefined;
}
resolve(value) {
if (this.isResolved()) return; // A node is not allowed to resolve twice: ignore
console.log(`Resolving "${this.source}" to "${value}"`);
this.value = value;
if (this.parent) this.parent.notify();
}
notify() {
// Check if all dependencies have been resolved
let value = "";
for (let i = 0; i < this.children.length; i++) {
const child = this.children[i];
if (!child.isResolved()) { // Not ready yet
console.log(`"${this.source}" is getting notified, but not all dependecies are ready yet`);
return;
}
value += this.texts[i] + child.value;
}
console.log(`"${this.source}" is getting notified, and all dependecies are ready:`);
this.resolve(value + this.texts.at(-1));
}
}
function makeTree(s) {
const leaves = {}; // nodes keyed by atomic names (like "x" "y" in the example)
const tokens = s.split(/([{}])/);
let i = 0; // Index in s
function dfs(parent=null) {
const node = new Node(parent);
const start = i;
while (tokens.length) {
const token = tokens.shift();
i += token.length;
if (token == "}") break;
if (token == "{") {
node.children.push(dfs(node));
} else {
node.texts.push(token);
}
}
node.source = s.slice(start, i - (tokens.length ? 1 : 0));
if (node.children.length == 0) { // It's a leaf
const label = node.texts[0];
leaves[label] ??= []; // Define as empty array if not yet defined
leaves[label].push(node);
}
return node;
}
dfs();
return leaves;
}
// ------------------- DEMO --------------------
let s = "foo{a{x}-{y}}-{baz{one}-{two}}-foo{c}";
const leaves = makeTree(s);
// Create a random order in which to resolve the atomic variables:
function shuffle(array) {
for (var i = array.length - 1; i > 0; i--) {
var j = Math.floor(Math.random() * (i + 1));
[array[j], array[i]] = [array[i], array[j]];
}
return array;
}
const names = shuffle(Object.keys(leaves));
// Use a timer to resolve the variables one by one in the given random order
let index = 0;
function resolveRandomVariable() {
if (index >= names.length) return; // all done
console.log("\n---------------- timer tick --------------");
const name = names[index++];
console.log(`Variable ${name} gets a value: "${index}". Calling resolve() on the connected node instance(s):`);
for (const node of leaves[name]) node.resolve(index);
setTimeout(resolveRandomVariable, 1000);
}
setTimeout(resolveRandomVariable, 1000);
your idea of building a dependency tree it's really likeable.
Anyway I tryed to find a solution as simplest possible.
Even if it already works, there are many optimizations possible, take this just as proof of concept.
The background idea it's produce a List of Strings which you can read in order where each element it's what you need to solve progressively. Each element might be mandatory to solve something that come next in the List, hence for the overall expression. Once you solved all the chunks you have all pieces to solve your original expression.
It's written in Java, I hope it's understandable.
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Objects;
public class StackOverflow {
public static void main(String[] args) {
String exp = "foo{a{x}-{y}}-{baz{one}-{two}}-foo{c}";
List<String> chunks = expToChunks(exp);
//it just reverse the order of the list
Collections.reverse(chunks);
System.out.println(chunks);
//output -> [c, two, one, baz{one}-{two}, y, x, a{x}-{y}]
}
public static List<String> expToChunks(String exp) {
List<String> chunks = new ArrayList<>();
//this first piece just find the first inner open parenthesys and its relative close parenthesys
int begin = exp.indexOf("{") + 1;
int numberOfParenthesys = 1;
int end = -1;
for(int i = begin; i < exp.length(); i++) {
char c = exp.charAt(i);
if (c == '{') numberOfParenthesys ++;
if (c == '}') numberOfParenthesys --;
if (numberOfParenthesys == 0) {
end = i;
break;
}
}
//this if put an end to recursive calls
if(begin > 0 && begin < exp.length() && end > 0) {
//add the chunk to the final list
String substring = exp.substring(begin, end);
chunks.add(substring);
//remove from the starting expression the already considered chunk
String newExp = exp.replace("{" + substring + "}", "");
//recursive call for inner element on the chunk found
chunks.addAll(Objects.requireNonNull(expToChunks(substring)));
//calculate other chunks on the remained expression
chunks.addAll(Objects.requireNonNull(expToChunks(newExp)));
}
return chunks;
}
}
Some details on the code:
The following piece find the begin and the end index of the first outer chunk of expression. The background idea is: in a valid expression the number of open parenthesys must be equal to the number of closing parenthesys. The count of open(+1) and close(-1) parenthesys can't ever be negative.
So using that simple loop once I find the count of parenthesys to be 0, I also found the first chunk of the expression.
int begin = exp.indexOf("{") + 1;
int numberOfParenthesys = 1;
int end = -1;
for(int i = begin; i < exp.length(); i++) {
char c = exp.charAt(i);
if (c == '{') numberOfParenthesys ++;
if (c == '}') numberOfParenthesys --;
if (numberOfParenthesys == 0) {
end = i;
break;
}
}
The if condition provide validation on the begin and end indexes and stop the recursive call when no more chunks can be found on the remained expression.
if(begin > 0 && begin < exp.length() && end > 0) {
...
}
I tried the "for loop" and "do ... while ...", both of them stopped in the middle of the looping process, and the rest of the codes, which come after the loop, did not run. This becomes an issue when I loop through hundreds of rows.
I know that the use of array is a better solution as the code execution is faster, but I have a difficulty in setting the borders in batch as there is no ".setBorders(Array)" function in Google Sheets.
The sheet provided here has been simplified only to show the looping issues. The actual sheet is written to automatically create hundreds of tables with different values, font weights and horizontal alignments.
What I want to do:
Choose the option "Yes" to start the looping (the current row of looping is also tracked and recorded in "CURRENT ROW" and the "STATUS" shows "Processing ...")
The program will check the "SCORE" column, if "SCORE" is empty (""), than the font weight for that row is set to "bold", else, the font weight is set to "normal"
If the looping is done until the last row, the "STATUS" shows "Done".
The following is the copy of the Google App Script:
let app = SpreadsheetApp;
let ss = app.getActiveSpreadsheet();
let activeSheet = ss.getActiveSheet();
let sheetName = activeSheet.getName();
let sheet1 = ss.getSheetByName("Sheet1");
let loopOptionRange = sheet1.getRange(4, 4);
let loopOption = loopOptionRange.getValue();
let loopStatusRange = sheet1.getRange(4, 7);
function onEdit(e) {
}
function onOpen() {
loopOptionRange.setValue("Choose");
sheet1.getRange(4, 5).setValue(5);
loopStatusRange.setValue("");
}
function loopTest() {
const startRow = 4; //table head
const lastRow = sheet1.getLastRow();
sheet1.getRange(4, 6).setValue(lastRow);
const startRowLoop = startRow + 1; //first row of looping
try {
for (i = startRowLoop; i <= lastRow; i++) {
const testStatus = sheet1.getRange(i, 3).getValue();
sheet1.getRange(4, 5).setValue(i);
if (testStatus == "") {
sheet1.getRange(i, 1, 1, 2).setFontWeight("bold");
} else {
sheet1.getRange(i, 1, 1, 3).setFontWeight("normal");
}
}
loopStatusRange.setValue("Done");
loopOptionRange.setValue("Choose");
} catch (error) {
app.getUi().alert(`An error occured.`);
}
}
if (sheetName === "Sheet1"){
if (loopOption == "Yes") {
loopStatusRange.setValue("Processing ...");
loopTest();
} else if (loopOption === "Cancel") {
loopOptionRange.setValue("Choose");
}
}
LOOP TEST - Google Sheets file
When I saw your script, getValue, setValue and setFontWeight are used in a loop. In this case, the process cost will become high. I thought that this might be the reason for your issue. In order to reduce the process cost of your script, how about the following modification?
From:
for (i = startRowLoop; i <= lastRow; i++) {
const testStatus = sheet1.getRange(i, 3).getValue();
sheet1.getRange(4, 5).setValue(i);
if (testStatus == "") {
sheet1.getRange(i, 1, 1, 2).setFontWeight("bold");
} else {
sheet1.getRange(i, 1, 1, 3).setFontWeight("normal");
}
}
To:
const range = sheet1.getRange(startRow, 3, lastRow - startRowLoop);
const values = range.getDisplayValues().map(([c]) => [c ? null : "bold"]);
range.offset(0, -1).setFontWeights(values);
Note:
About Is it really because of "the high process cost" you mentioned in your answer?, when I saw your script for the 1st time, I thought that the reason for your issue might be due to the process cost. Because, when the script is run by OnEdit trigger, the maximum execution time is 30 seconds. And, when I tested your script and your Spreadsheet, when I saw the log after the script was stopped, an error related to the over of the maximum execution time. By this, I resulted that the reason for your issue is due to the process cost of the script.
References:
map()
setFontWeights(fontWeights)
sometime based on some condition it may want to jump (or move forward) a few steps inside the for loop,
how to do it is kolin?
a simplified use case:
val datArray = arrayOf(1, 2, 3......)
/**
* start from the index to process some data, return how many data has been
consumed
*/
fun processData_1(startIndex: Int) : Int {
// process dataArray starting from the index of startIndex
// return the data it has processed
}
fun processData_2(startIndex: Int) : Int {
// process dataArray starting from the index of startIndex
// return the data it has processed
}
in Java it could be:
for (int i=0; i<datArray.lenght-1; i++) {
int processed = processData_1(i);
i += processed; // jump a few steps for those have been processed, then start 2nd process
if (i<datArray.lenght-1) {
processed = processData_2(i);
i += processed;
}
}
How to do it in kotlin?
for(i in array.indices){
val processed = processData(i);
// todo
}
With while:
var i = 0
while (i < datArray.length - 1) {
var processed = processData_1(i)
i += processed // jump a few steps for those have been processed, then start 2nd process
if (i < datArray.length - 1) {
processed = processData_2(i)
i += processed
}
i++
}
You can do that with continue as stated in the Kotlin docs here: https://kotlinlang.org/docs/reference/returns.html
Example:
val names = arrayOf("james", "john", "jim", "jacob", "johan")
for (name in names) {
if(name.length <= 4) continue
println(name)
}
This would only print names longer than 4 characters (as it skips names with a length of 4 and below)
Edit: this only skips one iteration at a time. So if you want to skip multiple, you could store the process state somewhere else and check the status for each iteration.
I am doing an edit distance with the user input. I am storing my values in array. then the edit distance will compare the user input with my array of strings. I am doing a loop that if the edit distance is more than 2 it will display invalid else valid.
The only problem I've got is that although the program is working out fine, the output is the result of all the '28' strings that I have in my array. I would like to display only invalid or valid once.
Test is my array of strings and user is - String user - the user input.
void testingLD()
{
for (int i=0; i<test.length; i++)
{
if(getLevenshteinDistance(test[i],user) > 2)
{
println ("Invalid re-input");
}
else
{
println ("Valid");
}
}
}
You have your print line functions inside your loop so they get printed once per iteration.
Try this.
void testingLD()
{
boolean isValid = true; // assume true, check each and update
// begin loop
for (int i=0; i<test.length; i++)
{
if(getLevenshteinDistance(test[i],user) > 2)
{
isValid = false;
break; // don't need to test the rest of the loop if we already found a false
}
}
// end loop
if(isValid){
println("Valid");
}else{
println("Invalid re-input");
}
}
Similarly you could count the number of valid int validCount = 0; validCount++ and then display stats about how many were valid, the percentage etc. Or keep an array of the invalid strings and display those as the ones that fail etc!
Wrap up:
When you want to check an entire collection or array for some condition and output one answer make sure to have your output outside of the loop!
I wrote a script that gets a rows data from a spreadsheet and loops through them, calling a function to send an SMS if the rows' data meets certain conditions (having a phone number and not having already been sent for example).
However after adding about 600 rows, the script execution time exceeds it's limit, that seems to be 5 minutes according to my research. I'm using JavaScript objects to read data and a for loop to iterate through the rows.
Can anyone tel me if it is possible to make it faster? I'm very new to programming but this seems such a light task for all this computing power that I can't understand why it takes so long
Thanks in advance!
Here's the code of the function I'm using:
// Will send SMS on the currently active sheet
function sendSms() {
// Use the send sms menu to trigger reconcile
var user = ScriptProperties.getProperty(PROPERTY_USER_RECONCILE);
if (user == null)
reconcileUser();
// The sheets
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Registo");
var settingsSheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Settings");
// Fetch values for each row in the Range.
var startRow = 2;
var apiKey = settingsSheet.getRange("B2").getValue();
var apiSecret = settingsSheet.getRange("B3").getValue();
var prefix = settingsSheet.getRange("B4").getValue();
var numRows = sheet.getMaxRows() - 1;
var numCols = 16;
var statusColNum = 15; // IMPT: To keep track status in col 15
var dataRange = sheet.getRange(startRow, 1, numRows, numCols);
// Make sure there is API key and secret
if (apiKey == "" || apiSecret == "") {
Browser.msgBox("You MUST fill in your API key and secret in Settings sheet first!");
return;
}
// Create one JavaScript object per row of data.
var objects = getRowsData(sheet, dataRange);
var totalSent = 0;
for (var i = 0; i < objects.length; ++i) {
// Get a row object
var rowData = objects[i];
var ss = SpreadsheetApp.getActiveSpreadsheet();
var templateSheet = ss.getSheetByName("SMS Modelo");
var template = templateSheet.getRange("A1").getValue();
// jump loop iteration if conditions not satisied
if (rowData.resolv == "x" || rowData.contactoUtente == null || rowData.contactoUtente == "" || rowData.reserv == null || rowData.reserv == "" || rowData.cont == "x" || rowData.sms !== null) continue;
var message = fillInTemplateFromObject(template, rowData);
var senderName = "Farm Cunha"
var mobile = rowData.contactoUtente;
// Send via Nexmo API
var response = nexmoSendSms(apiKey, apiSecret,"+351" + mobile, message, senderName);
if (response.getResponseCode() == 200) {
var object = JSON.parse(response.getContentText());
if (object.messages[0]['status'] == "0") {
// Set to QUEUE status - We assumed SENT, as we don't handle delivery status.
//sheet.getRange(startRow + i, statusColNum).setValue(STATUS_QUEUE);
sheet.getRange(startRow + i, statusColNum).setValue(STATUS_SENT);
// Set the reference id
sheet.getRange(startRow + i, 19).setValue(object.messages[0]['message-id']);
// sheet.getRange(startRow + i, statusColNum+3).setValue(new Date()); linha pode ser activada para fazer timestamp do envio
totalSent++;
}
else {
// If status is not 0, then it is an error.
// Set status to the error text
sheet.getRange(startRow + i, statusColNum).setValue(object.messages[0]['error-text']);
}
}
else {
// Non 200 OK response
sheet.getRange(startRow + i, statusColNum).setValue("Error Response Code: " + response.getResponseCode);
}
SpreadsheetApp.flush();
// Need a wait. Need to throttle else will have "Route Busy" error.
Utilities.sleep(2000);
}
// Update total sent
var lastTotalSent = parseInt(ScriptProperties.getProperty(PROPERTY_SMS_SENT_FOR_RECONCILE));
if (isNaN(lastTotalSent)) lastTotalSent = 0;
ScriptProperties.setProperty(PROPERTY_SMS_SENT_FOR_RECONCILE, (lastTotalSent + totalSent).toString());
Logger.log("Last sent: " + lastTotalSent + " now sent: " + totalSent);
reconcileApp();
}
You have a few things in your loop that are too time consuming : spreadsheet readings and API calls + 2 seconds sleep !.
I would obviously advise you to take these out of the loop (specially the template sheet reading that is always the same!). A possible solution would be to check the conditions from the row objects and to save the valid entries in an array... THEN iterate in this array to call the API.
If this is still too long then proceed by small batches, saving the end position of the partial iteration in scriptproperties and using a timer trigger that will continue the process every 5 minutes until it is completed (and kill the trigger at the end).
There are a few example of this kind of "mechanics" on this forum, one recent example I suggested is here (it's more like a draft but the idea is there)
Ok, I've solved it by taking these 3 lines out of the loop as Serge (thanks) had told me to:
var ss = SpreadsheetApp.getActiveSpreadsheet();
var templateSheet = ss.getSheetByName("SMS Modelo");
var template = templateSheet.getRange("A1").getValue();
It's so simple that I don't know how I was not seeing that.
This simple change made the script much faster. For example, going through 600 rows would take more than 5 minutes. Now, more than 5000 rows only take seconds.