Calculating Processing and Request time for multiple Asynchronous Requests - ajax

For data visualization request in my application, I am sending multiple AJAX requests to a servlet in order to get the data in chunks and on callback of each request, the data received is rendered over map.
For this request, I am trying to calculate:
Request Time (how much total time it took for client to get data from server)
Processing Time (how much total time it took for client to render the data on client side)
In order to do this, I am capturing start time of each request before sending it to server (using jquery "beforeSend") and "onSuccess" event of each request, the end time is captured.
Once the all requests are completed, I am deducting the "start time" of first request from the "end time" of last request in order to calculate the total time the client took for fetching records from server. (Similarly for Processing Time)
But somehow my calculation doesn't produce correct results. Could any one please provide me some suggestions for this issue?
for explaining my question in more better way:
var dataProviderRequestsStartTime = [];
var dataProviderRequestsEndTime = [];
var dataParsingStartTime = [];
var dataParsingEndTime = [];
getResults(ids);
var getResults = function(totalIds) {
for(var i=0; i<10; i++;) {
requestResultForOneChunk(totalIds[i]);
}
};
var requestResultForOneChunk = function(streetIds) {
$.ajax({
beforeSend: function() {
var requestStartTime = new Date().getTime();
dataProviderRequestsStartTime.push(requestStartTime);
},
type : 'POST',
url : "myServlet",
contentType : "application/x-www-form-urlencoded",
data : {
"ids" : streetIds,
},
success : function(response) {
//Request Finished
var dataProvideRequestEndTime = new Date().getTime();
dataProviderRequestsEndTime.push(dataProvideRequestEndTime);
addFeaturesToMap(response);
},
error : function(x, e) {
alert("Something went wrong in the request" + e);
}
});
};
var addFeaturesToMap = function(measurements) {
//Parsing Started
var featureParsingStartTime = new Date().getTime();
dataParsingStartTime.push(featureParsingStartTime);
doParsing(measurements);
//Parsing Finished
featureParsingEndTime = new Date().getTime();
dataParsingEndTime.push(featureParsingEndTime);
};
$("#loading").bind(
"ajaxStart",
function(options) {
ajaxStartTime = options.timeStamp;
}).bind("ajaxStop", function(options) {
var ajaxEndTime = options.timeStamp;
var totalTime = (ajaxEndTime - ajaxStartTime);
calculateTimeBreakDown();
});
var calculateTimeBreakDown = function() {
var totalValues = dataProviderRequestsEndTime.length;
var lastValueIndex = totalValues - 1;
// Request Time calculation
var endTimeOfLastRequest = dataProviderRequestsEndTime[lastValueIndex];
var startTimeOfFirstRequest = dataProviderRequestsStartTime[0];
var totalRequestTime = (endTimeOfLastRequest - startTimeOfFirstRequest);
// Parsing Time Calculation
var endTimeOfLastParsing = dataParsingEndTime[lastValueIndex];
var startTimeOfFirstParsing = dataParsingStartTime[0];
var totalParsingTime = (endTimeOfLastParsing - startTimeOfFirstParsing);
};
Finally, I have requestTime(totalRequestTime) and parsingTime(totalParsingTime). But the problem is adding these both doesn't produce value near to total time which is calculated using ajax start and stop.

look at the .ajaxStart() and .ajaxStop() events for "total time", (<- those are also great for progressbars)
http://api.jquery.com/ajaxStart/
http://api.jquery.com/ajaxStop/
and .ajaxSend() and .ajaxComplete() events for "cumulative time" calculations.
http://api.jquery.com/ajaxSend/
http://api.jquery.com/ajaxComplete/
look at this code:
var totalTime = null;
var cachedTime = null;
function alertLoadingTime() {
if(!totalTime) return;
var loadingTime = totalTime / 1000;
console.log("loaded " + loadingTime + " seconds");
}
function timingStart() {
cachedTime = new Date;
}
function timingEnd() {
var endTime = new Date;
totalTime += endTime - cachedTime;
cachedTime = null;
alertLoadingTime();
}
$(document).ajaxStart(timingStart);
$(document).ajaxStop(timingEnd);
note that it will only account for time spent doing ajax calls and won't include the initial page loading time.
to time the parsing:
use the same functions as before but change totalTime to totalParsingTime. (Note: you can achieve this by changing totalTime to reference some other variable)
Call timingStart() right before you append the result of the ajax call to the dom tree.
Have the server add timingEnd() to the end of every response.
totalTime will then be set to the time it took to add everything to the DOM tree.

solution for you would to rely on jquery ajax callback methods
ajaxStart : Register a handler to be called when the first Ajax request begins.
ajaxStop : Register a handler to be called when all Ajax requests have completed including success and error callbacks
I have used the below code snippet in my application and it works perfectly fine to report page rendering time including ajaxs.
var startTime = 0,endTime = 0;
$(document).ajaxStart(function(){
startTime = new Date();
});
$(document).ajaxStop(function(){
endTime = new Date();
console.log("total time required : ",endTime - startTime); //Total time in milliseconds including time spent in success or error callbacks
});

Related

Google Users: List users data of a specific group

I am in the need of listing the users data belonging to a specific group within the organization. The documentation does not specify if this is possible. I was really hoping there could be some kind of query that would allow this. For example email in (1#domain.com,2#domain.com). However, I don't see that being possible. The only way I could think to accomplish this would be:
Get a list of all the members in the group (https://developers.google.com/admin-sdk/directory/reference/rest/v1/members/list)
Get each user data by email (https://developers.google.com/admin-sdk/directory/reference/rest/v1/users/get)
The problem with the above approach is that if a group contains 50+ members, this means that I have to make all that amount of requests, which is counter productive. Imagine how long that would take.
Any ideas? Greatly appreciate it.
Unfortunately I don’t think you can skip this two step process, but you can speed it up using batch requests. This
allows you to request up to 1000 calls in a single request. The steps would be:
Make a batch request to get all the members of all the groups you want (using members.list).
Make a batch request to get all the user info that you need using their id (using user.get).
Notice that the data in the result won’t be sorted, but they will be tagged by Content-ID.
References
Sending Batch Requests (Directory API)
Method: members.list (Directory API)
Method: users.get (Directory API)
I thought about the batching request a couple of hours after I posted the question. The problem with Node JS is that it does not has built in support for batch requests, unlike the php client library for example; Therefore, I had to spent some time implementing support for it on my own since I was not able to find any example. I'll share the solution in case it helps someone else or for my future reference.
async function getGroupMembersData(){
const groupEmail = "group#domain.com"; //google group email
const groupMembers = await getGroupMembers(groupEmail).catch(error=>{
console.error(`Error querying group members: ${error.toString()}`);
});
if(!groupMembers){ return; }
const url = "https://www.googleapis.com/batch/admin/directory_v1";
const scopes = ["https://www.googleapis.com/auth/admin.directory.user.readonly"];
const requests = [];
for(let i=0; i<groupMembers.length; ++i){
const user = groupMembers[i];
const request = {
email: user,
endpoint: `GET directory_v1/admin/directory/v1/users/${user}?fields=*`
};
requests.push(request);
}
const batchRequestData = await batchProcess(url, scopes, requests).catch(error=>{
console.error(`Error processing batch request: ${error.toString()}`);
});
if(!batchRequestData){ return; }
const usersList = batchRequestData.map(i=>{
return i.responseBody;
});
console.log(usersList);
}
//get group members using group email address
async function getGroupMembers(groupKey){
const client = await getClient(scopes); //function to get an authorized client, you have to implement on your own
const service = google.admin({version: "directory_v1", auth: client});
const request = await service.members.list({
groupKey,
fields: "members(email)",
maxResults: 200
});
const members = !!request.data.members ? request.data.members.map(i=>i.email) : [];
return members;
}
//batch request processing in groups of 100
async function batchProcess(batchUrl, scopes, requests){
const client = await getClient(scopes); //function to get an authorized client, you have to implement on your own
let results = [];
const boundary = "foobar99998888"; //boundary line definition
let batchBody = ""; const nl = "\n";
const batchLimit = 100; //define batch limit (max supported = 100)
const totalRounds = Math.ceil(requests.length / batchLimit);
let batchRound = 1;
let batchItem = 0;
let roundLimit = batchLimit;
do{
roundLimit = roundLimit < requests.length ? roundLimit : requests.length;
//build the batch request body
for(batchItem; batchItem<roundLimit; batchItem++){
const requestData = requests[batchItem];
batchBody += `--${boundary}${nl}`;
batchBody += `Content-Type: application/http${nl}`;
batchBody += `Content-Id: <myapprequest-${requestData.email}>${nl}${nl}`;
batchBody += `${requestData.endpoint}${nl}`;
}
batchBody += `--${boundary}--`;
//send the batch request
const batchRequest = await client.request({
url: batchUrl,
method: "POST",
headers: {
"Content-Type": `multipart/mixed; boundary=${boundary}`
},
body: batchBody
}).catch(error=>{
console.log("Error processing batch request: " + error);
});
//parse the batch request response
if(!!batchRequest){
const batchResponseData = batchRequest.data;
const responseBoundary = batchRequest.headers["content-type"].split("; ")[1].replace("boundary=", "");
const httpResponses = batchResponseParser(batchResponseData, responseBoundary);
results.push(...httpResponses);
}
batchRound++;
roundLimit += batchLimit;
} while(batchRound <= totalRounds);
return results;
};
//batch response parser
function batchResponseParser(data, boundary){
const nl = "\r\n";
data = data.replace(`--${boundary}--`,"");
const responses = data.split(`--${boundary}`);
responses.shift();
const formattedResponses = responses.map(i=>{
const parts = i.split(`${nl}${nl}`);
const responseMetaParts = (parts[0].replace(nl, "")).split(nl);
let responseMeta = {};
responseMetaParts.forEach(part=>{
const objectParts = part.split(":");
responseMeta[objectParts[0].trim()] = objectParts[1].trim();
});
const responseHeadersParts = parts[1].split(nl);
let responseHeaders = {};
responseHeadersParts.forEach(part=>{
if(part.indexOf("HTTP/1.1") > -1){
responseHeaders.status = part;
} else {
const objectParts = part.split(":");
responseHeaders[objectParts[0].trim()] = objectParts[1].trim();
}
});
const reg = new RegExp(`${nl}`, "g");
const responseBody = JSON.parse(parts[2].replace(reg, ""));
const formatted = {
responseMeta: responseMeta,
responseHeaders: responseHeaders,
responseBody: responseBody
};
return formatted;
});
return formattedResponses;
}

Is there a way to speed up/batch Google Calendar read/writes?

I am new to Google Apps Script and learning javascript as I go about this project. Over the course of the introductory codelabs I noted the best practice to read all the data into an array with one command, perform operations, and then write it with one command.
I understood how to do this working with Google Sheets but how do I achieve this working with Google Calendar? I have come across a few links discussing batching with Google Calendar API and Advanced Google Services but I didn't understand how to make use of the information.
I basically hope to batch edit events instead of accessing Google Calendar repeatedly in a for loop.
function deleteMonth() {
// Set Date range to delete
var today = new Date();
var firstDay = new Date(today.getFullYear(), today.getMonth(), 1);
var lastDay = new Date(today.getFullYear(), today.getMonth() + 1, 0);
// read spreadsheet data and get User Info from ss
var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
var idSheet = spreadsheet.getSheetByName('User Info');
//Get users from sheet in array of objects with properties from column header in
//'User Info' (name, email, year, calName, calID, early, late)
var userInfo = getSheetData(idSheet);
var deletedNames = "";
for (i = 0; i < userInfo.length; i++) {
var calID = userInfo[i].calID;
// if we have calID proceed to delete events
if (calID) {
console.time("get events");
var calendar = CalendarApp.getCalendarById(calID);
var events = calendar.getEvents(firstDay, lastDay);
console.timeEnd("get events");
// Delete events and add deleted name to string
// deletedNames
for (i = 0; i < events.length; i++) {
console.time("delete event");
deletedNames += events[i].getTitle() + ", ";
events[i].deleteEvent();
console.timeEnd("delete event");
}
}
}
spreadsheet.toast("Deleted events: \n" + deletedNames);
}
Time output from console.time():
Other related links which sounded relevant:
Using advanced google services (apps script resource)
Google Developer blog?
I believe your goal as follows
You want to delete all events in a month for several calendars using the batch process with Google Apps Script.
You want to reduce the process cost of above process.
For this, how about this answer?
Issue and workaround:
Calendar API can process with the batch requests. The batch requests can run 100 requests by one API call and can process with the asynchronous process. By this, I think that the process cost can bereduced. But, in the current stage, unfortunately, several calendar IDs cannot be used in one batch. When the requests including several calendar IDs, an error of Cannot perform operations on different calendars in the same batch request. occurs. So in your case, all events in a month in one calendar ID can be deleted in one batch requests. It is required to request the number of calendar IDs. I would like to propose this as the current workaround.
By the way, as the modification point of your scrit, in your script, the variable i is used in 1st for loop and 2nd for loop. By this, all values of userInfo are not used. Please be careful this.
Sample script:
Before you run the script, please enable Calendar API at Advanced Google services.
function deleteMonth() {
var today = new Date();
var firstDay = new Date(today.getFullYear(), today.getMonth(), 1);
var lastDay = new Date(today.getFullYear(), today.getMonth() + 1, 0);
var spreadsheet = SpreadsheetApp.getActiveSpreadsheet();
var idSheet = spreadsheet.getSheetByName('User Info');
var userInfo = getSheetData(idSheet);
var deletedNames = "";
var requests = []; // For batch requests.
for (i = 0; i < userInfo.length; i++) {
var req = [];
var calID = userInfo[i].calID;
if (calID) {
var calendar = CalendarApp.getCalendarById(calID);
var events = calendar.getEvents(firstDay, lastDay);
for (j = 0; j < events.length; j++) {
deletedNames += events[j].getTitle() + ", ";
var e = events[j];
req.push({
method: "DELETE",
endpoint: `https://www.googleapis.com/calendar/v3/calendars/${calID}/events/${e.getId().replace("#google.com", "")}`,
});
}
}
requests.push(req);
}
// Run batch requests.
requests.forEach(req => {
const limit = 100;
const split = Math.ceil(req.length / limit);
const boundary = "xxxxxxxxxx";
for (let i = 0; i < split; i++) {
const object = {batchPath: "batch/calendar/v3", requests: req.splice(0, limit)};
const payload = object.requests.reduce((s, e, i) => s += "Content-Type: application/http\r\nContent-ID: " + i + "\r\n\r\n" + e.method + " " + e.endpoint + "\r\nContent-Type: application/json; charset=utf-8\r\n\r\n" + JSON.stringify(e.requestBody) + "\r\n--" + boundary + "\r\n", "--" + boundary + "\r\n");
const params = {method: "post", contentType: "multipart/mixed; boundary=" + boundary, payload: payload, headers: {Authorization: "Bearer " + ScriptApp.getOAuthToken()}, muteHttpExceptions: true};
var res = UrlFetchApp.fetch("https://www.googleapis.com/" + object.batchPath, params);
console.log(res.getContentText())
}
})
spreadsheet.toast("Deleted events: \n" + deletedNames);
}
Note:
Please use this script with V8.
References:
Advanced Google services
Sending Batch Requests
Events: delete

Activity Page Token

I'm trying to keep track within the activities list, but the pageToken is really useless!!
Just to test, I ran this (javascript) code:
function Get(Url)
{
var xmlHttp = new XMLHttpRequest();
xmlHttp.open( "GET", Url, false );
xmlHttp.send( null );
return JSON.parse(xmlHttp.responseText);
}
pageToken = "";
do {
console.log(pageToken = (x = Get("https://www.googleapis.com/appsactivity/v1/activities?\
access_token=***ACCESSTOKEN***\
&source=drive.google.com\
&drive.ancestorId=0B8dK0Hw8TTYoX2pmOTZBcnlzVFE\
&pageSize=10\
&fields=*\
&pageToken="+pageToken)).nextPageToken);
} while (pageToken);
I get the same Activities list (x) everytime, with a different and random nextPageToken. There's not following like with the Drive.Changes API.
Should I just result to using the Changes API? I've spent 2 days trying to get this working! ​

Google Chrome Average request time

On my project, I would like to optimize an ajax request and to know, on average, how many ms I have gained.
So, thanks to the Google Chrome network tab, I have the time of a request, something like that :
Is there a feature to have some stats about our request ? For example the average time.
If no, how to do that ?
Thanks !
Not too difficult to roll your own code in JavaScript.
var times = [];
var sum = 0;
var tries = 10
for(var i=0; i<tries; i++) {
var xhr = new XMLHttpRequest();
xhr.open("GET", window.location.href, false);
xhr.onload = (function() {
var time = (Date.now() - this.start);
times.push(time);
sum += time;
console.log("#" + this.number + " " + time + "ms");
}).bind(xhr);
xhr.number = (i + 1);
xhr.start = Date.now();
xhr.send(null);
}
console.log("avg: " + (sum / tries) + "ms");
Go to url: chrome://net-internals
Description here

Pass a variable into ScrollTo

SOLVED
Here's the result:
window.onload = function timeScroll()
{
var current_date = new Date();
hour_value = current_date.getHours();
var big_number = hour_value*833.3
$(window).scrollTop(big_number)
}
PREVIOUS QUESTION
I'm working on a page, that takes the time of day (hour) and uses that time to scroll to a particular point on the page. I multiply the getHour in order to get a variable I've called big_number. However, I can't seem to pass that variable into my scroll function.
This tells me that it's calculating the number correctly:
window.onload = function timeScroll()
{
var current_date = new Date();
hour_value = current_date.getHours();
var big_number = hour_value*833.3
window.alert(big_number);
}
I'm looking to do something like this:
window.onload = function timeScroll()
{
var current_date = new Date();
hour_value = current_date.getHours();
var big_number = hour_value*833.3
**scrollTo(x,big_number)**
}
Anything you've got helps. Also, here's the site live...
http://newmedia.emerson.edu/~caleb_ungewitter/weather/index.html

Resources