How to do paging with Exchange Web Services CalendarView - exchange-server

If I do this:
_calendar = (CalendarFolder)Folder.Bind(_service, WellKnownFolderName.Calendar);
var findResults = _calendar.FindAppointments(
new CalendarView(startDate.Date, endDate.Date)
);
I sometimes get an exception that too many items were found.
"You have exceeded the maximum number of objects that can be returned for the find operation. Use paging to reduce the result size and try your request again."
CalendarView supports a constructor that will let me specify MaxItemsReturned, but I can't figure out how I would call it again, specifying the offset for paging. ItemView has this constructor:
public ItemView(int pageSize, int offset)
And the usage of that is obvious.
What about CalendarView? How does one do paging with a CalendarView?
I could reduce the date range to be a shorter span, but there's still no way of determining if it will work for sure.

CalendarView is not actually derived from PagedView, so all of the paging logic that you expect isn't possible. MaxItemsReturned is more of an upper limit than a page size. The error that's returned is more relevant to the PagedView derived view types.
I played around with some PowerShell to emulate paging by rolling the CalendarView window based on the last item returned, but unfortunately the logic behind the CalendarView and Appointment expansion make it impossible to get exactly what you need. Basically as it does the expansion, it's going to stop at "N" items, but you might have more than one appointment that starts at the exact same time and it may give you one, but not the rest. Also, any appointments that overlap the window will get included, so the below code would go into an infinite loop if you had 50 appointments on the calendar that all had the same start time.
Add-Type -Path "C:\Program Files\Microsoft\Exchange\Web Services\1.2\Microsoft.Exchange.WebServices.dll"
$service = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService
$cred = New-Object Microsoft.Exchange.WebServices.Data.WebCredentials ($user , $passwd)
$service.UseDefaultCredentials = $false
$service.Credentials = $cred
$service.AutodiscoverUrl($user)
$num=50
$total=0
$propsetfc = [Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties
$calfolder = [Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Calendar
$service.UserAgent = "EWSCalViewTest"
$calview = New-Object Microsoft.Exchange.WebServices.Data.CalendarView("1/1/2012","12/31/2012", $num)
$calview.PropertySet = $propsetfc
do {
$findresults = $service.FindAppointments($calfolder,$calview)
write-host "Found:" $findresults.Items.Count "of" $findresults.TotalCount
$calview.StartDate = $findresults.Items[$findresults.Items.Count-1].Start
$total+=$findresults.Items.Count
} while($findresults.MoreAvailable)
write-host $total "total found (including dups)"
Unfortunately the expansion and overlap logic mean you'll get duplicates this way, at least one duplicate for each call beyond the first.
If I had to write code using CalendarView, I'd probably use a MaxItemsReturned of 1000 (this is also the limit that throws you into the error condition if you don't specify MaxItemsReturned). If you get them all in one call, you're good. If you have to make a second call, then you'll have to do some extra work to dedup the result set. I'd also try to limit the burden on the server by using as narrow of a date window as possible in the CalendarView since you're asking Exchange to calculate the expansion of recurring appointments across the entire time span. It can be a fairly expensive operation for the server.

You can use ItemView and SearchFilter to query appointments:
var itemView = new ItemView(100, 0);
itemView.PropertySet = new PropertySet(BasePropertySet.IdOnly,
ItemSchema.Subject, AppointmentSchema.Start, AppointmentSchema.End);
var filter = new SearchFilter.SearchFilterCollection(LogicalOperator.And,
new SearchFilter.IsEqualTo(ItemSchema.ItemClass, "IPM.Appointment"),
new SearchFilter.IsGreaterThanOrEqualTo(AppointmentSchema.Start, startDate),
new SearchFilter.IsLessThan(AppointmentSchema.Start, endDate));
bool moreAvailable = true;
while (moreAvailable)
{
var result = _service.FindItems(WellKnownFolderName.Calendar, filter, itemView);
foreach (var appointment in result.OfType<Appointment>())
{
DateTime start = appointment.Start;
DateTime end = appointment.End;
string subject = appointment.Subject;
// ...
}
itemView.Offset += itemView.PageSize;
moreAvailable = result.MoreAvailable;
}

You can still paginate the FindAppointments function manipulating the CalendarView start dates.
var cal = CalendarFolder.Bind(_service, WellKnownFolderName.Calendar);
var cv = new CalendarView(start, end, 1000);
var appointments = new List<Appointment>();
var result = cal.FindAppointments(cv);
appointments.AddRange(result);
while (result.MoreAvailable)
{
cv.StartDate = appointments.Last().Start;
result = cal.FindAppointments(cv);
appointments.AddRange(result);
}
Though I don't know if they come in order. If they don't you might have to use the last envent start date and remove the duplicates.

Related

Google AppMaker: Fetch a MAX value

I am not able to fetch a max value from a number field in AppMaker. The field is filled with unique integers from 1 and up. In SQL I would have asked like this:
SET #tKey = (SELECT MAX(ID) FROM GiftCard);
In AppMaker I have done the following (with a bit help from other contributors in this forum) until now, and it returns tKey = "NaN":
var tKey = google.script.run.MaxID();
function MaxID() {
var ID_START_FROM = 11000;
var lock = LockService.getScriptLock();
lock.waitLock(3000);
var query = app.models.GiftCard.newQuery();
query.sorting.ID._descending();
query.limit = 1;
var records = query.run();
var next_id = records.length > 0 ? records[0].ID : ID_START_FROM;
lock.releaseLock();
return next_id;
}
There is also a maxValue() function in AppMaker. However, it seems not to work in that way I use it. If maxvalue() is better to use, please show :-)
It seems that you are looking in direction of auto incremented fields. The right way to achieve it would be using Cloud SQL database. MySQL will give you more flexibility with configuring your ids:
ALTER TABLE GiftCard AUTO_INCREMENT = 11000;
In case you strongly want to stick to Drive Tables you can try to fix your script as follow:
google.script.run
.withSuccessHandler(function(maxId) {
var tKey = maxId;
})
.withFailureHandler(function(error) {
// TODO: handle error
})
.MaxID();
As a side note I would also recommend to set your ID in onBeforeCreate model event as an extra security layer instead of passing it to client and reading back since it can be modified by malicious user.
You can try using Math.max(). Take into consideration the example below:
function getMax() {
var query = app.models.GiftCard.newQuery();
var allRecords = query.run();
allIds = [];
for( var i=0; i<allRecords.length;i++){
allIds.push(allRecords[i].ID);
}
var maxId = Math.max.apply(null, allIds);
return maxId;
}
Hope it helps!
Thank you for examples! The Math.max returned an undefined value. Since this simple case is a "big" issue, I will solve this in another way. This value is meant as a starting value for a sequence only. An SQL base is better yes!

Java8 stream average of object property in collection

I'm new to Java so if this has already been answered somewhere else then I either don't know enough to search for the correct things or I just couldn't understand the answers.
So the question being:
I have a bunch of objects in a list:
try(Stream<String> logs = Files.lines(Paths.get(args))) {
return logs.map(LogLine::parseLine).collect(Collectors.toList());
}
And this is how the properties are added:
LogLine line = new LogLine();
line.setUri(matcher.group("uri"));
line.setrequestDuration(matcher.group("requestDuration"));
....
How do I sort logs so that I end up with list where objects with same "uri" are displayed only once with average requestDuration.
Example:
object1.uri = 'uri1', object1.requestDuration = 20;
object2.uri = 'uri2', object2.requestDuration = 30;
object3.uri = 'uri1', object3.requestDuration = 50;
Result:
object1.uri = 'uri1', 35;
object2.uri = 'uri2', 30;
Thanks in advance!
Take a look at Collectors.groupingBy and Collectors.averagingDouble. In your case, you could use them as follows:
Map<String, Double> result = logLines.stream()
.collect(Collectors.groupingBy(
LogLine::getUri,
TreeMap::new,
Collectors.averagingDouble(LogLine::getRequestDuration)));
The Collectors.groupingBy method does what you want. It is overloaded, so that you can specify the function that returns the key to group elements by, the factory that creates the returned map (I'm using TreeMap here, because you want the entries ordered by key, in this case the URI), and a downstream collector, which collects the elements that match the key returned by the first parameter.
If you want an Integer instead of a Double value for the averages, consider using Collectors.averagingInt.
This assumes LogLine has getUri() and getRequestDuration() methods.

ParseQuery not taking into consideration Where clause WindowsPhone

I made a new query to select from Article Class with where clause for each item selected. However, it keeps getting the whole list every time although there are selected fields!
Here is my code:
ParseQuery<Article> query = new ParseQuery<Article>();
if (souCategorie.SelectedIndex >= 0)
{
query.WhereEqualTo("idSCategorie", listeSouCategorie.ElementAt(souCategorie.SelectedIndex));
}
if(motcle.Text.Length > 0)
{
query.WhereContains("nom", motcle.Text);
// query.WhereContains("description", motcle.Text);
}
if(distance.Text.Length>0)
if (Convert.ToDouble(distance.Text) > 0)
{
Debug.WriteLine(distance.Text);
ParseGeoPoint geo = new ParseGeoPoint();
geo.Latitude = geoposition.Coordinate.Latitude;
geo.Longitude = geoposition.Coordinate.Longitude;
query.WhereWithinDistance("coordonnees", geo, ParseGeoDistance.FromKilometers(Convert.ToDouble(distance.Text)));
}
IEnumerable<Article> lst = await query.FindAsync();
rechercheResult.DataContext = lst.ToList();
What could possibly be wrong?
I know that queries can do funky stuff when you start trying to use GeoPoint stuff. I would try setting up two queries, one that just queries for objects within a distance, then pass that query into the second query that has the whereEqualTo and whereContains calls.

How to retrieve total view count of large number of pages combined from the GA API

We are interested in the statistics of the different pages combined from the Google Analytics core reporting API. The only way I found to query statistics multiple pages at the same is by creating a filter like so:
ga:pagePath==page?id=a,ga:pagePath==page?id=b,ga:pagePath==page?id=c
And this get escaped inside the filter parameter of the GET query.
However when the GET query gets over 2000 characters I get the following response:
414. That’s an error.
The requested URL /analytics/v3/data/ga... is too large to process. That’s all we know.
Note that just like in the example call the only part that is different per page is a GET parameter in the pagePath, but we have to OR a new filter specifying both the metric (pagePath) as well as the part of the path that is always identical.
Is there any way to specify a large number of different pages to query without hitting this limit in the GET query (I can't find any documentation for doing POST requests)? Or are there alternatives to creating batches of a max of X different pages per query and adding them up on my end?
Instead of using ga:pagePath as part of a filter you should use it as a dimension. You can get up to 10,000 rows per query this way and paginate to get all results. Then parse the results client side to get what you need. Additionally use a filter to scope the results down if possible based on your site structure or page names.
I am sharing a sample code where you can fetch more then 10,000 record data via help of Items PerPage
private void GetDataofPpcInfo(DateTime dtStartDate, DateTime dtEndDate, AnalyticsService gas, List<PpcReportData> lstPpcReportData, string strProfileID)
{
int intStartIndex = 1;
int intIndexCnt = 0;
int intMaxRecords = 10000;
var metrics = "ga:impressions,ga:adClicks,ga:adCost,ga:goalCompletionsAll,ga:CPC,ga:visits";
var r = gas.Data.Ga.Get("ga:" + strProfileID, dtStartDate.ToString("yyyy-MM-dd"), dtEndDate.ToString("yyyy-MM-dd"),
metrics);
r.Dimensions = "ga:campaign,ga:keyword,ga:adGroup,ga:source,ga:isMobile,ga:date";
r.MaxResults = 10000;
r.Filters = "ga:medium==cpc;ga:campaign!=(not set)";
while (true)
{
r.StartIndex = intStartIndex;
var dimensionOneData = r.Fetch();
dimensionOneData.ItemsPerPage = intMaxRecords;
if (dimensionOneData != null && dimensionOneData.Rows != null)
{
var enUS = new CultureInfo("en-US");
intIndexCnt++;
foreach (var lstFirst in dimensionOneData.Rows)
{
var objPPCReportData = new PpcReportData();
objPPCReportData.Campaign = lstFirst[dimensionOneData.ColumnHeaders.IndexOf(dimensionOneData.ColumnHeaders.FirstOrDefault(h => h.Name == "ga:campaign"))];
objPPCReportData.Keywords = lstFirst[dimensionOneData.ColumnHeaders.IndexOf(dimensionOneData.ColumnHeaders.FirstOrDefault(h => h.Name == "ga:keyword"))];
lstPpcReportData.Add(objPPCReportData);
}
intStartIndex = intIndexCnt * intMaxRecords + 1;
}
else break;
}
}
Only one thing is problamatic that your query length shouldn't exceed around 2000 odd characters

Google calendar query returns at most 25 entries

I'm trying to delete all calendar entries from today forward. I run a query then call getEntries() on the query result. getEntries() always returns 25 entries (or less if there are fewer than 25 entries on the calendar). Why aren't all the entries returned? I'm expecting about 80 entries.
As a test, I tried running the query, deleting the 25 entries returned, running the query again, deleting again, etc. This works, but there must be a better way.
Below is the Java code that only runs the query once.
CalendarQuery myQuery = new CalendarQuery(feedUrl);
DateFormat dfGoogle = new SimpleDateFormat("yyyy-MM-dd'T00:00:00'");
Date dt = Calendar.getInstance().getTime();
myQuery.setMinimumStartTime(DateTime.parseDateTime(dfGoogle.format(dt)));
// Make the end time far into the future so we delete everything
myQuery.setMaximumStartTime(DateTime.parseDateTime("2099-12-31T23:59:59"));
// Execute the query and get the response
CalendarEventFeed resultFeed = service.query(myQuery, CalendarEventFeed.class);
// !!! This returns 25 (or less if there are fewer than 25 entries on the calendar) !!!
int test = resultFeed.getEntries().size();
// Delete all the entries returned by the query
for (int j = 0; j < resultFeed.getEntries().size(); j++) {
CalendarEventEntry entry = resultFeed.getEntries().get(j);
entry.delete();
}
PS: I've looked at the Data API Developer's Guide and the Google Data API Javadoc. These sites are okay, but not great. Does anyone know of additional Google API documentation?
You can increase the number of results with myQuery.setMaxResults(). There will be a maximum maximum though, so you can make multiple queries ('paged' results) by varying myQuery.setStartIndex().
http://code.google.com/apis/gdata/javadoc/com/google/gdata/client/Query.html#setMaxResults(int)
http://code.google.com/apis/gdata/javadoc/com/google/gdata/client/Query.html#setStartIndex(int)
Based on the answers from Jim Blackler and Chris Kaminski, I enhanced my code to read the query results in pages. I also do the delete as a batch, which should be faster than doing individual deletions.
I'm providing the Java code here in case it is useful to anyone.
CalendarQuery myQuery = new CalendarQuery(feedUrl);
DateFormat dfGoogle = new SimpleDateFormat("yyyy-MM-dd'T00:00:00'");
Date dt = Calendar.getInstance().getTime();
myQuery.setMinimumStartTime(DateTime.parseDateTime(dfGoogle.format(dt)));
// Make the end time far into the future so we delete everything
myQuery.setMaximumStartTime(DateTime.parseDateTime("2099-12-31T23:59:59"));
// Set the maximum number of results to return for the query.
// Note: A GData server may choose to provide fewer results, but will never provide
// more than the requested maximum.
myQuery.setMaxResults(5000);
int startIndex = 1;
int entriesReturned;
List<CalendarEventEntry> allCalEntries = new ArrayList<CalendarEventEntry>();
CalendarEventFeed resultFeed;
// Run our query as many times as necessary to get all the
// Google calendar entries we want
while (true) {
myQuery.setStartIndex(startIndex);
// Execute the query and get the response
resultFeed = service.query(myQuery, CalendarEventFeed.class);
entriesReturned = resultFeed.getEntries().size();
if (entriesReturned == 0)
// We've hit the end of the list
break;
// Add the returned entries to our local list
allCalEntries.addAll(resultFeed.getEntries());
startIndex = startIndex + entriesReturned;
}
// Delete all the entries as a batch delete
CalendarEventFeed batchRequest = new CalendarEventFeed();
for (int i = 0; i < allCalEntries.size(); i++) {
CalendarEventEntry entry = allCalEntries.get(i);
BatchUtils.setBatchId(entry, Integer.toString(i));
BatchUtils.setBatchOperationType(entry, BatchOperationType.DELETE);
batchRequest.getEntries().add(entry);
}
// Get the batch link URL and send the batch request
Link batchLink = resultFeed.getLink(Link.Rel.FEED_BATCH, Link.Type.ATOM);
CalendarEventFeed batchResponse = service.batch(new URL(batchLink.getHref()), batchRequest);
// Ensure that all the operations were successful
boolean isSuccess = true;
StringBuffer batchFailureMsg = new StringBuffer("These entries in the batch delete failed:");
for (CalendarEventEntry entry : batchResponse.getEntries()) {
String batchId = BatchUtils.getBatchId(entry);
if (!BatchUtils.isSuccess(entry)) {
isSuccess = false;
BatchStatus status = BatchUtils.getBatchStatus(entry);
batchFailureMsg.append("\nID: " + batchId + " Reason: " + status.getReason());
}
}
if (!isSuccess) {
throw new Exception(batchFailureMsg.toString());
}
There is a small quote on the API page
http://code.google.com/apis/calendar/data/1.0/reference.html#Parameters
Note: The max-results query parameter for Calendar is set to 25 by default,
so that you won't receive an entire
calendar feed by accident. If you want
to receive the entire feed, you can
specify a very large number for
max-results.
So to get all events from a google calendar feed, we do this:
google.calendarurl.com/.../basic?max-results=999999
in the API you can also query with setMaxResults=999999
I got here while searching for a Python solution;
Should anyone be stuck in the same way, the important line is the fourth:
query = gdata.calendar.service.CalendarEventQuery(cal, visibility, projection)
query.start_min = start_date
query.start_max = end_date
query.max_results = 1000
Unfortunately, Google is going to limit the maximum number of queries you can retrieve. This is so as to keep the query governor in their guidelines (HTTP requests not allowed to take more than 30 seconds, for example). They've built their whole architecture around this, so you might as well build the logic as you have.

Resources