I currently am trying to use TransformBlocks to make my code run faster. Instead, I find that I have achieved essentially no parallelization:
As you can see, there is quite a bit of dead space, with very little I/O or other issues preventing things from running in parallel (note: all the green blocks are the main thread).
The basic structure of the calling code is as follows:
var options = new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 8 };
var download = new TransformBlock<string, Tuple<string, string>>(s => sendAndReciveRequest(s), options);
var process = new TransformBlock<Tuple<string, string, TransformBlock<string, Tuple<string, string>>>, List<string>>(s => Helpers.ParseKBDL(s), options);
var toObjects = new TransformBlock<List<string>, List<Food>>(list => toFood(list), options);
for (char char1 = 'a'; char1 < 'z' + 1; char1++)
download.Post(char1.ToString());
while ((download.InputCount != 0 || download.OutputCount != 0 || process.InputCount != 0) || (Form1.downloadCount != Form1.processCount))
{
if (download.OutputCount == 0 && download.InputCount == 0)
{
continue;
}
var res = download.Receive();
process.Post(new Tuple<string, string, TransformBlock<string, Tuple<string, string>>>(res.Item1, res.Item2, download));
}
Note: The reason for the messy checks and tuples is because occasionally process needs to add more to the download block. I am very open to changing how this is organized.
So my question: Why is this code achieving no speedup? How can I restructure it so that I do get a speedup?
Related
This InDesign Javascript iterates over textStyleRanges and converts text with a few specific appliedFont's and later assigns a new appliedFont:-
var textStyleRanges = [];
for (var j = app.activeDocument.stories.length-1; j >= 0 ; j--)
for (var k = app.activeDocument.stories.item(j).textStyleRanges.length-1; k >= 0; k--)
textStyleRanges.push(app.activeDocument.stories.item(j).textStyleRanges.item(k));
for (var i = textStyleRanges.length-1; i >= 0; i--) {
var myText = textStyleRanges[i];
var converted = C2Unic(myText.contents, myText.appliedFont.fontFamily);
if (myText.contents != converted)
myText.contents = converted;
if (myText.appliedFont.fontFamily == 'Chanakya'
|| myText.appliedFont.fontFamily == 'DevLys 010'
|| myText.appliedFont.fontFamily == 'Walkman-Chanakya-905') {
myText.appliedFont = app.fonts.item("Utsaah");
myText.composer="Adobe World-Ready Paragraph Composer";
}
}
But there are always some ranges where this doesn't happen. I tried iterating in the forward direction OR in the backward direction OR putting the elements in an array before conversion OR updating the appliedFont in the same iteration OR updating it a different one. Some ranges are still not converted completely.
I am doing this to convert the Devanagari text encoded in glyph based non-Unicode encoding to Unicode. Some of this involves repositioning vowel signs etc and changing the code to work with find/replace mechanism may be possible but is a lot of rework.
What is happening?
See also: http://cssdk.s3-website-us-east-1.amazonaws.com/sdk/1.0/docs/WebHelp/app_notes/indesign_text_frames.htm#Finding_and_changing_text
Sample here: https://www.dropbox.com/sh/7y10i6cyx5m5k3c/AAB74PXtavO5_0dD4_6sNn8ka?dl=0
This is untested since I'm not able to test against your document, but try using getElements() like below:
var doc = app.activeDocument;
var stories = doc.stories;
var textStyleRanges = stories.everyItem().textStyleRanges.everyItem().getElements();
for (var i = textStyleRanges.length-1; i >= 0; i--) {
var myText = textStyleRanges[i];
var converted = C2Unic(myText.contents, myText.appliedFont.fontFamily);
if (myText.contents != converted)
myText.contents = converted;
if (myText.appliedFont.fontFamily == 'Chanakya'
|| myText.appliedFont.fontFamily == 'DevLys 010'
|| myText.appliedFont.fontFamily == 'Walkman-Chanakya-905') {
myText.appliedFont = app.fonts.item("Utsaah");
myText.composer="Adobe World-Ready Paragraph Composer";
}
}
A valid approach is to use hyperlink text sources as they stick to the genuine text object. Then you can edit those source texts even if they were actually moved elsewhere in the flow.
//Main routine
var main = function() {
//VARS
var doc = app.properties.activeDocument,
fgp = app.findGrepPreferences.properties,
cgp = app.changeGrepPreferences.properties,
fcgo = app.findChangeGrepOptions.properties,
text, str,
found = [], srcs = [], n = 0;
//Exit if no documents
if ( !doc ) return;
app.findChangeGrepOptions = app.findGrepPreferences = app.changeGrepPreferences = null;
//Settings props
app.findChangeGrepOptions.properties = {
includeHiddenLayers:true,
includeLockedLayersForFind:true,
includeLockedStoriesForFind:true,
includeMasterPages:true,
}
app.findGrepPreferences.properties = {
findWhat:"\\w",
}
//Finding text instances
found = doc.findGrep();
n = found.length;
//Looping through instances and adding hyperlink text sources
//That's all we do at this stage
while ( n-- ) {
srcs.push ( doc.hyperlinkTextSources.add(found[n] ) );
}
//Then we edit the stored hyperlinks text sources 's texts objects contents
n = srcs.length;
while ( n-- ) {
text = srcs[n].sourceText;
str = text.contents;
text.contents = str+str+str+str;
}
//Eventually we remove the added hyperlinks text sources
n = srcs.length;
while ( n-- ) srcs[n].remove();
//And reset initial properties
app.findGrepPreferences.properties = fgp;
app.changeGrepPreferences.properties = cgp;
app.findChangeGrepOptions.properties =fcgo;
}
//Running script in a easily cancelable mode
var u;
app.doScript ( "main()",u,u,UndoModes.ENTIRE_SCRIPT, "The Script" );
I am trying to write some logic to determine if all values of a certain property of an object in a collection are numeric and greater than zero. I can easily write this using ForEach but I'd like to do it using Linq to Object. I tried this:
var result = entity.Reports.Any(
x =>
x.QuestionBlock == _question.QuestionBlock
&& (!string.IsNullOrEmpty(x.Data)) && Int32.TryParse(x.Data, out tempVal)
&& Int32.Parse(x.Data) > 0);
It does not work correctly. I also tried this, hoping that the TryParse() on Int32 will return false the first time it encounter a string that cannot be parsed into an int. But it appears the out param will contain the first value string value that can be parsed into an int.
var result = entity.GranteeReportDataModels.Any(
x =>
x.QuestionBlock == _question.QuestionBlock
&& (!string.IsNullOrEmpty(x.Data)) && Int32.TryParse(x.Data, out tempVal));
Any help is greatly appreciated!
If you want to test if "all" values meet a condition, you should use the All extension method off IEnumerable<T>, not Any. I would write it like this:
var result = entity.Reports.All(x =>
{
int result = 0;
return int.TryParse(x.Data, out result) && result > 0;
});
I don't believe you need to test for an null or empty string, because int.TryPrase will return false if you pass in a null or empty string.
var allDataIsNatural = entity.Reports.All(r =>
{
int i;
if (!int.TryParse(r.Data, out i))
{
return false;
}
return i > 0;
});
Any will return when the first row is true but, you clearly say you would like to check them all.
You can use this extension which tries to parse a string to int and returns a int?:
public static int? TryGetInt(this string item)
{
int i;
bool success = int.TryParse(item, out i);
return success ? (int?)i : (int?)null;
}
Then this query works:
bool all = entity.Reports.All(x => {
if(x.QuestionBlock != _question.QuestionBlockint)
return false;
int? data = x.Data.TryGetInt();
return data.HasValue && data.Value > 0;
});
or more readable (a little bit less efficient):
bool all = entityReports
.All(x => x.Data.TryGetInt().HasValue && x.Data.TryGetInt() > 0
&& x.QuestionBlock == _question.QuestionBlockint);
This approach avoids using a local variable as out parameter which is an undocumented behaviour in Linq-To-Objects and might stop working in future. It's also more readable.
I wrote a script that gets a rows data from a spreadsheet and loops through them, calling a function to send an SMS if the rows' data meets certain conditions (having a phone number and not having already been sent for example).
However after adding about 600 rows, the script execution time exceeds it's limit, that seems to be 5 minutes according to my research. I'm using JavaScript objects to read data and a for loop to iterate through the rows.
Can anyone tel me if it is possible to make it faster? I'm very new to programming but this seems such a light task for all this computing power that I can't understand why it takes so long
Thanks in advance!
Here's the code of the function I'm using:
// Will send SMS on the currently active sheet
function sendSms() {
// Use the send sms menu to trigger reconcile
var user = ScriptProperties.getProperty(PROPERTY_USER_RECONCILE);
if (user == null)
reconcileUser();
// The sheets
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Registo");
var settingsSheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Settings");
// Fetch values for each row in the Range.
var startRow = 2;
var apiKey = settingsSheet.getRange("B2").getValue();
var apiSecret = settingsSheet.getRange("B3").getValue();
var prefix = settingsSheet.getRange("B4").getValue();
var numRows = sheet.getMaxRows() - 1;
var numCols = 16;
var statusColNum = 15; // IMPT: To keep track status in col 15
var dataRange = sheet.getRange(startRow, 1, numRows, numCols);
// Make sure there is API key and secret
if (apiKey == "" || apiSecret == "") {
Browser.msgBox("You MUST fill in your API key and secret in Settings sheet first!");
return;
}
// Create one JavaScript object per row of data.
var objects = getRowsData(sheet, dataRange);
var totalSent = 0;
for (var i = 0; i < objects.length; ++i) {
// Get a row object
var rowData = objects[i];
var ss = SpreadsheetApp.getActiveSpreadsheet();
var templateSheet = ss.getSheetByName("SMS Modelo");
var template = templateSheet.getRange("A1").getValue();
// jump loop iteration if conditions not satisied
if (rowData.resolv == "x" || rowData.contactoUtente == null || rowData.contactoUtente == "" || rowData.reserv == null || rowData.reserv == "" || rowData.cont == "x" || rowData.sms !== null) continue;
var message = fillInTemplateFromObject(template, rowData);
var senderName = "Farm Cunha"
var mobile = rowData.contactoUtente;
// Send via Nexmo API
var response = nexmoSendSms(apiKey, apiSecret,"+351" + mobile, message, senderName);
if (response.getResponseCode() == 200) {
var object = JSON.parse(response.getContentText());
if (object.messages[0]['status'] == "0") {
// Set to QUEUE status - We assumed SENT, as we don't handle delivery status.
//sheet.getRange(startRow + i, statusColNum).setValue(STATUS_QUEUE);
sheet.getRange(startRow + i, statusColNum).setValue(STATUS_SENT);
// Set the reference id
sheet.getRange(startRow + i, 19).setValue(object.messages[0]['message-id']);
// sheet.getRange(startRow + i, statusColNum+3).setValue(new Date()); linha pode ser activada para fazer timestamp do envio
totalSent++;
}
else {
// If status is not 0, then it is an error.
// Set status to the error text
sheet.getRange(startRow + i, statusColNum).setValue(object.messages[0]['error-text']);
}
}
else {
// Non 200 OK response
sheet.getRange(startRow + i, statusColNum).setValue("Error Response Code: " + response.getResponseCode);
}
SpreadsheetApp.flush();
// Need a wait. Need to throttle else will have "Route Busy" error.
Utilities.sleep(2000);
}
// Update total sent
var lastTotalSent = parseInt(ScriptProperties.getProperty(PROPERTY_SMS_SENT_FOR_RECONCILE));
if (isNaN(lastTotalSent)) lastTotalSent = 0;
ScriptProperties.setProperty(PROPERTY_SMS_SENT_FOR_RECONCILE, (lastTotalSent + totalSent).toString());
Logger.log("Last sent: " + lastTotalSent + " now sent: " + totalSent);
reconcileApp();
}
You have a few things in your loop that are too time consuming : spreadsheet readings and API calls + 2 seconds sleep !.
I would obviously advise you to take these out of the loop (specially the template sheet reading that is always the same!). A possible solution would be to check the conditions from the row objects and to save the valid entries in an array... THEN iterate in this array to call the API.
If this is still too long then proceed by small batches, saving the end position of the partial iteration in scriptproperties and using a timer trigger that will continue the process every 5 minutes until it is completed (and kill the trigger at the end).
There are a few example of this kind of "mechanics" on this forum, one recent example I suggested is here (it's more like a draft but the idea is there)
Ok, I've solved it by taking these 3 lines out of the loop as Serge (thanks) had told me to:
var ss = SpreadsheetApp.getActiveSpreadsheet();
var templateSheet = ss.getSheetByName("SMS Modelo");
var template = templateSheet.getRange("A1").getValue();
It's so simple that I don't know how I was not seeing that.
This simple change made the script much faster. For example, going through 600 rows would take more than 5 minutes. Now, more than 5000 rows only take seconds.
Im getting the error:
'new GeoCoordinate(MyEntity.Lat, MyEntity.Lon).GetDistanceTo(14.4416616666667, 5.71794583333333)' is not supported in a 'Where' Mobile Services query expression.
From the following query:
GeoCoordinate mypos = new GeoCoordinate(MainPage.myEntity.Lat, MainPage.myEntity.Lon);
var items = await App.MobileService
.GetTable<MyEntity>()
.Where(MyEntity => MyEntity.Id != MainPage.myEntity.Id && MyEntity.IsSelected == MainPage.myEntity.IsSelected
&& (MyEntity.Lat != 0 && MyEntity.Lon != 0)
&& new GeoCoordinate(MyEntity.Lat, MyEntity.Lon).GetDistanceTo(mypos) <= 5000)
.Take(20)
.ToListAsync();
I'm trying to get 20 entities from a table that is not me and within 5 km of my position.
One thing you need to be aware is that the line expression is not executed locally. It's instead translated into a query string which is sent to the server - otherwise you'd be downloading a lot more data than you need (which you still can do, by the way). Given that, there's no way to translate the method GeoCoordinate.GetDistanceTo into something which can be sent over the wire.
There are some operations which are supported in the server, such as arithmetic (+, -, *, /), and you could potentially implement something similar to what you need. Something like the code below should work (haven't tried it yet). Notice that you'll need to do the conversion between distance in coordinates (lat / lon) and whatever unit is returned by the GetDistanceTo method.
GeoCoordinate mypos = new GeoCoordinate(MainPage.myEntity.Lat, MainPage.myEntity.Lon);
double maxDistance = 1;
var items = await App.MobileService
.GetTable<MyEntity>()
.Where(e => e.Id != MainPage.myEntity.Id &&
e.IsSelected == MainPage.myEntity.IsSelected &&
(e.Lat != 0 && e.Lon != 0) &&
( ((e.Lat - mypos.Lat) * (e.Lat - mypos.Lat) +
(e.Lon - mypos.Lon) * (e.Lon - mypos.Lon)) < maxDistance )
.Take(20)
.ToListAsync();
Or, another alternative as I mentioned before would be to simply not to filter by the distance on that call, and filter it afterwards. Something like the code below:
GeoCoordinate mypos = new GeoCoordinate(MainPage.myEntity.Lat, MainPage.myEntity.Lon);
List<MyEntity> items = new List<MyEntity>();
int skip = 0;
while (items.Count < 20) {
var temp = await App.MobileService
.GetTable<MyEntity>()
.Where(e => e.Id != MainPage.myEntity.Id &&
e.IsSelected == MainPage.myEntity.IsSelected &&
(e.Lat != 0 && e.Lon != 0)
.Skip(skip)
.Take(20)
.ToListAsync();
if (temp.Count == 0) break;
foreach (var item in temp)
{
if (new GeoCoordinate(item.Lat, item.Lon).GetDistanceTo(mypos) <= 5000)
{
items.Add(item);
}
}
}
How can we do validation for percentage numbers in textbox .
I need to validate these type of data
Ex: 12-3, 33.44a, 44. , a3.56, 123
thanks in advance
sri
''''Add textbox'''''
<asp:TextBox ID="PERCENTAGE" runat="server"
onkeypress="return ispercentage(this, event, true, false);"
MaxLength="18" size="17"></asp:TextBox>
'''''Copy below function as it is and paste in tag..'''''''
<script type="text/javascript">
function ispercentage(obj, e, allowDecimal, allowNegative)
{
var key;
var isCtrl = false;
var keychar;
var reg;
if (window.event)
{
key = e.keyCode;
isCtrl = window.event.ctrlKey
}
else if (e.which)
{
key = e.which;
isCtrl = e.ctrlKey;
}
if (isNaN(key)) return true;
keychar = String.fromCharCode(key);
// check for backspace or delete, or if Ctrl was pressed
if (key == 8 || isCtrl)
{
return true;
}
ctemp = obj.value;
var index = ctemp.indexOf(".");
var length = ctemp.length;
ctemp = ctemp.substring(index, length);
if (index < 0 && length > 1 && keychar != '.' && keychar != '0')
{
obj.focus();
return false;
}
if (ctemp.length > 2)
{
obj.focus();
return false;
}
if (keychar == '0' && length >= 2 && keychar != '.' && ctemp != '10') {
obj.focus();
return false;
}
reg = /\d/;
var isFirstN = allowNegative ? keychar == '-' && obj.value.indexOf('-') == -1 : false;
var isFirstD = allowDecimal ? keychar == '.' && obj.value.indexOf('.') == -1 : false;
return isFirstN || isFirstD || reg.test(keychar);
}
</script>
You can further optimize this expression. Currently its working for all given patterns.
^\d*[aA]?[\-.]?\d*[aA]?[\-.]?\d*$
If you're talking about checking that a given text is a valid percentage, you can do one of a few things.
validate it with a regex like ^[0-9]+\.?[0-9]*$ then just convert that to a floating point value and check it's between 0 and 100 (that particular regex requires a zero before the decimal for values less than one but you can adapt it to handle otherwise).
convert it to a float using a method that raises an exception on invalid data (rather than just stopping at the first bad character.
use a convoluted regex which checks for valid entries without having to convert to a float.
just run through the text character by character counting numerics (a), decimal points (b) and non-numerics (c). Provided a is at least one, b is at most one, and c is zero, then convert to a float.
I have no idea whether your environment support any of those options since you haven't actually specified what it is :-)
However, my preference is to go for option 1, 2, 4 and 3 (in that order) since I'm not a big fan of convoluted regexes. I tend to think that they do more harm than good when thet become to complex to understand in less than three seconds.
Finally i tried a simple validation and works good :-(
function validate(){
var str = document.getElementById('percentage').value;
if(isNaN(str))
{
//alert("value out of range or too much decimal");
}
else if(str > 100)
{
//alert("value exceeded");
}
else if(str < 0){
//alert("value not valid");
}
}