I have a complex azure function which creates/updates approx 400 records in a table in dataverse while checking several conditions
I am using ExecuteTransactionRequest to create/update the records and it takes around 15 seconds to perform the logic. Is there a way I can make it faster, introduce parallel processing along with executetransation?
I'm not sure if my answer is applicable in your scenario, but I'll mention it for reference.
WebApi accepts a batch requests which saves you the latency time of sequential and concurrent requests. One request with multiple internal requests is sent once to the api and will run on their servers at once and get back the results.
That is if you are using WebApi and not using connectors to communicate with CDS.
If so, please refer to the following example to post 2 currencies and get all:
--batch_ABC123
Content-Type: multipart/mixed; boundary=changeset_ABC111
--changeset_ABC111
Content-Type: application/http
Content-Transfer-Encoding: binary
POST YOUR-RESOURCE_URL/api/data/v9.1/transactioncurrencies HTTP/1.1
Content-ID: 1
Accept: application/json;
Content-Type: application/json;type=entry
{"currencyname": "Dansk Krona1","currencysymbol": "$","isocurrencycode": "PPV","exchangerate": 0.90}
--changeset_ABC111
Content-Type: application/http
Content-Transfer-Encoding: binary
POST YOUR-RESOURCE_URL/api/data/v9.1/transactioncurrencies HTTP/1.1
Content-ID: 2
Accept: application/json;
Content-Type: application/json;type=entry
{"currencyname": "Dansk Krona2","currencysymbol": "$","isocurrencycode": "UPV","exchangerate": 0.90}
--changeset_ABC111--
--batch_ABC123
Content-Type: application/http
Content-Transfer-Encoding:binary
GET YOUR-RESOURCE_URL/api/data/v9.1/transactioncurrencies HTTP/1.1
Accept: application/json
--batch_ABC123--
Related
During these days, I am obsessed by a big problem —— how to get the vector tile by Geoserver? I have looking for many docs and blogs, and there are some useful resource for the problem, the operation is very easy if step by step:
http://suite.opengeo.org/docs/latest/dataadmin/vectortiles/index.html;
But after these operations, I still don't get success, i wonder to konw the reason, if you have some idea, please sharing with me, thanks for your kind~
ps:there is some message from console,maybe it's useful:
about gwc seeding:
10 一月 10:27:45 INFO [seed.SeedTask] - GWC Seeder Thread-4 completed (re)seeding layer cn:roads after 192 tiles and 2 seconds.
10 一月 10:27:45 INFO [seed.SeedTask] - GWC Seeder Thread-3 completed (re)seeding layer cn:roads after 80 tiles and 2 seconds.
10 一月 10:27:45 INFO [seed.GWCTask] - Thread group finished SEED task after 2.578 seconds
I don't have enough reputation, so I copy the message to here instead of sending a image, sorry
//*************************************************************************
I have already added the vector-tiles plugin,
and there are some errors when I request the tiles:
http://localhost:8080/geoserver/gwc/service/tms/1.0.0/cite:DLZXX_2011_PL_10000#EPSG%3A4326#geojson/11/1673/891.geojson 404 (Not Found)
//the code to get vector tiles
var vectorTile = new ol.layer.VectorTile({
//get source
source: new ol.source.VectorTile({
format:new ol.format.GeoJSON(),
tileGrid: ol.tilegrid.createXYZ({
maxZoom:14
}),
tilePixelRatio:1,
url:'/geoserver/gwc/service/tms/1.0.0/'+layerName
+'#EPSG%3A4326#geojson/{z}/{x}/{y}.geojson'
}),
style:initStyle
});
map.addLayer(vectorTile);
Header of each Tile GET request
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encodin: gzip, deflate
Accept-Language: zh-CN,zh;q=0.8,en-US;q=0.5,en;q=0.3
Cookie: JSESSIONID=9679B11893A04A0ACE21D49D93AB7CDC
Host: localhost:8080
Referer: http://localhost:8080/23DPlatform/index.jsp
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0
Header of response
Content-Disposition :inline; filename=geoserver-dispatch.text
Content-Encoding: gzip
Content-Type: text/plain
Date: Tue, 10 Jan 2017 13:01:09 GMT
Server: Apache-Coyote/1.1
Transfer-Encoding:chunked
//***************************************************************
Maybe you can read this question, there are some similar
Ionic / Leaflet - Can't get Tiles 404 Not Found (from cache)
I have already got the vector tiles by geoserver and openLayers3.
1)The vector tiles in geoserver(GeoWebcache) are created dynamically, so you
don't need to click the seeding button on GeoWebcache admin page;
2)The projection of your data is a very important param, the default projection in OL3 is EPSG:3857. so you need to make sure the projections(layer,source,tilegrid) are same;
PS:I had written down the process on my blog:
http://www.cnblogs.com/escage/p/6387529.html
sorry, it's Chinese blog, if you have any question about these topic, you can
contact me with email:
ericliuneverlost#163.com
My application load regulary the number of viewers watching a livestream on dailymotion.
I use the dailymotion API with the field "audience" to do that.
But the server is sending to me a really really old cached version of the precious JSON file.
For example, a streamer is live since 2 hours, but the API is sending this to me :
curl https://api.dailymotion.com/video/x25eyo8?fields=audience
{"audience":0}
If i add another field just a few seconds later, just to see :
curl https://api.dailymotion.com/video/x25eyo8?fields=audience,onair
{"audience":1177, onair: true}
And if I re-send the first request
curl https://api.dailymotion.com/video/x25eyo8?fields=audience
{"audience":0}
More interesting, the headers sent by the server show this :
curl -I https://api.dailymotion.com/video/x25eyo8?fields=audience
HTTP/1.1 200 OK
Server: DMS/1.0.42
X-Dm-Api-Object: video
X-DM-BackNode: web-011.adm.dailymotion.com:80
Cache-Control: public, max-age=10, stale-if-error=900
X-Dm-Api-Method: info
Content-Type: application/json; charset=UTF-8
X-DM-LB: 195.8.215.130
Access-Control-Allow-Origin: *
X-DM-BackNode-Response-Time: 47
Etag: W/"8xO_txIM6arAYYIALcRUgg"
X-Robots-Tag: noindex
Last-Modified: Fri, 06 Nov 2015 20:20:31 GMT
Access-Control-Allow-Headers: Content-Type, Authorization
Access-Control-Allow-Methods: GET, POST, DELETE
X-Dm-Page: fr.rest.rest_api
Via: 1.1 varnish
Fastly-Debug-Digest: 1d9daef237214a02cb79d06c44fdaa26329d5bd16c9afda535f3f9b104438b84
Content-Length: 17
Accept-Ranges: bytes
Date: Mon, 30 Nov 2015 01:16:59 GMT
Via: 1.1 varnish
Age: 116
Connection: keep-alive
X-Served-By: cache-fra1225-FRA, cache-lhr6323-LHR
X-Cache: HIT, MISS
X-Cache-Hits: 1, 0
Vary: X-DM-EC-Geo-Country, Accept-Encoding
Age : 116 with a max-age of 10 ?
And even when i receive an "up to date" version (age < 10), the file still contains 0 viewers whereas the stream is online and 1000+ viewers are watching.
Now there are 2 questions :
Why this happens ?
Can I force a non-cached version ?
Thanks for your help.
EDIT :
It looks like it's the same problem when you watch a stream on http://games.dailymotion.com/. For all channels, numbers of viewers are correct inside the player but for some ones, the number displayed under it is not (most of the time, it indicate 0)
Example of wrong number of viewers
I'm trying to get sessions from a startDate to a EndDate, but every time i try return always a empty record
HTTP/1.1 200 OK
Content-length: 3
X-xss-protection: 1; mode=block
Content-location: https://www.googleapis.com/fitness/v1/users/me/sessions?startTime=2015-11-20T00:00:00.00Z&endTime=2015-11-30T23:59:59.99Z
X-content-type-options: nosniff
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Vary: Origin,X-Origin
Server: GSE
Etag: ""
Pragma: no-cache
Cache-control: no-cache, no-store, max-age=0, must-revalidate
Date: Mon, 30 Nov 2015 16:49:52 GMT
X-frame-options: SAMEORIGIN
Content-type: application/json; charset=UTF-8
{}
Heres the URL :https://www.googleapis.com/fitness/v1/users/me/sessions?startTime=2015-11-20T00:00:00.00Z&endTime=2015-11-30T23:59:59.99Z
What is wrong?
I have data in that range and for example if i execute the dataset it return data (https://www.googleapis.com/fitness/v1/users/me/dataSources/derived:com.google.step_count.delta:com.google.android.gms:estimated_steps/datasets/XXXXXXX000000000000-XXXXXX000000000000 )
Sessions represent a time interval during which users perform a fitness activity.
It will only appear to the sessions resource items that have been converted to a session.
Normally, automatic tracks (the ones that you don't do anything to track) only become sessions when you put a name on it (you can try it using Google Fit App, selecting one of the auto track activities, put a name of it, like "lunch walk"). It takes a while to become available in the API, in my case, it took one hour to become available to the sessions list.
Another aproach to see itens on the session list is to create a session : create a new activity on Google Fit, with start and stop actions.
You may find more information about the session concept here: https://developers.google.com/fit/android/using-sessions
I'm new in Performance Tests, I started using JMeter and creating my own scripts. I am doing stress performance test on an API, until now POST, GET, PATCH all were working, but I stopped at PUT method. I need to send a file using PUT method, in POSTMAN is working (in body I am using file type with the selected file, in the header multipart/form-data).
I tried to put the file path in "Send files with the request", Parameter Name: file, MIME type: form-data, Content encoding: utf-8.
In the request it doesn't give me the file.
PUT http://10.111.30.12/api/tasks/2
PUT data:
[no cookies]
Request Headers:
Connection: keep-alive
X-AuthToken:
MjEzNUZFMEMxMzFEQTVBMUMxQzYxMDU0MjE0OEFFRTJDRjU0ODQ0QkRCNDUyQkQ0QTgxREU0M0Y5MDQwMTk1RDJGMEE2RDNERTIxNjFBRjE3MEQ0QTNFQzM1OTVBRjMyQUI0MkJFN0MwMjYxMkFDRTBFMTQyMzYyNjYwMkREMTU0RkMxQTlBMjJDOUJFQkMwRjEwNDdFOTEwNjgyRDAwMTVBOTlEQ0ExQ0FFQTBGQjA2MEVDRUNFQjgzOEQ1MTA4ODVGOUYxMDhBQUM0RTc5N0JDQTA2RkYyNjYxQURGODE3NUM0MDlFN0RENEM0MTc0Nzc4MzczRjNDQ0VDQzM3Q0Y2QzU4REE2ODg2QzAyNEE1MzY0QThDN0IwMjhEMjdE
Content-Type: multipart/form-data
Content-Length: 0
Host: 10.111.30.12
Proxy-Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.2.6 (java 1.5)"
The sampler result:
Thread Name: API Thread Group 1-1
Sample Start: 2015-09-21 15:33:53 EEST
Load time: 22
Connect Time: 0
Latency: 22
Size in bytes: 202
Headers size in bytes: 202
Body size in bytes: 0
Sample Count: 1
Error Count: 1
Response code: 415
Response message: Unsupported Media Type
Also tried to put the file in body data as: "file: C:\apache-jmeter-2.13\bin\API Performance Test\file.txt" but now I am getting 400 bad request.
Please if anyone got any idea on how to do this, tell me too.
Since you're testing an API my expectation is that you need to add a HTTP Header Manager to send Content-Type header with the value of application/json.
The best way to get to the bottom of the issue is using a sniffer tool like Wireshark to compare what's being sent by Postman and JMeter and make sure that there are no differences.
I'm having a problem getting the PayPal Payflow Gateway accept my HTTPS POST with non US-ASCII characters in it. Not matter what I post with these special characters, it seems to want to accept only US-ASCII encoded bytes. If I send UTF-8 encoded bytes, it still works, but fails to parse some of the request NVP values. I know it is possible because I've posted with another developer's test page to my account (http://ccaples.com/ NVP Quick Test) and that post seems to preserve the special characters.
Here's an example. I've tried with MANY different Content-Type headers and Accept/Accept-Charset values including specifying "; charset=UTF-8" or "; charset=utf-8" in the Content-Type after the text/namevalue.
POST https://pilot-payflowpro.paypal.com/ HTTP/1.1
Content-Type: text/namevalue
User-Agent: KLMS Payflow API for Java
X-VPS-Request-ID: 36A4ED051A8B492ABF70E6BE51CB13D5
X-VPS-CLIENT-TIMEOUT: 20
Connection: close
X-VPS-VIT-INTEGRATION-PRODUCT: KLMS Payflow API for Java
X-VPS-VIT-INTEGRATION-VERSION: 2.0.008
X-VPS-VIT-PROXY: Y
X-VPS-VIT-RUNTIME-VERSION: 20.45-b01
X-VPS-VIT-OS-ARCHITECTURE: amd64
X-VPS-VIT-OS-VERSION: 6.1
X-VPS-VIT-OS-NAME: Windows 7
Cache-Control: no-cache
Pragma: no-cache
Host: pilot-payflowpro.paypal.com
Content-Length: 694
USER[8]=XXXXXXXX&VENDOR[13]=XXXXXXXXXXXXX&PARTNER[6]=PayPal&PWD[8]=XXXXXXXX&VERBOSITY[4]=HIGH&BILLTOEMAIL[28]=XXXXXXXX#kineticlearning.com&TRXTYPE[1]=A&TENDER[1]=C&ACCT[16]=4111111111111111&EXPDATE[4]=1117&CVV2[3]=123&BILLTOFIRSTNAME[4]=Josè&BILLTOLASTNAME[7]=Elkjærd&BILLTOSTREET[14]=123 Elm Street&BILLTOCITY[9]=Elm Creek&BILLTOSTATE[2]=VA&BILLTOZIP[5]=22203&BILLTOCOUNTRY[2]=US&BILLTOPHONENUM[12]=763-221-5593&CUSTBROWSER[108]=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 Safari/537.36&CUSTHOSTNAME[12]=192.168.1.10&CUSTIP[12]=192.168.1.10&AMT[6]=600.00&CURRENCY[3]=USD&COMMENT1[26]=Law Enforcement Curriculum&COMMENT2[16]=Standard Version
HTTP/1.1 200 OK
Connection: close
Server: VPS-3.033.00
X-VPS-Request-ID: 36A4ED051A8B492ABF70E6BE51CB13D5
Date: Sun, 25 Jan 2015 17:45:05 GMT
Content-type: text/namevalue
Content-length: 199
RESULT=104&PNREF=B70P7B6EBAE7&RESPMSG=Timeout waiting for Processor response&TRANSTIME=2015-01-25 09:44:47&BILLTOFIRSTNAME=Jos &BILLTOLASTNAME=NotProvided&AMT=600.00&ACCT=1111&EXPDATE=1117&CARDTYPE=0
Ignore the 104 timeout for now. PayPal is working on that. What you can see is that the BILLTOFIRSTNAME is Jos(weird character that's not è) and the æ character in BILLTOLASTNAME must have caused the server's NVP parsing algorithm to have completely failed to parse because it says NotProvided.
Here's the code that converts the Java string in NVP formatted form to bytes:
final byte[] requestBytes = requestString.getBytes(Charsets.UTF_8);
Any idea what character encoding the server-side NVP parser is looking for by default, or how to tell it what character encoding I'm sending the body of the POST in?
I have found the solution. It appears that it is not supported by the Payflow Gateway Docs. The docs seem to be out of date and need updating. I worked with PayPal support and it does appear that the Payflow Gateway supports UTF-8. Here is the trick: either you need to NOT specify the length parameter in the name/value pairs you submit (i.e. BILLTOFIRSTNAME=Josè) or you have to specify in the length parameter as not the length in CHARACTERS (i.e. not value.length()), but the actual length in BYTES of the UTF-8 encoded byte stream for the value (i.e. value.getBytes("UTF-8").length). This transaction worked, stored the UTF-8 characters and displayed properly in the PayPal Manager, and the proper value was returned in the name/value pairs of the Payflow Gateway response. As you can see, the difference from above is that the BILLTOFIRSTNAME and BILLTOLASTNAME length tags are now larger and specify the length in bytes instead of the length in characters. Josè [4 characters but 5 bytes: 4A 6F 73 C3 A8] and Elkjærd & Grün [14 characters but 16 bytes: 45 6C 6B 6A C3 A6 72 64 20 26 20 47 72 C3 BC 6E].
POST https://pilot-payflowpro.paypal.com/ HTTP/1.1
Content-Type: text/namevalue
Connection: close
User-Agent: KLMS Payflow for Java/2.0.008
X-VPS-Request-ID: B8DABD9BDFE246EC909B4CF741030133
X-VPS-CLIENT-TIMEOUT: 20
X-VPS-VIT-INTEGRATION-PRODUCT: KLMS Payflow for Java
X-VPS-VIT-INTEGRATION-VERSION: 2.0.008
X-VPS-VIT-PROXY: Y
X-VPS-VIT-RUNTIME-VERSION: 20.45-b01
X-VPS-VIT-OS-ARCHITECTURE: amd64
X-VPS-VIT-OS-VERSION: 6.1
X-VPS-VIT-OS-NAME: Windows 7
Cache-Control: no-cache
Pragma: no-cache
Host: pilot-payflowpro.paypal.com
Content-Length: 771
USER[8]=XXXXXXXX&VENDOR[13]=XXXXXXXXXXXXX&PARTNER[6]=PayPal&PWD[8]=XXXXXXXX&VERBOSITY[4]=HIGH&BILLTOEMAIL[28]=XXXXXX#kineticlearning.com&TRXTYPE[1]=A&TENDER[1]=C&ACCT[16]=4111111111111111&EXPDATE[4]=0216&CVV2[3]=123&BILLTOFIRSTNAME[5]=Josè&BILLTOLASTNAME[16]=Elkjærd & Grün&BILLTOSTREET[14]=123 Elm Street&BILLTOSTREET2[10]=Elm & Vine&BILLTOCITY[9]=Elm Creek&BILLTOSTATE[2]=VA&BILLTOZIP[5]=12345&BILLTOCOUNTRY[2]=US&BILLTOPHONENUM[12]=763-221-5593&CUSTBROWSER[108]=Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 Safari/537.36&CUSTHOSTNAME[12]=192.168.1.10&CUSTIP[12]=192.168.1.10&AMT[6]=600.00&CURRENCY[3]=USD&COMMENT1[45]=Victim Advocate Curriculum (Standard Version)&COMMENT2[36]=Purchased for J.C. Hamlin (jchamlin)
HTTP/1.1 200 OK
Connection: close
Server: VPS-3.033.00
X-VPS-Request-ID: B8DABD9BDFE246EC909B4CF741030133
Date: Wed, 28 Jan 2015 01:45:29 GMT
Content-type: text/namevalue
Content-length: 209
RESULT=104&PNREF=B10P7D2F1643&RESPMSG=Timeout waiting for Processor response&TRANSTIME=2015-01-27 17:45:12&BILLTOFIRSTNAME=Josè&BILLTOLASTNAME[16]=Elkjærd & Grün&AMT=600.00&ACCT=1111&EXPDATE=0216&CARDTYPE=0
For anyone else stumbling across this I ran into the same issue as Devin where the è character was being replaced with è when I looked up the transaction in Payflow. I had to convert the encoding to ISO-8859-1. I'm using c# not java so this may not be the same but it was as simple as specifying the encoding in my Stream Writer.
using (StreamWriter sw = new StreamWriter(requestStream, Encoding.GetEncoding("ISO-8859-1")))
An explanation of why this may need to be done can be found here: https://www.i18nqa.com/debug/bug-utf-8-latin1.html