I have this block of code which works fine
https://jsfiddle.net/jejyjose/v7woyd8t/8/
Inside the JS part, if I disable "Part 1" and enable "Part 2" which gives me the same Json as a string , the script doesn't work. Can you guys help me on that?
When I call the remoteservice, I have limited control of what is spit out. The output of the webservice from the browser is like below- Once I read the json data, it gives me the correct data. Does a conversion to array required forthis?
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/"> { "rows": [ { "Dexc": "Fresh import for Sale 2015 Range rover Evoque HSE, Si4, Red, 5 door, Petrol, 1990 cc, Automatic, 4wd, Feul Economy start 3.5/6", "Location": "C:\Joseph\Source\OS\_Picture", "items": "", "ItemDescription": "hello1|hello2|hello3|hello4|hello5|hello6|hello7|hello8", "Phone": "09432d145", "Email": "someone#gmail.com", "Mobile": "02102628951" ,"Website": "www.somesite.com" }, { "Dexc": "2Fresh import for Sale 2015 Range rover Evoque HSE, Si4, Red, 5 door, Petrol, 1990 cc, Automatic, 4wd, Feul Economy start 3.5/6", "Location": "2C:\Joseph\Source\OS\_Picture", "items": "", "ItemDescription": "2hello1|hello2|hello3|hello4|hello5|hello6|hello7|hello8", "Phone": "209432d145", "Email": "2someone#gmail.com", "Mobile": "202102628951" ,"Website": "www.somesite.com" }, ] } </string>
Related
The problem I am having is :
Sharepoint Get File Files (Properties Only) can only do one filter for ODATA, not a a second AND clause so I need to use Filter Array to make secondary filter work. And it does work....
But now I need to take my filtered array and somehow get the {FullPath} property and get the file content via passing a path and I get this error...
[ {
"#odata.etag": ""1"",
"ItemInternalId": "120",
"ID": 120,
"Modified": "2022-03-21T15:03:31Z",
"Editor": {
"#odata.type": "#Microsoft.Azure.Connectors.SharePoint.SPListExpandedUser",
"Claims": "i:0#.f|membership|dev#email.com",
"DisplayName": "Bob dole",
"Email": "dev#email.com",
"Picture": "https://company.sharepoint.us/sites/devtest/_layouts/15/UserPhoto.aspx?Size=L&AccountName=dev#email.com",
"Department": "Information Technology",
"JobTitle": "Senior Applications Developer II"
},
"Editor#Claims": "data",
"Created": "2022-03-21T15:03:31Z",
"Author": {
"#odata.type": "#Microsoft.Azure.Connectors.SharePoint.SPListExpandedUser",
"Claims": "i:0#.f|membership|dev#email.com",
"DisplayName": "Bob Dole",
"Email": "dev#email.com",
"Picture": "https://company.sharepoint.us/sites/devtest/_layouts/15/UserPhoto.aspx?Size=L&AccountName=dev#email.com",
"Department": "Information Technology",
"JobTitle": "Senior Applications Developer II"
},
"Author#Claims": "i:0#.f|membership|dev#email.com",
"OData__DisplayName": "",
"{Identifier}": "Shared%2bDocuments%252fSDS%252fFiles%252fA10_NICKEL%2bVANADIUM%2bPRODUCT_PIS-USA_French.pdf",
"{IsFolder}": false,
"{Thumbnail}": ...DATA,
"{Link}": "https://company.sharepoint.us/sites/devtest/Shared%20Documents/SDS/Files/A10_NICKEL%20VANADIUM%20PRODUCT_PIS-USA_French.pdf",
"{Name}": "A10_NICKEL VANADIUM PRODUCT_PIS-USA_French",
"{FilenameWithExtension}": "A10_NICKEL VANADIUM PRODUCT_PIS-USA_French.pdf",
"{Path}": "Shared Documents/SDS/Files/",
"{FullPath}": "Shared Documents/SDS/Files/A10_NICKEL VANADIUM PRODUCT_PIS-USA_French.pdf",
"{IsCheckedOut}": false,
"{VersionNumber}": "1.0" } ]
So from what I can see, I think it's what I thought. Even though you're filtering an array down to a single element, you need to treat it like an array.
I'm going to make an assumption that you're always going to retrieve a single item as a result of your filter step.
I created a variable (SharePoint Documents) to store your "filtered" array so I could then do the work to extract the {FullPath} property.
I then created variable that is initialised with the first (again, I'm making the assumption that your filter will only ever return a single element) and used this expression ...
variables('SharePoint Documents')?[0]['{FullPath}']
This is the result and you can use that in your next step to get the file content from SharePoint ...
If my assumption is wrong and you can have more than one then you'll need to throw it in a loop and do the same sort of thing ...
This is the expression contained within ...
items('For_Each_in_Array')['{FullPath}']
Result ...
I actually ended up doing this and it works.
Get the type of the house that is Villa:
<div class="property__feature">
<h3 class="property__feature-title property__feature-title--b-spacing">Property Details</h3>
<ul class="property__details-list">
<li class="property__details-item">
Here is where I need to get the info:
<span class="property__details-item--cat">Type:</span>
Villa
// ***
They are using client-side template rendering, so there is no xpath selector you can use to extract that element using Scrapy because Scrapy only deals with HTML, not JavaScript
That said, the data you are looking for is sent down inside the page, and thus can be extracted with a little massaging:
def parse(self, response):
import json
model_json = ''.join(response.xpath(
'.//script[#type="text/javascript" and contains(text(), "var model")]/text()'
).re(r'var model = (\{.+\});'))
model_data = json.loads(model_json)
property_data = model_data['Property']
"""
{
"Property": {
"Id": 80765,
"Type": 32,
"Status": 1,
"Area": {
"Id": 32,
"Name": "Geri",
"District": {
"Id": 1,
"Name": "Nicosia"
}
},
"IsForRent": false,
"RentPrice": 0,
"IsForSale": true,
"SalePrice": 125000,
"PriceIsPublic": true,
"SaleVAT": 0,
"SaleVATType": 1,
"Description": "<p class=\"MsoNormal\" style=\"text-align: justify;\"><span style=\"mso-fareast-language: EN-GB;\">We are pleased to present the new project of under division residential plots in one of the most sought after areas of Geri. It is located in a quiet residential area, 3 km away from the General Hospital of Nicosia, 3.5 km from the Mall of Cyprus, 2.5 km from the University of Cyprus and with easy access to the motorway. The under division plot has an area of 565 sqm and it falls into residential planning zone Κα8 with 60% building density, 35% coverage, 2 floors and a maximum height of 10m.</span></p>\r\n<p class=\"MsoNormal\" style=\"text-align: justify;\">Subject to VAT.</p>",
"EnergyEfficiencyRating": {
"Type": 1,
"CurrentRating": 0,
"PotentialRating": 0,
"Key": null
},
"EnvironmentalImpactRating": {
"Type": 2,
"CurrentRating": 0,
"PotentialRating": 0,
"Key": null
},
"ShowOnMap": true,
"Coordinates": {
"Latitude": 35.119343,
"Longitude": 33.407802
},
If you want to convert those "Type": 32 into "Type": "Building Plot", you'll have to go the extra step of also extracting their .init data from the other script text
json.loads(''.join(response.xpath(
'.//script[#type="text/javascript" and contains(text(), ".init(")]/text()'
).re(r'\.init\((\{.+\})\)')))
"""{
"Resources": {
"PropertyTypes": {
"1": "Apartment",
"2": "House",
"8": "Commercial",
"32": "Building Plot",
"64": "Land",
"128": "Industrial",
"256": "Investments",
"512": "Villa",
"1024": "Mansion"
},
"PropertyStatuses": {
"1": "Resale",
"2": "Rented",
"3": "Sold",
In the future, if you don't see the information you are looking for when you use the "View Source" of your browser (or in the output of response.body_as_unicode() from inside Scrapy), then it is being loaded dynamically and thus needs some special handling. Most of the time, that information is loaded via XHR and you're actually in luck because reproducing that in Scrapy is usually painless and gets you out of the HTML scraping game entirely. This is one of those unfortunate situations where the data is being loaded dynamically, but from within the page, and thus the preceeding silliness is required.
I am using Microsoft Graph API to get mails.
GET /v1.0/me/messages
It returns
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('576552d5-3bc0-42a6-a23d-bfceb405db23')/messages",
"#odata.nextLink": "https://graph.microsoft.com/v1.0/me/messages?$skip=11",
"value": [
{
"#odata.etag": "W/\"HwAAABYAAACpTc/InBsuTYwTUBb+VIb4AACqi2tx\"",
"id": "AAMkADBlZTUwNTkxLWVmODgtNDVhNC1iZjhlLTdjNjA1ODZlMDI2MgBGAAAAAACUbnk-iwQZRbXMgkfKtmYhBwCpTc-InBsuTYwTUBb_VIb4AAAAAAEMAACpTc-InBsuTYwTUBb_VIb4AACqNTk9AAA=",
"createdDateTime": "2017-12-06T21:57:09Z",
"lastModifiedDateTime": "2017-12-06T21:57:19Z",
"changeKey": "HwAAABYAAACpTc/InBsuTYwTUBb+VIb4AACqi8tx",
"categories": [],
"receivedDateTime": "2017-12-06T21:57:09Z",
"sentDateTime": "2017-12-06T21:56:16Z",
"hasAttachments": false,
"internetMessageId": "<e74a536a53d245e49d779d47f774f4a0#CO2PR00MB0214.namprd00.prod.outlook.com>",
"subject": "Automatic reply: Hi",
"bodyPreview": "I am OOF.",
"importance": "normal",
"parentFolderId": "AAMkADBlZTUwNTkxLWVmODgtNDVhNC1iZjhlLTdjNjA2ODZlMDI5MgAuAAAAAACUbnk-iwQZRbXMgkfKtmYhAQCpTc-InBsuTYwTUBb_VIb4AAAAAAEMAAA=",
"conversationId": "AAQkADBlZTUwNTkxLWVmODgtNDVhNC1iZjhlLTdjNjA2ODZlMDI5MgAQAPekscpearpHmBFbhG0DKuc=",
"isDeliveryReceiptRequested": null,
"isReadReceiptRequested": false,
"isRead": true,
"isDraft": false,
"webLink": "https://outlook.office365.com/owa/?ItemID=AAMkADBlZTUwNTkxLWVmODgtNDVhNC1iZjhlLTdjNjA1ODZlMDI5MgBGAAAAAACUbnk%2FiwQZRbXMgkfKtmYhBwCpTc%2FInBsuTYwTUBb%2BVIb4AAAAAAEMAACpTc%2FInBsuTYwTUBb%2BVIb4AACqNTk2AAA%3D&exvsurl=2&viewmodel=ReadMessageItem",
"inferenceClassification": "focused",
"body": {
"contentType": "html",
"content": "hi"
},
"sender": {
"emailAddress": {
"name": "Jack",
"address": "jack#example.com"
}
},
"from": {
"emailAddress": {
"name": "Jack",
"address": "jack#example.com"
}
},
"toRecipients": [
{
"emailAddress": {
"name": "Rose",
"address": "rose#example.com"
}
}
],
"ccRecipients": [],
"bccRecipients": [],
"replyTo": []
}
]
}
I didn't find any field related with determine whether it is an auto reply mail.
Right now I am using
mail.subject.startsWith('Automatic reply:')
to determine whether is auto reply mail in code.
However, it is not reliable. Because sometimes I got mails starting with a different language such as Resposta automática:.
So how to know it is auto reply mail correctly?
As #Horkrine said there is no officially guaranteed way of detecting if an email is an auto reply or not.
But there are two ways that may be useful:
Method 1 : Detect the response time
If you are capable, consider checking the amount of time between the email sent and the response. If that time is within a certain threshold, it is almost certainly an auto reply. Consider a reply received within seconds, for example. This has a lot of correlations with modern-day spam-robot detection techniques.
Method 2 : Keywords
The other way to do it is to look for keywords, just as you are doing now. However, you also have to account for other languages, variations on spelling, misspellings, etc. You will not get everything.
For example:
mail.subject.contains('Automatic') OR mail.subject.contains('Auto-matic') OR mail.subject.contains('Away') OR mail.subject.contains('out of office')
...
OR mail.subject.contains('automática') ...
Rather than typing out such a list, I would recommend doing a quick search on the internet and see if there are any such lists you can copy-paste from, as surely someone has done this sort of thing before and has some free code.
I'm no expert but I don't believe there's any way to determine whether or not an email is an automatic reply unless the email actually contains a string saying "This is an automatic reply" or something.
Just found another interesting API getMailTips, however this can only help determine the auto mail if the other user is Outlook or Office 365 user.
Copy the demo below for convenience.
POST https://graph.microsoft.com/api/beta/users/{id|userPrincipalName}/getMailTips
{
"EmailAddresses": [
"danas#contoso.onmicrosoft.com",
"fannyd#contoso.onmicrosoft.com"
],
"MailTipsOptions": "automaticReplies, mailboxFullStatus"
}
It will return something like
{
"#odata.context":"https://graph.microsoft.com/api/beta/$metadata#Collection(microsoft.graph.mailTips)",
"value":[
{
"emailAddress":{
"name":"",
"address":"danas#contoso.onmicrosoft.com"
},
"automaticReplies":{
"message":"<style type=\"text/css\" style=\"\">\r\n<!--\r\np\r\n\t{margin-top:0;\r\n\tmargin-bottom:0}\r\n-->\r\n</style>\r\n<div dir=\"ltr\">\r\n<div id=\"x_divtagdefaultwrapper\" style=\"font-size:12pt; color:#000000; background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif\">\r\n<p>Hi, I am on vacation right now. I'll get back to you after I return.<br>\r\n</p>\r\n</div>\r\n</div>",
"messageLanguage":{
"locale":"en-US",
"displayName":"English (United States)"
}
},
"mailboxFull":false
},
{
"emailAddress":{
"name":"",
"address":"fannyd#contoso.onmicrosoft.com"
},
"automaticReplies":{
"message":""
},
"mailboxFull":false
}
]
}
I am trying to integrate with a service that provides FHIR ImageStudy messages in JSON format. Once I have the JSON message I need to convert the message to XML.
I am using the FHIR-net-api found here, https://github.com/ewoutkramer/fhir-net-api I posted earlier and got help using this library to parse standard image study messages. Here is a link to my earlier post, FHIR JSON to XML decoding in BizTalk
The service I am connecting to has added some extensions to the image study message and when I attempt to parse it I get an error that the parser failed line 1 character 1.
My understanding is that if the extension are done correctly the FHIR-net-api library should be able to parse the JSON to XML. Is this correct?
Can anyone identify if the test message below is compliant to the FHIR standard, if not what is wrong with it? I have shortened the message to only contain a single image study but the service returns multiple in a bundle. I have also removed identifying information.
{
"resourceType": "Bundle",
"total": 15,
"entry": [
{"resource": {
"resourceType": "ImagingStudy",
"id": "LALA.e1e6683d-f6d9-e311-ae0e-0050568f64",
"contained": [
{
"resourceType": "Organization",
"text": {"div": "LALA"},
"name": "LALA"
},
{
"resourceType": "Procedure",
"id": "Procedure1",
"code": {"coding": [ {
"code": "RAD-HANB",
"display": "HANDS BIL"
}]}
}
],
"extension": [ {
"url": "https://someplace.org/fhir/extensions/imagingstudy-examstatus",
"valueString": "Finalized"
}],
"started": "2013-12-03T12:30:00-08:00",
"accession": {"value": "A12345BH"},
"procedure": [{"reference": "#Procedure1"}],
"series": [ {
"modality": {
"system": "http://www.dicomlibrary.com/dicom/modality/",
"code": "CR"
},
"bodySite": {"code": "UEX"},
"instance": [
{"title": "DiagnosticReport"},
{
"title": "DiagnosticImage",
"content": [
{
"url": "/fhir/Patient/91111/ImagingStudy?_query=imageUrl&_id=6683d-f6d9-e311-ae0e-0050568f6477&-mrn=12345T&-organization=lala&accession=tester&-status=F",
"title": "Something"
},
{
"url": "/fhir/Patient/9111111/ImagingStudy?_query=html5Url&_id=e1e6683d-f6d9-e311-ae0e-0050568f6&-mrn=123345&-organization=lala&accession=testing&-status=F",
"title": "HTML5"
}
]
}
]
}]
}}
]
}
I suspect that you got a message like this: Error parsing XHTML: Incorrect document syntax. at line 1 row 1. source = " at line 8 col 13
That's what I get once I clean up the instance a little bit to include only the resource and not the wrapper from the Bundle and check it against http://fhir2.healthintersections.com.au/open/.
The first issue is that the narrative inside your div tag is not valid. It needs to look like this:
"div": "<div>LALA</div>"
However there are a bunch of others. The narrative is missing status. Narrative isn't actually allowed on contained resources, you're missing a bunch of mandatory elements, etc. Just go to the link above and paste your JSON into the "upload" box at the bottom of the page and select "validate". That will give you a full report of the issues. (Not all of those will necessarily impact your ability to convert between JSON and XML, but presumably you'll want to fix them regardless.)
Consider there is a action card response from the MS bot & it looks as follows in skype:
When this similar response comes in the REST APIs i.e using Direct Line APIs. The following is the relevant part of JSON response.
{
"id": "1t90Ym3PEry|000000000000000014",
"conversationId": "1t90Ym3PEry",
"created": "2016-12-06T09:34:55.6280699Z",
"from": "rich3cards",
"images": [
"https://upload.wikimedia.org/wikipedia/commons/thumb/7/7c/Seattlenighttimequeenanne.jpg/320px-Seattlenighttimequeenanne.jpg"
],
"attachments": [],
"eTag": "W/\"datetime'2016-12-06T09%3A34%3A54.94083Z'\""
},
{
"id": "1t90Ym3PEry|000000000000000014",
"conversationId": "1t90Ym3PEry",
"created": "2016-12-06T09:34:55.6280699Z",
"from": "rich3cards",
"text": "Hero Card\n\nSpace Needle\n\nThe <b>Space Needle</b> is an observation tower in Seattle, Washington, a landmark of the Pacific Northwest, and an icon of Seattle.\n\n(Current Weather) action?weather=Seattle, WA",
"images": [],
"attachments": [],
"eTag": "W/\"datetime'2016-12-06T09%3A34%3A54.94083Z'\""
}
Now, the question is about how do we parse this json to get the button data [(Current Weather) action?weather=Seattle, WA"] out of the text attribute? Is the only way is patter match ?
Has anyone faced or know solution, please put some light here too ;)
Update: If its different channel like skype/webchat/etc.. the JSON response looks very proper to consume, following is the sample JSON.
{
"type": "message",
"id": "5AdoK89rtSc|000000000000000018",
"timestamp": "2016-12-06T09:53:20.4777291Z",
"channelId": "webchat",
"from": {
"id": "rich3cards",
"name": "RichCards"
},
"conversation": {
"id": "5AdoK89rtSc"
},
"attachments": [
{
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"title": "Hero Card",
"subtitle": "Space Needle",
"text": "The <b>Space Needle</b> is an observation tower in Seattle, Washington, a landmark of the Pacific Northwest, and an icon of Seattle.",
"images": [
{
"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/7/7c/Seattlenighttimequeenanne.jpg/320px-Seattlenighttimequeenanne.jpg"
}
],
"buttons": [
{
"type": "postBack",
"value": "action?weather=Seattle, WA",
"title": "Current Weather"
}
]
}
}
]
As mentioned in the comments, you are using DirectLine v1.1. Unfortunately, v1.1 doesn't support attachments/cards and so there isn't a good way to understand/parse the card.
You might want to consider moving to DirectLine v3 which has full support for attachments.
Alternatively, if you want to support Cards, you might have to do something custom as shown in the DirectLine sample. There, the bot is sending the hero card through the ChannelData field and the client is parsing that accordingly. However, you might have to add the logic to detect who is talking to the bot so you send the cards as ChannelData only if the caller is DirectLine and not one of the other clients (such as skype)