After we switched from Mandrill to SparkPost we encountered issues when sending emails using transmission. In Mandrill merge_vars were not HTML encoded and we sometimes put HTML in them, however in SparkPost substitution_data does appear to be HTML encoded and it's messing some of our emails. Is there a global setting that allows to turn this off or at least disable it for a transmission?
Edit: I forgot to mention that we use csharp-sparkpost library, which means it might be a problem directly related to the library rather than SparkPost API and I need to investigate it further.
Edit2: I tested sending an email directly using JSON and the result was the same, so I can conclude that the HTML encoding is done by SparkPost and not by the c# library.
You can render HTML in substitution variables without escaping by using 3 braces around your variables. e.g. With this in your transmission:
{
"substitution_data": {
"firstName": "<em>Jimbo</em>"
},
"content": {
"html": "<p>Hi {{{firstName}}}</p>"
}
}
...you get this in your HTML message body:
<p>Hi <em>Jimbo</em></p>
There are more details in the SparkPost reference docs: https://developers.sparkpost.com/api/#/introduction/substitutions-reference/escaping-html-values
Related
I am using Bot Composer to publish my first chatbot. I need to construct the chatbot to send out an HTTP POST request to fetch external resources from a remote website. As specified by the composer interface, I can embed JSON, form data, or string in the body of the HTTP POST request. Instead of hard-coding the body part of the POST request, I need to pass in one or multiple properties (chatbot's variable) to generate the body of the HTTP POST dynamically. Here are my questions:
(1) can I pass a variable to the body part of the HTTP REQUEST (such as POST)? can I embed a property such as $(user. name) in the HTTP POST body?
For example, can I embed a property such as $(user.name) in a string or form data (such as fname=$(user. name) to construct the body part of the HTTP POST REQUEST?
(2) The document specifies that there is a pre-build function JSON to serialize data. If I understand correctly, I can't pass a variable (such as $(user. name) to the JSON pre-built function. Therefore, I will probably need to embed an expression in the body to pass the variable. Yet, I couldn't find any detailed information. Is there anywhere I can find a good example showing how to write an expression inside the body part of the HTTP REQUEST
Thanks for any information/assistance.
Yes, you can do this. The simplest way is to set the body to Object and then put in your structured json, something similar to:
{
"userinfo": {
"username": "${user.username}",
"name": "${user.personalname}",
"favoritecolor": "${user.favcolor}",
"profileupdated":"${dialog.userprofileuptodate}"
}
}
I am trying to figure out how to set it up in an adaptive expression in LG, and then be able to refrence it with something like:
# APIBodyTemplate()
-```
{
"userinfo": {
"username": "${user.username}",
"name": "${user.personalname}",
"favoritecolor": "${user.favcolor}",
"profileupdated":"${dialog.userprofileuptodate}"
}
}
```
And then using something like the following in an expression in the body field:
=json(APIBodyTemplate()), but that is not quite working yet. Might be a bug. I will update when I have more info.
I've been working on a MailChimp integration and have run into an issue where their server truncates data sent via merge_fields when including emoji characters. The simplest example that reproduces the behaviour is a PUT request to /lists/xx/members/ with the following body:
{
"email_address": "my.test#email.address",
"status": "subscribed",
"merge_fields": {
"FNAME": "Ian",
"LNAME": "Cool🐯Cat"
}
}
Using the playground on their developer website, the response comes back with the "LNAME" set correctly to "Cool🐯Cat". The issue is that when I view the data later, either via a cURL lookup or generating a campaign email, The "LNAME" field has been truncated to "Cool", stripping the emoji and everything after it.
If anyone has a fix for this behaviour, that would be appreciated. I suspect it's just an issue with MailChimp's database (I know there have been some issues with storing emoji to certain versions of MySQL, for example).
Edit: In the meantime, I'm using this solution to simply strip out the offending non-BMP Unicode characters, but I'd really like an answer that doesn't kill my data.
We have some publicly shared S3 files that we want to make sure won't be indexed by Google. I can't seem to find any documentation on how to do this. Is there a way to set a "noindex" x-robots-tag response header on individual S3 objects?
(We're using the Ruby AWS client)
There does not appear to be a way to do this.
Only certain headers from an S3 PUT object request are documented as being returned when the object is fetched.
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html
Anything else you send appears to be simply disregarded, as long as it doesn't actually invalidate the request.
Actually, that's what I thought before researching this, and it's almost true.
The documentation here seems incomplete, and elsewhere suggests the following request headers, if sent with the upload, will appear in the download:
Cache-Control
Content-Disposition
Content-Encoding
Content-Type
x-amz-meta-*
Other headers are listed at the latter link, but some of these like Expect wouldn't make sense on a GET request, so they logically wouldn't appear.
So far, this is all consistent with my experience with S3.
If you send a random but not-invalid header with your request, it's ignored. Example:
X-Foo: bar
S3 seems to accepts this on upload, but discards it (presumably doesn't store it)... downloading the object does not return the X-Foo header.
But X-Robots-Tag appears to be an undocumented exception to this.
Uploading a file with X-Robots-Tag: noindex (for example) does indeed result in the same header and value being returned with the object when you GET it.
Unless somebody can cite the documentation that explains why this works, we're operating in distinctly undocumented territory.
But, if you're interested in going there, the simple answer appears to be, you just add this header to the HTTP PUT request you send to the REST API to upload the object.
"Not so fast," you say, "I'm using the Ruby SDK." Indeed. The AWS Ruby client seems to be too "helpful" to let you get away with this, at least, not easily. The docs there show how to add "metadata" --
:metadata (Hash) — A hash of metadata to be included with the object. These will be sent to S3 as headers prefixed with x-amz-meta. Each name, value pair must conform to US-ASCII.
Well, that's not going to work, because you'd get x-amz-meta-x-robots-tag.
How do you set other headers in the upload? Every other header you'd normally set is an element of the options hash, like :cache_control, which turns into Cache-Control: in the upload request. Unless they're blindly applying the keys from that hash to the upload transaction (which would be terrible design combined with excellent luck) then you may not have a straightforward way to get here from there. I can't be much more specific, because the only I really know about Ruby is the same thing I know about Java -- from what I've seen of it, I don't like it. :)
But X-Robots-Tag does appear to be a custom header S3 supports, to some extent, without clear documentation of that fact. It's, at least, accepted by the REST API.
Failing the above, you can manually add this header to the metadata in the S3 console after uploading the object. (Note, X-Foo: Bar doesn't work from the S3 console, either -- it's silently discarded, with no error -- but X-Robots-Tag: works fine).
You can also, of course, put a publicly-readable robots.txt file (with the appropriate directives in it) in the root of the bucket. Depending on your cobtent mix, path hierarchy, and other factors, that isn't (perhaps) as simple as selectively setting headers, but if the entire bucket is comprised of information you don't want indexed, it should easily accomplish what you want, since content should not be indexed if disallowed in robots.txt, even when a search spider follows a link to it from another site -- every domain (and subdomain)'s robots.txt file stands alone.
#Michael - sqlbot is correct. The SDKs don't support it by default and it won't show in the AWS Console, but if you set it directly with the REST API it works. For those who don't want to figure out the REST API and its authentication method, I was able to modify the node.js aws-sdk to support this feature.
Amazon stores the method params configuration and validation in a large json file: apis/s3-2006-03-01.min.json . I guess that the other SDKs may implement their validation in the same way.
You can go to the "PutObject" command, and under "input.members" you can add a new parameter "XRobotsTag". Configure it as a "header" and set the location to "X-Robots-Tag".
"XRobotsTag": {
"location": "header",
"locationName": "X-Robots-Tag"
}
Your local aws-sdk is now configured to support X-Robots-Tag on your putObject requests. In node.js this would look like this:
s3.putObject({
ACL: "public-read",
Body: "hello world",
Bucket: "my-bucket",
CacheControl: "public, max-age=31536000",
ContentType: "text/plain",
Key: "hello.txt",
XRobotsTag: "noindex, nofollow"
}, function(err, resp){});
I'm trying to work with the IUCN Red List web services API (here's an example output). Unfortunately I haven't been able to find any documentation other than this one-off Gist. It looks as though the API is constructing an HTML document rather than returning a data object, which isn't something I've experienced in the past. I also notice that in the example there is no mention of a ?callback=JSON_CALLBACK in the URL, which I would expect when dealing with JSONP.
I've constructed an http request in AngularJS like so:
atRiskApp.controller('IucnController', ['$scope', '$routeParams', '$http', function ($scope, $routeParams, $http) {
$scope.iucn = $routeParams.iucn; // pulling a number from the URL: ex. 22718591
$scope.getIUCN = function () {
var iucnUrl = 'http://api.iucnredlist.org/details/' + $scope.iucn + '/0.js';
$http.jsonp( url )
.success( function (response) {
console.log(response);
})
.error( function (response) {
console.log(response);
});
};
}]);
Although the HTML document is being successfully passed to my app I'm getting the following error message:
Uncaught SyntaxError: Unexpected token <
It seems like the app is expecting to get Javascript, and is instead getting an HTML document, which it apparently can't parse. I've tried adding a config object to the request based on the angular docs: $http.jsonp( {url: iucnUrl, responseType: 'text'} ) without any luck.
My question is, how do I work with the returned HTML document, or am I way off track here?
Response from the API is an HTML document with a javascript extension:
On the page you linked to in your comment , I found some potentially useful information under the heading API Index.
You can actually get JSON for all levels of taxonomy, including your example Aneides aeneus. However, this JSON doesn't include all of the data from the HTML version, so it's not as useful. Hopefully this helps a little.
API Index (excerpt)
It is also possible to retrieve the row(s) of the index corresponding to an individual species:
http://iucn-redlist-api.heroku.com/index/species/panthera-leo.json
You can use dashes for spaces, as a convenient replacement for the standard URL escape, %20.
The HTML format contains direct links to the species account pages. The CSV and JSON formats include a species_id column which can be used to construct species account URLs as follows:
http://iucn-redlist-api.heroku.com/details/species_id/0
To use the index JSON in Web pages directly, you may need JSONP padding; use the “.js” extension and add a “callback” parameter with the name of the function to use.
http://iucn-redlist-api.heroku.com/index/genus/Dioscorea.js?callback=show
I diagonally looked over the website and its sitemap and found no reference to a public API. All the output is HTML, and it makes sense that json parse method jsonp will not be able to make sense of it. First < it encounters, it will fail (as is apparent).
First of all, I would contact the site admin to simply ask if there is an API that will yield you XML or json or some other object notation that's convenient to work with.
Then there's the scenario where his or her answer would be 'no':
Parsing HTML is not something to be taken lightly and certainly not something you would write yourself unless absolutely necessary.
Luckily, there are ways to get data from html using jQuery.parseHTML(), pure ('vanilla') javascript ways you can use from within AngularJS and full-blown HTML parsing libraries such as HTML Agility Pack(for use in .NET), all of which can get you to the heart of the data within the DOM nodes you're trying to poke at.
There are many other libraries that might serve you better, but these examples will give you a good starting point to canvas the landscape of HTML parsing. This will take some looking into, but it will be more than worth it.
I have a form with a fileupload control in it, and I call form.submit with a success function.
My server side does the usual trick of setting the content type to text/html to get it to arrive in one piece.
In the success function, action.response.responseText does contain the JSON which I sent.
When it leaves the server, it looks like:
{
html: "<div>a div</div>"
}
When it arrives in the success function, the tags are missing. What's going on? Do I need to put some sort of html cdata wrapper around the entire response on the server to avoid this?
A string in a string in JSON. As long as it is well-formed you are allowed to put HTML in string values (making sure you escape quotes etc.).
It's probably the function you're using to insert the HTML that's stripping the tags.
Here's the situation. When you ask ExtJS or JQuery to do Ajax for a form with a file upload, it has to use an iframe. For the response to come back correctly, it has to be of content type text/html whatever is in it. So it has to have it's HTML characters escaped for HTML, which I accomplished with a function from CommonsLang.