Using the content of an HTML file in a JSON string in shell or Makefile (AWS) - bash

I am attempting to use the AWS apigateway CLI to set an Integration Response mapping template for one of my endpoints in my API. This particular case involves using text/html as my Content-Type and as such my mapping template is raw HTML code. Depending on certain circumstances, I would like to use the CLI to set the mapping template to different HTML code. As such, I am attempting to read the HTML content from a specified file into the --response-template JSON string, which will fill in the mapping template.
The command to do this is as follows:
aws apigateway put-integration-response
--rest-api-id $(restApiId) --resource-id $(resourceId) \
--http-method POST --status-code 200 \
--selection-pattern "" \
--response-parameters '{"method.response.header.Content-Type":"'"'"'text/html'"'"'"}' \
--response-templates '{"text/html": "<!DOCTYPE html><html lang="en"><body><p>Hello, world</p></body></html>"}'
I was originally attempting to do this in a Makefile, hence the variables, but I am open to creating a shell script to achieve the same thing. My HTML file is much larger than the example above and contains scripts with functions, brackets, JSON objects, plenty of strings to stylesheets and whatnot - in other words, many characters that need to be escaped. I cannot manually change the HTML file nor am I sure if I can modify it with escape characters since the mapping template needs to be the exact HTML content as that is what is being rendered.
Is it possible for me to place the entire contents of an HTML file into the value portion of that JSON string? If so, how can I get this to work?
Any help is appreciated.

Simon offered a possible solution to this while using the AWS CLI approach but I have solved this in another way.
I decided to use a Python script with the AWS Python SDK, Boto3. I used Python's json library to dump the content of the HTML file into valid JSON format using json.dumps(). I put that as the value to the JSON object that eventually was used in the Boto3 function below and did an inline replacement of the value as I sent it with json.loads(). I followed the top answer from this link on how to convert html source code to json object.
I used the Boto3 put_integration_response() function and sent that JSON object as the --response-template, which worked for me.

Related

Gatsby & GraphQL: Transforming a String field into markdown

I have a GraphQL schema on a headless CMS. I'm using gatsby-source-graphql on a Gatsby site to get data from it.
There is a content field MyType_BlogPost.body of type String. This contains markdown code.
Is it possible to transform (with gatsby-transformer-remark) that into MarkdownRemark (or similar) so that it will automatically get subfields like html that contain the data one would expect (html code transformed from the markdown source)? I suspect subfields would be the way to do this, but I'm not 100 % of that.
I also suspect this could help but I'm not sure: https://www.gatsbyjs.org/docs/schema-customization/
Thank you.
If the field contains markdown code which you would like to transform into html, you should be able to do that using remark.
Remark is the library that gatsby-transformer-remark uses under the hood.
https://remark.js.org/
Specifically, look at the API section in their docs.

VuePress: How to modify the markdown content

VuePress (v1) provides this API to access the page context: https://v1.vuepress.vuejs.org/plugin/option-api.html#extendpagedata
I can access these properties but I can't change the value of _content for instance.
I would like to modify (replace some regexp) the markdown before it get's rendered by Vue (an even before it's get parsed by the markdown parser).
Maybe it makes more sense to do it with a markdown-it plugin using the chainmarkdown API but I don't know how exactly. But the question is more how to write a plugin for markdown-it which can modify the content before even parsing, because I need the raw text.
Here is one markdown-it plugin that allows you to do exactly that by defining regular expressions: https://www.npmjs.com/package/markdown-it-regexp

Properly Nesting Arrays Within Objects In Bash

I'm building a Bash shell script to run as a post-build task in Jenkins.
This script sets some information conditionally and then sends it to Slack's Inbound Webhook API.
The information is sent via a cURL request, and the JSON object that I send can accept an attachments key which points to an array of objects.
This is how my code for the attachments array and overall JSON object
attachments="[\"{\"fallback\":\"This\u0020is\u0020a\u0020fallback\u0020message\u0020just\u0020in\u0020case\",\"color\":\"#36a64f\",\"author_name\":\"$author_name\",\"text\":\"$text\"}\"]"
json="{\"channel\":\"$channel\",\"username\":\"$username\",\"icon_emoji\":\"$emoji\",\"attachments\":\"$attachments\"}"
In Jenkins' console output, I'm seeing the following translation in my cURL request:
curl -X POST --data '{"channel":"#jenkinsslacktest","username":"Jenkins-Bot","icon_emoji":":rocket:","attachments":"["{"fallback":"This\u0020is\u0020a\u0020fallback\u0020message\u0020just\u0020in\u0020case","color":"#36a64f","author_name":"TestAuthor","text":":rocket:\u0020:rocket:\u0020SUCCESS!\u0020:rocket:\u0020:rocket:"}"]"}' https://hooks.slack.com/services/link-to-my-webhook
In the way it's formatted, my webhook does not trigger appropriately and no message is sent into my chosen Slack channel.
If I replaced the attachments key/value pair and just placed the text line in its place, this call is successful.
It seems like I'm just not escaping or formatting this attachments value properly. What should I be doing differently?
The only issue here is the JSON syntax of the attachments array. You need to remove the quotes around [ and ] to make it valid JSON syntax. Then it will work.
Here is the correct bash line:
curl -X POST --data '{"channel":"#jenkinsslacktest","username":"Jenkins-Bot","icon_emoji":":rocket:","attachments":[{"fallback":"This\u0020is\u0020a\u0020fallback\u0020message\u0020just\u0020in\u0020case","color":"#36a64f","author_name":"TestAuthor","text":":rocket:\u0020:rocket:\u0020SUCCESS!\u0020:rocket:\u0020:rocket:"}]}' https://hooks.slack.com/services/link-to-my-webhook
You can use a tool like JSONLint or JSONViewer to verify your JSON syntax is correct.

Passing whole body as a variable to a POST request

I'm trying to create a simple stress test using JMeter. I have mostly GET requests and a couple POST requests. My main goal is to make this test as reusable as possible. I want to implement it in a way that the user would have to provide a CSV file with the following headers:
method;path;postBody
The values would look something like:
GET;/path/to/resource;''
POST;/path/to/resource;'{"key":"value","key":"value","key":"value"}'
Now POST (PUT, PATCH etc ..) bodies differ from one request to another. Providing ${postBody} to Body Data tab does not work "${postBody}" as well.
Is there a way to achieve this? Command line solutions are more than welcome as well.
EDIT: To clarify, I'm using the UI interface. When I input ${postBody} in the Body Data tab the UI complains. When switching from the Body Data tab to another one I get the following prompt:
Remove "'" around the request and it should work.
Regarding the warning you get, it is not an error, it is just that in JMeter those 2 tabs are exclusive:
Parameters tabs is for input of parameters in the form as name=value
Body data is for your kind of requirement
So can you test my hypothesis which is to remove the quote around the request in CSV file ?
If it still fails, please show the logs.
You can remain in Body Data tab,
Add after pathPost your optional query parameters for GET request:
${pathPost}?${getPramaters}
Don't worry about the ? it's just seperate path from parameters
Also consider changing variable name to path, more suitable because it can be POST.
In JSON , seperate between values while in CSV default is also ,
I suggest you can your CSV delimiter, In CSV Data Set Config Choose different Delimiter as ; and add your data in CSV accordingly (remove extra ' characters):
POST;/path/to/resource;{"key":"value","key":"value","key":"value"}
Notice: Allow quoted data keep default value False

How to include an image in an http POST request

I'm trying to make a POST request from the command line to my Flask app, and I want it to include an image. But I don't know how to include it with the command. I've only used strings as data successfully.
So, if my POST request looks like this:
curl -i -H "Content-Type: application/json" -X POST -d '{"username":"user1", "password":"password", "image":##What do I put here?##}' http://localhost:5000/my_app/api/users
I don't know what to put in that image part of the JSON. I'm tagging flask in this question because it might be a specific answer with regards to flask.
I would like to include an actual image here, and then on the Flask side of things, put the image in a folder of the app where all the uploads go, then save the path to the image in the database for later access. But, to do that, I need to know how to send an image in the first place. Any thoughts?
Seems you're mixing things up here.
From your example seems you want to upload an image in a JSON object. This is generally bad for 2 reasons:
Overhead: the image data should be encoded in printable characters, e.g. using base64. This creates a huge overhead on the data itself causing the JSON decoder to slow down.
Testing: You can't try this using curl on the commandline. You should make some command line utility to test the request.
HTTP knows about data uploads. So to mantain the JSON structure without the slowdown of above, you should upload your image as traditional data upload using a field, and another field for the JSON data structure.
Using curl this is achieved with the -F option.
curl -i -Ffiledata=#file.jpg -Fdata='{"username":"user1", "password":"password"}' http://localhost:5000/my_app/api/users
With the above command you are sending an HTTP POST with a file upload named filedata and a 'data` field containing your 'JSON' payload to parse in the receiving view handler.
You must use 2 HTTP fields because in HTTP upload you're using multipart encoding.
Behind the scenes
You're sending the image contents encoded in base64 but since the data is decoded by the application framework knowing exactly what to do (JSON parse must figure it out while parsing) it's a lot faster.

Resources