How to send a POST request with Kotlin/Native using libcurl? - spring-boot

I'm trying to send a JSON through a POST request with a Kotlin/Native application using libcurl. I'm working on a Windows 11 machine, and the endpoint lies under a Spring Boot (version 2.7.8) backend written with Kotlin and Java 11.
The following is the code I wrote to accomplish this task.
import kotlinx.cinterop.*
import libcurl.*
fun main() {
val curl = curl_easy_init()
curl?.let {
var headers: CValuesRef<curl_slist>? = null
headers = curl_slist_append(headers, "Content-Type: application/json")
setCurl(curl, headers)
val res = curl_easy_perform(curl)
if (res != CURLE_OK) println("curl_easy_perform() failed ${curl_easy_strerror(res)?.toKString()}")
curl_easy_cleanup(curl)
curl_slist_free_all(headers)
} ?: println("curl_easy_init() failed to return curl easy handle")
}
private fun setCurl(curl: COpaquePointer?, headers: CPointer<curl_slist>?) {
val body = "{ JSON object }"
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers)
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body)
curl_easy_setopt(curl, CURLOPT_URL, "http://localhost:8080/dishes/add/")
}
The JSON string I need to send represents a simple Dish object having three fields: name, type and description.
These are the attempts I made to correctly format the JSON string to initialize the val body and the relative outputs that the Spring Boot endpoint return:
{\"name\":\"Fish\",\"type\":\"Second\",\"description\":\"Fry\"} => HTTP 200;
{\"name\":\"Fish fry\",\"type\":\"Second course\",\"description\":\"Lots of fresh fish to delight the palate with an excellent, fried second course\"} => HTTP 400: JSON parse error: Unexpected character (' ' (code 160)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false');
{\\\"name\\\":\\\"Fish fry\\\",\\\"type\\\":\\\"Second course\\\",\\\"description\\\":\\\"Lots of fresh fish to delight the palate with an excellent, fried second course\\\"} => HTTP 400: JSON parse error: Illegal character ((CTRL-CHAR, code 0)): only regular white space (\r, \n, \t) is allowed between tokens;
"\"{\\\"name\\\":\\\"Fish fry\\\",\\\"type\\\":\\\"Second course\\\",\\\"description\\\":\\\"Lots of fresh fish to delight the palate with an excellent, fried second course\\\"}\"" => HTTP 400: JSON parse error: Unexpected character (' ' (code 160)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
From the above tests, I can effectively POST something on the backend only with the first body string, but if I try to send a longer one with even the same format (see the second one), the POST request fails too.
Besides, I also tried to execute a POST request using Postman and the terminal (with the following command: curl -H "Content-Type: application/json" -X POST -d "{\"name\":\"Fish fry\",\"type\":\"Second course\",\"description\":\"Lots of fresh fish to delight the palate with an excellent, fried second course\"}" http://localhost:8080/dishes/add/), and with both, I can correctly send the POSTs. And since using the commands prompt, curl accepts the last body JSON I tried, maybe it could be the right approach to format the string, but I'm not sure.
What am I missing?
Thanks for your precious time!
UPDATE 1:
I just discovered the --libcurl curl parameter, which lets you convert a curl command into libcurl code.
Using this helpful tool, I converted my working cmd POST request
curl -H "Content-Type: application/json" -X POST -d "{\"name\":\"Fish fry\",\"type\":\"Second course\",\"description\":\"Lots of fresh fish to delight the palate with an excellent, fried second course\"}" http://localhost:8080/dishes/add/
Into the following, Kotlin adapted, C snippet:
private fun setCurl(hnd: COpaquePointer?, certPath: String, url: String) {
var headers: CValuesRef<curl_slist>? = null
headers = curl_slist_append(headers, "Content-Type: application/json")
curl_easy_setopt(hnd, CURLOPT_BUFFERSIZE, 102400L)
curl_easy_setopt(hnd, CURLOPT_URL, "http://localhost:8080/dishes/add/")
curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 1L)
curl_easy_setopt(
hnd,
CURLOPT_POSTFIELDS,
"{\"name\":\"Fish fry\",\"type\":\"Second course\",\"description\":\"Lots of fresh fish to delight the palate with an excellent, fried second course\"}"
)
val postFieldSize: curl_off_t = 138
curl_easy_setopt(hnd, CURLOPT_POSTFIELDSIZE_LARGE, postFieldSize)
curl_easy_setopt(hnd, CURLOPT_HTTPHEADER, headers)
curl_easy_setopt(hnd, CURLOPT_USERAGENT, "curl/7.83.1")
curl_easy_setopt(hnd, CURLOPT_MAXREDIRS, 50L)
curl_easy_setopt(hnd, CURLOPT_CUSTOMREQUEST, "POST")
curl_easy_setopt(hnd, CURLOPT_FTP_SKIP_PASV_IP, 1L)
curl_easy_setopt(hnd, CURLOPT_TCP_KEEPALIVE, 1L)
}
But still, my Kotlin/Native application failed to execute the request with Spring Boot returning the same error: Unexpected character (' ' (code 160)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false').
At this time, I ran out of ideas. Please, let me know about any other solutions.
UPDATE 2:
Since the first val body initialization was the only one to be successful (despite the body string having to be small), I started to do some other tests with that type of formatted JSON, so I found that the POST request is successful if the entire string does not cross the 63 chars of length otherwise, the Spring Boot endpoint fires the error regarding the code 160 unexpected character.
The body string I'm currently using is the following one, which length is exactly 63 chars.
{\"name\":\"Fettuccine Alfredo\",\"type\":\"Main co\",\"description\":\"\"}
I don't know why this situation is happening, and I'm very frustrated.
Every tip of advice is much appreciated.

Related

Curl/GraphQL command failing with 200

I am trying to write a shell script that executes a curl against a GraphQL API and I've never interacted with GQL before. I am getting some strange errors and although I understand this community doesn't have access to the GQL server I was hoping someone could take a look at the script and make sure I'm not doing anything flagrantly wrong syntax-wise (both in the shell script layer as well as the GQL query itself).
My script:
#!/bin/bash
BSEE_WEB_SERVER_DNS=https://mybsee.example.com
BSEE_API_KEY=abc123
siteId=1
scanConfigId=456
runScanQuery='mutation CreateScheduleItem { create_schedule_item(input: {site_id: "$siteId" scan_configuration_ids: "$scanConfigId"}) { schedule_item { id } } }'
runScanVariables='{ "input": "site_id": $scanId }}'
runScanOperationName='CreateScheduleItem'
curl -i --request POST \
--url $BSEE_WEB_SERVER_DNS/graphql/v1 \
--header "Authorization: $BSEE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{"query":"$runScanQuery","variables":{$runScanVariables},"operationName":"${runScanOperationName}"}'
And the output when I run the script off the terminal:
HTTP/2 200
<OMITTED RESPONSE HEADERS>
{"errors":[{"message":"Invalid JSON : Unexpected character (\u0027$\u0027 (code 36)): was expecting double-quote to start field name, Line 1 Col 38","extensions":{"code":3}}]}%
I am omitting the HTTP response headers for security and brevity reasons.
I am wondering if my use of quotes/double-quotes is somehow wrong, or if there is anything about the nature of the GQL query itself (via curl) that looks off to anyone.
I verified with the team that manages the server that the HTTP 200 OK response code is correct. 200 shows that the request succeeded to the GQL API, but that GQL is responding with this error to indicate the query itself is incorrect.
We need to modify the GraphQL bits and fix the bash string quoting.
runScanQuery GraphQL operation
Fix the GraphQL syntax. Use a GraphQL operation name CreateScheduleItem with variables $site_id in the arguments input: { site_id: $siteId, scan_configuration_ids: $scanConfigId:
mutation CreateScheduleItem($site_id: String!, $scanConfigId: String!) {
create_schedule_item(
input: { site_id: $siteId, scan_configuration_ids: $scanConfigId }
) {
schedule_item {
id
}
}
}
runScanVariables: JSON
Our mutation expects two variables, which GraphQL will substitute into CreateScheduleItem($site_id: String!, $scanConfigId: String!). Provide the GraphQL variables as JSON. Here is the expected output after bash variable substitution:
{ "$site_id": "1", "$scanConfigId": "456" }
Get the bash quoting right
Finally, translate the inputs into bash-friendly syntax:
runScanQuery='mutation CreateScheduleItem($site_id: String!, $scanConfigId: String!) { create_schedule_item(input: {site_id: $siteId scan_configuration_ids: $scanConfigId}) { schedule_item { id } } }'
runScanVariables='{"$site_id":"'"$siteId"'","$scanConfigId":"'"$scanConfigId"'"}' # no spaces!
runScanOperationName='CreateScheduleItem'
data='{"query":"'"$runScanQuery"'","variables":'$runScanVariables',"operationName":"'"$runScanOperationName"'"}'
Check our bash formats. Paste the terminal output into a code-aware editor like VSCode. Expect the editor to parse the output correctly.
echo $runScanQuery # want string in graphql format
echo $runScanVariables # want JSON
echo $data # want JSON
Edit: add a public API example
Here's a complete working example using the public Star Wars API:
#!/bin/bash
filmId=1
data='{"query":"query Query($filmId: ID) { film(filmID: $filmId) { title }}","variables":{"filmId":"'"$filmId"'"}}'
curl --location --request POST 'https://swapi-graphql.netlify.app/.netlify/functions/index' \
--header 'Content-Type: application/json' \
--data "$data"
Responds with {"data":{"film":{"title":"A New Hope"}}}.
In GraphQL it's normal to always have 200 status code; client must check response body searching for failures.
The reason is simple: In REST, http is part of the protocol and status code has semantics but in GraphQL http is not part of the protocol, you can have GraphQL over serveral transport protocols:
http: typical scenario docs
WebSocket: does not provide any "status code like" payload. sample
MQTT: does not provide any "status code like" payload
...
The only way that server tells you something (even failures) is the body.
In your case I suggest you jq to parse json via bash script searching error property.
Your error is completely unrelated to GraphQL. You really have wrong JSON.
Error message says Unexpected character (\u0027$\u0027 (code 36)): was expecting double-quote to start field name, Line 1 Col 38",
You can replace escaped \u0027 with apostrophe and you will get
Unexpected character ('$' (code 36)): was expecting double-quote to start field name, Line 1 Col 38",
So it hates dollar sign at position 38 in what you send as data to curl
data='{"query":"'"$runScanQuery"'","variables":'$runScanVariables'
^
this
First - all field names and values in JSON should be wrapped with double quotes, not single.
Second - if you want curl to expand env variable, put it to double quotes, not single.

How to delete all attributes from the schema in solr?

Deleting all documents from solr is
curl http://localhost:8983/solr/trans/update?commit=true -d "<delete><query>*:*</query></delete>"
Adding a (static) attribute to the schema is
curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field":{"name":"trans","type":"string","stored":true, "indexed":true},}' http://localhost:8983/solr/trans/schema
Deleting one attribute is
curl -X POST -H 'Content-type:application/json' -d '{ "delete-field":{"name":"trans"}}' http://arteika:8983/solr/trans/schema
Is there a way to delete all attributes from the schema?
At least in version 6.6 of the Schema API and up to the current version 7.5 of it, you can pass multiple commands in a single post (see 6.6 and 7.5 documenation, respectively). There are multiple accepted formats, but the most intuitive one (I think) is just passing an array for the action you want to perform:
curl -X POST -H 'Content-type: application/json' -d '{
"delete-field": [
{"name": "trans"},
{"name": "other_field"}
]
}' 'http://arteika:8983/solr/trans/schema'
So. How do we obtain the names of the fields we want to delete? That can be done by querying the Schema:
curl -X GET -H 'Content-type: application/json' 'http://arteika:8983/solr/trans/schema'
In particular, the copyFields, dynamicFields and fields keys in the schema object in the response.
I automated clearing all copy field rules, dynamic field rules and fields as follows. You can of course use any kind of script that is available to you. I used Python 3 (might work with Python 2, I did not test that).
import json
import requests
# load schema information
api = 'http://arteika:8983/solr/trans/schema'
r = requests.get(api)
# delete copy field rules
names = [(o['source'], o['dest']) for o in r.json()['schema']['copyFields']]
payload = {'delete-copy-field': [{'source': name[0], 'dest': name[1]} for name in names]}
requests.post(api, data = json.dumps(payload),
headers = {'Content-type': 'application/json'})
# delete dynamic fields
names = [o['name'] for o in r.json()['schema']['dynamicFields']]
payload = {'delete-dynamic-field': [{'name': name} for name in names]}
requests.post(api, data = json.dumps(payload),
headers = {'Content-type': 'application/json'})
# delete fields
names = [o['name'] for o in r.json()['schema']['fields']]
payload = {'delete-field': [{'name': name} for name in names]}
requests.post(api, data = json.dumps(payload),
headers = {'Content-type': 'application/json'})
Just a note: I received status 400 responses at first, with null error messages. Had a bit of a hard time figuring out how to fix those, so I'm sharing what worked for me. Changing the default of updateRequestProcessorChain in solrconfig.xml to false (default="${update.autoCreateFields:false}") and restarting the Solr service made those errors go away for me. The fields I was deleting were created automatically, that may have something to do with that.

Create inventory Square API / UNIREST Picture Upload

I want to create a product on my website and have it be created on square (which is working). However I also want to set the initial inventory which is seems there is no way to do it from the documentation. https://docs.connect.squareup.com/api/connect/v1/#post-inventory-variationid
If I go into my square account I can manually set up an initial amount, then query that entry and get the id and update it, but who wants to do anything manually. It defeats the purpose. Is there a way to create an inventory entry?
My second struggle is with uploading an image using unirest.
function uploadItemImage($itemId, $image_file)
{
global $accessToken, $locationId, $connectHost;
$requestHeaders = array
(
'Authorization' => 'Bearer ' . $accessToken,
'Accept' => 'application/json',
'Content-Type' => 'multipart/form-data;'
);
$request_body = array
(
'image_data'=>Unirest\Request\Body::file($image_file, 'text/plain', myproduct.jpg')
);
$response = Unirest\Request::post($connectHost . '/v1/' . $locationId . '/items/'.$itemId.'/image', $requestHeaders, $request_body);
print(json_encode($response->type, JSON_PRETTY_PRINT));
}
where $itemId is taken from the product created earlier and $image_file is the direct link to the file on my server
I keep getting this error...
> PHP Fatal error: Uncaught exception 'Unirest\Exception' with message
> 'couldn't open file "https://somewebsite/myPicture.jpg" ' in
> rootFolder/Unirest/Request.php:479 Stack trace:
> #0 rootFolder/Unirest/Request.php(292): Unirest\Request::send('POST', 'https://connect...', Array, Array, NULL, NULL)
> #1 rootFolder/
Any help is much appreciated!
Way to maximise the use of your question!
There is not currently a way to set initial inventory via API, but new item and inventory management APIs are in the works, read more on the Square Blog
I'm assuming that you are not literally using "https://somewebsite/myPicture.jpg" but it seems like unirest thinks you are trying to use a web url instead of getting a file from your filesystem. Try the following curl command and see if you can match up all the parts to unirest:
:)
curl --request POST \
--url https://connect.squareup.com/v1/XXXXXX/items/XXXXX/image \
--header 'authorization: Bearer sq0atp-XXXXX' \
--header 'cache-control: no-cache' \
--header 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
--form image_data=#/Users/ManuEng13/Desktop/test.png

JMeter: Body & File content

The WebAPI request has a POST method which expects Content body. I've tried to use both Parameters and Body options but I receive error responses - 'Invalid Request' with 400 Status code, etc.
JMeter request Sample Content Body:
{
"ParamA": 111,
"ParamB": "Char String",
"ParamC": "VarType"
}
OR
{ "ParamA": 111, "ParamB": "Char String", "ParamC": "VarType"}
Listener Request:
POST data:
--8vpH3B6WcV4f1La46_wccVi4c25lrLJaGcN--
Listener Response:
{"message":"The request is invalid.","modelState":{"value":["An error
has occurred."]}}
Any insight into viable options? Eventually, I'm planning on reading the Body string from a .csv file so I can parameterize the request. Reading from a .CSV file only reads the first line of the request body - for example: '{'
Any help would be greatly appreciated.
Best,
Ray
HTTP Request
Request
Uncheck in HTTP request the option:
Use multipart/form data for POST
Also check your CSV does not contain some data that contains the CSV separator which is '\t' by default.
Ensure it doesn't by changing separator to '|' for example if you're sure your JSON will never contain it.

Converting curl header format to Ruby's header format

I am trying to convert this request header into Ruby format:
curl http://example.com/api/v1/info -H 'Authorization: Token token="asklasjqwAiSo1s2dj5ias23dkl"'
I am trying to add it to an HTTP GET request:
http = Net::HTTP.new(endpoint, 80)
http.get(path, authorization_header_with_token)
How would I build the header I used in the cURL request to work with the Ruby request?
The header hash parameter should look like this:
http.get(path, {'Authorization' => 'Token token="asklasjqwAiSo1s2dj5ias23dkl"'})

Resources