Curl/GraphQL command failing with 200 - shell

I am trying to write a shell script that executes a curl against a GraphQL API and I've never interacted with GQL before. I am getting some strange errors and although I understand this community doesn't have access to the GQL server I was hoping someone could take a look at the script and make sure I'm not doing anything flagrantly wrong syntax-wise (both in the shell script layer as well as the GQL query itself).
My script:
#!/bin/bash
BSEE_WEB_SERVER_DNS=https://mybsee.example.com
BSEE_API_KEY=abc123
siteId=1
scanConfigId=456
runScanQuery='mutation CreateScheduleItem { create_schedule_item(input: {site_id: "$siteId" scan_configuration_ids: "$scanConfigId"}) { schedule_item { id } } }'
runScanVariables='{ "input": "site_id": $scanId }}'
runScanOperationName='CreateScheduleItem'
curl -i --request POST \
--url $BSEE_WEB_SERVER_DNS/graphql/v1 \
--header "Authorization: $BSEE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{"query":"$runScanQuery","variables":{$runScanVariables},"operationName":"${runScanOperationName}"}'
And the output when I run the script off the terminal:
HTTP/2 200
<OMITTED RESPONSE HEADERS>
{"errors":[{"message":"Invalid JSON : Unexpected character (\u0027$\u0027 (code 36)): was expecting double-quote to start field name, Line 1 Col 38","extensions":{"code":3}}]}%
I am omitting the HTTP response headers for security and brevity reasons.
I am wondering if my use of quotes/double-quotes is somehow wrong, or if there is anything about the nature of the GQL query itself (via curl) that looks off to anyone.
I verified with the team that manages the server that the HTTP 200 OK response code is correct. 200 shows that the request succeeded to the GQL API, but that GQL is responding with this error to indicate the query itself is incorrect.

We need to modify the GraphQL bits and fix the bash string quoting.
runScanQuery GraphQL operation
Fix the GraphQL syntax. Use a GraphQL operation name CreateScheduleItem with variables $site_id in the arguments input: { site_id: $siteId, scan_configuration_ids: $scanConfigId:
mutation CreateScheduleItem($site_id: String!, $scanConfigId: String!) {
create_schedule_item(
input: { site_id: $siteId, scan_configuration_ids: $scanConfigId }
) {
schedule_item {
id
}
}
}
runScanVariables: JSON
Our mutation expects two variables, which GraphQL will substitute into CreateScheduleItem($site_id: String!, $scanConfigId: String!). Provide the GraphQL variables as JSON. Here is the expected output after bash variable substitution:
{ "$site_id": "1", "$scanConfigId": "456" }
Get the bash quoting right
Finally, translate the inputs into bash-friendly syntax:
runScanQuery='mutation CreateScheduleItem($site_id: String!, $scanConfigId: String!) { create_schedule_item(input: {site_id: $siteId scan_configuration_ids: $scanConfigId}) { schedule_item { id } } }'
runScanVariables='{"$site_id":"'"$siteId"'","$scanConfigId":"'"$scanConfigId"'"}' # no spaces!
runScanOperationName='CreateScheduleItem'
data='{"query":"'"$runScanQuery"'","variables":'$runScanVariables',"operationName":"'"$runScanOperationName"'"}'
Check our bash formats. Paste the terminal output into a code-aware editor like VSCode. Expect the editor to parse the output correctly.
echo $runScanQuery # want string in graphql format
echo $runScanVariables # want JSON
echo $data # want JSON
Edit: add a public API example
Here's a complete working example using the public Star Wars API:
#!/bin/bash
filmId=1
data='{"query":"query Query($filmId: ID) { film(filmID: $filmId) { title }}","variables":{"filmId":"'"$filmId"'"}}'
curl --location --request POST 'https://swapi-graphql.netlify.app/.netlify/functions/index' \
--header 'Content-Type: application/json' \
--data "$data"
Responds with {"data":{"film":{"title":"A New Hope"}}}.

In GraphQL it's normal to always have 200 status code; client must check response body searching for failures.
The reason is simple: In REST, http is part of the protocol and status code has semantics but in GraphQL http is not part of the protocol, you can have GraphQL over serveral transport protocols:
http: typical scenario docs
WebSocket: does not provide any "status code like" payload. sample
MQTT: does not provide any "status code like" payload
...
The only way that server tells you something (even failures) is the body.
In your case I suggest you jq to parse json via bash script searching error property.

Your error is completely unrelated to GraphQL. You really have wrong JSON.
Error message says Unexpected character (\u0027$\u0027 (code 36)): was expecting double-quote to start field name, Line 1 Col 38",
You can replace escaped \u0027 with apostrophe and you will get
Unexpected character ('$' (code 36)): was expecting double-quote to start field name, Line 1 Col 38",
So it hates dollar sign at position 38 in what you send as data to curl
data='{"query":"'"$runScanQuery"'","variables":'$runScanVariables'
^
this
First - all field names and values in JSON should be wrapped with double quotes, not single.
Second - if you want curl to expand env variable, put it to double quotes, not single.

Related

How to send a POST request with Kotlin/Native using libcurl?

I'm trying to send a JSON through a POST request with a Kotlin/Native application using libcurl. I'm working on a Windows 11 machine, and the endpoint lies under a Spring Boot (version 2.7.8) backend written with Kotlin and Java 11.
The following is the code I wrote to accomplish this task.
import kotlinx.cinterop.*
import libcurl.*
fun main() {
val curl = curl_easy_init()
curl?.let {
var headers: CValuesRef<curl_slist>? = null
headers = curl_slist_append(headers, "Content-Type: application/json")
setCurl(curl, headers)
val res = curl_easy_perform(curl)
if (res != CURLE_OK) println("curl_easy_perform() failed ${curl_easy_strerror(res)?.toKString()}")
curl_easy_cleanup(curl)
curl_slist_free_all(headers)
} ?: println("curl_easy_init() failed to return curl easy handle")
}
private fun setCurl(curl: COpaquePointer?, headers: CPointer<curl_slist>?) {
val body = "{ JSON object }"
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers)
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body)
curl_easy_setopt(curl, CURLOPT_URL, "http://localhost:8080/dishes/add/")
}
The JSON string I need to send represents a simple Dish object having three fields: name, type and description.
These are the attempts I made to correctly format the JSON string to initialize the val body and the relative outputs that the Spring Boot endpoint return:
{\"name\":\"Fish\",\"type\":\"Second\",\"description\":\"Fry\"} => HTTP 200;
{\"name\":\"Fish fry\",\"type\":\"Second course\",\"description\":\"Lots of fresh fish to delight the palate with an excellent, fried second course\"} => HTTP 400: JSON parse error: Unexpected character (' ' (code 160)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false');
{\\\"name\\\":\\\"Fish fry\\\",\\\"type\\\":\\\"Second course\\\",\\\"description\\\":\\\"Lots of fresh fish to delight the palate with an excellent, fried second course\\\"} => HTTP 400: JSON parse error: Illegal character ((CTRL-CHAR, code 0)): only regular white space (\r, \n, \t) is allowed between tokens;
"\"{\\\"name\\\":\\\"Fish fry\\\",\\\"type\\\":\\\"Second course\\\",\\\"description\\\":\\\"Lots of fresh fish to delight the palate with an excellent, fried second course\\\"}\"" => HTTP 400: JSON parse error: Unexpected character (' ' (code 160)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
From the above tests, I can effectively POST something on the backend only with the first body string, but if I try to send a longer one with even the same format (see the second one), the POST request fails too.
Besides, I also tried to execute a POST request using Postman and the terminal (with the following command: curl -H "Content-Type: application/json" -X POST -d "{\"name\":\"Fish fry\",\"type\":\"Second course\",\"description\":\"Lots of fresh fish to delight the palate with an excellent, fried second course\"}" http://localhost:8080/dishes/add/), and with both, I can correctly send the POSTs. And since using the commands prompt, curl accepts the last body JSON I tried, maybe it could be the right approach to format the string, but I'm not sure.
What am I missing?
Thanks for your precious time!
UPDATE 1:
I just discovered the --libcurl curl parameter, which lets you convert a curl command into libcurl code.
Using this helpful tool, I converted my working cmd POST request
curl -H "Content-Type: application/json" -X POST -d "{\"name\":\"Fish fry\",\"type\":\"Second course\",\"description\":\"Lots of fresh fish to delight the palate with an excellent, fried second course\"}" http://localhost:8080/dishes/add/
Into the following, Kotlin adapted, C snippet:
private fun setCurl(hnd: COpaquePointer?, certPath: String, url: String) {
var headers: CValuesRef<curl_slist>? = null
headers = curl_slist_append(headers, "Content-Type: application/json")
curl_easy_setopt(hnd, CURLOPT_BUFFERSIZE, 102400L)
curl_easy_setopt(hnd, CURLOPT_URL, "http://localhost:8080/dishes/add/")
curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 1L)
curl_easy_setopt(
hnd,
CURLOPT_POSTFIELDS,
"{\"name\":\"Fish fry\",\"type\":\"Second course\",\"description\":\"Lots of fresh fish to delight the palate with an excellent, fried second course\"}"
)
val postFieldSize: curl_off_t = 138
curl_easy_setopt(hnd, CURLOPT_POSTFIELDSIZE_LARGE, postFieldSize)
curl_easy_setopt(hnd, CURLOPT_HTTPHEADER, headers)
curl_easy_setopt(hnd, CURLOPT_USERAGENT, "curl/7.83.1")
curl_easy_setopt(hnd, CURLOPT_MAXREDIRS, 50L)
curl_easy_setopt(hnd, CURLOPT_CUSTOMREQUEST, "POST")
curl_easy_setopt(hnd, CURLOPT_FTP_SKIP_PASV_IP, 1L)
curl_easy_setopt(hnd, CURLOPT_TCP_KEEPALIVE, 1L)
}
But still, my Kotlin/Native application failed to execute the request with Spring Boot returning the same error: Unexpected character (' ' (code 160)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false').
At this time, I ran out of ideas. Please, let me know about any other solutions.
UPDATE 2:
Since the first val body initialization was the only one to be successful (despite the body string having to be small), I started to do some other tests with that type of formatted JSON, so I found that the POST request is successful if the entire string does not cross the 63 chars of length otherwise, the Spring Boot endpoint fires the error regarding the code 160 unexpected character.
The body string I'm currently using is the following one, which length is exactly 63 chars.
{\"name\":\"Fettuccine Alfredo\",\"type\":\"Main co\",\"description\":\"\"}
I don't know why this situation is happening, and I'm very frustrated.
Every tip of advice is much appreciated.

Testing REST routes with cURL - data not saving

I am running through the tutorials in Open REST Routes but I am now stuck at posting data to the mongodb posts collection.
When I cURL it executes without error but my title and link are not saved and the upvotes defaults to 0 as defined in the schema.
My Post schema is:
var mongoose = require('mongoose');
var PostSchema = new mongoose.Schema({
title: String,
link: String,
upvotes: {type: Number, default: 0},
comments: [{type: mongoose.Schema.Types.ObjectId, ref: 'Comment'}]
});
mongoose.model('Post', PostSchema);
I get the result below:
curl http://localhost:3000/posts -d '{"title":"Go Bigdadi! Nicely done!!!","link":"http://www.foo.com","upvotes":2}'
{"upvotes":0,"comments":[],"_id":"5b740c875bdf6a326c677cd3","__v":0}`
Please suggest where I am likely to be messing it up.
I think I have figured it out....
I tried changing the parameters round a bit and was able to get the expected result.
I tried:
curl http://localhost:3000/posts -i -X POST -d "title=Bigdadi Rules Posts&link=http://www.foo.com&upvotes=2"
It works!!!

How to delete all attributes from the schema in solr?

Deleting all documents from solr is
curl http://localhost:8983/solr/trans/update?commit=true -d "<delete><query>*:*</query></delete>"
Adding a (static) attribute to the schema is
curl -X POST -H 'Content-type:application/json' --data-binary '{ "add-field":{"name":"trans","type":"string","stored":true, "indexed":true},}' http://localhost:8983/solr/trans/schema
Deleting one attribute is
curl -X POST -H 'Content-type:application/json' -d '{ "delete-field":{"name":"trans"}}' http://arteika:8983/solr/trans/schema
Is there a way to delete all attributes from the schema?
At least in version 6.6 of the Schema API and up to the current version 7.5 of it, you can pass multiple commands in a single post (see 6.6 and 7.5 documenation, respectively). There are multiple accepted formats, but the most intuitive one (I think) is just passing an array for the action you want to perform:
curl -X POST -H 'Content-type: application/json' -d '{
"delete-field": [
{"name": "trans"},
{"name": "other_field"}
]
}' 'http://arteika:8983/solr/trans/schema'
So. How do we obtain the names of the fields we want to delete? That can be done by querying the Schema:
curl -X GET -H 'Content-type: application/json' 'http://arteika:8983/solr/trans/schema'
In particular, the copyFields, dynamicFields and fields keys in the schema object in the response.
I automated clearing all copy field rules, dynamic field rules and fields as follows. You can of course use any kind of script that is available to you. I used Python 3 (might work with Python 2, I did not test that).
import json
import requests
# load schema information
api = 'http://arteika:8983/solr/trans/schema'
r = requests.get(api)
# delete copy field rules
names = [(o['source'], o['dest']) for o in r.json()['schema']['copyFields']]
payload = {'delete-copy-field': [{'source': name[0], 'dest': name[1]} for name in names]}
requests.post(api, data = json.dumps(payload),
headers = {'Content-type': 'application/json'})
# delete dynamic fields
names = [o['name'] for o in r.json()['schema']['dynamicFields']]
payload = {'delete-dynamic-field': [{'name': name} for name in names]}
requests.post(api, data = json.dumps(payload),
headers = {'Content-type': 'application/json'})
# delete fields
names = [o['name'] for o in r.json()['schema']['fields']]
payload = {'delete-field': [{'name': name} for name in names]}
requests.post(api, data = json.dumps(payload),
headers = {'Content-type': 'application/json'})
Just a note: I received status 400 responses at first, with null error messages. Had a bit of a hard time figuring out how to fix those, so I'm sharing what worked for me. Changing the default of updateRequestProcessorChain in solrconfig.xml to false (default="${update.autoCreateFields:false}") and restarting the Solr service made those errors go away for me. The fields I was deleting were created automatically, that may have something to do with that.

What is the Proper Method of Assigning JSON within a Ruby Variable?

The following JSON is a transaction what will be sent to the Ripple Network to query accounts that hold cryptographic assets at a Gateway (somewhat like a bank, more like a trust account between its clients). This script is to be used in conjunction with PHP to fetch a Gateway's issued balances and ignored it's hot-wallet or day-to-day operations wallet. My question is what is the proper way to:
a. Assign JSON within a Ruby variable?
b. What is the best way to escape double quotes and deal with newlines where brackets and square brackets occur within the JSON syntax?
The JSON follows:
ripple_path="/home/rippled/build/rippled"
conf = "--conf /etc/rippled/rippled.cfg"
puts "About to set the JSON lines "
gatewayStart = "\"method\": \"gateway_balances\","
paramsLine = "\"params\": [ {"
accountLine = "\"account\": \"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH\","
hotwalletLine = "\"hotwallet\": \"rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C\","
liLine = "\"ledger_index\": \"validated\","
strictLine = "\"strict\": "
trueLine = true
endLine = " } ] }"
balancesLine = "#{gatewayStart} #{paramsLine} #{accountLine} #>{hotwalletLine} #{liLine} #{strictLine} #{trueLine} #{endLine}"
lineString = "#{balancesLine.to_s}"
linetoJSON = "#{lineString}"
puts "linetoJSON: #{linetoJSON} "
cmd2=`#{ripple_path} #{conf} json gateway_balances #{linetoJSON}`
cmder="#{ripple_path} #{conf} json gateway_balances #{linetoJSON}"
puts "Done."
The output is:
root#xagate:WorkingDirectory# ruby gatewaybal.rb
About to set the JSON lines
linetoJSON: "method": "gateway_balances", "params": [ { "account":
"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH", "hotwallet": "rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C", "ledger_index": "validated", "strict":rue } ] }
Loading: "/etc/rippled/rippled.cfg"
rippled [options] <command> <params>
General Options:
-h [ --help ] Display this message.
.....
Done.
It is noteworthy that this command also returns a badSyntax error when executed manually via the command line. Please see here for the mirror of this issue raised on the ripple forums.
jsonLine = "'{ \"account\": \"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH\", \"hotwallet\": \"rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C\", \"ledger_index\": \"validated\", \"strict\": true }'"
Is the proper way to assign this JSON within a single variable; this solution was provided by JoelKatz. The completed code is now available on GitHub.

Couchdb view Queries

Could you please help me in creating a view. Below is the requirement
select * from personaccount where name="srini" and user="pup" order by lastloggedin
I have to send name and user as input to the view and the data should be sorted by lastloggedin.
Below is the view I have created but it is not working
{
"language": "javascript",
"views": {
"sortdatetimefunc": {
"map": "function(doc) {
emit({
lastloggedin: doc.lastloggedin,
name: doc.name,
user: doc.user
},doc);
}"
}
}
}
And this the curl command iam using:
http://uta:password#localhost:5984/personaccount/_design/checkdatesorting/_view/sortdatetimefunc?key={\"name:srini\",\"user:pup\"}
My Questions are
As sorting will be done on key and I want it on lastloggedin so I have given that also in emit function.
But iam passing name and user only as parameters. Do we need to pass all the parameters which we give it in key?
First of all I want to convey to you for the reply, I have done the same and iam getting errors. Please help
Could you please try this on your PC, iam posting all the commands :
curl -X PUT http://uta:password#localhost:5984/person-data
curl -X PUT http://uta:password#localhost:5984/person-data/srini -d '{"Name":"SRINI", "Idnum":"383896", "Format":"NTSC", "Studio":"Disney", "Year":"2009", "Rating":"PG", "lastTimeOfCall": "2012-02-08T19:44:37+0100"}'
curl -X PUT http://uta:password#localhost:5984/person-data/raju -d '{"Name":"RAJU", "Idnum":"456787", "Format":"FAT", "Studio":"VFX", "Year":"2010", "Rating":"PG", "lastTimeOfCall": "2012-02-08T19:50:37+0100"}'
curl -X PUT http://uta:password#localhost:5984/person-data/vihar -d '{"Name":"BALA", "Idnum":"567876", "Format":"FAT32", "Studio":"YELL", "Year":"2011", "Rating":"PG", "lastTimeOfCall": "2012-02-08T19:55:37+0100"}'
Here's the view as you said I created :
{
"_id": "_design/persondestwo",
"_rev": "1-0d3b4857b8e6c9e47cc9af771c433571",
"language": "javascript",
"views": {
"personviewtwo": {
"map": "function (doc) {\u000a emit([ doc.Name, doc.Idnum, doc.lastTimeOfCall ], null);\u000a}"
}
}
}
I have fired this command from curl command :
curl -X GET http://uta:password#localhost:5984/person-data/_design/persondestwo/_view/personviewtwo?startkey=["SRINI","383896"]&endkey=["SRINI","383896",{}]descending=true&include_docs=true
I got this error :
[4] 3000
curl: (3) [globbing] error: bad range specification after pos 99
[5] 1776
[6] 2736
[3] Done descending=true
[4] Done(3) curl -X GET http://uta:password#localhost:5984/person-data/_design/persondestwo/_view/personviewtwo?startkey=["SRINI","383896"]
[5] Done endkey=["SRINI","383896"]
I am not knowing what this error is.
I have also tried passing the parameters the below way and it is not helping
curl -X GET http://uta:password#localhost:5984/person-data/_design/persondestwo/_view/personviewtwo?key={\"Name\":\"SRINI\",\"Idnum\": \"383896\"}&descending=true
But I get different errors on escape sequences
Overall I just want this query to be satisfied through the view :
select * from person-data where Name="SRINI" and Idnum="383896" orderby lastTimeOfCall
My concern is how to pass the multiple parameters from curl command as I get lot of errors if I do the above way.
First off, you need to use an array as your key. I would use:
function (doc) {
emit([ doc.name, doc.user, doc.lastLoggedIn ], null);
}
This basically outputs all the documents in order by name, then user, then lastLoggedIn. You can use the following URL to query.
/_design/checkdatesorting/_view/sortdatetimefunc?startkey=["srini","pup"]&endkey=["srini","pup",{}]&include_docs=true
Second, notice I did not output doc as the value of your query. It takes up much more disk space, especially if your documents are fairly large. Just use include_docs=true.
Lastly, refer to the CouchDB Wiki, it's pretty helpful.
I just stumbled upon this question. The errors you are getting are caused by not escaping this command:
curl -X GET http://uta:password#localhost:5984/person-data/_design/persondestwo/_view/personviewtwo?startkey=["SRINI","383896"]&endkey=["SRINI","383896",{}]descending=true&include_docs=true
The & character has a special meaning on the command-line and should be escaped when part of an actual parameter.
So you should put quotes around the big URL, and escape the quotes inside it:
curl -X GET "http://uta:password#localhost:5984/person-data/_design/persondestwo/_view/personviewtwo?startkey=[\"SRINI\",\"383896\"]&endkey=[\"SRINI\",\"383896\",{}]descending=true&include_docs=true"

Resources