I defined a proto in Golang shown below (and the corresponding API path) and am trying to test the POST API call as shown below, but am getting the following unexpected token error.
I verified that this error is from the award_map section (defined to be map<string, People>), in which People is an array of strings. What is the proper data format for award_map in the curl request? Thanks
Error:
{"code":3, "message":"proto: syntax error : unexpected token [","details":[]}
Post API request:
curl --header "Content-Type: application/json" \
-- request POST \
-- data '{"school_name":"MIT", "awards":["Summa-cum-laude", "Magna-cum-laude"], "award_map":{"Summa-cum-laude":[], "Magna-cum-laude": ["john", "mark"]}}' \
<POST_ENDPOINT>
Proto definitions:
message People {
repeated string names = 1;
}
message School {
string school_name = 1;
repeated string awards = 2;
map<string, People> award_map = 3;
}
The award_map values need to be full People messages, not just arrays of strings. Try this:
{
"school_name":"MIT",
"awards": [
"Summa-cum-laude",
"Magna-cum-laude"
],
"award_map": {
"Summa-cum-laude": {
"names": []
},
"Magna-cum-laude": {
"names": [
"john",
"mark"
]
}
}
}
Copy/paste version:
'{"school_name":"MIT","awards":["Summa-cum-laude","Magna-cum-laude"],"award_map":{"Summa-cum-laude":{"names":[]},"Magna-cum-laude":{"names":["john","mark"]}}}'
Note that it isn't possible to define something like map<string, repeated string>.
Related
I've been using REST APIs (both as a consumer and a developer of) for many years and am just starting to work with a GraphQL API for the first time, specifically, BurpSuite's Enterprise GraphQL API. I like it but I'm definitely missing a few critical concepts.
I am trying to hit their GetScan endpoint:
curl -k -i -H "Content-Type: application/json" -H "Authorization: <MY_API_KEY>" -X GET -d '{ "query": "query GetScan ($id: ID!) { scan(id: $id) { id status agent { id name } site_application_logins { login_credentials { label username } recorded_logins { label } } audit_items { id issue_counts { total } number_of_requests } scan_configurations { id name } } }"}' 'https://mybsee.example.com/graphql/v1'
For easier reading, that graphql query is also:
query GetScan ($id: ID!) {
scan(id: $id) {
id status
agent { id name }
site_application_logins {
login_credentials { label username }
recorded_logins { label }
}
audit_items {
id
issue_counts { total }
number_of_requests
}
scan_configurations { id name }
}
},
variables {
$agent: "Firefox"
}
Above <MY_API_KEY> has a real value when I run it (I can't post it here, obviously). Same with the URL (I have an on-premise version of BurpSuite running off, say, mybsee.example.com).
When I run that curl, I get the following output:
About to execute: query GetScan ($id: ID!) { scan(id: $id) { id status agent { id name } site_application_logins { login_credentials { label username } recorded_logins { label } } audit_items { id issue_counts { total } number_of_requests } scan_configurations { id name } } }
HTTP/2 200
date: Wed, 20 Oct 2021 23:33:50 GMT
x-frame-options: DENY
<lots of response headers omitted>
{"errors":[{"message":"Unexpected exception occurred. Check logs for more details.","extensions":{"code":77}}]}
I don't have access to the server logs, so I'm trying to rule out an issue on the client-side (bad call to GetScan on my part). Anything look wrong to anybody? Should I be injecting values to any of the fields in the query, such as id or status, etc.? If so, how (specifically) could I change the query to be valid? I'm also not sure if I need to append a "query" JSON field name into the actual query or not.
Thanks for any and all help here.
Update
I do realize that this is not a perfectly answerable problem, because the BurpSuite API is unfortunately proprietary and you cannot obtain an API key from them unless you purchase the product or go through a lot of rigamarole to get a free trial license.
But more importantly, I'm not looking for anyone to fish for me here, I'm hoping someone can teach me how to fish. In reality I need to integrate with a lot more GraphQL endpoints besides this one, but if someone can show me the proper way to construct one query, I can take it from there.
The safest way to do this is to use jq to generate correctly-escaped JSON with both your query and its parameters.
Building a function to automate that:
# usage: gqlQuery http://ENDPOINT/graphql "$query" arg1=val1 arg2=val2 ...
gqlQuery() {
local endpoint query postBody
local -a jqArgs=( )
endpoint=$1; shift || return
query=$1; shift || return
while (( $# )); do
case $1 in
# the variable name "query" is used for passing in, well, the query
query=*) echo "ERROR: variable name query is reserved" >&2; return 1;;
# 'foo=JSON:["bar", "baz"]' passes ["bar", "baz"] as a list, not a string
*=JSON:*) jqArgs+=( --argjson "${1%%=*}" "${1#*=}" ) ;;
# without JSON:, everything gets passed as a string to the child
*=*) jqArgs+=( --arg "${1%%=*}" "${1#*=}" ) ;;
# arguments without a = are not recognized
*) echo "ERROR: Argument $1 not well-formed" >&2; return 1;;
esac
shift
done
postBody=$(jq -nc \
--arg query "$query" \
"${jqArgs[#]}" \
'{ "query": $query, "variables": ($ARGS.named | del(.query)) }'
) || return
curl --fail -XPOST \
-H 'Content-Type: application/json' \
-d "$postBody" "$endpoint"
}
...you might use such a function as:
getScanQuery='query GetScan ($id: ID!) { scan(id: $id) { id status agent { id name } site_application_logins { login_credentials { label username } recorded_logins { label } } audit_items { id issue_counts { total } number_of_requests } scan_configurations { id name } } }'
myUrl=https://mybsee.example.com/graphql/v1
scanIdToRetrieve=1000 # replace with whatever is appropriate
result=$(gqlQuery "$myUrl" "$getScanQuery" id="$scanIdToRetrieve")
curl -k -i -H "Content-Type: application/json" -H "Authorization: Bearer <API_TOKEN>" -X POST -d "{ \"query\": \"$string\"}" '<API_URL>'
You did almost everything right. Just Change the HTTP Method to POST instead of GET. GET will try to fetch the whole page, POST on the other hand will interact with the GraphQL API.
Nevertheless stated in the GraphQL serving over HTTP page that GET Method should also work with queries under certain limtations. Refer the links for more Information.
Also consider visting other Webpages which lead me to this resolution;
Introduction to GraphQL on GitHub
Making GraphQL Requests using HTTP Methods
Burp Suite Enterprise GraphQL forum discussion with the same Error code of yours
I have been experimenting with Content Negotiation as backend versioning for my SpringBoot/Kotlin application. I have the following:
#GetMapping("/user", produces = [MediaType.APPLICATION_JSON_VALUE])
fun getUsers() {
//some code here
}
I have found this project combining accept" header and a "Accept-Version" custom header. I wonder whether this is the correct way of implementing a content negotiation approach and if not how can I fix it?
#GetMapping("/user", produces = [MediaType.APPLICATION_JSON_VALUE], headers = ["Accept-Version=$CUSTOM_ACCEPT_HEADER"])
fun getUsers() {
//some code here
}
object VersioningUtility {
const val CUSTOM_ACCEPT_HEADER = "vnd.sample.com-v1+json"
//here more constants as each controller can be versioned independently
}
Thank you
Yes, you can implement API versioning using content negotiation by having a custom header and header value as you have specified. However, since that is not a standard header, there are other scenarios which you might have to handle by yourself, such as:
default representation when the header is not present
exception scenarios when invalid media type values are passed as part of the header.
In case you are working with only json responses, the JSON API standard for content negotiation is to send the Accept header with the value application/vnd.api+json. Since Accept is a standard request header, using that is preferred. In case you need to handle other types of responses, you can still go ahead with the custom header.
You can implement content negotiation as below:
#RestController
class UserController {
#GetMapping("/users", headers = ["Accept=${VersioningUtility.VERSION_1_HEADER}"])
fun getUser(): ResponseEntity<Any> {
return ResponseEntity(listOf(User("Abraham Lincoln")), HttpStatus.OK)
}
#GetMapping("/users", headers = ["Accept=${VersioningUtility.VERSION_2_HEADER}"])
fun getNewUser(): ResponseEntity<Any> {
return ResponseEntity(listOf(NewUser(Name("Abraham", "Lincoln"))), HttpStatus.OK)
}
}
data class User(val name: String)
data class NewUser(val name: Name)
data class Name(val firstName: String, val lastName: String)
object VersioningUtility {
const val VERSION_1_HEADER = "application/vnd.v1+json"
const val VERSION_2_HEADER = "application/vnd.v2+json"
}
The above with enable you to have 2 versions of the GET /users endpoint with the Accept header.
When the curl request is made with v1 of the header value, the response would be according to the version v1
curl -L -X GET 'http://localhost:8080/users' \
-H 'Accept: application/vnd.v1+json'
[
{
"name": "Abraham Lincoln"
}
]
When the curl request is made with v2 of the header value, the response would be according to the version v2
curl -L -X GET 'http://localhost:8080/users' \
-H 'Accept: application/vnd.v2+json'
[
{
"name": {
"firstName": "Abraham",
"lastName": "Lincoln"
}
}
]
When an invalid header value is sent, it would respond with a 406 Not Acceptable
curl -L -X GET 'http://localhost:8080/users' \
-H 'Accept: application/vnd.abc+json'
{
"timestamp": "2020-04-01T18:33:16.393+0000",
"status": 406,
"error": "Not Acceptable",
"message": "Could not find acceptable representation",
"path": "/users"
}
When no Accept header is sent, it would respond with the default version, ie v1 here
curl -L -X GET 'http://localhost:8080/users'
[
{
"name": "Abraham Lincoln"
}
]
Even GitHub has implemented versioning with content negotiation in a similar way and you can have a look at that in their documentation.
I am trying to post some data to an elastic search server. I am using curl for this. The code is:
curl -X PUT https://username:password#someurl:443/index_name?pretty -H 'Content-Type: application/json' -d ' {"mappings": {"properties": {"my_field": {"type": "search_as_you_type"}}}}'
only the basic stuff of retrieving index info via cat/_indices?v works
Errors include
"Content-Type header [application/x-www-form-urlencoded] is not supported", curl: (6) Could not resolve host: application
curl: (3) [globbing] unmatched brace in column 1
curl: (3) [globbing] unmatched brace in column 1
curl: (3) [globbing] unmatched brace in column 1
curl: (3) [globbing] unmatched brace in column 1
curl: (3) [globbing] unmatched close brace/bracket in column 19
Error message is clear that you are using the wrong content type application/x-www-form-urlencoded, in very first line of error message.
"Content-Type header [application/x-www-form-urlencoded] is not
supported",
As given mapping seems to be copied from the official ES https://www.elastic.co/guide/en/elasticsearch/reference/7.x/search-as-you-type.html link, there only you can find the corresponding CURL format.
Pasting that format as well as screen-shot from the link, how can you copy the entire curl command.
here and I tried it in my local and it worked, note the only difference is content type which should be application/json
curl -X PUT "localhost:9500/so_index?pretty" -H 'Content-Type: application/json' -d'
{
"mappings": {
"properties": {
"my_field": {
"type": "search_as_you_type"
}
}
}
}
'
Figured it out after some trial and error. Here is the code used (after creating the index).
import requests
import json
url = 'http://localhost:9200/companies_list/_mapping'
headers1 = {'Content-Type':'application/json'}
data_obj = {
"properties": {
"date": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss"
}
}
}
response = requests.put(url, headers=headers1, data=json.dumps(data_obj))
print(response.content)
Here are some useful commands for people starting out on elasticsearch or upgrading to the latest version
import requests
import json
headers1 = {'Content-Type':'application/json'}
#create index
url = 'http://localhost:9200/my_index'
response = requests.put(url)
#view indexes
url = 'http://localhost:9200/_cat/indices?v'
response = requests.get(url)
#create mappings. if mappings are not created, then dynamic mapping will be created by elastic search
url = 'http://localhost:9200/my_index/_mapping'
response = requests.post(url, headers=headers1, data=json.dumps(data_obj))
data_obj = {
"properties": {
"names": {
"type": "search_as_you_type",
}
}
}
#insert data. if mappings have not been created initially, then dynamic mapping will be created by elastic search
url = 'http://localhost:9200/my_index/_doc/'
my_list = ['test name 1', 'test name 2', 'test name 3']
for i,j in enumerate(my_list):
response = requests.post(url+str(i), headers=headers1, data=json.dumps({"names":j}))
print(response.content)
#retrieve data.
url = 'http://localhost:9200/my_index/_doc/_search'
url = 'http://localhost:9200/my_index/_search' #works with both
data_obj = {
"query": {
"multi_match": {
"query": "test name 2",
"fields": [
"names",
"names._2gram",
"names._3gram"
]
}
}
}
response = requests.get(url, headers=headers1, data=json.dumps(data_obj))
print(response.content)
x = json.loads(response.content)
#x = json.loads(response.content.decode("utf-8")) #in case the response is in bytes and not str
for i in x["hits"]["hits"]:
print(i["_source"]["names"], i["_score"])
I am new to elastic search. Will keep updating the post with more basics
I've beat on this issue for what seems like a week and nothing online solves it. No data object is returned, not even "null". I'm converting from a working REST CRUD app to GraphQL. Not as easy as I expected.
My mutation in Playground:
mutation createMember($input: MemberInput!) {
createMember (input: $input) {
first_name
last_name
user_name
}
}
Playground Query Variables below: (Not in the headers section.)
{
"input": {
"first_name": "John",
"last_name": "Creston",
"user_name": "jc"
}
}
The schema: (The queries work fine and the Member type, an entity in TypeORM, works with them and full REST CRUD.)
input MemberInput {
first_name: String!
last_name: String!
user_name: String!
}
type Mutation {
createMember(input: MemberInput!): Member!
}
type Query {
getMembers: [Member]
getMember(member_id: Int!): Member!
checkUserName(user_name: String): Member
checkEmail(email: String): Member
}
I don't see how the resolver could be the problem for this error message but I'll add the Nestjs resolver:
#Mutation('createMember')
async createMember(#Args('input') input: MemberInput): Promise<Members> {
console.log('input in resolver: ', input);
return await this.membersService.addItem(input);
}
The service works with REST and the queries so that should be a problem. The console.log in the resolver never appears in terminal.
From the Copy CURL button:
curl 'http://localhost:3000/graphql' -H 'Accept-Encoding: gzip,
deflate, br' -H 'Content-Type: application/json' -H 'Accept:
application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin:
http://localhost:3000' --data-binary '{"query":"mutation
createMember($input: MemberInput!) {\n createMember (input: $input)
{\n first_name\n middle_initial\n last_name\n
user_name\n pitch\n main_skill_title\n \tskill_id_array\n
skills_comments\n other_skills\n links\n country\n
email\n member_status\n }\n}"}' --compressed
I had same problem, in variables i had comma at the end of last parameter. gql gave me same error.
change this:
{
"input": {
"param1": "val1",
"param2": "val2",
}
}
to this:
{
"input": {
"param1": "val1",
"param2": "val2" // no comma here
}
}
A bit complicated to explain but we've probably all be there. I was trying different "solutions" and the remains of one was accidentally left in my JSON data object. The full object has more properties than my brief one above. An array shouldn't be "[]" in JSON. I was getting tired and desperate last night. I changed the resolver to what I posted above but it didn't fix the problem because of the experiment in JSON caused the same error as before. So I was looking in the wrong places. Once I noticed and removed the quotes my full object's data was posted to the Postgres db.
The code above works.
It would be nice if there were more specific GraphQL errors. I suspected that this one is very general but that didn't help. It didn't indicate that the original problem was in the resolver and later that I had a data input error.
In my case, my code is implementing this plugin graphql-query-complexity for which there is a known bug: https://github.com/slicknode/graphql-query-complexity/issues/69.
Wrapping the plugin in a try/catch I am able to skip the error validation.
I am unable to retrieve all the fields, which i post for ProcedureRequest to a FHIR database.
Step 1: Post Request
curl -X POST https://fhir.dstu2.safetylabs.org/api/smartdstu2/open/ProcedureRequest \
-H 'Content-type: application/json+fhir' \
--data #ProcedureRequest.js
Result of POST
{
"resourceType":"OperationOutcome",
"text":{
"status":"generated",
"div":"<div xmlns=\"http://www.w3.org/1999/xhtml\"><h1>Operation Outcome</h1><table border=\"0\"><tr><td style=\"font-weight: bold;\">information</td><td>[]</td><td><pre>Successfully created resource "ProcedureRequest/7660/_history/1" in 7ms</pre></td>\n\t\t\t\t\t\n\t\t\t\t\n\t\t\t</tr>\n\t\t</table>\n\t</div>"
},
"issue":[
{
"severity":"information",
"code":"informational",
"diagnostics":"Successfully created resource \"ProcedureRequest/7660/_history/1\" in 7ms"
}
]
}
This request was successful.
The JSON was validated from JSON validator, and It is also validated from hapi test Server from given link
" https://fhirtest.uhn.ca/resource?serverId=home_21&pretty=false&resource=ProcedureRequest"
Step 2: Retrieve Request
curl -X GET https://fhir.dstu2.safetylabs.org/api/smartdstu2/open/ProcedureRequest/7660
Results of GET
{
"resourceType":"ProcedureRequest",
"id":"7660",
"meta":{
"versionId":"1",
"lastUpdated":"2017-11-18T13:49:23.000+00:00"
},
"text":{
"status":"generated",
"div":"<div xmlns=\"http://www.w3.org/1999/xhtml\"><p><b>Generated Narrative with Details</b></p><p><b>id</b>: subrequest</p><p><b>definition</b>: Protocol for alergies</p><p><b>basedOn</b>: Original Request</p><p><b>replaces</b>: Previous allergy test</p><p><b>requisition</b>: A13848392</p><p><b>status</b>: active</p><p><b>intent</b>: instance-order</p><p><b>priority</b>: routine</p><p><b>code</b>: Peanut IgG <span>(Details : {LOINC code '35542-0' = 'Peanut IgG Ab [Mass/volume] in Serum)</span></p><p><b>subject</b>: <a>Patient/dicom</a></p><p><b>occurrence</b>: 08/05/2013 9:33:27 AM</p><h3>Requesters</h3><table><tr><td>-</td><td><b>Agent</b></td></tr><tr><td>*</td><td><a>Dr. Adam Careful</a></td></tr></table><p><b>performerType</b>: Nurse <span>(Details : {[not stated] code 'null' = 'null', given as 'Qualified nurse'})</span></p><p><b>reasonCode</b>: Check for Peanut Allergy <span>(Details )</span></p><p><b>bodySite</b>: Right arm <span>(Details : {[not stated] code 'null' = 'null', given as 'Right arm'})</span></p></div>"
},
"subject":{
"reference":"Patient/SL88812358"
},
"code":{
"coding":[
{
"system":"http://loinc.org",
"code":"35542-0"
}
],
"text":"Peanut IgG"
},
"bodySite":[
{
"coding":[
{
"display":"Right arm"
}
],
"text":"Right arm"
}
],
"status":"active",
"priority":"routine"
}
Note that Retrieve did not retrieve the following fields
occurrenceTiming
occurrenceDateTime
performerType
reasonCode
requester
Question : How do I request all the fields I posted?
That's because you posted to a DSTU2 server and those elements don't exist in DSTU2 - so the server ignored them (as servers are allowed to do with unrecognized elements). If you post to a DSTU3 server, the elements should be stored - if the server supports them, which most test servers do.