I am trying to make a post request in powershell using curl. But I seem to be unfortunately getting this error.
I have tried removing spaces here and there, and googling the problem but have not found a solution.
curl.exe -X 'POST' -H #{'Content-Type'='application/json'; 'accept'='application/json'} -d \"{\"name\":\"test3\", \"auto_init\":true, \"default_branch\": \"master\", \"description\": \"My Test\", \"gitignores\": \"Vim\", \"issue_labels\":\"Default\", \"license\": \"DOC\", \"name\":\"test2\", \"private\":false, \"readme\":\"Default\",\"template\":false,\"trust_model\":\"default\"}\" http://localhost:3000/api/v1/user/repos?access_token=c11ceb97fa594a7e6c4b5519e4327908be3274b9
Re -H:
curl.exe is an external program, which means that you cannot meaningfully pass a PowerShell hashtable (#{ ... }) as an argument, because it will (uselessly) be passed as literal string System.Collections.Hashtable.
Instead, pass strings, as multiple -H options, each in the form '<name>: <value>'
Re -d:
PowerShell's escape character is ` (the so-called backtick), not \.
Since your argument is to be passed verbatim (contains no variable references to be interpolated), use a verbatim (single-quoted) string ('...').
However: The sad reality as of PowerShell 7.2 is that an extra, manual layer of \-escaping of embedded " characters is required in arguments passed to external programs. This may get fixed in a future version, which may require opt-in. See this answer to the linked duplicate for details.
To put it all together:
curl.exe -H 'Content-Type: application/json' -H 'accept: application/json' -d '{\"name\":\"test3\", \"auto_init\":true, \"default_branch\": \"master\", \"description\": \"My Test\", \"gitignores\": \"Vim\", \"issue_labels\":\"Default\", \"license\": \"DOC\", \"name\":\"test2\", \"private\":false, \"readme\":\"Default\",\"template\":false,\"trust_model\":\"default\"}' 'http://localhost:3000/api/v1/user/repos?access_token=c11ceb97fa594a7e6c4b5519e4327908be3274b9'
Note: I've omitted -X 'POST', because, as Daniel Stenberg notes, a POST request is implied when you use the -d option.
Related
So, I've been beating my head around this for a few days and could use some help.
Initially, I found the gitlab blog post from 2017: https://about.gitlab.com/blog/2017/09/05/how-to-automatically-create-a-new-mr-on-gitlab-with-gitlab-ci/
However it seems like a lot of it is outdated(?) but I was able to use POSTMAN to carve down a working model (using token as a parameter instead of a header, etc.)
Here's the thing though, I cannot get the Merge Request to trigger and I believe it's due to --data (-d) not interpolating the variables.
I've hardcoded the payload with the values exactly, and pushed to my branch and it generated the MR just fine.
But any method I use to try and incorporate variables will result in the cURL going through, but no MR being generated.
What's more, the GL Runner does not output the response - so it's difficult for me to figure out exactly what the issue is.
I've tried escaping with back slashes, I've tried single quote setups with escaping the single quotes and doing inner double quotes with the ${} variables, but nothing.
I've validated that the variables being sent in from the yml exists (echoing them out before the curl call and the body setup), but I'm just lost at this point.
I've still yet to try JQ, but I'm not sure if the runner has it installed, but I'll find out when I return back to test.
Any help would be lovely.
General format of the scripts code:
#!/usr/bin/env bash
TARGET_BRANCH="test"
echo "Host: ${HOST}"
echo "Project Id: ${CI_PROJECT_ID}
echo "SOURCE BRANCH: ${CI_COMMIT_REF_NAME}
echo "Target Branch: ${TARGET_BRANCH}
BODY="{
\"id\": ${CI_PROJECT_ID},
\"source_branch\": \"${CI_COMMIT_REF_NAME}\",
\"target_branch\": \"${TARGET_BRANCH}\",
\"remove_source_branch\": false,
\"title\": \"WIP: ${CI_COMMIT_REF_NAME}\",
}";
echo "Body: ${BODY}"
# Create MR
curl -X POST https://${HOST}/api/v4/projects/${CI_PROJECT_ID}/merge_requests?private_token${PRIVATE_TOKEN}"\
--header "PRIVATE_TOKEN: ${PRIVATE_TOKEN}"\
--header "Content-Type: application/json"\
--data "${BODY}"
Now I want to note, that after testing this and seeing the curl go through, but no MR generated, I modified the --data value to go from ${BODY} to:
(assume to be valid value, just obscuring for reasons)
--data '{
"id": <x>,
"source_branch": "dev",
"target_branch": "test",
"remove_source_branch": false,
"title":"WIP: dev"
}'
And this worked! I the MR generated just fine.
So I I then tested by replacing the single quote with double quotes, and escaping the inner double quotes but still keeping the hardcoded values. Back to no MR being generated.
I'm quote stumped - I even went back to the BODY=... block, and tried to reformat that.
Then I went to the outer single, inner double, escaping
'{
"id": '"${CI_PROJECT_ID}"',
...
'}
and still nothing.
So I'm taking a break after all the attempts and speaking with some teammates to no avail, and throwing a hail marry Stack Overflow for some advice.
Added note:
This is not a typical application project repo, so any discussion on "this isn't a good standard practice, use issues as an anchor point for generating MRs" doesn't apply when the use of the repo will not be using issues, have minimum user hands on, etc.
I value those kinds of insights, but again, the usecase for this repo is not an actual "project", so to speak.
Thank you
I'm a bit confused about what it is you're asking. It sounds like you got it to work just fine by passing the json string directly to curl's data flag, and you're just wondering why the other attempts didn't work. If that's the case, here's what I see which would make your other attempts fail:
# ...
BODY="{
\"id\": ${CI_PROJECT_ID},
\"source_branch\": \"${CI_COMMIT_REF_NAME}\",
\"target_branch\": \"${TARGET_BRANCH}\",
\"remove_source_branch\": false,
\"title\": \"WIP: ${CI_COMMIT_REF_NAME}\",
}";
echo "Body: ${BODY}"
# Create MR
curl -X POST https://${HOST}/api/v4/projects/${CI_PROJECT_ID}/merge_requests?private_token${PRIVATE_TOKEN}"\
--header "PRIVATE_TOKEN: ${PRIVATE_TOKEN}"\
--header "Content-Type: application/json"\
--data "${BODY}"
In this first option, the variable looks just fine, but when you're using it in your curl command, you're quoting the variable (ie, "$BODY"). There's nothing wrong with this when you're doing concatenation, but here it's likely causing a syntax issue since when the variable is extracted, the result is a json string that's doubly quoted:
""{
\"id\": ${CI_PROJECT_ID},
\"source_branch\": \"${CI_COMMIT_REF_NAME}\",
\"target_branch\": \"${TARGET_BRANCH}\",
\"remove_source_branch\": false,
\"title\": \"WIP: ${CI_COMMIT_REF_NAME}\",
}""
I would try passing the $BODY variable without any quotes:
# Create MR
curl -X POST https://${HOST}/api/v4/projects/${CI_PROJECT_ID}/merge_requests?private_token${PRIVATE_TOKEN}"\
--header "PRIVATE_TOKEN: ${PRIVATE_TOKEN}"\
--header "Content-Type: application/json"\
--data $BODY
For the third attempt, I assume there's a typo in your question since you have what should be the final single-quote in front of the closing brace. The other issue is that you're terminating the single-quoted string in the middle of the json:
'{
"id": '"${CI_PROJECT_ID}"',
...
}'
Since this starts with a single-quote, the first single quote it sees will terminate the string. The first single quote is around "${CI_PROJECT_ID}"', so the string being passed to curl is:
'{
"id": '
To clarify why this curl command isn't throwing an exit code, this is still a legal string, and is perfectly fine to curl, so it sends it on to GitLab API. When parsing the data however, GitLab can't parse it, so nothing happens over there. It would result in a 4xx HTTP code however, but as you mentioned, you can't see some of that information in the job output.
To fix this, I'd remove the single quotes around $CI_PROJECT_ID:
'{
"id": "${CI_PROJECT_ID}",
...
}'
or, since $CI_PROJECT_ID is and always will be an integer, you can remove all the quotes around it:
'{
"id": $CI_PROJECT_ID,
...
}'
I have to insert many data in my application and through the graphical interface it takes many time. For this reason I want to create a bash script and make the requests through curl using the REST API (I have to manually specify the id).
The problem is that i get the error: The server refused this request because the request entity is in a format not supported by the requested resource for the requested method.
Here is the code
#!/bin/bash
for i in {1..1}
do
CURL='/usr/bin/curl -X POST'
RVMHTTP="http://192.168.1.101:8080/sitewhere/api/devices
-H 'accept:application/json'
-H 'content-type:application/json'
-H 'x-sitewhere-tenant:sitewhere1234567890'
--user admin:password"
DATA=" -d '{\"hardwareId":\"$i",\"siteToken\":\"4e6913db-c8d3-4e45-9436-f0a99b502d3c\",\"specificationToken\":\"82043707-9e3d-441f-bdcc-33cf0f4f7260\"}'"
# or you can redirect it into a file:
$CURL $RVMHTTP $DATA >> /home/bluedragon/Desktop/tokens
done
The format of my request has to be json
#!/usr/bin/env bash
rvmcurl() {
local url
url="http://192.168.1.101:8080/sitewhere/${1#/}"
shift || return # function should fail if we weren't passed at least one argument
curl -XPOST "${rvm_curl_args[#]}" "$url" "$#"
}
i=1 # for testing purposes
rvm_curl_args=(
-H 'accept:application/json'
-H 'content-type:application/json'
-H 'x-sitewhere-tenant:sitewhere1234567890'
--user admin:password
)
data=$(jq -n --arg hardwareId "$i" '
{
"hardwareId": $hardwareId,
"siteToken": "4e6913db-c8d3-4e45-9436-f0a99b502d3c",
"specializationToken": "82043707-9e3d-441f-bdcc-33cf0f4f7260"
}')
rvmcurl /api/devices -d "$data"
Note:
Commands, or command fragments intended to be parsed into multiple words, should never be stored in strings. Use an array or a function instead. Quotes inside such strings are not parsed as syntax, and instead (when parsed without eval, which carries its own serious risks and caveats) become literal values. See BashFAQ #50 for a full explanation.
Use a JSON-aware tool, such as jq, to ensure that generated data is legit JSON.
Fully-qualifying paths to binaries is, in general, an antipattern. It doesn't result in a significant performance gain (the shell caches PATH lookups), but it does reduce your scripts' portability and flexibility (preventing you from installing a wrapper for curl in your PATH, in an exported shell function, or otherwise).
All-caps variable names are in a namespace used for variables with meaning to the shell and operating system. Use names with at least one lowercase character for your own variables to prevent any chance of conflict.
I have data phone in phone.txt
+6285712341234
+6285712341235
+6285712341236
+6285712341237
+6285712341238
but I don't know how to use this data to curl, here's what I tried:
curl -X POST "https://rest-api.moceansms.com/rest/1/sms" -d "mocean-api-key={api_key}&mocean-api-secret={api_secret}&mocean-from={name}&mocean-to={phone.txt}&mocean-text=Hello"
I should use phone data to send SMS to everyone; I googled for a solution, but with no luck (I don't even know whether the keywords I used to look for a solution where correct or not).
You should use --data flag:
Check:
https://curl.haxx.se/mail/archive-2007-03/0097.html
https://curl.haxx.se/docs/manpage.html#-d
Here the entire explanation from man:
-d, --data
(HTTP) Sends the specified data in a POST request to the HTTP server,
in the same way that a browser does when a user has filled in an HTML
form and presses the submit button. This will cause curl to pass the
data to the server using the content-type
application/x-www-form-urlencoded. Compare to -F, --form.
--data-raw is almost the same but does not have a special interpretation of the # character. To post data purely binary, you
should instead use the --data-binary option. To URL-encode the value
of a form field you may use --data-urlencode.
If any of these options is used more than once on the same command
line, the data pieces specified will be merged together with a
separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would
generate a post chunk that looks like 'name=daniel&skill=lousy'.
If you start the data with the letter #, the rest should be a file
name to read the data from, or - if you want curl to read the data
from stdin. Multiple files can also be specified. Posting data from a
file named 'foobar' would thus be done with -d, --data #foobar. When
--data is told to read from a file like that, carriage returns and newlines will be stripped out. If you don't want the # character to
have a special interpretation use --data-raw instead.
See also --data-binary and --data-urlencode and --data-raw. This
option overrides -F, --form and -I, --head and --upload.
I am trying to pass parameters to cURL through the command line, this way:
curl -s -X POST -H "Content-Type: text/xml" -H "Cache-Control: no-cache" -d '<Data Token="someToken" Name='"$appName"' ID='"$someVar"' ParseAppID='"$someVar"' ParseRESTKey='"$someVar"' AndroidPackage='"$someVar"' Version="1"></Data>' 'https://prefix.something.com/somePath?InputType=Xml'
(This line is actually extracted from the Postman app).
I Googled this issue and found whole lot of solutions that did not work for me (links are to SO past questions...):
I tried isolating the variables by ending the single quotes, this way: 'before...'"${someVar}"'...after...'. Could not complete the request.
I tried passing the variables using a file (-d #fileName). Failed to post.
I tried replacing the single quotes surrounding the <Data> tokens with double quotes - but the command apparently cannot accept such substitution.
The errors I get are either <Error></Error> or The server encountered an error and could not complete your request.
Is there any chance that there exists some other solution?
Has anyone encoutered such problem before?
I would be greatful for any help.
You aren't supplying quotes around the value of ID like you are for Name. That is, you need
'<Data Token="someToken" Name="'"$appName"'" ...>'
^^^
|||
||+- shell quote to protect $appName
|+- shell quote enclosing the XML
+- literal quote embedded in the XML
which results in the string (assuming appName=foo)
<Data Token="someToken" Name="foo" ...>
I have a curl command that needs to make a post and send my ssh key as one of the params, the command I am using looks like this:
SSH_KEY=`cat ~/.ssh/id_rsa.pub`;
curl -d "login=username&token=apikey&title=test&key=$SSH_KEY" http://github.com/api/v2/yaml/repos/key/username/somerepo/add
The problem is that there is a + in the ssh key and the string is terminating at that point, how can I avoid this?
Apparently in version 7.18.0 curl added the --data-urlencode option:
curl --data-urlencode "login=username&token=apikey&title=test&key=$SSH_KEY" http://github.com/api/v2/yaml/repos/key/username/somerepo/add
Use --data-urlencode instead of -d.
urlencode()
Is what I think your looking for? You can have special characters automatically replaced by using urlencode() on the string.
http://php.net/manual/en/function.urlencode.php for more information on it.
Also more information on: curl.haxx.se/docs/manpage.html do a page search for: urlencode
Hope that helps :)
To send a file add '#' in front of the filename
curl -d "login=username&token=apikey&title=test&key=#~/.ssh/id_rsa.pub" \
http://github.com/api/v2/yaml/repos/key/username/somerepo/add
Hope this helps.