Help with a curl command and reading from a file - bash

I have a curl command that needs to make a post and send my ssh key as one of the params, the command I am using looks like this:
SSH_KEY=`cat ~/.ssh/id_rsa.pub`;
curl -d "login=username&token=apikey&title=test&key=$SSH_KEY" http://github.com/api/v2/yaml/repos/key/username/somerepo/add
The problem is that there is a + in the ssh key and the string is terminating at that point, how can I avoid this?

Apparently in version 7.18.0 curl added the --data-urlencode option:
curl --data-urlencode "login=username&token=apikey&title=test&key=$SSH_KEY" http://github.com/api/v2/yaml/repos/key/username/somerepo/add

Use --data-urlencode instead of -d.

urlencode()
Is what I think your looking for? You can have special characters automatically replaced by using urlencode() on the string.
http://php.net/manual/en/function.urlencode.php for more information on it.
Also more information on: curl.haxx.se/docs/manpage.html do a page search for: urlencode
Hope that helps :)

To send a file add '#' in front of the filename
curl -d "login=username&token=apikey&title=test&key=#~/.ssh/id_rsa.pub" \
http://github.com/api/v2/yaml/repos/key/username/somerepo/add
Hope this helps.

Related

curl: (3) Port number ended with ' ' [Powershell]

I am trying to make a post request in powershell using curl. But I seem to be unfortunately getting this error.
I have tried removing spaces here and there, and googling the problem but have not found a solution.
curl.exe -X 'POST' -H #{'Content-Type'='application/json'; 'accept'='application/json'} -d \"{\"name\":\"test3\", \"auto_init\":true, \"default_branch\": \"master\", \"description\": \"My Test\", \"gitignores\": \"Vim\", \"issue_labels\":\"Default\", \"license\": \"DOC\", \"name\":\"test2\", \"private\":false, \"readme\":\"Default\",\"template\":false,\"trust_model\":\"default\"}\" http://localhost:3000/api/v1/user/repos?access_token=c11ceb97fa594a7e6c4b5519e4327908be3274b9
Re -H:
curl.exe is an external program, which means that you cannot meaningfully pass a PowerShell hashtable (#{ ... }) as an argument, because it will (uselessly) be passed as literal string System.Collections.Hashtable.
Instead, pass strings, as multiple -H options, each in the form '<name>: <value>'
Re -d:
PowerShell's escape character is ` (the so-called backtick), not \.
Since your argument is to be passed verbatim (contains no variable references to be interpolated), use a verbatim (single-quoted) string ('...').
However: The sad reality as of PowerShell 7.2 is that an extra, manual layer of \-escaping of embedded " characters is required in arguments passed to external programs. This may get fixed in a future version, which may require opt-in. See this answer to the linked duplicate for details.
To put it all together:
curl.exe -H 'Content-Type: application/json' -H 'accept: application/json' -d '{\"name\":\"test3\", \"auto_init\":true, \"default_branch\": \"master\", \"description\": \"My Test\", \"gitignores\": \"Vim\", \"issue_labels\":\"Default\", \"license\": \"DOC\", \"name\":\"test2\", \"private\":false, \"readme\":\"Default\",\"template\":false,\"trust_model\":\"default\"}' 'http://localhost:3000/api/v1/user/repos?access_token=c11ceb97fa594a7e6c4b5519e4327908be3274b9'
Note: I've omitted -X 'POST', because, as Daniel Stenberg notes, a POST request is implied when you use the -d option.

Curl Post with File

I have data phone in phone.txt
+6285712341234
+6285712341235
+6285712341236
+6285712341237
+6285712341238
but I don't know how to use this data to curl, here's what I tried:
curl -X POST "https://rest-api.moceansms.com/rest/1/sms" -d "mocean-api-key={api_key}&mocean-api-secret={api_secret}&mocean-from={name}&mocean-to={phone.txt}&mocean-text=Hello"
I should use phone data to send SMS to everyone; I googled for a solution, but with no luck (I don't even know whether the keywords I used to look for a solution where correct or not).
You should use --data flag:
Check:
https://curl.haxx.se/mail/archive-2007-03/0097.html
https://curl.haxx.se/docs/manpage.html#-d
Here the entire explanation from man:
-d, --data
(HTTP) Sends the specified data in a POST request to the HTTP server,
in the same way that a browser does when a user has filled in an HTML
form and presses the submit button. This will cause curl to pass the
data to the server using the content-type
application/x-www-form-urlencoded. Compare to -F, --form.
--data-raw is almost the same but does not have a special interpretation of the # character. To post data purely binary, you
should instead use the --data-binary option. To URL-encode the value
of a form field you may use --data-urlencode.
If any of these options is used more than once on the same command
line, the data pieces specified will be merged together with a
separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would
generate a post chunk that looks like 'name=daniel&skill=lousy'.
If you start the data with the letter #, the rest should be a file
name to read the data from, or - if you want curl to read the data
from stdin. Multiple files can also be specified. Posting data from a
file named 'foobar' would thus be done with -d, --data #foobar. When
--data is told to read from a file like that, carriage returns and newlines will be stripped out. If you don't want the # character to
have a special interpretation use --data-raw instead.
See also --data-binary and --data-urlencode and --data-raw. This
option overrides -F, --form and -I, --head and --upload.

Dynamicly build string path for curl capath

Curl has the option to add capath as one of its arguments.
This argument can contain one path or several paths in this format:
curl --capath /certs/path1:/certs/path2:/certs/path3 https://domain.com
Is it possible to use curl capath arg with subfolders by only adding the root dir such as /certs/ ?
And if not i would like to build the string which automatically expands to this
format: /certs/path1:/certs/path2:/certs/path3
When i echo this command :
echo /certs/*
/certs/path1 /certs/path2 /certs/path3
required output:
/certs/path1:/certs/path2:/certs/path3
The idea is to have some automatic expanding method that will do that without sed awk or external tool.
something like this:
curl --capath /certs/*{:} https://domain.com
will automatlcy result with :
curl --capath /certs/path1:/certs/path2/:/certs/path3 https://domain.com
Unfortunately, I don't see any way to do it without an externl program.
Well, you could do
for s in /certs/*
do
path+="$s:"
done
But I don't think you are looking for this.
The point is that you are using the wildcard character '*' and it is interpreted by your shell as a list of string, separated with space.
Else just put in a variable
var=`/certs/* | tr ' ' ':'`
Hope this will answer. And if someone can find a real solution, then I want to know it too =)

How to urlencode data into a URL, with bash or curl

How can a string be urlencoded and embedded into the URL? Please note that I am not trying to GET or POST data, so the -G and --data and --data-urlencode options of curl don't seem to do the job.
For example, if you used
curl -G http://example.com/foo --data-urlencode "bar=spaced data"
that would be functionally equivalent to
curl http://example.com/foo?bar=spaced%20data"
which is not desired.
I have a string foo/bar which must be urlencoded foo%2fbar and embedded into the URL.
curl http://example.com/api/projects/foo%2fbar/events
One hypothetical solution (if I could find something like this) would be to preprocess the data in bash, if there exists some kind of urlencode function.
DATA=foo/bar
ENCODED=`urlencode $DATA`
curl http://example.com/api/projects/${ENCODED}/events
Another hypothetical solution (if I could find something like this) would be some switch in curl, similar to this:
curl http://example.com/api/projects/{0}/events --string-urlencode "0=foo/bar"
The specific reason I'm looking for an answer to this question is the Gitlab API. For example, gitlab get single project NAMESPACE/PROJECT_NAME is URL-encoded, eg. /api/v3/projects/diaspora%2Fdiaspora (where / is represented by %2F). Further to this, you can request individual properties in the project, so you end up with a URL such as http://example.com/projects/diaspora%2Fdiaspora/events
Although this question is gitlab-specific, I imagine it's generally applicable to REST API's in general, and I'm surprised I can't find a pre-existing answer on stackoverflow or internet search.
The urlencode function you propose is easy enough to implement:
urlencode() {
python -c 'import urllib, sys; print urllib.quote(sys.argv[1], sys.argv[2])' \
"$1" "$urlencode_safe"
}
...used as:
data=foo/bar
encoded=$(urlencode "$data")
curl "http://example.com/api/projects/${encoded}/events"
If you want to have some characters which are passed through literally -- in many use cases, this is desired for /s -- instead use:
encoded=$(urlencode_safe='/' urlencode "$data")

Github API /issues - pagination trouble

I am using curl from a bash command line to GET Github issues like this:
curl -o myoutput --user "myuser:mypasswd" -G https://api.github.com/issues?filter=all
This is working fine and returns 52 open issues.
I know there are more issues, so I am also examining the headers (using -i) which provides links to the next & last pages, https://api.github.com/issues?filter=all&page=2 & https://api.github.com/issues?filter=all&page=14 respectively
However, using curl with these link URI's produces the same 52 results as before. In fact any page number I try returns the same most recent issues. I am deleting myoutput each time.
What am I missing?
Any words of wisdom on this would be much appreciated.
Thanks
What am I missing?
Use a single quoted string for the URL to make sure the ampersand (e.g &page=2) is not interpreted as a control operator:
curl -o myoutput2 --user "user:pwd" \
'https://api.github.com/issues?filter=all&page=2'
Without doing so you systematically perform a https://api.github.com/issues?filter=all request, which is why the output is always the same.

Resources