I am having a problem with my bash script. It is producing an error of
curl (6) couldn't resolve host
What have I done wrong?
The following is my bash script.
#!/bin/bash
(set -o igncr) 2>/dev/null && set -o igncr; # this comment is needed
CookieFileName=cookies.txt
TEST="curl -k --cookie $CookieFileName --cookie-jar $CookieFileName POST -F "passUID=xxx&passUCD=xxx" https://wp1.coned.com/retailaccess/default.asp"
echo $TEST
RESPONSE=`$TEST`
echo $RESPONSE
Try this instead :
#!/bin/bash
set -o igncr
CookieFileName='cookies.txt'
curl -k \
--cookie "$CookieFileName" \
--cookie-jar "$CookieFileName" \
--data "passUID=xxx&passUCD=xxx" \
"https://wp1.coned.com/retailaccess/default.asp" # POST request
If you need to load another page after that, simply chains cURL commands with the previous line :
curl -k \
--cookie "$CookieFileName" \
--cookie-jar "$CookieFileName" \
"https://wp1.coned.com/retailaccess/another_page.asp" # GET request
Note
Command Substitution: The $(foo bar) causes the command 'foo' to be executed with the argument 'bar' and $(..) will be replaced by the output. See http://mywiki.wooledge.org/BashFAQ/002, http://mywiki.wooledge.org/CommandSubstitution, and http://mywiki.wooledge.org/BashFAQ/082
Related
EDIT:
I have managed to make it work with
response=$(
curl -k -X PUT -d "$marathon_payload" --write-out %{http_code} --silent --output "$tmp"\
-H "Authorization: token=$dcos_token" -H "$header_content_type" $app_id_url
)
The single quotes were causing the problem. It took a few gyrations but all good.
MORAL: quotes inside the value don't matter if the value is properly quoted UNLESS you eval the whole thing, and I should have known that. Occam's wins again.
end edit
I am initiating Mesosphere microservice deployments with curl, but it won't succeed without using eval. Since I recently inherited this code I've been trying to scrub the eval out of it just as a matter of habit, but it's thwarting me.
The script initiates the deployment with
response=$(
eval curl -k -X PUT -d "'$marathon_payload'" --write-out %{http_code} --silent --output $tmp\
-H "'Authorization: token=$dcos_token'" -H "'$header_content_type'" $app_id_url
)
If it gets a 200 or a 201, it loops a curl to effectively screen-scrape the deployments page till the request disappears.
chkDeploy() { rm -f $tmp;
eval curl -k -X GET --silent --write-out %{http_code} --silent --output $tmp\
-H "'Authorization: token=$dcos_token'" -H "'$header_content_type'" $deployments_url
}
response=$( chkDeploy )
$dcos_token is a base64 encoded string.
It then checks the service with another curl loop to the info page so it can verify the version number. This one is working fine with no eval.
chkCode() {
curl -k -X GET --write-out %{http_code} --silent --output $tmp $info_url;
}
response=$( chkCode )
The first two return 401, authentication failure.
I'm guessing the auth token quoting is off.
There's no reason to use eval here; you just need to quote the arguments to -H properly.
response=$(
curl -k -X PUT -d "$marathon_payload" \
--write-out %{http_code} \
--silent --output "$tmp" \
-H "Authorization: token=$dcos_token" \
-H "$header_content_type" "$app_id_url"
)
I wrote curl which returns only http status code :
curl --write-out %{http_code} \n
--silent \
--output /dev/null \
$URL
It works fine if I execute this from console. But after I have puted it into script, like this:
HTTP_STATUS=$(curl --write-out %{http_code} \n
--silent \
--output /dev/null \
$URL)
And try to echo $HTTP_STATUS, result is 200000000000000000000000000000000000000000000000000
How can I fix it?
I wrote curl which returns only http status code
There are couple of issues with your script.
Your use of UPPERCASE variables might override shell environment variables.
The --write-out argument of the curl can ideally be within double quotes.
status=$(curl --write-out "%{http_code}" --silent --output /dev/null "$url")
echo "$status" # Would give you just the status
Note: As pointed out in this comment you don't need the newline too since you're assigning the value to a variable.
I'm writing my first bash script
LANG="en_US.UTF8" ; export LANG
PROXY=$(shuf -n 1 proxy.txt)
export https_proxy=$PROXY
RUID=$(php -f randuid.php)
curl --data "mydata${RUID}" --user-agent "myuseragent" https://myurl.com/url -o "ticket.txt"
This script also use curl, but if proxy is down it gives me this error:
failed to connect PROXY:PORT
How can I make bash script run again, so it can get another proxy address from proxy.txt
Thanks in advance
Run it in a loop until the curl succeeds, for example:
export LANG="en_US.UTF8"
while true; do
PROXY=$(shuf -n 1 proxy.txt)
export https_proxy=$PROXY
RUID=$(php -f randuid.php)
curl --data "mydata${RUID}" --user-agent "myuseragent" https://myurl.com/url -o "ticket.txt" && break
done
Notice the && break at the end of the curl command.
That is, if the curl succeeds, break out of the infinite loop.
If you have multiple curl commands and you need all of them to succeed,
then chain them all together with &&, and add the break after the last one:
curl url1 && \
curl url2 && \
break
Lastly, as #Inian pointed out,
you could use the --proxy flag to pass a proxy URL to curl without the extra step of setting https_proxy, for example:
curl --proxy "$(shuf -n 1 proxy.txt)" --data "mydata${RUID}" --user-agent "myuseragent"
Lastly, note that due to the randomness, a randomly selected proxy may come up more than once until you find one that works.
Avoid that, you could read iterate over the shuffled proxies instead of an infinite loop:
export LANG="en_US.UTF8"
shuf proxy.txt | while read -r proxy; do
ruid=$(php -f randuid.php)
curl --proxy "$proxy" --data "mydata${ruid}" --user-agent "myuseragent" https://myurl.com/url -o "ticket.txt" && break
done
I also lowercased your user-defined variables,
as capitalization is not recommended for those.
I know i accepted #janos answer but since I can't edit his I'm going to add this
response=$(curl --proxy "$proxy" --silent --write-out "\n%{http_code}\n" https://myurl.com/url)
status_code=$(echo "$response" | sed -n '$p')
html=$(echo "$response" | sed '$d')
case "$status_code" in
200) echo 'Working!'
;;
*)
echo 'Not working, trying again!';
exec "$0" "$#"
esac
This will run my script again if it gives 503 status code which i wanted :)
And with #janos code it will run again if proxy is not working.
Thank you everyone i achieved what i wanted.
I'm working with installr API.
I'm trying to do the following curl request via a script :
curl -H "X-InstallrAppToken: mytoken" https://www.installrapp.com/apps.json/ \
-F 'qqfile=#'$APKPATH \
-F 'releaseNotes=These are my release notes' \
-F 'notify=true'
and it works perfectly.
However, when I try to get my release notes from a file with a variable like this :
RELEASENOTES=`cat "release_notes/test.md"`
curl -H "X-InstallrAppToken: mytoken" https://www.installrapp.com/apps.json/ \
-F 'qqfile=#'$APKPATH \
-F 'releaseNotes='$RELEASENOTES \
-F 'notify=true' > /dev/null
it doesn't work at all, only the first word is sent. For the others, I have the error Could not resolve host: xxx.
I did a echo on these two curl request and the exact same thing is printed.
is that the catcommand which return a specific format ?
Probably an issue with the quotes and spaces. You can use double-quotes around a variable to allow variable expansion in the shell.
RELEASENOTES=$(cat "release_notes/test.md")
curl -H "X-InstallrAppToken: mytoken" https://www.installrapp.com/apps.json/ \
-F "qqfile=#${APKPATH}" \
-F "releaseNotes=${RELEASENOTES}" \
-F 'notify=true' > /dev/null
I've got the following call in a bash script I'm creating
response= curl -X POST $URL -u "$USER:$PASSWORD" --data-urlencode "key=$key" --data "label=pub_key" -o /dev/null --silent --write-out "%{http_code}"
I can see 200 being written to the console however $response always ends up null.
I've also tried the following but it was not better.
response= $(curl -X POST $URL -u "$USER:$PASSWORD" --data-urlencode "key=$key" --data "label=pub_key" -o /dev/null --silent --write-out "%{http_code}")
Any help for a bash noob would be appreciated.
Space -- the final frontier:
response=$(curl -X ...)
Note: no spaces around the =. The shell is white-space sensitive in a few places and variable assignments are one of them.
With the space, as in var= command args, you set var as empty in a one-shot assignment and then run command.