Is it possible to set timeout value for curl globally? E.g. via an environment variable or config file?
I have a shell script with some 20 curl commands scattered all over and would like to avoid specifying --connect-timeout everywhere.
This is for shell scripting, not php or C/C++ or ...
As far as I can see on the man page, there is no such environment variable.
You could make a function called curl:
curl () {
command curl --connect-timeout 60 "$#"
}
So whenever you call curl, it will call this function, which in turn calls the curl command (command suppresses shell function lookup) with the same arguments.
Related
I want to automate and create a bash script where I have this call:
curl --header "Authorization: Bearer <PANDA TOKEN HERE>" https://www.websitehere.com/api/v2/groups/GROUP NAME/details
My goal is for the bash script to iterate over a list of group names (GROUP NAME).I have my panda token working. Unfortunately I have no previous experience with bash. I have installed WSL and curl.
I tried this bash script, but it doesn't work:
#!/bin/bash
group_details=("acton_team_coordinators" "acs-peripherals" "admingrocery" "ae-central-leads")
for group in "${group_details[#]}"
do
curl -v GET " https://www.websitehere.com/api/v2/groups/acton_team_coordinators/details" -H "<PANDA TOKEN HERE>”
done
I got error in CMD like:
enter image description here
You forgot to code the "group" variable as a variable, by prefixing it with the "$".
It is good practice to surround variable strings with a pair of braces ("{}"), because there are cases where confusion can occur in determining what is the actual variable name.
You also had space inside the double-quotes, before the "http ...". I am not sure if that could create issues, but best to remove that space and avoid them.
The modified script would look like this:
#!/bin/bash
out_capture="somewhere"
group_details=("acton_team_coordinators" "acs-peripherals" "admingrocery" "ae-central-leads")
for group in "${group_details[#]}"
do
curl -v GET "https://www.websitehere.com/api/v2/groups/${group}/details" -H "<PANDA TOKEN HERE>" >${out_capture}/${group}.out
done
I'm testing Pushgateway along Prometheus, pushing simple data about my VMs' services every minute with a bash script and curl, triggered by crontab.
I have two bash scripts.
The first one, cpu_usage.sh, parse and send data about ps -aux command to pushgateway, via curl :
#!/usr/bin/env bash
z=$(ps aux)
while read -r z
do
var=$var$(awk '{print "cpu_usage{process=\""$11"\", pid=\""$2"\", cpu=\""$3"\"}", $3z}');
done <<< "$z"
curl -X POST -H "Content-Type: text/plain" --data "$var
" http://localhost:9091/metrics/job/top/instance/machine
Everything's ok on this one, my crontab send every minute the formatted data to localhost:9091, my pushgateway instance.
Now, I'm trying to send a parsed result of the service --status-all command, with a script called services_list.sh, exactly the way I've dealed with the cpu_usage.sh script :
#!/usr/bin/env bash
y=$(service --status-all)
while read -r y
do
varx=$varx$(awk '{print "services_list{service=\""$y"\"}", 1}');
done <<< "$y"
curl -X POST -H "Content-Type: text/plain" --data "$varx
" http://localhost:9091/metrics/job/top/instance/machine
When executing both scripts manually, like ./cpu_usage.sh and ./services_list.sh, everything's fine, Pushgateway is successfully retrieving data from both scripts.
But when I'm passing theses calls by CRON, only cpu_usage.sh is sending data to pushgateway (timestamp of last push on services_list method on Pushgateway stays unchanged).
My crontab syntax is like : * * * * * cd /path/ && ./script.sh
Scripts are in 777 / root:root, crontab is always edited as root. I've tried to concatenate both scipts in one bash file, and no matter the order I put them, the curl call for services_list method is never made (but everything's ok on cpu_usage method).
I'm a bit lost, as the two scripts are very similar, and manual calls on services_list.sh are working fine. Any thoughts ? Thank you.
Running on Debian 9 with Pushgateway 0.10.
Fixed :
The command service is not callable from crontab. I have to furnish the full path of the service ressource when calling it from cron, like /usr/sbin/service --options.
In this context, everything's running fine with y=$(/usr/sbin/service --status-all) in place of y=$(service --status-all) in my services_list.sh script.
Trying to run a curl command to a test page results in "no URL specified!" when attempting to run via a bash script. This is a test that runs through a proxy (1.2.3.4 in the code example)
The command runs as expected from the command line. I've tried a few variations of quotes around the components of the command however still manage the same "no URL specified" result.
echo $(curl -x http://1.2.3.4:80 https://example.company.com)
The expected result is an HTML file, however instead the above issue is found only when run from this small bash script file.
The command itself runs just fine as "curl -x http://1.2.3.4:80 https://example.company.com" from the command line.
Appreciate in advance any help you can provide!
I have literally edited it down to the following. Everything else is commented out. Still the same error.
#!/bin/bash
curl -x http://1.2.3.4:80 https://example.company.com
In your example you want to use double quotes around the subshell and single qutes for the curl parameters
Make sure to set the correct shebang (#!/bin/bash).
You could also try to run it as:
cat <(curl -x http://1.2.3.4:80 https://example.company.com)
However I am not able to reproduce your example
I am using Ubuntu bash shell on a windows machine and i am trying to create a function to test my endpoints of a spring application. First i must authenticate with the server via the following curl command. The output of the statement run is below.
$> curl -i -X "POST" -d"${auth}" -H"${contentType}" "${host}/login"
When i copy and paste this command into a function it seems to blow up leading to non-execution of curl as intended which should have the same output as above.
$> function springlogin(){curl -i -X "POST" -d"${auth}" -H"${contentType}" "${host}/login";};
$> springlogin
What am i missing here? Is this due to some variable expansion im not aware of? Or something else entirely.
My end goal would be to use the output of this function and use it to authorize my endpoint api calls if i happen to be on my command line.
Thanks to Gordon Davisson i figured it out. This is something that we do at work but i didn't think to do it in my home code. I need to unset the function in my profile before i declare it again.
unset springlogin
function springlogin(){curl -i -X "POST" -d"${auth}" -H"${contentType}" "${host}/login";};
I want to incorporate a simple monitoring into my application so I need to send an HTTP request that contains the number of documents in the mongodb collection from the crontab.
The requests are described on the page http://countersrv.com/ as follows:
curl http://countersrv.com/ID -d value=1
I need to query the mongodb from the command line and get the number of documents in the collection. It should be something like db.my_docs.count().
I want to send this number every hour so need to add something like this into crontab:
0 * * * * curl http://countersrv.com/ID -d value=...query mongo here...?
Not meaning to detract from the timely answer given by Victor, but the "one liner" form of this would be:
mongo --quiet --eval 'var db = db.getSiblingDB("database"); print( "value=" + db.collection.count() );' | curl -X POST http://countersrv.com/[edit endpoint] -d #-
The --quiet suppresses the startup message on the shell and --eval alows the commands to pass through on the command line.
To select the database you use .getSiblingDB() as the method helper for the interactive shell use database with the "database" name you want. After this either just the "collection" name or .getCollection() method can be used along with the basic function.
Simply print() the "key/value" pair required and pipe to curl at the "edit endpoint" for countersrv, which is the default viewing page. The #- construct takes stdin.
I would avoid using commands directly on crontab, you probably have a directory /etc/cron.hourly and crontab already have calls to run all the scripts in the specific folders, in determined intervals, hourly, daily for example
Then, inside /etc/cron.hourly you can create a monitor.sh. You can set the execution privilege of this script with
chmod +x /etc/cron.hourly/monitor.sh
Them, you make a js code to retrieve the data, for example, mongoscript.js:
use yourdb
db.my_docs.count()
And you final monitor.sh will probably be something like
#!/bin/bash
mongo mongoscript.js > output.js
curl http://countersrv.com/ID -d value=#output.js