Why doesn't command substitution work? - bash

I have a bash script that calls "curl" to post parameters. Some of these parameters have spaces in their values, so we need to surround it with quotes when passing it to curl. Otherwise, only the first part before the space is passed as a parameter.
I tried storing the command in a variable, using escaped quotes, but I cannot get it to work in command substitution except by using eval. I can also get it working by calling command substitution directly on the string (and not storing the command in a variable).
Is it possible to store the command in a variable and use command substitution on that variable? This is what I am trying - and failing - to do in attempt 1 in the code below. I don't understand why it doesn't work:
#!/bin/bash
yesterday=$(date +"%Y-%m-%d %H:%M:%S" -d "1 day ago") #2013-09-04 01:15:51
now=$(date +"%Y-%m-%d %H:%M:%S" ) #2013-09-05 01:15:51
echo -e "INPUTS: Today: $now, Yesterday: $yesterday"
#Attempt 1: Does not work
cmd="curl -s -u scott:tiger -d url=http://localhost:8080 -d \"start=$yesterday\" -d \"end=$now\" -d \"async=y\" http://192.168.1.46:8080/cmd/doSendMinimalToServer"
output=$($cmd)
echo "Output from executing cmd variable: $output"
#Result: The quotes get passed into the HTTP POST request. This is not good.
#Attempt 2: Using eval on the variable. Works.
output_eval=$(eval $cmd)
echo "Output from eval'ing variable: $output_eval"
#Result: This works, but I would prefer not to use eval
#Attempt 3: Using Command substitution directly on the string. Works.
output_direct=$(curl -s -u scott:tiger -d url=http://localhost:8080 -d "start=$yesterday" -d "end=$now" -d "async=y" http://192.168.1.46:8080/cmd/doSendMinimalToServer)
echo "Output from executing string: $output_direct"
#Result: This works. The HTTP POST parameters correctly have spaces in their values and no quotes.
I have also tried passing in the parameters as an array, unsuccessfully.

Store your command arguments in an array and run it like this:
#!/bin/bash
yesterday=$(date +"%Y-%m-%d %H:%M:%S" -d "1 day ago") # 2013-09-04 01:15:51
now=$(date +"%Y-%m-%d %H:%M:%S" ) #2013-09-05 01:15:51
echo -e "INPUTS: Today: $now, Yesterday: $yesterday"
cmd=(curl -s -u scott:tiger -d url=http://localhost:8080 -d "start=$yesterday" -d "end=$now" -d "async=y" "http://192.168.1.46:8080/cmd/doSendMinimalToServer")
output=$("${cmd[#]}")
echo "Output from executing cmd variable: $output"
Also I think you could use date '+%F %T' for simplicity.

Related

Unable to pass variable in gremlin query in shell script

I am trying to connection to Neptune DB and getting vertices details using CURL command. I have shell script for it. But somehow variable data is not going through it gremlin query. I have one Orgid.txt file where tenantid is present and my shell script reading the file and passing it to "name" variable
#!/bin/bash
i=1
rm VerticesCount1
while IFS= read -r line
do
name="$line"
#echo "Orgid name is "$i": $name"
curl -X POST https://<Neptune_endpoint>:<port>/gremlin -d '{"gremlin":"g.V().has(\"system.tenantId\",\"$name\").count()"}' >> VerticesCount1
#printf "\n"
echo >> VerticesCount1
((i=i+1))
done < Orgid.txt
As with your other question I tested with a simple data file and it works fine. However, note how I changed the type of quotes used by curl.
i=1
while IFS= read -r line
do
name="$line"
curl -X POST https://mydbcluster.cluster-xxxxxxxxxxxx.us-east-1.neptune.amazonaws.com:8182/gremlin -d \
"{\"gremlin\":\"g.V().has('code','$name').count()\"}"
((i=i+1))
done < values.txt
which produces
{"requestId":"4e3e80ed-efcb-40a7-b92b-366c6f391d4e","status":{"message":"","code":200,"attributes":{"#type":"g:Map","#value":[]}},"result":{"data":{"#type":"g:List","#value":[{"#type":"g:Int64","#value":1}]},"meta":{"#type":"g:Map","#value":[]}}}{"requestId":"6a269b5b-32f6-49d2-a31d-c51dd52eba29","status":{"message":"","code":200,"attributes":{"#type":"g:Map","#value":[]}},"result":{"data":{"#type":"g:List","#value":[{"#type":"g:Int64","#value":1}]},"meta":{"#type":"g:Map","#value":[]}}}
it is working fine with this code.
while IFS= read -r line
do
name="$line"
#echo "Orgid name is "$i": $name"
curl -X POST https://<Neptune_endpoint>:<port>/gremlin -d '{"gremlin":"g.V().has(\"system.tenantId\",\"'$name'\").count()"}' >> VerticesCount1
echo >> VerticesCount1
done < Orgid.txt

How to pass a date to a bash shell and calulcate from it?

I am trying to pass a date string to a bash shell and calculate a new date from it, here is the test1.sh
#!/bin/bash
echo "$1"
MYDATE=$1
days = 5
echo $MYDATE
DATE_FROM=$($MYDATE "--date=-$days day" +%F)
and I call it with:
sh ./test1.sh 2017-07-10
it got following output:
2017-07-10
2017-07-10
./test1.sh: 8: ./test1.sh: 2017-07-10: not found
any idea ? thanks
Updated:
here is a working version based on #John Goofy:
#!/bin/bash
MYDATE="$1"
#DAYS=5 #or comment this line out and pass DAYS as argument $2
if [ -z "$2" ]; then
DAYS = 10
else
DAYS="$2"
fi
NEWDATE=$(date --date="${MYDATE} + ${DAYS} day" +%Y-%m-%d)
echo $NEWDATE
calling this:
sh ./test1.sh 2017-07-10 5
days is set to 5, if calling it without $2, I'd like it to be 10, but got:
DAYS: not found
You're missing a call to date command:
DATE_FROM=$(date --date "$MYDATE - $days day" +%F)
Also, to use a default value for days, you can use the default value parameter expansion ${parameter:-word} (see examples).
Additionally, since environment variables are ALL_CAPS as defined by POSIX, it's best to keep your script variables in lowercase.
And note that the +%Y-%m-%d date format introduced in your update is identical to +%F (only more verbose).
All of this taken together yields a script that does what you need, and does it properly:
#!/bin/bash
start_date="$1"
days="${2:-10}"
end_date=$(date --date="${start_date} + ${days} day" +%F)
echo "${end_date}"
For example:
$ bash test1.sh 2017-07-10 5
2017-07-15
(Note that if you call your script with sh it doesn't matter that it has bash in hashbang, it's run with sh. Hashbang is only used if you make your script an executable with chmod +x test1.sh and then call it like ./test.sh .... Only then the shell will read from the hashbang which interpreter should use for the script.)
Your script should look like this:
#!/bin/bash
MYDATE="$1"
DAYS=5 #or use DAYS="$2" to pass DAYS as argument $2
NEWDATE=$(date --date="${MYDATE} + ${DAYS} day" +%Y-%m-%d)
echo $NEWDATE
If you don't pass an argument the script will calculate from now.

How to use current date in a curl command in shell script?

#!/bin/bash
currentTime=$(date +"%R")
currentdate=$(date +'%m/%d/%Y')
oldtime=$(date +%R -d "50 min ago")
echo "Current time : $currentTime"
echo "Current Date : $currentdate"
echo "Old time : $oldtime"
Response=$(curl -X GET -H "Authorization: Basic Token=" "http://targetserver.company.com:8080/v1/organizations/company/environments/environmentname/stats/apis?select=sum(is_error)&timeRange="$currentdate"%20"$currentTime"~"$currentdate"%20"$oldtime"")
echo -e "the response is: \n $Response"
Not getting any response? Please help how to use system date-time/current date-time in curl URL in shell-script.
it looks consistent, but not entirely sure what looks wrong. have you tried to echo the Response line to see what it looks like? how about trying it with wrapping the bash variable names in curly brace and removing the quotes in the string?
Response=$(curl -X GET -H "Authorization: Basic Token=" "http://targetserver.company.com:8080/v1/organizations/company/environments/environmentname/stats/apis?select=sum(is_error)&timeRange=${currentdate}%20${currentTime}~${currentdate}%20${oldtime}")

Shell script on passing arguments

My code was like this I'm passing 4 arguments to a script
ex.sh "wavpath" "featpath"
"ex.sh"
code is
#!/bin/bash
wavPath=$1
featPath=$2
rm -f $scpFile
echo $wavPath
echo $featPath
for dir in `ls -R $wavPath|grep ":"|cut -d':' -f1`
do
mkdir -p ${dir/$wavPath/$featPath}
done
The error message:
bad substitution
and it is at ${dir/$wavPath/$featPath}
and its showing both the paths
can anyone help
Try ${dir}/${wavPath}/${featPath}
maybe you meant $dir/$wavPath/$featPath
try changing
mkdir -p ${dir/$wavPath/$featPath}
to
echo $dir/$wavPath/$featPath
and see if the output is what you expected for the input of mkdir.
Also, you're not setting a value for the variable $scpFile before you use it.

Bash script in Prey project formatted incorrectly? Nested backtick issue?

I'm a terrible beginner at bash scripting, and am hoping someone can help me out with this issue.
Having a problem with the Prey project standalone scripts. There's a line that's supposed to send an email, and apparently its not formatted correctly.
response=`mailsender -f "$mail_from" -t "$mail_to" -u "$complete_subject" \
-s $smtp_server -a $file_list -o message-file="$trace_file.msg" \
tls=auto username=$smtp_username \
password=\`decrypt \"$smtp_password\"\``
Where mailsender is an alias to Brandon Zehm's PERL sendEmail script, $smtp_password is a pointless base64 encoding of the password, and decrypt is:
decrypt() {
echo "$1" | openssl enc -base64 -d
}
So can anyone tell me what's wrong with the script? For reference, if I just replace the entire decrypt part with the plaintext password, it works fine. i.e.:
response=`mailsender -f "$mail_from" -t "$mail_to" -u "$complete_subject" \
-s $smtp_server -a $file_list -o message-file="$trace_file.msg" \
tls=auto username=$smtp_username password=actual_password`
The simplest thing to do is avoid backticks, and use $() instead -- they nest cleanly, with no special escaping needed:
response=$(Documents/Projects/Shell\ Scripting/printargs -f "$mail_from" \
-t "$mail_to" -u "$complete_subject" -s $smtp_server -a $file_list \
-o message-file="$trace_file.msg" tls=auto username=$smtp_username \
password="$(decrypt "$smtp_password")")
I think this script is isomorphic with yours:
decrypt()
{
echo "$1" | tr 'a-z' 'A-Z'
}
xxx=`echo xxx=yyy pass=\`decrypt \"xyz abc\"\``
echo "$xxx"
When run with 'sh -x xxx' (where 'sh' is 'bash' in disguise):
$ sh -x xxx
+++ decrypt '"xyz' 'abc"'
+++ echo '"xyz'
+++ tr a-z A-Z
++ echo xxx=yyy 'pass="XYZ'
+ xxx='xxx=yyy pass="XYZ'
+ echo 'xxx=yyy pass="XYZ'
xxx=yyy pass="XYZ
$
You can see where there are problems - if you know how to look. The decrypt command line has two arguments where the intention was to have just one, and the arguments include a double quote before the first and another at the end of the second.
So, in your script, the argument passed to decrypt includes a pair of double quotes, which probably isn't what you wanted.
If we rewrite the script using the '$(...)' notation, which nests much more neatly, then we get:
decrypt()
{
echo "$1" | tr 'a-z' 'A-Z'
}
yyy=$(echo zzz=yyy pass=$(decrypt "xyz abc"))
echo "$yyy"
The trace from this looks like:
$ sh -x xxx
+++ decrypt 'xyz abc'
+++ echo 'xyz abc'
+++ tr a-z A-Z
++ echo zzz=yyy pass=XYZ ABC
+ yyy='zzz=yyy pass=XYZ ABC'
+ echo 'zzz=yyy pass=XYZ ABC'
zzz=yyy pass=XYZ ABC
$
I'm one of the guys from Prey. This bug was confimed yesterday and a fix has already been commited.
I do agree that $() is much easier to read than backticking -- and specially back-backticking --, and actually that's one of the things we're working on (big code refactoring).
Lately I've been working on a Bash framework called Skull which provides a much nicer interface for writing shell scripts. Hopefully Prey 0.6 will be based completely on it, and excessive backticking will be replaced with $() to make it easier for everyone to read.

Resources