I'm having trouble in finding the reason why my BASH script behaves differently if run alone or inside cron. I have this snippet:
#!/bin/bash
RESPONSE=$(curl -fs -XPOST -H "Content-type: application/json" -d '{"id" : 4}' https://myserver.com)
echo $RESPONSE
if [ -z "$RESPONSE" ]; then
echo "empty response"
return 0
fi
COMMAND=$(echo $RESPONSE | python -c "import sys, json; print json.load(sys.stdin)['command']")
if [ -z "$COMMAND" ]; then
echo "empty command"
elif [ "$COMMAND" = "SYS_INFO" ];
then
#business logic
fi
that prints two different responses in the two environments:
$RESPONSE Running from console:
{"id":"1f78d8d0-e754-4a23-a2f0-448fbeb42995", "key":"\n4RHDFAnTull1Z+aHGbO1zXcAGghuaEUz0w8sT7dlpc80jG6ZaWnbDox4G0f8sKY\ng0WZ80zWf8ftNgX3nes9MWYEq00nM5jJWCSavmGSKCKjoGD2XqBod8W0Z5w/KAHTSitGVMFgMjda91+xozw8uMlzR/t3Y8FP2k/NHj\n"}
$RESPONSE Running from :
{"id":"1f78d8d0-e754-4a23-a2f0-448fbeb42995", "key":"
4RHDFAnTull1Z+aHGbO1zXcAGghuaEUz0w8sT7dlpc80jG6ZaWnbDox4G0f8sKYj
g0WZ80zWf8ftNgX3nes9MWYEq00nM5jJWCSavmGSKCKjoGD2XqBod8W0Z5w/KAHTSitGVMFgMjda91+xozw8uMlzR/t3Y8FP2k/NHj
"}
Please notice the \n that the server returns into key field that are present when running from console and NOT present (actually, they are encoded as newline) when running from crontab
What I've tried:
adding source ~/.bashrc as suggested here
changing value of PATH evaluating the differences of the two environments as suggested here
However, nothing seems to work.
Related
I have .sh script in UbuntuServer (VM):
response=$(curl -H 'Accept: application/vnd.twitchtv.v5+json' -H 'Authorization: OAuth 9f65dd6onr07vhpdqblbiix5rl0tch' -X GET 'https://api.twitch.tv/kraken/streams/127060528' | jq -r '.stream');
now=$(date +"%Y-%m-%d %T");
echo "$now" >> log.txt;
echo "$response" >> log.txt;
if [[ "$response" == "null" ]]
then
echo "ZERO"
else
streamlink -o "dump/stream_$now.mp3" twitch.tv/mahetirecords/clip/FreezingEncouragingCougarSuperVinlin audio
echo "STREAM"
fi
When I run the script through the bash, the THEN-way is triggered.
When I run the script through crontab, the ELSE-way is triggered.
WHY?
Crontab -e:
* * * * * /home/chesterlife/twitch-interceptor/script.sh
If the stream is offline, then "$response" return null. This is text-null because var=""; if [ "$var" == "$response" ]; then echo "true"; else echo "false"; fi return false
Any ideas?
By default crontab executes using sh shell. You can change it to bash using the information from the link below:
https://unix.stackexchange.com/questions/94456/how-to-change-cron-shell-sh-to-bash
Also read this Stack Overflow question about the difference between [ and [[ in Bash
While [ is POSIX, [[ is only supported by some shells including bash.
You're missing a shebang in the script beginning, eg:
#!/usr/bin/env bash
Also, you should add your current user's path (obtained via echo $PATH) to your script, so you're running in a sane environnment, identical to your user's.
I have a curl command that looks like this:
curl -X PUT -H "myheader:coca-cola" -d '{ "name":"harrypotter" }' http://mygoogle.com/service/books/123
Running this command as is via terminal returns the expected results.
I am trying to incorporate this curl command in my bash script as follows:
#!/bin/bash
MYURL=http://mygoogle.com/service/books/123
# Generate body for curl request
generate_put_data()
{
cat <<EOF
{
"name":"harrypotter"
}
EOF
}
put_data=$(echo "$(generate_put_data)")
put_data_with_single_quotes="'$put_data'"
# Generate headers for curl request
header=myheader:coca-cola
header_with_double_quotes="\"$header\""
# The following function takes two inputs - a simple string variable (with no spaces or quotes) and the curl command string
function run_cmd() {
echo $1
echo $2
#Run the curl command
"$2"
#Check return code of the curl command
if [ "$?" -ne 0 ]; then
#do something with simple string variable
echo "$1"
echo "Job failed"
exit 1
else
#do something with simple string variable
echo "$1"
echo "Job Succeeded"
fi
}
# Run the bash function - run_cmd
run_cmd "mysimplestring" "curl -X PUT -H $header_with_double_quotes -d $put_data_with_single_quotes $MYURL"
However, when I try to run the above bash script, it fails at the point where I call run_cmd() function with the two inputs. I get the following error:
curl -X PUT -H "myheader:coca-cola" -d '{
"name":"harrypotter"
}' http://mygoogle.com/service/books/123: No such file or directory
Job failed
This error occurs on the line where "$2" is being executed in the run_cmd() function declaration.
Could someone help me understand where I am going wrong? Thanks!
"$2"
This will take the second argument and try to run it without doing any word splitting. It treats it as one string.
You're going to run into trouble passing in the curl command as one string. You'll do better if you pass it without quotes, just as if you typed it on the command line. You'll want to quote each of the variables but not quote the command as a whole.
run_cmd "mysimplestring" curl -X PUT -H "$header" -d "$put_data" "$MYURL"
Notice that you don't need the "with_quotes" variables any more. You don't have to do anything like that. The original plain values will work.
Now you can access the command using array syntax:
function run_cmd() {
local name=$1; shift
local cmd=("$#")
#Run the curl command
"${cmd[#]}"
}
By the way, this is a useless use of echo:
put_data=$(echo "$(generate_put_data)")
Make that:
put_data=$(generate_put_data)
Trying to debug my bash script. What's wrong with my syntax here? I'm trying to evaluate a parameter entered by the user and based on that run one of my IF-THEN statements. However, I'm getting a command not found.
Here's my script thus far:
if [[ $# != 4 ]]; then
echo "Usage: ./test.sh <ABC|XYZ> <owner> <db> <TARGETHOST>" 2>&1
exit 1
fi
case $1 in
ABC|XYZ)
filename="get-$1.sql"
;;
*)echo "Must enter ABC or XYZ"
exit 1
;;
esac
export OWNER=$2
export DB=$3
export HOST_NM=$4
export PORT=5432
export LOG="test-$1.log"
PSQL=`which psql`
if [[$1=="ABC"]]; then
RUNCLI=$("$PSQL" -h $HOST_NM -p $PORT -U $OWNER $DB -F $'\t' --no-align -f get-$1.sql | tee >> $LOG)
exit 1
else
echo "Error running report ..."
fi
if [[$1=="XYZ"]]; then
RUNCLI2=$("$PSQL" -h $HOST_NM -p $PORT -U $OWNER $DB -a -f get-$1.sql | tee >> $LOG)
exit 1
else
echo "Error running report ..."
fi
Error:
./test.sh: line 41: [[XYZ==ABC]]: command not found
Error running report ...
./test.sh: line 51: [[XYZ==XYZ]]: command not found
Error running report ...
Although the question is already answered in the comment section I want to give an answer and share some knowledge which is not obvious (at least it was not for me).
The if in bash just checks the return code of the following command, which means that instead of if [ condition ]... or if [[ condition ]]... you could also write if ./configure && make....
[ and [[ are commands or shell-built-ins, respectively, as well and not part of the if syntax. An which [ for instance returns /bin/[.
At this point it is obvious that you need spaces between the brackets and the condition since it is just just a set of parameters passed to a command.
If you have this in mind, you will never forget the spaces again.
When I run this by its self in the command line it seems to work fine, but when I have another script execute this, it doesn't work. Any ideas? I'm guessing it has to do with quotes, but not sure.
#!/bin/sh
#Required csvquote from https://github.com/dbro/csvquote
#TODO: Clean CSV File using CSVFix
#Version 3
echo "File Name: $1"
function quit {
echo "Quitting Script"
exit 1
}
function fileExists {
if [ ! -f "$1" ]
then
echo "File $1 does not exists"
quit
fi
}
function getInfo {
#Returns website url like: "http://www.website.com/info"
#Reads last line of a csv file, and gets the 2nd item.
RETURN=$(tail -n 1 $1 | csvquote | cut -d ',' -f 2 | csvquote -u)
echo $RETURN
}
function work {
CURLURL="http://127.0.0.1:9200/cj/_query"
URL=$(getInfo)
echo "URL: $URL"
CURLDATA='{ "query" : { "match" : { "PROGRAMURL" : '$URL' } } }'
#URL shows up as blank...???
echo "Curl Data: $CURLDATA"
RESPONSE=$(curl -XDELETE "$CURLURL" -d "$CURLDATA" -vn)
echo $RESPONSE
echo "Sleeping Allowing Time To Delete"
sleep 5s
}
fileExists $1
work $1
I cant see why a simpler version wont work: functions are useful, but I think there are too many, overcomplicating things, if what you are posting is the entirety of your script (in my opinion)
Your script is doing things using a broken lucky pattern: $1 variables are also arguments to shell functions as well as the main script. Think of them as local variables to a function. So when you are calling $(getInfo) it is calling that function with no argument, so actually runs tail -n 1 which falls back to stdin, which you are specifying to work as < $1. You could see this for yourself by putting echo getInfo_arg_1="$1" >&2 inside the function...
Note also you are not quoting $1 anywhere, this script is not whitespace in file safe, although this is only more likely to be a problem if you are having to deal with files sent to you from a Windows computer.
In the absence of other information, the following 'should' work:
#!/bin/bash
test -z "$1" && { echo "Please specify a file." ; exit 1; }
test -f "$1" || { echo "Cant see file '$1'." ; exit 1; }
FILE="$1"
function getInfo() {
#Returns website url like: "http://www.website.com/info"
#Reads last line of a csv file, and gets the 2nd item.
tail -n 1 "$1" | csvquote | cut -d ',' -f 2 | csvquote -u
}
CURLURL="http://127.0.0.1:9200/cj/_query"
URL=$(getInfo "$FILE")
echo "URL: $URL"
CURLDATA='{ "query" : { "match" : { "PROGRAMURL" : '$URL' } } }'
curl -XDELETE "$CURLURL" -d "$CURLDATA" -vn
echo "Sleeping Allowing Time To Delete"
sleep 5s
If it still fails you really need to post your error messages.
One other thing, especially if you are calling this from another script, chmod +x the script so you can run it without having to invoke it with bash directly. If you want to turn on debugging then put set -x near the start somewhere.
I have a script that runs curl. I want to be able to optionally add a -H parameter, if a string isn't empty. What's complex is the levels of quoting and spaces.
caption="Test Caption"
if [ "${caption}" != "" ]; then
CAPT=-H "X-Caption: ${caption}"
fi
curl -A "$UA" -H "Content-MD5: $MD5" -H "X-SessionID: $SID" -H "X-Version: 1" $CAPT http://upload.example.com/$FN
The idea is that the CAPT variable is either empty, or contains the desired -H header in the same form as the others, e.g., -H "X-Caption: Test Caption"
The problem is when run, it interprets the assignment as a command to be executed:
$bash -x -v test.sh
+ '[' 'Test caption' '!=' '' ']'
+ CAPT=-H
+ 'X-Caption: Test caption'
./test.sh: line 273: X-Caption: Test caption: command not found
I've tried resetting IFS before the code, but it didn't make a difference.
The key to making this work is to use an array.
caption="Test Caption"
if [[ $caption ]]; then
CAPT=(-H "X-Caption: $caption")
fi
curl -A "$UA" -H "Content-MD5: $MD5" -H "X-SessionID: $SID" -H "X-Version: 1" "${CAPT[#]}" "http://upload.example.com/$FN"
If you only need to know whether or not the caption is there, you can interpolate it when it needs to be there.
caption="Test Caption"
NOCAPT="yeah, sort of, that would be nice"
if [ "${caption}" != "" ]; then
unset NOCAPT
fi
curl ${NOCAPT--H "X-Caption: ${caption}"} -A "$UA" ...
To recap, the syntax ${var-value} produces value if var is unset.
I finally did get it to work. Part of the problem is specific to curl, in that when using the -H option to set custom headers, it seems to work best when everything after the -H (that is, both the custom header name and value) are protected by single quotes. Then, I needed to pass the constructed string through eval to get it to work.
To make this easier to read, I store a single quote in a variable named TICK.
Example:
TICK=\'
#
HDRS=""
HDRS+=" -H ${TICK}Content-MD5: ${MD5}${TICK}"
HDRS+=" -H ${TICK}X-SessionID: ${SID}${TICK}"
HDRS+=" -H ${TICK}X-Version: 1.1.1${TICK}"
HDRS+=" -H ${TICK}X-ResponseType: REST${TICK}"
HDRS+=" -H ${TICK}X-ID: ${ID}${TICK}"
if [ "${IPTC[1]}" != "" ]; then
HDRS+=" -H ${TICK}X-Caption: ${IPTC[1]}${TICK}"
fi
if [ "${IPTC[2]}" != "" ]; then
HDRS+=" -H ${TICK}X-Keywords: ${IPTC[2]}${TICK}"
fi
#
# Set curl flags
#
CURLFLAGS=""
CURLFLAGS+=" --cookie $COOKIES --cookie-jar $COOKIES"
CURLFLAGS+=" -A \"$UA\" -T ${TICK}${the_file}${TICK} "
eval curl $CURLFLAGS $HDRS -o $OUT http://upload.example.com/$FN