Issue in echo statement in shell scripting - bash

I have a very peculiar issue with a script that I have wrote today. I am trying to form an ip address from two variables namely url and port. I am getting the url value from a library script which echos 10.241.1.8 and the port number is 10000. Now if I concatenate both the url and the port into another variable ip, I get completely a strange result(:10000241.1.8). I have my code and its result below. Please help me with your suggestions to fix this.
clear
echo $(date +'%H:%M:%S')'>> "Sample Records" Script started...'
usage() {
echo ">> $ script.sh -ctoff 89 -env c -ns reporting -depPath /user/release/audit_prime_oozie"
echo "Usage: $ script.sh -ctoff <Cutoff number> -env <testing cluster. ex: s for staging,c,d,p and a> -ns <optional: hive namespace> -depPath <deployment path>"
}
# Function to validate if value of a parameter is not empty
validate () {
if [[ $flag != 1 ]]; then
if [[ $tmpVar == *"-"* ]] || [[ -z $tmpVar ]]; then
usage
exit 1
fi
fi
}
options=$#
if [[ -z $options ]]; then
usage
exit 1
fi
arguments=($options)
index=0
# Function to extract the parameter values
check (){
for x in $options
do
index=`expr $index + 1`
case $x in
-ctoff)
cutOff="${arguments[index]}"
tmpVar=$cutOff
validate $tmpVar
;;
-env)
env="${arguments[index]}"
tmpVar=$env
validate $tmpVar
;;
-ns)
ns="${arguments[index]}"
tmpVar=$ns
validate $tmpVar
;;
-depPath)
depPath="${arguments[index]}"
tmpVar=$depPath
validate $tmpVar
;;
esac
if [[ -z $ns ]];then
ns=reporting
fi
done
}
check $#
error_exit(){
echo "$1" 1>&2
exit 1
}
# Create the execution directory
user=$(id -u -n)
PWD=`pwd`
INSTALL_ROOT=$PWD
LOCAL_DIR="/tmp/$user/sample_duns"
if [[ ! -d $LOCAL_DIR ]]; then
mkdir -p $LOCAL_DIR
echo ">> Created local directory $LOCAL_DIR"
if [[ $? -ne 0 ]]; then
echo ">> Unable to create $LOCAL_DIR, writing to current folder $INSTALL_ROOT"
LOCAL_DIR=$INSTALL_ROOT
fi
fi
if [[ $(ls -A $LOCAL_DIR) ]]; then
echo ">> Removed the temp files from $LOCAL_DIR"
rm -r $LOCAL_DIR/*
fi
# create the file name
datestamp=$(date '+%Y%m%d%H')
outFile=sample_duns_$datestamp.txt
# Copy the contents from HDFS to Local directory
echo ">> Copying required files from HDFS"
hdfs dfs -copyToLocal $depPath/data-warehouse/config/server.properties $LOCAL_DIR || error_exit "Cannot copy files from HDFS! Exiting now.."
hdfs dfs -copyToLocal $depPath/data-warehouse/reporting/lib_getHiveServer2ip.sh $LOCAL_DIR || error_exit "Cannot copy files from HDFS! Exiting now.."
if [[ $? -ne 0 ]]; then
echo ">> Files missing. Exiting now.."
exit 1
fi
# Call the lib script to get appropriate hiveserver2 ip address from the supplied environment for beeline execution
echo ">> Reading the HiveServer2 ip"
chmod +x $LOCAL_DIR/lib_getHiveServer2ip.sh
url=$($LOCAL_DIR/lib_getHiveServer2ip.sh $env $LOCAL_DIR/server.properties)
echo url=$url
port=10000
echo ip=$url:$b
Here is my output from the terminal.
11:18:16>> "Sample Records" Script started...
>> Removed the temp files from /tmp/user/sample_duns
>> Copying required files from HDFS
>> Reading the HiveServer2 ip
url=10.241.1.8
:10000241.1.8
I am expecting the below result
ip=10.241.1.8:10000
Adding the lib_getHiveServer2ip.sh script below
. $2 # read properties file
if [[ $1 == "d" ]]; then
ip=$devHSer
elif [[ $1 == "c" ]]; then
ip=$crankHSer
elif [[ $1 == "s" ]]; then
ip=$stgHSer
elif [[ $1 == "p" ]]; then
ip=$prdHSer
elif [[ $1 == "a" ]]; then
ip=$alpHSer
else
echo ">> Invalid cluster ip encountered. Exiting now ..."
exit 1
fi
echo $ip

Your url variable contains a carriage return character for some reason. Check lib_getHiveServer2ip.sh for weirdness.
Pipe your echo output to hexdump to confirm.
Edit: looks like your properties file has bad line endings. Use the file utility to check.

Related

How to Ask User for Confirmation: Shell

I am new to shell, and my code takes two arguments from the user. I would like to confirm their arguments before running the rest of the code. I would like a y for yes to prompt the code, and if they type n for no, then the code will ask again for new arguments
Pretty much, if i type anything when I am asked to confirm, the rest of the code runs anyways. I tried inserting the rest of the code after the first then statement, but that didn't work either. I have also checked my code with ShellCheck and it all appears to be legal syntax. Any advice?
#!/bin/bash
#user passes two arguments
echo "Enter source file name, and the number of copies: "
read -p "Your file name is $1 and the number of copies is $2. Press Y for yes N for no " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]
then
echo "cloning files...."
fi
#----------------------------------------REST OF CODE
DIR="."
function list_files()
{
if ! test -d "$1"
then echo "$1"; return;
fi
cd ... || $1
echo; echo "$(pwd)":; #Display Directory name
for i in *
do
if test -d "$i" #if dictionary
then
list_files "$i" #recursively list files
cd ..
else
echo "$i"; #Display File name
fi
done
}
if [ $# -eq 0 ]
then list_files .
exit 0
fi
for i in "$#*"
do
DIR=$1
list_files "$DIR"
shift 1 #To read next directory/file name
done
if [ ! -f "$1" ]
then
echo "File $1 does not exist"
exit 1
fi
for ((i=0; i<$2; i++))
do
cp "$1" "$1$i.txt"; #copies the file i amount of times, and creates new files with names that increment by 1
done
status=$?
if [ "$status" -eq 0 ]
then
echo 'File copied succeaful'
else
echo 'Problem copying'
fi
Moving the prompts into a while loop might help here. The loop will re-prompt for the values until the user confirms them. Upon confirmation, the target code will be executed and the break statement will terminate the loop.
while :
do
echo "Enter source file name:"
read source_file
echo "Number of copies"
read number_of_copies
echo "Your file name is $source_file and the number of copies is $number_of_copies."
read -p "Press Y for yes N for no " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then
echo "cloning files...."
break ### <<<---- terminate the loop
fi
echo ""
done
#----------------------------------------REST OF CODE

Can anyone suggest why my code for postgresql walarchiveclean up is not working...I give two arguments as parameters but still it does not go forward

This is the code for wal archive clean up in postgresql. I am passing archive path and age before when wal needs to be cleaned up. Its a wrapper scrip but somehow its not working. Whenever I pass arguments it throws first message for passing correct parameters even though I am giving correct one.
#!/bin/bash
if [ "$#" -ne 2 ]; then
echo -e "This script helps in cleaning up archived log files in postgres. Give the command with parameters in order\t
sh walarchivecleanup.sh -p archivepath -a age (days) "
echo "Usage : walarchivecleanup.sh -p archivepath -a age"
echo -e "\t -p <value> -- Path to the archived WAL logs (e.g. /pg_data/pg_xlog/archive)"
echo -e "\t -a <value> -- Age of archived logs to keep (days), anything older will be deleted"
exit 1
else
echo -e "Do Nothing"
fi
archivepath=$1
age=$2
##########################################################
while getopts "p:a" opt;
do
case ${opt} in
p) archivepath=${OPTARG};;
a) age=${OPTARG};;
\? )
echo "Usage: sh walarchivecleanup.sh -p archivepath -a age (days) "
;;
esac
done
###############################################################
if [[ -z $archivepath ]]; then echo "Error: Missing archivepath"; exit 1; fi
if [[ -z $age ]]; then echo "Error: Age (-a) must be given"; exit 1; fi
if ! [[ -d $archivepath ]]; then
echo "Error: archivepath not found"; exit 1
else
cmd_path=$archivepath
fi
if [[ -n $archivecleanup ]]; then
if ! [[ -x $archivecleanup ]]; then
echo "Error: Command $archivecleanup not found or no permission to execute"; exit 1;
else
cmd_command="$archivecleanup"
fi
else
if ! `which pg_archivecleanup 1>/dev/null`; then echo "Error: Command pg_archivecleanup not found"; exit 1; fi
cmd_command="pg_archivecleanup"
fi
if [[ -n $age ]]; then
cmd_file="$(find ${archivepath}/ -type f -mtime +${age} -printf "%C# %f\n" |sort -n | tail -n 1 | awk '{print $NF}')"
else
cmd_file="$archivefile"
fi
execute="$cmd_command $cmd_path $cmd_file"
`$execute`
exit $?
echo "Unknown Error - Should never reach this part"
exit 1

unary operator expected with more than 1 argument

for var in "$#"
do
if test -z $var
then
echo "missing operand"
elif [ -d $var ]
then
echo "This is a directory"
elif [ ! -f $var ]
then
echo "The file does not exist"
else
basename=$(basename $var)
dirname=$(readlink -f $var)
inodeno=$(ls -i $var| cut -d" " -f1)
read -p "remove regular file $#" input
if [ $input = "n" ]
then exit 1
fi
mv $var "$var"_"$inodeno"
echo "$basename"_"$inodeno":"$dirname" >> $HOME/.restore.info
mv "$var"_"$inodeno" $HOME/deleted
fi
done
**Hello, the above code is trying to mimic the rm command in unix. Its purpose is to remove the file .
Eg if I type in bash safe_rm file1 , it works however if type in
bash safe_rm file1 file 2 , it prompts me to remove file 1 twice and gives me a unary operater expected for line 27(if [ $input = "n" ]).
Why does it not work for two files, ideally I would like it to prompt me to remove file1 and file 2.
Thanks
read -p "remove regular file $#" input
should probably be
read -p "remove regular file $var" input
That's the basic.
And this is how I'd prefer to do it:
for T in "$#"; do
if [[ -z $T ]]; then
echo "Target is null."
elif [[ ! -e $T ]]; then
echo "Target does not exist: $T"
elif [[ -d $T ]]; then
echo "Target can't be a directory: $T"
else
BASE=${T##*/}
DIRNAME=$(exec dirname "$T") ## Could be simpler but not sure how you want to use it.
INODE_NUM=$(exec stat -c '%i' "$T")
read -p "Remove regular file $T? "
if [[ $REPLY == [yY] ]]; then
# Just copied. Not sure about its logic.
mv "$T" "${T}_${INODE_NUM}"
echo "${BASE}_${INODE_NUM}:${DIRNAME}" >> "$HOME/.restore.info"
mv "${T}_${INODE_NUM}" "$HOME/deleted"
fi
fi
done

How to escape path separators in file path?

I'm writing my first bash script and having trouble assigning a file path to a variable:
$target="/etc/httpd/conf/httpd.conf"
It seems bash wants to interpret this with the "=" assignment operator resulting in the script throwing an error to the effect "No such file or directory."
Is there an easy way to do this? I've discovered I can assign a full path to a constant like this:
readonly TARGET=/etc/httpd/conf/httpd.conf
but that seems rather cumbersome. How would I perform string ops to modify/manipulate?
I've also discovered I can put full paths in an array like this:
declare -a cfile=('/root/.bashrc' '/etc/fstab')
All well and good, but how do I assign a file path to a variable?
== == == ==
finished! my first bash script - a basic config file manager
#!/bin/bash
# cfmgr.sh - configuration file manager bash script
# options: -get, -put
# '-get' creates SOURCEDIR/USERDIR and copies config files to USERDIR
# '-put' copies files in SOURCEDIR/USERDIR to system-defined locations on server
# purpose: helps with moving LAMP VMs to different hosts, bulk edits of
# of config files in editors like Notepad++, and backing up config files.
readonly SOURCEDIR=/usr/bin/_serverconfig
while [[ $# > 0 ]]
do
arg="$1"
shift
case $arg in
-put)
put=true
;;
-get)
get=true
;;
*)
badarg=true
;;
esac
done
clear
if [ $badarg ]; then
echo "Invalid argument. Use either 'scf.sh -put' or 'scf.sh -get' to put"\
"or get config files."
exit
elif [ $get ]; then
echo "Enter directory name to store files cfmgr will GET from this server:"
elif [ $put ]; then
echo "Enter directory name containing files cfmgr will PUT to this server:"
else
echo "Use either 'scf.sh -put' or 'scf.sh -get' to put or get config files."
exit
fi
read -e -i $SOURCEDIR"/" USERDIR
pattern=" |'"
if [[ $USERDIR =~ $pattern ]]; then
echo "Spaces not allowed. Please try again."
exit
fi
declare -a cfile=('/root/.bashrc' '/etc/fstab' '/etc/hosts' '/etc/networks'\
'/etc/php.ini' '/etc/nsswitch.conf' '/etc/ntp.conf' '/etc/resolv.conf'\
'/etc/sysctl.conf' '/etc/httpd/conf/httpd.conf' '/etc/selinux/config'\
'/etc/samba/smb.conf' '/etc/samba/smbusers' '/etc/security/limits.conf'\
'/etc/sysconfig/network' '/etc/sysconfig/network-scripts/ifcfg-eth0'\
'/etc/sysconfig/network-scripts/ifcfg-eth1')
if [ $get ]; then
if [[ -d "$USERDIR" ]]; then
echo $USERDIR "directory already exists. Please try again."
exit
else
mkdir -m 755 $USERDIR
fi
for file in ${cfile[#]}
do
if [ -e $file ]; then
rsync -q $file $USERDIR
if [ $? -eq 0 ]; then
sleep 0.1
printf "# "$file"\n"
fi
else
printf "not found: "$file"\n"
fi
done
elif [ $put ]; then
if [[ ! -d "$USERDIR" ]]; then
echo $USERDIR "directory does not exist. Please try again."
exit
fi
id=0
cd $USERDIR
for item in *
do
if [[ -f $item ]]; then
cdir[$id]=$item
id=$(($id+1))
fi
done
for file in ${cdir[#]}
do
case $file in
.bashrc)
idx=0
;;
fstab)
idx=1
;;
hosts)
idx=2
;;
networks)
idx=3
;;
php.ini)
idx=4
;;
nsswitch.conf)
idx=5
;;
ntp.conf)
idx=6
;;
resolv.conf)
idx=7
;;
sysctl.conf)
idx=8
;;
httpd.conf)
idx=9
;;
config)
idx=10
;;
smb.conf)
idx=11
;;
smbusers)
idx=12
;;
limits.conf)
idx=13
;;
network)
idx=14
;;
ifcfg-eth0)
idx=15
;;
ifcfg-eth1)
idx=16
;;
*)
printf "not found: "$file"\n"
continue
esac
target=${cfile[$idx]}
if [[ -e $target ]]; then
dtm=$(date +%Y-%m-%d)
mv $target $target"."$dtm
fi
source=$USERDIR"/"$file
dos2unix -q $source
rsync -q $source $target
if [ $? -eq 0 ]; then
sleep 0.1
printf "# "$target"\n"
fi
done
read -p "reboot now? (y|n)" selection
case $selection in
[Yy]*)
`reboot`
;;
*)
exit
;;
esac
fi
exit 0
Instead of
$target="/etc/httpd/conf/httpd.conf"
Use:
target="/etc/httpd/conf/httpd.conf"
When bash sees the former, it first substitutes in for "$target". If target was empty, then the line that bash tries to execute, after the variable substitution and quote removal steps, is:
=/etc/httpd/conf/httpd.conf
Since there is no file named "=/etc/httpd/conf/httpd.conf", bash returns with a "No such file or directory" error.

Check if passed argument is file or directory in Bash

I'm trying to write an extremely simple script in Ubuntu which would allow me to pass it either a filename or a directory, and be able to do something specific when it's a file, and something else when it's a directory. The problem I'm having is when the directory name, or probably files too, has spaces or other escapable characters are in the name.
Here's my basic code down below, and a couple tests.
#!/bin/bash
PASSED=$1
if [ -d "${PASSED}" ] ; then
echo "$PASSED is a directory";
else
if [ -f "${PASSED}" ]; then
echo "${PASSED} is a file";
else
echo "${PASSED} is not valid";
exit 1
fi
fi
And here's the output:
andy#server~ $ ./scripts/testmove.sh /home/andy/
/home/andy/ is a directory
andy#server~ $ ./scripts/testmove.sh /home/andy/blah.txt
/home/andy/blah.txt is a file
andy#server~ $ ./scripts/testmove.sh /home/andy/blah\ with\ a\ space.txt
/home/andy/blah with a space.txt is not valid
andy#server~ $ ./scripts/testmove.sh /home/andy\ with\ a\ space/
/home/andy with a space/ is not valid
All of those paths are valid, and exist.
That should work. I am not sure why it's failing. You're quoting your variables properly. What happens if you use this script with double [[ ]]?
if [[ -d $PASSED ]]; then
echo "$PASSED is a directory"
elif [[ -f $PASSED ]]; then
echo "$PASSED is a file"
else
echo "$PASSED is not valid"
exit 1
fi
Double square brackets is a bash extension to [ ]. It doesn't require variables to be quoted, not even if they contain spaces.
Also worth trying: -e to test if a path exists without testing what type of file it is.
At least write the code without the bushy tree:
#!/bin/bash
PASSED=$1
if [ -d "${PASSED}" ]
then echo "${PASSED} is a directory";
elif [ -f "${PASSED}" ]
then echo "${PASSED} is a file";
else echo "${PASSED} is not valid";
exit 1
fi
When I put that into a file "xx.sh" and create a file "xx sh", and run it, I get:
$ cp /dev/null "xx sh"
$ for file in . xx*; do sh "$file"; done
. is a directory
xx sh is a file
xx.sh is a file
$
Given that you are having problems, you should debug the script by adding:
ls -ld "${PASSED}"
This will show you what ls thinks about the names you pass the script.
Using -f and -d switches on /bin/test:
F_NAME="${1}"
if test -f "${F_NAME}"
then
echo "${F_NAME} is a file"
elif test -d "${F_NAME}"
then
echo "${F_NAME} is a directory"
else
echo "${F_NAME} is not valid"
fi
Using the "file" command may be useful for this:
#!/bin/bash
check_file(){
if [ -z "${1}" ] ;then
echo "Please input something"
return;
fi
f="${1}"
result="$(file $f)"
if [[ $result == *"cannot open"* ]] ;then
echo "NO FILE FOUND ($result) ";
elif [[ $result == *"directory"* ]] ;then
echo "DIRECTORY FOUND ($result) ";
else
echo "FILE FOUND ($result) ";
fi
}
check_file "${1}"
Output examples :
$ ./f.bash login
DIRECTORY FOUND (login: directory)
$ ./f.bash ldasdas
NO FILE FOUND (ldasdas: cannot open `ldasdas' (No such file or directory))
$ ./f.bash evil.php
FILE FOUND (evil.php: PHP script, ASCII text)
FYI: the answers above work but you can use -s to help in weird situations by checking for a valid file first:
#!/bin/bash
check_file(){
local file="${1}"
[[ -s "${file}" ]] || { echo "is not valid"; return; }
[[ -d "${file}" ]] && { echo "is a directory"; return; }
[[ -f "${file}" ]] && { echo "is a file"; return; }
}
check_file ${1}
Using stat
function delete_dir () {
type="$(stat --printf=%F "$1")"
if [ $? -ne 0 ]; then
echo "$1 directory does not exist. Nothing to delete."
elif [ "$type" == "regular file" ]; then
echo "$1 is a file, not a directory."
exit 1
elif [ "$type" == "directory" ]; then
echo "Deleting $1 directory."
rm -r "$1"
fi
}
function delete_file () {
type="$(stat --printf=%F "$1")"
if [ $? -ne 0 ]; then
echo "$1 file does not exist. Nothing to delete."
elif [ "$type" == "directory" ]; then
echo "$1 is a regular file, not a directory."
exit 1
elif [ "$type" == "regular file" ]; then
echo "Deleting $1 regular file."
rm "$1"
fi
}
https://linux.die.net/man/2/stat
https://en.m.wikipedia.org/wiki/Unix_file_types
A more elegant solution
echo "Enter the file name"
read x
if [ -f $x ]
then
echo "This is a regular file"
else
echo "This is a directory"
fi
Answer based on the title:
Check if passed argument is file or directory in Bash
This works also if the provided argument has a trailing slash .e.g. dirname/
die() { echo $* 1>&2; exit 1; }
# This is to remove the the slash at the end: dirName/ -> dirName
fileOrDir=$(basename "$1")
( [ -d "$fileOrDir" ] || [ -f "$fileOrDir" ] ) && die "file or directory $fileOrDir already exists"
Testing:
mkdir mydir
touch myfile
command dirName
# file or directory mydir already exists
command dirName/
# file or directory mydir already exists
command filename
# file or directory myfile already exists
#!/bin/bash
echo "Please Enter a file name :"
read filename
if test -f $filename
then
echo "this is a file"
else
echo "this is not a file"
fi
One liner
touch bob; test -d bob && echo 'dir' || (test -f bob && echo 'file')
result is true (0)(dir) or true (0)(file) or false (1)(neither)
This should work:
#!/bin/bash
echo "Enter your Path:"
read a
if [[ -d $a ]]; then
echo "$a is a Dir"
elif [[ -f $a ]]; then
echo "$a is the File"
else
echo "Invalid path"
fi

Resources