Pass wildcard to scp in a wrapper bash script - bash

I am writing a bash wrapper for scp'ing into and from a certain host with a certain username, like:
johny#bonjour:~/bin$ cat scpphcl
#!/bin/bash
download=false
upload=false
local=""
remote=""
usage()
{
echo "Usage: $0 -d[-u] -l <LocalPath> -r <RemotePath>"
exit 1
}
while getopts "h?dul:r:" opt; do
case "$opt" in
h|\?)
usage
;;
d)
download=true
upload=false
;;
u)
download=false
upload=true
;;
l)
local=$OPTARG
;;
r)
remote=$OPTARG
;;
esac
done
if [[ -z $local || -z $remote ]]; then
echo "Need to provide local and remote path."
usage
fi
if $download; then
scp somebody#somehost:"$remote" $local
elif $upload; then
scp $local somebody#somehost:"$remote"
else
echo "Neither download nor upload?"
exit 1
fi
if [[ $? -ne 0 ]]; then
echo "Something wrong happened in the scp process."
exit 1
fi
exit 0
It works well with the usual filenames, but if there is any wildcard in the local filename field, it will not work right.
johny#bonjour:~/test$ scpphcl -u -l * -r /u/somebody/temp
Need to provide local and remote path.
Usage: /Users/johny/bin/scpphcl -d[-u] -l <LocalPath> -r <RemotePath>
There is a walkaround, using sinqle quotes around the local file argument if there is a wildcard in it:
johny#bonjour:~/test$ scpphcl -u -l '*' -r /u/somebody/temp
But even this walkaround will not work, if the command is issued outside the folder test:
johny#bonjour:~/test$ cd ..
johny#bonjour:~$ scpphcl -u -l 'test/*' -r /u/somebody/temp
This doesn't work and will hang in the scp process.
Any help in how to pass the wildcard in local filenames with the bash wrapper?

It's probably best not to require your users to quote wildcard patterns. I'd instead change the interface of your program to accept any number of local paths, after the option arguments:
echo "Usage: $0 [-d|-u] [-r <RemotePath>] <LocalPath>..."
When reading options, consume them with shift:
while getopts "h?dur:" opt; do
case "$opt" in
h|\?)
usage
exit 0
;;
d)
download=true
upload=false
;;
u)
download=false
upload=true
;;
r)
remote="$OPTARG"
;;
*)
usage >&2
exit 1
;;
esac
done
shift $((OPTIND-1))
Now the remaining positional arguments are the local filenames (and can be accessed with "$#" - note the all-important double-quotes there):
if test -z "$*" # no LocalPath arguments!
then usage >&2; exit 1
elif $download
then exec scp somebody#somehost:"$remote" "$#"
elif $upload
then exec scp "$#" somebody#somehost:"$remote"
fi

Related

Shell script with parameters?

I have a shell script that takes parameters, below is the code..
Right now it will only accept parameters if passed if called like this: script.sh --mode=load (or -m=load)
Is there a way to modify this so that it can be called with or without the "=" sign, so that I can call: script.sh --mode load (or -m load)
Ideally needs to work in pure bash as I don't have access to install additional tools, etc.
for i in "$#"
do
case $i in
-m=*|--mode=*)
MODE="${i#*=}"
if [[ $MODE =~ ^(dump|load)$ ]]; then
echo "" > /dev/null
else
bark "Invalid --mode set, set this to dump or load.";
exit 1
fi
;;
-p=*|--db-path=*)
DBPATH="${i#*=}"
;;
-d=*|--dump-dir=*)
DUMPDIR="${i#*=}"
;;
-l=*|--list-file=*)
TABLES="${i#*=}"
# check if file exists on disk
if [ -e $TABLES ]
then
echo "" >> /dev/null
else
bark "Table file not found!";
exit 1
fi
;;
-t=*|--tenant-name=*)
TENANT="${i#*=}"
# check if tenant is correct
if [[ $TENANT =~ ^($TENANT_LIST)$ ]]; then
echo "" >> /dev/null
else
bark "Tenant name does not match, aborting.";
exit 1
fi
;;
-s|--shared)
SHARED=YES
;;
*) usage # unknown option
;;
esac
done
My bash version:
bash --version
GNU bash, version 4.3.22(1)-release (powerpc-ibm-aix5.1.0.0)
Loop on $#. When $1 is "-m", do a shift. So in the next loop $1 will now be the argument to the -m option.
script.sh --mode load
# FIRST LOOP
$# is "--mode load"
$1 is "--mode"
shift
# SECOND LOOP
$# is "load"
$1 is "load"
This is also useful if you can specify many arguments instead of just one like you have right now. Error checking should be done to validate your argument values, and if a user did script.sh --mode with no other argument.
Don't reinvent the wheel.
If you're OK with just 1 character options, use the bash builtin getopts
#!/bin/bash
while getopts :m:p:d:l:t:s opt; do
case $opt in
m) mode=$OPTARG ;;
p) dbpath=$OPTARG ;;
d) dumpdir=$OPTARG ;;
l) tables=$OPTARG
# test file existence
;;
t) tenant=$OPTARG
# test tenant
;;
s) shared=YES ;;
:) echo "Missing argument for option -$OPTARG" >&2
exit 2
;;
*) echo "Invalid option -$OPTARG" >&2
exit 2
;;
esac
done
shift $((OPTIND - 1))
cat << SHOW_VARS
I have:
mode=$mode
dbpath=$dbpath
dumpdir=$dumpdir
tables=$tables
tenant=$tenant
shared=$shared
rest of args=$*
SHOW_VARS
Otherwise, you may be able to use the external getopt program to help parse your args. I don't have an AIX box to test on, so YMMV
tempargs=$(
getopt \
-o m:d:l:t:s \
--long mode:,db-path:,dump-dir:,list-file:,tenant-name:,shared \
-- "$#"
)
if [[ $? -ne 0 ]]; then echo "Error..." >&2; exit 2; fi
eval set -- "$tempargs"
while true; do
case $1 in
-m|--mode) mode=$2; shift 2;;
-p|--db-path) dbpath=$2; shift 2;;
-d|--dump-dir) dumpdir=$2; shift 2;;
-l|--list-file) tables=$2
# test file existence
shift 2
;;
-t|--tenant-name) tenant=$2
# test tenant
shift 2
;;
-s|--shared) shared=YES; shift;;
--) shift; break ;;
*) echo "Error..." >&2; exit 2 ;;
esac
done

Processing command line options with multiple arguments in Bash [duplicate]

This question already has an answer here:
Storing bash script argument with multiple values
(1 answer)
Closed 5 years ago.
I have been researching on using bash scripts to process command-line arguments. I have multiple optional arguments, each of which have one or more operands. An example is:
./script.sh -f file1 file2 -s server1 server2
-f itself is optional, but must be followed by filename; -s is optional, can be used without any operands or operands.
I know I can force putting "" on operands so I only deal with arguments with one operand and can use case $1 $2 shift to process it.
But I am interested in doing so without quotes, just to save some typing for the users.
A rough idea would be read in "$#" as one string, and separate them by space, then locate arguments with -/-- and assign operands following them. Maybe I can use an array to do that?
Any suggestions would be welcome.
Thanks folks for your wonderful suggestions. After spending some more time I resolved to the solution below:
Simply put, I use case and few checks to determine if the argument is an option or not. I use only alter flag variables during argument processing and then use the flags to determine what functions I will perform. In a way that I can have options in different order.
main(){
# flags, 1 is false, 0 is true. it's the damn bash LOCAL_DEPLOY=1
SERVER_DEPLOY=1 DRY_RUN=0
FILES=("${ALLOWEDFILES[#]}");
DEPLOYTARGET=("${ALLOWEDSERVERS[#]}");
if [ $# -eq 0 ]
then
printf -- "Missing optins, perform DRY RUN\nFor help, run with -h/--help\n"
for target in "${FILES[#]}"; do generate "$target"; done
echo "....dry run: markdown files generated in rendered/"
exit 0
fi
while true ; do
case "$1" in
-f |--file) # required operands
case "$2" in
"") die $1 ;;
*)
FILES=($2)
for i in "${FILES[#]}"; do
if is_option $i; then die $1; fi # check for option
if ! check_allowed $i ${ALLOWEDFILES[#]}; then exit 1; fi
done;
shift 2;; # input FILES are good
esac ;;
-l|--local) # no operands expected
DRY_RUN=1 # turn off dryrun
LOCAL_DEPLOY=0 # turn on local deploy
shift ;;
-s|--server) # optional operands
case "$2" in
"") shift ;;
*)
DEPLOYTARGET=($2) # use input
for i in "${DEPLOYTARGET[#]}"; do
if is_option $i; then die $1; fi # check for option
if ! check_allowed $i ${ALLOWEDSERVERS[#]}; then exit 1; fi
done ; shift 2;; # use input value
esac
DRY_RUN=1
SERVER_DEPLOY=0
;;
-n|--dryrun) # dry-run:generate markdown files only
DRY_RUN=0
shift ;;
-h|--help) # docs
print_help
exit 0
;;
--) shift; break ;;
-?*)
printf 'ERROR: Unkown option: %s\nExisting\n\n' "$1" >&2
print_help
exit 1
shift
;;
*)
break ;;
esac
done
echo "choose files: ${FILES[#]}"
echo ""
# dry-run
if [ $DRY_RUN == 0 ]; then
echo "..perform dry run.."
for target in "${FILES[#]}"; do generate "$target"; done
echo "....dry run: markdown files generated in rendered/"
exit 0
fi
# local-deploy
if [ $LOCAL_DEPLOY == 0 ] && [ $SERVER_DEPLOY != 0 ]; then
echo "..deploy locally"
for target in "${FILES[#]}"; do
generate "$target" > /dev/null
deploylocal "$target"
done;
# sync hexo-gcs hexo-yby
cd "$(dirname $HEXOLOCATION)"
./syncRepo.sh
printf -- "....hexo-gcs hexo-yby synced\n"
cd $CURRENTLOCATION
fi
# server-deploy
if [ $SERVER_DEPLOY == 0 ]; then
echo "..deploy on servers: ${DEPLOYTARGET[#]}"
echo ""
for target in "${FILES[#]}"; do # deploy locally
generate "$target" > /dev/null
deploylocal "$target"
done
# sync hexo-gcs hexo-yby
cd "$(dirname $HEXOLOCATION)"
./syncRepo.sh
printf -- "....hexo-gcs hexo-yby synced\n"
cd $CURRENTLOCATION
# deploy to selected server: git or gcp
for dt in "${DEPLOYTARGET[#]}"; do
deployserver $dt
done
fi
}

Bash getopts doesn't show error for second option

I want a script to take in two options, both are required. if I pass one in, the script doesn't print an error requesting you to pass in a second one.
-bash-4.2$ bash test.sh -b
Invalid option: b requires an argument
-bash-4.2$ bash test.sh -p
Invalid option: p requires an argument
-bash-4.2$ bash test.sh -b sdfsfd
-bash-4.2$ bash test.sh -p sdfsfd
-bash-4.2$ bash test.sh -b sdfsfd -s sfd
Invalid option: s
Code
showHelp()
{
cat << EOF
Find files in client's folder and upload to S3 bucket.
Usage: $(basename $0) [-p PATH_TO_SEARCH] [-b S3 bucket]
OPTIONS:
-h Show this help message
-p Path to search
-b S3 Bucket
EOF
exit 1
}
while getopts ":p:b:h" o; do
case "${o}" in
h)
showHelp
;;
p)
p=${OPTARG}
;;
b)
b=${OPTARG}
;;
\? )
echo "Invalid option: $OPTARG";;
: )
echo "Invalid option: ${OPTARG} requires an argument";;
esac
done
shift $((OPTIND-1))
if [ -z "${p}" ]; then
showHelp
fi
if [ -z "${b}" ]; then
showHelp
fi
If you want to ensure you get both options, you can use something like:
no_p=1
no_b=1
while getopts ":p:b:h" o; do
case "${o}" in
h)
showHelp
;;
p)
p=${OPTARG}
no_p=0
;;
b)
b=${OPTARG}
no_b=0
;;
\? )
echo "Invalid option: $OPTARG";;
: )
echo "Invalid option: ${OPTARG} requires an argument";;
esac
done
[[ $no_p -eq 1 ]] && echo "No -p provided" && exit 1
[[ $no_b -eq 1 ]] && echo "No -b provided" && exit 1

How to escape path separators in file path?

I'm writing my first bash script and having trouble assigning a file path to a variable:
$target="/etc/httpd/conf/httpd.conf"
It seems bash wants to interpret this with the "=" assignment operator resulting in the script throwing an error to the effect "No such file or directory."
Is there an easy way to do this? I've discovered I can assign a full path to a constant like this:
readonly TARGET=/etc/httpd/conf/httpd.conf
but that seems rather cumbersome. How would I perform string ops to modify/manipulate?
I've also discovered I can put full paths in an array like this:
declare -a cfile=('/root/.bashrc' '/etc/fstab')
All well and good, but how do I assign a file path to a variable?
== == == ==
finished! my first bash script - a basic config file manager
#!/bin/bash
# cfmgr.sh - configuration file manager bash script
# options: -get, -put
# '-get' creates SOURCEDIR/USERDIR and copies config files to USERDIR
# '-put' copies files in SOURCEDIR/USERDIR to system-defined locations on server
# purpose: helps with moving LAMP VMs to different hosts, bulk edits of
# of config files in editors like Notepad++, and backing up config files.
readonly SOURCEDIR=/usr/bin/_serverconfig
while [[ $# > 0 ]]
do
arg="$1"
shift
case $arg in
-put)
put=true
;;
-get)
get=true
;;
*)
badarg=true
;;
esac
done
clear
if [ $badarg ]; then
echo "Invalid argument. Use either 'scf.sh -put' or 'scf.sh -get' to put"\
"or get config files."
exit
elif [ $get ]; then
echo "Enter directory name to store files cfmgr will GET from this server:"
elif [ $put ]; then
echo "Enter directory name containing files cfmgr will PUT to this server:"
else
echo "Use either 'scf.sh -put' or 'scf.sh -get' to put or get config files."
exit
fi
read -e -i $SOURCEDIR"/" USERDIR
pattern=" |'"
if [[ $USERDIR =~ $pattern ]]; then
echo "Spaces not allowed. Please try again."
exit
fi
declare -a cfile=('/root/.bashrc' '/etc/fstab' '/etc/hosts' '/etc/networks'\
'/etc/php.ini' '/etc/nsswitch.conf' '/etc/ntp.conf' '/etc/resolv.conf'\
'/etc/sysctl.conf' '/etc/httpd/conf/httpd.conf' '/etc/selinux/config'\
'/etc/samba/smb.conf' '/etc/samba/smbusers' '/etc/security/limits.conf'\
'/etc/sysconfig/network' '/etc/sysconfig/network-scripts/ifcfg-eth0'\
'/etc/sysconfig/network-scripts/ifcfg-eth1')
if [ $get ]; then
if [[ -d "$USERDIR" ]]; then
echo $USERDIR "directory already exists. Please try again."
exit
else
mkdir -m 755 $USERDIR
fi
for file in ${cfile[#]}
do
if [ -e $file ]; then
rsync -q $file $USERDIR
if [ $? -eq 0 ]; then
sleep 0.1
printf "# "$file"\n"
fi
else
printf "not found: "$file"\n"
fi
done
elif [ $put ]; then
if [[ ! -d "$USERDIR" ]]; then
echo $USERDIR "directory does not exist. Please try again."
exit
fi
id=0
cd $USERDIR
for item in *
do
if [[ -f $item ]]; then
cdir[$id]=$item
id=$(($id+1))
fi
done
for file in ${cdir[#]}
do
case $file in
.bashrc)
idx=0
;;
fstab)
idx=1
;;
hosts)
idx=2
;;
networks)
idx=3
;;
php.ini)
idx=4
;;
nsswitch.conf)
idx=5
;;
ntp.conf)
idx=6
;;
resolv.conf)
idx=7
;;
sysctl.conf)
idx=8
;;
httpd.conf)
idx=9
;;
config)
idx=10
;;
smb.conf)
idx=11
;;
smbusers)
idx=12
;;
limits.conf)
idx=13
;;
network)
idx=14
;;
ifcfg-eth0)
idx=15
;;
ifcfg-eth1)
idx=16
;;
*)
printf "not found: "$file"\n"
continue
esac
target=${cfile[$idx]}
if [[ -e $target ]]; then
dtm=$(date +%Y-%m-%d)
mv $target $target"."$dtm
fi
source=$USERDIR"/"$file
dos2unix -q $source
rsync -q $source $target
if [ $? -eq 0 ]; then
sleep 0.1
printf "# "$target"\n"
fi
done
read -p "reboot now? (y|n)" selection
case $selection in
[Yy]*)
`reboot`
;;
*)
exit
;;
esac
fi
exit 0
Instead of
$target="/etc/httpd/conf/httpd.conf"
Use:
target="/etc/httpd/conf/httpd.conf"
When bash sees the former, it first substitutes in for "$target". If target was empty, then the line that bash tries to execute, after the variable substitution and quote removal steps, is:
=/etc/httpd/conf/httpd.conf
Since there is no file named "=/etc/httpd/conf/httpd.conf", bash returns with a "No such file or directory" error.

Exclude string from wildcard in bash

I'm trying to adapt a bash script from "Sams' Teach Yourself Linux in 24 Hours" which is a safe delete command called rmv. The files are removed by calling rmv -d file1 file2 etc. In the original script a max of 4 files can by removed using the variables $1 $2 $3 $4.
I want to extend this to an unlimited number of files by using a wildcard.
So I do:
for i in $*
do
mv $i $HOME/.trash
done
The files are deleted okay but the option -d of the command rmv -d is also treated as an argument and bash objects that it cannot be found. Is there a better way to do this?
Thanks,
Peter
#!/bin/bash
# rmv - a safe delete program
# uses a trash directory under your home directory
mkdir $HOME/.trash 2>/dev/null
# four internal script variables are defined
cmdlnopts=false
delete=false
empty=false
list=false
# uses getopts command to look at command line for any options
while getopts "dehl" cmdlnopts; do
case "$cmdlnopts" in
d ) /bin/echo "deleting: \c" $2 $3 $4 $5 ; delete=true ;;
e ) /bin/echo "emptying the trash..." ; empty=true ;;
h ) /bin/echo "safe file delete v1.0"
/bin/echo "rmv -d[elete] -e[mpty] -h[elp] -l[ist] file1-4" ;;
l ) /bin/echo "your .trash directory contains:" ; list=true ;;
esac
done
if [ $delete = true ]
then
for i in $*
do
mv $i $HOME/.trash
done
/bin/echo "rmv finished."
fi
if [ $empty = true ]
then
/bin/echo "empty the trash? \c"
read answer
case "$answer" in
y) rm -i $HOME/.trash/* ;;
n) /bin/echo "trashcan delete aborted." ;;
esac
fi
if [ $list = true ]
then
ls -l $HOME/.trash
fi
You can make use of shift here.
Once you find -d is one of the options in the switch, you can shift and get rid of -d from the positional parameters. Next you can
until [ -z $1 ] ; do
mv $1 $HOME/.trash
shift
done
getopts sets OPTIND to the index of the first argument after the options. (#)
So after parsing the options you can do:
shift $OPTIND-1
to remove the options from the argument list.
Then use "$#" rather than $*, and you can handle files with spaces in them.
Thanks a lot!
I changed the code to read:
#!/bin/bash
# rmv - a safe delete program
# todo: add ability to handle wildcards
# uses a trash directory under your home directory
mkdir $HOME/.trash 2>/dev/null
# four internal script variables are defined
cmdlnopts=false
delete=false
empty=false
list=false
# uses getopts command to look at command line for any options
while getopts "dehl" cmdlnopts; do
case "$cmdlnopts" in
d ) echo -e "deleting: \n" "${#:2}" ; delete=true ;;
e ) echo -e "emptying the trash..." ; empty=true ;;
h ) echo -e "safe file delete v1.0"
echo -e "rmv -d[elete] -e[mpty] -h[elp] -l[ist] file [...]" ;;
l ) echo -e "your .trash directory contains:" ; list=true ;;
esac
done
shift $OPTIND-1
if [ $delete = true ]
then
for i in $#
do
mv $i $HOME/.trash
done
echo "rmv finished."
fi
then
/bin/echo "empty the trash? \c"
read answer
case "$answer" in
y) rm -i $HOME/.trash/* ;;
n) /bin/echo "trashcan delete aborted." ;;
esac
fi
if [ $list = true ]
then
ls -l $HOME/.trash
fi
This deletes the files as desired but I get this error:
/home/peter/rmv: line 21: shift: 2-1: numeric argument required
mv: invalid option -- 'd'
Try `mv --help' for more information.
You need to use
shift $(($OPTIND - 1))
to get red of the processed command line args. Try this version:
#!/bin/bash
# rmv - a safe delete program
# uses a trash directory under your home directory
mkdir -p $HOME/.trash
# uses getopts command to look at command line for any options
while getopts "dehl" cmdlnopts; do
case "$cmdlnopts" in
d ) delete=true;;
e ) echo "emptying the trash..." ; empty=true ;;
h ) echo "safe file delete v1.0"
echo "rmv -d[elete] -e[mpty] -h[elp] -l[ist] files" ;;
l ) echo "your .trash directory contains:" ; list=true ;;
esac
done
shift $(($OPTIND - 1))
if [ -n "${delete}" ]; then
echo "deleting: " "${#}"
mv ${#} $HOME/.trash
echo "rmv finished."
fi
if [ -n "${empty}" ]; then
read -p "empty the trash? " answer
case "$answer" in
y) rm -i $HOME/.trash/* ;;
n) echo "trashcan delete aborted." ;;
esac
fi
if [ -n "${list}" ]; then
ls -l $HOME/.trash
fi
Late to the party, but for Googlers, this will generate the error Peter describes:
shift $OPTIND-1
while the syntax in Jurgen's reply will not:
shift $(($OPTIND-1))
The problem is that $OPTIND-1 is interpreted as a string, and shift can't use a string as an argument. $(( )) is Bash's arithmetic expansion operator. You put a string inside it, the string is evaluated as an arithmetic expression, and the value returned.

Resources