Changing the value of an argument and using options in bash - bash

I'm writing a bash script to automate some job, and in the meantime practice with bash. I'm writing a script to automate some git cloning and updating stuff. My script will have three options (-g, -f, -h). When someone types -h it should display the help message (which I have written but omitted it below), when someone writes -g, it should accept at least one argument, but it can also accept second one as optional. From which, the first one will be the url of the repository to clone and second one will be the directory, where to clone it. On the other hand, if someone types -f, the script should get just one argument, just a file name. Then I want to read the file, line by line, and do git cloning and updating for each git url inside the file.
If I run the script I get the following error message for each option, and if I call it without any option, or with a valid option followed by some other argument, it still doesn't do anything, just returns:
./gitupdate.sh: option requires an argument -- g
I guess it doesn't use $2 and $3 somehow in my code. But then again, when I pass -h, all it has to do is call the help function and display the message, doesn't need to use any other argument.
I guess the problem is because I have something wrong at the bottom, where I use getopts to get the option name specified by the user. In my code I assume that the option is the first argument, $1, and then the second one $2 is the url or filename, according to the option specified, and the $3 is the optional one that works only with -g option.
Below you can find my code:
#!/bin/bash
declare default_directory=$HOME
declare action
declare src
function clone_repo() {
if [ -n "$2" ]; then
if [ "$(ls -A "$2" 2>/dev/null)" ]; then
cd "$2"
else
git clone "$1" "$2"
fi
else
git clone "$1"
#TODO: Get the directory name created by cloning
# and cd to it.
fi
git remote add upstream "$1"
git fetch upstream
}
function read_repos_from_file() {
if [ -f "$1" ]; then
while read -r line; do
clone_repo "$line" "$2"
done < "$1"
else
echo -e "Error: The specified file could not be found."
exit 1
fi
}
while getopts "f:h:r" option
do
case "${option}" in
f) action=read_repos_from_file; src="$OPTARG";;
g) action=clone_repo; src="$OPTARG";;
h) help ; exit 1 ;;
esac
done
shift $((OPTIND-1))
[ -z "$action" ] && ( help; exit 1 )
If someone could help me, I would be glad.
EDIT: Corrected some typo mistakes in the above code. And updated the error message. The script doesn't work correctly. I guess I need to change something in my code, to make it do something, and actually get $2 and $3 arguments. It even doesn't display the help message, when -h option is passed, and all I do there is call the help function that I have previously created. Maybe I need to somehow change my getopts part.
EDIT 2: Made the advised changes, and changed the above code.

git() is the beginning of a function definition (the keyword function is optional when the function name is followed by parentheses). If you want to call a function git() you need to define it first, and call it without the parentheses:
function git() {
# do stuff
}
git
It's not good practice to create functions with the same name as existing binaries, though. In your case you should probably just call git clone with the line read from the file:
while read -r line; do
git clone "$line"
done < "${file}"
Edit: Updated, since the question changed significantly.
Your argument processing is … weird, to be frank. When you're using an option parser, you shouldn't work around the way that option parser works. "g:" means an option -g with exactly one argument. Don't try to make it an option with more than one argument, one of them being optional on top of it. If you need an additional (optional) argument for an output directory, make that either another option (e.g. "d:") or a non-option argument.
I'd suggest to change your option-processing to something like this:
while getopts "f:g:h" option; do
case "$option" in
f) action=file; src="$OPTARG";;
g) action=repo; src="$OPTARG";;
h) help; exit 1;;
esac
done
shift $((OPTIND-1))
[ -z "$action" ] && ( help; exit 1 )
After this "$#" holds only non-option arguments (in this case the optional output directory), so you could call your functions like this:
$action $src "$#"
with the functions simplified to something like this:
function repo() {
if [ -n "$2" ]; then
if [ "$(ls -A "$2" 2>/dev/null)" ]; then
cd "$2"
else
git clone "$1" "$2"
fi
else
git clone "$1"
fi
...
}
function file() {
if [ -f "$1" ]; then
while read -r line; do
repo "$line" "$2"
done < "$1"
else
echo "Error: The specified file could not be found."
exit 1
fi
}
On a more general note, you should make your names more self-explanatory. Better names for functions for cloning a repository or reading repositories from a file would be something like clone_repo and read_repos_from_file respectively. Also, -c or -r would be better mnemonics for an option that triggers the cloning of a repository.

Related

How can I use getopts in a script that appends lines from files in a separate directory to a new file?

I am trying to write a bash script that takes in a directory, reads each file in the directory, and then appends the first line of each file in that directory to a new file. When I hard-code the variables in my script, it works fine.
This works:
#!/bin/bash
rm /local/SomePath/multigene.firstline.btab
touch /local/SomePath/multigene.firstline.btab
btabdir=/local/SomePath/test/*
outfile=/local/SomePath/multigene.firstline.btab
for f in $btabdir
do
head -1 $f >> $outfile
done
This does not work:
#!/bin/bash
while getopts ":d:o:" opt; do
case ${opt} in
d) btabdir=$OPTARG;;
o) outfile=$OPTARG;;
esac
done
rm $outfile
touch $outfile
for f in $btabdir
do
head -1 $f >> $outfile
done
Here is how I call the script:
bash /local/SomePath/Scripts/btab.besthits.wBp-q_wBm-r.sh -d /local/SomePath/test/* -o /local/SomePath/out.test/multigene.firstline.btab
And here is what I get when I run it:
rm: missing operand
Try 'rm --help' for more information.
touch: missing file operand
Try 'touch --help' for more information.
/local/SomePath/Scripts/btab.besthits.wBp-q_wBm-r.sh: line 23: $outfile: ambiguous redirect
Any suggestions? I'd like to be able to use getopts so I can make the script more generic. Thanks!
You have to pay extra attention to quoting and globbing when writing bash scripts.
When you call the script with a glob (* here) it gets expanded and split into words by your shell. This happends before your script even gets executed.
If you for example do cat *.txt cat will get all .txt files in the directory as its arguments. It will be the same as calling cat afile.txt nextfile.txt (and so on). Cat will never see the asterisk.
In your script it means that the input -d /local/SomePath/test/* gets expanded som something like /local/SomePath/test/someFile /local/SomePath/test/someOtherFile /test/someThirdFile.
Subsequently getopts only takes the first file after -d as for $btabdir and the -o doesn't get handled in the case switch.
I suggest you start by quoting every variable, preferable in the "${name}" style, and only invoke the script with quoted input.
It might also be send in a directory path, test that it is a directory (test -d), and change your for loop to for f in "${btabdir}"/*
This also works:
head -n1 -q /local/SomePath/test/* >> /local/SomePath/out.test/multigene.firstline.btab
I think the right answer here is "don't do it that way." :-)
The reason your current script isn't working may be that the wildcard is expanded by your interactive shell, not by your script. Try running your command with an echo at the beginning of the line for a hint at what's really happening. Once getopts sees the second of the matched files in the glob, it stops processing options, so -o never gets read, and $outfile remains unset. And since you don't quote your variable in rm $outfile, it's as if you're running rm without options. Test the difference in your shell between rm alone and rm "".
Also, what happens to your for loop if there's a space in a filename? Since you have bash, you have arrays. And arrays are much better for processing lists of files.
Perhaps use something like this instead:
#!/bin/bash
# initialize an array
files=()
while getopts :d:o: opt; do
case "$opt" in
d)
if [[ ! -d "$OPTARG" ]]; then
printf 'ERROR: not a directory: %s\n' "$OPTARG" >&2
exit 65
fi
# add to the array
files+=( "$OPTARG"/* )
;;
o) outfile="$OPTARG" ;;
*)
printf 'ERROR: unknown option: %s\n' "$opt" >&2
exit 64
;;
esac
done
if ! rm -f "$outfile" && touch "$outfile"; then
printf 'ERROR: cannot create %s\n' "$outfile" >&2
exit 73
fi
for f in "${files[#]}"; do
read -r < "$f"
printf '%s\n' "$REPLY"
done > "$outfile"
Here are some highlights of the changes....
We're using arrays, of course. The array ${files[#]} will contain one-file-per-record, without relying on whitespace, so with proper quoting you'll avoid problems with special characters in filenames.
We test for more error conditions, and actually show errors and exit if we see them. (The exit values are sysexits.)
Instead of using head, we use read and a single redirect to $outfile. This saves multiple forks to an external program, and multiple fopen() calls to your output file.
Note that the argument to -d should be a directory, not a glob. And you can specify options multiple times. Multiple -d options will be added together, but only the last -o option will be used.

Shellscript - Show error for specific argument when using mv

I'm currently writing code for a script to move files to a "dustbin".
As it stands - the code works fine but I want it to produce a message when a file has been moved correctly and a message when a specific file has failed to move/doesn't exist.
My code will accept multiple filenames for input, e.g.
# del test1.txt *.html testing.doc
# Successfully moved to Dustbin
However if only one of these does not exist it still produces an error message.
How do I do this but still allow it to accept arguments as seen in the above example?
My code is as follows:
#!/bin/sh
mv -u "$#" /root/Dustbin 2>/dev/null
# END OF SCRIPT
Sorry for what is probably an obvious question! I'm completely new to shellscript !
Many thanks
You would have to iterate over the arguments and try to move each one:
for path in "$#"; do
if mv -u "$path" /root/Dustbin 2>/dev/null; then
echo "Success"
else
printf 'Failed to move %s\n' "$path"
fi
done
As a shorthand for iterating over the arguments you can omit in "$#" like
for path; do
if mv -u "$path" /root/Dustbin 2>/dev/null; then
echo "Success"
else
printf 'Failed to move %s\n' "$path"
fi
done

Create shell sub commands by hierarchy

I'm trying to create a system for my scripts -
Each script will be located in a folder, which is the command itself.
The script itself will act as a sub-command.
For example, a script called "who" inside a directory called "git",
will allow me to run the script using git who in the command line.
Also, I would like to create a sub command to a psuedo-command, meaning a command not currently available. E.g. some-arbitrary-command sub-command.
Is that somehow possible?
I thought of somehow extending https://github.com/basecamp/sub to accomplish the task.
EDIT 1
#!/usr/bin/env bash
command=`basename $0`
subcommand="$1"
case "$subcommand" in
"" | "-h" | "--help" )
echo "$command: Some description here" >&2
;;
* )
subcommand_path="$(command -v "$command-$subcommand" || true)"
if [[ -x "$subcommand_path" ]]; then
shift
exec "$subcommand_path" "${#}"
return $?
else
echo "$command: no such command \`$subcommand'" >&2
exit 1
fi
;;
esac
This is currently the script I run for new custom-made commands.
Since it's so generic, I just copy-paste it.
I still wonder though -
can it be generic enough to just recognize the folder name and create the script by its folder name?
One issue though is that it doesn't seem to override the default command name, if it supposed to replace it (E.g. git).
EDIT 2
After tinkering around a bit this is what I came to eventuall:
#!/usr/bin/env bash
COMMAND=`basename $0`
SUBCOMMAND="$1"
COMMAND_DIR="$HOME/.zsh/scripts/$COMMAND"
case "$SUBCOMMAND" in
"" | "-h" | "--help" )
cat "$COMMAND_DIR/help.txt" 2>/dev/null ||
command $COMMAND "${#}"
;;
* )
SUBCOMMAND_path="$(command -v "$COMMAND-$SUBCOMMAND" || true)"
if [[ -x "$SUBCOMMAND_path" ]]; then
shift
exec "$SUBCOMMAND_path" "${#}"
else
command $COMMAND "${#}"
fi
;;
esac
This is a generic script called "helper-sub" I symlink to all the script directories I have (E.g. ln -s $HOME/bin/helper-sub $HOME/bin/ssh).
in my zshrc I created this to call all the scripts:
#!/usr/bin/env bash
PATH=${PATH}:$(find $HOME/.zsh/scripts -type d | tr '\n' ':' | sed 's/:$//')
export PATH
typeset -U path
for aliasPath in `find $HOME/.zsh/scripts -type d`; do
aliasName=`echo $aliasPath | awk -F/ '{print $NF}'`
alias ${aliasName}=${aliasPath}/${aliasName}
done
unset aliasPath
Examples can be seen here: https://github.com/iwfmp/zsh/tree/master/scripts
You can't make a directory executable as a script, but you can create a wrapper that calls the scripts in the directory.
You can do this either with a function (in your profile script or a file in your FPATH) or with a wrapper script.
A simple function might look like:
git() {
local subPath='/path/to/your/git'
local sub="${1}" ; shift
if [[ -x "${subPath}/${1}" ]]; then
"${subPath}/${sub}" "${#}"
return $?
else
printf '%s\n' "git: Unknown sub-command '${sub}'." >&2
return 1
fi
}
(This is the same way that the sub project you linked works, just simplified.)
Of course, if you actually want to create a sub-command for git specifically (and that wasn't just an example), you'll need to make sure that the built-in git commands still work. In that case you could do like this:
git() {
local subPath='/path/to/your/git'
local sub="${1}"
if [[ -x "${subPath}/${sub}" ]]; then
shift
"${subPath}/${sub}" "${#}"
return $?
else
command git "${#}"
return 1
fi
}
But it might be worth pointing out in that case that git supports adding arbitrary aliases via git config:
git config --global alias.who '!/path/to/your/git/who'

Finding error with shell script

I wrote a script with mbratch's help.
I run as below;
./scriptname folder1
However, I see neither error nor results and I'm not sure what's wrong.
sh -x ./scriptname folder1
+ MAIN
+ check_directories
+ [ -d ]
This works fine for me:
Note: updated to support additional options.
opt="$1"
folder1="$2"
folder2="$3"
case "$opt" in
-d)
if [ -d "${folder1}" ] && [ -d "${folder2}" ] ; then
for i in "${folder1}/*" ; do
echo "test <$i>"
if [ -f "${folder2}/${i##*/}" ] ; then
echo "<${i##*/}>"
else
echo "! <${i##*/}>"
fi
done
fi
;;
# Example option
-h)
# Print help and exit.
;;
# Default case
*)
echo "Unknown option '$opt'" >&2
exit 1
;;
esac
Try replacing ~/ with $HOME/, and be sure to set folder1 and folder2 before using them. Note also that this will break if your directory or file names include spaces. In that case, use find; check the find man page for details.
Is that the entirety of your script? The variables $folder1 are never defined anywhere. By default, the program will take the first two chunks of text in as $1 and $2, so use those variables instead.
Put some echo statements in there to see what variables have what values.
You have a for loop already, so go with that. However you might have to put the part where you are getting a file list inside of $() to have it assigned to a variable, and then loop over it.
Do a quick search on "Looping through files in bash" and you will find a good template for the for loop.

Bash function - return parent script file path

I have a bash script containing a function which is sourced by a number of different bash scripts. This function may fail based on its input, and I'd like to create logging within the function to identify what script(s) are causing failures.
E.g.,
source /path/to/function.sh
The closest I've come is this:
ps --no-heading -ocmd -p $$
This works well enough if the full file path is used to run the parent script, returning:
/bin/bash /path/to/parent.sh
But it fails to provide the full path if the parent script is run from a relative path, returning:
/bin/bash ./parent.sh
Ideally, I'd like a way to reliably return the parent script file path for both cases.
I suppose I could have each parent script pass its file path to the function (via $0 or similar), but that seems hard to enforce and not terribly elegant.
Any ideas, or alternative approaches? Should I not worry about the relative path case, and just use full/absolute file paths for everything?
Thanks!
I'm using Centos 5.9.
Bash version -
GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)
You can use readlink to follow all symbolic links to get an absolute path.
echo $(readlink -f $0)
As soon as the parent script starts export
"`pwd`/$0"
or so, into an env variable, say ORIG_SCRIPT, then in the function just use ORIG_SCRIPT.
You need to do this as soon as the script starts because $0 may be relative to the PWD and if you later change PWD before you need the value of ORIG_SCRIPT, it gets unnecessarily complicated.
Update:
Since you know the pid by $$, you may get something from /proc/<PID>/cmdline but I don't know how exactly this one works right now.
You could use ${BASH_SOURCE[1]} to get the script that calls the function but that is not always on absolute path form. You could get the absolute path of it by readlink -m, realpath, or other shell-script based solutions, but if your script changes directory from time to time, conversion of relative paths to absolute paths would no longer be accurate as those tools base from the current directory to get the actual form.
There's a workaround however but this requires that you won't change directories in your scripts before calling (sourcing) the script that contains the function. You would have to save the current directory in that script itself then base forming of absolute paths through that directory. You are free to change directories after the script has already been included. As an example:
ORIGINAL_PWD=$PWD
function x {
local CALLING_SCRIPT="${BASH_SOURCE[1]}"
if [[ -n $CALLING_SCRIPT ]]; then
if [[ $CALLING_SCRIPT == /* ]]; then
CALLING_SCRIPT=$(readlink -m "$CALLING_SCRIPT")
else
CALLING_SCRIPT=$(readlink -m "$ORIGINAL_PWD/$CALLING_SCRIPT")
fi
echo "Calling script: $CALLING_SCRIPT"
else
echo "Caller is not a script."
fi
}
Or
ORIGINAL_PWD=$PWD
function getabspath {
local -a T1 T2
local -i I=0
local IFS=/ A
case "$1" in
/*)
read -r -a T1 <<< "$1"
;;
*)
read -r -a T1 <<< "/$PWD/$1"
;;
esac
T2=()
for A in "${T1[#]}"; do
case "$A" in
..)
[[ I -ne 0 ]] && unset T2\[--I\]
continue
;;
.|'')
continue
;;
esac
T2[I++]=$A
done
case "$1" in
*/)
[[ I -ne 0 ]] && __="/${T2[*]}/" || __=/
;;
*)
[[ I -ne 0 ]] && __="/${T2[*]}" || __=/.
;;
esac
}
function x {
local CALLING_SCRIPT="${BASH_SOURCE[1]}"
if [[ -n $CALLING_SCRIPT ]]; then
if [[ $CALLING_SCRIPT == /* ]]; then
getabspath "$CALLING_SCRIPT"
else
getabspath "$ORIGINAL_PWD/$CALLING_SCRIPT"
fi
echo "Calling script: $__"
else
echo "Caller is not a script."
fi
}
You could also play around with FUNCNAME and BASH_LINENO to be more specific with the errors. I'm just not sure if they're already supported in Bash 3.2.
If you actually had Bash 4.0+ you could make use of associative arrays to map absolute paths with it but if there are two scripts with the same names or are called with almost similar names, one value could be overridden. There's no fix to that since we can't choose our keys from BASH_SOURCE.
Added Note: You could also prevent your script from being unnecessarily sourced multiple times as it only requires to be once through a solution like Shell Script Loader. You might find convenience through it as well.

Resources