Shell scripting debug help - Iterating through files in a directory - shell

#!/bin/sh
files = 'ls /myDir/myDir2/myDir3/'
for file in $files do
echo $file
java myProg $file /another/directory/
done
What i'm trying to do is iterate through every file name under /myDir/myDir2/myDir3/, then use that file name as the first argument in calling a java program (second argument is "/another/directory")
When I run this script: . myScript.sh
I get this error:
-bash: files: command not found
What did I do wrong in my script? Thanks!

Per Neeaj's answer, strip off the whitespace from files =.
Better yet, use:
#!/bin/sh -f
dir=/myDir/MyDir2/MyDir3
for path in $dir/*; do
file=$(basename $path)
echo "$file"
java myProg "$file" arg2 arg3
done
Bash is perfectly capable of expanding the * wildcard itself, without spawning a copy of ls to do the job for it!
EDIT: changed to call basename rather than echo to meet OP's (previously unstated) requirement that the path echoed be relative and not absolute. If the cwd doesn't matter, then even better I'd go for:
#!/bin/sh -f
cd /myDir/MyDir2/MyDir3
for file in *; do
echo "$file"
java myProg "$file" arg2 arg3
done
and avoid the calls to basename altogether.

strip off the whitespace in and after files = as files=RHS of assignment

Remove the space surrounding the '=' : change
files = 'ls /myDir/myDir2/myDir3/'
into:
files='ls /myDir/myDir2/myDir3/'
and move the 'do' statement to its own line:
for file in $files
do
....

quote your variables and no need to use ls.
#!/bin/sh
for file in /myDir/myDir2/*
do
java myProg "$file" /another/directory/
done

Related

loop in bash for parameters that uses multiple sources

I have a bash script which works like this;
File structure;
get.sh
loop.sh
config/param1.conf
config/param2.conf
Usage of the main script, get.sh;
./get.sh <param> i.e ./get.sh param1, ./get.sh param2
So when you run the script with specific params it fetches the config files from config/<param>.conf
What I'm trying to do is to run this second script, ./loop.sh so it runs the ./get.sh <param> for you in a loop using the params inside config folder, without .conf extensions.
Here's my loop.sh;
#!/bin/bash
# run the script with the first param you found inside ~/config/
# folder without including it's .conf extension,
# wait for 5 seconds and then do the same with the 2nd param you found
for i in $(find ~/config -name '*.conf'); do
./get.sh $(basename $i) | cut -d'.' -f 1
sleep 5
done
but this one is just displaying the params inside config folder and doing nothing else.
`
Inside of the config/param1.conf;
var=Hello1
Inside of the config/param2.conf;
var=Hello2
Inside of the get.sh;
#!/bin/bash
function testFunction {
echo "$var"
}
cfg_file=$1
if [ -f "$cfg_file" ]; then
. "$cfg_file"
testFunction $1
exit 1
else
echo "$1.conf doesn't exist"
exit 1
fi
So after all, when you run the loop.sh, the expected behavior should be printing the Hello1 and Hello2 strings into shell.
How can I fix this?
In loop (using .sh extensions for bash scripts is not great practice):
#!/bin/bash
for i in ~/config/*.conf; do
i_basename=${i##*/} # change ~/config/foo.conf to just foo.conf
i_basename=${i_basename%.conf} # change foo.conf to just foo
./get "$i_basename"
sleep 5
done
The ${var##prefix} and ${var%suffix} syntax is parameter expansion; with ## it removes the longest match from the beginning (so for */, everything up to the last /); with %, it removes the shortest successful match starting from the end.
In get:
#!/bin/bash
# Using POSIX function syntax; see http://wiki.bash-hackers.org/scripting/obsolete
testFunction() {
echo "$var"
}
cfg_file="config/$1.conf"
if [ -f "$cfg_file" ]; then
. "$cfg_file"
testFunction
else
echo "$cfg_file doesn't exist"
exit 1
fi
With respect to lack of .sh extensions -- UNIX commands don't conventionally have extensions (you run ls, not ls.elf); similarly, when you install a Python module built with setuptools, it doesn't put .py extensions on shims it creates in /usr/local/bin, even if the libraries those executable shims invoke do have such extensions. Moreover, bash and POSIX sh are two different languages: Bash scripts often don't work correctly when started with sh some-bash-only-script.sh (as unlike bash, sh isn't guaranteed to support language features like arrays), but the extension implies that they will.
-name expects the parameter to be just a filename, not a whole pathname. You should use the directory as a regular argument to find, not as part of -name.
for i in $(find ~/config -name '*.conf'); do
Don't use basename, you should pass the entire pathname to script.sh.
Then in script.sh you should should use $1 as the whole path to the config file, rather than concatenating it with a directory prefix.
cfg_file=$1
I don't see any point in this:
case $1 in
$1) cfg=$1 ;;
esac
The case will always be true, how can $1 not match $1? Remove that code.

How do I move all arguments entered in a shell script to a folder?

I am trying to move all the arguments that are given when the script is run, to a RecycleBin directory. I am setting RecycleBin to be a directory in my home folder using RecycleBin="$HOME/Recycled".
I then want to move all the arguments files / directories to the RecycleBin directory.
This is what I've got so far:
for i in $*
do
if [ $* -e ]
then
mv $i /path/to/RecycleBin/$*.`date +"%Y%m%d.%H%M%S"`
else
echo "The file does not exist"
fi
done
where .date +"%Y%m%d.%H%M%S" is appending the version (date command) of each file
and for i in $* is checking for all the arguments in the command.
Output will look something like this ./script.sh file1 file2 file3
- where 3 files are being moved into RecycleBin
This is not working as my loop function is incorrect.
Would appreciate help
Thanks
Pete
[ $* -e ]
That test should be this:
[ -e "$i" ]
Also in the mv command you should be using $i rather than $*. Personally I'd just get rid of the if statement entirely. mv will print out an error message if a file doesn't exist, no need to do it yourself.
To handle file names with spaces in them it's best practice to use "$#" in place of $*, and to surround your variable names with with quotes, like so:
for file in "$#"; do
mv "$file" "/path/to/RecycleBin/$file.$(date +%Y%m%d.%H%M%S)"
done

Shell script to browse one or more directories passed as parameters

I made this script that should receive one or more parameter, and those parameter are all directories, and it has to browse those directories (one by one) and do some operations.
The operations work fine if the parameter is 1 (only one directory),
How should I modify my script to make it works if more than 1 parameter is passed
Example if I want it to do the same operations in 2 or 3 directories at the same time?
Thanks
#!/bin/sh
cd $1
for file in ./* # */
do
if [[ -d $file ]]
then
ext=dir
else
ext="${file##*.}"
fi
mv "${file}" "${file}.$ext"
done
First, if you are using bash use bash shebang (#! /bin/bash).
Then use
#! /bin/bash
for d in "$#"
do
echo "Do something with $d"
done
to iterate over the command line arguments (dirs in your case)
#!/bin/sh
for dir in "$#"; do
for file in "$dir"/*; do
echo "Doing something with '$file'"
done
done

Shell Script and spaces in path

I have larger shell script which handles different things.
It will get it's own location by the following...
BASEDIR=`dirname $0`/..
BASEDIR=`(cd "$BASEDIR"; pwd)`
then BASEDIR will be used create other variables like
REPO="$BASEDIR"/repo
But the problem is that this shell script does not work if the path contains spaces where it is currently executed.
So the question is: Does exist a good solution to solve that problem ?
Be sure to double-quote anything that may contain spaces:
BASEDIR="`dirname $0`"
BASEDIR="`(cd \"$BASEDIR\"; pwd)`"
The answer is "Quotes everywhere."
If the path you pass in has a space in it then dirname $0 will fail.
$ cat quote-test.sh
#!/bin/sh
test_dirname_noquote () {
printf 'no quotes: '
dirname $1
}
test_dirname_quote () {
printf 'quotes: '
dirname "$1"
}
test_dirname_noquote '/path/to/file with spaces/in.it'
test_dirname_quote '/path/to/file with spaces/in.it'
$ sh quote-test.sh
no quotes: usage: dirname string
quotes: /path/to/file with spaces
Also, try this fun example
#!/bin/sh
mkdir -p /tmp/foo/bar/baz
cd /tmp/foo
ln -s bar quux
cd quux
cat >>find-me.sh<<"."
#!/bin/sh
self_dir="$(dirname $0)"
base_dir="$( (cd "$self_dir/.." ; pwd -P) )"
repo="$base_dir/repo"
printf 'self: %s\n' "$self_dir"
printf 'base: %s\n' "$base_dir"
printf 'repo: %s\n' "$repo"
.
sh find-me.sh
rm -rf /tmp/foo
Result when you run it:
$ sh example.sh
self: .
base: /tmp/foo
repo: /tmp/foo/repo
Quote your full variable like this:
REPO="$BASEDIR/repo"
There is no reliable and/or portable way to do this correctly.
See How do I determine the location of my script? as to why
The best answer is the following, which is still OS dependent
BASEDIR=$(readlink -f $0)
Then you can do things like REPO="$BASEDIR"/repo , just be sure to quote your variables as you did.
Works perfectly fine for me. How are you using REPO? What specifically "doesn't work" for you?
I tested
#!/bin/sh
BASEDIR=`dirname $0`/..
BASEDIR=`(cd "$BASEDIR"; pwd)`
REPO="$BASEDIR"/repo
echo $REPO
in a ".../a b/c d" directory. It outputs ".../a b/repo", as expected.
Please give the specific error that you are receiving... A "doesn't work" bug report is the least useful bug report, and every programmer absolutely hates it.
Using spaces in directory names in unix is always an issue so if they can be avoided by using underscores, this prevents lots of strange scripting behaviour.
I'm unclear why you are setting BASEDIR to be the parent directory of the directory containing the current script (..) and then resetting it after changing into that directory
The path to the directory should still work if it has ..
e.g. /home/trevor/data/../repo
BASEDIR=`dirname $0`/..
I think if you echo out $REPO it should have the path correctly assigned because you used quotes when assigning it but if you then try to use $REPO somewhere else in the script, you will need to use double quotes around that too.
e.g.
#!/bin/ksh
BASEDIR=`dirname $0`/..
$REPO="$BASEDIR"/repo
if [ ! -d ["$REPO"]
then
echo "$REPO does not exist!"
fi
Use speech marks as below:
BASEDIR=`dirname "${0}"`/..

Get current directory or folder name (without the full path)

How could I retrieve the current working directory/folder name in a bash script, or even better, just a terminal command.
pwd gives the full path of the current working directory, e.g. /opt/local/bin but I only want bin.
No need for basename, and especially no need for a subshell running pwd (which adds an extra, and expensive, fork operation); the shell can do this internally using parameter expansion:
result=${PWD##*/} # to assign to a variable
result=${result:-/} # to correct for the case where PWD=/
printf '%s\n' "${PWD##*/}" # to print to stdout
# ...more robust than echo for unusual names
# (consider a directory named -e or -n)
printf '%q\n' "${PWD##*/}" # to print to stdout, quoted for use as shell input
# ...useful to make hidden characters readable.
Note that if you're applying this technique in other circumstances (not PWD, but some other variable holding a directory name), you might need to trim any trailing slashes. The below uses bash's extglob support to work even with multiple trailing slashes:
dirname=/path/to/somewhere//
shopt -s extglob # enable +(...) glob syntax
result=${dirname%%+(/)} # trim however many trailing slashes exist
result=${result##*/} # remove everything before the last / that still remains
result=${result:-/} # correct for dirname=/ case
printf '%s\n' "$result"
Alternatively, without extglob:
dirname="/path/to/somewhere//"
result="${dirname%"${dirname##*[!/]}"}" # extglob-free multi-trailing-/ trim
result="${result##*/}" # remove everything before the last /
result=${result:-/} # correct for dirname=/ case
Use the basename program. For your case:
% basename "$PWD"
bin
$ echo "${PWD##*/}"
​​​​​
Use:
basename "$PWD"
OR
IFS=/
var=($PWD)
echo ${var[-1]}
Turn the Internal Filename Separator (IFS) back to space.
IFS=
There is one space after the IFS.
You can use a combination of pwd and basename. E.g.
#!/bin/bash
CURRENT=`pwd`
BASENAME=`basename "$CURRENT"`
echo "$BASENAME"
exit;
How about grep:
pwd | grep -o '[^/]*$'
This thread is great! Here is one more flavor:
pwd | awk -F / '{print $NF}'
basename $(pwd)
or
echo "$(basename $(pwd))"
I like the selected answer (Charles Duffy), but be careful if you are in a symlinked dir and you want the name of the target dir. Unfortunately I don't think it can be done in a single parameter expansion expression, perhaps I'm mistaken. This should work:
target_PWD=$(readlink -f .)
echo ${target_PWD##*/}
To see this, an experiment:
cd foo
ln -s . bar
echo ${PWD##*/}
reports "bar"
DIRNAME
To show the leading directories of a path (without incurring a fork-exec of /usr/bin/dirname):
echo ${target_PWD%/*}
This will e.g. transform foo/bar/baz -> foo/bar
echo "$PWD" | sed 's!.*/!!'
If you are using Bourne shell or ${PWD##*/} is not available.
Surprisingly, no one mentioned this alternative that uses only built-in bash commands:
i="$IFS";IFS='/';set -f;p=($PWD);set +f;IFS="$i";echo "${p[-1]}"
As an added bonus you can easily obtain the name of the parent directory with:
[ "${#p[#]}" -gt 1 ] && echo "${p[-2]}"
These will work on Bash 4.3-alpha or newer.
There are a lots way of doing that I particularly liked Charles way because it avoid a new process, but before know this I solved it with awk
pwd | awk -F/ '{print $NF}'
For the find jockeys out there like me:
find $PWD -maxdepth 0 -printf "%f\n"
i usually use this in sh scripts
SCRIPTSRC=`readlink -f "$0" || echo "$0"`
RUN_PATH=`dirname "${SCRIPTSRC}" || echo .`
echo "Running from ${RUN_PATH}"
...
cd ${RUN_PATH}/subfolder
you can use this to automate things ...
Just use:
pwd | xargs basename
or
basename "`pwd`"
Below grep with regex is also working,
>pwd | grep -o "\w*-*$"
If you want to see only the current directory in the bash prompt region, you can edit .bashrc file in ~. Change \w to \W in the line:
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u#\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
Run source ~/.bashrc and it will only display the directory name in the prompt region.
Ref: https://superuser.com/questions/60555/show-only-current-directory-name-not-full-path-on-bash-prompt
I strongly prefer using gbasename, which is part of GNU coreutils.
Just run the following command line:
basename $(pwd)
If you want to copy that name:
basename $(pwd) | xclip -selection clipboard
An alternative to basname examples
pwd | grep -o "[^/]*$"
OR
pwd | ack -o "[^/]+$"
My shell did not come with the basename package and I tend to avoid downloading packages if there are ways around it.
You can use the basename utility which deletes any prefix ending in / and the suffix (if present in string) from string, and prints the
result on the standard output.
$basename <path-of-directory>
Just remove any character until a / (or \, if you're on Windows). As the match is gonna be made greedy it will remove everything until the last /:
pwd | sed 's/.*\///g'
In your case the result is as expected:
λ a='/opt/local/bin'
λ echo $a | sed 's/.*\///g'
bin
Here's a simple alias for it:
alias name='basename $( pwd )'
After putting that in your ~/.zshrc or ~/.bashrc file and sourcing it (ex: source ~/.zshrc), then you can simply run name to print out the current directories name.
The following commands will result in printing your current working directory in a bash script.
pushd .
CURRENT_DIR="`cd $1; pwd`"
popd
echo $CURRENT_DIR

Resources