I'm quite new to bash scripting and I've ran out of ideas in my homework script.
The script takes 4 arguments - pathToDirectory, [-c/-m/-r] == copy/move/remove, ext1 and ext2 (example of running a script: script.sh /home/user/somefolder -c a.txt b.sh ).
The script should find all files in /home/user/someFolder (and its all subfolders) that contain 'a.txt' in their names and (in -c and -m case) rename that 'a.txt' part to 'b.sh' and depending on -c/-m argument either create a new file or just rename an existing file (in -r case it just removes the file) and then write in stdout something like 'old name => new name'.
example output of a script mentioned above:
/home/user/someFolder/bbb.txt => /home/user/someFolder/bba.txt
Well, that was not a problem to implement, everything worked until I posted my code to our upload system (evaluates our script).
The very first Upload System's try to run my script looked like "script.sh /something/graph 1 -c .jpg .jpeg".
The problem now is, that the whole '/something/graph 1' is a path and that whitespace before '1' ruins it all.
expected output: ./projekty/graph 1.jpg => ./projekty/graph 1.jpeg
my script output: ./projekty/graph => ./projekty/graph.jpeg
1.jpg => 1.jpeg
What I have so far:
if [ "$2" = "-r" ]; then
for file in $(find $1 -name "*$3"); do
echo $file
rm -f $file
done
elif [ "$2" = "-c" ]; then
for file in $(find "$1" -name "*$3") ; do
cp "$file" "${file//$3/$4}"
echo $file "=>" ${file%$3}$4
done
elif [ "$2" = "-m" ]; then
for file in $(find $1 -name "*$3"); do
mv "$file" "${file//$3/$4}"
echo $file "=>" ${file%$3}$4
done
else
echo Unknown parameter >&2
fi
My tried¬working&probablystupid idea: as the -r/-c/-m parameter should be at $2, I was able to detect that $2 is something else (assumpting something that still belongs to the path) and append that $2 thing to $1, so then I had a variable DIR which was the whole path. Using shift I moved all parameters to the left (because of the whitespace, the -r/-m/-c parameter was not on $2 but on $3, so I made it $2 again) and then the code looked like: (just the -c part)
DIR=$1
if [ "$2" != "-r" ] && [ "$2" != "-c" ] && [ "$2" != "-m" ]; then
DIR+=" $2"
shift
fi
if [ "$2" = "-c" ]; then
for file in $(find "$DIR" -name "*$3") ; do
cp "$file" "${file//$3/$4}"
echo $file "=>" ${file%$3}$4
done
fi
when i echoed "$DIR", it showed the whole path (correctly), but it still didn't work..
Is there any other/better/any way how to fix this please ? :/
Thanks in advance !
As the target string needs to be replaced only at the very end of a filename, "${file//$3/$4}" is a bad idea.
Example: ./projekty/graph.jpg.jpg.jpg.graph.jpg
Passing a string prone to unquoted expansion to a loop is a no better idea either.
The fact is that find works as expected and its output looks like:
./projekty/graph 1.jpg
But inside a loop it is expanded incorrectly:
./projekty/graph
1.jpg
To avoid this, you can save the output of find to a variable and then tokenize it until no text is left:
list="$(find $1 -name "*$3")"
while [ -n "$list" ]; do
file="${list%%$'\n'*}"
list="${list#$file}"
list="${list#$'\n'}"
# your commands here
# ...
done
Related
I was doing this little script in which the first argument must be a path to an existing directory and the second any other thing.
Each object in the path indicated in the first argument must be renamed so that the new
name is the original that was added as a prefix to the character string passed as the second argument. Example, for the string "hello", the object OBJECT1 is renamed hello.OBJECT1 and so on
Additionally, if an object with the new name is already present, a message is shown by a standard error output and the operation is not carried out continuing with the next object.
I have the following done:
#! /bin/bash
if [ "$#" != 2 ]; then
exit 1
else
echo "$2"
if [ -d "$1" ]; then
echo "directory"
for i in $(ls "$1")
do
for j in $(ls "$1")
do
echo "$i"
if [ "$j" = "$2"."$i" ]; then
exit 1
else
mv -v "$i" "$2"."$i"
echo "$2"."$i"
fi
done
done
else
echo "no"
fi
fi
I am having problems if I run the script from another file other than the one I want to do it, for example if I am in /home/pp and I want the changes to be made in /home/pp/rr, since that is the only way It does in the current.
I tried to change the ls to catch the whole route with
ls -R | sed "s;^;pwd;" but the route catches me badly.
Using find you can't because it puts me in front of the path and doesn't leave the file
Then another question, to verify that that object that is going to create new is not inside, when doing it with two for I get bash errors for all files and not just for coincidences
I'm starting with this scripting, so it has to be a very simple solution thing
An obvious answer to your question would be to put a cd "$2 in the script to make it work. However, there are some opportunities in this script for improvement.
#! /bin/bash
if [ "$#" != 2 ]; then
You might put an error message here, for example, echo "Usage: $0 dir prefix" or even a more elaborate help text.
exit 1
else
echo $2
Please quote, as in echo "$2".
if [ -d $1 ]; then
Here, the quotes are important. Suppose that your directory name has a space in it; then this if would fail with bash: [: a: binary operator expected. So, put quotes around the $1: if [ -d "$1" ]; then
echo "directory"
This is where you could insert the cd "$1".
for i in $(ls $1)
do
It is almost always a bad idea to parse the output of ls. Once again, this for-loop will fail if a file name has a space in it. A possible improvement would be for i in "$1"/* ; do.
for j in $(ls $1)
do
echo $i
if [ $j = $2.$i ]; then
exit 1
else
The logic of this section seems to be: if a file with the prefix exists, then exit instead of overwriting. It is always a good idea to tell why the script fails; an echo before the exit 1 will be helpful.
The question is why you use the second loop? a simple if [ -f "$2.$i ] ; then would do the same, but without the second loop. And it will therefore be faster.
mv -v $i $2.$i
echo $2.$i
Once again: use quotes!
fi
done
done
else
echo "no"
fi
fi
So, with all the remarks, you should be able to improve your script. As tripleee said in his comment, running shellcheck would have provided you with most of the comment above. But he also mentioned basename, which would be useful here.
With all that, this is how I would do it. Some changes you will probably only appreciate in a few months time when you need some changes to the script and try to remember what the logic was that you had in the past.
#!/bin/bash
if [ "$#" != 2 ]; then
echo "Usage: $0 directory prefix" >&2
echo "Put a prefix to all the files in a directory." >&2
exit 1
else
directory="$1"
prefix="$2"
if [ -d "$directory" ]; then
for f in "$directory"/* ; do
base=$(basename "$f")
if [ -f "Sdirectory/$prefix.$base" ] ; then
echo "This would overwrite $prefix.$base; exiting" >&2
exit 1
else
mv -v "$directory/$base" "$directory/$prefix.$base"
fi
done
else
echo "$directory is not a directory" >&2
fi
fi
So I have this bash script which will rename all the files from the current directory. I need help modifying it so I can instead specify only certain files which will be renamed, but also still have the ability to pass it a directory instead. I'm not super familiar with bash so it's fairly confusing to me.
#!/bin/bash
#
# Filename: rename.sh
# Description: Renames files and folders to lowercase recursively
# from the current directory
# Variables: Source = x
# Destination = y
#
# Rename all directories. This will need to be done first.
#
# Process each directory’s contents before the directory itself
for x in `find * -depth -type d`;
do
# Translate Caps to Small letters
y=$(echo $x | tr '[A-Z]' '[a-z]');
# check if directory exits
if [ ! -d $y ]; then
mkdir -p $y;
fi
# check if the source and destination is the same
if [ "$x" != "$y" ]; then
# check if there are files in the directory
# before moving it
if [ $(ls "$x") ]; then
mv $x/* $y;
fi
rmdir $x;
fi
done
#
# Rename all files
#
for x in `find * -type f`;
do
# Translate Caps to Small letters
y=$(echo $x | tr '[A-Z]' '[a-z]');
if [ "$x" != "$y" ]; then
mv $x $y;
fi
done
exit 0
Your script has a large number of beginner errors, but the actual question in the title has some merit.
For a task like this, I would go for a recursive solution.
tolower () {
local f g
for f; do
# If this is a directory, process its contents first
if [ -d "$f" ]; then
# Recurse -- run the same function over directory entries
tolower "$f"/*
fi
# Convert file name to lower case (Bash 4+)
g=${f,,}
# If lowercased version differs from original, move it
if [ "${f##*/}" != "${g##*/}" ]; then
mv "$f" "$g"
fi
done
}
Notice how variables which contain file names always need to be quoted (otherwise, your script will fail on file names which contain characters which are shell metacharacters) and how Bash has built-in functionality for lowercasing a variable's value (in recent versions).
Also, tangentially, don't use ls in scripts and try http://shellcheck.net/ before asking for human debugging help.
I'm writing very simple bash script to change encoding of
.html files and want to handle directories recursively.
The function working properly for first level directory only.
Can you tell me where I'm wrong? Here is me code.
#!/bin/bash
handleFiles () {
local REGEXP='.+\.html$'
echo $1
for f in $1/*
do
if [[ -d $f ]]
then
handleFiles "$f"
elif [[ $f =~ $REGEXP ]]
then
echo "Converting $f"
enconv -L bg -x UTF-8 "$f"
fi
done
}
# The script show all .html files in test
# enter into subdirectory but not working ..
handleFiles "test"
Here is structure of test directory:
test$ tree
.
├── test.html
└── Untitled Folder
└── test1.html
1 directory, 2 files
When I run the script I get following output:
./converter.sh
test
Converting test/test.html
test/Untitled Folder
To be the whole story I post my final solution. I hope this will be useful to someone with a similar problem.
#!/bin/bash
########################################################
# This bash script assume directory as a argument
# and convert all .html,js and xml files from
# windows-1251 encoding into utf-8 encoding.
# #author Georgi Naumov
# #email gonaumov#gmail.com for contacts and
# suggestions.
########################################################
if [[ $# -ne 1 ]] ; then
echo "Usage $0 <<directory to change encoding reqursively>>"
exit 1
fi
handleFiles () {
local REGEXP='.+\.(html|js|xml)$'
for f in "$1"/*
do
if [[ -d "$f" ]]
then
handleFiles "$f"
elif [[ "$f" =~ $REGEXP ]]
then
echo "Converting $f"
enconv -L bg -x UTF-8 "$f"
fi
done
}
handleFiles "$1"
The whole approach is rather more complicated than necessary. I recommend using find:
find test -name '*.html' -exec enconv -L bg -x UTF-8 '{}' \;
If you want to do it manually, you have to put $f in double quotes everywhere (i.e., "$f"), or it will break if a directory contains spaces, as you have noticed, because the shell will expand it into two (or more, as the case may be) separate tokens.
Quote also $1 to handle whitespaces (e.g. Untitled Folder):
for f in "$1"/*
I want to see if files exist in a certain folder and if they exist run certain event else skip this. Bash see files in my empty folder.
I tried a few different ways and all show that a file exits.
$ [ -f ] && echo "Found" || echo "Not found"
Found
$ [ -r ] && echo "Found" || echo "Not found"
Found
$ [ -e ] && echo "Found" || echo "Not found"
Found
Whereas it does not:
$ ls -lrt
total 0
What am I missing here?
My actual code is:
get_last_parsed_file_time ()
{
if [ -s "$DATA_DIR$EPG_XML_FILES" ]
then
NEWEST_FILE=$( ls -tr | tail -1 )
LAST_PARSED_TIME=$( stat -c %Y $NEWEST_FILE )
else
NO_FILE=1
fi
}
[ -f filename ] checks if filename is a file that exists.
[ str ] checks whether str is non-empty.
In your case, str is -f, but it still just checks whether string "-f" is nonempty, which it obviously isn't (it's two characters, a dash and an "f").
This is especially puzzling in the case of [ -f $filename ]. When $filename is empty and unquoted, the command will become [ -f ] and will be true. Always quote your variables.
that other guy's answer explains why [ -f ] doesn't work.
More generally, it is important to note that you cannot pass globs (filename patterns such as * to a bash file-test operator such as -f. The file-test operators expect a single, literal filename or file path (either as a string literal or as a string stored in a variable).
[[ -f '/path/to/file' ]] # correct
[[ -f /path/to/* ]] # WRONG
Here's a snippet that tests whether a folder is empty, i.e. whether it contains ANY items - whether files, hidden files, or subfolders:
folder='/path/to/folder'
[[ -n $(ls -A "$folder" | head -n 1) ]] && echo "NON-empty" || echo "EMPTY"
To only consider non-hidden items, remove -A from the ls command.
To quietly ignore the case where the input folder doesn't exist or is not accessible, append 2> /dev/null to the ls command.
I want to check whether a directory has files or not in bash.
My code is here.
for d in {,/usr/local}/etc/bash_completion.d ~/.bash/completion.d
do
[ -d "$d" ] && [ -n "${d}/*" ] &&
for f in $d/*; do
[ -f "$f" ] && echo "$f" && . "$f"
done
done
The problem is that "~/.bash/completion.d" has no file.
So, $d/* is regarded as simple string "~/.bash/completion.d/*", not empty string which is result of filename expansion.
As a result of that code, bash tries to run
. "~/.bash/completion.d/*"
and of course, it generates error message.
Can anybody help me?
If you set the nullglob bash option, through
shopt -s nullglob
then globbing will drop patterns that don't match any file.
# NOTE: using only bash builtins
# Assuming $d contains directory path
shopt -s nullglob
# Assign matching files to array
files=( "$d"/* )
if [ ${#files[#]} -eq 0 ]; then
echo 'No files found.'
else
# Whatever
fi
Assignment to an array has other benefits, including desirable (correct!) handling of filenames/paths containing white-space, and simple iteration without using a sub-shell, as the following code does:
find "$d" -type f |
while read; do
# Process $REPLY
done
Instead, you can use:
for file in "${files[#]}"; do
# Process $file
done
with the benefit that the loop is run by the main shell, meaning that side-effects (such as variable assignment, say) made within the loop are visible for the remainder of script. Of course, it's also way faster, if performance is an issue.
Finally, an array can also be inserted in command line arguments (without splitting arguments containing white-space):
$ md5sum fileA "${files[#]}" fileZ
You should always attempt to correctly handle files/paths containing white-space, because one day, they will happen!
You could use find directly in the following way:
for f in $(find {,/usr/local}/etc/bash_completion.d ~/.bash/completion.d -maxdepth 1 -type f);
do echo $f; . $f;
done
But find will print a warning if some of the directory isn't found, you can either put a 2> /dev/null or put the find call after testing if the directories exist (like in your code).
find() {
for files in "$1"/*;do
if [ -d "$files" ];then
numfile=$(ls $files|wc -l)
if [ "$numfile" -eq 0 ];then
echo "dir: $files has no files"
continue
fi
recurse "$files"
elif [ -f "$files" ];then
echo "file: $files";
:
fi
done
}
find /path
Another approach
# prelim stuff to set up d
files=`/bin/ls $d`
if [ ${#files} -eq 0 ]
then
echo "No files were found"
else
# do processing
fi