If I have a file name with spaces and a random set of numbers that looks like this:
file name1234.csv
I want to rename it to this (assuming date is previously specified):
file_name_${date}.csv
I am able to do it like this:
mv 'file name'*'.csv file_name_${date}.csv
However, in a situation that 'file name*.csv' can actually match multiple files, I want to specify that it's 'file name[random numbers].csv'
I've searched around and can't find any relevant answers.
You need what is called a "pathname expansion", to match one or more digits:
+([0-9])
A functional script could be like this one:
date=$(date +'%Y-%m-%d')
shopt -s extglob nullglob
for f in 'file name'+([[:digit:]]).csv; do
file="${f%%[0-9]*}"
echo mv "$f" "${file// /_}_${date}.csv"
done
Warning: all files found will be renamed to just one name, make sure that that is what you want before removing the echo.
To activate the extended version of "Pathname Expansion" we use shopt -s extglob.
To avoid the case where no file is matched, we also need the nullglob set.
We can set the positional arguments to the result of the above expansion.
Then we loop over all files found to change each of their names.
The ${f%%[0-9]*} removes all from the digits to the end.
The ${file// /_} replaces spaces with underscores.
The mv is not actually done with the script presented because of the echo.
If after running a test, you want the change(s) performed, remove the echo.
Use Extended Globs and Parameter Expansion
You can do what you want with Bash extended globs and a few parameter expansions, without resorting to external or non-standard utilities.
date="2016-11-21"
shopt -s extglob
for file in 'file name'+([[:digit:]]).csv; do
newfile="${file%%[0-9]*}"
newfile="${newfile// /_}"
mv "$file" "${newfile}_${date}.csv"
done
Related
When I pass a flag containing [...] to my bash script, getops gives me an empty string when I try to grab the value with $OPTARG.
shopt -s nullglob
while getopts ":f:" opt; do
case $opt in
f)
str=$OPTARG
;;
esac
done
echo ${str}
Running the script:
$ script.sh -f [0.0.0.0]
<blank line>
How can I get the original value back inside the script?
Short summary: Double-quote your variable references. And use shellcheck.net.
Long explanation: When you use a variable without double-quotes around it (e.g. echo ${str}), the shell tries to split its value into words, and expand anything that looks like a wildcard expression into a list of matching files. In the case of [0.0.0.0], the brackets make it a wildcard expression that'll match either the character "0" or "." (equivalent to [0.]). If you had a file named "0", it would expand to that string. With no matching file(s), it's normally left unexpanded, but with the nullglob set it expands to ... null.
Turning off nullglob solves the problem if there are no matching files, but isn't really the right way do it. I remember (but can't find right now) a question we had about a script that failed on one particular computer, and it turned out the reason was that one computer happened to have a file that matched a bracket expression in an unquoted variable's value.
The right solution is to put double-quotes around the variable reference. This tells the shell to skip word splitting and wildcard expansion. Here's an interactive example:
$ str='[0.0.0.0]' # Quotes aren't actually needed here, but they don't hurt
$ echo $str # This works without nullglob or a matching file
[0.0.0.0]
$ shopt -s nullglob
$ echo $str # This fails because of nullglob
$ shopt -u nullglob
$ touch 0
$ echo $str # This fails because of a matching file
0
$ echo "$str" # This just works, no matter whether file(s) match and/or nullglob is set
[0.0.0.0]
So in your script, simply change the last line to:
echo "${str}"
Note that double-quotes are not required in either case $opt in or str=$OPTARG because variables in those specific contexts aren't subject to word splitting or wildcard expansion. But IMO keeping track of which contexts it's safe to leave the double-quotes off is more hassle than it's worth, and you should just double-quote 'em all.
BTW, shellcheck.net is good at spotting common mistakes like this; I recommend feeding your scripts through it, since this is probably not the only place you have this problem.
Assuming that shopt -s nullglob is needed in the bigger script.
You can temporary disable shopt -s nullglob using shopt -u nullglob
shopt -s nullglob
shopt -u nullglob
while getopts ":f:" opt; do
case $opt in
f)
str=$OPTARG
;;
esac
done
echo ${str}
shopt -s nullglob
I am making a shell script that allows you to select a file from a directory using YAD. I am doing this:
list='';
exc='!'
for f in "$SHOTS_NOT_CONVERTED_DIR"/*;do
f=`basename $f`
list="${list}${exc}${f}"
done
The problem is that if there are no files in that directory, I end up with a selection with *.
What's the easiest, most elegant way to make this work in Bash?
The goal is to have an empty list if there are no files there.
* expansion is called a glob expressions. The bash manual calls it filename expansion.
You need to set the nullglob option. Doing so gives you an empty result if the glob expression does not find files:
shopt -s nullglob
list='';
exc='!'
for f in "$SHOTS_NOT_CONVERTED_DIR"/*;do
# Btw, use $() instead of ``
f=$(basename "$f")
list="${list}${exc}${f}"
done
This must be simple but I can't figure it out.
for filename in *[^\.foo].jpg
do
echo $filename
done
Instead of the matched filenames, echo shows the pattern:
+ echo '*[^.foo].jpg'
*[^.foo].jpg
Intention is to find all files ending in .jpg but not .foo.jpg.
EDIT: Tried this as per (misunderstood) advice:
for filename in *[!".foo"].jpg
Still not there!
You actually can do this, with an extglob. To demonstrate, copy-and-paste the following code:
shopt -s extglob
cd "$(mktemp -d "${TMPDIR:-/tmp}/test.XXXXXX")" || exit
touch hello.txt hello.foo hello.foo.jpg hello.jpg
printf '%q\n' !(*.foo).jpg
Output should be:
hello.jpg
In bash, if a glob pattern has no matches bash will return the pattern itself. You can change this behavior with the nullglob shell option, which can be turned on like this:
shopt -s nullglob
This is described in the section titled Filename Expansion in the bash man page.
As to why it doesn't match, it's simply that you don't have any files that match. This is possibly due to your use of ^ which isn't normally a valid glob meta character. As far as glob is concerned, ^ simply matches a literal ^. Also, [...] probably doesn't do what you think it does either.
For an explanation of valid glob meta-characters, see the Pattern Matching section of the bash man page.
You can't write a glob pattern that returns "all files ending in .jpg but not .foo.jpg.". The easiest thing to do is glob over all jpg files (*.jpg) and then filter out the ones that end in foo.jpg inside the code block.
for filename in *.jpg
do
[[ $filename = *.foo.jpg ]] && continue
echo $filename
done
What's the proper way to do a for loop over a file mask?
E.g. if the mask doesn't expand to any file, then the loop should not run; else, it should run over all the files that it expands to.
The problem with naive approach is that if the * doesn't expand to anything, then the whole loop will run once as if the * part is an actual part of a filename (of a file that doesn't actually exist).
One way to do this is:
for f in abc*; do if [[ -f "$f" ]]; then
# Do something with "$f"
fi; done
That will also filter directories out of the list, which might or might not be what you want.
If you want to keep directories, use -e instead of -f.
Another way to do it is to set the shell option nullglob (may not be available on all bash versions). With nullglob set, patterns which don't match any filename will result in nothing instead of being unaltered.
shopt -s nullglob
for f in abc*; do
# do something with $f
done
shopt -u nullglob
(That leaves nullglob set for the duration of the for loop. You can unset it inside the loop instead of waiting for the end, at the smallish cost of executing the shopt -u on every loop.)
Use the nullglob shell option to make a wildcard that doesn't match anything expand into nothing instead of returning the wildcard itself:
shopt -s nullglob
for file in abc*
do
...
done
This question already has answers here:
How to skip the for loop when there are no matching files?
(2 answers)
Closed 3 years ago.
I have this bash "for in" loop that looks for pdf files in a directory and prompt them (simplified for the example)
#!/bin/bash
for pic in "$INPUT"/*.pdf
do
echo "found: ${pic}"
done
This script works well when there are pdf files in $INPUT directory, however, when there are no pdf files in the directory, I get :
found: /home/.../input-folder/*.pdf
Is it the expected behavior ?
How can I deal with it with a for in loop ?
Do I need to use ls or find ?
I tried with and without quotes around "$INPUT". There are no spaces in files names and directory names.
This is the expected behavior. According to the bash man page, in the Pathname Expansion section:
After word splitting, unless the -f option has been set, bash scans each word for the characters *, ?, and [. If one of these characters appears, then the word is regarded as a pattern, and replaced with an alphabetically sorted list of file names matching the pattern. If no matching file names are found, and the shell option nullglob is not enabled, the word is left unchanged.
As a result, if no matches for "$INPUT"/*.pdf are found, the loop will be executed on the pattern itself. But in the next sentence of the man page:
If the nullglob option is set, and no matches are found, the word is removed.
That's what you want! So just do:
#!/bin/bash
shopt -s nullglob
for pic in "$INPUT"/*.pdf
do
echo "found: ${pic}"
done
(But be aware that this may change the behavior of other things in unexpected ways. For example, try running shopt -s nullglob; ls *.unmatchedextension and see what happens.)
I would just add a file exists test like this:
#!/bin/bash
if test -e `echo "$INPUT"/*.pdf | cut -d' ' -f1`
then
for pic in "$INPUT"/*.pdf
do
echo "found: ${pic}"
done
fi