Matching *.jpg but not *.foo.jpg in bash - bash

This must be simple but I can't figure it out.
for filename in *[^\.foo].jpg
do
echo $filename
done
Instead of the matched filenames, echo shows the pattern:
+ echo '*[^.foo].jpg'
*[^.foo].jpg
Intention is to find all files ending in .jpg but not .foo.jpg.
EDIT: Tried this as per (misunderstood) advice:
for filename in *[!".foo"].jpg
Still not there!

You actually can do this, with an extglob. To demonstrate, copy-and-paste the following code:
shopt -s extglob
cd "$(mktemp -d "${TMPDIR:-/tmp}/test.XXXXXX")" || exit
touch hello.txt hello.foo hello.foo.jpg hello.jpg
printf '%q\n' !(*.foo).jpg
Output should be:
hello.jpg

In bash, if a glob pattern has no matches bash will return the pattern itself. You can change this behavior with the nullglob shell option, which can be turned on like this:
shopt -s nullglob
This is described in the section titled Filename Expansion in the bash man page.
As to why it doesn't match, it's simply that you don't have any files that match. This is possibly due to your use of ^ which isn't normally a valid glob meta character. As far as glob is concerned, ^ simply matches a literal ^. Also, [...] probably doesn't do what you think it does either.
For an explanation of valid glob meta-characters, see the Pattern Matching section of the bash man page.
You can't write a glob pattern that returns "all files ending in .jpg but not .foo.jpg.". The easiest thing to do is glob over all jpg files (*.jpg) and then filter out the ones that end in foo.jpg inside the code block.
for filename in *.jpg
do
[[ $filename = *.foo.jpg ]] && continue
echo $filename
done

Related

How to convert files in Unix using iconv?

I'm new to Bash scripting. I have a requirement to convert multiple input files in UTF-8 encoding to ISO 8859-1.
I am using the below command, which is working fine for the conversion part:
cd ${DIR_INPUT}/
for f in *.txt; do iconv -f UTF-8 -t ISO-8859-1 $f > ${DIR_LIST}/$f; done
However, when I don't have any text files in my input directory ($DIR_INPUT), it still creates an empty .txt file in my output directory ($DIR_LIST).
How can I prevent this from happening?
The empty file *.txt is being created in your output directory because by default, bash expands an unmatched expansions to the literal string that you supplied. You can change this behaviour in a number of ways, but what you're probably looking for is shopt -s nullglob. Observe:
$ for i in a*; do echo "$i"; done
a*
$ shopt -s nullglob
$ for i in a*; do echo "$i"; done
$
You can find documentation about this in the bash man page under Pathname Expansion. Or here or here.
In your case, I'd probably rewrite this in this way:
shopt -s nullglob
for f in "$DIR_INPUT"/*.txt; do
iconv -f UTF-8 -t ISO-8859-1 "$f" > "${DIR_LIST}/${f##*/}"
done
This avoids the need for the initial cd, and uses parameter expansion to strip off the path portion of $f for the output redirection. The nullglob will obviously eliminate the work being done on a nonexistent file.
As #ghoti pointed out, in the absence of files matching the wildcard expression a* the expression itself becomes the result of pathname expansion. By default (when nullglob option is unset), a* is expanded to, literally, a*.
You can set nullglob option, of course. But then you should be aware of the fact that all subsequent pathname expansions will be affected, unless you unset the option after the loop.
I would rather use find command which has a clear interface (and, in my opinion, is less likely to perform implicit conversions as opposed to the Bash globbing). E.g.:
cmd='iconv --verbose -f UTF-8 -t ISO-8859-1 "$0" > "$1"/$(basename "$0")'
find "${DIR_INPUT}/" \
-mindepth 1 \
-maxdepth 1 \
-type f \
-name '*.txt' \
-exec sh -c "$cmd" {} "${DIR_LIST}" \;
In the example above, $0 and $1 are positional arguments for the file path and ${DIR_LIST} respectively. The command is invoked via standard shell (sh) because of the need to refer to the file path {} twice. Although most modern implementations of find may handle multiple occurrences of {} correctly, the POSIX specification states:
If more than one argument containing the two characters "{}" is present, the behavior is unspecified.
As in the for loop, the -name pattern *.txt is evaluated as true if the basename of the current pathname matches the operand (*.txt) using the pattern matching notation. But, unlike the for loop, filename expansion do not apply as this is a matching operation, not an expansion.

Bash Script to remove leading and trailing spaces for all files in a directory

I would like to just add it to Automator and let the user choose the directory in which it runs. One Drive will not upload files with space. I managed to remove all spaces but not remove all spaces from the beginning and the end.
My code:
for f in "$1"/*; do
dir=$(dirname "$f")
file=$(basename "$f")
mv "$f" "${dir}/${file//[^0-9A-Za-z.]}"
done
#!/usr/bin/env bash
shopt -s extglob # Enable extended globbing syntax
for path in "$1"/*; do
file=${path##*/} # Trim directory name
file=${file##+([[:space:]])} # Trim leading spaces
file=${file%%+([[:space:]])} # Trim trailing spaces
if [[ $file != "${path##*/}" ]]; then # Skip files that aren't changed
mv -- "$path" "$1/${file}"
fi
done
Notes:
A shell needs to be started with bash, not sh, to ensure that extensions (such as extglobbing and [[ ]]) are available.
There's no need to call dirname, since we always know the directory name: It's in $1.
extglob syntax extends regular glob expressions to have power comparable to regexes. +([[:space:]]) is extglob for "one or more spaces", whereas the ${var%%pattern} and ${var##pattern} remove as many characters matching pattern as possible from the back or front, respectively, of a variable's value.
There's no point to running a mv when the filename didn't need to change, so we can optimize a bit by checking first.

use bash to rename a file with spaces and regex

If I have a file name with spaces and a random set of numbers that looks like this:
file name1234.csv
I want to rename it to this (assuming date is previously specified):
file_name_${date}.csv
I am able to do it like this:
mv 'file name'*'.csv file_name_${date}.csv
However, in a situation that 'file name*.csv' can actually match multiple files, I want to specify that it's 'file name[random numbers].csv'
I've searched around and can't find any relevant answers.
You need what is called a "pathname expansion", to match one or more digits:
+([0-9])
A functional script could be like this one:
date=$(date +'%Y-%m-%d')
shopt -s extglob nullglob
for f in 'file name'+([[:digit:]]).csv; do
file="${f%%[0-9]*}"
echo mv "$f" "${file// /_}_${date}.csv"
done
Warning: all files found will be renamed to just one name, make sure that that is what you want before removing the echo.
To activate the extended version of "Pathname Expansion" we use shopt -s extglob.
To avoid the case where no file is matched, we also need the nullglob set.
We can set the positional arguments to the result of the above expansion.
Then we loop over all files found to change each of their names.
The ${f%%[0-9]*} removes all from the digits to the end.
The ${file// /_} replaces spaces with underscores.
The mv is not actually done with the script presented because of the echo.
If after running a test, you want the change(s) performed, remove the echo.
Use Extended Globs and Parameter Expansion
You can do what you want with Bash extended globs and a few parameter expansions, without resorting to external or non-standard utilities.
date="2016-11-21"
shopt -s extglob
for file in 'file name'+([[:digit:]]).csv; do
newfile="${file%%[0-9]*}"
newfile="${newfile// /_}"
mv "$file" "${newfile}_${date}.csv"
done

How to deal with `*` expansion when there are no files

I am making a shell script that allows you to select a file from a directory using YAD. I am doing this:
list='';
exc='!'
for f in "$SHOTS_NOT_CONVERTED_DIR"/*;do
f=`basename $f`
list="${list}${exc}${f}"
done
The problem is that if there are no files in that directory, I end up with a selection with *.
What's the easiest, most elegant way to make this work in Bash?
The goal is to have an empty list if there are no files there.
* expansion is called a glob expressions. The bash manual calls it filename expansion.
You need to set the nullglob option. Doing so gives you an empty result if the glob expression does not find files:
shopt -s nullglob
list='';
exc='!'
for f in "$SHOTS_NOT_CONVERTED_DIR"/*;do
# Btw, use $() instead of ``
f=$(basename "$f")
list="${list}${exc}${f}"
done

Replace underscores to whitespaces using bash script

How can I replace all underscore chars with a whitespace in multiple file names using Bash Script? Using this code we can replace underscore with dash. But how it works with whitespace?
for i in *.mp3;
do x=$(echo $i | grep '_' | sed 's/_/\-/g');
if [ -n "$x" ];
then mv $i $x;
fi;
done;
Thank you!
This should do:
for i in *.mp3; do
[[ "$i" = *_* ]] && mv -nv -- "$i" "${i//_/ }"
done
The test [[ "$i" = *_* ]] tests if file name contains any underscore and if it does, will mv the file, where "${i//_/ }" expands to i where all the underscores have been replaced with a space (see shell parameter expansions).
The option -n to mv means no clobber: will not overwrite any existent file (quite safe). Optional.
The option -v to mv is for verbose: will say what it's doing (if you want to see what's happening). Very optional.
The -- is here to tell mv that the arguments will start right here. This is always good practice, as if a file name starts with a -, mv will try to interpret it as an option, and your script will fail. Very good practice.
Another comment: When using globs (i.e., for i in *.mp3), it's always very good to either set shopt -s nullglob or shopt -s failglob. The former will make *.mp3 expand to nothing if no files match the pattern (so the loop will not be executed), the latter will explicitly raise an error. Without these options, if no files matching *.mp3 are present, the code inside loop will be executed with i having the verbatim value *.mp3 which can cause problems. (well, there won't be any problems here because of the guard [[ "$i" = *_* ]], but it's a good habit to always use either option).
Hope this helps!
The reason your script is failing with spaces is that the filename gets treated as multiple arguments when passed to mv. You'll need to quote the filenames so that each filename is treated as a single agrument. Update the relevant line in your script with:
mv "$i" "$x"
# where $i is your original filename, and $x is the new name
As an aside, if you have the perl version of the rename command installed, you skip the script and achieve the same thing using:
rename 's/_/ /' *.mp3
Or if you have the more classic rename command:
rename "_" " " *.mp3
Using tr
tr '_' ' ' <file1 >file2

Resources