I have a directory filled with a lot of files, some of which have brackets in them; e.g. a(file).ext,an(other)file.ext, etc. My goal is to rename them into something like this: a_file_.ext or an?other?file.ext. (doesn't matter what character).
the reason for this is because certain console applications can't deal with these brackets and think it's some kind of command.
things I already tried:
$ rename ( ? *(*
$ for f in *(*; do mv $f ${f//(/?}; done
$ for f in "*(*"; do mv $f ${f//\"(\"/\"?\"}; done
and the like.
It could be that I'm not understanding these rename functions. (I do know that these only work for "(" and that I have to do them again for ")")
So could someone also give some more explanation about it's syntax and why it won't work?
All in Bash.
Consider:
(shopt -s nullglob; for f in *[\(\)]*; do mv "$f" "${f//[()]/_}"; done)
( and ) are syntax, and need to be escaped to be unambiguously referred to as literals.
Setting the nullglob option makes the glob expand to nothing at all, rather than itself, if no files match. Putting the code in a subshell prevents this configuration change from persisting beyond the single command.
Using quotes around expansions is mandatory if you don't want those expansions to be subject to string-splitting and globbing.
Related
Good day,
I have a bunch of files that need to be batch renamed like so:
01-filename1.txt > filename1.txt
02-filename2.txt > filename2.txt
32-filename3.txt > filename3.txt
322-filename4.txt > filename4.txt
31112-filename5.txt > filename5.txt
I run into an example of achieving this using bash ${string#substring} string operation, so this almost works:
for i in `ls`; do mv $i ${i#[0-9]}; done
However, this removes only a single digit and adding regex '+' does not seem to work. Is there a way to strip ALL preceding digits characters?
Thank you!
With Perl's standalone rename command:
rename -n 's/.*?-//' *.txt
If output looks okay, remove -n.
See: The Stack Overflow Regular Expressions FAQ
If you have a single character that always marks the end of the prefix, Pattern Matching makes it very simple.
for f in *; do
mv -nv "$f" "${f#*-}";
done;
Things worth noting:
In your case, the use of ls does not cause problems, but for a more generalized solution, certain filenames would break it. Additionally, the lack of quotes around parameter expansions would cause issues for files with newlines, spaces or tabs in them.
The pattern *- matches any string ending with - combined with lazy prefix removal (one # instead of 2), leads to ${f#*-} evaluating to "$f" with the shortest prefix ending in - removed (if one exists).
Bash's pattern matching is different from and inferior to RegEx, but you can get a little more power by enabling extended pattern matching with shopt -s extglob. Some distributions have this enabled by default.
Also, I threw the -nv flags in mv to ensure no mishaps when playing around with parameter expansion.
More Pattern Matching tricks I often use:
If you want to remove all leading digits and don't always have a single character terminating the prefix, extended pattern matching is helpful: "${f##+([0-9])}"
for i in *
do
name=$( echo "$i" | cut -d "-" -f 2 )
mv "$i" "$name" 2>/dev/null
done
I have been following the answers given in these questions
Shellscript Looping Through All Files in a Folder
How to iterate over files in a directory with Bash?
to write a bash script which goes over files inside a folder and processes them. So, here is the code I have:
#!/bin/bash
YEAR="2002/"
INFOLDER="/local/data/datasets/Convergence/"
for f in "$INFOLDER$YEAR*.mdb";
do
echo $f
absname=$INFOLDER$YEAR$(basename $f)
# ... the rest of the script ...
done
I am receiving this error: basename: extra operand.
I added echo $f and I realized that f contains all the filenames separated by space. But I expected to get one at a time. What could be the problem here?
You're running into problems with quoting. In the shell, double-quotes prevent word splitting and wildcard expansion; generally, you don't want these things to happen to variable's values, so you should double-quote variable references. But when you have something that should be word-split or wildcard-expanded, it cannot be double-quoted. In your for statement, you have the entire file pattern in double-quotes:
for f in "$INFOLDER$YEAR*.mdb";
...which prevents word-splitting and wildcard expansion on the variables' values (good) but also prevents it on the * which you need expanded (that's the point of the loop). So you need to quote selectively, with the variables inside quotes and the wildcard outside them:
for f in "$INFOLDER$YEAR"*.mdb;
And then inside the loop, you should double-quote the references to $f in case any filenames contain whitespace or wildcards (which are completely legal in filenames):
echo "$f"
absname="$INFOLDER$YEAR$(basename "$f")"
(Note: the double-quotes around the assignment to absname aren't actually needed -- the right side of an assignment is one of the few places in the shell where it's safe to skip them -- but IMO it's easier and safer to just double-quote all variable references and $( ) expressions than to try to keep track of where it's safe and where it's not.)
Just quote your shell variables if they are supposed to contain strings with spaces in between.
basename "$f"
Not doing so will lead to splitting of the string into separate characters (see WordSplitting in bash), thereby messing up the basename command which expects one string argument rather than multiple.
Also it would be a wise to include the * outside the double-quotes as shell globbing wouldn't work inside them (single or double-quote).
#!/bin/bash
# good practice to lower-case variable names to distinguish them from
# shell environment variables
year="2002/"
in_folder="/local/data/datasets/Convergence/"
for file in "${in_folder}${year}"*.mdb; do
# break the loop gracefully if no files are found
[ -e "$file" ] || continue
echo "$file"
# Worth noting here, the $file returns the name of the file
# with absolute path just as below. You don't need to
# construct in manually
absname=${in_folder}${year}$(basename "$file")
done
just remove "" from this line
for f in "$INFOLDER$YEAR*.mdb";
so it looks like this
#!/bin/bash
YEAR="2002/"
INFOLDER="/local/data/datasets/Convergence/"
for f in $INFOLDER$YEAR*.mdb;
do
echo $f
absname=$INFOLDER$YEAR$(basename $f)
# ... the rest of the script ...
done
I am trying to copy a .nii file (Gabor3.nii) path to a variable but even though the file is found by the find command, I can't copy the path to the variable.
find . -type f -name "*.nii"
Data= '/$PWD/"*.nii"'
output:
./Gabor3.nii
./hello.sh: line 21: /$PWD/"*.nii": No such file or directory
What went wrong
You show that you're using:
Data= '/$PWD/"*.nii"'
The space means that the Data= parts sets an environment variable $Data to an empty string, and then attempts to run '/$PWD/"*.nii"'. The single quotes mean that what is between them is not expanded, and you don't have a directory /$PWD (that's a directory name of $, P, W, D in the root directory), so the script "*.nii" isn't found in it, hence the error message.
Using arrays
OK; that's what's wrong. What's right?
You have a couple of options. The most reliable is to use an array assignment and shell expansion:
Data=( "$PWD"/*.nii )
The parentheses (note the absence of spaces before the ( — that's crucial) makes it an array assignment. Using shell globbing gives a list of names, preserving spaces etc in the names correctly. Using double quotes around "$PWD" ensures that the expansion is correct even if there are spaces in the current directory name.
You can find out how many files there are in the list with:
echo "${#Data[#]}"
You can iterate over the list of file names with:
for file in "${Data[#]}"
do
echo "File is [$file]"
ls -l "$file"
done
Note that variable references must be in double quotes for names with spaces to work correctly. The "${Data[#]}" notation has parallels with "$#", which also preserves spaces in the arguments to the command. There is a "${Data[*]}" variant which behaves analogously to "$*", and is of similarly limited value.
If you're worried that there might not be any files with the extension, then use shopt -s nullglob to expand the globbing expression into an empty list rather than the unexpanded expression which is the historical default. You can unset the option with shopt -u nullglob if necessary.
Alternatives
Alternatives involve things like using command substitution Data=$(ls "$PWD"/*.nii), but this is vastly inferior to using an array unless neither the path in $PWD nor the file names contain any spaces, tabs, newlines. If there is no white space in the names, it works OK; you can iterate over:
for file in $Data
do
echo "No white space [$file]"
ls -l "$file"
done
but this is altogether less satisfactory if there are (or might be) any white space characters around.
You can use command substitution:
Data=$(find . -type f -name "*.nii" -print -quit)
To prevent multiline output, the -quit option stop searching after the first file was found(unless you're sure only one file will be found or you want to process multiple files).
The syntax to do what you seem to be trying to do with:
Data= '/$PWD/"*.nii"'
would be:
Data="$(ls "$PWD"/*.nii)"
Not saying it's the best approach for whatever you want to do next of course, it's probably not...
If I have a file name with spaces and a random set of numbers that looks like this:
file name1234.csv
I want to rename it to this (assuming date is previously specified):
file_name_${date}.csv
I am able to do it like this:
mv 'file name'*'.csv file_name_${date}.csv
However, in a situation that 'file name*.csv' can actually match multiple files, I want to specify that it's 'file name[random numbers].csv'
I've searched around and can't find any relevant answers.
You need what is called a "pathname expansion", to match one or more digits:
+([0-9])
A functional script could be like this one:
date=$(date +'%Y-%m-%d')
shopt -s extglob nullglob
for f in 'file name'+([[:digit:]]).csv; do
file="${f%%[0-9]*}"
echo mv "$f" "${file// /_}_${date}.csv"
done
Warning: all files found will be renamed to just one name, make sure that that is what you want before removing the echo.
To activate the extended version of "Pathname Expansion" we use shopt -s extglob.
To avoid the case where no file is matched, we also need the nullglob set.
We can set the positional arguments to the result of the above expansion.
Then we loop over all files found to change each of their names.
The ${f%%[0-9]*} removes all from the digits to the end.
The ${file// /_} replaces spaces with underscores.
The mv is not actually done with the script presented because of the echo.
If after running a test, you want the change(s) performed, remove the echo.
Use Extended Globs and Parameter Expansion
You can do what you want with Bash extended globs and a few parameter expansions, without resorting to external or non-standard utilities.
date="2016-11-21"
shopt -s extglob
for file in 'file name'+([[:digit:]]).csv; do
newfile="${file%%[0-9]*}"
newfile="${newfile// /_}"
mv "$file" "${newfile}_${date}.csv"
done
I am trying achieve the same effect as typing
mv ./images/*.{pdf,eps,jpg,svg} ./images/junk/
at the command line, from inside a bash script. I have:
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='{pdf,eps,jpg,svg}'
mv "$MYDIR/"*.$SUFFIXES "$OTHERDIR/"
which, when run, gives the not unexpected error:
mv: rename ./images/*.{pdf,eps,jpg,svg} to ./images/junk/*.{pdf,eps,jpg,svg}:
No such file or directory
What is the correct way to quote all this so that mv will actually do the desired expansion? (Yes, there are plenty of files that match the pattern in ./images/.)
A deleted answer was on the right track. A slight modification to your attempt:
shopt -s extglob
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='#(pdf|eps|jpg|svg)'
mv "$MYDIR/"*.$SUFFIXES "$OTHERDIR/"
Brace expansion is done before variable expansion, but variable expansion is done before pathname expansion. So the braces are still braces when the variable is expanded in your original, but when the variable instead contains pathname elements, they have already been expanded when the pathname expansion gets done.
You'll need to eval that line in order for it to work, like so:
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='{pdf,eps,jpg,svg}'
eval "mv \"$MYDIR\"/*.$SUFFIXES \"$OTHERDIR/\""
Now, this has problems, in particular, if you don't trust $SUFFIXES, it might contain an injection attack, but for this simple case it should be alright.
If you are open to other solutions, you might want to experiment with find and xargs.
You can write a function:
function expand { for arg in "$#"; do [[ -f $arg ]] && echo $arg; done }
then call it with what you want to expand:
expand "$MYDIR/"*.$SUFFIXES
You can also make it a script expand.sh if you like.