I want to read the full file name, but it always auto separator by space
#!/bin/dash
for file in `ls | grep -E '(jpg)$' | sed 's/\.jpg//g'`
do
echo ${file}
done
This will work when the file name do not contain space or other symbol.
But it do not work when the file name like : A B C.jpg
This will print
A
B
C
It sepator the file name by space, how can i avoid this situation
Unquoted command substitutions are split on IFS (white space by default).
You can use shell glob expansion, and shell suffix removal:
for file in *.jpg; do
[ -f "$file" ] || continue
name=${file%.jpg}
echo "$name"
done
*.[Jj][Pp][Gg] for a case insensitive match (bash also has shopt -s nocasematch).
Related
I am working on a short script to search a large number of folders on a NAS for this odd character and delete the character. I am on a Synology NAS running Linux. This is what I have so far.
#!/bin/bash
for file in "$(find "/volume1/PLNAS/" -depth -type d -name '**')";
do
echo "$file";
mv "$file" "$(echo $file | sed s/// )";
done
Current problem is that the Kernel does not appear to be passing each MV command separately. I get a long error message that appears to list every file in one command, truncated error message below. There are spaces in my file path and that it why I have tried to quote every variable.
mv: failed to access '/volume1/PLNAS/... UT Thickness Review ': File name too long
Several issues. The most important is probably that for file in "$(find...)" iterates only once with file set to the full result of your search. This is what the double quotes are for: prevent word splitting.
But for file in $(find...) is not safe: if some file names contain spaces they will be split...
Assuming the character is unicode 0xf028 ( ) try the following:
while IFS= read -r -d '' file; do
new_file="${file//$'\uf028'}"
printf 'mv %s %s\n' "$file" "$new_file"
# mv "$file" "$new_file"
done < <(find "/volume1/PLNAS/" -depth -type d -name $'*\uf028*' -print0)
Uncomment the mv line if things look correct.
As your file names are unusual we use the -d '' read separator and the print0 find option. This will use the NUL character (ASCII code zero) as separator between the file names instead of the default newline characters. The NUL character is the only one that you cannot find in a full file name.
We also use the bash $'...' expansion to represent the unwanted character by its unicode hexadecimal code, it is safer than copy-pasting the glyph. The new name is computed with the bash pattern substitution (${var//}).
Note: do not use echo with unusual strings, especially without quoting the strings (e.g. your echo $file | ...). Prefer printf or quoted here strings (sed ... <<< "$file").
Below is a script and its output describing the problem I found today. Even though ls output is quoted, bash still breaks at the whitespaces. I changed to use for file in *.txt, just want to know why bash behaves this way.
[chau#archlinux example]$ cat a.sh
#!/bin/bash
FILES=$(ls --quote-name *.txt)
echo "Value of \$FILES:"
echo $FILES
echo
echo "Loop output:"
for file in $FILES
do
echo $file
done
[chau#archlinux example]$ ./a.sh
Value of $FILES:
"b.txt" "File with space in name.txt"
Loop output:
"b.txt"
"File
with
space
in
name.txt"
Why bash ignored the quotation in ls output?
Because word splitting happens on the result of variable expansion.
When evaluating a statement the shell goes through different phases, called shell expansions. One of these phases is "word splitting". Word splitting literally does split your variables into separate words, quoting from the bash manual:
The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.
The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words using these characters as field terminators. . If IFS is unset, or its value is exactly <space><tab><newline>, the default, then sequences of <space>, <tab>, and <newline> at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words. ...
When shell has a $FILES, that is not within double quotes, it firsts does "parameter expansion". It expands $FILES to the string "b.txt" "File with space in name.txt". Then word splitting occurs. So with the default IFS, the resulting string is split/separated on spaces, tabs or newlines.
To prevent word splitting the $FILES has to be inside double quotes itself, no the value of $FILES.
Well, you could do this (unsafe):
ls -1 --quote-name *.txt |
while IFS= read -r file; do
eval file="$file"
ls -l "$file"
done
tell ls to output newline separated list -1
read the list line by line
re-evaulate the variable to remove the quotes with evil. I mean eval
I use ls -l "$file" inside the loop to check if "$file" is a valid filename.
This will still not work on all filenames, because of ls. Filenames with unreadable characters are just ignored by my ls, like touch "c.txt"$'\x01'. And filenames with embedded newlines will have problems like ls $'\n'"c.txt".
That's why it's advisable to forget ls in scripts - ls is only for nice-pretty-printing in your terminal. In scripts use find.
If your filenames have no newlines embedded in them, you can:
find . -mindepth 1 -maxdepth 1 -name '*.txt' |
while IFS= read -r file; do
ls -l "$file"
done
If your filenames are just anything, use a null-terminated stream:
find . -mindepth 1 -maxdepth 1 -name '*.txt' -print0 |
while IFS= read -r -d'' file; do
ls -l "$file"
done
Many, many unix utilities (grep -z, xargs -0, cut -z, sort -z) come with support for handling zero-terminated strings/streams just for handling all the strange filenames you can have.
You can try the follwing snippet:
#!/bin/bash
while read -r file; do
echo "$file"
done < <(ls --quote-name *.txt)
I'm trying to write a bash script that read user's input (some files so user can use TAB completion) and copy them into a specific folder.
#/bin/bash
read -e files
for file in $files
do
echo $file
cp "$file" folder/"$file"
done
It's ok for: file1 file2 ...
Or with : file* (even if there is a filename with space in the folder).
But it's not working for filenames with space escaped with backslash \ like : file\ with\ space escaped spaces are ignored and string is split on each spaces, even escaped.
I saw information on quoting, printf, IFS, read and while... I think it's very basic bash script but I can't find a good solution. Can you help me?
Clearing IFS prior to your unquoted expansion will allow globbing to proceed while preventing string-splitting:
IFS=$' \t\n' read -e -a globs # read glob expressions into an array
IFS=''
for glob in "${globs[#]}"; do # these aren't filenames; don't claim that they are.
files=( $glob ) # expand the glob into filenames
# detect the case where no files matched by checking whether the first result exists
# these *would* need to be quoted, but [[ ]] turns off string-splitting and globbing
[[ -e $files || -L $files ]] || {
printf 'ERROR: Glob expression %q did not match any files!\n' "$glob" >&2
continue
}
printf '%q\n' "${files[#]}" # print one line per file matching
cp -- "${files[#]}" folder/ # copy those files to the target
done
Note that we're enforcing the default IFS=$' \t\n' during the read operation, which ensures that unquoted whitespace is treated as a separator between array elements at that stage. Later, with files=( $glob ), by contrast, we have IFS='', so whitespace no longer can break individual names apart.
You can read the filenames into an array, then loop over the array elements:
read -e -a files
for file in "${files[#]}"; do
echo "$file"
cp "$file" folder/"$file"
done
Reading into a single string won't work no matter how you quote: the string will either be split up at each space (when unquoted) or not at all (when quoted). See this canonical Q&A for details (your case is the last item in the list).
This prevents globbing, i.e., file* is not expanded. For a solution that takes this into account, see Charles' answer.
There is a fully functional solution for files and globs.
With the help of using xargs (which is able to preserve quoted strings). But you need to write files with spaces inside quotes:
"file with spaces"
When you use the script: Unquote the read and quote the assignment for listOfFiles.
I am also taking advantage of some ideas on the post of #CharlesDuffy (thanks Charles).
#!/bin/bash
# read -e listOfFiles
listOfFiles='file1 file* "file with spaces"'
IFS=''
while IFS='' read glob; do # read each file expressions into an array
files=( $glob ) # try to expand the glob into filenames
# If no file match the split glob
# Then assume that the glob is a file and test its existence
[[ -e $files || -L $files ]] || {
files="$glob"
[[ -e $files || -L $files ]] || {
printf 'ERROR: Glob "%q" did not match any file!\n' "$glob" >&2
continue
}
}
printf '%q\n' "${files[#]}" # print one line per file matching
cp -- "${files[#]}" folder/ # copy those files to the target
done < <(xargs -n1 <<<"$listOfFiles")
Note that the answers of both Charles Duffy and user2350426 do not preserve escaped *s; they will expand them, too.
Benjamin's approach, however, won't do globbing at all. He is mistaken in that you can first put your globs in a string and then load them into an array.
Then it will work as desired:
globs='file1 file\ 2 file-* file\* file\"\"' # or read -re here
# Do splitting and globbing:
shopt -s nullglob
eval "files=( $globs )"
shopt -u nullglob
# Now we can use ${files[#]}:
for file in "${files[#]}"; do
printf "%s\n" "$file"
done
Also note the use of nullglob to ignore non-expandable globs.
You may also want to use failglob or, for more fine-grained control, code like in the aforementioned answers.
Inside functions, you probably want to declare variables, so they stay local.
How can I replace all underscore chars with a whitespace in multiple file names using Bash Script? Using this code we can replace underscore with dash. But how it works with whitespace?
for i in *.mp3;
do x=$(echo $i | grep '_' | sed 's/_/\-/g');
if [ -n "$x" ];
then mv $i $x;
fi;
done;
Thank you!
This should do:
for i in *.mp3; do
[[ "$i" = *_* ]] && mv -nv -- "$i" "${i//_/ }"
done
The test [[ "$i" = *_* ]] tests if file name contains any underscore and if it does, will mv the file, where "${i//_/ }" expands to i where all the underscores have been replaced with a space (see shell parameter expansions).
The option -n to mv means no clobber: will not overwrite any existent file (quite safe). Optional.
The option -v to mv is for verbose: will say what it's doing (if you want to see what's happening). Very optional.
The -- is here to tell mv that the arguments will start right here. This is always good practice, as if a file name starts with a -, mv will try to interpret it as an option, and your script will fail. Very good practice.
Another comment: When using globs (i.e., for i in *.mp3), it's always very good to either set shopt -s nullglob or shopt -s failglob. The former will make *.mp3 expand to nothing if no files match the pattern (so the loop will not be executed), the latter will explicitly raise an error. Without these options, if no files matching *.mp3 are present, the code inside loop will be executed with i having the verbatim value *.mp3 which can cause problems. (well, there won't be any problems here because of the guard [[ "$i" = *_* ]], but it's a good habit to always use either option).
Hope this helps!
The reason your script is failing with spaces is that the filename gets treated as multiple arguments when passed to mv. You'll need to quote the filenames so that each filename is treated as a single agrument. Update the relevant line in your script with:
mv "$i" "$x"
# where $i is your original filename, and $x is the new name
As an aside, if you have the perl version of the rename command installed, you skip the script and achieve the same thing using:
rename 's/_/ /' *.mp3
Or if you have the more classic rename command:
rename "_" " " *.mp3
Using tr
tr '_' ' ' <file1 >file2
I have to remove .. character from a file in Bash script. Example:
I have some string like:
some/../path/to/file
some/ab/path/to/file
And after replace, it should look like
some/path/to/file
some/ab/path/to/file
I have used below code
DUMMY_STRING=/../
TEMP_FILE=./temp.txt
sed s%${DUMMY_STRING}%/%g ${SRC_FILE} > ${TEMP_FILE}
cp ${TEMP_FILE} ${SRC_FILE}
It is replacing the /../ in line 1; but it is also removing the line /ab/ from second line. This is not desired. I understand it is considering /../ as some regex and /ab/ matches this regex. But I want only those /../ to be replaced.
Please provide some help.
Thanks,
NN
The . is a metacharacter in sed meaning 'any character'. To suppress its special meaning, escape it with a backslash:
sed -e 's%/\.\./%/%g' $src_file > $temp_file
Note that you are referring to different files after you eliminate the /../ like that. To refer to the same name as before (in the absence of symlinks, which complicate things), you would need to remove the directory component before the /../. Thus:
some/../path/to/file
path/to/file
refer to the same file, assuming some is a directory and not a symlink somewhere else, but in general, some/path/to/file is a different file (though symlinks could be used to confound that assertion).
$ x="some/../path/to/file
> some/ab/path/to/file
> /some/path/../to/another/../file"
$ echo "$x"
some/../path/to/file
some/ab/path/to/file
/some/path/../to/another/../file
$ echo "$x" | sed -e 's%/\.\./%/%g'
some/path/to/file
some/ab/path/to/file
/some/path/to/another/file
$ echo "$x" | sed -e "s%/\.\./%/%g"
some/path/to/file
some/ab/path/to/file
/some/path/to/another/file
$ echo "$x" | sed -e s%/\.\./%/%g
some/path/file
some/path/file
/some/path/to/another/file
$ echo "$x" | sed -e s%/\\.\\./%/%g
some/path/to/file
some/ab/path/to/file
/some/path/to/another/file
$
Note the careful use of double quotes around the variable "$x" in the echo commands. I could have used either single or double quotes in the assignment and would have gotten the same result.
Test on Mac OS X 10.7.4 with the standard sed (and shell is /bin/sh, aka bash 3.2.x), but the results would be the same on any system.