In bash, when I want to iterate in a recursive list of pdf files, without the extension, I could do the following:
for file in `find mypath -type f -name '*.pdf' -printf "%f\n"`
do
echo "${file%.*}"
done
This works perfectly, and I get a list of the pdf files without the extension.
But if I try to do the same in a Makefile, I get empty output:
my_test:
#for file in `find mypath -type f -name '*.pdf' -printf "%f\n"`; \
do \
echo "${file%.*}"; \
done; \
do you have an idea why this is happening?
thanks in advance
Just put in an extra $:
echo "$${file%.*}"; \
In your command Make expands the first $, interprets ${ as nothing, and things unravel fast. With $$, the first $ escapes the second and the ${...} gets passed to the shell.
Related
Hello stackoverflow community,
I'm facing a problem with removing files that contain spaces in filename, i have this part of code which is responsible of deleting files that we get from a directory,
for f in $(find $REP -type f -name "$Filtre" -mtime +${DelAvtPurge})
do
rm -f $f
I know that simple or double quotes are working for deleting files with spaces, it works for me when i try them in a command line, but when i put them in $f in the file it doesn't work at all.
Could anybody help me to find a solution for this ?
GNU find has -delete for that:
find "$REP" -type f -name "$Filtre" -mtime +"$DelAvtPurge" -delete
With any other find implementation, you can use bulk-exec:
find "$REP" -type f -name "$Filtre" -mtime +"$DelAvtPurge" -exec rm -f {} +
For a dry-run, drop -delete from the first and see the list of files to be deleted; for second, insert echo before rm.
The other answer has shown how to do this properly. But fundamentally the issue in your command is the lack of quoting, due to the way the shell expands variables:
rm -f $f
needs to become
rm -f "$f"
In fact, always quoting your variables is safe and generally a good idea.
However, this will not fix your code. Now filenames with spaces will work, but filenames with other valid characters (to wit, newlines) won’t. Try it:
touch foo$'\n'bar
for f in $(find . -maxdepth 1 -name foo\*); do echo "rm -f $f"; done
Output:
rm -f ./foo
rm -f bar
Clearly that won’t do. In fact, you mustn’t parse the output of find, for this reason. The only way of making this safe, apart from the solution via find -exec is to use the -print0 option:
find "$REP" -type f -name "$Filtre" -mtime +"$DelAvtPurge" -print0 \
| IFS= while read -r -d '' f; do
rm -f "$f"
done
Using -print0 instead of (implicit) -print causes find to delimit hits by the null character instead of newline. Correspondingly, IFS= read -r -d '' reads a null-character delimited input string, which we do in a loop using while (the -r option prevents read from interpreting backslashes as escape sequences).
This command works
find . -name \*.txt -print
and outputs two filenames
This command works
bash -c 'echo . $0 $1 -print' "-name" "\*.txt"
and outputs this result:
. -name *.txt -print
But this command
bash -c 'find . $0 $1 -print' "-name" "\*.txt"
does not give an error but does not output anything either.
Can anyone tell me what is happening here?
It looks like you're trying to use "\*.txt" to forestall glob-expansion so that the find command sees *.txt instead of e.g. foo.txt.
However, what it ends up seeing is \*.txt. No files match that pattern, so you see no output.
To make find see *.txt as its 3rd argument, you could do this:
bash -c 'find . $0 "$1" -print' "-name" "*.txt"
Edit: Are you really getting . -name *.txt -print as the output of the first command where you replaced find with echo? When I run that command, I get . -name \*.txt -print.
Well the suggestions from francesco work. But I am still confused by the behaviour here.
We know that putting unquoted wild cards in a find command will usually result in an error. To wit:
find . -name *.txt -print
find: paths must precede expression: HowTo-Word-Split.txt' find:
possible unquoted pattern after predicate-name'?
However putting the wild card in single quotes (or escaping it if it is only 1 char) will work like so:
find . -name \*.txt -print
which gives this output (on two separate lines)
> ./HowTo-Word-Split.txt
> ./bash-parms.txt
So in the bash -c version what I was thinking was this:
bash -c 'find . $0 $1 -print' "-name" "*.txt"
would result in the *.txt being expanded even before being passed in to the cmd string,
and using single quotes around it would result in trying to execute (after the arg substitution and the -c taking effect)
find . -name *.txt -print
which as I just demonstrated does not work.
However there seems to be some sort of magic associated with the -c switch as demonstrated by setting -x at the bash prompt, like so:
$ set -x
$ bash -c ' find . $0 "$1" -print' "-name" "*.txt"
+ bash -c ' find . $0 "$1" -print' -name '*.txt'
./HowTo-Word-Split.txt
./bash-parms.txt
Note that even though I used double quotes in the -c line, bash actually executed the find with single quotes put around the argument, thus making find work.
Problem solved. :)!
I am facing a problem with the following shell script:
#!/bin/bash
searchPattern=".*\/.*\.abc|.*\/.*\.xyz|.*\/.*\.[0-9]{3}"
subFolders=$(find -E * -type d -regex ".*201[0-4][0-1][0-9].*|.*20150[1-6].*" -maxdepth 0 | sed 's/.*/"&"/')
echo "subFolders: $subFolders"
# iterate through subfolders
for thisFolder in $subFolders
do
echo "The current subfolder is: $thisFolder"
find -E $thisFolder -type f -iregex $searchPattern -maxdepth 1 -print0 | xargs -0 7z a -mx=9 -uz1 -x!.DS_Store ${thisFolder}/${thisFolder}_data.7z
done
The idea behind it is to archive filetypes with the ending .abc, .xyz and .000-.999 in one 7z archive per subfolder. However, I can't manage to deal with folders including spaces. When I run the script as shown above I always get the following error:
find: "20130117_test": No such file or directory
If I run the script with the line
subFolders=$(find -E * -type d -regex ".*201[0-4][0-1][0-9].*|.*20150[1-6].*" -maxdepth 0 | sed 's/.*/"&"/')
changed to
subFolders=$(find -E * -type d -regex ".*201[0-4][0-1][0-9].*|.*20150[1-6].*" -maxdepth 0)
the script works like charm, but of course not for folders containing space.
Strangely enough, when I execute the following line directly in shell, it works as expected:
find -E "20130117_test" -type f -iregex ".*\/.*\.abc|.*\/.*\.xyz|.*\/.*\.[0-9]{3}" -maxdepth 1 -print0 | xargs -0 7z a -mx=9 -uz1 -x!.DS_Store "20130117_test"/"20130117_test"_data.7z
I know the issue is somehow related to the storing of a list of folders (in quotes) in the subFolders variable, but I simply cannot find a way to make it work properly.
I hope someone more advanced in shell can help me out here.
In general, you should not use find in an attempt to generate a list of file names. You especially cannot build a quoted list the way you are attempting; there is a difference between quotes in a parameter value and quotes around a parameter expansion. Here, especially, you can just use simple patterns:
shopt -s nullglob
subFolders=(
*201[0-4][0-1][0-9]*
*20150[1-6]*
)
for thisFolder in "${subFolders[#]}"; do
echo "The current subfolder is: $thisFolder"
to_archive=(
*/*.abc
*/*.xyz
*/*.[0-9][0-9][0-9]
)
7z a -mx9 -uz1 -x!.DS_Store "$thisFolder/$thisFolder_data.7z" "${to_archive[#]}"
done
Combining the input from gniourf_gniourf and chepner I was able to produce the following code, which does exactly what I want.
#!/bin/bash
shopt -s nullglob
find -E "$PWD" -type d -maxdepth 1 -regex ".*201[0-5][0-1][0-9].*" -print0 | while IFS="" read -r -d "" thisFolder ; do
echo "The current folder is: $thisFolder"
to_archive=( "$thisFolder"/*.[Aa][Bb][Cc] "$thisFolder"/*.[Xx][Yy][Zz] "$thisFolder"/*.[0-9][0-9][0-9] )
if [ ${#to_archive[#]} != 0 ]
then
7z a -mx=9 -uz1 -x!.DS_Store "$thisFolder"/"${thisFolder##*/}"_data.7z "${to_archive[#]}" && rm "${to_archive[#]}"
fi
done
shopt -s nullglob leads to ignorance towards non-matching characters
find... searches for directories matching the regex pattern and streams each matching folder to the while loop using the null separator.
inside the while loop I can safely quote the $thisFolder variable expansion and therefore deal with possible spaces.
using absolute paths instead of relative paths instructs 7z to create no folders inside the archive
I am writing a script that wraps the find command to search for specific source file types under a given directory. A sample invocation would be :
./find_them.sh --java --flex --xml dir1
The above command would search for .java, .as and .xml files under dir1.
To do this manually I came up with the following find command :
find dir1 -type f -a \( -name "*.java" -o -name "*.as" -o -name "*.xml" \)
As I am doing this in a script where I want to be able specify different file sets to search for you end up with the following structure :
find_cmd_file_sets=$(decode_file_sets) # Assume this creates a string with the file sets e.g. -name "*.java" -o -name "*.as" etc
dirs=$(get_search_dirs) # assume this gives you the list of dirs to search, defaulting to the current directory
for dir in $dirs
do
find $dir -type f -a \( $find_cmd_file_sets \)
done
The above script doesn't behave as expected, you execute the script and the find command churns for a while before returning no results.
I'm certain the equivalents of decode_file_sets and get_search_dirs I've created are generating the correct results.
A simpler example if to execute the following directly in a bash shell
file_sets=' -name "*.java" -o -name "*.as" '
find dir -type f -a \( $file_sets \) # Returns no result
# Executing result of below command directly in the shell returns correct result
echo find dir -type f -a \\\( $file_sets \\\)
I don't understand why variable expansion in brackets of the find command would change the result. If it makes any difference I am using git-bash under Windows.
This is really frustrating. Any help would be much appreciated. Most importantly I would like to understand why the variable expansion of $file_sets is behaving as it is.
Hope this will work, Its tested on bash.
file_sets=' -name "*.java" -o -name "*.as" '
command=`echo "find $dir -type f -a \( $file_sets \)"`
eval $command
TLDR: Don't use quotes in find_cmd_file_sets variable and disable pathname expansion (set -f) before calling find.
When you have "special" character in a variable content and then you try to expand that variable without quotes than bash will surround each word with "special" character with single quotes, e.g.:
#!/usr/bin/env bash
set -x
VAR='abc "def"'
echo $VAR
The output is:
+ VAR='abc "def"'
+ echo abc '"def"'
abc "def"
As you can see, bash surrounded "def" with single quotes. In your case, the call to find command becomes:
find ... -name '"*.java"' ...
So it tries to find files which start with " and end with .java"
To prevent that behavior, the only thing you can do (which I'm aware of) is to use double quotes when expanding the variable, e.g.:
#!/usr/bin/env bash
set -x
VAR='abc "def"'
echo "$VAR"
The output is:
+ VAR='abc "def"'
+ echo 'abc "def"'
abc "def"
The only problem, as you probably noticed already, is that now the whole variable is in quotes and is treated as single argument. So this won't work in your find command.
The only option left is to not use quotes, neither in variable content nor when expanding the variable. But then, of course, you have a problem with pathname expansion:
#!/usr/bin/env bash
set -x
VAR='abc *.java'
echo $VAR
The output is:
+ VAR='abc *.java'
+ echo abc file1.java file2.java
abc file1.java file2.java
Fortunately you can disable pathname expansion using set -f:
#!/usr/bin/env bash
set -x
VAR='abc *.java'
set -f
echo $VAR
The output is:
+ VAR='abc *.java'
+ set -f
+ echo abc '*.java'
abc *.java
To sum up, the following should work:
#!/usr/bin/env bash
pattern='-name *.java'
dir="my_project"
set -f
find "$dir" -type f -a \( $pattern \)
bash arrays were introduced to allow this kind of nested quoting:
file_sets=( -name "*.java" -o -name "*.as" )
find dir -type f -a \( "${file_sets[#]}" \)
I have a lot of directory that end with "_ and 6 digits", eg:
diff_gb_and_pf_2voids_158543
I would like to find all that folders in the current folder, and rename them by deleting the "_" and the 6 digits at the end.
So far I'm stuck with this command:
find . -type d -print |grep '.*[0-9]\{6\}$' |xargs -I {} bash -c 'for i in {}; do mv "$i" ????; done;'
I can't find how to do the last step. I would try and call sed, but how ?
Also, if there is a nicer way, please tell.
Thanks
Here's one way using your shell:
for i in $(find . -mindepth 1 -type d -regextype posix-extended -regex '.*_[0-9]{6}'); do
mv "$i" "${i%_*}";
done
Here you go:
find /path -regex '.*_[0-9]\{6\}' -exec sh -c 'n="{}"; echo mv "{}" "${n%_*}"' \;
Check the output, if it looks good then drop the echo in there.
Explanation: for each matched file, we run a sub-shell, where we assign the filename to variable n, so that we can use pattern substitution ${n%_*}, which cuts off the last _ character and everything after it until the end of the filename.
Or here's a more portable way that should work in older systems too:
find /path -name '*_[0-9][0-9][0-9][0-9][0-9][0-9]' | sed -ne 's/\(.*\)_[0-9]\{6\}$/mv "&" "\1"/p'
Check the output, if it looks good than pipe it to sh (append this: | sh)
Explanation:
The sed command receives the list of files to rename
In the pattern we capture the first part of the filename within \( ... \)
We replace the pattern with the text mv "&" "\1", where & is substituted with the pattern that was matched, in this case the entire original filename, and \1 is substituted with the part we captured within \( ... \)
#!/usr/bin/env bash
set -x
ls -d diff* > dirlist
while IFS='_' read field1 field2 field3 field4 field5 field6
do
mv -v "${field1}_${field2}_${field3}_${field4}_${field5}_${field6}" \
"${field1}_${field2}_${field3}_${field4}_${field5}"
done < dirlist