I have a directory $dir that contains .txt.xy files and subdirectories with .txt.xy files. I try to iterate over each file and pass the whole path as well as the path without $dir as argument to a program like this:
dir="/path/to/"
suffix=".xy"
find "$dir" -name "*.txt.xy" -print0 | xargs -0 -I {} sh -c 'program "$1" |
subprogram filename="$2"' _ {} "$(echo {} | sed -e "s#^${dir}##" -e "s#${suffix}\$##")"
$1 shold be the full path (e.g. /path/to/subdir/file.txt.xy)
$2 should be the full path without $dir and $suffix (e.g. subdir/file.txt)
$1 is the propper full path but $2 is also the full path as if the pipe in $(...) is never executed. What am I missing here?
Your attempt seems rather roundabout. It sounds like you are looking for
find /path/to -name "*.txt.xy" -exec sh -c '
for f; do
g=${f##*/}
program "$f" | subprogram filename="${g%.xy}"
done' _ {} +
If you really need your parameters to be in variables, maybe pass in the suffix as $0 which isn't used for anything useful here anyway. It's a bit obscure, but helps avoid the mess you had with double quotes.
find /path/to -name "*.txt.xy" -exec sh -c '
for f; do
g=${f##*/}
program "$f" | subprogram filename="${g%"$0"}"
done' ".xy" {} +
The above simply trims g to the basename which I guess on closer reading you don't want. So pass /path/to in $0 instead, and hardcode .xy inside:
find /path/to -name "*.txt.xy" -exec sh -c '
for f; do
g=${f#"$0"/}
program "$f" | subprogram filename="${g%.xy}"
done' "/path/to" {} +
or if you really need both to be command-line parameters,
dir="/path/to"
suffix=".xy"
find "$dir" -name "*.txt$suffix" -exec sh -c '
suffix=$1
shift
for f; do
g=${f#"$0"/}
program "$f" | subprogram filename="${g%"$suffix"}"
done' "$dir" "$suffix" {} +
One reason for the failure, is that the command substitution with xargs under double quotes is expanded by the shell even before the former is executed. One way to avoid that would be to do the whole substitution inside the sub-shell created by sh -c as
find "$dir" -name "*.txt.xy" -print0 |
xargs -0 -I {} sh -c '
f="{}"
g="$(echo "$f" | sed -e 's|"'^"${dir}"'"||' -e 's|"'\\"${suffix}"$'"||' )"
program "$f" | subprogram filename="$g"
'
Related
I'm trying to use the cut function to parse filenames, but am encountering difficulty while doing so in a find loop With the intention of converting my music library from ARTIST - TITLE.EXT to TITLE.EXT
So If I had the file X - Y.EXT it should yield Y.EXT as an output.
The current function is something like this:
find . -iname "*.mp3" -exec cut -d "-" -f 2 <<< "`echo {}`" \;
It should be noted that the above syntax looks a bit strange, why not just use <<< {} \; instead of the echo {}. cut seems to parse the file instead of the filename if it's not given a string.
Another attempt I had looked something like:
find . -iname "*.mp3" -exec TRACKTITLE=`echo {} | cut -d '-' -f2` \; -exec echo "$TRACKTITLE" \;
But this fails with find: ‘TRACKTITLE=./DAN TERMINUS - Underwater Cities.mp3’: No such file or directory.
This (cut -d "-" -f 2 <<< FILENAME) command works wonderfully for a single instance (although keeps the space after the "-" character frustratingly).
How can I perform this operation in a find loop?
First thing is try to extract what you want in your file name with Parameter Expansion.
file="ARTIST - TITLE.EXT"
echo "${file#* - }"
Output
TITLE.EXT
Using find and invoking a shell with a for loop.
find . -type f -iname "*.mp3" -exec sh -c 'for music; do echo mv -v "$music" "${music#* - }"; done' sh {} +
If there are .mp3 files in sub directories, just change
-exec
with
-execdir
if available/supported by your find
For whatever reason -execdir is not available.
find . -type f -iname "*.mp3" -exec sh -c '
for music; do
pathname="${music%/*}"
filename="${music##*/}"
new_music="${filename#* - }"
echo mv -v "$music" "$pathname/$new_music"
done' sh {} +
Remove the echo if you're satisfied with the output.
See Understanding -exec option to Find
Below command would say what it would do, remove echo to actually
run mv:
find . -iname "*.mp3" -exec sh -c 'echo mv "$1" "$(echo "$1" | cut -d - -f2)"' sh {} \;
Example output:
$ find . -iname "*.mp3" -exec sh -c 'echo mv "$1" "$(echo "$1" | cut -d - -f2)"' sh {} \;
mv ./X - Y.mp3 Y.mp3
mv ./ARTIST - TITLE.mp3 TITLE.mp3
Also notice that your cut command will leave a whitespace at the
beginning of the new filename:
$ echo ARTIST\ -\ TITLE.mp3 | cut -d - -f2-
TITLE.mp3
You don't need the find nor the cut for this task.
for f in *' - '*.mp3; do mv -i "$f" "${f##* - }"; done
will do the job for the current directory.
If you want to descend through directories, then:
shopt -s globstar
for f in ./**/*' - '*.mp3; do
mv -i "$f" "${f%/*}/${f##* - }"
done
I want to find all files with certain name (Myfile.txt) that do not contain certain string (my-wished-string) and then do a sed in order to do a replace in the found files. I tried with:
find . -type f -name "Myfile.txt" -exec grep -H -E -L "my-wished-string" {} + | sed 's/similar-to-my-wished-string/my-wished-string/'
But this only displays me all files with wished name that miss the "my-wished-string", but does not execute the replacement. Do I miss here something?
With a for loop and invoking a shell.
find . -type f -name "Myfile.txt" -exec sh -c '
for f; do
grep -H -E -L "my-wished-string" "$f" &&
sed -i "s/similar-to-my-wished-string/my-wished-string/" "$f"
done' sh {} +
You might want to add a -q to grep and -n to sed to silence the printing/output to stdout
You can do this by constructing two stacks; the first containing the files to search, and the second containing negative hits, which will then be iterated over to perform the replacement.
find . -type f -name "Myfile.txt" > stack1
while read -r line;
do
[ -z $(sed -n '/my-wished-string/p' "${line}") ] && echo "${line}" >> stack2
done < stack1
while read -r line;
do
sed -i "s/similar-to-my-wished-string/my-wished-string/" "${line}"
done < stack2
With some versions of sed, you can use -i to edit the file. But don't pipe the list of names to sed, just execute sed in the find:
find . -type f -name Myfile.txt -not -exec grep -q "my-wished-string" {} \; -exec sed -i 's/similar-to-my-wished-string/my-wished-string/g' {} \;
Note that any file which contains similar-to-my-wished-string also contains the string my-wished-string as a substring, so with these exact strings the command is a no-op, but I suppose your actual strings are different than these.
I want to insert an integer into filenames with various extensions in multiple subdirectories using bash.
Examples:
./trial2/foo.hhh --> ./trial2/foo1.hhh
./trial2/trial3/foo.txt--> ./trial2/trial3/foo1.txt
I tried to separate the filename from the extension and insert the integer in between with:
i=123
find . -type f -exec sh -c ' echo mv "$0" "${0%.*}${i}.${0##*.}" ' {} \;
mv ./trial2/foo.hhh ./trial2/foo.hhh
But the variable output ${i} is not printed. Why?
If I change to:
find . -type f -exec sh -c ' mv "$0" "${0%.*}123.${0##*.}" ' {} \;
mv ./trial2/foo.hhh ./trial2/foo123.hhh
The number is printed. However, I need the variable ${i} as it will be defined in a wrapping for-loop.
Edit
You are almost there; you just hit a quoting subtlety. i is declared in the current shell, so needs to be passed into the sh run by find somehow. In your current command, ${i} is in single quotes, so is not interpreted by bash before the quoted material is passed to sh.
As suggested by Charles Duffy:
find . -type f -exec sh -c ' echo mv "$0" "${0%.*}${1}.${0##*.}" ' {} "$i" \;
Within the sh command, $0 is the {}, as you know. $1 is the second parameter to the sh, which is "$i" (i expanded as a single word). Instead of ${i}, the sh command uses ${1} to access that parameter (a copy of i).
Original
In this example, i is interpolated in the current shell.
Before: find . -type f -exec sh -c ' echo mv "$0" "${0%.*}${i}.${0##*.}" ' {} \;
After: find . -type f -exec sh -c ' echo mv "$0" "${0%.*}'"${i}"'.${0##*.}" ' {} \;
^^ ^^
The '"${i}"' drops you out of the single quotes, then expands i, then takes you back into the single quotes. That way the command you are passing to sh includes the value of i you want.
Pass $i in $0, and your filenames in $1 and onward, and you can change from -exec ... {} \; to -exec ... {} +, thus using only one sh instance to rename potentially several files.
The following requires bash, but generates mv commands that are guaranteed to be correct even in the presence of malicious or confusing filenames (with spaces, newlines, control characters, etc):
find . -type f -exec bash -c '
logcmd() { printf "%q " "$#"; printf "\n"; } # a more accurate replacement for echo
for f; do
logcmd mv "$f" "${f%.*}${0}.${f##*.}"
done' "$i" {} +
I am trying to use find -exec with multiple commands without any success. Does anybody know if commands such as the following are possible?
find *.txt -exec echo "$(tail -1 '{}'),$(ls '{}')" \;
Basically, I am trying to print the last line of each txt file in the current directory and print at the end of the line, a comma followed by the filename.
find accepts multiple -exec portions to the command. For example:
find . -name "*.txt" -exec echo {} \; -exec grep banana {} \;
Note that in this case the second command will only run if the first one returns successfully, as mentioned by #Caleb. If you want both commands to run regardless of their success or failure, you could use this construct:
find . -name "*.txt" \( -exec echo {} \; -o -exec true \; \) -exec grep banana {} \;
find . -type d -exec sh -c "echo -n {}; echo -n ' x '; echo {}" \;
One of the following:
find *.txt -exec awk 'END {print $0 "," FILENAME}' {} \;
find *.txt -exec sh -c 'echo "$(tail -n 1 "$1"),$1"' _ {} \;
find *.txt -exec sh -c 'echo "$(sed -n "\$p" "$1"),$1"' _ {} \;
Another way is like this:
multiple_cmd() {
tail -n1 $1;
ls $1
};
export -f multiple_cmd;
find *.txt -exec bash -c 'multiple_cmd "$0"' {} \;
in one line
multiple_cmd() { tail -1 $1; ls $1 }; export -f multiple_cmd; find *.txt -exec bash -c 'multiple_cmd "$0"' {} \;
"multiple_cmd()" - is a function
"export -f multiple_cmd" - will export it so any other subshell can see it
"find *.txt -exec bash -c 'multiple_cmd "$0"' {} \;" - find that will execute the function on your example
In this way multiple_cmd can be as long and as complex, as you need.
Hope this helps.
There's an easier way:
find ... | while read -r file; do
echo "look at my $file, my $file is amazing";
done
Alternatively:
while read -r file; do
echo "look at my $file, my $file is amazing";
done <<< "$(find ...)"
Extending #Tinker's answer,
In my case, I needed to make a command | command | command inside the -exec to print both the filename and the found text in files containing a certain text.
I was able to do it with:
find . -name config -type f \( -exec grep "bitbucket" {} \; -a -exec echo {} \; \)
the result is:
url = git#bitbucket.org:a/a.git
./a/.git/config
url = git#bitbucket.org:b/b.git
./b/.git/config
url = git#bitbucket.org:c/c.git
./c/.git/config
I don't know if you can do this with find, but an alternate solution would be to create a shell script and to run this with find.
lastline.sh:
echo $(tail -1 $1),$1
Make the script executable
chmod +x lastline.sh
Use find:
find . -name "*.txt" -exec ./lastline.sh {} \;
Thanks to Camilo Martin, I was able to answer a related question:
What I wanted to do was
find ... -exec zcat {} | wc -l \;
which didn't work. However,
find ... | while read -r file; do echo "$file: `zcat $file | wc -l`"; done
does work, so thank you!
1st answer of Denis is the answer to resolve the trouble. But in fact it is no more a find with several commands in only one exec like the title suggest. To answer the one exec with several commands thing we will have to look for something else to resolv. Here is a example:
Keep last 10000 lines of .log files which has been modified in the last 7 days using 1 exec command using severals {} references
1) see what the command will do on which files:
find / -name "*.log" -a -type f -a -mtime -7 -exec sh -c "echo tail -10000 {} \> fictmp; echo cat fictmp \> {} " \;
2) Do it: (note no more "\>" but only ">" this is wanted)
find / -name "*.log" -a -type f -a -mtime -7 -exec sh -c "tail -10000 {} > fictmp; cat fictmp > {} ; rm fictmp" \;
I usually embed the find in a small for loop one liner, where the find is executed in a subcommand with $().
Your command would look like this then:
for f in $(find *.txt); do echo "$(tail -1 $f), $(ls $f)"; done
The good thing is that instead of {} you just use $f and instead of the -exec … you write all your commands between do and ; done.
Not sure what you actually want to do, but maybe something like this?
for f in $(find *.txt); do echo $f; tail -1 $f; ls -l $f; echo; done
should use xargs :)
find *.txt -type f -exec tail -1 {} \; | xargs -ICONSTANT echo $(pwd),CONSTANT
another one (working on osx)
find *.txt -type f -exec echo ,$(PWD) {} + -exec tail -1 {} + | tr ' ' '/'
A find+xargs answer.
The example below finds all .html files and creates a copy with the .BAK extension appended (e.g. 1.html > 1.html.BAK).
Single command with multiple placeholders
find . -iname "*.html" -print0 | xargs -0 -I {} cp -- "{}" "{}.BAK"
Multiple commands with multiple placeholders
find . -iname "*.html" -print0 | xargs -0 -I {} echo "cp -- {} {}.BAK ; echo {} >> /tmp/log.txt" | sh
# if you need to do anything bash-specific then pipe to bash instead of sh
This command will also work with files that start with a hyphen or contain spaces such as -my file.html thanks to parameter quoting and the -- after cp which signals to cp the end of parameters and the beginning of the actual file names.
-print0 pipes the results with null-byte terminators.
for xargs the -I {} parameter defines {} as the placeholder; you can use whichever placeholder you like; -0 indicates that input items are null-separated.
I found this solution (maybe it is already said in a comment, but I could not find any answer with this)
you can execute MULTIPLE COMMANDS in a row using "bash -c"
find . <SOMETHING> -exec bash -c "EXECUTE 1 && EXECUTE 2 ; EXECUTE 3" \;
in your case
find . -name "*.txt" -exec bash -c "tail -1 '{}' && ls '{}'" \;
i tested it with a test file:
[gek#tuffoserver tmp]$ ls *.txt
casualfile.txt
[gek#tuffoserver tmp]$ find . -name "*.txt" -exec bash -c "tail -1 '{}' && ls '{}'" \;
testonline1=some TEXT
./casualfile.txt
Here is my bash script that you can use to find multiple files and then process them all using a command.
Example of usage. This command applies a file linux command to each found file:
./finder.sh file fb2 txt
Finder script:
# Find files and process them using an external command.
# Usage:
# ./finder.sh ./processing_script.sh txt fb2 fb2.zip doc docx
counter=0
find_results=()
for ext in "${#:2}"
do
# #see https://stackoverflow.com/a/54561526/10452175
readarray -d '' ext_results < <(find . -type f -name "*.${ext}" -print0)
for file in "${ext_results[#]}"
do
counter=$((counter+1))
find_results+=("${file}")
echo ${counter}") ${file}"
done
done
countOfResults=$((counter))
echo -e "Found ${countOfResults} files.\n"
echo "Processing..."
counter=0
for file in "${find_results[#]}"
do
counter=$((counter+1))
echo -n ${counter}"/${countOfResults}) "
eval "$1 '${file}'"
done
echo "All files have been processed."
I have files in a dir.
I need to append a new line and the file name at the end of each file.
This should do:
for f in *; do echo >> $f; echo $f >> $f; done
First echo a new-line, then echo the filename.
The >> says "append at the end of the file".
Edit: aioobe's answer has been updated to show the -e flag that I didn't see when I first answered this. Thus I'm now just showing an example which includes a directory and how to eliminate the directory name:
#!/bin/bash
for fn in dir/*
do
shortname=${fn/#*\//}
echo -e "\n$shortname" >> $fn
done
If you want the directory name, take out the shortname=${fn/#*\//} line and replace $shortname with $fn in the echo.
Let xargs do the looping:
# recursive, includes directory name
find -type f -print0 | xargs -0 -I% bash -c 'echo -e "\n%" >> %'
or
# non-recursive, doesn't include directory name
find -maxdepth 1 -type f -exec basename {} \; | xargs -I% bash -c 'echo -e "\n%" >> %'
or
# non-recursive, doesn't include directory name
find -maxdepth 1 -type f -printf "%f\0" | xargs -0 -I% bash -c 'echo -e "\n%" >> %'
or
# recursive, doesn't include directory name
find -type f -print0 | xargs -0 -I% bash -c 'f=%; echo -e "\n${f##*/}" >> %'
Another method using ex (or vim) :
ex -c 'args **/*' -c 'set hidden' -c 'argdo $put =bufname(".")' -c 'wqa'