I use the following Bash script to count lines of code in one of my projects:
echo "--- CLIENT"
cd "/mypath/client"
# Count classes:
a=`find . -name \*.java -print | wc -l`
echo ""
echo "Number of Java classes: $a"
# Total count:
b=`find . -name \*.java -exec cat {} \; | wc -l`
echo ""
echo "Java lines: $b"
c=`find . -name \*.css -exec cat {} \; | wc -l`
echo ""
echo "CSS lines: $c"
d=`find . -name \*.json -exec cat {} \; | wc -l`
echo ""
echo "JSON lines: $d"
f=$((`find . -name \*.h -exec cat {} \; | wc -l` + `find . -name \*.m -exec cat {} \; | wc -l`))
echo ""
echo "iOS Objective-C lines: $f"
echo ""
echo "--- SERVER"
cd "/mypath/server"
# Count classes:
h=`find . -name \*.java -print | wc -l`
echo ""
echo "Number of Java classes: $h"
# Total count:
i=`find . -name \*.java -exec cat {} \; | wc -l`
echo ""
echo "Java lines: $i"
echo ""
echo "Total lines of code: $((b + c + d + e + f + i))"
cd ~
This script worked fine as long as all the source code was searchable this way. Now I have a different use case: some of the source code is still reachable with this script, and some of it is inside compressed zip files (located in various subfolders of "/mypath/client"). These zip files can contain the sources in the root or in various subfolders within them.
I suppose it's possible to adapt my script to take into account the zipped files in the count, but I don't know how to do it.
Counting Files
When you search for .xyz files, also search for .zip files and search their file list.
You can list all filenames in a zip archive using zipinfo archive.zip. zipinfo also supports wildcards to print only matching filenames. For instance, zipinfo archive.zip '*.java' prints only filenames ending with .java.
find . -name \*.java -print \
-o -name \*.zip -exec zipinfo -1 {} '*.java' \; |
wc -l
This command assumes that filenames do not contain linebreaks.
Counting Lines
You can print zipped files without explicitly extracting them using unzip -p archive.zip file1 file2 .... This command also accepts wildcards.
By the way: You can drastically simplify your script by using a function, since find . -name \*.xyz -exec cat {} \; | wc -l is often the same, except for xyz. Also, -exec cat {} + is way faster than -exec cat {} \;.
#! /bin/bash
countLines() {
local ext=$1
find . -name "*.$ext" -exec cat {} + \
-o -name \*.zip -exec unzip -p {} "*.$ext" \; |
wc -l
}
for ext in java css json; do
echo "$ext lines: $(countLines "$ext")"
done
unzip -p archive.zip '*.java' may print the warning caution: filename not matched: *.java if there are no .java files. You can suppress this by adding 2> /dev/null after the find command.
Keep in mind that this approach is very inefficient. find has to run for each file extension. And the zip files are read multiple times too. It would be faster to filter out all files that you want to inspect first, then run wc -l on all of them, and then sum up their line counts.
How do I properly escape the path to come out of find to a new command argument?
#!/bin/bash
for f in $(find . -type f -name '*.flac')
do
if flac -cd "$f" | lame -bh 320 - "${f%.*}".mp3; then
rm -f "$f"
echo "removed $f"
fi
done
returns
lame: excess arg Island of the Gods - 3.mp3
Using a Bash for loop is not ideal for the results of find or ls. There are other ways to do it.
You may want to use -print0 and xargs to avoid word splitting issues.
$ find [path] -type f -name *.flac -print0 | xargs -0 [command line {xargs puts in fn}]
Or use -exec primary in find:
$ find [path] -type f -name *.flac -exec [process {find puts in fn}] \;
Alternative, you can use a while loop:
find [path] -type f -name *.flac | while IFS= read -r fn; do # fn not quoted here...
echo "$fn" # QUOTE fn here!
# body of your loop
done
I'm working on a bash script to help organize files and I want to use mv -i to make sure I don't write over something important.
The script is working right now except for the -i for the mv.
It shows (y/n [n]) not overwritten part, but then goes and and doesn't allow me to interact with it.
createList()
{
ls *.epub | sed 's/-.*//' |uniq >> list.txt
ls *.mobi | sed 's/-.*//' |uniq >> list2.txt
}
atag()
{
find /Users/j/Desktop/Source -maxdepth 1 -iname "*.epub" -type f -print0 | xargs -0 -I '{}' tag -a Purple {}
find /Users/j/Desktop/Source -maxdepth 1 -iname "*.mobi" -type f -print0 | xargs -0 -I '{}' tag -a Purple {}
}
moveEpub()
{
while read -r line; do
if [ -d "/Users/j/Desktop/Dest/$line" ]; then
if [ -d "/Users/j/Desktop/Dest/$line/EPUB" ]; then
find /Users/j/Desktop/Source/ -maxdepth 1 -iname "*$line*" -and ! -iname ".*$line*" -type f -print0 | xargs -0 -I '{}' mv -i {} /Users/j/Desktop/Dest/"$line"/EPUB/
else
mkdir "/Users/j/Desktop/Dest/$line/EPUB"
find /Users/j/Desktop/Source/ -maxdepth 1 -iname "*$line*" -and ! -iname ".*$line*" -type f -print0 | xargs -0 -I '{}' mv -i {} /Users/j/Desktop/Dest/"$line"/EPUB/
fi
fi
done < "list.txt"
}
moveMobi()
{
while read -r line; do
if [ -d "/Users/j/Desktop/Dest/$line" ]; then
if [ -d "/Users/j/Desktop/Dest/$line/MOBI" ]; then
find /Users/j/Desktop/Source/ -maxdepth 1 -iname "*$line*" -and ! -iname ".*$line*" -type f -print0 | xargs -0 -I '{}' mv -i {} /Users/j/Desktop/Dest/"$line"/MOBI/
else
mkdir "/Users/j/Desktop/Dest/$line/MOBI"
find /Users/j/Desktop/Source/ -maxdepth 1 -iname "*$line*" -and ! -iname ".*$line*" -type f -print0 | xargs -0 -I '{}' mv --interactive {} /Users/j/Desktop/Dest/"$line"/MOBI/
fi
fi
done < "list2.txt"
}
clear
createList
atag
moveEpub
moveMobi
rm list.txt
rm list2.txt
If you want mv -i to interact with the terminal, that means its stdin needs to be attached to that terminal. There are several places, here, where you're overriding stdin.
For instance:
# THIS LOOP OVERRIDES STDIN
while read -r line
...
done <list.txt
...redirects stdin for the entire duration of the loop, so instead of reading from the user, mv reads from list.txt. To change this, use a different file descriptor:
# This loop uses FD 3 for stdin
while read -r line <&3
...
done 3<list.txt
Another place is in calling xargs. Instead of:
# Overrides stdin for xargs and mv to contain output from find
find ... -print0 | xargs -0 -I '{}' mv -i '{}' "$dest"
...use:
# directly executes mv from find, stdin not modified
find ... -exec mv -i '{}' "$dest" ';'
That said, I would suggest ditching list.txt and list2.txt altogether; you simply don't need them; for that matter, you don't need find either.
dest=/Users/j/Desktop/Dest
source=/Users/j/Desktop/Source
moveEpub() {
local -A finished=( ) # WARNING: This requires bash 4.0 or newer.
for name in *.epub; do
prefix=${name%%-*} # remove everything past the first dash
[[ ${finished[$prefix]} ]] && continue # skip if already done with this prefix
finished[$prefix]=1 # set flag to skip other files w/ this prefix
[[ -d $dest/$prefix ]] || continue # skip if no directory exists for this prefix
mkdir -p "$dest/$prefix/EPUB" # create destination if not existing
mv -i "$source"/*"$prefix"* "$dest/$prefix/EPUB"
done
}
You can use built in find action -exec instead of piping to xargs :
find /Users/j/Desktop/Source/ -maxdepth 1 \
-iname "*$line*" -and ! -iname ".*$line*" -type f \
-exec mv -i {} /Users/j/Desktop/Dest/"$line"/EPUB/ \;
I am trying to use find -exec with multiple commands without any success. Does anybody know if commands such as the following are possible?
find *.txt -exec echo "$(tail -1 '{}'),$(ls '{}')" \;
Basically, I am trying to print the last line of each txt file in the current directory and print at the end of the line, a comma followed by the filename.
find accepts multiple -exec portions to the command. For example:
find . -name "*.txt" -exec echo {} \; -exec grep banana {} \;
Note that in this case the second command will only run if the first one returns successfully, as mentioned by #Caleb. If you want both commands to run regardless of their success or failure, you could use this construct:
find . -name "*.txt" \( -exec echo {} \; -o -exec true \; \) -exec grep banana {} \;
find . -type d -exec sh -c "echo -n {}; echo -n ' x '; echo {}" \;
One of the following:
find *.txt -exec awk 'END {print $0 "," FILENAME}' {} \;
find *.txt -exec sh -c 'echo "$(tail -n 1 "$1"),$1"' _ {} \;
find *.txt -exec sh -c 'echo "$(sed -n "\$p" "$1"),$1"' _ {} \;
Another way is like this:
multiple_cmd() {
tail -n1 $1;
ls $1
};
export -f multiple_cmd;
find *.txt -exec bash -c 'multiple_cmd "$0"' {} \;
in one line
multiple_cmd() { tail -1 $1; ls $1 }; export -f multiple_cmd; find *.txt -exec bash -c 'multiple_cmd "$0"' {} \;
"multiple_cmd()" - is a function
"export -f multiple_cmd" - will export it so any other subshell can see it
"find *.txt -exec bash -c 'multiple_cmd "$0"' {} \;" - find that will execute the function on your example
In this way multiple_cmd can be as long and as complex, as you need.
Hope this helps.
There's an easier way:
find ... | while read -r file; do
echo "look at my $file, my $file is amazing";
done
Alternatively:
while read -r file; do
echo "look at my $file, my $file is amazing";
done <<< "$(find ...)"
Extending #Tinker's answer,
In my case, I needed to make a command | command | command inside the -exec to print both the filename and the found text in files containing a certain text.
I was able to do it with:
find . -name config -type f \( -exec grep "bitbucket" {} \; -a -exec echo {} \; \)
the result is:
url = git#bitbucket.org:a/a.git
./a/.git/config
url = git#bitbucket.org:b/b.git
./b/.git/config
url = git#bitbucket.org:c/c.git
./c/.git/config
I don't know if you can do this with find, but an alternate solution would be to create a shell script and to run this with find.
lastline.sh:
echo $(tail -1 $1),$1
Make the script executable
chmod +x lastline.sh
Use find:
find . -name "*.txt" -exec ./lastline.sh {} \;
Thanks to Camilo Martin, I was able to answer a related question:
What I wanted to do was
find ... -exec zcat {} | wc -l \;
which didn't work. However,
find ... | while read -r file; do echo "$file: `zcat $file | wc -l`"; done
does work, so thank you!
1st answer of Denis is the answer to resolve the trouble. But in fact it is no more a find with several commands in only one exec like the title suggest. To answer the one exec with several commands thing we will have to look for something else to resolv. Here is a example:
Keep last 10000 lines of .log files which has been modified in the last 7 days using 1 exec command using severals {} references
1) see what the command will do on which files:
find / -name "*.log" -a -type f -a -mtime -7 -exec sh -c "echo tail -10000 {} \> fictmp; echo cat fictmp \> {} " \;
2) Do it: (note no more "\>" but only ">" this is wanted)
find / -name "*.log" -a -type f -a -mtime -7 -exec sh -c "tail -10000 {} > fictmp; cat fictmp > {} ; rm fictmp" \;
I usually embed the find in a small for loop one liner, where the find is executed in a subcommand with $().
Your command would look like this then:
for f in $(find *.txt); do echo "$(tail -1 $f), $(ls $f)"; done
The good thing is that instead of {} you just use $f and instead of the -exec … you write all your commands between do and ; done.
Not sure what you actually want to do, but maybe something like this?
for f in $(find *.txt); do echo $f; tail -1 $f; ls -l $f; echo; done
should use xargs :)
find *.txt -type f -exec tail -1 {} \; | xargs -ICONSTANT echo $(pwd),CONSTANT
another one (working on osx)
find *.txt -type f -exec echo ,$(PWD) {} + -exec tail -1 {} + | tr ' ' '/'
A find+xargs answer.
The example below finds all .html files and creates a copy with the .BAK extension appended (e.g. 1.html > 1.html.BAK).
Single command with multiple placeholders
find . -iname "*.html" -print0 | xargs -0 -I {} cp -- "{}" "{}.BAK"
Multiple commands with multiple placeholders
find . -iname "*.html" -print0 | xargs -0 -I {} echo "cp -- {} {}.BAK ; echo {} >> /tmp/log.txt" | sh
# if you need to do anything bash-specific then pipe to bash instead of sh
This command will also work with files that start with a hyphen or contain spaces such as -my file.html thanks to parameter quoting and the -- after cp which signals to cp the end of parameters and the beginning of the actual file names.
-print0 pipes the results with null-byte terminators.
for xargs the -I {} parameter defines {} as the placeholder; you can use whichever placeholder you like; -0 indicates that input items are null-separated.
I found this solution (maybe it is already said in a comment, but I could not find any answer with this)
you can execute MULTIPLE COMMANDS in a row using "bash -c"
find . <SOMETHING> -exec bash -c "EXECUTE 1 && EXECUTE 2 ; EXECUTE 3" \;
in your case
find . -name "*.txt" -exec bash -c "tail -1 '{}' && ls '{}'" \;
i tested it with a test file:
[gek#tuffoserver tmp]$ ls *.txt
casualfile.txt
[gek#tuffoserver tmp]$ find . -name "*.txt" -exec bash -c "tail -1 '{}' && ls '{}'" \;
testonline1=some TEXT
./casualfile.txt
Here is my bash script that you can use to find multiple files and then process them all using a command.
Example of usage. This command applies a file linux command to each found file:
./finder.sh file fb2 txt
Finder script:
# Find files and process them using an external command.
# Usage:
# ./finder.sh ./processing_script.sh txt fb2 fb2.zip doc docx
counter=0
find_results=()
for ext in "${#:2}"
do
# #see https://stackoverflow.com/a/54561526/10452175
readarray -d '' ext_results < <(find . -type f -name "*.${ext}" -print0)
for file in "${ext_results[#]}"
do
counter=$((counter+1))
find_results+=("${file}")
echo ${counter}") ${file}"
done
done
countOfResults=$((counter))
echo -e "Found ${countOfResults} files.\n"
echo "Processing..."
counter=0
for file in "${find_results[#]}"
do
counter=$((counter+1))
echo -n ${counter}"/${countOfResults}) "
eval "$1 '${file}'"
done
echo "All files have been processed."
I have files in a dir.
I need to append a new line and the file name at the end of each file.
This should do:
for f in *; do echo >> $f; echo $f >> $f; done
First echo a new-line, then echo the filename.
The >> says "append at the end of the file".
Edit: aioobe's answer has been updated to show the -e flag that I didn't see when I first answered this. Thus I'm now just showing an example which includes a directory and how to eliminate the directory name:
#!/bin/bash
for fn in dir/*
do
shortname=${fn/#*\//}
echo -e "\n$shortname" >> $fn
done
If you want the directory name, take out the shortname=${fn/#*\//} line and replace $shortname with $fn in the echo.
Let xargs do the looping:
# recursive, includes directory name
find -type f -print0 | xargs -0 -I% bash -c 'echo -e "\n%" >> %'
or
# non-recursive, doesn't include directory name
find -maxdepth 1 -type f -exec basename {} \; | xargs -I% bash -c 'echo -e "\n%" >> %'
or
# non-recursive, doesn't include directory name
find -maxdepth 1 -type f -printf "%f\0" | xargs -0 -I% bash -c 'echo -e "\n%" >> %'
or
# recursive, doesn't include directory name
find -type f -print0 | xargs -0 -I% bash -c 'f=%; echo -e "\n${f##*/}" >> %'
Another method using ex (or vim) :
ex -c 'args **/*' -c 'set hidden' -c 'argdo $put =bufname(".")' -c 'wqa'