I'm new to Bash, and I'm trying to find files in a certain set of folders. I want to create a txt report for image files in each /check/ folder.
Here's what I've been working with:
# Find images
for f in */check/ ; do
find ./ -iname "*jpg*" -o -iname "*png*" > find_images.txt
echo "finished $f"
done
I can't figure out how to only look at subfolders named "check", and I also want to pass the variable so that I get separate text files named after the parent folders. Any suggestions?
You're close, but you're not using $f which contains the folder's name:
# Find images
for f in */check/ ; do
# Removing front-slashes from $f to use in log name
# http://mywiki.wooledge.org/BashGuide/Parameters#Parameter_Expansion
log_f="${f//\//_}"
# Only search inside $f, saving results to find_images_[foldername].txt
find "$f" -iname "*jpg*" -o -iname "*png*" > "find_images_${log_f}.txt"
echo "finished $f"
done
Use grep command and pipe it with find command
find . | grep check
The find command supports searching for directories (folders), e.g.
find . -name "check" -type d
You could use these results to then look for the files you want. The variable $f will be the name of the folder, so use that in the inner find command. Then if you want separate output files each time through the loop, use a variable in the filename. The $f variable will have slashes in the content, so you probably don't want to use that in the name of your output file. In my example, I use a counter to make sure each output file has a unique name.
count=1
for f in `find . -name "check" -type d` ; do
find $f -iname "*jpg*" -o -iname "*png*" > find_images_$count_.txt
count=$((count+1))
done
Related
new here, learning bash for first time.
I'm trying to iterate over files named "list.txt" placed in subfolders, manipulate and create a new files, under the same subfolder. The nest could be like this:
inventory/product_names1/list.txt
inventory/product_names2/list.txt
As product_names is completly random, I would like to iterate over all list.txt files with unix cms like sed/grep/cut and create a new file, under the same random product_names folders.
for f in $( find . -name 'list.txt'); do for list in $f; do cat $f | cut -d']' -f2- > "$f/new_file.txt" ; done ; done
I can access files into the nest using find command. How can I redirect output in the right subfolder if the product_names is random?
inventory/product_names1/list.txt
inventory/product_names1/new_file.txt
inventory/product_names2/list.txt
inventory/product_names2/new_file.txt
This script is intended to work in the root folder, pointing and working with entime path "inventory". $f access to inventory/product_names1/list.txt but I need the output in inventory/product_names1. How can I redirect correctly if I don't have the right value/variable?
You can either use parameter expansion to remove the file name from the path, or you can iterate over all the directories and only work on them if they contain the list.txt file.
#!/bin/bash
for list in inventory/*/list.txt ; do
new=${list%/*}/new_list.txt
echo "$list" "$new"
done
# OR
for dir in inventory/* ; do
if [[ -f $dir/list.txt ]] ; then
echo "$dir"/list.txt "$dir"/new_list.txt
fi
done
find can not only find files but also execute commands when a file is found:
find . -type f -name 'list.txt' -execdir sh -c 'cut -d"]" -f2 list.txt > new_file.txt' \;
Explanations:
-type f condition added to skip directories named list.txt. If some of your list.txt files can be symbolic links and you want to consider them too, use -type f,l with GNU find. With other find you may need to use \(-type f -o -type l\).
-execdir runs the command in the directory where the file was found.
By default find does not print when -execdir is used. If you need it add the -print command:
find . -type f -name 'list.txt' -execdir sh -c 'cut -d"]" -f2 list.txt > new_file.txt' \; -print
I'm new(ish) to using Bash and I'm trying to figure out how to combine a few different things into one script.
I'm looking for file transfers that were interrupted. These folders contain image files (either jpgs or pngs), but are missing another specific file (finished.txt).
Here is what I'm using to find folders with images (from here):
for f in */incoming/ ; do
log_f="${f//\//}"
echo "searching $f"
find "$f" -iname "*jpg*" -o -iname "*png*" > "/output/${log_f}.txt"
echo "$f finished"
done
Then, I'm running this command to find folders that are missing the finished.txt file (from here):
find -mindepth 2 -maxdepth 2 -type d '!' -exec test -e "{}/finished.txt" ';' -print
Is there a way to combine them so I have a list of folders which have jpg or png files, but don't have finished.txt? Also, If I want to add -mtime, where do I put that?
Alternatively, if there's a better/faster way to do this, I'm interested in that too.
Thanks!
From the first pass when you get the files with jpg/png you can get the directory by using dirname. The list of directories can then be used for iterating over and looking for finished.txt file. On finding you can skip the directory if not print it out.
Something as below should do the needful
for i in `find "$f" -iname "*jpg*" -o -iname "*png*" -exec dirname {} \;`
do
ls $i | grep finished >/dev/null
if [ $? -eq 1 ]; then
echo $i
fi
done
Add " | sort | uniq" at the end of find command to perhaps remove the duplicates. Something like
find "$f" -iname "jpg" -o -iname "png" -exec dirname {} \; | sort | uniq
The scenario is that I want to convert all of my music files from .mp3 to .ogg. They are in a folder called "Music". In this folder there are folders and files. The files are .mp3s. The directories may contain .mp3s or directories which further contain .mp3s or directories, and so on. This is because some artists have albums which have parts and some do not, etc.
I want to write a script that converts each file using avconv.
Basically, what I am going to do is manually cd into every directory and run the following:
for file in $(ls); do avconv -i $file `echo \`basename $file .mp3\`.ogg`; done
This successfully gets me what I want. However, this is not great as I have a lot of folders, and manually going into each of them and executing this is slow.
My question, then, is how do I write a script that runs this in any directory that has .mp3s, and then goes into any subdirectory it finds and recursively calls itself? My intuition tells me to use Perl or Python because of the complex nature of this.
Thanks for any suggestions!
I'm not familiar with avconv but assuming your command is:
avconv -i inputname outputname
And you want to convert all inputname.mp3 to inputname.ogg in their original directories below Music, then the following should work in bash:
#!/bin/bash
while read -r fname; do
avconv -i "$fname" "${fname%.mp3}.ogg"
done < <(find /path/to/Music -type f -name "*.mp3")
Note: this does not remove the original .mp3, and the space between < < is required. Also note, for file in $(ls) is filled with potential for errors.
You can do it with bash in one liner:
First you find all files (of type file (-type f) ) that match next pattern "*.mp3". To read each one you use 'while' and invoke avconf.
For exchange extension I prefer 'sed' command, that keep folder so you don't need the 'cd' command.
Notice that you must put quotes on $FN variable because it can contain spaces.
find -type f -iname "*.mp3" | while read "FN" ; do avconf -i "$FN" $(echo "$FN" | sed 's/\.mp3/\.ogg/g') ; done
find <music-folder> -type f -name '*.mp3' | \
xargs -I{} bash -c 'mp3="$0"; ogg="${mp3%.mp3}.ogg"; avconv -i "$mp3" "$ogg";' {}
This should survive in cases of "weird" filenames with spaces, quotes and other strange symbols within.
You can list directories with absolute paths and recursively cd into every directory using find $PWD -type d syntax:
Just inside from Music directory run:
for d in $(find $PWD -type d)
do
cd $d
for file in $(find . -maxdepth 1 -type f)
do
echo $file
avconv -i $file `echo \`basename $file .mp3\`.ogg`
done
done
I'm struggling with this task:
Write a script that takes as input a directory (path) name and a
filename base (such as ".", "*.txt", etc). The script shall search the
given directory tree, find all files matching the given filename, and
bundle them into a single file. Executing the given file as a script
should return the original files.
Can anyone help me?
First i tried to do the find part like this:
#!/bin/bash
filebase=$2
path=$1
find $path \( -name $base \)
Then i found this code for bundle, but I dont know how to combine them.
for i in $#; do
echo "echo unpacking file $i"
echo "cat > $i <<EOF"
cat $i
echo "EOF"
done
Going on tripleee's comment you can use shar to generate a self extracting archive.
You can take the output of find and pass it through to shar in order to generate the archive.
#!/bin/bash
path="$1"
filebase="$2"
archive="$3"
find "$path" -type f -name "$filebase" | xargs shar > "$archive"
The -type f option passed to find will restrict the search to files (i.e. excludes directories), which seems to be a required limitation.
If the above script is called archive_script.sh, and is executable, then you can call it as below for example:
./archive_script.sh /etc '*.txt' etc-text.shar
This will create a shar archive of all the .txt files in /etc.
I want to write a bash script that (recursively) processes all files of a certain type.
I know I can get the matching file list by using find thusly:
find . -name "*.ext"
I want to use this in a script:
recursively obatin list of files with a given extension
obtain the full file pathname
pass the full pathname to another script
Check the return code from the script. If non zero, log the name of the file that could not be processed.
My first attempt looks (pseudocode) like this:
ROOT_DIR = ~/work/projects
cd $ROOT_DIR
for f in `find . -name "*.ext"`
do
#need to lop off leading './' from filename, but I havent worked out how to use
#cut yet
newname = `echo $f | cut -c 3
filename = "$ROOT_DIR/$newname"
retcode = ./some_other_script $filename
if $retcode ne 0
logError("Failed to process file: $filename")
done
This is my first attempt at writing a bash script, so the snippet above is not likely to run. Hopefully though, the logic of what I'm trying to do is clear enough, and someone can show how to join the dots and convert the pseudocode above into a working script.
I am running on Ubuntu
find . -name '*.ext' \( -exec ./some_other_script "$PWD"/{} \; -o -print \)
Using | while read to iterate over file names is fine as long as there are no files with carrier return to be processed:
find . -name '*.ext' | while IFS=$'\n' read -r FILE; do
process "$(readlink -f "$FILE")" || echo "error processing: $FILE"
done