I am trying to find instances of the word package in line 171 of any file in a certain directory. What is the best way to do this?
You can do, from the directory you want to check, recursively:
find . -type f -exec bash -c '[[ $(sed -n '171p' "$1") =~ package ]] && echo "$1"' _ {} +
this will show you the filenames that contain package in their 171-th line.
Non-recursively:
find . -maxdepth 1 -type f -exec bash -c '[[ $(sed -n '171p' "$1") =~ package ]] && echo "$1"' _ {} +
Example:
I am looking for bar:
$ cat foo
foo
bar
$ find . -type f -exec bash -c '[[ $(sed -n '2p' "$1") =~ bar ]] && echo "$1"' _ {} +
./foo
#This will search second line of all the files and grep over them.
#Pass search pattern as argument.
pattern=$1
for file in *
do
cat $file |sed -n '2p' |grep $pattern
done
Related
We have few csv and xml files in following formats
String_YYYY_MM_DD_HH_MM.csv
String_YYYY_MM_DD_HH_MM.xml
String.xml
String.csv
Examples:
Reference_Categories_2021_02_24_17_14.csv
CD_CategoryTree_2021_02_24_17_14.csv
New_Categories.xml
Mobile_Footnote_2021_03_05_16_21.csv
Campaign_Version_2018_09_24_20_00.xml
Campaign_new.csv
Now we have to remove _YYYY_MM_DD_HH_MM from filenames so result will be
Reference_Categories.csv
CD_CategoryTree.csv
New_Categories.xml
Mobile_Footnote.csv
Campaign_Version.xml
Campaign_new.csv
Any idea how to do that in bash?
In pure bash:
pat='_[0-9][0-9][0-9][0-9]_[0-9][0-9]_[0-9][0-9]_[0-9][0-9]_[0-9][0-9]'
for f in *$pat*.{csv,xml}; do echo mv "$f" "${f/$pat}"; done
Delete the echo if the output looks fine.
With bash Something like:
shopt -s nullglob
for f in *.{xml,csv}; do
ext="${f##*.}"
[[ "${f%%_[0-9]*}" = *.#(xml|csv) ]] && continue
echo mv -v -- "$f" "${f%%_[0-9]*}.$ext"
done
With the =~ operator and BASH_REMATCH
shopt -s nullglob
regexp='^(.{1,})(_[[:digit:]]{4}_[[:digit:]]{2}_[[:digit:]]{2}_[[:digit:]]{2}_[[:digit:]]{2})([.].*)$'
for f in *.{xml,csv}; do
[[ "$f" =~ $regexp ]] &&
echo mv -v -- "$f" "${BASH_REMATCH[1]}${BASH_REMATCH[-1]}"
done
Remove the echo if you're satisfied with the output.
Using bash, find, and awk:
Use find to find files with .csv or .xml suffix in the current directory. Pipe the find output to awk and create the mv commands that are output and passed to bash.
bash < <(find * -type f \( -name '*.csv' -o -name '*.xml' \) | awk '{orig=$0; gsub(/_[0-9]{4}_[0-9]{2}_[0-9]{2}_[0-9]{2}_[0-9]{2}/,""); print "mv "orig" "$0}')
Directory contents before:
find * -type f
CD_CategoryTree_2021_02_24_17_14.csv
Campaign_Version_2018_09_24_20_00.xml
Campaign_new.csv
Mobile_Footnote_2021_03_05_16_21.csv
New_Categories.xml
Reference_Categories_2021_02_24_17_14.csv
Directory contents after:
find * -type f
CD_CategoryTree.csv
Campaign_Version.xml
Campaign_new.csv
Mobile_Footnote.csv
New_Categories.xml
Reference_Categories.csv
How to move all files which content begins with foo to another folder with command line ?
I tried this to echo filenames when matching:
for f in *.txt; do if [ $(head -c5 $f) = "foo" ]; then echo $f; fi; done;
but I'm often getting this error:
-bash: [: too much arguments
Use a Perl one-liner in combination with find and xargs, like so:
echo foo > 1.txt
echo "foo\nbar" > 2.txt
echo "bar\nfoo" > 3.txt
mkdir foodir
find . -maxdepth 1 -name '[123].txt' -exec perl -lne 'print $ARGV if /^foo/; last;' {} \; | xargs -I{} mv {} foodir
find foodir -type f
# foodir/2.txt
# foodir/1.txt
Find with awk
find . -type f -exec awk -F \/ "NR < 6 && /foo/ { fnd=1 } END { if (fnd==1) { split(FILENAME,arr,\"/\");print \"mv -f \"FILENAME\" newdir/\"arr[length(arr)] } }" '{}' \;
Use awk to process the first 5 lines (NR < 6). Search for foo and if it exists, set a fnd variable to 1. At the end, if fnd is 1, print the mv command, using split to get the filename without the directories. Check that everything looks as expected and then run with:
find . -type f -exec awk -F \/ "NR < 6 && /foo/ { fnd=1 } END { if (fnd==1) { split(FILENAME,arr,\"/\");print \"mv -f \"FILENAME\" newdir/\"arr[length(arr)] } }" '{}' \; | bash
I am having trouble looping and searching. It seems that the loop is not waiting for the find to finish. What am I doing wrong?
I made a loop the reads a file line by line. I then want to use that "name" to search a directory looking to see if a folder has that name. If it exists copy it to a drive.
#!/bin/bash
DIRFIND="$2"
DIRCOPY="$3"
if [ -d $DIRFIND ]; then
while IFS='' read -r line || [[ -n "$line" ]]; do
echo "$line"
FILE=`find "$DIRFIND" -type d -name "$line"`
if [ -n "$FILE" ]; then
echo "Found $FILE"
cp -a "$FILE" "$DIRCOPY"
else
echo "$line not found."
fi
done < "$1"
else
echo "No such file or directory"
fi
Have you tried xargs...
Proposed Solution
cat filenamelist | xargs -n1 -I {} find . -type d -name {} -print | xargs -n1 -I {} mv {} .
what the above does is pipe a list of filenames into find (one at a time), when found find prints the name and passes to xarg which moves the file...
Expansion
file = yogo
yogo -> | xargs -n1 -I yogo find . -type d -name yogo -print | xargs -n1 -I {} mv ./<path>/yogo .
I hope the above helps, note that xargs has the advantage that you do not run out of command line buffer.
I'm trying to find all files in a file structure above a certain file size, list them, then delete them. What I currently have looks like this:
filesToDelete=$(find $find $1 -type f -size +$2k -ls)
if [ -n "$filesToDelete" ];then
echo "Deleting files..."
echo $filesToDelete
$filesToDelete | xargs rm
else
echo "no files to delete"
fi
Everything works, except the $filesToDelete | xargs rm, obviously. Is there a way to use pipe on a variable? Or is there another way I could do this? My google-fu didn't really find anything, so any help would be appreciated.
Edit: Thanks for the information everyone. I will post the working code here now for anyone else stumbling upon this question later:
if [ $(find $1 -type f -size +$2k | wc -l) -ge 1 ]; then
find $1 -type f -size +$2k -exec sh -c 'f={}; echo "deleting file $f"; rm $f' {} \;
else
echo "no files above" $2 "kb found"
fi
As already pointed out, you don't need piping a var in this case. But just in case you needed it in some other situation, you can use
xargs rm <<< $filesToDelete
or, more portably
echo $filesToDelete | xargs rm
Beware of spaces in file names.
To also output the value together with piping it, use tee with process substitution:
echo "$x" | tee >( xargs rm )
You can directly use -exec to perform an action on the files that were found in find:
find $1 -type f -size +$2k -exec rm {} \;
The -exec trick makes find execute the command given for each one of the matches found. To refer the match itself we have to use {} \;.
If you want to perform more than one action, -exec sh -c "..." makes it. For example, here you can both print the name of the files are about to be removed... and remove them. Note the f={} thingy to store the name of the file, so that it can be used later on in echo and rm:
find $1 -type f -size +$2k -exec sh -c 'f={}; echo "removing $f"; rm $f' {} \;
In case you want to print a message if no matches were found, you can use wc -l to count the number of matches (if any) and do an if / else condition with it:
if [ $(find $1 -type f -size +$2k | wc -l) -ge 1 ]; then
find $1 -type f -size +$2k -exec rm {} \;
else
echo "no matches found"
fi
wc is a command that does word count (see man wc for more info). Doing wc -l counts the number of lines. So command | wc -l counts the number of lines returned by command.
Then we use the if [ $(command | wc -l) -ge 1 ] check, which does an integer comparison: if the value is greater or equal to 1, then do what follows; otherwise, do what is in else.
Buuuut the previous approach was using find twice, which is a bit inefficient. As -exec sh -c is opening a sub-shell, we cannot rely on a variable to keep track of the number of files opened. Why? Because a sub-shell cannot assign values to its parent shell.
Instead, let's store the files that were deleted into a file, and then count it:
find . -name "*.txt" -exec sh -c 'f={}; echo "$f" >> /tmp/findtest; rm $f' {} \;
if [ -s /tmp/findtest ]; then #check if the file is empty
echo "file has $(wc -l < /tmp/findtest) lines"
# you can also `cat /tmp/findtest` here to show the deleted files
else
echo "no matches"
fi
Note that you can cat /tmp/findtest to see the deleted files, or also use echo "$f" alone (without redirection) to indicate while removing. rm /tmp/findtest is also an option, to do once the process is finished.
You don't need to do all this. You can directly use find command to get the files over a particular size limit and delete it using xargs.
This should work:
#!/bin/bash
if [ $(find $1 -type f -size +$2k | wc -l) -eq 0 ]; then
echo "No Files to delete"
else
echo "Deleting the following files"
find $1 -size +$2 -exec ls {} \+
find $1 -size +$2 -exec ls {} \+ | xargs rm -f
echo "Done"
fi
I need to write a bash script that will iterate through the contents of a directory (including subdirectories) and perform the following replacements:
replace 'foo' in any file names with 'bar'
replace 'foo' in the contents of any files with 'bar'
So far all I've got is
find . -name '*' -exec {} \;
:-)
With RH rename:
find -f \( -exec sed -i s/foo/bar/g \; , -name \*foo\* -exec rename foo bar {} \; \)
find "$#" -depth -exec sed -i -e s/foo/bar/g {} \; , -name '*foo*' -print0 |
while read -d '' file; do
base=$(basename "$file")
mv "$file" "$(dirname "$file")/${base//foo/bar}"
done
UPDATED: 1632 EST
Now handles whitespace but 'while read item' never terminates. Better,
but still not right. Will keep
working on this.
aj#mmdev0:~/foo_to_bar$ cat script.sh
#!/bin/bash
dirty=true
while ${dirty}
do
find ./ -name "*" |sed -s 's/ /\ /g'|while read item
do
if [[ ${item} == "./script.sh" ]]
then
continue
fi
echo "working on: ${item}"
if [[ ${item} == *foo* ]]
then
rename 's/foo/bar/' "${item}"
dirty=true
break
fi
if [[ ! -d ${item} ]]
then
cat "${item}" |sed -e 's/foo/bar/g' > "${item}".sed; mv "${item}".sed "${item}"
fi
dirty=false
done
done
#!/bin/bash
function RecurseDirs
{
oldIFS=$IFS
IFS=$'\n'
for f in *
do
if [[ -f "${f}" ]]; then
newf=`echo "${f}" | sed -e 's/foo/bar/g'`
sed -e 's/foo/bar/g' < "${f}" > "${newf}"
fi
if [[ -d "${f}" && "${f}" != '.' && "${f}" != '..' && ! -L "${f}" ]]; then
cd "${f}"
RecurseDirs .
cd ..
fi
done
IFS=$oldIFS
}
RecurseDirs .
bash 4.0
#!/bin/bash
shopt -s globstar
path="/path"
cd $path
for file in **
do
if [ -d "$file" ] && [[ "$file" =~ ".*foo.*" ]];then
echo mv "$file" "${file//foo/bar}"
elif [ -f "$file" ];then
while read -r line
do
case "$line" in
*foo*) line="${line//foo/bar}";;
esac
echo "$line"
done < "$file" > temp
echo mv temp "$file"
fi
done
remove the 'echo' to commit changes
for f in `tree -fi | grep foo`; do sed -i -e 's/foo/bar/g' $f ; done
Yet another find-exec solution:
find . -type f -exec bash -c '
path="{}";
dirName="${path%/*}";
baseName="${path##*/}";
nbaseName="${baseName/foo/bar}";
#nbaseName="${baseName//foo/bar}";
# cf. http://www.bash-hackers.org/wiki/doku.php?id=howto:edit-ed
ed -s "${path}" <<< $'H\ng/foo/s/foo/bar/g\nwq';
#sed -i "" -e 's/foo/bar/g' "${path}"; # alternative for large files
exec mv -iv "{}" "${dirName}/${nbaseName}"
' \;
correction to find-exec approach by gregb (adding quotes):
# compare
bash -c '
echo $'a\nb\nc'
'
bash -c '
echo $'"'a\nb\nc'"'
'
# therefore we need
find . -type f -exec bash -c '
...
ed -s "${path}" <<< $'"'H\ng/foo/s/foo/bar/g\nwq'"';
...
' \;