I am trying to extract all files with a line count greater than x using the following code.
for i in massive*;
do
if [ wc -l $i | cut -d ' ' -f 1 > 50 ]; then
mv $i subset_massive_subcluster_num_gt50/;
fi;
done
However I am getting the following error everytime it goes through the loop:
cut: ]: No such file or directory
-bash: [: missing `]'
Any ideas?
Change this:
for i in massive*;
do
if [ wc -l $i | cut -d ' ' -f 1 > 50 ]; then
mv $i subset_massive_subcluster_num_gt50/;
fi;
done
To this:
for i in massive*;
do
if [ "$(wc -l "$i" | cut -d ' ' -f 1)" -gt 50 ]; then
mv "$i" subset_massive_subcluster_num_gt50/;
fi;
done
Maybe you can try:
for file in massive*
do
[[ $(grep -c '' "$file") > 50 ]] && echo mv "$file" subset_massive_subcluster_num_gt50/
done
the grep -c '' is nicer (and safer) than wc -l | cut
The above is for "dry run". Remove the echo if satisfied.
Related
I am not able to execute this command
/test/filename_*; do $(wc -l "$file" | cut -d' ' -f1) -eq 1 && m $file ; done
It worked till yesterday, somehow this is not working now.
Are you sure you didn't have this command instead:
for file in /test/filename_*; do [ "$(wc -l "$file" | cut -d' ' -f1)" -eq 1 ] && rm "$file" ; done
I'm trying to run a command that should find PHP files that contain "base64_decode" and/or "eval", echo the file name, print the top three lines, if the file contains more than 3 lines, also the bottom 3.
I have the following at the moment:
for file in $(find . -name "*.php" -exec grep -il "base64_decode\|eval" {} \;); do echo $file; head -n 3 $file; if [ wc -l < $file -gt 3 ]; then tail -n 3 $file fi; done | less
This returns the following error:
bash: syntax error near unexpected token `done'
I would to use the following
while read -r file
do
echo ==$file==
head -n 3 "$file"
[[ $(grep -c '' "$file") > 3 ]] && (echo ----last-3-lines--- ; tail -n 3 "$file")
done < <(find . -name \*.php -exec grep -il 'base64_decode\|eval' {} \+)
Using while over the for is better, because the filenames could contain spaces. /probably not in this case, but anyway :)/
using grep -c '' "$file" is sometimes better (when the last line in the file, doesn't contains the \n character (the wc counts the \n characters in the file)
the find with the \+ instead of the \; is more efficient
Problem seems to be here:
if [ wc -l < $file -gt 3 ]; then
Since you need to use command substitution here to make sure wc -l command executes first and then compare the result:
if [[ $(wc -l < "$file") -gt 3 ]]; then
You want to execute your wc, more like:
if [[ $(wc -l < $file) -gt 3 ]]; then
try this:
#!/bin/bash
for file in $(grep -H "base64_decode\|eval" ./*.php | cut -d: -f1);
do
echo $file;
head -n 3 $file;
if [[ $(wc -l < $file) -gt 3 ]];
then
tail -n 3 $file
fi;
done
I tested and seems to work fine.
But, be carefull ... if php has 4 lines, you will see:
line1
line2
line3
line2
line3
line4
EDIT: changed the script above to grep inside files.
cat a.php
asdasd
asd
base64_decode
l
a
and result
./test2.sh
./a.php
asdasd
asd
base64_decode
base64_decode
l
a
to do that
#!/bin/bash
find ./ -name '*.mp3' | while read -r i; do
echo "----------------------------------------"
if [ $(mp3info -x "$i" | grep Audio | awk '{print $2}') < 130 ]
then
read -p "Delete? " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
rm -f "$i" && echo "$i succesfully deleted!"
fi
fi
echo "----------------------------------------"
done
it stops with this output:
Error opening MP3: /Like A Prayer/Madonna - Act Of Contrition.mp3: No such file or directory
It looks like there is an error with the filepath, cause the leading dot is missing.
I think IFS is set to a value with ".".
Also, to compare integers, use [[ ]]:
#!/bin/bash
find ./ -name '*.mp3' | while IFS='' read -r i; do
echo "----------------------------------------"
if [[ $(mp3info -x "$i" | grep Audio | awk '{print $2}') -lt 130 ]]; then
read -p "Delete? " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then
rm -f "$i" && echo "$i succesfully deleted!"
fi
fi
echo "----------------------------------------"
done
Btw I think you have to add your file to the question:
read -p "Delete $i? " -n 1 -r
Here's my script:
while [[ $startTime -le $endTime ]]
do
thisfile=$(find * -type f | xargs grep -l $startDate | xargs grep -l $startTime)
fordestination=`cut -d$ -f2 $thisfile | xargs cut -d ~ -f4`
echo $fordestination
startTime=$(( $startTime + 1 ))
done
I think your cut and grep commands could get stuck. You probably should make sure that their parameters aren't empty, by using the [ -n "$string" ] command to see if $string isn't empty. In your case, if it were empty, it wouldn't add any files to the command that would use it afterwards, meaning that the command would probably wait for input from the command line (ex: if $string is empty and you do grep regex $string, grep wouldn't receive input files from $string and would instead wait for input from the command line). Here's a "complex" version that tries to show where things could go wrong:
while [[ $startTime -le $endTime ]]
do
thisfile=$(find * -type f)
if [ -n "$thisfile" ]; then
thisfile=$(grep -l $startDate $thisfile)
if [ -n "$thisfile" ]; then
thisfile=$(grep -l $startTime $thisfile)
if [ -n "$thisfile" ]; then
thisfile=`cut -d$ -f2 $thisfile`
if [ -n "$thisfile" ]; then
forDestination=`cut -d ~ -f4 $thisfile`
echo $fordestination
fi
fi
fi
fi
startTime=$(( $startTime + 1 ))
done
And here's a simpler version:
while [[ $startTime -le $endTime ]]
do
thisfile=$(grep -Rl $startDate *)
[ -n "$thisfile" ] && thisfile=$(grep -l $startTime $thisfile)
[ -n "$thisfile" ] && thisfile=`cut -d$ -f2 $thisfile`
[ -n "$thisfile" ] && cut -d ~ -f4 $thisfile
startTime=$(( $startTime + 1 ))
done
The "-R" tells grep to search files recursively, and the && tells bash to only execute the command that follows it if the command before it succeeded, and the command before the && is the test command (used in ifs).
Hope this helps =)
This is what I came up with. It works perfectly -- I'm just curious if there's a smaller/crunchier way to do it. (wondering if possible without a loop)
files='file1|file2|file3|file4|file5'
path='/my/path'
found=$(find "$path" -regextype posix-extended -type f -regex ".*\/($files)")
for file in $(echo "$files" | tr '|', ' ')
do
if [[ ! "$found" =~ "$file" ]]
then
echo "$file"
fi
done
You can do this without invoking any external tools:
IFS="|"
for file in $files
do
[ -f "$file" ] || printf "%s\n" "$file"
done
Your code will break if you have file names with whitespace. This is how I would do it, which is a bit more concise.
echo "$files" | tr '|' '\n' | while read file; do
[ -e "$file" ] || echo "$file"
done
You can probably play around with xargs if you want to get rid of the loop all together.
$ eval "ls $path/{${files//|/,}} 2>&1 1>/dev/null | awk '{print \$4}' | tr -d :"
Or use awk
$ echo -n $files | awk -v path=$path -v RS='|' '{printf("! [[ -e %s ]] && echo %s\n", path"/"$0, path"/"$0) | "bash"}'
without whitespace in filenames:
files=(mbox todo watt zoff xorf)
for f in ${files[#]}; do test -f $f || echo $f ; done