I am trying to write a shell script for school that searches your entire home directory for all files with the .java extension. For each such file, list the number of lines in the file along with its location (that is, its full path).
my script looks like
#!/bin/bash
total=0
for currfile in $(find ~ -name "*.java" -print)
do
total=$[total+($(wc -l $currfile| awk '{print $1}'))]
echo -n 'total=' $total
echo -e -n '\r'
done
echo 'total=' $total
when i run it from the konsole i get error
./fileQuest.sh: line 5: total+(): syntax error: operand expected (error token is ")")
I am a novice and cannot figure out what the error is telling me. Any help would be appreciated
total+()
This is the expression that's being evaluated inside of $[...]. Notice that the parentheses are empty. There should be a number there. It indicates that the $(wc | awk) bit is yielding an empty string.
total=$[total+($(wc -l $currfile| awk '{print $1}'))]
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If that part is blank then you get:
total=$[total+()]
Note that wc can handle multiple file names natively. You don't need to write your own loop. You could use find -exec to call it directly instead.
find ~ -name "*.java" -exec wc {} +
Related
Sorry, I am new to bash. I have a fairly simple script that is something like the below:
#!/bin/bash
grep file.txt | awk '{print $2}' | while read -r line; do
log "$line"
log "find path/to/csv -name $line"
command=$(find path/to/csv -name $line)
log "$command"
done
}
What I am trying to do is grab the second field for every line in the file.txt file, pipe it to the line variable and then use that to find any csv file in the /path/to/csv directory with a name that matches any name in file.txt. This isn't working so I have stripped it down to be as simple as I can.
When I use the $line variable in the log command, the log file brings the second field for every line as expected. However when I use the $line variable with the find command, it is not working and the log shows a blank output.
20200702-16h-12m-33s filename*.csv
20200702-16h-12m-33s find path/to/csv -name filename*.csv
20200702-16h-12m-33s
Just wondering if I need to anything to my $line variable within the find command? I have tried:
$line
"$line"
"*line*"
${line}
{$line}
Any help would be massively appreciated
Don't store the command in a variable.
That is, don't do this: command=$(find path/to/csv -name $line)
Instead just store the "find" arguments in an array and then the run find command with it.
So, in your case, it would be something like this:
args=(path/to/csv -name "$line")
find "${args[#]}"
I have 1000 files with following names:
something-345-something.txt
something-5468-something.txt
something-100-something.txt
something-6200-something.txt
and a lot more...
And I have one txt file, with only numbers in it. f.e:
1000
500
5468
6200
699
usw...
Now I would like to move all files, which have a number in their filenames which is in my txt file.
So in my example above the following files should be moved only:
something-5468-something.txt
something-6200-something.txt
Is there an easy way to achieve this?
What about on the fly moving files by doing this:
for i in `cat you-file.txt`; do
find . -iname "*-$i-*" -exec mv '{}' /target/dir \;
; done
For every line in your text file, the find command will try to find only does matching the pattern *-$i-* (something-6200-something.txt) and move it to your target dir.
Naive implementation: for file in $(ls); do grep $(echo -n $file | sed -nr 's/[^-]*-([0-9]+).*/\1/p') my-one-txt.txt && mv $file /tmp/somewhere; done
In English: For every file in output of ls: parse number part of filename with sed and grep for it in your text file. grep returns a non-zero exit code if nothing is found, so mv is in evaluated in that case.
Script file named move (executable):
#!/bin/bash
TARGETDIR="$1"
FILES=`find . -type f` # build list of files
while read n # read numbers from standard input
do # n contains a number => filter list of files by that number:
echo "$FILES" | grep "\-$n-" | while read f
do # move file that passed the filter because its name matches n:
mv "$f" "$TARGETDIR"
done
done
Use it like this:
cd directory-with-files
./move target-directory < number-list.txt
Here's a crazy bit of bash hackery
shopt -s extglob nullglob
mv -t /target/dir *-#($(paste -sd "|" numbers.txt))-*
That uses paste to join all the lines in your numbers file with pipe characters, then uses bash extended pattern matching to find the files matching any one of the numbers.
I assume mv from GNU coreutils for the -t option.
I need to find just extension of all files in directory (if there are 2 same extensions, its just one). I already have it. But the output of my script is like
test.txt
test2.txt
hello.iso
bay.fds
hellllu.pdf
Im using grep -e -e '.' and it just highlight DOTs
And i need just these extensions give in one variable like txt,iso,fds,pdf
Is there anyone who could help? I already had it one time but i had it on array. Today I found out It's has to work on dash too.
You can use find with awk to get all unique extensions:
find . -type f -name '?*.?*' -print0 |
awk -F. -v RS='\0' '!seen[$NF]++{print $NF}'
can be done with find as well, but I think this is easier
for f in *.*; do echo "${f##*.}"; done | sort -u
if you want to assign a comma separated list of the unique extensions, you can follow this
ext=$(for f in *.*; do echo "${f##*.}"; done | sort -u | paste -sd,)
echo $ext
csv,pdf,txt
alternatively with ls
ls -1 *.* | rev | cut -d. -f1 | rev | sort -u | paste -sd,
rev/rev is required if you have more than one dot in the filename, assuming the extension is after the last dot. For any other directory simply change the part *.* to dirpath/*.* in all scripts.
I'm not sure I understand your comment. If you don't assign to a variable, by default it will print to the output. If you want to pass directory name as a variable to a script, put the code into a script file and replace dirpath with $1, assuming that will be your first argument to the script
#!/bin/bash
# print unique extension in the directory passed as an argument, i.e.
ls -1 "$1"/*.* ...
if you have sub directories with extensions above scripts include them as well, to limit only to file types replace ls .. with
find . -maxdepth 1 -type f -name "*.*" | ...
I'm trying to list all PDF files under a given directory $1 (and its subdirectories), get the number of pages in each file and calculate two numbers using the pagecount. My script used to work, but only on filenames that don't contain spaces and only in one directory that is only filled with PDF files. I've modified it a bit already (using quotes around variables and such), but now I'm a bit stuck.
The problem I'm having is that, as it is now, the script only processes the first file found by find . -name '*.pdf'. How would I go about processing the rest?
#!/bin/bash
wd=`pwd`
pppl=0.03 #euro
pppnl=0.033 #eruo
cd $1
for entry in "`find . -name '*.pdf'`"
do
filename="$(basename "$entry")"
pagecount=`pdfinfo "$filename" | grep Pages | sed 's/[^0-9]*//'`
pricel=`echo "$pagecount * $pppl" | bc`
pricenl=`echo "$pagecount * $pppnl" | bc`
echo -e "$filename\t\t$pagecount\t$pricel\t$pricenl"
done
cd "$wd"
The problem with using find in a for loop, is that if you don't quote the command, the filenames with spaces will be split, and if you do quote the command, then the entire results will be parsed in a single iteration.
The workaround is to use a while loop instead, like this:
find . -name '*.pdf' -print0 | while IFS= read -r -d '' entry
do
....
done
Read this article for more discussion: http://mywiki.wooledge.org/ParsingLs
It's a bad idea to use word splitting. Use a while loop instead.
while read -r entry
do
filename=$(basename "$entry")
pagecount=$(pdfinfo "$filename" | grep Pages | sed 's/[^0-9]*//')
pricel=$(echo "$pagecount * $pppl" | bc)
pricenl=$(echo "$pagecount * $pppnl" | bc)
echo -e "$filename\t\t$pagecount\t$pricel\t$pricenl"
done < <(exec find . -name '*.pdf')
Also prefer $() over backticks when possible. You also don't need to place around "" variables or command substitutions when they are being used for assignment.
filename=$(basename "$entry")
As well could simply be just
filename=${entry##*/}
Why does this work
for myfile in `find . -name "R*VER" -mtime +1`
do
SHELLVAR=`grep ^err $myfile || echo "No error"`
ECHO $SHELLVAR
done
and outputs
No error
err ->BIST Login Fail 3922 err
No error
err ->IR Remote Key 1 3310 err
But this does not
for myfile in `find . -name "R*VER" -mtime +1`
do
SHELLVAR=`grep ^err $myfile || echo "No error"`
awk -v awkvar=${SHELLVAR} '{print awkvar}'
done
and outputs
awk: cmd. line:1: fatal: cannot open file `{print awkvar}' for reading (No such file or directory)
What am I missing?
Does $SHELLVAR contain a space? If so, your awk script is getting misparsed, and the {print awkvar} is being assumed to be a file name and not the actual AWK program.
You also have a problem where both your for loop and the awk program are both slurping STDIN. In fact, your for loop would only be executed once since the AWK will read in all the STDIN until it finishes. In the end, you'll get the same line over and over, and then your program will stop running as the awk awaits for more STDIN.
I think you want to do something like this...
find . -name "R*VER" -mtime +1 | while read myfile
do
echo "$SHELLVAR" | awk '{print $0}'
done
This way, your echo command feeds into your awk which will prevent awk from reading from the find statement.
As an aside, you're better off doing this:
find . -name "R*VER" -mtime +1 | while read myfile
do
...
done
Rather than this:
for myfile in `find . -name "R*VER" -mtime +1`
do
...
done
This is for several reasons:
The command line buffer could overflow, and you'll lose file names, but you'll never see an error.
The find command in the second example must first complete before the for loop can start to execute. In the first example, the find feeds into the while loop.
ADDENDUM
Now that I saw just-my-correct-opinion's answer, I realize what you've really done wrong: You forgot the file name in the awk command:
find . -name "R*VER" -mtime +1 | while read myfile
do
SHELLVAR=`grep ^err $myfile || echo "No error"`
awk -v awkvar=${SHELLVAR} '{print awkvar}' $myfile
done
Now, the question is what exactly are you doing with the awk. You're not printing anything except the value of $SHELVAR for each and every line in the file. That's probably not what you want to do. In fact, why not simply do this:
find . -name "R*VER" -mtime +1 | while read myfile
do
SHELLVAR=$(grep -q "^err" $myfile")
if [ "x$SHELLVAR" != "x" ]
then
echo "$SHELLVAR"
fi
done
That way, you print out $SHELLVAR, but only if $SHELLVAR is empty.
Or, you can use awk to print out only those lines that match your regex:
find . -name "R*VER" -mtime +1 | while read myfile
do
awk '/^err/ {print $0}' $myfile
done
What you're trying to do is possible but ... a bit quirky. Here's an alternative for you:
for f in $(find . -name "R*VER" -mtime +1)
do
echo "$f"
awk 'BEGIN{ec=0} /^err/{ec+=1;print} END{if(ec==0){print "No error"}}' "$f"
done
This way you don't have to worry about grep, don't have to worry about shell variables and can keep your logic all in one place in one language.
Note that this code is only partially tested since I don't have your data files, but the testing I did worked fine.
If you'd like you can even go a step farther and write the whole thing in awk. It is, after all, a full, general-purpose programming language (with, IMO, a far cleaner syntax than bash). This way you avoid the need for find, grep and bash entirely. I won't be writing that script, however. You'll have to pick up the awk man page and read up on file I/O yourself.
You need to quote $SHELLVAR, to prevent the shell from splitting it.
awk -v awkvar="${SHELLVAR}" '{print awkvar}'
You have already got alternative solutions. Regardless of that, I just want to answer your question, ie the thing that you are missing:
awk -v awkvar=${SHELLVAR} '{print awkvar}'
Here awk is seeking to read the input from STDIN. And that's the problem. Awk seeks to read input from STDIN, unless you specify input file(s). The given commands to awk are executed for each RECORD in the input (and by default a record is a line). See man awk to read more on awk.
But here is a hack, if you want it to proceed without any input:
awk -v awkvar=${SHELLVAR} 'BEGIN{print awkvar}'
BEGIN block is executed as soon as the awk is called. And if awk doesn't find any other block except BEGIN, it executes those BEGIN blocks and exits.
I hope you got the problem behind the error, and a quick solution for that as well.