At first I want to assure that I was looking for the answer for a few hours by now and I've read a lot similar questions but none of them solved my problem.
Straight to the point now:
I have two scripts in bash: one is "tool" that do some stuff for me and the second one is main "for user" script.
I want to pass to the tool script various patterns (like "[A-Za-z0-9]*" or "&")
And here is some code:
#!/bin/bash
SET() {
wz1=`./PREP2.sh $1 $2 '[0-9A-Za-z]\*'`
wz2=`./PREP2.sh $1 $2 '&'`
echo $wz1
echo $wz2
}
SET $1 $2
Tool script is actually working if I declare patterns inside like this:
line='[0-9A-Za-z]*'
But when I pass the same pattern with
'\*'
I can't get rid of "\" without interpreting "*" as "show all files in catalog".
I've been trying to use eval inside the tool like this:
eval echo '$3'
But it didn't work.
Full code follow.
User script:
#!/bin/bash
SET() {
#echo '[0-9A-Za-z]*'
wzor1=$(./PREP2.sh "$1" "$2" '[0-9A-Za-z]*')
wzor2=`./PREP2.sh $1 $2 '&'`
echo $wzor1
echo $wzor2
}
SET $1 $2 $4
Tool code
#!/bin/bash
PREP2() {
#echo "$3"
wzor="`./PREP.sh $1 $2 | tee linie.txt`"
#tmp="`echo $wzor | sed 's/,/,%/'`"
#echo $tmp;
./ZAMIEN_WSZYSTKIE_WYSTAPIENIA.sh linie.txt , #%
#tmp="`echo $wzor | tr '#' '\n x' | tee linie.txt`"
tmp="`tr '#' '\n x' < linie.txt | tee linie.txt`"
llini=`echo "$tmp" | wc -l`
#echo liczba lini $llini
i=1
wzor=""
while [ $i -le $llini ];
do
linia="`eval sed -n -e $i\p linie.txt | cut -d '%' -f2`"
if [ -z "$linia" ];then
#linia='[0-9A-Za-z]*'
linia=`eval '$3'`
#echo $linia
fi
if [ $i -ne 1 ];then
#echo "kolejna wartosc"
wzor=$wzor\,$linia
else
#echo "pierwsza wartosc"
wzor=$linia
fi
i=`expr $i + 1`
done
echo $wzor
#wynik="`grep -v "$wzor" $1`"
#echo "$wynik" > $1
#echo $nowy_wpis >> $1
}
eval echo "$3"
#PREP2 $1 $2 $3
And just to clear things up I don't actually go into procedure because I know it is working weird because of the arguments I put into it.
Quotes, quotes, quotes and more quotes. And prefer $() to backticks, that saves some quoting problems.
#!/bin/bash
SET() {
wz1=$(./PREP2.sh "$1" "$2" '[0-9A-Za-z]*')
wz2=$(./PREP2.sh "$1" "$2" '&')
echo "$wz1"
echo "$wz2"
}
SET "$1" "$2"
(BTW: it's unusual to have function names all uppercase. That's usually for environment variables.)
Related
I'm trying to iterate over a folder, running a grep on each file, and putting them into separate files, tagged with a .res extension. Here's what I have so far....
#!/bin/bash
directory=$(pwd)
searchterms="searchterms.txt"
extension=".end"
usage() {
echo "usage: fmat [[[-f file ] [-d directory ] [-e ext]] | [-h]]"
echo " file - text file containing a return-delimited list of materials"
echo " directory - directory to process"
echo " ext - file extension of files to process"
echo ""
}
while [ "$1" != "" ]; do
case $1 in
-d | --directory ) shift
directory=$1
;;
-f | --file ) shift
searchterms=$1
;;
-e | --extension ) shift
extension=$1
;;
-h | --help ) usage
exit
;;
* ) usage
exit 1
esac
shift
done
if [ ! -d "$directory" ]; then
echo "Sorry, the directory '$directory' does not exist"
exit 1
fi
if [ ! -f "$searchterms" ]; then
echo "Sorry, the searchterms file '$searchterms' does not exist"
exit 1
fi
echo "Searching '$directory' ..."
for file in "${directory}/*"; do
printf "File: %s\n" ${file}
[ -e "$file" ] || continue
printf "%s\n" ${file}
if [ ${file: -3} == ${extension} ]; then
printf "%s will be processed\n" ${file}
#
# lots of processing here
#
fi
done
I know that it's down to my poor understanding of of globbing... but I can't get the test on the extension to work.
Essentially, I want to be able to specify a source directory, a file with search terms, and an extension to search for.
NOW, I realise there may be quicker ways to do this, e.g.
grep -f searchterms.txt *.end > allchanges.end.res
but I may have other processing I need to do to the files, and I want to save them into separate files: so bing.end, bong.end, would be grep'ed into bing.end.res, bong.end.res .
Please let me know, just how stupid I'm being ;-)
Just for completeness sake, here's the last part, working, thanks to #chepner and #Gordon Davisson :
echo "Searching '$directory' ..."
for file in "${directory}"/*; do
[ -e "$file" ] || continue
# show which files will be processed
if [[ $file = *.${extension#.} ]]; then
printf "Processing %s \n" "$file"
head -n 1 "${file}" > "${file}.res"
grep -f $searchterms "${file}" >> "${file}.res"
fi
done
You just need to leave the * out of the quotes, so that it isn't treated as a literal *:
for file in "${directory}"/*; do
Unlike most languages, the quotes don't define a string (as everything in bash is already a string: it's the only data type). They simply escape each character inside the quotes. "foo" is exactly the same as \f\o\o, which (because escaping most characters doesn't really have any effect) is the same as foo. Quoted or not, all characters not separated by word-splitting characters are part of the same word.
http://shellcheck.net will catch this, although not with the most useful error message. (It will also catch the other parameter expansions that you did not quote but should.)
I am new to bash scripting and I have to create this script that takes 3 directories as arguments and copies in the third one all the files in the first one that are NOT in the second one.
I did it like this:
#!/bin/bash
if [ -d $1 && -d $2 && -d $3 ]; then
for FILE in [ ls $1 ]; do
if ! [ find $2 -name $FILE ]; then
cp $FILE $3
done
else echo "Error: one or more directories are not present"
fi
The error I get when I try to execute it is: "line 7: syntax error near unexpected token `done' "
I don't really know how to make it work!
Also even if I'm using #!/bin/bash I still have to explicitly call bash when trying to execute, otherwise it says that executing is not permitted, anybody knows why?
Thanks in advance :)
Couple of suggestions :
No harm double quoting variables
cp "$FILE" "$3" # prevents wordsplitting, helps you filenames with spaces
for statement fails for the fundamental reason -bad syntax- it should've been:
for FILE in ls "$1";
But then, never parse ls output. Check [ this ].
for FILE in ls "$1"; #drastic
Instead of the for-loop in step2 use a find-while-read combination:
find "$1" -type f -print0 | while read -rd'' filename #-type f for files
do
#something with $filename
done
Use lowercase variable names for your script as uppercase variables are reserved for the system. Check [this].
Use tools like [ shellcheck ] to improve script quality.
Edit
Since you have mentioned the input directories contain only files, my alternative approach would be
[[ -d "$1" && -d "$2" && -d "$3" ]] && for filename in "$1"/*
do
[ ! -e "$2/${filename##*/}" ] && cp "$filename" "$3"
done
If you are baffled by ${filename##*/} check [ shell parameter expansion ].
Sidenote: In linux, although discouraged it not uncommon to have non-standard filenames like file name.
Courtesy: #chepner & #mklement0 for their comments that greatly improved this answer :)
Your script:
if ...; then
for ...; do
if ...; then
...
done
else
...
fi
Fixed structure:
if ...; then
for ...; do
if ...; then
...
fi # <-- missing
done
else
...
fi
If you want the script executable, then make it so:
$ chmod +x script.sh
Notice that you also have other problems in you script. It is better written as
dir1="$1"
dir2="$2"
dir3="$3"
for f in "$dir1"/*; do
if [ ! -f "$dir2/$(basename "$f")" ]; then
cp "$f" "$dir3"
fi
done
this is not totally correct:
for FILE in $(ls $1); do
< whatever you do here >
done
There is a big problem with that loop if in that folder there is a filename like this: 'I am a filename with spaces.txt'.
Instead of that loop try this:
for FILE in "$1"/*; do
echo "$FILE"
done
Also you have to close every if statement with fi.
Another thing, if you are using BASH ( #!/usr/bin/env bash ), it is highly recommended to use double brackets in your test conditions:
if [[ test ]]; then
...
fi
For example:
$ a='foo bar'
$ if [[ $a == 'foo bar' ]]; then
> echo "it's ok"
> fi
it's ok
However, this:
$ if [ $a == 'foo bar' ]; then
> echo "it's ok";
> fi
bash: [: too many arguments
You've forgot fi after the innermost if.
Additionally, neither square brackets nor find do work this way. This one does what your script (as it is now) is intended to on my PC:
#!/bin/bash
if [[ -d "$1" && -d "$2" && -d "$3" ]] ; then
ls -1 "$1" | while read FILE ; do
ls "$2/$FILE" >/dev/null 2>&1 || cp "$1/$FILE" "$3"
done
else echo "Error: one or more directories are not present"
fi
Note that after a single run, when $2 and $3 refer to different directories, those files are still not present in $2, so next time you run the script they will be copied once more despite they already are present in $3.
I want to write a script that take 1 command line argument( a directory) and then prompt for 2 number, it will then print out any file(each in a different line) that has the size between that 2 number, this is my script
echo -n "Enter the first number: "
read a
echo -n "Enter the second, bigger number: "
read b
if
[ $b -lt $a ]
then
echo 'The first number must be smaller'
else
echo The files in $1 that are between $a and $b bytes are the following
echo
for var in 'ls $1'
do
if
[ -f $var ]
then
size='ls -l $var | '{ print $5 }''
if
[ $size -le $b && $size -ge $a ]
then
echo $var is $size bytes
fi
fi
done
fi
The problem is after I enter the numbers, it will print out "The files..." and then nothing else. Also, I use Vi to edit it,but the color of last three lines is not quite right(the color should match the first "fi" but it not). Can anyone show me what was wrong? Thank you.
Your immediate problem is that you used single quotes where you wanted command substitution. However, this is the wrong way to iterate over files. You should use pattern matching instead. Your for loop should read
for var in $1/*
do
if [ -f "$var" ]
then
# Check 'man stat' for the correct format string on your system
size=$(stat +%s "$var")
if [ $size -le $b ] && [ $size -ge $a ]
then
echo $var is $size bytes
fi
fi
done
There are a couple of problems here, but the one that I think has you stuck is that the single-quote character (') is used in a couple of places where the backtick character (`) should be used. This is a subtle typographical distinction, so sometimes people that haven't encountered it before don't pick up on the distinction. On my keyboard, you get a backtick character by hitting the key just to the left of the number 1, it is paired with the tilde (~), but your keyboard may be different.
The backtick allows you to assign the output of a command to a variable, for example:
my_variable=`ls - l` # <- uses backtick, assigns output of 'ls -l' command to $my_variable
#As opposed to:
my_variable='ls -l' # <- uses single-quote, makes $my_variable equal to the text "ls -l"
Note, this will also fix your vi issue if you replace the correct single-quotes w/backticks.
As stated by others, use a shebang and use backticks for your commands. Other things that were wrong, ls -l $var | '{ print $5 }' should be ls -l "$1$var" | awk '{ print $5 }' (awk command was missing), and when testing the files you should use the full path to the file like [ -f "$1$var" ] since the user may not be in the same directory as the path they provide as an argument to the script. Another problem is [ $size -le $b && $size -ge $a ]. You can't use the && operator that way, instead use [ $size -le $b ] && [ $size -ge $a ].
These are all the changes I made to your code. Hope it works for you.
echo -n "Enter the first number: "
read a
echo -n "Enter the second, bigger number: "
read b
if [ $b -lt $a ]
then
echo 'The first number must be smaller'
else
echo The files in "$1" that are between "$a" and "$b" bytes are the following
echo
for var in `ls "$1"`
do
if [ -f $1$var ]
then
size=`ls -l "$1$var" | awk '{ print $5 }'`
if [ $size -le $b ] && [ $size -ge $a ]
then
echo "$var" is "$size" bytes
fi
fi
done
fi
I wrote a bash script that uses sed to create a command that filters text by keywords and then colors the output. I did a lot of trial and error, and googling for techniques. I know this can be improved. Can anybody offer suggestions?
I'd like to be able to use this like tail too - filter and color tail output in real time. Any thoughts?
Thanks in advance!
function multigrep(){
#THIS WORKS - Recreate this, using input parameters
#sed -En '/(App)|(Spe)/p' ./flashlog.txt;
filename="/Users/stevewarren/Library/Preferences/Macromedia/Flash\ Player/Logs/flashlog.txt";
paramString="";
for element in "$#"
do
#echo $element;
paramString="$paramString($element)|";
done
#TRIM FINAL | OFF PARAMSTRING
paramString=${paramString:0:${#paramString}-1};
#CREATE SED EXPRESSION - '/($1)|($2)|(...)/p'
paramString="'/$paramString/p'";
#CREATE SED FUNCTION, CALL ON FILE
paramString="sed -En $paramString ./flashlog.txt"
echo $paramString;
echo "${txtbld}$(tput setaf 7)" > ./flashlog_output.txt;
eval $paramString >> ./flashlog_output.txt;
echo >> ./flashlog_output.txt;
#cat ./flashlog_output.txt;
cat ./flashlog_output.txt | while read LINE
do
[[ $1 && ${1-x} ]] &&
if grep -q $1 <<<$LINE; then
echo "$(tput setaf 3)$LINE"
fi
[[ $2 && ${2-x} ]] &&
if grep -q $2 <<<$LINE; then
echo "$(tput setaf 7)$LINE"
fi
[[ $3 && ${3-x} ]] &&
if grep -q $3 <<<$LINE; then
echo "$(tput setaf 6)$LINE"
fi
done
}
Do you know about colortail? I'm using it too. You can configure the colorization using regexes in the config file. The output will look like: (yeah! :)
I'm writing a script that asks the user for several options and then, via a series of echo statements, creates and writes to a separate script file. That script will also be dependent on at least one command line argument when executed.
Given the original statement
if [ "` echo $1 | egrep ^[[:digit:]]+$`" = "" ]
that determines if the first argument ($1) is an integer, how can I include that in the echo statement to be written to the new file while maintaining the command line argument access?
I tried to escape the double quotes and dollar signs like
echo "if [ \"` echo \$1 | egrep ^[[:digit:]]+\$`\" = \"\" ]" >> generatePanos
but that just resulted in
if [ "" = "" ]
However, echo "\$1" results in $1 being printed in the file.
Since you are using bash, you can use bash's builtin regex:
if [[ $1 =~ ^[[:digit:]]+$ ]]; then
...
fi
Even without bash's builtin regex, there is no need for the test command ( [ ), or echo:
if grep -q -E '^[[:digit:]]+$' <<< "$1"; then
...
fi
If this must also work on shells other than bash, then you can keep the echo:
if echo "$1" | grep -q -E '^[[:digit:]]+$'; then
...
fi
The last two work because if tests the exit status of a command. The grep command returns non-zero if a match is not found.
I'm not entirely certain what you mean, but here are two things you can try:
If you're trying to print the entire line as-is (including the $1), use single-quotes to tell echo to not interpret anything:
$ echo 'if [ "` echo $1 | egrep ^[[:digit:]]+$`" = "" ]'
if [ "` echo $1 | egrep ^[[:digit:]]+$`" = "" ]
If you're trying to print the entire line but substitute in the current value for $1:
$ echo "if [ \"\` echo $1 | egrep ^[[:digit:]]+\$\`\" = \"\" ]"
if [ "` echo <<some value>> | egrep ^[[:digit:]]+$`" = "" ]
If you're trying to substitute the entire portion of the command that's in backticks (evaluated using the current value of $1), it's probably best to use an intermediate variable:
temp=$(echo $1 | egrep ^[[:digit:]]+$)
echo "if [ \"$temp\" = \"\" ]"
put a back slash before every offending character: echo if [ \"` echo $1 \| egrep ^[[:digit:]]+$`\" = \"\" ]
echo "\$1" should do exactly what you said it did.
To echo the CONTENTS of '1' then echo $1 (without the backslash).
When using variables in bash scripts, it's often good practice to double quote them (echo "$1", func_name "$1", etc) or to escape them ( echo "${1}" ).
As to the first part, are you wanting to echo the entire 'if' statement to the file? That's what it looks like. If not, then you should do this:
if [ conditions ]; then echo "$1" >> $filename