Grep $value `grep $value2 `<command>`` - Nested grep? - bash

I'm a complete noob at awk/sed so forgive me if I'm missing something obvious here.
Basically I'm trying to do a nested grep, i.e. something akin to:
grep $value `exim -Mvh $(`exim -bpru | grep $eximID | more`)`
Breakdown:
grep $value IN COMMAND
--> exim -Mvh (print exim mail headers) FROM RESULTS OF
---> exim -bpru | grep $eximID | more
$value is the string I'm looking for
$eximID is the string I'm looking for within exim -bpru (list all exim thingies)
No idea if what I'm trying to accomplish would be easier with awk/sed hence the question really.
I tried to make that as legible as possible but nested nesting is hard yo
Edit
Tada! My script is now workings thanks to you guys! Here it is, unfinished, but working:
#!/usr/bin/bash
echo "Enter the email address you want to search for + compare sender info via exim IDs."
read searchTarget
echo "Enter the target domain the email is coming from."
read searchDomain
#domanList is array for list of exim IDs needed
domainList=($(exim -bpru | grep "$searchDomain" | awk '{ print $3 }'))
for i in "${domainList[#]}"
do
echo "$(exim -Mvh $i | grep $searchTarget)"
#echo "$(grep $searchTarget $(exim -Mvh $i))"
done

grep $value `exim -Mvh $(`exim -bpru | grep $eximID | more`)`
This isn't right. The backticks (`command`) and $(command) do the same thing, it's just an alternative syntax. The advantage of using $() is that it's better nestable, so it's a good habit to always use that.
So, let's fix this, we now end up with:
grep "$value" "$(exim -Mvh "$(exim -bpru | grep "$eximID")")" | more
I relocated the more command, for what I think will be obvious reasons. more just paginates data for the user, feeding the output of more to something else almost never makes sense.
I've also quoted the variables, this is also a good habit, because otherwise things will break when there are certain characters in your variable (most common is the a space).
I can't test if this gives you the output you want, if it doesn't, then update your answer with a few lines of example data, and the expected output.

If you're going to do it with back-quotes (not recommended; it is hard work), then you have to write:
grep $value `exim -Mvh $(\`exim -bpru | grep $eximID\`)`
(where I've removed the more since when used like that it behaves like cat and there's no point in using cat at the end of the commands like that either).
It would be more sane to use the $(…) notation throughout:
grep $value $(exim -Mvh $( $(exim -bpru | grep $eximID)))
And it seems more plausible that you don't need quite that many sets of indirection and this is what you're really after:
grep $value $(exim -Mvh $(exim -bpru | grep $eximID))
You should look at:
Why didn't back quotes in a shell script help me cd to a directory?
What is the benefit of using $(…) instead of back ticks in shell scripts?
Why does \$ reduce to $ inside backquotes [though not inside $(…)]?
and no doubt there are other related questions too.

Related

Bash grep, awk o sed to reverse find

I am creating a script to look for commonly used patterns in a password.Although I have security policies in the hosting panel, servers have been outdated due to incompatibilities.
Example, into the file words.txt, i put in there, the word test, when i execute grep -c test123 words.txt. When I look for that pattern I need it to find it but I think that with the command grep it won't work for me.
Script:
EMAILPASS=`/root/info.sh -c usera | grep #`
for PAR in ${EMAILPASS} ; do
EMAIL=$(echo "${PAR}" | grep # | cut -f1 -d:)
PASS=$(echo "${PAR}" | cut -d: -f 2)
PASS="${PASS,,}"
FINDSTRING=$(grep -ic "${PASS}" /root/words.txt)
echo -e ""
echo -e "Validating password ${EMAIL}"
echo -e ""
if [ $FINDSTRING -ge 1 ] ; then
echo "Insecre"
else
echo "Secure"
fi
the current output of the command is as follows
# grep -c test123 /root/words.txt
0
I think grep is not good for what I need, maybe someone can help me.
I could also use awk or sed but I can't find an option to help me.
Regardsm
Reverse your application.
echo test123 | grep -f words.txt
Each line of the text file will be used as a pattern to test against the input.
edit
Apparently you actually do want to see if the whole password is an actual word, rather than just checking to see if it's based on a dictionary word. That's considerably less secure, but easy enough to do. The logic you have will not report test123 as insecure unless the whole passwword is an exact match for a word in the dictionary.
You said you were putting test in the dictionary and using test123 as the password, so I assumed you were looking for passwords based on dictionary words, which was the structure I suggested above. Will include as commented alternate lines below.
Also, since you're doing a case insensitive search, why bother to downcase the password?
declare -l pass # set as always lowecase
would do it, but there's no need.
Likewise, unless you are using it again later, it isn't necessary to put everything into a variable first, such as the grep results. Try to remove anything not needed -- less is more.
Finally, since we aren't catching the grep output in a variable and testing that, I threw it away with -q. All we need to see is whether it found anything, and the return code, checked by the if, tells us that.
/root/info.sh -c usera | grep # | # only lines with at signs
while IFS="$IFS:" read email pass # parse on read with IFS
do printf "\n%s\n\n" "Validating password for '$email'"
if grep -qi "$pass" /root/words.txt # exact search (-q = quiet)
#if grep -qif /root/words.txt <<< "$pass" # 'based on' search
then echo "Insecure"
else echo "Secure" # well....
fi
done
I think a better paradigm might be to just report the problematic ones and be silent for those that seem ok, but that's up to you.
Questions?

Regex - Pattern Matching in Shell

I am trying to match a pattern and extract the values that comes after it. I have used below regex pattern matchching, it it dint help me. No values got extracted as I got blank value when I echoed it.
Someone let me know what mistake I made.
Sample regex:
class="remove_link_style">Site Issue - Please check</a></td><td>
Working</td><td>
<ahref="/0051043899"class="remove_link_style">
patten used: text=$(echo "class="remove_link_style">Site Issue - Please check</a></td><td>Working</td><td><ahref="/0051043899"class="remove_link_style">" | grep -o --perl-regexp "(?class="remove_link_style")[a-zA-Z0-9_]+"")
I also wanted to extract the string that comes after class="remove_link_style" but before </a></td><td>
I think you would find a lot of references and advice not to parse XML with bash tools like grep/sed/awk . With this context, I would advise using any of the parsing tools like http://xmlsoft.org/xmllint.html or http://xmlstar.sourceforge.net/doc/xmlstarlet.txt . But if you'd like to quickly extract the contents, you can combine grep and cut as below.
echo 'class="remove_link_style">GB|Trekkinn-UK|Manualcrawlrequest|1</a></td><td>WorkInProgress</td><td><ahref="/0051043899"class="remove_link_style">' | grep -Eo 'style"[^<>]*>[^<>]+' | cut -f2 -d">"
This prints out:
GB|Trekkinn-UK|Manualcrawlrequest|1
WorkInProgress
EDIT : As per OP's ask, store the output into an array.
If you need the output to be stored in an array, you need to set the IFS since you have white spaces in your elements.
IFS=$'\n'
result=($(echo 'class="remove_link_style">Site Issue - Please check</a></td><td>Working</td><td><ahref="/0051043899"class="remove_link_style">' | grep -Eo 'style"[^<>]*>[^<>]+' | cut -f2 -d">"))
unset IFS
for i in "${result[#]}"; do echo $i; done
Site Issue - Please check
Working

BASH Palindrome Checker

This is my first time posting on here so bear with me please.
I received a bash assignment but my professor is completely unhelpful and so are his notes.
Our assignment is to filter and print out palindromes from a file. In this case, the directory is:
/usr/share/dict/words
The word lengths range from 3 to 45 and are supposed to only filter lowercase letters (the dictionary given has characters and uppercases, as well as lowercase letters). i.e. "-dkas-das" so something like "q-evvavve-q" may count as a palindrome but i shouldn't be getting that as a proper result.
Anyways, I can get it to filter out x amount of words and return (not filtering only lowercase though).
grep "^...$" /usr/share/dict/words |
grep "\(.\).\1"
And I can use subsequent lines for 5 letter words and 7 and so on:
grep "^.....$" /usr/share/dict/words |
grep "\(.\)\(.\).\2\1"
But the prof does not want that. We are supposed to use a loop. I get the concept but I don't know the syntax, and like I said, the notes are very unhelpful.
What I tried was setting variables x=... and y=.. and in a while loop, having x=$x$y but that didn't work (syntax error) and neither did x+=..
Any help is appreciated. Even getting my non-lowercase letters filtered out.
Thanks!
EDIT:
If you're providing a solution or a hint to a solution, the simplest method is prefered.
Preferably one that uses 2 grep statements and a loop.
Thanks again.
Like this:
for word in `grep -E '^[a-z]{3,45}$' /usr/share/dict/words`;
do [ $word == `echo $word | rev` ] && echo $word;
done;
Output using my dictionary:
aha
bib
bob
boob
...
wow
Update
As pointed out in the comments, reading in most of the dictionary into a variable in the for loop might not be the most efficient, and risks triggering errors in some shells. Here's an updated version:
grep -E '^[a-z]{3,45}$' /usr/share/dict/words | while read -r word;
do [ $word == `echo $word | rev` ] && echo $word;
done;
Why use grep? Bash will happily do that for you:
#!/bin/bash
is_pal() {
local w=$1
while (( ${#w} > 1 )); do
[[ ${w:0:1} = ${w: -1} ]] || return 1
w=${w:1:-1}
done
}
while read word; do
is_pal "$word" && echo "$word"
done
Save this as banana, chmod +x banana and enjoy:
./banana < /usr/share/dict/words
If you only want to keep the words with at least three characters:
grep ... /usr/share/dict/words | ./banana
If you only want to keep the words that only contain lowercase and have at least three letters:
grep '^[[:lower:]]\{3,\}$' /usr/share/dict/words | ./banana
The multiple greps are wasteful. You can simply do
grep -E '^([a-z])[a-z]\1$' /usr/share/dict/words
in one fell swoop, and similarly, put the expressions on grep's standard input like this:
echo '^([a-z])[a-z]\1$
^([a-z])([a-z])\2\1$
^([a-z])([a-z])[a-z]\2\1$' | grep -E -f - /usr/share/dict/words
However, regular grep does not permit backreferences beyond \9. With grep -P you can use double-digit backreferences, too.
The following script constructs the entire expression in a loop. Unfortunately, grep -P does not allow for the -f option, so we build a big thumpin' variable to hold the pattern. Then we can actually also simplify to a single pattern of the form ^(.)(?:.|(.)(?:.|(.)....\3)?\2?\1$, except we use [a-z] instead of . to restrict to just lowercase.
head=''
tail=''
for i in $(seq 1 22); do
head="$head([a-z])(?:[a-z]|"
tail="\\$i${tail:+)?}$tail"
done
grep -P "^${head%|})?$tail$" /usr/share/dict/words
The single grep should be a lot faster than individually invoking grep 22 or 43 times on the large input file. If you want to sort by length, just add that as a filter at the end of the pipeline; it should still be way faster than multiple passes over the entire dictionary.
The expression ${tail+:)?} evaluates to a closing parenthesis and question mark only when tail is non-empty, which is a convenient way to force the \1 back-reference to be non-optional. Somewhat similarly, ${head%|} trims the final alternation operator from the ultimate value of $head.
Ok here is something to get you started:
I suggest to use the plan you have above, just generate the number of "." using a for loop.
This question will explain how to make a for loop from 3 to 45:
How do I iterate over a range of numbers defined by variables in Bash?
for i in {3..45};
do
* put your code above here *
done
Now you just need to figure out how to make "i" number of dots "." in your first grep and you are done.
Also, look into sed, it can nuke the non-lowercase answers for you..
Another solution that uses a Perl-compatible regular expressions (PCRE) with recursion, heavily inspired by this answer:
grep -P '^(?:([a-z])(?=[a-z]*(\1(?(2)\2))$))++[a-z]?\2?$' /usr/share/dict/words

shell script : how to replace file name

I want to change some file names with full path similar to this:
/home/guest/test
⟶ /home/guest/.test.log
I tried the command below but it cannot search "/"
string="/home/guest/test"
substring="/"
replacement="/."
echo ${string/%substring/replacement}.log
You can do something like:
for file in /home/guest/*; do
name=${file##*/}
path=${file%/*}
mv "$file" "$path"'/.'"$name"'.log'
done
Created using bash on a mac, so it might work with whatever shell you are using...
string="/home/guest/test"
echo $string | sed 's/\/\([^\/]\{0,\}\)$/\/.\1.log/'
Using simple shell string replacement wasn't going to work since I know of no way you can target the last occurrence of the / sign as the only replacement.
Update:
Actually I came to think of a alternative way if you know that it is always "/two/directories/in"
string="/home/guest/test"
firstpartofstring=$(echo $string | cut -d\/ -f1-3)
lastpartofstring=$(echo $string | cut -d\/ -f4)
echo ${firstpartofstring}/.${lastpartofstring}.log

BashScripting: Reading out a specific variable

my question is actually rather easy, but I suck at bash scripting and google was no help either. So here is the problem:
I have an executable that writes me a few variables to stdout. Something like that:
MrFoo:~$ ./someExec
Variable1=5
Another_Weird_Variable=12
VARIABLENAME=42
What I want to do now is to read in a specific one of these variables (I already know its name), store the value and use it to give it as an argument to another executable.
So, a simple call like
./function2 5 // which comes from ./function2 Variable1 from above
I hope you understand the problem and can help me with it
With awk you can do something like this (this is for passing value of 1st variable)
./someExec | awk -F= 'NR==1{system("./function2 " $2)}'
or
awk -F= 'NR==1{system("./function2 " $2)}' <(./someExec)
Easiest way to go is probably to use a combination of shell and perl or ruby. I'll go with perl since it's what I cut my teeth on. :)
someExec.sh
#!/bin/bash
echo Variable1=5
echo Another_Weird_Variable=12
echo VARIABLENAME=42
my_shell_script.sh
#!/bin/bash
myVariable=`./someExec | perl -wlne 'print $1 if /Variable1=(.*)/'`
echo "Now call ./function2 $myVariable"
[EDIT]
Or awk, as Jaypal pointed out 58 seconds before I posted my answer. :) Basically, there are a lot of good solutions. Most importantly, though, make sure you handle both security and error cases properly. In both of the solutions so far, we're assuming that someExec will provide guaranteed well-formed and innocuous output. But, consider if someExec were compromised and instead provided output like:
./someExec
5 ; rm -rf / # Uh oh...
You can use awk like this:
./function2 $(./someExec | awk -F "=" '/Variable1/{print $2}')
which is equivalent to:
./function2 5
If you can make sure someExec's output is safe you can use eval.
eval $(./someExec)
./function2 $Variable1
You can use this very simple and straight forward way:
./exp1.sh | grep "Variable1" | awk -F "=" '{print $2}'
If you want to use only one variable from the file use the below
eval $(grep 'Variable1' ./someExec )
./function2 $Variable1
And, if you want to use all the variables of a file, use
eval $(./someExec)
./function2 $<FILE_VARIBALE_NAME>

Resources