Escaping Shebang in grep - shell

in a shell script, i'm trying to find out if another file is a shell script. i'm doing that by grepping the shebang line. but my grep statement doesn't work:
if [[ $($(cat $file) | grep '^#! /bin' | wc -l) -gt 0 ]]
then
echo 'file is a script!'
else
echo "no script"
fi
i always get the error "bash: #!: command not found". i tried several things to escape the shebang but that didn't work.
maybe you can help me with that? :)
cheers,
narf

I would suggest that you change your condition to this:
if grep -q '^#! */bin' "$file"
The -q option to grep is useful in this case as it tells grep to produce no output, exiting successfully if the pattern is matched. This can be used with if directly; there's no need to wrap everything in a test [[ (and especially no need for a useless use of cat).
I also modified your pattern slightly so that the space between #! and /bin is optional.
It's worth noting that this will produce false positives in cases where the match is on a different line of the file, or when another shebang is used. You could work around the first issue by piping head -n 1 to grep, so that only the first line would be checked:
if head -n 1 "$file" | grep -q '^#! */bin'
If you are searching for a known list of shebangs, e.g. /bin/sh and /bin/bash, you could change the pattern to something like ^#! */bin/\(sh\|bash\).

Related

How to make a bash script fail based on grep regex?

I have the following bash script.
The problem I try to solve is really easy. When a commit in Git no starts with gh-1234 ... it should fail.
What is wrong with this bash script?
commit_regex='(gh-[0-9]+|merge)'
error_msg="Aborting commit. Your commit message is missing either a Github Issue ('gh-1111') or 'Merge'."
if ! grep -q "$commit_regex" <<< "$1"; then
echo "$error_msg" >&2
exit 1
fi
As mentioned in the comments, you need to do grep -E "$commit_regex"
From the man grep page-
-E, --extended-
Interpret PATTERN as an extended regular expression (ERE, see below). (-E is specified by POSIX.)
That should solve your problem since it forces grep to expand the variable.
Also another way to achieve the same without grep :
if [[ $git_message=~ $commit_regex ]]
then
echo "$error_msg"
exit 1;
fi
Also : If the commit it "gh-1234" is mandated to be at the start of the message , you should add ^ in your regex.

Read file for value, loop until value = $foo?

I'm writing a shell script that greps for $foo then counts the number of occurrences then runs a command. Each time that command is run, there is one less instance of $foo in that file. Uncertain on how to continuously read that file and reduce the value in the variable I set.
$count= `grep -o $foo /some/file |wc -w`
until [ $count -eq 0 ]
do
some_command_that_deletes_foo_in_file
done
However I realize that $count is set once at runtime and is not updated. What I want is $count to be updated to the current count in /some/file while the script is looping through /some/file until there is 0 instances of the phrase I'm grepping for. Uncertain to what the best approach is.
Unless you have additional code that you haven't showed us that depends on $count, you don't actually need to count occurrences; you just need to know whether the string appears in the file. For that, you can write:
while grep -q -- "$foo" /some/file ; do
some_command_that_deletes_foo_in_file
done
(using the fact that grep returns success when it finds the value, and failure when it does not, and using the -q flag to suppress its console output).
You could add the grep command inside the loop:
count=$(grep -o "$foo" /some/file |wc -w)
until (( count == 0 ))
do
some_command_that_deletes_foo_in_file
count=$(grep -o "$foo" /some/file |wc -w)
done
You simply want to delete the string "$foo"? Use sed:
sed "/$foo/d" /some/file > /some/other/file
The sed command is an editor. The /$foo/ is taking a regular expression (whatever the value of $foo), finding it in the file. The d tells it to delete the line.
sed doesn't usually do an in place edit. You usually have to write to another file and then to a move. However, some sed commands may have such a parameter. You can check your manage.
Second Try
I think it must take some action or perform some processing or something, and one of its effects is that one of the $foos is gone. (But I could be wrong.) – ruakh yesterday
This is what I get answering these questions at night.
You can take advantage of the fact that grep returns true (i.e. exit value = 0) if it can find your regular expression and false (i.e. exit value not equal to 0) otherwise:
while grep -o -q "$foo" "/some/file"
do
some_command_that_deletes_foo_in_file
done
The -q tells grep not to output anything. Instead, the exit value of grep will be true if the string is found and false if it isn't.
Read your manpage on grep. Some grep commands don't have the -q parameter. In that case, you'l need to redirect both STDOUT and STDERR to /dev/null.
Another tact may be to do your count of the number of lines, and then use that as a counter:
count=$(grep -o "$foo" "/some/file" | wc -w) # $(...) is the preferred syntax over `...`
for loop in {1..$count}
do
some_command_that_deletes_foo_in_file
done
The advantage is that you only grep through the file once (which maybe a long operation). However, your count maybe incorrect if $foo is on more than one line.
A few notes:
The $(...) is preferred over backticks
When you set a variable, you don't use the dollar sign in front of that variable. Note I have count= and not $count=.
Watch spaces. count= $(...) is wrong because there's a space after the equals sign.

grep behavior in cli and script differs

I've got a web server with a bunch of domains that have different name servers and I'm trying to clean up the mess. I'm trying to get a list of the domains and their name servers. I've got this simple script written and it almost works:
#!/bin/bash
for f in `cat mydomains.txt`
do
echo $f " " >> mydns.txt
dig ns $f | grep '^$f' | cut -d $'\t' -f 5 >> mydns.txt
echo "" >> mydns.txt
done
As of right now, all this does is echo $f " " >> mydns.txt.
If I take the dig line and substitute what $f should be in the command line, I get the expected results. However, I get nothing in my script. I know that the $f variable is populated because it echoes $f in the previous line. Why doesn't it work in the script ?
Did you mean to grep for '^$f'? It should be '^'"$f", i.e., without the ticks around the variable. It won't be expanded that way.
Single quotes prevent the shell from expanding the variable. You should use double quotes:
dig ns "$f" | grep "^$f" | ...
Note that I also used double quotes around $f in the dig call. Quoting your variables is good practice; see this other SO question for details.
It's possible that it's a Windows-style/Unix-style newline issue. It might help to convert your newlines to \n instead of \r\n.
Have you tried the dos2unix suggestion given here: grep fails inside bash script but works on command line?

overwrite a file then append

I have a loop in my script that will append a list of email address's to a file "$CRN". If this script is executed again, it will append to this old list. I want it to overwrite with the new list rather then appending to the old list. I can submit my whole script if needed. I know I could test if "$CRN" exists then remove file, but I'm interested in some other suggestions? Thanks.
for arg in "$#"; do
if ls /students | grep -q "$arg"; then
echo "${arg}#mail.ccsf.edu">>$CRN
((students++))
elif ls /users | grep -q "$arg$"; then
echo "${arg}#ccsf.edu">>$CRN
((faculty++))
fi
Better do this :
CRN="/path/to/file"
:> "$CRN"
for arg; do
if printf '%s\n' /students/* | grep -q "$arg"; then
echo "${arg}#mail.ccsf.edu" >> "$CRN"
((students++))
elif printf '%s\n'/users/* | grep -q "${arg}$"; then
echo "${arg}#ccsf.edu" >> "$CRN"
((faculty++))
fi
done
don't parse ls output ! use bash glob instead. ls is a tool for interactively looking at file information. Its output is formatted for humans and will cause bugs in scripts. Use globs or find instead. Understand why: http://mywiki.wooledge.org/ParsingLs
"Double quote" every expansion, and anything that could contain a special character, eg. "$var", "$#", "${array[#]}", "$(command)". See http://mywiki.wooledge.org/Quotes http://mywiki.wooledge.org/Arguments and http://wiki.bash-hackers.org/syntax/words
take care to false positives like arg=foo and glob : foobar, that will match. You need grep -qw then if you want word boundaries. UP2U

Passing multiple arguments to a UNIX shell script

I have the following (bash) shell script, that I would ideally use to kill multiple processes by name.
#!/bin/bash
kill `ps -A | grep $* | awk '{ print $1 }'`
However, while this script works is one argument is passed:
end chrome
(the name of the script is end)
it does not work if more than one argument is passed:
$end chrome firefox
grep: firefox: No such file or directory
What is going on here?
I thought the $* passes multiple arguments to the shell script in sequence. I'm not mistyping anything in my input - and the programs I want to kill (chrome and firefox) are open.
Any help is appreciated.
Remember what grep does with multiple arguments - the first is the word to search for, and the remainder are the files to scan.
Also remember that $*, "$*", and $# all lose track of white space in arguments, whereas the magical "$#" notation does not.
So, to deal with your case, you're going to need to modify the way you invoke grep. You either need to use grep -F (aka fgrep) with options for each argument, or you need to use grep -E (aka egrep) with alternation. In part, it depends on whether you might have to deal with arguments that themselves contain pipe symbols.
It is surprisingly tricky to do this reliably with a single invocation of grep; you might well be best off tolerating the overhead of running the pipeline multiple times:
for process in "$#"
do
kill $(ps -A | grep -w "$process" | awk '{print $1}')
done
If the overhead of running ps multiple times like that is too painful (it hurts me to write it - but I've not measured the cost), then you probably do something like:
case $# in
(0) echo "Usage: $(basename $0 .sh) procname [...]" >&2; exit 1;;
(1) kill $(ps -A | grep -w "$1" | awk '{print $1}');;
(*) tmp=${TMPDIR:-/tmp}/end.$$
trap "rm -f $tmp.?; exit 1" 0 1 2 3 13 15
ps -A > $tmp.1
for process in "$#"
do
grep "$process" $tmp.1
done |
awk '{print $1}' |
sort -u |
xargs kill
rm -f $tmp.1
trap 0
;;
esac
The use of plain xargs is OK because it is dealing with a list of process IDs, and process IDs do not contain spaces or newlines. This keeps the simple code for the simple case; the complex case uses a temporary file to hold the output of ps and then scans it once per process name in the command line. The sort -u ensures that if some process happens to match all your keywords (for example, grep -E '(firefox|chrome)' would match both), only one signal is sent.
The trap lines etc ensure that the temporary file is cleaned up unless someone is excessively brutal to the command (the signals caught are HUP, INT, QUIT, PIPE and TERM, aka 1, 2, 3, 13 and 15; the zero catches the shell exiting for any reason). Any time a script creates a temporary file, you should have similar trapping around the use of the file so that it will be cleaned up if the process is terminated.
If you're feeling cautious and you have GNU Grep, you might add the -w option so that the names provided on the command line only match whole words.
All the above will work with almost any shell in the Bourne/Korn/POSIX/Bash family (you'd need to use backticks with strict Bourne shell in place of $(...), and the leading parenthesis on the conditions in the case are also not allowed with Bourne shell). However, you can use an array to get things handled right.
n=0
unset args # Force args to be an empty array (it could be an env var on entry)
for i in "$#"
do
args[$((n++))]="-e"
args[$((n++))]="$i"
done
kill $(ps -A | fgrep "${args[#]}" | awk '{print $1}')
This carefully preserves spacing in the arguments and uses exact matches for the process names. It avoids temporary files. The code shown doesn't validate for zero arguments; that would have to be done beforehand. Or you could add a line args[0]='/collywobbles/' or something similar to provide a default - non-existent - command to search for.
To answer your question, what's going on is that $* expands to a parameter list, and so the second and later words look like files to grep(1).
To process them in sequence, you have to do something like:
for i in $*; do
echo $i
done
Usually, "$#" (with the quotes) is used in place of $* in cases like this.
See man sh, and check out killall(1), pkill(1), and pgrep(1) as well.
Look into pkill(1) instead, or killall(1) as #khachik comments.
$* should be rarely used. I would generally recommend "$#". Shell argument parsing is relatively complex and easy to get wrong. Usually the way you get it wrong is to end up having things evaluated that shouldn't be.
For example, if you typed this:
end '`rm foo`'
you would discover that if you had a file named 'foo' you don't anymore.
Here is a script that will do what you are asking to have done. It fails if any of the arguments contain '\n' or '\0' characters:
#!/bin/sh
kill $(ps -A | fgrep -e "$(for arg in "$#"; do echo "$arg"; done)" | awk '{ print $1; }')
I vastly prefer $(...) syntax for doing what backtick does. It's much clearer, and it's also less ambiguous when you nest things.

Resources