I have a funny issue with grep. Basically, I am trying to match certain control characters in a file and get the count.
grep -ocbUaE $"\x07\|\x08\|\x0B\|\x0C\|\x1A\|\x1B" <file>
Funny enough, in CLI it matches all control characters and returns the correct count, but if I use it in a bash script, it doesn't match anything.
Any ideas what I am doing wrong?
Tested on: MacOS and CentOS - same issue.
Thank you for your help!
I think you should change your command to:
grep -cUaE $'[\x07\x08\x0B\x0C\x1A\x1B]' file
I removed the extra output flags, which get ignored when -c is present. I assume that you include -U and -a for a reason.
The other changes are to use $'' with single quotes (you don't want a double-quoted string here), and replace your series of ORs with a bracket expression, which matches if any one of the characters match.
Note that C-style strings $'' don't work in all shells, so if you want to use bash you should call your script like bash script.sh and/or include the shebang #!/bin/bash if it is executable. sh script.sh does not behave in the same way as bash script.sh.
Related
I'm developing a shell script where I have the variable RUN as a result of a command substitution.
Something like this:
RUN="$(kubectl logs ${POD_LISTENER} | grep ${FROM_DATE})"
OUTPUT=$(eval $RUN)
Problem is with the grep portion.
The pattern I'm searching with grep is a date, so I need to add single quote around the variable ${FROM_DATE} to match exactly what I need.
From the terminal, I run the command below and get the result I need
kubectl logs cortex-listener-prod-6b8884d45b-mlmzz | grep '2018-08-11'
And it work well, but I can't make it run from the script.
I don't see, why you need to quote your FROM_DATE, because this string contains only digits and hyphens, and neither one is treated as a filename character in a POSIX shell. However, your outer double quotes don't make sense. Just use
RUN=$(kubectl logs $POD_LISTENER | grep $FROM_DATE)
I have an applescript
do shell script "echo -n -e \\\\x61\\\\x61\\\\x61 > /tmp/file.txt"
But the file.txt does not contain "aaa"!
It contains "-n -e aaa\n" instead.
Can someone help me with that problem?
Different versions of echo are hopelessly inconsistent in how they interpret command options (like -n and -e) and/or escape sequences in the string. It's not just bash vs. sh as cdarke said; it's much messier than that. The best thing to do is just avoid either one by using printf instead. It's a bit more complicated to use than echo, but completely worth it because your scripts won't break just because the latest version of the shell was compiled with different options(!).
In this case, using printf is actually even simpler than using echo, because it always interprets escape sequences (in its first argument, the "format string" -- the rest are different), and doesn't print a newline at the end (unless you explicitly tell it to with \n at the end of the format string). So your script becomes:
do shell script "printf \\\\x61\\\\x61\\\\x61 > /tmp/file.txt"
...although you can simplify it further by using single-quotes to keep the shell from interpreting escapes before they get to printf:
do shell script "printf '\\x61\\x61\\x61' > /tmp/file.txt"
(The escapes are still doubled, because they're being interpreted by AppleScript. But at least they don't need to be quadrupled anymore.)
(p.s. relevant xkcd)
We started using shellcheck to check our scripts for errors/warnings,
Now common warning what we see in all our scripts is unquoted variables.
is there any script to correct those simple warnings/errors ?
I have below command which I use to change $VAR to ${VAR}
sed -i -r 's:\$([_a-zA-Z?][_a-zA-Z0-9]*):${\1}:g' <scriptname>
I modified it as follows,
sed -i -r 's:\$([_a-zA-Z?][_a-zA-Z0-9]*):"${\1}":g' <scriptname>
above command works fine when variables are unquoted but when they are quoted e.g. "$VAR" it changes to ""${VAR}""
any suggestion to whether continue doing it with sed or better write script to do it ?
any particular suggestions?
Edit carefully.
When you write echo "This is example ${var} in the middle of the line" you do not want to put quotes around ${var}.
You should put all variables (except PATH, PWD and some other system vars) in lowercase.
You might want to add some mappings in .vimrc, that will execute your sed first or second commandline using F4 of F5 (something like . ! ~/bin/make_my_var) making the editing easier. In make_my_var you can add logic for lowercasing the vars when they are not one of a list of exceptions.
And (edited):
You might want some more standards, perhaps use a styleguide.
I want to issue this command from the bash script
sed -e $beginning,$s/pattern/$variable/ file
but any possible combination of quotes gives me an error, only one that works:
sed -e "$beginning,$"'s/pattern/$variable/' file
also not good, because it do not dereferences the variable.
Does my approach can be implemented with sed?
Feel free to switch the quotes up. The shell can keep things straight.
sed -e "$beginning"',$s/pattern/'"$variable"'/' file
You can try this:
$ sed -e "$beginning,$ s/pattern/$variable/" file
Example
file.txt:
one
two
three
Try:
$ beginning=1
$ variable=ONE
$ sed -e "$beginning,$ s/one/$variable/" file.txt
Output:
ONE
two
three
There are two types of quotes:
Single quotes preserve their contents (> is the prompt):
> var=blah
> echo '$var'
$var
Double quotes allow for parameter expansion:
> var=blah
> echo "$var"
blah
And two types of $ sign:
One to tell the shell that what follows is the name of a parameter to be expanded
One that stands for "last line" in sed.
You have to combine these so
The shell doesn't think sed's $ has anything to do with a parameter
The shell parameters still get expanded (can't be within single quotes)
The whole sed command is quoted.
One possibility would be
sed "$beginning,\$s/pattern/$variable/" file
The whole command is in double quotes, i.e., parameters get expanded ($beginning and $variable). To make sure the shell doesn't try to expand $s, which doesn't exist, the "end of line" $ is escaped so the shell doesn't try anything funny.
Other options are
Double quoting everything but adding a space between $ and s (see Ren's answer)
Mixing quoting types as needed (see Ignacio's answer)
Methods that don't work
sed '$beginning,$s/pattern/$variable/' file
Everything in single quotes: the shell parameters are not expanded (doesn't follow rule 2 above). $beginning is not a valid address, and pattern would be literally replaced by $variable.
sed "$beginning,$s/pattern/$variable/" file
Everything in double qoutes: the parameters are expanded, including $s, which isn't supposed to (doesn't follow rule 1 above).
the following form worked for me from within script
sed $beg,$ -e s/pattern/$variable/ file
the same form will also work if executed from the shell
I want to do (in bash script):
NEWBASE=`echo $NAME | sed "s/${DIR}//g" | sed 's/.\///g'`
I read in the net, that I have to replace single quote with double quote.
This is unfortunately not working. Why? Thanks
sed is overkill for this. Use parameter expansion:
NEWBASE=${NAME//$DIR//}
NEWBASE=${NEWBASE//.\//}
It is important to understand that bash and sed are two completely independent things. When you give bash a command, it first processes it according to its rules, in order to come up with a utility name and a set of arguments for that utility (in this case sed), and then calls the utility with the arguments.
Probably $DIR contains a slash character. Perhaps it looks something like /usr/home/codyline/src.
So when bash substitutes that into the argument to the sed command:
"s/${DIR}//g"
the result is
s//usr/home/codyline/src//g
which is what is then passed to sed. But sed can't understand that commabnd: it has (many) too many / characters.
If you really want to use sed for this purpose, you need to use a delimiter other than /, and it needs to be a character you are confident will never appear in $DIR. Fortunately, the sed s command allows you to use any character as a delimiter: whatever character follows the s is used as the delimiter. But there always must be exactly three of them in the command.
For example, you might believe that no directory path contains a colon (:), in which case you could use:
sed "s:${DIR}::g"
Of course, someday that will fail precisely because you have a directory with a colon in its name. So you could make things more general by using bash's substitute-and-replace feature to backslash-escape all the colons:
sed "s:${DIR//:/\:}::g"
But you could have used this bash feature in order to avoid the use of sed altogether:
NEWBASE=${NAME//$DIR}
Unfortunately, you can't nest bash substitute-and-replaces, so you need to do them sequentially:
NEWBASE=${NEWBASE//.\/}
Note: I used ${var//...}, which is the equivalent of specifying the g flag in a sed s command, but I really don't know if it is appropriate. Do you really expect multiple instances of $DIR in a single path? If there are multiple instances, do you really want to remove all of them? You'll have to decide.