How to handle variables that contain ";"? - bash

I have a configuration file that contains lines like "hallo;welt;" and i want to do a grep on this file.
Whenever i try something like grep "$1;$2" my.config or echo "$1;$2 of even line="$1;$2" my script fails with something like:
: command not found95: line 155: =hallo...
How can i tell bash to ignore ; while evaluating "..." blocks?
EDIT: an example of my code.
# find entry
$line=$(grep "$1;$2;" $PERMISSIONSFILE)
# splitt line
reads=$(echo $line | cut -d';' -f3)
writes=$(echo $line | cut -d';' -f4)
admins=$(echo $line | cut -d';' -f5)
# do some stuff on the permissions
# replace old line with new line
nline="$1;$2;$reads;$writes;$admins"
sed -i "s/$line/$nline/g" $TEMPPERM
my script should be called like this: sh script "table" "a.b.*.>"
EDIT: another, simpler example
$test=$(grep "$1;$2;" temp.authorization.config)
the temp file:
table;pattern;read;write;stuff
the call sh test.sh table pattern results in: : command not foundtable;pattern;read;write;stuff

Don't use $ on the left side of an assignment in bash -- if you do it'll substitute the current value of the variable rather than assigning to it. That is, use:
test=$(grep "$1;$2;" temp.authorization.config)
instead of:
$test=$(grep "$1;$2;" temp.authorization.config)
Edit: also, variable expansions should be in double-quotes unless there's a good reason otherwise. For example, use:
reads=$(echo "$line" | cut -d';' -f3)
instead of:
reads=$(echo $line | cut -d';' -f3)
This doesn't matter for semicolons, but does matter for spaces, wildcards, and a few other things.

A ; inside quotes has no meaning at all for bash. However, if $1 contains a doublequote itself, then you'll end up with
grep "something";$2"
which'll be parsed by bash as two separate commands:
grep "something" ; other"
^---command 1----^ ^----command 2---^
Show please show exactly what your script is doing around the spot the error is occurring, and what data you're feeding into it.

Counter-example:
$ cat file.txt
hello;welt;
hello;world;
hell;welt;
$ cat xx.sh
grep "$1;$2" file.txt
$ bash -x xx.sh hello welt
+ grep 'hello;welt' file.txt
hello;welt;
$
You have not yet classified your problem accurately.
If you try to assign the result of grep to a variable (like I do) your example breaks.
Please show what you mean. Using the same data file as before and doing an assignment, this is the output I get:
$ cat xx.sh
grep "$1;$2" file.txt
output=$(grep "$1;$2" file.txt)
echo "$output"
$ bash -x xx.sh hello welt
+ grep 'hello;welt' file.txt
hello;welt;
++ grep 'hello;welt' file.txt
+ output='hello;welt;'
+ echo 'hello;welt;'
hello;welt;
$
Seems to work for me. It also demonstrates why the question needs an explicit, complete, executable, minimal example so that we can see what the questioner is doing that is different from what people answering the question think is happening.
I see you've provided some sample code:
# find entry
$line=$(grep "$1;$2;" $PERMISSIONSFILE)
# splitt line
reads=$(echo $line | cut -d';' -f3)
writes=$(echo $line | cut -d';' -f4)
admins=$(echo $line | cut -d';' -f5)
The line $line=$(grep ...) is wrong. You should omit the $ before line. Although it is syntactically correct, it means 'assign to the variable whose name is stored in $line the result of the grep command'. That is unlikely to be what you had in mind. It is, occasionally, useful. However, those occasions are few and far between, and only for people who know what they're doing and who can document accurately what they're doing.
For safety if nothing else, I would also enclose the $line values in double quotes in the echo lines. It may not strictly be necessary, but it is simple protective programming.
The changes lead to:
# find entry
line=$(grep "$1;$2;" $PERMISSIONSFILE)
# split line
reads=$( echo "$line" | cut -d';' -f3)
writes=$(echo "$line" | cut -d';' -f4)
admins=$(echo "$line" | cut -d';' -f5)
The rest of your script was fine.

It seems like you are trying to read a semicolon-delimited file, identify a line starting with 'table;pattern;' where table is a string you specify and pettern is a regular expression grep will understand. Once the line is identified you wish to replaced the 3rd, 4th and 5th fields with different data and write the updated line back to the file.
Does this sound correct?
If so, try this code
#!/bin/bash
in_table="$1"
in_pattern="$2"
file="$3"
while IFS=';' read -r -d$'\n' tuple pattern reads writes admins ; do
line=$(cut -d: -f1<<<"$tuple")
table=$(cut -d: -f2<<<"$tuple")
# do some stuff with the variables
# e.g., update the values
reads=1
writes=2
admins=12345
# replace the old line with the new line
sed -i'' -n $line'{i\
'"$table;$pattern;$reads;$writes;$admins"'
;d;}' "$file"
done < <(grep -n '^'"${in_table}"';'"${in_pattern}"';' "${file}")
I chose to update by line number here to avoid problems of unknown characters in the left hand of the substitution.

Related

How to print the last column of a row only using "grep" and "cut" bash command

I need to parse the line written bold below:
line="eth1 Link encap:Ethernet HWaddr 11:11:11:11:11:11"
This line may have more words unexpectedly such as
line="eth1 Link encap:Ethernet Extra HWaddr 11:11:11:11:11:11"
So, for parsing the MAC address correctly, I need to parse the line accordingly with a bash command.
echo $line | cut -d' ' -f5* works for the first line, while *echo $line | cut -d' ' -f6* works for the second. So, I need to parse only the last column of the line.
However, because of the device restriction, I can only use grep and cut command. Not sed, awk, rev,reverse, etc.
With grep:
echo $line | grep -o -E '[^ ]+$'
With cut, a solution can be made with an extra computation based on the word count, assuming the delimiter is a space:
nw=$(echo $line | wc -w)
echo $line | cut -d ' ' -f$nw-
If the MAC address is the last sequence of characters after a space, you can remove the longest match of "* " (asterisk and a space) pattern using pure Bash:
echo "${line##* }"
You can also extract the last 17 characters from the string:
echo "${line: -17}"
If you want a strict match at the end of the line (due to .*):
echo $(expr match "$line" '.*\(\([a-zA-Z0-9]\{2\}\:\)\{5\}[a-zA-Z0-9]\{2\}\)')
Using GNU grep:
grep -o -P '(?:[a-zA-Z0-9]{2}:){5}[a-zA-Z0-9]{2}' <<< "$line"
In the latter case, you may want to add the $ anchor for the end of the line. Of course, you don't have to use here string. You may want to use a pipe instead: echo "$line" | grep -o -P ....

creating a variable from sed output

I am banging my head against the keyboard on this simple piece of code.
#!/bin/bash
connstate="Connected"
vpnstatus=$(/opt/cisco/anyconnect/bin/vpn state | (grep -m 1 'state:'))
echo $vpnstatus
vpnconn=$(echo $vpnstatus | sed -e 's/>>\ state: //g' | sed "s/ //g")
echo "$vpnconn" "$connstate"
if [ "$vpnconn" = "$connstate" ];then
echo $vpnconn
else echo "this script still fails"
fi
echo done
This is the output from the above code:
>> state: Connected
Connected Connected
this script still fails
done
I believe the issue revolves around the vpnconn=$ if I comment that section of code out and fill the variable vpnconn="Connected" this code works fine. Something with how the sed is working on the input from vpnstatus and outputting the results to vpnconn is making what looks like a correct result incorrect when doing the compare in the if then.
I have tried splitting up the vpnconn line into two separate lines and that did not change anything, I took out the sed "s/ //g" and replaced it with a trim -d ' ' and that did not change the results. I know this is something small in this tiny piece of code that I am missing.
Did you try?
vpnconn=$(echo "$vpnstatus" | awk '{print $3}')
Something like:
vpnstatus=$(/opt/cisco/anyconnect/bin/vpn state|grep -m 1 'state:'|awk '{print 3}')
should do the work.

How Do I Convert A Cut Command In Bash Into Grep With Given Code?

I've written a template engine script that uses cut to extract certain elements from a file, but I want to use grep in place of the cut. Here is the code I have written:
#!/bin/bash
IFS=# #makes # a delimiter.
while read line
do
dataspace=`echo ${line} | cut -d'=' -f1`
value=`echo ${line} | cut -d"=" -f2`
printf -v $dataspace "$value" #make the value stored in value into the name of a dataspace.
done < 'template.vars' #read template.vars for standard input.
skipflag=false #initialize the skipflag to false
while read line #while it is reading standard input one line at a time
Just came to the conclusion that the code blocks system here does not support bash.
Anyway, since stackoverflow isn't letting me put Bash into codeblocks, I am not putting the entire script since it would look nasty. Based on what is currently high-lighted, how would I go about changing the part using the cut command into a line using the grep command?
As has been noted, you should give more information for a better answer. Going with what you have, I would say that awk is a better option than grep
dataspace=$(awk '$0=$1' FS== <<< "$line")
value=$(awk '$0=$2' FS== <<< "$line")

Why does this sed command do nothing? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question appears to be off-topic because it lacks sufficient information to diagnose the problem. Describe your problem in more detail or include a minimal example in the question itself.
Closed 9 years ago.
Improve this question
I am trying to cat a file to create a copy of itself, but at the same time replace some values
my command is:
cat ${FILE} | sed "s|${key}|${value}|g" > ${TEMP_FILE}
However, when I open the temp file, none of the keys have been replaced- just a straight swap. I have echoed the values of key and value and they are correct - they come from an array element.
Yet if I use a plain string not a variable, it works fine for one type of key - i.e:
cat ${FILE} | sed "s|example_key|${value}|g" > ${TEMP_FILE}
The example_key instances within the file are replaced which is what I want.
However, when I try to use my array $key parameter, it does nothing. No idea why :-(
Command usage:
declare -a props
...
....
for x in "${props[#]}"
do
key=`echo "${x}" | cut -d '=' -f 1`
value=`echo "${x}" | cut -d '=' -f 2`
# global replace on the $FILE
cat ${FILE} | sed "s|${key}|${value}|g" > ${TEMP_FILE}
#cat ${FILE} | sed "s|example_key|${value}|g" > ${TEMP_FILE}
done
array elements are stored in the following format: $key=$value
key='echo "${x}" | cut -d '=' -f 1
value='echo "${x}" | cut -d '=' -f 2
Use back-ticks, not single-quotes, if you want to do command substitution.
key=`echo "${x}" | cut -d '=' -f 1`
value=`echo "${x}" | cut -d '=' -f 2`
Also note that as you loop over the series of key=value pairs, you're overwriting your temp file each time, using only one substitution applied to the original file.. So after the loop is finished, the best you can hope for is that only the last substitution will be applied.
I'd also suggest not doing this in multiple passes -- do it by passing multiple expressions to sed:
for x in "${props[#]}" ; do
subst="$subst -e 's=$x=g'"
done
sed $subst "${FILE}" > "${TEMP_FILE}"
I'm using a trick, by using = as the delimiter for the sed substitution expression, we don't have to separate the key from the value. The command simply becomes:
sed -e 's=foo=1=g' -e 's=bar=2=g' "${FILE}" > "${TEMP_FILE}"
Thanks to #BillKarwin for spotting the crux of the problem: each iteration of the loop wipes out the previous iterations' replacements, because the result of a single key-value pair replacement replaces the entire output file every time.
Try the following:
declare -a props
# ...
cp "$FILE" "$TEMP_FILE"
for x in "${props[#]}"; do
IFS='=' read -r key value <<<"$x"
sed -i '' "s|${key}|${value}|g" "${TEMP_FILE}"
done
Copies the input file to the output file first, then replaces the output file in-place (using sed's -i option) in every iteration of the loop.
I also streamlined the code to parse each line into key and value, using read.
Also note that I consistently double-quoted all variable references.
#anubhava makes a good general point: depending on the variable values, a different regex delimiter may be needed (in your case: if the keys or values contained '|', you couldn't use '|' to delimit the regexes).
Update: #BillKarwin makes a good point: performing the replacements one by one, in a loop, is inefficient.
Here's a one-liner that avoids loops altogether:
sed -f <(awk -F'=' '{ if ($0) print "s/" $1 "/" substr($0, 1+length($1)) "/g" }' \
"$FILE") "$FILE" > "$TEMP_FILE"
Uses awk to build up the entire set of substitution commands for sed (one per line).
Then feeds the result via process substitution as a command file to sed with -f.
Handles the case where values have embedded = chars. correctly.

How to remove the last character from a bash grep output

COMPANY_NAME=`cat file.txt | grep "company_name" | cut -d '=' -f 2`
outputs something like this
"Abc Inc";
What I want to do is I want to remove the trailing ";" as well. How can i do that? I am a beginner to bash. Any thoughts or suggestions would be helpful.
This will remove the last character contained in your COMPANY_NAME var regardless if it is or not a semicolon:
echo "$COMPANY_NAME" | rev | cut -c 2- | rev
I'd use sed 's/;$//'. eg:
COMPANY_NAME=`cat file.txt | grep "company_name" | cut -d '=' -f 2 | sed 's/;$//'`
foo="hello world"
echo ${foo%?}
hello worl
I'd use head --bytes -1, or head -c-1 for short.
COMPANY_NAME=`cat file.txt | grep "company_name" | cut -d '=' -f 2 | head --bytes -1`
head outputs only the beginning of a stream or file. Typically it counts lines, but it can be made to count characters/bytes instead. head --bytes 10 will output the first ten characters, but head --bytes -10 will output everything except the last ten.
NB: you may have issues if the final character is multi-byte, but a semi-colon isn't
I'd recommend this solution over sed or cut because
It's exactly what head was designed to do, thus less command-line options and an easier-to-read command
It saves you having to think about regular expressions, which are cool/powerful but often overkill
It saves your machine having to think about regular expressions, so will be imperceptibly faster
I believe the cleanest way to strip a single character from a string with bash is:
echo ${COMPANY_NAME:: -1}
but I haven't been able to embed the grep piece within the curly braces, so your particular task becomes a two-liner:
COMPANY_NAME=$(grep "company_name" file.txt); COMPANY_NAME=${COMPANY_NAME:: -1}
This will strip any character, semicolon or not, but can get rid of the semicolon specifically, too.
To remove ALL semicolons, wherever they may fall:
echo ${COMPANY_NAME/;/}
To remove only a semicolon at the end:
echo ${COMPANY_NAME%;}
Or, to remove multiple semicolons from the end:
echo ${COMPANY_NAME%%;}
For great detail and more on this approach, The Linux Documentation Project covers a lot of ground at http://tldp.org/LDP/abs/html/string-manipulation.html
Using sed, if you don't know what the last character actually is:
$ grep company_name file.txt | cut -d '=' -f2 | sed 's/.$//'
"Abc Inc"
Don't abuse cats. Did you know that grep can read files, too?
The canonical approach would be this:
grep "company_name" file.txt | cut -d '=' -f 2 | sed -e 's/;$//'
the smarter approach would use a single perl or awk statement, which can do filter and different transformations at once. For example something like this:
COMPANY_NAME=$( perl -ne '/company_name=(.*);/ && print $1' file.txt )
don't have to chain so many tools. Just one awk command does the job
COMPANY_NAME=$(awk -F"=" '/company_name/{gsub(/;$/,"",$2) ;print $2}' file.txt)
In Bash using only one external utility:
IFS='= ' read -r discard COMPANY_NAME <<< $(grep "company_name" file.txt)
COMPANY_NAME=${COMPANY_NAME/%?}
Assuming the quotation marks are actually part of the output, couldn't you just use the -o switch to return everything between the quote marks?
COMPANY_NAME="\"ABC Inc\";" | echo $COMPANY_NAME | grep -o "\"*.*\""
you can strip the beginnings and ends of a string by N characters using this bash construct, as someone said already
$ fred=abcdefg.rpm
$ echo ${fred:1:-4}
bcdefg
HOWEVER, this is not supported in older versions of bash.. as I discovered just now writing a script for a Red hat EL6 install process. This is the sole reason for posting here.
A hacky way to achieve this is to use sed with extended regex like this:
$ fred=abcdefg.rpm
$ echo $fred | sed -re 's/^.(.*)....$/\1/g'
bcdefg
Some refinements to answer above. To remove more than one char you add multiple question marks. For example, to remove last two chars from variable $SRC_IP_MSG, you can use:
SRC_IP_MSG=${SRC_IP_MSG%??}
cat file.txt | grep "company_name" | cut -d '=' -f 2 | cut -d ';' -f 1
I am not finding that sed 's/;$//' works. It doesn't trim anything, though I'm wondering whether it's because the character I'm trying to trim off happens to be a "$". What does work for me is sed 's/.\{1\}$//'.

Resources