Given the following string:
toto="function(param"
I want to get the substring function from the string above, in bash.
I tried the following:
echo "${toto%(}"
Which gives:
function(param
However, with this example:
echo "${toto%param}"
I get:
function(
As expected.
The expansion did not take place when expanding the the character "(".
Why is that ? How can I extract only the beginning (before the "(" of this string ?
To cut ( and anything after it you have match exactly that.
echo "${toto%(*}"
Related
I have string this:
release/3.0.2.344
And I want to extract this:
release/3.0.2
pattern is like:
release/<number>.<number>.<number>.<number>
Use parameter expansion in bash:
$ v="release/3.0.2.344"
$ echo "${v%.*}"
release/3.0.2
% cuts off of the end of the variable, and the search pattern is a dot . followed by any characters *.
How can I escape double quotes inside a double string in Bash?
For example, in my shell script
#!/bin/bash
dbload="load data local infile \"'gfpoint.csv'\" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY \"'\n'\" IGNORE 1 LINES"
I can't get the ENCLOSED BY '\"' with double quote to escape correctly. I can't use single quotes for my variable, because I want to use variable $dbtable.
Use a backslash:
echo "\"" # Prints one " character.
A simple example of escaping quotes in the shell:
$ echo 'abc'\''abc'
abc'abc
$ echo "abc"\""abc"
abc"abc
It's done by finishing an already-opened one ('), placing the escaped one (\'), and then opening another one (').
Alternatively:
$ echo 'abc'"'"'abc'
abc'abc
$ echo "abc"'"'"abc"
abc"abc
It's done by finishing already opened one ('), placing a quote in another quote ("'"), and then opening another one (').
More examples: Escaping single-quotes within single-quoted strings
Keep in mind that you can avoid escaping by using ASCII codes of the characters you need to echo.
Example:
echo -e "This is \x22\x27\x22\x27\x22text\x22\x27\x22\x27\x22"
This is "'"'"text"'"'"
\x22 is the ASCII code (in hex) for double quotes and \x27 for single quotes. Similarly you can echo any character.
I suppose if we try to echo the above string with backslashes, we will need a messy two rows backslashed echo... :)
For variable assignment this is the equivalent:
a=$'This is \x22text\x22'
echo "$a"
# Output:
This is "text"
If the variable is already set by another program, you can still apply double/single quotes with sed or similar tools.
Example:
b="Just another text here"
echo "$b"
Just another text here
sed 's/text/"'\0'"/' <<<"$b" #\0 is a special sed operator
Just another "0" here #this is not what i wanted to be
sed 's/text/\x22\x27\0\x27\x22/' <<<"$b"
Just another "'text'" here #now we are talking. You would normally need a dozen of backslashes to achieve the same result in the normal way.
Bash allows you to place strings adjacently, and they'll just end up being glued together.
So this:
echo "Hello"', world!'
produces
Hello, world!
The trick is to alternate between single and double-quoted strings as required. Unfortunately, it quickly gets very messy. For example:
echo "I like to use" '"double quotes"' "sometimes"
produces
I like to use "double quotes" sometimes
In your example, I would do it something like this:
dbtable=example
dbload='load data local infile "'"'gfpoint.csv'"'" into '"table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '"'"'"' LINES "'TERMINATED BY "'"'\n'"'" IGNORE 1 LINES'
echo $dbload
which produces the following output:
load data local infile "'gfpoint.csv'" into table example FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "'\n'" IGNORE 1 LINES
It's difficult to see what's going on here, but I can annotate it using Unicode quotes. The following won't work in Bash – it's just for illustration:
dbload=‘load data local infile "’“'gfpoint.csv'”‘" into ’“table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '”‘"’“' LINES ”‘TERMINATED BY "’“'\n'”‘" IGNORE 1 LINES’
The quotes like “ ‘ ’ ” in the above will be interpreted by bash. The quotes like " ' will end up in the resulting variable.
If I give the same treatment to the earlier example, it looks like this:
echo “I like to use” ‘"double quotes"’ “sometimes”
Store the double quote character in a variable:
dqt='"'
echo "Double quotes ${dqt}X${dqt} inside a double quoted string"
Output:
Double quotes "X" inside a double quoted string
Check out printf...
#!/bin/bash
mystr="say \"hi\""
Without using printf
echo -e $mystr
Output: say "hi"
Using printf
echo -e $(printf '%q' $mystr)
Output: say \"hi\"
Make use of $"string".
In this example, it would be,
dbload=$"load data local infile \"'gfpoint.csv'\" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY \"'\n'\" IGNORE 1 LINES"
Note (from the man page):
A double-quoted string preceded by a dollar sign ($"string") will cause the string to be translated according to the current locale. If the current locale is C or POSIX, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted.
For use with variables that might contain spaces in you Bash script, use triple quotes inside the main quote, e.g.:
[ "$(date -r """$touchfile""" +%Y%m%d)" -eq "$(date +%Y%m%d)" ]
Add "\" before double quote to escape it, instead of \
#! /bin/csh -f
set dbtable = balabala
set dbload = "load data local infile "\""'gfpoint.csv'"\"" into table $dbtable FIELDS TERMINATED BY ',' ENCLOSED BY '"\""' LINES TERMINATED BY "\""'\n'"\"" IGNORE 1 LINES"
echo $dbload
# load data local infile "'gfpoint.csv'" into table balabala FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY "''" IGNORE 1 LINES
I have key value pairs in a string like this:
key1 = "value1"
key2 = "value2"
key3 = "value3"
In a bash script, I need to extract the value of one of the keys like for key2, I should get value2, not in quote.
My bash script needs to work in both Redhat and Ubuntu Linux hosts.
What would be the easiest and most reliable way of doing this?
I tried something like this simplified script:
pattern='key2\s*=\s*\"(.*?)\".*$'
if [[ "$content" =~ $pattern ]]
then
key2="${BASH_REMATCH[1]}"
echo "key2: $key2"
else
echo 'not found'
fi
But it does not work consistently.
Any better/easier/more reliable way of doing this?
To separate the key and value from your $content variable, you can use:
[[ $content =~ (^[^ ]+)[[:blank:]]*=[[:blank:]]*[[:punct:]](.*)[[:punct:]]$ ]]
That will properly populate the BASH_REMATCH array with both values where your key is in BASH_REMATCH[1] and the value in BASH_REMATCH[2].
Explanation
In bash the [[...]] treats what appears on the right side of =~ as an extended regular expression and matched according to man 3 regex. See man 1 bash under the section heading for [[ expression ]] (4th paragraph). Sub-expressions in parenthesis (..) are saved in the array variable BASH_REMATCH with BASH_REMATCH[0] containing the entire portion of the string (your $content) and each remaining elements containing the sub-expressions enclosed in (..) in the order the parenthesis appear in the regex.
The Regular Expression (^[^ ]+)[[:blank:]]*=[[:blank:]]*[[:punct:]](.*)[[:punct:]]$ is explained as:
(^[^ ]+) - '^' anchored at the beginning of the line, [^ ]+ match one or more characters that are not a space. Since this sub-expression is enclosed in (..) it will be saved as BASH_REMATCH[1], followed by;
[[:blank:]]* - zero or more whitespace characters, followed by;
= - an equal sign, followed by;
[[:blank:]]* - zero or more whitespace characters, followed by;
[[:punct:]] - a punctuation character (matching the '"', which avoids caveats associated with using quotes within the regex), followed by the sub-expression;
(.*) - zero or more characters (the rest of the characters), and since it is a sub-expression in (..) it the characters will be stored in BASH_REMATCH[2], followed by;
[[:punct:]] - a punctuation character (matching the '"' ... ditto), at the;
$ - end of line anchor.
So if you match what your key and value input lines separated by an = sign, it will separate the key and value into the array BASH_REMATCH as you wanted.
Bash supports BRE only and you cannot use \s and .*?.
As an alternative, please try:
while IFS= read -r content; do
# pattern='key2\s*=\s*\"(.*)\".*$'
pattern='key2[[:blank:]]*=[[:blank:]]*"([^"]*)"'
if [[ $content =~ $pattern ]]
then
key2="${BASH_REMATCH[1]}"
echo "key2: $key2"
(( found++ ))
fi
done < input-file.txt
if (( found == 0 )); then
echo "not found"
fi
What you start talking about key-value pairs, it is best to use an associative array:
declare -A map
Now looking at your lines, they look like key = "value" where we assume that:
value is always encapsulated by double quotes, but also could contain a quote
an unknown number of white spaces is before and/or after the equal sign.
So assuming we have a variable line which contains key = "value", the following operations will extract that value:
key="${line%%=*}"; key="${key// /}"
value="${line#*=}"; value="${value#*\042}"; value="${value%\042*}"
IFS=" \t=" read -r value _ <<<"$line"
This allows us now to have something like:
declare -A map
while read -r line; do
key="${line%%=*}"; key="${key// /}"
value="${line#*=}"; value="${value#*\042}"; value="${value%\042*}"
map["$key"]="$value"
done <inputfile
With awk:
awk -v key="key2" '$1 == key { gsub("\"","",$3);print $3 }' <<< "$string"
Reading the output of the variable called string, pass the required key in as a variable called key and then if the first space delimited field is equal to the key, remove the quotes from the third field with the gsub function and print.
Ok, after spending so many hours, this is how I solved the problem:
If you don't know where your script will run and what type of file (win/mac/linux) are you reading:
Try to avoid non-greedy macth in linux bash instead of tweaking diffrent switches.
don't trus end of line match $ when you might get data from windows or mac
This post solved my problem: Non greedy text matching and extrapolating in bash
This pattern works for me in may linux environments and all type of end of lines:
pattern='key2\s*=\s*"([^"]*)"'
The value is in BASH_REMATCH[1]
I want to use backticks in ruby for a programm call.
The parameter is a String variable containing one or more backticks, i.e.
"&E?##A`?". The following command yields a new label as its return value:
echo "&E?##A\`?" | nauty-labelg 2>/dev/null
From a ruby program I can call it as follows and get the correct result:
new_label = `echo "&E?##A\\\`?" | nauty-labelg 2>/dev/null`
I want to achieve the same using a variable for the label.
So I have to insert three slashes into my variable label = "&E?##A`?" in order to escape the backtick. The following seems to work, though it is not very elegant:
escaped_label = label.gsub(/`/, '\\\`').gsub(/`/, '\\\`').gsub(/`/, '\\\`')
But the new variable cannot be used in the program call:
new_label = `echo "#{escaped_label}" | nauty-labelg 2>/dev/null`
In this case I do not get an answer from nauty-labelg.
So I have to insert three slashes into my variable label = "&E?##A`?" in order to escape the backtick.
No, you only need to add one backslash for the output. To escape the ` special bash character. The other other two are only for representation proposes, otherwise it isn't valid Ruby code.
new_label = `echo "&E?##A\\\`?" | nauty-labelg 2>/dev/null`
The first backslash will escape the second one (outputting one single backslash). The third backslash escapes the ` character (outputting one single `).
You should only add backslashes before characters that have a special meaning within double quoted bash context. These special characters are: $, `, \ and \n. Those can be escaped with the following code:
def escape_bash_string(string)
string.gsub(/([$`"\\\n])/, '\\\\\1')
end
For label = "&E?##A`?" only the ` should be escaped.
escaped_string = escape_bash_string("&E?##A\`?")
puts escaped_string
# &E?##A\`?
I have a text file in which I want to find all of ID:= "abc123" when it finds that I want it to take that value of abc123 and create a new line and have a set string, newId:= "abc123 How can I do this within terminal?
I'd like to use bash, below are some examples, find the string '"ID": ", copy the value (abc123) and make a new line with this data.
"ID": "abc123"
"newID": "abc123"
You can do this:
sed -e 's/^"ID": "\(.*\)"/&\
"newID": "\1"/' myfile.txt
First, I'll try to explain the regular expression that searches for matches:
^ Matches the start of the line
"ID": " Matches that exact string
\(.*\) Matches a sequence of zero or more (*) of any character (.). Placing this expression between backslashed parenthesis creates a "capture", which allows us to store the resulting part of the match into an auxiliary variable \1.
" Matches the double-quote character
When it finds a match, it replaces it with:
& the match itself. This operator is an auxiliary variable that represents what was matched.
\<new-line> the backslash followed by an actual new line character escapes a new line, ie. it allows us to print a new line character into the replacement
"newId": " prints that exact string
\1 prints the contents of our capture, so it prints the ID we found
" prints a double quote character.
Hope this helps =)
Try doing this :
sed -r 's#^"ID": "([a-Z0-9]+)"#"newID": "\1"#' file.txt
sed : the executable
-r : extented mode (no need to backslash parenthesis)
s : we perform a substitution, skeleton is s#origin#replacement# (the separator can be anything)
^ : means start of line in regex
( ) : parenthesis is a capture
"newID": is the start of the new string
\1 : is the end of the substituted string (the captured string)
Considering your question is very vague I made some assumptions which will become apparent in my implementation.
INPUT FILE -- call it t
ID="one"
dkkd
ID="two"
ffkjf
ID="three"
ldl
Command ran on input file
for line in `cat t`; do newID=`echo $line | grep ID | cut -d= -f2`; if [[ "$newID" != "" ]]; then echo $line >> t2; echo newID=$newID >> t2; else echo $line >> t2; fi; done
OUTPUT FILE -- Name is t2 (apparent from the command)
ID="one"
newID="one"
dkkd
ID="two"
newID="two"
ffkjf
ID="three"
newID="three"
ldl
Basically this command goes line by line in the file (in this case called t) looks for an ID line. If it finds one it gets its value, prints the original line with the ID and then prints another one with a newID following right after. If the line in question does not have and ID then it just prints the line it self.
Things to note:
If you have any other line in the file that contains "ID" in it but is not the normal ID that you requested, this will not work.