ns/APAC_BankTransfers_Publish/CMB/services/svcPublishBankTransfers/flow.xml
In the above line and similar lines like this I want to extract whatever is present in between services and flow.xml and save it to a variable DIST.
The output should be svcPublishBankTransfers.
Using parameter expansion mechanisms available in POSIX sh:
s=ns/APAC_BankTransfers_Publish/CMB/services/svcPublishBankTransfers/flow.xml
s=${s%/flow.xml} # remove "/flow.xml"
s=${s##*/services/} # remove everything before "services"
echo "$s"
This has the advantages of being purely in-process (so faster than approaches that require piping through an external tool), and compatible with all POSIX shells (ash, dash, ksh, etc).
References:
BashFAQ #100 ("How do I do string manipulation in bash?")
BashFAQ #73 ("How can I use parameter expansion? How can I get substrings?")
Using BASH regex:
s='ns/APAC_BankTransfers_Publish/CMB/services/svcPublishBankTransfers/flow.xml'
[[ "$s" =~ /([^/]+)/[^/]*$ ]] && echo "${BASH_REMATCH[1]}"
svcPublishBankTransfers
OR else:
[[ "$s" =~ /services/([^/]+)/flow\.xml ]] && echo "${BASH_REMATCH[1]}"
svcPublishBankTransfers
bash$ echo "ns/APAC_BankTransfers_Publish/CMB/services/svcPublishBankTransfers/flow.xml" | cut -d '/' -f 5
svcPublishBankTransfers
bash$
Related
I wrote the following code in Bash:
#!/bin/sh
host=$1
regex="^(((git|ssh|http(s)?)|(git#[\w\.]+))(:(\/\/)?)([A-Za-z0-9.#:_/-]+)\.com)(.*)"
if [[ "$host" =~ $regex ]]; then
d=${BASH_REMATCH[1]}
if [[ "$d" = *github* ]]; then
return
fi
fi
die "Current repository is not stored in Github."
I want to learn how to write a better Bash code so I use the shellcheck.net.
Line 5:
if [[ "$host" =~ $regex ]]; then
^-- SC2039: In POSIX sh, [[ ]] is undefined.
Line 6:
d=${BASH_REMATCH[1]}
^-- SC2039: In POSIX sh, array references are undefined.
Line 7:
if [[ "$d" = *github* ]]; then
^-- SC2039: In POSIX sh, [[ ]] is undefined.
I'm trying to understand how to fix those warnings. I understand that in order to fix [[ ]] I need it to switch to [ ] but then I get an error due globs. Also how should I replace the =~ operator?
When you write #!/bin/sh then you shouldn't use bash-specific features like [[. But you don't need to change [[ to [ or anything like that; just change the shebang line to #!/bin/bash. Then you can use all the bash features you like.
Use grep and sed in posix.
# use grep -q to match with regex
if printf "%s\n" "$host" | grep -q '\(git\|ssh\|http\(s\)\)etc. etc. etc.'; then
# use sed to extract part of the string matching regex
d=$(printf "%s\n" "$host" | sed 's/\(g\|ssh\|http\(s\)\)etc. etc. etc./\2/')
if printf "%s\n" "$d" | grep -q github; then
return
fi
fi
Finding out proper regexes is left to others.
You could try to parse out the different parts with parameter expansions though it's going to get a bit tedious. (The link is to the Bash manual; only a few of the expansions supported by Bash are POSIX.)
Assuming the input is a valid, well-formed URL (which may or may not be warranted) maybe try
host=$1
tail=${1#*://*/}
case $tail in "$host") tail=${host#*/};; esac
case ${host%/$tail} in
*github.com) return ;;
esac
die "Current repository is not stored in Github."
(where of course we assume that this is in a context where return makes sense, and where die is defined separately, like we have to assume in the original code).
This is quite a lot simpler than the regex you presented, and definitely does not cover all the strings that the regex would be able to handle; but perhaps it doesn't have to be all that complex if we can assume that the URL has gone through some sort of validation (i.e. if it's the output from git remote it's pretty safe to assume that the user has verified it by other means already).
I writing a shell script.
The line will look like:
"holderIdentity":"leader-elector1-5cd5b9d76d-ztfgf","numberofLeader":{"":0,"node2":1,"node3":2,"node4":2}"
The character 5cd5b9d76d-ztfgf will be created randomly, so my keyword to grep is just only leader-elector. And the string ,"numberofLeader":{"":0,"node2":1,"node3":2,"node4":2}" is also randomly (it is diffent everytime)
How can I get just this string leader-elector1-5cd5b9d76d-ztfgf from the line.
Thank you so much!
using GNU grep
grep -Eo 'leader-elector[^"]*' <<< '"holderIdentity":"leader-elector1-5cd5b9d76d-ztfgf","numberofLeader":{"":0,"node2 ":1,"node3":2,"node4":2}"'
otherwise with bash regex feature
var='"holderIdentity":"leader-elector1-5cd5b9d76d-ztfgf","numberofLeader":{"":0,"node2 ":1,"node3":2,"node4":2}"'
re='leader-elector[^"]*'
if [[ $var =~ $re ]] ; then
declare -p BASH_REMATCH
echo "${BASH_REMATCH[0]}"
fi
I looked some other posts and learnt to match file extension in the following way but why my code is not working? Thanks.
1 #!/bin/sh
2
3 for i in `ls`
4 do
5 if [[ "$i" == *.txt ]]
6 then
7 echo "$i is .txt file"
8 else
9 echo "$i is NOT .txt file"
10 fi
11 done
eidt:
I realized #!/bin/sh and #!/bin/bash are different, if you are looking at this post later, remember to check which one you are using.
The [[ ]] expression is only available in some shells, like bash and zsh. Some more basic shells, like dash, do no support it. I'm guessing you're running this on a recent version of Ubuntu or Debian, where /bin/sh is actually dash, and hence doesn't recognize [[. And actually, you shouldn't use [[ ]] with a #!/bin/sh shebang anyway, since it's unsafe to depend on a feature that the shebang doesn't request.
So, what to do about it? You'll have the [ ] type of test expression available, but it doesn't do pattern matching (like *.txt). There are a number of alternate ways to do it:
The case statement is available in even basic shells, and has the same pattern matching capability as [[ = ]]. This is the most common way to do this type of thing, especially when you have a list of different patterns to check against.
More indirectly, you can use ${var%pattern} to try remove .txt from the end of the end of the value (see "Remove Smallest Suffix Pattern" here), and then check to see if that changed the value:
if [ "$i" != "${i%.txt}" ]
More explanation: suppose $i is "file.txt"; then this expands to [ "file.txt" != "file" ], so they're not equal, and the test (for !=) succeeds. On the other hand, if $i is "file.pdf", then it expands to [ "file.pdf" != "file.pdf" ], which fails because the strings are the same.
Other notes: when using [ ], use a single equal sign for string comparison, and be sure to properly double-quote all variable references to avoid confusion. Also, if you use anything that has special meaning to the shell (like < or >), you need to quote or escape them.
You could use the expr command's : operator to do regular expression matching. (Regular expressions are a different type of pattern from the basic wildcard or "glob" expression.) You could do this, but don't.
#!/bin/sh
for i in `ls`
do
if [[ "$i" = *".txt" ]] ; then
echo "$i is .txt file"
else
echo "$i is NOT .txt file"
fi
done
You don't have to loop in ls output, and sh implementation might vary among OS distributions.
Consider:
#! /bin/sh
for i in *
do
if [[ "$i" == *.txt ]]
then
echo "$i is txt file"
else
echo "$i is NOT txt file"
fi
done
I have a list of alias definitions in a file I want to unalias in a batch.
The file looks like this:
please=sudo
po='git push origin'
I have come this far but I'm not sure how to pass the alias names to the unalias com
cat old.txt | cut -d = -f 1
To allow the input file to contain comments, you might do something like:
while IFS== read -r name val; do
[[ $val ]] || continue # skip any line that didn't have a "="
[[ $name =~ [#] ]] && continue # skip any line that had a # anywhere before the "="
unalias "$name"
done <old.txt
This avoids relying on any tools external to the shell itself -- all processing is done with bash-native logic. (Sometimes this is the right thing, sometimes it's not -- bash's string processing tends to be slower than general-purpose tools, but those tools typically also have significant startup-time costs, making them undesirable to run in a loop).
The while read idiom is documented in BashFAQ #1. Setting IFS== means that we split into fields when an = is seen; providing name and val means that the first field goes into name, and all subsequent fields go into val.
[[ $var =~ $regex ]] does POSIX ERE-style regex matching.
... <in.txt is both more efficient than cat in.txt | ..., and avoids triggering the issues described in BashFAQ #24 (which can happen when piping data into a loop).
This did the trick:
unalias $(cat old.txt | cut -d = -f 1)
I'm trying to add a function to my bash_profile for msysgit:
function git-unpushed {
brinfo=$(git branch -v | grep git-branch-name)
if [[ $brinfo =~ ("[ahead "([[:digit:]]*)]) ]]
then
echo "(${BASH_REMATCH[2]})"
fi
}
But I get the following error:
bash: conditional binary operator expected`
bash: syntax error near =~'
From what I can find, the "equals tilde" operator (=~) evaluates as regex in bash.
Why is =~ is throwing an error?
UPDATE: Here's a screenshot of inputting it manually (this is running sh.exe):
I had the same error on Bash 3.1.0 from Git installation on Windows. Ultimately I changed it to:
if echo $var | grep -E 'regexp' > /dev/null
then
...
fi
According to https://groups.google.com/forum/#!topic/msysgit/yPh85MPDyfE this is because msys doesn't ship libregex along with bash. Supposedly if you compile/find an msys built libregex, and put it in the library path, =~ starts working fine.
Update 2015: msysgit is now obsolete.
You should use the bash which comes with git-for-windows.
As mentioned in this answer, it uses a much more recent bash (4.3+), for which the =~ syntax will work.
Original answer (march 2013)
The bash packaged with msysgit might simply be too old to fully support this operator.
It is certainly too old to compare with unquoted regex, as mentioned in "Bash, version 3" and "How do I use regular expressions in bash scripts?":
As of version 3.2 of Bash, expression to match no longer quoted.
Actually, mklement0 mentions in the comments:
=~ was introduced in bash 3.0 and always supported an unquoted token on the RHS.
Up to 3.1.x, quoted tokens were treated the same as unquoted tokens: both were interpreted as regexes.
What changed in 3.2 was that quoted tokens (or quoted substrings of a token) are now treated as literals.
But I tried with quotes (in the latest msysgit 1.8.1.2), and it still fails:
vonc#voncvb /
$ /bin/bash --version
GNU bash, version 3.1.0(1)-release (i686-pc-msys)
Copyright (C) 2005 Free Software Foundation, Inc.
vonc#voncvb /
$ variable="This is a fine mess."
vonc#voncvb /
$ echo "$variable"
This is a fine mess.
vonc#voncvb /
$ if [[ "$variable" =~ T.........fin*es* ]] ; then echo "ok" ; fi
bash: conditional binary operator expected
bash: syntax error near `=~'
vonc#voncvb /
$ if [[ "$variable" =~ "T.........fin*es*" ]] ; then echo "ok" ; fi
bash: conditional binary operator expected
bash: syntax error near `=~'
vonc#voncvb /
Here is a solution that supports extracting matched strings. If the operator =~ is not supported by bash, then the sed command is used (installed with msysgit)
if eval "[[ a =~ a ]]" 2>/dev/null; then
regexMatch() { # (string, regex)
eval "[[ \$1 =~ \$2 ]]"
return $?
}
elif command -v /bin/sed >/dev/null 2>&1; then
regexMatch() { # (string, regex)
local string=$1
if [[ ${2: -1} = $ ]]; then
local regex="(${2%$})()()()()()()()()$"
else
local regex="($2)()()()()()()()().*"
fi
regex=${regex//\//\\/}
local replacement="\1\n\2\n\3\n\4\n\5\n\6\n\7\n\8\n\9\n"
local OLD_IFS=$IFS
IFS=$'\n'
BASH_REMATCH=($(echo "$string" | /bin/sed -rn "s/$regex/$replacement/p" | while read -r; do echo "${REPLY}"; done))
IFS=$OLD_IFS
[[ $BASH_REMATCH ]] && return 0 || return 1
}
else
error "your Bash shell does not support regular expressions"
fi
Usage example:
if regexMatch "username#host.domain" "(.+)#(.+)"; then
echo ${BASH_REMATCH[0]}
echo ${BASH_REMATCH[1]}
echo ${BASH_REMATCH[2]}
fi