How can I check for a string in a file path in bash? I am trying:
if [[$(echo "${filePathVar}" | sed 's#//#:#g#') == *"File.java"* ]]
to replace all forward slashes with a colon (:) in the path. It's not working. Bash is seeing the file path string as a file path and throws the error "No such file or directory". The intention is for it to see the file path as a string.
Example: filePathVar could be
**/myloc/src/File.java
in which case the check should return true.
Please note that I am writing this script inside a Jenkins job as a build step.
Updates as of 12/15/15
The following returns Not found, which is wrong.
#!/bin/bash
sources="**/src/TESTS/A.java **/src/TESTS/B.java"
if [[ "${sources}" = ~B.java[^/]*$ ]];
then
echo "Found!!"
else
echo "Not Found!!"
fi
The following returns Found which also also wrong (removed the space around the comparator =).
#!/bin/bash
sources="**/src/TESTS/A.java **/src/TESTS/C.java"
if [[ "${sources}"=~B.java[^/]*$ ]];
then
echo "Found!!"
else
echo "Not Found!!"
fi
The comparison operation is clearly not working.
It is easier to use bash's builtin regex matching facility:
$ filePathVar=/myLoc/src/File.java
if [[ "$filePathVar" =~ File.java[^/]*$ ]]; then echo Match; else echo No Match; fi
Match
Inside [[...]], the operator =~ does regex matching. The regular expression File.java[^/]* matches any string that contains File.java optionally followed by anything except /.
It worked in a simpler way as below:
#!/bin/bash
sources="**/src/TESTS/A.java **/src/TESTS/B.java"
if [[ $sources == *"A.java"* ]]
then
echo "Found!!"
else
echo "Not Found!!"
fi
Related
Here is my code
vmname="$1"
EXCEPTLIST="desktop-01|desktop-02|desktop-03|desktop-04"
if [[ $vmname != #(${EXCEPTLIST}) ]]; then
echo "${vmname}"
else
echo "Its in the exceptlist"
fi
The above code works perfectly but my question is , the EXCEPTLIST can be a long line, say 100 server names. In that case its hard to put all that names in one line. In that situation is there any way to make the variable EXCEPTLIST to be a multiline variable ? something like as follows:
EXCEPTLIST="desktop-01|desktop-02|desktop-03| \n
desktop-04|desktop-05|desktop-06| \n
desktop-07|desktop-08"
I am not sure but was thinking of possibilities.
Apparently I would like to know the terminology of using #(${})- Is this called variable expansion or what ? Does anyone know the documentation/explain to me about how this works in bash. ?
One can declare an array if the data/string is long/large. Use IFS and printf for the format string, something like:
#!/usr/bin/env bash
exceptlist=(
desktop-01
desktop-02
desktop-03
desktop-04
desktop-05
desktop-06
)
pattern=$(IFS='|'; printf '#(%s)' "${exceptlist[*]}")
[[ "$vmname" != $pattern ]] && echo good
In that situation is there any way to make the variable EXCEPTLIST to be a multiline variable ?
With your given input/data an array is also a best option, something like:
exceptlist=(
'desktop-01|desktop-02|desktop-03'
'desktop-04|desktop-05|desktop-06'
'desktop-07|desktop-08'
)
Check what is the value of $pattern variable one way is:
declare -p pattern
Output:
declare -- pattern="#(desktop-01|desktop-02|desktop-03|desktop-04|desktop-05|desktop-06)"
Need to test/check if $vmname is an empty string too, since it will always be true.
On a side note, don't use all upper case variables for purely internal purposes.
The $(...) is called Command Substitution.
See LESS=+'/\ *Command Substitution' man bash
In addition to what was mentioned in the comments about pattern matching
See LESS=+/'(pattern-list)' man bash
See LESS=+/' *\[\[ expression' man bash
s there any way to make the variable EXCEPTLIST to be a multiline variable ?
I see no reason to use matching. Use a bash array and just compare.
exceptlist=(
desktop-01
desktop-02
desktop-03
desktop-04
desktop-05
desktop-06
)
is_in_list() {
local i
for i in "${#:2}"; do
if [[ "$1" = "$i" ]]; then
return 0
fi
done
return 1
}
if is_in_list "$vmname" "${EXCEPTLIST[#]}"; then
echo "is in exception list ${vmname}"
fi
#(${})- Is this called variable expansion or what ? Does anyone know the documentation/explain to me about how this works in bash. ?
${var} is a variable expansion.
#(...) are just characters # ( ).
From man bash in Compund commands:
[[ expression ]]
When the == and != operators are used, the string to the right of the operator is considered a pattern and matched according to the rules
described below under Pattern Matching, as if the extglob shell option were enabled. ...
From Pattern Matching in man bash:
#(pattern-list)
Matches one of the given patterns
[[ command receives the #(a|b|c) string and then matches the arguments.
There is absolutely no need to use Bash specific regex or arrays and loop for a match, if using grep for raw string on word boundary.
The exception list can be multi-line, it will work as well:
#!/usr/bin/sh
exceptlist='
desktop-01|desktop-02|desktop-03|
deskop-04|desktop-05|desktop-06|
desktop-07|deskop-08'
if printf %s "$exceptlist" | grep -qwF "$1"; then
printf '%s is in the exceptlist\n' "$1"
fi
I wouldn't bother with multiple lines of text. This is would be just fine:
EXCEPTLIST='desktop-01|desktop-02|desktop-03|'
EXCEPTLIST+='desktop-04|desktop-05|desktop-06|'
EXCEPTLIST+='desktop-07|desktop-08'
The #(...) construct is called extended globbing pattern and what it does is an extension of what you probably already know -- wildcards:
VAR='foobar'
if [[ "$VAR" == fo?b* ]]; then
echo "Yes!"
else
echo "No!"
fi
A quick walkthrough on extended globbing examples: https://www.linuxjournal.com/content/bash-extended-globbing
#!/bin/bash
set +o posix
shopt -s extglob
vmname=$1
EXCEPTLIST=(
desktop-01 desktop-02 desktop-03
...
)
if IFS='|' eval '[[ ${vmname} == #(${EXCEPTLIST[*]}) ]]'; then
...
Here's one way to load a multiline string into a variable:
fn() {
cat <<EOF
desktop-01|desktop-02|desktop-03|
desktop-04|desktop-05|desktop-06|
desktop-07|desktop-08
EOF
}
exceptlist="$(fn)"
echo $exceptlist
As to solving your specific problem, I can think of a variety of approaches.
Solution 1, since all the desktop has the same desktop-0 prefix and only differ in the last letter, we can make use of {,} or {..} expansion as follows:
vmname="$1"
found=0
for d in desktop-{01..08}
do
if [[ "$vmname" == $d ]]; then
echo "It's in the exceptlist"
found=1
break
fi
done
if (( !found )); then
echo "Not found"
fi
Solution 2, sometimes, it is good to provide a list in a maintainable clear text list. We can use a while loop and iterate through the list
vmname="$1"
found=0
while IFS= read -r d
do
if [[ "$vmname" == $d ]]; then
echo "It's in the exceptlist"
found=1
break
fi
done <<EOF
desktop-01
desktop-02
desktop-03
desktop-04
desktop-05
desktop-06
desktop-07
desktop-08
EOF
if (( !found )); then
echo "Not found"
fi
Solution 3, we can desktop the servers using regular expressions:
vmname="$1"
if [[ "$vmname" =~ ^desktop-0[1-8]$ ]]; then
echo "It's in the exceptlist"
else
echo "Not found"
fi
Solution 4, we populate an array, then iterate through an array:
vmname="$1"
exceptlist=()
exceptlist+=(desktop-01 desktop-02 desktop-03 deskop-04)
exceptlist+=(desktop-05 desktop-06 desktop-07 deskop-08)
found=0
for d in ${exceptlist[#]}
do
if [[ "$vmname" == "$d" ]]; then
echo "It's in the exceptlist"
found=1
break;
fi
done
if (( !found )); then
echo "Not found"
fi
I have an issue in finding a part of string variable in another string variable, I tried many methods but none worked out..
for example:
echo -e " > Required_keyword: $required_keyword"
send_func GUI WhereAmI
echo -e " > FUNCVALUE: $FUNCVALUE"
flag=`echo $FUNCVALUE|awk '{print match($0,"$required_keyword")}'`;
if [ $flag -gt 0 ];then
echo "Success";
else
echo "fail";
fi
But it always gives fail though there are certain words in variable which matches like
0_Menu/BAA_Record ($required_keyword output string)
Trying to connect to 169.254.98.226 ... OK! Executing sendFunc GUI
WhereAmI Sent Function WhereAmI [OK PageName:
"_0_Menu__47__BAA_Record" ($FUNCVALUE output string)
As we can see here the BAA_Record is common in both of the output still, it always give FAIL
The output echo is
> Required_keyword: 0_Menu/BAA_Record
> FUNCVALUE:
Trying to connect to 169.254.98.226 ... OK!
Executing sendFunc GUI WhereAmI
Sent Function WhereAmI [OK]
PageName: "_0_Menu__47__BAA_Record"
Bash can do wildcard and regex matches inside double square brackets.
if [[ foobar == *oba* ]] # wildcard
if [[ foobar =~ fo*b.r ]] # regex
In your example:
if [[ $FUNCVALUE = *$required_keyword* ]]
if [[ $FUNCVALUE =~ .*$required_keyword.* ]]
Not sure if I understand what you want, but if you need to find out if there's part of string "a" present in variable "b" you can use simply just grep.
grep -q "a" <<< "$b"
[[ "$?" -eq 0 ]] && echo "Found" || echo "Not found"
EDIT: To clarify, grep searches for string a in variable b and returns exit status (see man grep, hence the -q switch). After that you can check for exit status and do whatever you want (either with my example or with regular if statement).
I made a very simple script which tells me a file name and extension.
The script works as follows:
for file in * ; do
if [[ -f $file ]] ; then
filename=${file##*/}
basename=${filename%\.*}
extension=${filename##*.}
if [[ -n $extension ]] ; then
echo "FILE: " $basename " ; ESTENSIONE " $extension
fi
fi
done
The problem is that when I have a file without extension (e.g. Makefile) it says that the extension is the filename itself (e.g. extension= Makefile).
Am I doing something wrong?
Well, the result you get is the expected one; I don't know if that means you're doing something wrong or not.
The way the pattern replacements work is that if the pattern doesn't match, nothing is replaced. Here you have ${filename##*.} which says remove all characters up to and including the final period. But if there's no period in the name, then the pattern doesn't match and nothing is removed, so you simply get the same result as ${filename}.
I should point out that the backslash in ${filename%\.*} is useless: the pattern here is shell globbing not regular expressions, so you don't need to escape a period. You can just write ${filename%.*}.
ETA:
There's no way to do what you want in one step. You have two choices; you can either test to see if the extension is the same as the filename and if so set it to empty:
extension=${filename##*.}
[ "$extension" = "$filename" ] && extension=
or you can strip off the basename, which you already computed, then get rid of any leading periods:
extension=${filename#$basename}
extension=${extension##*.}
Extensions don't have any privileged status in Unix file systems; they are just a part of the file name that people treat specially. You'll have to check if the file contains a . first.
basename=${filename%\.*}
if [[ $filename = *.* ]]; then
extension=${filename##*.}
echo "FILE: " $basename " ; ESTENSIONE " $extension
else
extension=""
fi
I'm trying to match any strings that start with /John/ but does not contain / after /John/
if
[ $string == /John/[!/]+ ]; then ....
fi
This is what I got and it doesn't seem to be working.
So I tried
if
[[ $string =~ ^/John/[!/]+$ ]]; then ....
fi
It still didn't work, and so I changed it to
if
[[ $string =~ /John/[^/] ]]; then ....
fi
It worked but will match with all the strings that has / behind /John/ too.
For bash you want [[ $string =~ /John/[^/]*$ ]] -- the end-of-line anchor ensures there are no slashes after the last acceptable slash.
How about "the string starts with '/John/' and doesn't contain any slashes after '/John/'"?
[[ $string = /John/* && $string != /John/*/* ]]
Or you could compare against a parameter expansion that only expands if the conditions are met. This says "after stripping off everything including and after the last slash, the string is /John":
[[ ${string%/*} = /John ]]
In fact, this last solution is the only entirely POSIXLY_STRICT one I can come up with without multiple test expressions.
[ "${string%/*}" = /John ]
By the way, your problem is probably simply be using double-equals inside a single-bracket test expression. bash actually does accept them inside double-bracket test expressions, but a single equals is a better idea.
You can also use plain old grep:
string='/John Lennon/Yoko Ono'
if echo "$string" | grep -q "/John[^/]" ; then
echo "matched"
else
echo "no match found"
fi
This only fails if /John is at the very end of the string... if that's a possibility then you can tweak to handle that case, for instance:
string='/John Lennon/Yoko Ono'
if echo "$string" | grep -qP "(/John[^/])|(/John$)" ; then
echo "matched"
else
echo "no match found"
fi
Not sure what language you're using, but normal negative character classes are prefixed with a ^
e.g.
[^/]
You can also put in start/end qualifiers (clojure example, so Java's regex engine). Usually ^ at beginning and $ at end.
user => (re-matches #"^/[a-zA-Z]+[^/]$" "/John/")
nil
A little bit about my application:
I am writing a small application in bash script. The application must store personal settings to home directory.
My settings are in the form of a key/value pair which will be stored as filename/content:
for example:
~/my-github-app
├── github-account
├── github-token
My current solution for adding a key/value:
read KEY
read VALUE
# FIXME! should check for for valid filename.
# user can do injection hack by KEY="../../path/to/yourfile"
echo $VALUE > ~/my-github-app/$KEY
What is the simplest and safe way to validate $KEY?
A built-in function?
A regular expression?
I really need a reusable solution, not just for this application.
Edit:
"validate filename" mean check string for proper filename format, accepted by OS.
"bob" : good filename format
"" : bad because filename can not be empty
"*" : ?
"/" : ?
"con" : ?
....
The only way to make something secure is to use a whitelist. Which means instead of blacklisting bad characters you allow good ones. The reason why blacklists will always fail is that you can't blacklist all of the weird characted, you'd always forget something. Especially if you're working with unicode strings.
Filenames could contain anything. According to wikipedia:
Ext4 Allowed characters in filenames: All bytes except NUL ('\0') and '/'
Which means that whole bash scripts could be valid filenames.
So, if I were you, I would only allow a-zA-Z as valid characters. Problem solved.
That's how you do it:
# if [[ $key =~ [^a-zA-Z] ]]; then # or this. Whatever makes more sense to you
if ! [[ $key =~ ^[a-zA-Z]+$ ]]; then
echo 'Wrong key. Only a-zA-Z characters are allowed' >&2 # write to stderr
exit 1
fi
In addition to #Aleks-Daniel Jakimenko-A.'s answer, following script checks following conditions, If all conditions are set then True is returned:
a-z
A-Z
0-9
underscore (_)
dash (-)
period (.)
Max length is 255
First character should be a character or number: {a-z or A-z or 0-9}
my_script.sh:
#!/bin/bash
# To run: bash my_script.sh <my_string_is>
key=$1
val=$(echo "${#key}")
if [[ $key == "" ]]; then
echo "False";
exit
fi
if [[ $key == "." ]] || [[ $key == ".." ]]; then
# "." and ".." are added automatically and always exist, so you can't have a
# file named . or .. // https://askubuntu.com/a/416508/660555
echo "False";
exit
fi
if [ $val -gt 255 ]; then
# String's length check
echo "False";
exit
fi
if ! [[ $key =~ ^[0-9a-zA-Z._-]+$ ]]; then
# Checks whether valid characters exist
echo "False";
exit
fi
_key=$(echo $key | cut -c1-1)
if ! [[ $_key =~ ^[0-9a-zA-Z.]+$ ]]; then
# Checks the first character
echo "False";
exit
fi
echo "True";
If you just want to check if a file already exists, use the test command and use it like this for your validation :
if [[ ! -e "$KEY" ]]
then
#file doet not exists
fi
What do you want to validate, just that a key doesn't contain any path info?
KEY=$(basename $KEY)
This would remove any parts of the KEY that are part of the path. That said, there are plenty of things the user could enter that would probably be a bad idea. Can you perhaps have a list of allowed keys, then reject anything that isn't in that list?
If you're trying to see if a file is writable, you could check if a) it exists and is writable (-w) or b) just try to write to it and check for errors.