I have below statement in one of the steps in Jenkinsfile
steps {
sh '''
file=/sql/common/file1.sql
echo $file
if ["$file" = *"/common"* ]; then
echo "changes found in common directory "
fi
'''
}
For some reason shell is not working properly inside jenkinsfile. how do we
compare strings in shell in Jenkinsfile? do we have any specific syntax for these? Jenkins give error if I use == operator to compare the strings.
My assumption was shell should work same way in Jenkinsfile once we declare it inside sh '''. Is that not the case?
["$file"
is invalid. There must be a space between [ and the argument. [ is a command.
if [ "$file" = *"/common"* ];
doesn't mean what you think it does. *"/common"* undergoes filename expansion, so it is replaced by a list of files that match the pattern. Because there are most probably many files that match the filename expansion, [ program exits with some kind of a a syntax error.
If you want to match a string against a pattern in POSIX shell, either use grep with a regular expression:
if printf "%s\n" "$file" | grep -q '.*/common.*'; then
or use case with glob:
if case "$file" in *"/common"*) true; ;; *) false; ;; esac; then
Related
Something I am missing that I am not able to figure out.. Need some thoughts..
I am trying to check file extensions in a directory only file extensions I get .txt or .TXT ) .. but both should be treated as different as I am performing different validations for .txt and .TXT files..
I have the below files
aa.394.63.txt
aa.394.23.TXT
Here is my code
for file in "$SEARCH_DIR"/*; do
extn=$(echo $file | awk -F '.' '{print $NF}')
echo "extn:" $extn
if [ $extn=="txt" ]; then
echo "txt Loop"
elif [$extn=="TXT" ]; then
echo "TXT loop"
fi
But this script always be true for the "txt" validations never goes to the "TXT loop".. I think unix is case sensitive and it should be treated as separate.. Pls advise what am I missing ..
You're using test in the form of [] to test your conditions. You must include spaces around the brackets and the equality operators.
From the test man page:
[ is a synonym for test but requires a final argument of ]
...
Spaces around the brackets are important - each operator and operand must be a separate argument.
https://ss64.com/bash/test.html
This means that you need to pay careful attention to spaces in your test constructs. You should also note that variables should be quoted when you're testing them with [], as they may have undergone word splitting (not relevant in this case, but probably a good habit).
Because you're using [] to test conditons, rather than the bash [[]] construct, you should use a single = framed with whitespace as a test for string equality.
The following is a slightly amended version of your code and should work:
#!/bin/bash
SEARCH_DIR=./search
for file in "$SEARCH_DIR"/*; do
extn=$(echo $file | awk -F '.' '{print $NF}')
echo "extn:" "$extn"
if [ "$extn" = "txt" ]; then
echo "txt Loop"
elif [ "$extn" = "TXT" ]; then
echo "TXT loop"
fi
done
References
test man page
Comparison operators in bash
I have a bash statement to test a command line argument. If the argument passed to the script is "clean", then the script removes all .o files. Otherwise, it builds a program. However, not matter what is passed (if anything), the script still thinks that the argument "clean" is being passed.
#!/bin/bash
if test "`whoami`" != "root" ; then
echo "You must be logged in as root to build (for loopback mounting)"
echo "Enter 'su' or 'sudo bash' to switch to root"
exit
fi
ARG=$1
if [ $ARG == "clean" ] ; then
echo ">>> cleaning up object files..."
rm -r src/*.o
echo ">>> done. "
echo ">>> Press enter to continue..."
read
else
#Builds program
fi
Answer for first version of question
In bash, spaces are important. Replace:
[ $ARG=="clean" ]
With:
[ "$ARG" = "clean" ]
bash interprets $ARG=="clean" as a single-string. If a single-string is placed in a test statement, test returns false if the string is empty and true if it is non-empty. $ARG=="clean" will never be empty. Thus [ $ARG=="clean" ] will always return true.
Second, $ARG should be quoted. Otherwise, if it is empty, then the statement reduces to `[ == "clean" ] which is an error ("unary operator expected").
Third, it is best practices to use lower or mixed case for your local variables. The system uses upper-case shell variables and you don't want to accidentally overwrite one of them.
Lastly, with [...], the POSIX operator for equal, in the string sense, is =. Bash will accept either = or == but = is more portable.
first:
Every string must double quoted or will error absent argument.
second:
for string used only = or != not a == and also -n and -z commands.
third:
you may combine conditions by -a and -o commands but newer used enclose in () yous conditions so not to get error. Logical operators acts through operators presidence, fist calculate -o operator and then -a! For example
[ -n "$1" -a $1 = '-h' -o $1 = '--help' ] && { usage; exit 0; }
will work when passed to script at least 1 argument and is -h or --help. All spaces must be!!! Bush do short cycle logical evaluations. So don't trouble for case when $1 don't exist in second condition because of result of this expression is determined in first one. next don't calculate in this case. But if your argument may contains space symbols you need it double quote. You must do it also in command line too! Else you get error in script or split your arguments in two or more parts.
Operator == isn't used in test. For numbers(not siring) used -eq or -ne commands. See man 1 test for full descriptions. test EXPRESSION... equivalent of [ EXPRESSIONS... ]. More shirt form of test.
I have this (test) script:
#!/bin/bash
my_cmd_bad_ ( ) {
cmd="$#"
$cmd
}
my_cmd_good_ ( ) {
"$#"
}
my_cmd_bad_ ls -l "file with space"
my_cmd_good_ ls -l "file with space"
The output is (the file does not exist, which is not the point of this question):
ยป ~/test.sh
ls: cannot access file: No such file or directory
ls: cannot access with: No such file or directory
ls: cannot access space: No such file or directory
ls: cannot access file with space: No such file or directory
I am surprised that the first version does not work as expected: the parameter is not quoted, and instead of processing one file, it processes three. Why?
How can I save the command that I want to execute, properly quoted? I need to execute it later, where I do not have "$#" anymore.
A simple rework of this test script would be appreciated.
See similar question: How to pass command line parameters with quotes stored in single variable?
Use those utility functions ho save a command to a string for later execution:
bash_escape() {
# backtick indirection strictly necessary here: we use it to strip the
# trailing newline from sed's output, which Solaris/BSD sed *always* output
# (unlike GNU sed, which outputs "test": printf %s test | sed -e s/dummy//)
out=`echo "$1" | sed -e s/\\'/\\''\\\\'\\'\\'/g`
printf \'%s\' "$out"
}
append_bash_escape() {
printf "%s " "$1"
bash_escape "$2"
}
your_cmd_fixed_ ( ) {
cmd="$#"
while [ $# -gt 0 ] ; do
cmd=`append_bash_escape "$cmd" "$1"` ; shift
done
$cmd
}
You can quote any single parameter and evaluate it later:
my_cmd_bad_ ( ) {
j=0
for i in "$#"; do
cmd["$j"]=\"$"$i"\"
j=$(( $j + 1 ))
done;
eval ${cmd[*]}
}
You are combining three space-delimited strings "ls", "-l", and "file with space" into a single space-delimited string cmd. There's no way to know which spaces were originally quoted (in "file with space") and which spaces were introduced during the assignment to cmd.
Typically, it is not a good idea to try to build up command lines into a single string. Use functions, or isolate the actual command and leave the arguments in $#.
Rewrite the command like this:
my_cmd_bad_ () {
cmd=$1; shift
$cmd "$#"
}
See http://mywiki.wooledge.org/BashFAQ/050
Note that your second version is greatly preferred most of the time. The only exceptions are if you need to do something special. For example, you can't bundle an assignment or redirect or compound command into a parameter list.
The correct way to handle the quoting issue requires non-standard features. Semi-realistic example involving a template:
function myWrapper {
typeset x IFS=$' \t\n'
{ eval "$(</dev/fd/0)"; } <<-EOF
for x in $(printf '%q ' "$#"); do
echo "\$x"
done
EOF
}
myWrapper 'foo bar' $'baz\nbork'
Make sure you understand exactly what's going on here and that you really have a good reason for doing this. It requires ensuring side-effects can't affect the arguments. This specific example doesn't demonstrate a very good use case because everything is hard-coded so you're able to correctly escape things in advance and expand the arguments quoted if you wanted.
I am little unfamiliar with the 'if...then...fi' and the 'for' statements syntax.
Could anyone explain what the "$2/$fn" and "/etc/*release" in the code snippets below mean?...specifically on the use of the forward slash....and the asterisk...
if [ -f "$filename" ]; then
if [ ! -f "$2/$fn" ]; then
echo "$fn is missing from $2"
missing=$((missing + 1))
fi
fi
and
function system_info
{
if ls /etc/*release 1>/dev/null 2>&1; then
echo "<h2>System release info</h2>"
echo "<pre>"
for i in /etc/*release; do
# Since we can't be sure of the
# length of the file, only
# display the first line.
head -n 1 $i
done
uname -orp
echo "</pre>"
fi
} # end of system_info
...thx for the help...
/etc/*release : here the * will match any number of any characters, so any thing /etc/0release , /etc/asdfasdfr_release etc will be matched. Simply stated, it defined all the files in the /etc/ directory which ends with the string release.
The $2 is the 2nd commandline argument to the shell script, and $fn is some other shell variable. The "$2/$fn" after the variable substitutions will make a string, and the [ -f "$2/$fn" ] will test if the string formed after the substitution forms a path to a regular file which is specified by the -f switch. If it is a regular file then the body of if is executed.
In the for loop the loop will loop for all the files ending with the string release in the directory /etc (the path). At each iteration i will contain the next such file name, and for each iteration the first 1 line of the file is displayed with the head command by getting the file name from variable i within the body.
It is better to check the manual man bash and for if condition check man test . Here is a good resource: http://tldp.org/LDP/Bash-Beginners-Guide/html/
The forward slash is the path separator, and the * is a file glob character. $2/$fn is a path where $2 specifies the directory and $fn is the filename. /etc/*release expands to the space separated list of all the files in /etc whose name ends in "release"
Dollar sign marks variable. The "-f" operator means "file exsists".
So,
[ -f "$filename" ]
checks if there is file named the same as value contained in $filename variable.
Simmilar, if we assume that $2 = "some_folder", and $fn = "some_file", expression
[ ! -f "$2/$fn" ]
returns true if file some_folder/some_file doesn't exsist.
Now, about asterisk - it marks "zero or more of any character(s)". So, expression:
for i in /etc/*release; do
will iterate trough all folders named by that pattern, for example:
/etc/release, /etc/666release, /etc/wtf_release...
I hope this helps.
This question has 3 parts, and each alone is easy, but combined together is not trivial (at least for me) :)
Need write a script what should take as its arguments:
one name of another command
several arguments for the command
list of files
Examples:
./my_script head -100 a.txt b.txt ./xxx/*.txt
./my_script sed -n 's/xxx/aaa/' *.txt
and so on.
Inside my script for some reason I need distinguish
what is the command
what are the arguments for the command
what are the files
so probably the most standard way write the above examples is:
./my_script head -100 -- a.txt b.txt ./xxx/*.txt
./my_script sed -n 's/xxx/aaa/' -- *.txt
Question1: Is here any better solution?
Processing in ./my_script (first attempt):
command="$1";shift
args=`echo $* | sed 's/--.*//'`
filenames=`echo $* | sed 's/.*--//'`
#... some additional processing ...
"$command" "$args" $filenames #execute the command with args and files
This solution will fail when the filenames will contain spaces and/or '--', e.g.
/some--path/to/more/idiotic file name.txt
Question2: How properly get $command its $args and $filenames for the later execution?
Question3: - how to achieve the following style of execution?
echo $filenames | $command $args #but want one filename = one line (like ls -1)
Is here nice shell solution, or need to use for example perl?
First of all, it sounds like you're trying to write a script that takes a command and a list of filenames and runs the command on each filename in turn. This can be done in one line in bash:
$ for file in a.txt b.txt ./xxx/*.txt;do head -100 "$file";done
$ for file in *.txt; do sed -n 's/xxx/aaa/' "$file";done
However, maybe I've misinterpreted your intent so let me answer your questions individually.
Instead of using "--" (which already has a different meaning), the following syntax feels more natural to me:
./my_script -c "head -100" a.txt b.txt ./xxx/*.txt
./my_script -c "sed -n 's/xxx/aaa/'" *.txt
To extract the arguments in bash, use getopts:
SCRIPT=$0
while getopts "c:" opt; do
case $opt in
c)
command=$OPTARG
;;
esac
done
shift $((OPTIND-1))
if [ -z "$command" ] || [ -z "$*" ]; then
echo "Usage: $SCRIPT -c <command> file [file..]"
exit
fi
If you want to run a command for each of the remaining arguments, it would look like this:
for target in "$#";do
eval $command \"$target\"
done
If you want to read the filenames from STDIN, it would look more like this:
while read target; do
eval $command \"$target\"
done
The $# variable, when quoted will be able to group parameters as they should be:
for parameter in "$#"
do
echo "The parameter is '$parameter'"
done
If given:
head -100 test this "File name" out
Will print
the parameter is 'head'
the parameter is '-100'
the parameter is 'test'
the parameter is 'this'
the parameter is 'File name'
the parameter is 'out'
Now, all you have to do is parse the loop out. You can use some very simple rules:
The first parameter is always the file name
The parameters that follow that start with a dash are parameters
After the "--" or once one doesn't start with a "-", the rest are all file names.
You can check to see if the first character in the parameter is a dash by using this:
if [[ "x${parameter}" == "x${parameter#-}" ]]
If you haven't seen this syntax before, it's a left filter. The # divides the two parts of the variable name. The first part is the name of the variable, and the second is the glob filter (not regular expression) to cut off. In this case, it's a single dash. As long as this statement isn't true, you know you have a parameter. BTW, the x may or may not be needed in this case. When you run a test, and you have a string with a dash in it, the test might mistake it for a parameter of the test and not the value.
Put it together would be something like this:
parameterFlag=""
for parameter in "$#" #Quotes are important!
do
if [[ "x${parameter}" == "x${parameter#-}" ]]
then
parameterFlag="Tripped!"
fi
if [[ "x${parameter}" == "x--" ]]
then
print "Parameter \"$parameter\" ends the parameter list"
parameterFlag="TRIPPED!"
fi
if [ -n $parameterFlag ]
then
print "\"$parameter\" is a file"
else
echo "The parameter \"$parameter\" is a parameter"
fi
done
Question 1
I don't think so, at least not if you need to do this for arbitrary commands.
Question 3
command=$1
shift
while [ $1 != '--' ]; do
args="$args $1"
shift
done
shift
while [ -n "$1" ]; do
echo "$1"
shift
done | $command $args
Question 2
How does that differ from question 3?