I ran into a small problem when trying to create a shell line in groovy that is stapled from many variables.
Here is what it looks like:
sh("""
java -jar Report.jar settings.xml "$startTest" "$durationTest" "$opt" empty iReport:gReport:aReport:pReport="$iReport":"$gReport":"$aReport":"$pReport" $reportName
""")
The point is that the command in the shell must run along with the ", which here frames the values of the variables.
However, no matter how much I tried to screen them:
Put \ or \\ in front of "
Convert the string to 'text' + varibale + 'text'
Put the whole shell line in ' and leave " as is, or with the first point taken into account.
None of this works.
Here is an example output:
java -jar Report.jar settings.xml '2022-07-04 11:00' 01:00 'Offset=01:00;' empty iReport:gReport:aReport:pReport=true:true:true:true App.result
You may notice that for some reason two variables are surrounded by quotation marks ' for themselves, although this is not specified anywhere, which is probably the root of the problem.
Because if me use \", the quotes appear, but not everywhere, in general, me get some mishmash of " and '.
To understand, this is how a groovy file works in the Jenkins pipeline.
Hence my question, under what circumstances do variables start framing themselves '' and how to get rid of it?
Or maybe someone has a different solution to this problem, I would appreciate it.
Related
I've written a bunch of environment variables in Docker format, but now I want to use them outside of that context. How can I source them with one line of bash?
Details
Docker run and compose have a convenient facility for importing a set of environment variables from a file. That file has a very literal format.
The value is used as is and not modified at all. For example if the value is surrounded by quotes (as is often the case of shell variables), the quotes are included in the value passed
Lines beginning with # are treated as comments and are ignored
Blank lines are also ignored.
"If no = is provided and that variable is…exported in your local environment," docker "passes it to the container"
Thankfully, whitespace before the = will cause the run to fail
so, for example, this env-file:
# This is a comment, with an = sign, just to mess with us
VAR1=value1
VAR2=value2
USER
VAR3=is going to = trouble
VAR4=this $sign will mess with things
VAR5=var # with what looks like a comment
#VAR7 =would fail
VAR8= but what about this?
VAR9="and this?"
results in these env variables in the container:
user=ubuntu
VAR1=value1
VAR2=value2
VAR3=is going to = trouble
VAR4=this $sign will mess with things
VAR5=var # with what looks like a comment
VAR8= but what about this?
VAR9="and this?"
The bright side is that once I know what I'm working with, it's pretty easy to predict the effect. What I see is what I get. But I don't think bash would be able to interpret this in the same way without a lot of changes. How can I put this square Docker peg into a round Bash hole?
tl;dr:
source <(sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list)
You're probably going to want to source a file, whose contents
are executed as if they were printed at the command line.
But what file? The raw docker env-file is inappropriate, because it won't export the assigned variables such that they can be used by child processes, and any of the input lines with spaces, quotes, and other special characters will have undesirable results.
Since you don't want to hand edit the file, you can use a stream editor to transform the lines to something more bash-friendly. I started out trying to solve this with one or two complex Perl 5 regular expressions, or some combination of tools, but I eventually settled on one sed command with one simple and two extended regular expressions:
sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list
This does a lot.
The first expression prepends export to any line whose first character is anything but #.
As discussed, this makes the variables available to anything else you run in this session, your whole point of being here.
The second expression simply inserts a single-quote after the first = in a line, if applicable.
This will always enclose the whole value, whereas a greedy match could lop off some of (e.g.) VAR3, for example
The third expression appends a second quote to any line that has at least one =.
it's important here to match on the = again so we don't create an unmatched quotation mark
Results:
# This is a comment, with an =' sign, just to mess with us'
export VAR1='value1'
export VAR2='value2'
export USER
export VAR3='is going to = trouble'
export VAR4='this $sign will mess with things'
export VAR5='var # with what looks like a comment'
#VAR7 ='would fail'
export VAR8=' but what about this?'
export VAR9='"and this?"'
Some more details:
By wrapping the values in single-quotes, you've
prevented bash from assuming that the words after the space are a command
appropriately brought the # and all succeeding characters into the VAR5
prevented the evaluation of $sign, which, if wrapped in double-quotes, bash would have interpreted as a variable
Finally, we'll take advantage of process substitution to pass this stream as a file to source, bring all of this down to one line of bash.
source <(sed -E -e "s/^([^#])/export \1/" -e "s/=/='/" -e "s/(=.*)$/\1'/" env.list)
Et voilà!
I need to write a shell script such that I have to read .sh script and find a particular variable (for example, Variable_Name="variable1") and take out is value(variable1).
In other shell script if Variable_Name is used I need to replace it with its Value(variable1)
A simple approach, to build on, might be:
assignment=$(echo 'Variable_Name="variable1"' | sed -r 's/Variable_Name=(.*)/\1/')
echo $assignment
"variable1"
Depending on variable type, the value might be quoted or not, quoted with single apostrophs or quotes. That might be neccessary (String with or without blanks) or superflous. Behind the assignment there might be furter code:
pi=3.14;v=42;
or a comment:
user=janis # Janis Joplin
it might be complicated:
expr="foobar; O'Reilly " # trailing blank important
But only you may know, how complicated it might get. Maybe the simple case is already sufficient. If the new script looks similar, it might work, or not:
targetV=INSERT_HERE; secondV=23
# oops: secondV accidnetally hidden:
targetV="foobar; O'Reilly " # trailing blank important; secondV=23
If the second script is under your control, you can prevent such problems easily. If source and target language are identical, what worked here should work there too.
This question already has answers here:
Why a variable assignment replaces tabs with spaces
(2 answers)
Closed 7 years ago.
I'm having some troubles with the printf function in bash.
I wrote a little script on which I pass a name and two letters (such as "sh", "py", "ht") and it creates a file in the current working directory named "name.extension".
For instance, if I execute seed test py a file named test.py is created in the current working dir with the shebang #!/usr/bin/python3.
So far, so good, nothing fancy: I'm learning shell scripting and I thought this could be a simple exercise to test the knowledge gained so far.
The problem is when I want to create an HTML file. This is the function that I use:
creaHtml(){
head='<!--DOCTYPE html-->\n<html>\n\t<head>\n\t\t<meta charset=\"UTF-8\">\n\t</head>\n\t<body>\n\t</body>\n</html>'
percorso=$CARTELLA_CORRENTE/$NOME_FILE.html
printf $head>>$percorso
chmod 755 $percorso
}
If I run, for instance, seed test ht the correct function (creaHtml) is called, test.html is created but if I try to look into it I only see:
<!--DOCTYPE
And nothing else.
This is the trace for that function:
[sviluppo:~/bin]$ seed test ht
+ creaHtml
+ head='<!--DOCTYPE html-->\n<html>\n\t<head>\n\t\t<meta charset=\"UTF-8\">\n\t</head>\n\t<body>\n\t</body>\n</html>'
+ percorso=/home/sviluppo/bin/test.html
+ printf '<!--DOCTYPE' 'html-->\n<html>\n\t<head>\n\t\t<meta' 'charset=\"UTF-8\">\n\t</head>\n\t<body>\n\t</body>\n</html>'
+ chmod 755 /home/sviluppo/bin/test.html
+ set +x
However, if I try to run printf '<!--DOCTYPE html-->\n<html>\n\t<head>\n\t\t<meta charset=\"UTF-8\">\n\t</head>\n\t<body>\n\t</body>\n</html>' from the terminal, I see the correct output: the "skeleton" of an HTML file neatly displayed with indentation and everything. What am I missing here?
Try echo -e instead of printf. printf is for printing formatted strings. Since you didn't protect $head with quotes, bash splits the string to form the command. The first word (before first white space) forms the format string. The rest are just arguments for things you didn't specify to print.
echo -e "$head" > "$percorso"
The -e evaluates your \n into newlines. I changed your >> to > since it looks like you want this to be the whole file, rather than append to any existing file you might have.
You have to be careful with quotes in bash. One thing can become many things. This actually makes it more powerful, but it can be confusing for people learning. Notice that I also put the file name "$percorso" in double quotes too. This evaluates the variable and makes sure that it ends up as one thing. If you use single quotes, it will be one word, but not evaluated. Unlike Python, there is a big difference between single and double quotes.
If you want to use printf for compatibility as #chepner pointed out, just be sure to quote it:
printf "$head" > "$percorso"
Actually that is much simpler anyway.
The other day I stumbled upon a question on SO. If I wanted to extract the value of HOSTNAME in /etc/sysconfig/network which contains
NETWORKING=yes
HOSTNAME=foo
now I can do grep and cut to get the foo but there was some bash magic involved for a similar issue. I don't know what to search for that and I can't seem to find the question now. it involved something like #{HOSTNAME} . As if it was treating HOSTNAME as a key and foo as a value.
If that configuration file is compatible with shell syntax, simply include it as a shell script. IIRC the files in /etc/sysconfig on Red Hat-like distributions are indeed designed to be parsable by a shell. Note that this means that
If shell special characters may end up in a variable's value, they must be properly quoted. For example, var="value with spaces" requires the quotes. var="with\$dollar" requires the backslash.
The script may run arbitrary code that will be executed, so this is only ok if you trust its content.
If these assumptions are valid, then you can go the simple route:
. /etc/sysconfig/network
echo "$HOSTNAME"
Regarding the quoting and braces, see $VAR vs ${VAR} and to quote or not to quote.
I am trying to print the paths that are located in a file .chsrc which is easy to print with just echo-ing $path, but I need to add something to the end of each path that is listed. Ie:
/opt/local/bin: /usr/ucb: /usr/bin:
I can not edit or change the .chsrc file. I also tried to find something on concatenating in C Shell, but that seems to not really "exist" in C Shell from what I read. I am sorry if I sound arrogant in anyway, I am new to C Shell. If anyone has any pointers, advice is always great! Thank you!
echo "$PATH" | sed 's/:/: /g;s/ *$//'
's/'=substitute, '/:/: /'=targetPattern/replacmentPattern, 'g'=do replacment globally (on the current line), ';'= command separator, 's/ *$//'= substitute any trailing spaces at the end-of-the line '$', replacementPattern='//'=(nothing, empty, nada, zilch;-)
Best to always echo any env var surrounded by dbl-quotes unless you want any spaces in the variable to cause word splitting in the commandline evaluatio. Especially with PATH, as space char is legal in a path.
In general, concatenation works like this in CSH
set var1 = text1
set var2 = myText
echo "someText "$var1 " more stuff"$var2
# -----------^^^^^^^^^^^^^^^^--- deliberate, copy paste as is
I don't have a csh to print the output with, but cut and paste these lines and you'll see that spaces outside of " .... ", get reduced to 1 space, spaces inside of "...." stay in place, as many as you want, AND variables, bumped up next to "text Strings" do NOT insert a space char automatically, you have to put them in.
I don't see any arrogance in this question ;-)
But... before you spend 8+ years of your life using a 2nd rate shell ( ;-( ), read everything at A great Unix Primer , especially Why csh is less than perfect scripting language ;-)
P.S. Welcome to StackOverflow (S.O.) Please remeber to read the FAQs, http://tinyurl.com/2vycnvr , vote for good Q/A by using the gray triangles, http://i.imgur.com/kygEP.png , and to accept the one answer that best solves your problem, if any, by pressing the checkmark sign , http://i.imgur.com/uqJeW.png