I'm trying to include a bash script in an AWS SSM Document, via the Terraform templatefile function. In the aws:runShellScript section of the SSM document, I have a Bash for loop with an # sign that seems to be creating an error during terraform validate.
Version of terraform: 0.13.5
Inside main.tf file:
resource "aws_ssm_document" "magical_document" {
name = "magical_ssm_doc"
document_type = "Command"
document_format = "YAML"
target_type = "/AWS::EC2::Instance"
content = templatefile(
"${path.module}/ssm-doc.yml",
{
Foo: var.foo
}
)
}
Inside my ssm-doc.yaml file, I loop through an array:
for i in "$\{arr[#]\}"; do
if test -f "$i" ; then
echo "[monitor://$i]" >> $f
echo "disabled=0" >> $f
echo "index=$INDEX" >> $f
fi
done
Error:
Error: Error in function call
Call to function "templatefile" failed:
./ssm-doc.yml:1,18-19: Invalid character;
This character is not used within the language., and 1 other diagnostic(s).
I tried escaping the # symbol, like \#, but it didn't help. How do I
Although the error is pointing to the # symbol as being the cause of the error, it's the ${ } that's causing the problem, because this is Terraform interpolation syntax, and it applies to templatefiles too. As the docs say:
The template syntax is the same as for string templates in the main Terraform language, including interpolation sequences delimited with ${ ... }.
And the way to escape interpolation syntax in Terraform is with a double dollar sign.
for i in "$${arr[#]}"; do
if test -f "$i" ; then
echo "[monitor://$i]" >> $f
echo "disabled=0" >> $f
echo "index=$INDEX" >> $f
fi
done
The interpolation syntax is useful with templatefile if you're trying to pass in an argument, such as, in the question Foo. This argument could be accessed within the yaml file as ${Foo}.
By the way, although this article didn't give the answer to this exact issue, it helped me get a deeper appreciation for all the work Terraform is doing to handle different languages via the templatefile function. It had some cool tricks for doing replacements to escape for different scenarios.
Related
I have a nextflow script that creates a variable from a text file, and I need to pass the value of that variable to a command line order (which is a bioconda package). Those two processes happen inside the "script" part. I have tried to call the variable using the '$' symbol without any results, I think because using that symbol in the script part of a nextflow script is for calling variables defined in the input part.
To make myself clearer, here is a code sample of what I'm trying to achieve:
params.gz_file = '/path/to/file.gz'
params.fa_file = '/path/to/file.fa'
params.output_dir = '/path/to/outdir'
input_file = file(params.gz_file)
fasta_file = file(params.fa_file)
process foo {
//publishDir "${params.output_dir}", mode: 'copy',
input:
path file from input_file
path fasta from fasta_file
output:
file ("*.html")
script:
"""
echo 123 > number.txt
parameter=`cat number.txt`
create_report $file $fasta --flanking $parameter
"""
}
By doig this the error I recieve is:
Error executing process > 'foo'
Caused by:
Unknown variable 'parameter' -- Make sure it is not misspelt and defined somewhere in the script before using it
Is there any way to call the variable parameter inside the script without Nextflow interpreting it as an input file? Thanks in advance!
The documentation re the script block is useful here:
Since Nextflow uses the same Bash syntax for variable substitutions in
strings, you need to manage them carefully depending on if you want to
evaluate a variable in the Nextflow context - or - in the Bash
environment execution.
One solution is to escape your shell (Bash) variables by prefixing them with a back-slash (\) character, like in the following example:
process foo {
script:
"""
echo 123 > number.txt
parameter="\$(cat number.txt)"
echo "\${parameter}"
"""
}
Another solution is to instead use a shell block, where dollar ($) variables are managed by your shell (Bash interpreter), while exclamation mark (!) variables are handled by Nextflow. For example:
process bar {
echo true
input:
val greeting from 'Hello', 'Hola', 'Bonjour'
shell:
'''
echo 123 > number.txt
parameter="$(cat number.txt)"
echo "!{greeting} parameter ${parameter}"
'''
}
declare "parameter" in the top 'params' section.
params.parameter="1234"
(..)
script:
"""
(...)
create_report $file $fasta --flanking ${params.parameter}
(...)
"""
(...)
and call "nextflow run" with "--parameter 87678"
I've been trying to evaulate an expression inside a function as follows:
eval "fn() { $(cat fn.sh); }"
Where fn.sh contains the following:
#!/bin/sh
echo "You provided $1."
So that when I call:
fn "a phrase"`
it prints "You provided a phrase.". However I cannot get it to work.
What's particularly frustrating is that:
eval "$(cat fn.sh)"
works perfectly! What am I missing here?
What I've tried:
eval "fn() { \"\$(cat fn.sh)\"; }"
fn
# bash: #!/bin/sh
# echo "You provided $1."
# return 1: No such file or directory
eval "fn() { \$(cat fn.sh); }"
fn
# bash: #!/bin/sh: No such file or directory
and myriad other combinations, most of which at this point is guess work.
Found the answer:
eval "fn() { eval \"\$(cat "fn.sh")\"; }"
Mandatory reference that explains to the best degree I understand the security risks of using eval.
Just use source/. from inside the new function.
fn () {
. fn.sh "$1"
}
If the function is used often enough where you think repeated disk I/O would be an issue, the file will almost certainly be in a disk cache when you call fn.
I have a script that is (supposed to be) assigning a dynamic variable name (s1, s2, s3, ...) to a directory path:
savedir() {
declare -i n=1
sn=s$n
while test "${!sn}" != ""; do
n=$n+1
sn=s$n
done
declare $sn=$PWD
echo "SAVED ($sn): ${!sn}"
}
The idea is that the user is in a directory they'd like to recall later on and can save it to a shell variable by typing 'savedir'. It -does- in fact write out the echo statement successfully: if I'm in the directory /home/mrjones and type 'savedir', the script returns:
SAVED (s1): /home/mrjones
...and I can further type:
echo $sn
and the script returns:
s1
...but typing either...
> echo $s1
...or
echo ${!sn}
...both return nothing (empty strings). What I want, in case it's not obvious, is this:
echo $s1
/home/mrjones
Any help is greatly appreciated! [apologies for the formatting...]
To set a variable using a name stored in another variable I use printf -v, in this example:
printf -v "$sn" '%s' "$PWD"
declare here is creating a variable local to the function, which doesn't seem to be what you want. Quoting from help declare:
When used in a function, declare makes NAMEs local, as with the local
command. The -g option suppresses this behavior.
so you can either try the -g or with the printf
Use an array instead.
savedir() {
s+=("$PWD")
echo "SAVED (s[$((${#s[#]}-1))]): ${s[${#s[#]}-1]}"
}
Consider the following nonsense array:
# KIND[ID]=NAME
MONKEYS[1]="Oo Oo"
MONKEYS[2]="Aa Aa"
MONKEYS[3]="Ba Nana"
LIONS[5]="Mister Mufasa"
LIONS[7]="Cocoa Puff"
LIONS[8]="Lala Leo"
TIGERS[13]="Ben Gal"
TIGERS[15]="Tee Eye Double Guh Err"
TIGERS[22]="Oh Esex Diez Punto Cuatro"
With a given KIND and ID, I'm attempting to build a string that resembles $NAME[$ID] to get the associated name.
When explicitly stating an array name, the command behaves as expected echo "${LIONS[5]}"=>"Mister Mufasa"). However, whenever a variable is used, the shell responds with the given character in the string.
$LIONS[5] => 'e' # The fifth letter in "Mister Mufasa"
In other cases, I can't find a way to control interpolation to get the NAME
KIND="LIONS"
ID="5"
# Attempt to return value of `LIONS` when `KIND=LIONS`
echo $"${KIND}"; echo "\$${KIND}" #=> "$LIONS"
echo "$${KIND}" #=> "57800{KIND}" Interpolates "$$"
echo "\$\${KIND}"; "\$\${KIND}" #=> "$${KIND}"
I found the following works albeit "ugly"...
eval echo `echo \\$${KIND}`
However when introducing the ID things break once again:
eval echo `echo \\$${KIND}[$ID]`
#> title:5: no matches found: $LIONS[5]
#> no matches found: $LIONS[5]
I feel like I'm missing something very simple. I have a hunch I'm forgetting to escape something, but I'm not quite sure what.
Also, what "less redundant" alternatives to eval echo `echo... or eval echo `print... exist?
In bash, use indirect addressing:
REF="$KIND[$ID]" # Sets REF to "LIONS[5]"
echo "${!REF}" # Prints "Mister Mufasa"
EDIT: In zsh, use nested expansion instead:
echo "${(P)${KIND}[ID]}"
I want to save the variable name and its contents easily from my script.
Currently :-
LOGFILE=/root/log.txt
TEST=/file/path
echo "TEST : ${TEST}" >> ${LOGFILE}
Desired :-
LOGFILE=/root/log.txt
function save()
{
echo "$1 : $1" >> ${LOGFILE}
}
TEST=/file/path
save TEST
Obviously the above save function just saves TEST : TEST
Want I want it to save is TEST : /file/path
Can this be done? How? Many thanks in advance!
You want to use Variable Indirection. Also, don't use the function keyword, it is not POSIX and also not necessary as long as you have () at the end of your function name.
LOGFILE=/root/log.txt
save()
{
echo "$1 : ${!1}" >> ${LOGFILE}
}
TEST=/file/path
save TEST
Proof of Concept
$ TEST=foo; save(){ echo "$1 : ${!1}"; }; save TEST
TEST : foo
Yes, using indirect expansion:
echo "$1 : ${!1}"
Quoting from Bash reference manual:
The basic form of parameter expansion is ${parameter} [...] If the first character of parameter is an exclamation point (!), a level of variable indirection is introduced. Bash uses the value of the variable formed from the rest of parameter as the name of the variable; this variable is then expanded and that value is used in the rest of the substitution, rather than the value of parameter itself. This is known as indirect expansion
Consider using the printenv function. It does exactly what it says on the tin, prints your environment. It can also take parameters
$ printenv
SSH_AGENT_PID=2068
TERM=xterm
SHELL=/bin/bash
LANG=en_US.UTF-8
HISTCONTROL=ignoreboth
...etc
You could do printenv and then grep for any vars you know you have defined and be done in two lines, such as:
$printenv | grep "VARNAME1\|VARNAME2"
VARNAME1=foo
VARNAME2=bar