I'm looking at wurstmeister/kafka-docker, and found this pull suggesting using the IP address. Basically, it uses the docker info formatting:
HOSTNAME_COMMAND: "docker info -f '{{`{{.Swarm.NodeAddr}}`}}'"
I get the idea that it uses the formatting to get the address, but I don't understand what's going on with the extra {{...}} and the backticks (which would normally be an eval). docker-compose allows for ${variables}, but there's no dollar sign ($) here, so no need to escape. Why isn't this just:
HOSTNAME_COMMAND: "docker info -f '{{.Swarm.NodeAddr}}'"
I've seen this elsewhere, so I assume there's a reason.
This is basically a duplicate of this post. However, you might not find that one if you aren't aware that docker-compose files use Go templates (I didn't find any mention of it in the compose file reference). So basically, in go, the backquotes indicate a literal string, so the outer set of {{ }} is a template allowing for the backquotes to take everything between literally.
Related
I'm trying to provision a Windows virtual machine in VMWare using Salt Cloud wrapped in a bash script so that I can parameterise it but I'm having a problem with the escaping of the map_data.
my command is:
#!/bin/bash
salt salt-cloud cloud.map_run map_data='{"PROFILE":[{"HOSTNAME":{"folder":"FOLDER","devices":{"network":{"Network adapter 1":{"ip":"MYIP"}}}}}]}'
This works fine however I would like HOSTNAME, FOLDER and MYIP to be variables ($hostname $folder and $ip) and I'm struggling a bit with the escaping so that the variables are expanded and passed correctly to salt.
I have tried putting the variable inline in the command:
salt salt-cloud cloud.map_run map_data='{"PROFILE":[{"$hostname":{"folder":"$folder,"devices":{"network":{"Network adapter 1":{"ip":"$ip"}}}}}]}'
This gets as far as copying the template in the profile before bombing out with a vmware error about the variblised elements being incorrect
I have also tried to encapsulate the whole map data in a variable, escaping the double quotes and passing that, e.g,
data="'{\"PROFILE\":[{\"$hostname\":{\"folder\":\"$folder\",\"devices\":{\"network\":{\"Network adapter 1\":{\"ip\":\"$ip\"}}}}}]}'"
This appears to expand correctly if I echo it out but when I add it to my command:
salt salt-cloud cloud.map_run map_data=$data
I get the following error:
Passed invalid arguments to cloud.map_run: map_run() takes at most 1 argument (10 given)
I know that this is probably not strictly Salt's problem but I wondered if anyone out there could give me some pointers on how to proceed?
Did you try the concatenation of strings like that :
salt salt-cloud cloud.map_run map_data='{"PROFILE":[{"'$hostname'":{"folder":"'$folder',"devices":{"network":{"Network adapter 1":{"ip":"'$ip'"}}}}}]}'
I don't use the cloud app myself, so I can't test it but looking at the first command you give:
salt salt-cloud cloud.map_run map_data='{"PROFILE":[{"$hostname":{"folder":"$folder,"devices":{"network":{"Network adapter 1":{"ip":"$ip"}}}}}]}'
Because the variables are in single quotes, they won't expand. So that won't work.
The second command you gave:
data="'{\"PROFILE\":[{\"$hostname\":{\"folder\":\"$folder\",\"devices\":{\"network\":{\"Network adapter 1\":{\"ip\":\"$ip\"}}}}}]}'"
Looks correct, it will expand the variables, but compared to the first command it will also add single quotes to the string (I think you forgot to remove those?).
Also in your first command a " seems to be missing after $folder.
Fixing those mistakes gives me the command:
salt salt-cloud cloud.map_run map_data="{\"PROFILE\":[{\"$hostname\":{\"folder\":\"$folder\",\"devices\":{\"network\":{\"Network adapter 1\":{\"ip\":\"$ip\"}}}}}]}"
which I think would work. If you put an echo in front of your command, and just copy your json, you can copy/paste it into a json formatter like https://jsonformatter.curiousconcept.com/ and it will tell you if the json you used is correct. This will help you find things like missing quotes.
I have a docker-compose file with environment variables set like:
identity-api:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://0.0.0.0:80
- SpaClient=http://${ESHOP_EXTERNAL_DNS.NAME_OR_IP}:5104
It's just an example from https://github.com/dotnet-architecture/eShopOnContainers/blob/dev/src/docker-compose.override.yml.
The issue: I replaced underscore symbol with dot symbol here: ESHOP_EXTERNAL_DNS_NAME_OR_IP -> ESHOP_EXTERNAL_DNS.NAME_OR_IP.
After that, If I try to build docker-compose project in VS, it will not work. The error looks like:
Error DT1001 Invalid interpolation format for "environment" option in service "identity-api":"SpaClient=http://${ESHOP_EXTERNAL_DNS.NAME_OR_IP}:5104"
docker-compose C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Sdks\Microsoft.Docker.Sdk\build\Microsoft.VisualStudio.Docker.Compose.targets 202
So the question: what is wrong with the variable name here? Is dot symbol supported or not? Or maybe it can be escaped somehow?
In general, environment variable names with dots in them are allowed at a technical level but aren't well supported in any context, and I'd choose a different name.
In Docker Compose, it looks like the regexps used to identify variable expansions only support ASCII letters, digits, and underscores. This suggests dots in environment variable names aren't supported, and there's no way to escape them.
I think variables with dots in them also aren't supported in shell scripts, but I'm having trouble proving that to myself reading the POSIX spec. A Name also only consists of ASCII letters, digits, and underscores; parameters either have a name, a number, or a single symbol; and variables are parameters with names; but none of the formal description of this says a "name" is a Name.
One easy thing to demonstrate is running the bash shell in the ubuntu Docker image. We can use the docker run -e option to set arbitrary environment variables (that don't contain =) but we can't really use any shell features to expand variables with dots:
host$ docker run --rm -it -e foo=A -e foo.bar=B ubuntu
# Demonstrate the environment variable is set:
root#227bec28c674:/# env | grep foo
foo.bar=B
foo=A
# It won't get expanded without braces:
root#227bec28c674:/# echo $foo.bar
A.bar
# With braces, it's considered illegal syntax:
root#227bec28c674:/# echo ${foo.bar}
bash: ${foo.bar}: bad substitution
I have Terraform that is using a templated bash script to set my user data section for an AWS launch configuration.
data "template_file" "user_data" {
template = "${file("${path.module}/user-data.tpl")}"
vars {
file_system_id = "${aws_efs_mount_target.my_efs_alpha.dns_name}"
}
}
The file_system_id variable then needs to be used in my template:
sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 $$$${file_system_id}:/ /mnt/efs
Bash will interpret a single dollar sign as a bash variable. As I understand it, Terraform will interpret a double-dollar-sign as a Terraform variable. For added fun, the dollar signs in the template need to be escaped with another dollar sign -- hence the 4 dollar signs in front of file_system_id.
Looking at the user data in my Launch Config over in AWS Console, Terraform does not appear to be making any effort to replace my $$$${file_system_id) with the variable value from my template_file definition. Rather, it just shows up in the user data section as literally $${file_system_id}.
So, the question is, how do I get my EFS DNS name (or whatever other value I want) to replace the file_system_id variable in my template? What have I missed?
As BMW mentioned, you don't need to escape the dollar signs. ${file_system_id} works just fine.
Terraform's variable-replacement in templates will run first so you don't need to worry about how Bash will parse it until after the variables are replaced.
I want to do something like this in bash:
echo "Installing WordPress Development..."
noroot wp core install --url=development.local.dev --quiet --title="Local WordPress Dev" --admin_name=admin --admin_email="admin#local.dev" --admin_password="password"
As far as I understand "local" in bash is a reserved word so this code should fail because of this, right? How to escape this local so that it would be interpreted as a string?
There is no need to escape "local". This word has special meaning only when it occurs at the beginning of the line, like local SOME_VAR='some value'. Moreover, it can be used in a function only.
If you try to run the code in your question you will find out that it works and --url=development.local.dev is interpreted as a string without any special meaning.
The other day I stumbled upon a question on SO. If I wanted to extract the value of HOSTNAME in /etc/sysconfig/network which contains
NETWORKING=yes
HOSTNAME=foo
now I can do grep and cut to get the foo but there was some bash magic involved for a similar issue. I don't know what to search for that and I can't seem to find the question now. it involved something like #{HOSTNAME} . As if it was treating HOSTNAME as a key and foo as a value.
If that configuration file is compatible with shell syntax, simply include it as a shell script. IIRC the files in /etc/sysconfig on Red Hat-like distributions are indeed designed to be parsable by a shell. Note that this means that
If shell special characters may end up in a variable's value, they must be properly quoted. For example, var="value with spaces" requires the quotes. var="with\$dollar" requires the backslash.
The script may run arbitrary code that will be executed, so this is only ok if you trust its content.
If these assumptions are valid, then you can go the simple route:
. /etc/sysconfig/network
echo "$HOSTNAME"
Regarding the quoting and braces, see $VAR vs ${VAR} and to quote or not to quote.