This is a system Octopus Deploy Variable:
#{Octopus.Action[Deploy To Server].Output.Package.InstallationDirectoryPath}
The text "Deploy to Server" is the name of the step in my project that deploys the Nuget Package to the server. This variable gives the install location of the NugetPackage.
I am wondering if I can make this more generic:
#{Octopus.Action[#{DeploymentStep}].Output.Package.InstallationDirectoryPath}
#{DeploymentStep} is itself a variable with the value of "Deploy to Server"?
I tried this and it not did do the substitution when it tried to run. But I am hoping there is a different syntax for variable in variable substitution.
(I want to do this so I can make this the default value for a Step Template.)
It can be done; but you need to use slightly different syntax!
Variable substitution syntax: http://docs.octopusdeploy.com/display/OD/Variable+Substitution+Syntax
$deploymentStep = "#{DeploymentStep}"
$installationDirectory = $OctopusParameters["Octopus.Action[$deploymentStep].Output.Package.InstallationDirectoryPath"]
Have just had the same issue, but with a few tests I got it working in a 1 liner.
You need to encapsulate the inner variable in brackets with a dollar, and you need to change the double quotes within the variables to single quotes so it doesn't complain about the miss match of quotes. Double quotes on the outside and single quotes in the in.
The example below gets a step name with an octopus variable and also a the machine name it ran on variable to produce the result:
$OctopusParameters["Octopus.Action[$($OctopusParameters['Octopus.Step.Name'])].Output[$($OctopusParameters['Octopus.Machine.Name'])].MyVarFromMachineFromStep"]
Related
I have a situation where I have an environment variable with space character inside. Some tools do not like quoting the value of the variable, as they will treat the quote as part of the variable.
This is set in a .env file.
PIP_EXTRA_INDEX_URL="https://token#repo https://token#repo"
When I include and export this .env file in a Makefile, I get this warning.
WARNING: Location '"https://token#repo' is ignored:
it is either a non-existing path or lacks a specific scheme.
But I have seen this behavior as initially mentioned also with other tools. Is there a way to handle this?
In the Makefile, I include it like below.
include .env
export
build:
docker build --build-arg PIP_EXTRA_INDEX_URL -t myimage .
Makefiles are not shell scripts and it is not possible to use the same syntax to define variables in both the shell and in make, except in very limited situations.
In the shell, you can have multiple assignments on the same line or even run programs on the same line. So, if your assignment has whitespace in it you have to quote it as you've done here.
In make, the syntax of an assignment is that all text after the assignment (and leading whitespace) becomes the value of the variable and there is no quoting needed; any quotes that are seen are kept as part of the variable value.
So, in the shell this assignment:
PIP_EXTRA_INDEX_URL='https://token#repo https://token#repo'
sets the shell variable PIP_EXTRA_INDEX_URL to the value https://token#repo https://token#repo ... note the quotes are stripped from the value by the shell.
In make this assignment:
PIP_EXTRA_INDEX_URL='https://token#repo https://token#repo'
sets the shell variable PIP_EXTRA_INDEX_URL to the value 'https://token#repo https://token#repo' ... note the quotes are not stripped from the value by make.
So if you use this value in a recipe like this:
do something "$(PIP_EXTRA_INDEX_URL)"
then make will expand that variable and you'll get:
do something "'https://token#repo https://token#repo'"
(including quotes) and that's your problem.
It works like this.
build:
docker build --build-arg PIP_EXTRA_INDEX_URL=$(PIP_EXTRA_INDEX_URL) -t myimage .
I've got a CI script setup something like this with 3 files :
# file Vars
.def-vars:
STAGING_SSH_DEST: mysite.com
PROJECT_ROOT: myRoot
# file gitlab-ci
variables:
extends: .def-vars
STAGING_SSH_DEST: myrealsite.com
PROJECT_ROOT: /myRealRoot
deploy-stage:
extends: .deploy
variables:
SSH_DESTINATION: $STAGING_SSH_DEST
# file deploy
.deploy:
variables:
SSH_DESTINATION: mysite.com
RSYNC_DESTINATION: $SSH_DESTINATION:$PROJECT_ROOT
I have my files and variables split up like this to increase the re-usability of the scripts.
The idea was that since I have multi site destinations, staging prod, I want to be able to pass the ssh destination to each and have the job figure out the rsync on its own. Problem is, the variable expansion is not working the way I'd think it would.
In the deploy script I added a print and got the following :
$ echo $SSH_DESTINATION # This is the variable name local to job
myrealsite.com # Yep! printed the passed in value
$ echo $RSYNC_DESTINATION # $SSH_DESTINATION:$PROJECT_ROOT
$STAGING_SSH_DEST:/myRealRoot # That is the name of the variable passed in
The root and SSH_DESTINATION print just fine. When appending the two, the former seems to be expanded one too few times.
I've had the idea to just create the rsync variable within the script section but I'd like to avoid this as I want to be able to override the rsync variable without editing the .deploy job.
How can this be accomplished?
There is an issue with Gitlab CI variables, preventing you to expand variables correctly with your extends setup.
Your options to solve this are:
Use this solution with the help of before_script, posted inside aforementioned issue. Some limitations are there, but for simple stuff it is working pretty good.
before_script:
- export VAR1="${CI_PIPELINE_ID}"
- export VAR2="test-${VAR1}"
Do some kind of preparation via .env artifacts and downstream pipelines. This one is tougher to setup, but it will allow to create dynamic tasks (and pipelines), for example spawning multi-stage deploy after successful build.
"Setting a variable within another variable" should be easier with GitLab 15.6 (November 2022)
Support for special characters in CI/CD variables
Previously, it was difficult to use the $ character in a CI/CD variable because $ normally signifies the start of another variable.
GitLab would interpret it as a variable and try to expand it.
In this release, we are introducing the variable: expand: keyword which will allow you to mark a variable as “raw”.
A raw variable can contain any special characters and is not expanded when passed to the GitLab runner.
See Documentation and Issue.
And:
Example of variables:expand:
variables:
VAR1: value1
VAR2: value2 $VAR1
VAR3:
value: value3 $VAR1
expand: false
The result of VAR2 is value2 value1.
The result of VAR3 is value3 $VAR1.
I have Terraform that is using a templated bash script to set my user data section for an AWS launch configuration.
data "template_file" "user_data" {
template = "${file("${path.module}/user-data.tpl")}"
vars {
file_system_id = "${aws_efs_mount_target.my_efs_alpha.dns_name}"
}
}
The file_system_id variable then needs to be used in my template:
sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 $$$${file_system_id}:/ /mnt/efs
Bash will interpret a single dollar sign as a bash variable. As I understand it, Terraform will interpret a double-dollar-sign as a Terraform variable. For added fun, the dollar signs in the template need to be escaped with another dollar sign -- hence the 4 dollar signs in front of file_system_id.
Looking at the user data in my Launch Config over in AWS Console, Terraform does not appear to be making any effort to replace my $$$${file_system_id) with the variable value from my template_file definition. Rather, it just shows up in the user data section as literally $${file_system_id}.
So, the question is, how do I get my EFS DNS name (or whatever other value I want) to replace the file_system_id variable in my template? What have I missed?
As BMW mentioned, you don't need to escape the dollar signs. ${file_system_id} works just fine.
Terraform's variable-replacement in templates will run first so you don't need to worry about how Bash will parse it until after the variables are replaced.
Is it possible to concatenate a string to call a Bamboo variable.
Using a script task in Bamboo, I want to generalize the following:
python my.py moon ${bamboo.mynamespace.moon}
to
SET planet=MOON
python my.py %planet% ${bamboo.mynamespace.%planet%}
But doing it like the second example above results in my python script receiving
${bamboo.mynamespace.%planet%}
as a string and not the value of
${bamboo.mynamespace.moon}
I know... moon is not a planet
I don't think it's going to be possible in the way how you're using it. Because once you use ${bamboo.variableName} Bamboo tries to resolve the variable and substitute it with a variable value. Since there's no variable%planet% Bamboo can't reference it.
But I think you could reorganise your solution a bit and make use environment variables (all Bamboo variables are passed to process as environment variables). So e.g. if Bamboo variable's name is variable.name you're allowed to reference to it via ${bamboo_variable_name} (bamboo prefix + all dots are replaced with underscore)
Then I can imagine you could get variable which interests you via print os.environ['bamboo_mynamespace_' + 'planet'] (more info on env variables in python here)
I'm using maven and have a list of system properties -DA=alpha, -DB=beta -DS=random_stuff that I want to pass down to unit tests when maven runs, and which cannot go into the pom files.
Surefire and Failsafe plugins both basically say you can pass values down to tests by passing them via the -DargLine system property.
Usually it's defined in the pom.xml, e.g.
<argLine>-DA=alpha -DB=beta -DS=random_stuff</argLine>
But I have also seen examples where -DargLine=... was passed in from the command line.
I believe that I need to put quotes around the system properties I want to pass in via argLine, as they are space separated:
-DargLine="-DA=alpha -DB=beta -DS=random_stuff"
And now I want to pass them in via the MAVEN_OPTS environment variable (which also "carries" other system properties, not just argLine). What I have so far is:
MAVEN_OPTS="-Dxyz=abc ... -DargLine=\"-DA=alpha -DB=beta -DS=random_stuff\""
In other words, I escaped argLine's quotes. Unfortunately, the end quote is now treated as part of -DS's value, i.e. S now gets replaced by random_stuff" - not what I want.
How can I do what I want to do, i.e. make sure A, B, and S are passed in as part of argLine, and pass argLine through as part of MAVEN_OPTS?
P.S.: I also tired inner single quotes (equally unusable result: some_stuff') and unescaped inner double quotes (MAVEN_OPTS=" ... -DargLine="-DA=alpha ..."", resulting in errors when trying to source .bash_profile), both unsuccessfully...