I am new to devops and I am having the following problem:
I have cloudbuild file which configured to generate deployment for namespace named after tag.
For example: for tag v1.0.2/host/dev it should generate new deployment at "dev" namespace.
Here is part of cloudbuild.yaml code:
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args:
- '-c'
- |
export APP_VERSION
export NAMESPACE
export CLUSTER_NAME
IFS=/ read -r APP_VERSION CLUSTER_NAME NAMESPACE <<< "$TAG_NAME"
... here is my problem ...
export ENVIRONMENT
export X
export XX
IFS=/ read -r X XX ENVIRONMENT <<< "$TAG_NAME"
...
As you can see - variable NAMESPACE is set with last part of tag (like "dev")
Later in code i need to set one more variable - ENVIRONMENT - with same value ("dev"). I did it by copy the way it done before and use some X and XX unused variables.
How can this be done in more accurate way?
I tried:
export ENVIRONMENT=NAMESPACE
Or:
export ENVIRONMENT=${NAMESPACE}
Any ideas?
This should do it:
export ENVIRONMENT=${NAMESPACE}
Basically, you are doing an
export NAMESPACE
before the said assignment and in bash NAMESPACE should be available as a variable ${NAMESPACE}
Related
I have some JobStreams which have some Jobs in it.
I intend to create a config file to be used generic for any Jobstream and another config file to be used by each Job.So I would have many job_config_files and only one generic_config_file
My question, is, can I create a config file as source by another?
for example:
Job_config_file
export NAME_JOB='PP_AAAR'
export log_dir="/data/workdir/AAAR/aaar/logs/"
export cluster_logs_dir="/hadoop/cluster_logs/AAAR/"
test_general_config_file
source "/data/workdir/user/m292121/properties/test_job_config_file"
export TIMESTAMP=`date '+%Y%m%d_%H%M%S'`
export DT_INICIO=`date "+%Y-%m-%d %H:%M:%S"`
export HOST_NAME=`hostname -s`
export DIR=$(pwd)
export ULTIMO_ERRO='EXECU��O SEM ERROS'
export DATA_PROCESSAMENTO=`date '+%Y%m%d'`
export GF_NAME_JOB = $NAME_JOB
and I want to get the value of NAME_JOB from the Job_config_file and save it in the GF_NAME_JOB var in the general_config_file.
So, my sh script would be able to get the value of GF_NAME_JOB
sh_script
#!/bin/bash
source "/data/workdir/user/m292121/properties/test_general_config_file"
echo $GF_NAME_JOB
and when I ran this, I got the error:
sh test_shell.sh
/data/workdir/user/m292121/properties/test_general_config_file: line 10: export: `=': not a valid identifier
My question is quite simple, I'm trying to learn AzureDevOps. I have a pipeline. In this pipeline I have a task with a bash script. This task basically adds files to the archive. This archive format:
I want it to be MyPackage_09192022_MyDeploymentComment.zip
For this, I created a variable called DeploymentComment in the pipeline. When I start a queque from this pipeline, I fill in the DeploymentComment field. I added this bash script to the task as filepath, so it gives the address of the file on the machine. I also gave the $DeploymentComment variable to the arguments in this task.
My script is as follows
date="$(date +"%d%m%Y")"
zipName="MyPackage_"$date"_"$1
zip -r $zipname /home/admins/myDir/*
I am waiting for the content of the $DeploymentComment variable that I gave as an argument on the Pipeline to come to the part I specified as $1 in the script. In other words, when I start the queque, when I type my1stTry in the $DeploymentComment section, I expect the zip file created when I type my1stTry
I expect it to be MyPackage_09192022_my1stTry.zip but bash does not see this variable.
I can create a zip file as MyPackage_09192022_.zip.
What am I missing, can you help me?
This will achieve your requirements:
trigger:
- none
pool:
vmImage: ubuntu-latest
variables:
- name: DeploymentComment #Define the variable
value: MyDeploymentComment
steps:
- bash: |
xxx="$(DeploymentComment)" #Use the pipeline variable
date="$(date +"%d%m%Y")"
zipName="$(System.DefaultWorkingDirectory)/MyPackage_"$date"_"$xxx
echo $zipName
zip -r $zipName.zip ./*
ls
Successfully get the value on my side:
I have got a YAML configuration file in which I need to store an API key. As I do not want to commit my API Key to my Repo I figured a Gitlab CI/CD variable would be a good option. I have configured the variable in the Gitlab UI to be:
TOKEN = "123"
My .gitlab.ci.yml file contains:
image:
name: xxx
variables:
P_TOKEN: ${TOKEN}
And my YAML file has:
spec:
command: test.sh
env_vars:
- TOKEN=${P_TOKEN}
But it just sets TOKEN in the YAML file to ${P_TOKEN} instead of the contents of ${P_TOKEN}. If I echo out my variables in my CI/CD pipeline it is set correctly.
So your YAML file is actually a template for a YAML file. You'd need to run some sort of template engine on top of it to have your ${P_TOKEN} placeholder replaced.
A very simple example using sed that might suffice for your use case:
sed -i "s/\${P_TOKEN}/$TOKEN/" your_file.yaml
i'm currently stuck with a bash script that should be able to manage permission on files and directories at the end of the processing part.
Actually i have 4 components:
the main script which source .conf file and libs (.sh), process things, and call a function "ApplyPermissionFromCSVFile" with a .csv file as argument at the end, to ensure that rights are correctly set. This function should handle the job for managing permission on files
a script called "permission_lib.sh" which contains several functions, including the "ApplyPermissionFromCSVFile" one. This script is SOURCED at the begining of main script
a .conf file which contains some path defined as variables which is SOURCED at the beginning of main script
a .csv file containing paths (including "dynamic path", which refer to variables defined in conf file) for files and directories which is READ by the ApplyPermissionFromCSVFile function
At the moment, the main script runs correctly, is able to source both conf file and lib file but when i put some debug point inside the "ApplyPermissionFromCSVFile", it appears that "dynamic path" is not interpreted by bash.
Extract of Main Script:
#########################################
includes
##################################################
# this section can _almost_ be copied as-is ;-)
nameOfThisScript=$(basename "${BASH_SOURCE[0]}")
directoryOfThisScript="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
configFile="$directoryOfThisScript/$nameOfThisScript.conf"
functionsFile="$directoryOfThisScript/safeScriptsFunctions.sh"
permissionLib="$directoryOfThisScript/permission_lib.sh"
permissionFile="$directoryOfThisScript/$nameOfThisScript.permissionFile.csv"
for fileToSource in "$configFile" "$functionsFile" "$permissionLib"; do
source "$fileToSource" 2>/dev/null || {
echo "File '$fileToSource' not found"
exit 1
}
done
#Main Loop to read CSV File is called in permissionLib.sh
ApplyPermissionFromCSVFile $permissionFile
Extract of conf file (real filename replaced for exemple):
totovariable="/usr/local"
tatavariable="$totovariable/bin"
Extract of csv file:
$totovariable;someuser;somegroup;0600
$tatavariable;someuser;somegroup;0600
Extract of permission lib file:
function ApplyPermissionFromCSVFile {
local csvFileName="$1"
local fieldNumberFileName=1
local fieldNumberOwner=2
local fieldNumberGroupOwner=3
local fieldNumberPermissions=4
while read csvLine
do
fileName=$(getFieldFromCsvLine "$fieldNumberFileName" "$csvLine")
fileOwner=$(getFieldFromCsvLine "$fieldNumberOwner" "$csvLine")
fileGroupOwner=$(getFieldFromCsvLine "$fieldNumberGroupOwner" "$csvLine")
filePermissions=$(getFieldFromCsvLine "$fieldNumberPermissions" "$csvLine")
permissionArray[0,0]="$fileName|$fileOwner|$fileGroupOwner|$filePermissions"
echo "${permissionArray[0,0]}"
done < "$csvFileName"
}
getFieldFromCsvLine() {
csvFieldSeparator=';'
fieldNumber="$1"
csvLine="$2"
echo "$csvLine" | cut -d "$csvFieldSeparator" -f "$fieldNumber"
}
Don't bother for the fact that the loop overwrite value at each iterations, it's not the purpose here (but optionnal answer :p).
Which output results:
$totovariable|someuser|somegroup|0600
$tatavariable|someuser|somegroup|0600
Changing owner to someuser:somegroup for file $tatavariable
chown: cannot access '$tatavariable': No such file or directory
Changing permissions to 0600 for file $tatavariable
chmod: cannot access '$tatavariable': No such file or directory
After some investigations an research, it seems normal as:
conf file is SOURCED (by main script)
lib file is SOURCED (by main script)
csv file is not SOURCED but READ (by function in lib). So bash consider the variables contents as "pure-string", and not variables
The issue, is that i can't clearly see how and where i should replace the "pure-string" variable by its value (defined in .conf file and sourced by main script): at the main script level ? at the lib level with global variables ?
Actual solutions that i've found:
sed substitute
use eval
Any help could be appreciated.
Solution used:
fileName=eval echo "$fileName"
As the pathes in .csv file may contain "$" Symbol, Bash parameter expansion will not work in every case.
Exemple:
with the following csv content:
$tatavariable;someuser;somegroup;0600
$totovariable/thisotherfile.txt;someuser;somegroup;0660
$totovariable;someuser;somegroup;0600
/home/someuser/lolzy.txt;someuser;somegroup;0666
And the following conf file:
totovariable="/home/someuser/fack"
tatavariable="$totovariable/thisfile.txt"
The following bash code (based on eval which is not highly recommended in most case) will work on every case (containing $ symbol or not):
#!/bin/bash
#########################
# Function
#########################
getFieldFromCsvLine() {
local csvFieldSeparator=';'
local fieldNumber="$1"
local csvLine="$2"
echo "$csvLine" | cut -d "$csvFieldSeparator" -f "$fieldNumber"
}
#########################
#Core Script
#########################
source configFile.conf
csvFileName="permissionFile.csv"
fieldNumberFileName=1
fieldNumberOwner=2
fieldNumberGroupOwner=3
fieldNumberPermissions=4
while read csvLine
do
fileName=$(getFieldFromCsvLine "$fieldNumberFileName" "$csvLine")
fileOwner=$(getFieldFromCsvLine "$fieldNumberOwner" "$csvLine")
fileGroupOwner=$(getFieldFromCsvLine "$fieldNumberGroupOwner" "$csvLine")
filePermissions=$(getFieldFromCsvLine "$fieldNumberPermissions" "$csvLine")
#Managing Variables used as 'Dynamic path'
fileName=$(eval echo "$fileName")
echo "Content of \$fileName is $fileName"
done < "$csvFileName"
Results:
[someuser#SAFEsandbox:~]$ ./simpletest.sh
Content of $fileName is /home/someuser/fack/thisfile.txt
Content of $fileName is /home/someuser/fack/thisotherfile.txt
Content of $fileName is /home/someuser/fack
Content of $fileName is /home/someuser/lolzy.txt
Whereas the following bash code (base on bash parameters expansion) will throw errors:
#!/bin/bash
#########################
# Function
#########################
getFieldFromCsvLine() {
local csvFieldSeparator=';'
local fieldNumber="$1"
local csvLine="$2"
echo "$csvLine" | cut -d "$csvFieldSeparator" -f "$fieldNumber"
}
#########################
#Core Script
#########################
source configFile.conf
csvFileName="permissionFile.csv"
fieldNumberFileName=1
fieldNumberOwner=2
fieldNumberGroupOwner=3
fieldNumberPermissions=4
while read csvLine
do
fileName=$(getFieldFromCsvLine "$fieldNumberFileName" "$csvLine")
fileOwner=$(getFieldFromCsvLine "$fieldNumberOwner" "$csvLine")
fileGroupOwner=$(getFieldFromCsvLine "$fieldNumberGroupOwner" "$csvLine")
filePermissions=$(getFieldFromCsvLine "$fieldNumberPermissions" "$csvLine")
#Managing Variables used as 'Dynamic path'
fileName=${!fileName}
echo "Content of \$fileName is $fileName"
done < "$csvFileName"
Behaviour example when .csv contains $ symbol:
[someuser#SAFEsandbox:~]$ ./simpletest.sh
./simpletest.sh: line 35: $tatavariable: bad substitution
Behaviour example when you remove the $ symbol in .csv file, but there is still incremental path notions in it:
[someuser#SAFEsandbox:~]$ ./simpletest.sh
Content of $fileName is /home/someuser/fack/thisfile.txt
./simpletest.sh: line 35: totovariable/thisotherfile.txt: bad substitution
Using bash parameter expansion (which is recommended in most case) is not easy here, because it forces to manage 2 cases in the script:
path containing a leading $ or serveral $ (contatenated variables paths)
path not containing a leading $ or serveral $ (contatenated variables paths)
Regards
I have a file /tmp/components_list with a content like this:
ComponentA: '1263'
ComponentB: '989'
ComponentC: '1354'
I want to register variables in ansible (without quotes), according to the content of file and use them in the yml code.
So, as a result I need something like that:
- name: Get variables from file
Some actions with a file /tmp/components_list
- name: Using these variables
shell: docker run --name component artifactory:5100/radware/Component:{{ComponentA}}
So it should be a number in the variable ComponentA.
How can I do it by using ansible? Thanks!
You can use include_vars statement, see also How to include vars file in a vars file with ansible?