I am trying to create an AWS template using Troposphere and I am stuck at a point where I have to use my Bash Script as UserData. My main problem being Ref used in Bash Script. I am importing Bash Script as a file read and encode them to Base64 as:
UserData= Base64(Join("", [
open('path to my bashscript', "r")
.read()
.splitlines(True)
]))
My bash script has Ref used as:
#!/bin/bash
set -x
export -f check_instance_service_status
timeout Ref('ValueReferenced') bash -c "check_instance_service_status $ELB_NAME"
The output I am getting is:
"#!/bin/bash\n",
"set -x\n",
"\n",
" export -f check_instance_service_status\n",
" timeout Ref('ValueReferenced') bash -c \"check_instance_service_status $ELB_NAME\"\n",
" RC=$?\n",
However, I want the Ref to be converted to proper JSON compatible as:
" timeout \"",
{
"Ref": "ValueReferenced"
},
Related
I need to extract some variables and functions from a zsh script into a bash script. Is there any way to do this? What I've tried (some are embarrassingly wrong, but covering everything):
. /script/path.zsh (zsh-isms exist, so it fails)
exec zsh
. /script/path.zsh
exec bash
zsh << 'EOF'
. /script/path.zsh
EOF
chsh -s zsh
. /script/path.zsh
chsh -s bash
This thread is the closest I've found. Unfortunately, I have too many items to import for that to be feasible, and neither script is anywhere near a polyglot. However, the functions and variables that I need to import are polyglots.
You can "scrape" the zsh source file for what you need, then execute the code in bash using eval. Here's an example for doing this for a few functions:
File script.zsh:
test1() {
echo "Hello from test1"
}
test2() {
echo $((1 + $1))
}
File script.sh (bash):
# Specify source script and functions
source_filename="script.zsh"
source_functions=" \
test1 \
test2 \
"
# Perform "sourcing"
function_definitions="$(python -B -c "
import re
with open('${source_filename}', mode='r') as file:
content = file.read()
for func in '${source_functions}'.split():
print(re.search(func + r'\(\).*?\n}', content, flags=re.DOTALL).group())
" )"
eval "${function_definitions}"
# Try out test functions
test1 # Hello from test1
n=5
echo "$n + 1 is $(test2 $n)" # 5 + 1 is 6
Run the bash script and it will make use of the functions test1 and test2 defined in the zsh script:
bash script.sh
The above makes use of Python, specifically its re module. It simply looks for character sequences of the form funcname(), and assumes that the function ends at the first }. So it's not very general, but works if you write your functions in this manner.
I am trying to execute a shell script on a windows node using Jenkins.
The bash script uses sort -u flag in one of the steps to filter out unique elements from an existing array
list_unique=($(echo "${list[#]}" | tr ' ' '\n' | sort -u | tr '\n' ' '))
Note - shebang used in the script is #!/bin/bash
On calling the script from command prompt as - bash test.sh $arg1
I got the following error -
-uThe system cannot find the file specified.
I understand the issue was that with the above call, sort.exe was being used from command prompt and not the Unix sort command. To get around this I changed the path variable in Windows System variables and moved \cygwin\bin ahead of \Windows\System32
This fixed the issue and the above call gave me the expected results.
However, When the same script is called on this node using Jenkins, I get the same error again
-uThe system cannot find the file specified.
Jenkins stage calling the script
stage("Run Test") {
options {
timeout(time: 5, unit: 'MINUTES')
}
steps {
script {
if(fileExists("${Test_dir}")){
dir("${Test_dir}"){
if(fileExists("test.sh")){
def command = 'bash test.sh ${env.arg1}'
env.output = sh(returnStdout: true , script : "${command}").trim()
if (env.output == "Invalid"){
def err_msg = "Error Found."
sh "echo -n '" + err_msg + " ' > ${ERR_MSG_FILE}"
error(err_msg)
}
sh "echo Running tests for ${env.output}"
}
}
}
}
}
}
Kindly Help
I am trying to execute set of commands from jenkinsfile.
The problem is, when I try to assign the value of stdout to a variable it is not working.
I tried different combinations of double quotes and single quotes, but so far no luck.
Here I executed the script with latest version of jenkinsfile as well as old version. Putting shell commands inside """ """ is not allowing to create new variable and giving error like client_name command does not exist.
String nodeLabel = env.PrimaryNode ? env.PrimaryNode : "slave1"
echo "Running on node [${nodeLabel}]"
node("${nodeLabel}"){
sh "p4 print -q -o config.yml //c/test/gradle/hk/config.yml"
def config = readYaml file: 'devops-config.yml'
def out = sh (script:"client_name=${config.BasicVars.p4_client}; " +
'echo "client name: $client_name"' +
" cmd_output = p4 clients -e $client_name" +
' echo "out variable: $cmd_output"',returnStdout: true)
}
I want to assign the stdout from the command p4 clients -e $client_name to variable cmd_output.
But when I execute the code the error that is thrown is:
NoSuchPropertyException: client_name is not defined at line cmd_output = p4 clients -e $client_name
What am I missing here?
Your problem here is that all the $ are interpreted by jenkins when the string is in double quotes. So the first 2 times there's no problem since the first variable comes from jenkins and the second time it's a single quote string.
The the third variable is in a double quote string, therefore jenkins tries to replace the variable with its value but it can't find it since it's generated only when the shell script is executed.
The solution is to escape the $ in $client_name (or define client_name in an environment block).
I rewrote the block:
String nodeLabel = env.PrimaryNode ? env.PrimaryNode : "slave1"
echo "Running on node [${nodeLabel}]"
node("${nodeLabel}"){
sh "p4 print -q -o config.yml //c/test/gradle/hk/config.yml"
def config = readYaml file: 'devops-config.yml'
def out = sh (script: """
client_name=${config.BasicVars.p4_client}
echo "client name: \$client_name"
cmd_output = p4 clients -e \$client_name
echo "out variable: \$cmd_output"
""", returnStdout: true)
}
How do I escape the string to use a variable for the filename within the following script? Am I even getting the variable in a proper manner?
The scenario is I have a few dozen small flac files of voice messages. They need to be sent with the gcloud command/url individually and a return is sent and this return output is set as a variable. After I get the return response I need to send the same file and the return response to a database using the json file schema.
Edited: I have tried escaping by adding/wrapping in single quotes, double quotes and backticks. all return errors and all return errors show the url containing "$i" rather than a filename.flac.
When I try without escaping the string I get this error:
When I do it without escaping I get this:
add2.sh: line 4: syntax error near unexpected token `sudo'
add2.sh: line 4: ` sudo gcloud ml speech recognize `/var/www/html/library/422980-2560-WIN/$i --language-code='en-US' >STT.txt`'
Here is the script:
#!/bin/bash
cd /var/www/html/library/422980-2560-WIN
for i in *.flac;
sudo gcloud ml speech recognize `/var/www/html/library/422980-2560-WIN/"$i" --language-code='en-US' >STT.txt`
done
STT=`grep -Po '"transcript": *\K"[^"]*"' STT.txt | cut -d '"' -f2`
echo $STT
sudo gsutil cp /var/www/html/library/422980-2560-WIN/"$i".flac gs://422980
#rm -i -f -- /var/www/html/library/422980-2560-WIN/*.flac
sudo /usr/local/fuego --credentials /home/repeater/medialunaauth01-280236ff5e5f.json add 422980 '
{
"bucketObject": "https://storage.cloud.google.com/$i",
"fileDay": "filedaytest33",
"fileMonth": "filemonttest33",
"fileName": "filenametest33",
"fileTime": "filetimetest33",
"fileYear": "fileyeartest33",
"liveOnline": "0",
"qCChecked": "0",
"speechToText":"'"$STT"'",
"transcribedData": "",
}'
Say I have a Jenkinsfile. Within that Jenkinsfile, is the following sh step:
sh "myScript.sh"
Within myScript.sh, the following variable is declared:
MY_VARIABLE="This is my variable"
How can I access MY_VARIABLE, which is declared in myScript.sh, from my Jenkinsfile?
To import the variable defined in your script into the current shell, you can use the source command (see explanation on SU):
# Either via command
source myScript.sh
# Or via built-in synonym
. myScript.sh
Supposing your script does not output anything, you can then instead output the variable to fetch it in Jenkins:
def myVar = sh(returnStdout: true, script: '. myScript.sh && echo $MY_VARIABLE')
If indeed outputs comes from your script, you can fetch the last output either per shell:
(. myScript.sh && echo $MY_VARIABLE) | tail -n1
or via Groovy:
def out = sh(returnStdout: true, script: '. myScript.sh && echo $MY_VARIABLE')
def myVar = out.tokenize('\n')[-1]
The bash variable declared in .sh file is ending with the pipeline step: sh complete.
But you can make you .sh to generate a properties file, then use pipeline step: readProperties to read the file into object for accessing.
// myScript.sh
...
echo MY_VARIABLE=This is my variable > vars.properties
// pipeline
sh 'myScript.sh'
def props = readProperties file: 'vars.properties'
echo props.MY_VARIABLE