grep command usage in groovy - bash

How to write the below shell in groovy
process_name = spin_user
if grep -i ${process_name} /tmp/output.log ; then
echo "Success"
grep -i ${process_name} output.log > final_output.log
else
echo "failure"
fi

<< edited in response to comment >>
1. Pure Groovy Solution
If you just want to implement the functionality in your bash script in groovy, you can do something like this:
def processName = 'spin_user'
def outLog = new File('/tmp/output.log')
def finalLog = new File('final_output.log')
def lines = outLog.readLines()
def hasProcess = { it.toLowerCase().contains(processName) }
if(lines.any(hasProcess)) {
println "Sucess"
finalLog.text = lines.findAll(hasProcess).join('\n')
} else {
println "failure"
}
should be noted that if your log file is large, there are better ways of searching for a string that do not require you to load the entire file into memory.
2. Process Management Solution
If you were specifically looking to use the linux system grep command from within groovy, the above will naturally not help you. The following groovy code:
import java.util.concurrent.*
def processName = 'spin_user'
def outLog = '/tmp/output.log'
def finalLog = 'final_output.log'
def result = exec('grep', '-i', processName, outLog)
if (result) {
println "success"
new File(finalLog).text = result
} else {
println "failure"
}
String exec(String... args) {
def out = new StringBuffer()
def err = new StringBuffer()
def process = args.toList().execute()
process.consumeProcessOutput(out, err)
process.waitForOrKill(5000)
if (err) {
println "ERROR: executing ${args} returned ${process.exitValue()}"
println "STDERR: ${err}"
}
out
}
will execute the grep command and should get you closer to what you want.
It should be noted that the output redirection > in your shell command is as far as I know hard to do on an external process from java/groovy and we are therefore writing the output to the final_output.log file from within groovy instead of executing the command using output redirection.
I also added a five second max timeout on the grep process execution. This is not required and that line can safely be removed, it's just there as a safeguard for cases where the grep blocks indefinitely.

Related

How to return output of shell script into Jenkinsfile [duplicate]

I have something like this on a Jenkinsfile (Groovy) and I want to record the stdout and the exit code in a variable in order to use the information later.
sh "ls -l"
How can I do this, especially as it seems that you cannot really run any kind of groovy code inside the Jenkinsfile?
The latest version of the pipeline sh step allows you to do the following;
// Git committer email
GIT_COMMIT_EMAIL = sh (
script: 'git --no-pager show -s --format=\'%ae\'',
returnStdout: true
).trim()
echo "Git committer email: ${GIT_COMMIT_EMAIL}"
Another feature is the returnStatus option.
// Test commit message for flags
BUILD_FULL = sh (
script: "git log -1 --pretty=%B | grep '\\[jenkins-full]'",
returnStatus: true
) == 0
echo "Build full flag: ${BUILD_FULL}"
These options where added based on this issue.
See official documentation for the sh command.
For declarative pipelines (see comments), you need to wrap code into script step:
script {
GIT_COMMIT_EMAIL = sh (
script: 'git --no-pager show -s --format=\'%ae\'',
returnStdout: true
).trim()
echo "Git committer email: ${GIT_COMMIT_EMAIL}"
}
Current Pipeline version natively supports returnStdout and returnStatus, which make it possible to get output or status from sh/bat steps.
An example:
def ret = sh(script: 'uname', returnStdout: true)
println ret
An official documentation.
quick answer is this:
sh "ls -l > commandResult"
result = readFile('commandResult').trim()
I think there exist a feature request to be able to get the result of sh step, but as far as I know, currently there is no other option.
EDIT: JENKINS-26133
EDIT2: Not quite sure since what version, but sh/bat steps now can return the std output, simply:
def output = sh returnStdout: true, script: 'ls -l'
If you want to get the stdout AND know whether the command succeeded or not, just use returnStdout and wrap it in an exception handler:
scripted pipeline
try {
// Fails with non-zero exit if dir1 does not exist
def dir1 = sh(script:'ls -la dir1', returnStdout:true).trim()
} catch (Exception ex) {
println("Unable to read dir1: ${ex}")
}
output:
[Pipeline] sh
[Test-Pipeline] Running shell script
+ ls -la dir1
ls: cannot access dir1: No such file or directory
[Pipeline] echo
unable to read dir1: hudson.AbortException: script returned exit code 2
Unfortunately hudson.AbortException is missing any useful method to obtain that exit status, so if the actual value is required you'd need to parse it out of the message (ugh!)
Contrary to the Javadoc https://javadoc.jenkins-ci.org/hudson/AbortException.html the build is not failed when this exception is caught. It fails when it's not caught!
Update:
If you also want the STDERR output from the shell command, Jenkins unfortunately fails to properly support that common use-case. A 2017 ticket JENKINS-44930 is stuck in a state of opinionated ping-pong whilst making no progress towards a solution - please consider adding your upvote to it.
As to a solution now, there could be a couple of possible approaches:
a) Redirect STDERR to STDOUT 2>&1
- but it's then up to you to parse that out of the main output though, and you won't get the output if the command failed - because you're in the exception handler.
b) redirect STDERR to a temporary file (the name of which you prepare earlier) 2>filename (but remember to clean up the file afterwards) - ie. main code becomes:
def stderrfile = 'stderr.out'
try {
def dir1 = sh(script:"ls -la dir1 2>${stderrfile}", returnStdout:true).trim()
} catch (Exception ex) {
def errmsg = readFile(stderrfile)
println("Unable to read dir1: ${ex} - ${errmsg}")
}
c) Go the other way, set returnStatus=true instead, dispense with the exception handler and always capture output to a file, ie:
def outfile = 'stdout.out'
def status = sh(script:"ls -la dir1 >${outfile} 2>&1", returnStatus:true)
def output = readFile(outfile).trim()
if (status == 0) {
// output is directory listing from stdout
} else {
// output is error message from stderr
}
Caveat: the above code is Unix/Linux-specific - Windows requires completely different shell commands.
this is a sample case, which will make sense I believe!
node('master'){
stage('stage1'){
def commit = sh (returnStdout: true, script: '''echo hi
echo bye | grep -o "e"
date
echo lol''').split()
echo "${commit[-1]} "
}
}
For those who need to use the output in subsequent shell commands, rather than groovy, something like this example could be done:
stage('Show Files') {
environment {
MY_FILES = sh(script: 'cd mydir && ls -l', returnStdout: true)
}
steps {
sh '''
echo "$MY_FILES"
'''
}
}
I found the examples on code maven to be quite useful.
All the above method will work. but to use the var as env variable inside your code you need to export the var first.
script{
sh " 'shell command here' > command"
command_var = readFile('command').trim()
sh "export command_var=$command_var"
}
replace the shell command with the command of your choice. Now if you are using python code you can just specify os.getenv("command_var") that will return the output of the shell command executed previously.
How to read the shell variable in groovy / how to assign shell return value to groovy variable.
Requirement : Open a text file read the lines using shell and store the value in groovy and get the parameter for each line .
Here , is delimiter
Ex: releaseModule.txt
./APP_TSBASE/app/team/i-home/deployments/ip-cc.war/cs_workflowReport.jar,configurable-wf-report,94,23crb1,artifact
./APP_TSBASE/app/team/i-home/deployments/ip.war/cs_workflowReport.jar,configurable-temppweb-report,394,rvu3crb1,artifact
========================
Here want to get module name 2nd Parameter (configurable-wf-report) , build no 3rd Parameter (94), commit id 4th (23crb1)
def module = sh(script: """awk -F',' '{ print \$2 "," \$3 "," \$4 }' releaseModules.txt | sort -u """, returnStdout: true).trim()
echo module
List lines = module.split( '\n' ).findAll { !it.startsWith( ',' ) }
def buildid
def Modname
lines.each {
List det1 = it.split(',')
buildid=det1[1].trim()
Modname = det1[0].trim()
tag= det1[2].trim()
echo Modname
echo buildid
echo tag
}
If you don't have a single sh command but a block of sh commands, returnstdout wont work then.
I had a similar issue where I applied something which is not a clean way of doing this but eventually it worked and served the purpose.
Solution -
In the shell block , echo the value and add it into some file.
Outside the shell block and inside the script block , read this file ,trim it and assign it to any local/params/environment variable.
example -
steps {
script {
sh '''
echo $PATH>path.txt
// I am using '>' because I want to create a new file every time to get the newest value of PATH
'''
path = readFile(file: 'path.txt')
path = path.trim() //local groovy variable assignment
//One can assign these values to env and params as below -
env.PATH = path //if you want to assign it to env var
params.PATH = path //if you want to assign it to params var
}
}
Easiest way is use this way
my_var=`echo 2`
echo $my_var
output
: 2
note that is not simple single quote is back quote ( ` ).

Running bash script from pipeline always hangs

I've created a simple pipeline which is attempting to run a script and then I'll do something else with the output, however the script (CheckTagsDates.sh) never finishes according to Jenkins. If I SSH into the Jenkins slave node, su as the jenkins user, navigate to the correct workspace folder, I can execute the command successfully.
pipeline {
agent {label 'agent'}
stages {
stage('Check for releases in past 24hr') {
steps{
sh 'chmod +x CheckTagsDates.sh'
script {
def CheckTagsDates = sh(script: './CheckTagsDates.sh', returnStdout: true)
echo "${CheckTagsDates}"
}
}
}
}
}
Here is the contents of the CheckTagsDates.sh file
#!/bin/bash
while read line
do
array[ $i ]="$line"
(( i++ ))
done < <( curl -L -s 'https://registry.hub.docker.com/v2/repositories/library/centos/tags'|jq -r '."results"[] | "\(.name)&\(.last_updated)"')
for i in "${array[#]}"
do
echo $i | cut -d '&' -f 1
echo $i | cut -d '&' -f 2
done
Here is the output from the script in the console
latest
2020-01-18T00:42:35.531397Z
centos8.1.1911
2020-01-18T00:42:33.410905Z
centos8
2020-01-18T00:42:29.783497Z
8.1.1911
2020-01-18T00:42:19.111164Z
8
2020-01-18T00:42:16.802842Z
centos7.7.1908
2019-11-12T00:42:46.131268Z
centos7
2019-11-12T00:42:41.619579Z
7.7.1908
2019-11-12T00:42:34.744446Z
7
2019-11-12T00:42:24.00689Z
centos7.6.1810
2019-07-02T14:42:37.943412Z
How I told you in a comment, I think that is a wrong use of the echo instruction for string interpolation.
Jenkins Pipeline uses rules identical to Groovy for string interpolation. Groovy’s String interpolation support can be confusing to many newcomers to the language. While Groovy supports declaring a string with either single quotes, or double quotes, for example:
def singlyQuoted = 'Hello'
def doublyQuoted = "World"
Only the latter string will support the dollar-sign ($) based string interpolation, for example:
def username = 'Jenkins'
echo 'Hello Mr. ${username}'
echo "I said, Hello Mr. ${username}"
Would result in:
Hello Mr. ${username}
I said, Hello Mr. Jenkins
Understanding how to use string interpolation is vital for using some of Pipeline’s more advanced features.
Source: https://jenkins.io/doc/book/pipeline/jenkinsfile/#string-interpolation
As a workaround for this case, I would suggest you to do the parsing of the json content in Groovy, instead of shell, and limit the script to only retrieving the json.
pipeline {
agent {label 'agent'}
stages {
stage('Check for releases in past 24hr') {
steps{
script {
def TagsDates = sh(script: "curl -L -s 'https://registry.hub.docker.com/v2/repositories/library/centos/tags'", returnStdout: true).trim()
TagsDates = readJSON(text: TagsDates)
TagsDates.result.each {
echo("${it.name}")
echo("${it.last_updated}")
}
}
}
}
}
}

Jenkins pipeline I need to execute the shell command and the result is the value of def variable. What shall I do? Thank you

Jenkins pipeline I need to execute the shell command and the result is the value of def variable.
What shall I do? Thank you
def projectFlag = sh("`kubectl get deployment -n ${namespace}| grep ${project} | wc -l`")
//
if ( "${projectFlag}" == 1 ) {
def projectCI = sh("`kubectl get deployment ${project} -n ${namespace} -o jsonpath={..image}`")
echo "$projectCI"
} else if ( "$projectCI" == "${imageTag}" ) {
sh("kubectl delete deploy ${project} -n ${namespaces}")
def redeployFlag = '1'
echo "$redeployFlag"
if ( "$projectCI" != "${imageTag}" ){
sh("kubectl set image deployment/${project} ${appName}=${imageTag} -n ${namespaces}")
}
else {
def redeployFlag = '2'
}
I believe you're asking how to save the result of a shell command to a variable for later use?
The way to do this is to use some optional parameters available on the shell step interface. See https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#sh-shell-script for the documentation
def projectFlag = sh(returnStdout: true,
script: "`kubectl get deployment -n ${namespace}| grep ${project} | wc -l`"
).trim()
Essentially set returnStdout to true. The .trim() is critical for ensuring you don't pickup a \n newline character which will ruin your evaluation logic.

Using jenkins variables in mutline shells

Given the following script:
node {
def hello = "Hello"
stage("Greetings") {
echo "${hello}world!"
}
}
The logs display HelloWorld!
When I attempt to use this in a multi-line sh command
node {
def hello = "Hello"
stage("Greetings") {
sh '''
echo ${hello}world!
'''
}
}
The variable is regarded as an empty string resulting in world!
Why and how to fix it?
You can try using double quotes instead of single quotes. Try this, see if this will work.
node {
def hello = "Hello"
stage("Greetings") {
sh """
export GREETINGS=5
echo ${hello}world \$GREETINGS times!
"""
}
}

How to read the output AND the return value of a external program in shell script?

I'm making a script that reads a tracking code, looks at the results of posting the tracking to a website and prints some messages and has a return value.
Here's part of the python code:
# update return True if there was a change to the .msg file
def update(cod):
msg = extract_msg(cod)
if msg == 'ERROR':
print('ERROR: invalid code\n')
sys.exit(2)
file = open('.msg', "r+")
old_msg = file.read()
if msg == old_msg:
return False
else:
print('Previous message: ' + old_msg)
print('Latest message: ' + msg)
file = overwrite(file, msg)
file.close()
return True
def main(argv):
if len(argv) > 1:
cod_rastr = argv[1]
else:
print("Error: no arg, no code\n")
return -1
# Verify if file exists
if os.path.isfile(".msg") == False:
arq = open('.msg', 'w')
arq.close()
# post() returns the source code of the resulting page of the posted code.
cod = post(cod_rastr)
if update(cod) == False:
return 0
else:
print ('\n Message!\n')
return 1
And here, I want to read not only the prints (for the final user) but the return values (for conditional use). This script should read the output of the .py and send me an email in case there is an update from the last check (I'll put this script in the crontab):
#!/bin/bash
if [ -z "$1" ]; then
echo usage: $0 CODE
exit
fi
CODE=$1
STATUS=$(myscript.py $CODE 2>&1)
VAL=$?
FILE=$(<.msg)
# always prints 0 (zero)
echo $VAL
# I want to check for an existing update case
if [[ $STATUS == 'Message!' ]]
then
echo $STATUS
echo $FILE | mail myuser#mydomain.com -s '$CODE: Tracking status'
fi
The problem is that $? always returns 0, and my string check inside the if, is not working, because I think It reads the update() prints too, which has variables in the print.
How can I make this shell script run, without changing the python script?
Thanks in advance.
I suspect that you can do what you want with the subprocess module. Either use rc = subprocess.call(...) to get a return code while directing stdout to a file, or use p = subprocess.Popen(...) and then perhaps p.communicate to get output and p.returncode to get the returncode.

Resources