I have written a shell script in a groovy function which should return the the output (in a single line) as abcd-def-chart but, I am getting the output as shown below in 2 different lines:
abcd-def
-chart
My groovy code:
String getChartName(Map configuration = [:]) {
if (configuration.chartName != null) {
return configuration.chartName
}
chartName = ""
if (configuration.buildSystem == 'maven') {
chartName = getMavenProjectName() + "-chart"
}
echo "chartName: ${chartName}"
return chartName
}
String getMavenProjectName() {
echo "inside getMavenProjectName +++++++"
def mavenChartName = sh returnStdout:true, script: '''
#!/bin/bash
GIT_LOG=$(env -i git config --get remote.origin.url)
basename "$GIT_LOG" .git; '''
echo "mavenChartName: ${mavenChartName}"
return mavenChartName
}
Related
How to get the line number of yml file in jenkins pipeline.
I am able to get it using groovy
def yaml =new File ('C:/Users/senthiln-1/yaml.yml')
yaml.text.lines().eachWithIndex { String line, int lineNo ->
if (line.contains('key1')) {
println "Found matching $line on line $lineNo"
}
}
But how the same I can get in Jenkins pipeline
node {
datas = readYaml file: 'release.yml'
}
Check the following code.
def data = readFile file: 'test.yml'
data.split('\n').eachWithIndex { line, i ->
if (line.contains('key3')) {
println "Found matching $line on line $i"
}
}
I need to create a file with all hashicorp vault key value pairs data using shell script.
I want to dump all the data from vault to a flat file.
please advice best way to do it.
Thanks in advance
Prudhvi
Just for keys and values you can use my little Perl script 'vault-backup', that also freezes the data using the correct vault commands.
Please note that this does NOT create a full backup of your Vault! There are no methods being backed up, or any other (unlistable) stuff outside the secrets. It's only usable for simple keys and values. It also probably isn't usable for multiline or binary values. You can patch the script to support that, if you like. ;)
#!/usr/bin/perl
#
# Usage: vault-backup [<PATH> [stdout]]
use Data::Dumper;
use Storable qw(freeze thaw);
# Set vault environment variables
# Always end with a " && " for the actual command
my $setenv =
"VAULT_ADDR=https://myvault.somewhere.com:8200 && ".
"VAULT_CA_PATH=/etc/yourcertificates/ && ";
my $path = $ARGV[0] || "secret/";
if ($path!~/\/$/) {
$path="$path/";
}
push #list, getData($path);
if ($ARGV[1] eq "stdout") {
print Dumper(\#list);
} else {
my $fn="vault-backup-frozen-".time().".dat";
open W,">$fn";
print W freeze(\#list);
close W;
print STDERR "Wrote data to $fn\n";
}
sub getData {
my $path=shift;
print STDERR "Starting getData($path)\n";
my #ret=();
my $command="$setenv vault kv list -tls-skip-verify $path | tail -n+3 ";
print STDERR "starting command: $command\n";
my #lines = `$command`;
chomp #lines;
foreach my $line (#lines) {
if ($line=~/\/$/) {
my #result = getData($path.$line);
if (scalar(#result)>0) {
# Find deeper results
push #ret, #result;
} else {
# empty final dir, no values
push #ret, { path => $path.$line };
}
} else {
# Found a key!
my $command="$setenv vault kv get -tls-skip-verify $path$line";
print STDERR "starting command: $command\n";
my $values = `$command`;
push #ret, {path=>$path.$line, value=>$values};
}
}
return #ret;
}
To restore the data, you can use the script below. It handles data only, it does not act on metadata.
#!/usr/bin/perl
# Usage: vault-restore <backup-filename>
use Data::Dumper;
use Storable qw(thaw);
my %all_entries;
# Set vault environment variables
# Always end with a " && " for the actual command
my $setenv =
"VAULT_ADDR=https://myothervault.somewhere.com:8200 && ".
"VAULT_CA_PATH=/etc/mycertificates/ && ";
# Read the data
my $fn = $ARGV[0] || die("I need a filename with the frozen data");
open F,"<$fn";
my #list = #{ thaw(join("",<F>)) };
close F;
print STDERR "Read ".scalar(#list)." entries.\n";
# Process the data
foreach my $entry (#list) {
print STDERR "\n# adding entry -> $entry->{path}\n";
addEntry($entry);
}
foreach my $path (keys %all_entries) {
my $keyvalues="";
foreach my $key (keys %{$all_entries{$path}}) {
my $value=$all_entries{$path}{$key};
$keyvalues.="'$key=$value' ";
}
print STDERR "vault kv put $path $keyvalues\n";
# `$command`;
}
sub addEntry {
my $entry=shift;
my $path = $entry->{'path'};
if ($entry->{'value'}) {
my $values = $entry->{value};
my #list=split("\n", $values);
my $metadata_engage=0;
my $data_engage=0;
foreach my $keyvalue (#list) {
if ($keyvalue=~/==== Metadata ====/) {
$metadata_engage=1;
$data_engage=0;
} elsif ($keyvalue=~/==== Data ====/) {
$metadata_engage=0;
$data_engage=1;
} elsif ($data_engage) {
my ($key,$value)=($keyvalue=~/^([^ ]+) +(.*)$/);
if ($key ne "Key" && $key ne "---") {
# print STDERR "key=$key ; value=$value\n";
$all_entries{$path}{$key}=$value;
} else {
# print STDERR "-- separator\n";
}
}
}
} else {
print STDERR "Found a final but empty path: $path\n";
}
}
How to write the below shell in groovy
process_name = spin_user
if grep -i ${process_name} /tmp/output.log ; then
echo "Success"
grep -i ${process_name} output.log > final_output.log
else
echo "failure"
fi
<< edited in response to comment >>
1. Pure Groovy Solution
If you just want to implement the functionality in your bash script in groovy, you can do something like this:
def processName = 'spin_user'
def outLog = new File('/tmp/output.log')
def finalLog = new File('final_output.log')
def lines = outLog.readLines()
def hasProcess = { it.toLowerCase().contains(processName) }
if(lines.any(hasProcess)) {
println "Sucess"
finalLog.text = lines.findAll(hasProcess).join('\n')
} else {
println "failure"
}
should be noted that if your log file is large, there are better ways of searching for a string that do not require you to load the entire file into memory.
2. Process Management Solution
If you were specifically looking to use the linux system grep command from within groovy, the above will naturally not help you. The following groovy code:
import java.util.concurrent.*
def processName = 'spin_user'
def outLog = '/tmp/output.log'
def finalLog = 'final_output.log'
def result = exec('grep', '-i', processName, outLog)
if (result) {
println "success"
new File(finalLog).text = result
} else {
println "failure"
}
String exec(String... args) {
def out = new StringBuffer()
def err = new StringBuffer()
def process = args.toList().execute()
process.consumeProcessOutput(out, err)
process.waitForOrKill(5000)
if (err) {
println "ERROR: executing ${args} returned ${process.exitValue()}"
println "STDERR: ${err}"
}
out
}
will execute the grep command and should get you closer to what you want.
It should be noted that the output redirection > in your shell command is as far as I know hard to do on an external process from java/groovy and we are therefore writing the output to the final_output.log file from within groovy instead of executing the command using output redirection.
I also added a five second max timeout on the grep process execution. This is not required and that line can safely be removed, it's just there as a safeguard for cases where the grep blocks indefinitely.
Below is my pipeline snippet and I am trying to assign RSTATE variable a value at run time. This value is basically stored in a text file but we need to grep and cut it. So a shell command output should be its value.
pipeline
{
agent any
environment
{
RSTATE = 'R4C'
ISO_REV = 'TA'
BuildSource = '18'
}
stages
{
stage('get Rstate')
{
echo env.RSTATE
}
}
}
I am trying to assign RSTATE value like:
RSTATE = sh ( script: 'grep RSTATE /proj/MM/scm/com/iv_build/mm18_1/rstate/next_rstate.txt
|cut -d "=" -f2', returnStdout: true).trim()
But this is not working.
I also tried to run a shell script but that also not works. Only hard coded value is working. Please suggest.
I tested and worksm you need to validate if your script return the value you want.
pipeline
{
agent any
environment
{
RSTATE = 'R4C'
ISO_REV = 'TA'
BuildSource = '18'
}
stages
{
stage('get Rstate')
{
steps {
script {
def RSTATE2 = sh ( script: 'echo \${RSTATE}', returnStdout: true).trim()
echo env.RSTATE
echo RSTATE2
}
}
}
}
}
Given the following script:
node {
def hello = "Hello"
stage("Greetings") {
echo "${hello}world!"
}
}
The logs display HelloWorld!
When I attempt to use this in a multi-line sh command
node {
def hello = "Hello"
stage("Greetings") {
sh '''
echo ${hello}world!
'''
}
}
The variable is regarded as an empty string resulting in world!
Why and how to fix it?
You can try using double quotes instead of single quotes. Try this, see if this will work.
node {
def hello = "Hello"
stage("Greetings") {
sh """
export GREETINGS=5
echo ${hello}world \$GREETINGS times!
"""
}
}