What is the best possible way to read the data from a file using readFile and converting it to a List in groovy? - jenkins-pipeline

I'm trying to read values from text files and putting the values into the list using the below method.
def myKeys = []
new File( '/tmp/A.txt' ).eachLine { line ->
myKeys << line
}
def myValues = []
new File( '/tmp/B.txt' ).eachLine { line ->
myValues << line
}
Problem is, Jenkins doesn't allow this to run on a slave and I'm not sure how to use readFile method here because it doesn't solve the purpose. I want to create a List, which readFile couldn't do.

You can get the same result using readFile step. It reads a given file from your workspace and returns the content of the file as a string. Then you can use String.eachLine(closure) method to iterate every line and add it to the list you expect. Keep in mind one thing, however - if you want to use String.eachLine() method, you need to do it in the #NonCPS mode. Otherwise, you will get maybe a single element from the iteration at best.
Take a look at the following example:
pipeline {
agent any
stages {
stage("Read test.txt file") {
steps {
script {
final String content = readFile(file: "test.txt")
final List myKeys = extractLines(content)
echo "myKeys = ${myKeys}"
}
}
}
}
}
#NonCPS
List extractLines(final String content) {
List myKeys = []
content.eachLine { line ->
myKeys << line
}
return myKeys
}
In this example, we use simple test.text file with the following content:
$ cat test.txt
123
qwe
asd
zxc
Running this exemplary pipeline produces the following output:
Running on Jenkins in /home/wololock/.jenkins/workspace/jobA
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Read test.txt file)
[Pipeline] script
[Pipeline] {
[Pipeline] readFile
[Pipeline] echo
myKeys = [123, qwe, asd, zxc]
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
You could use a similar approach to extract keys and values from two different files, e.g.
def myKeys = extractLines(readFile(file:"/tmp/A.txt"))
def myValues = extractLines(readFile(file:"/tmp/B.txt"))

Related

Error exit code 1 code showing in Jenkins Consiole output ( I do not want to see it )

I have a job running in the Jenkins pipeline and the output is showing error exit code 1 because am using if statement to create NOT_BUILT in the stage. Is there any other way to not see the work error exit code 1. I do not want to use When statement but if possible to still use IF statement and have a blank stage but without this message error exit code 1 in the console output.
This is my script below :
if(route53 == 'false' ) {
catchError(buildResult: 'SUCCESS', stageResult: 'NOT_BUILT') {
sh "exit 1"
}
}
else if(route53 == 'true' && all == "Yes" ) {
catchError(buildResult: 'SUCCESS', stageResult: 'NOT_BUILT') {
sh "exit 1"
}
}
The result in the pipeline console output is showing this, the stage graphic is fine as it is showing a blank stage but the console output error code is what I really want to manipulate.
output result
+ exit 1
[Pipeline] }
[Pipeline] }
ERROR: script returned exit code 1
[Pipeline] }
ERROR: script returned exit code 1
[Pipeline] }
ERROR: script returned exit code 1
[Pipeline] }
When using declarative pipelines the NOT_BUILT state is preserved to a stage the was not executed because its when directive was evaluated as false, and there is not direct way to set it except with the catchError workaround. (btw you can control the error message by using error('Your Message') instead of exit 1)
Therefore, it is easiest to achieve using the when directive and also makes your pipeline more readable. If you insist on using if statements you can still use them inside a when directive with the generic expression option which allows you to run any groovy code and calculate the relevant Boolean value according to your needs.
So you can still use your original code and just update it to return a Boolean:
stage('Conditional stage') {
when {
expression {
if(route53 == 'false' ) {
return false
}
else if(route53 == 'true' && all == "Yes" ) {
return false
}
return true
}
}
steps {
...
}
}

In Jenkins pipeline, how can I convert a String to Date?

I just want to be able to convert 2019-11-05T08:43:43.488-0500 to a Date object? I see Groovy String to Date but that doesn't work in pipeline (I'm aware not all Groovy does work in pipeline).
You can use java.text.SimpleDateFormat to parse String to Date object in a Jenkins Pipipeline. And this is actually what the Date.parse(format,date) does under the hood - https://github.com/apache/groovy/blob/GROOVY_2_4_12/src/main/org/codehaus/groovy/runtime/DefaultGroovyStaticMethods.java#L186
You will need, however, approve using DateFormat.parse(date) method when you run it for the first time in the Jenkins Pipeline.
Scripts not permitted to use method java.text.DateFormat parse java.lang.String. Administrators can decide whether to approve or reject this signature.
[Pipeline] End of Pipeline
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use method java.text.DateFormat parse java.lang.String
at org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.StaticWhitelist.rejectMethod(StaticWhitelist.java:175)
When you approve it, the following code should work for you:
import java.text.SimpleDateFormat
pipeline {
agent any
stages {
stage("Test") {
steps {
script {
def date = "2019-11-05T08:43:43.488-0500"
def format = "yyyy-MM-dd'T'HH:mm:ss.SSSZ"
def parsed = new SimpleDateFormat(format).parse(date)
echo "date = ${parsed}"
}
}
}
}
}
The output:
Running on Jenkins in /home/wololock/.jenkins/workspace/pipeline-sandbox
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
date = Tue Nov 05 14:43:43 CET 2019
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

I have 3 stages to build in jenkins using pipeline code (Scripted0

I have 3 stages(a,b,c) to run on jenkins using pipeline code(scripted), I
need to run stage a,b in parallel and run c after a is success (I am
doing this using pipeline code) but in blue ocean it showing only task
name but I wanna see stage names(in this case I have only 2 tasks with 3
stages and stage a and c are in one task). can someone help how can view
all three stages according to this situation.
def stages = [failFast: false]
def testList = ["a", "b", "c"]
def tasks = [:]
tasks["a-and-c"] = {
stage ("a"){
ansiColor('xterm') {
sh " ls -lart; sleep 30 "
}
if (currentBuild.currentResult == 'SUCCESS') {
stage("c") {
ansiColor('xterm') {
sh " ls -lart "
}
}
} else {
sh 'exit'
}
}
}
tasks["c"] = {
stage ("c"){
ansiColor('xterm') {
sh " ls -lart; sleep 20"
}
}
}
parallel tasks
I am expecting to have a separate view in blueocean for all three stages,
right now I am getting a-and-c and b parallel but I looking for a,b as
parallel and c after a is success. Thank you in advance.

std::process::Command cannot run hdiutil on macOS (mount failed - No such file or directory) but the command works fine when run in the terminal

hdiutils, when fed a correct path to a valid file, returns error 2, no such file or directory. When I join the indices of the command array with " ", print them, copy them and run the exact string in a terminal, it works fine.
This is the function edited to contain only the relevant bits. In order to reproduce my error, you will need a disk image located at ~/Downloads/StarUML.dmg.
use std::env;
use std::fs;
use std::process::Command;
fn setup_downloads(download_name: &str) {
let downloads_path: String = {
if cfg!(unix) {
//these both yield options to unwrap
let path = env::home_dir().unwrap();
let mut downloads_path = path.to_str().unwrap().to_owned();
downloads_path += "/Downloads/";
downloads_path
} else {
"we currently only support Mac OS".to_string()
}
};
let files_in_downloads =
fs::read_dir(&downloads_path).expect("the read_dir that sets files_in_downloads broke");
let mut file_path: String = "None".to_string();
for file_name in files_in_downloads {
let file_name: String = file_name
.expect("the pre string result which sets file_name has broken")
.file_name()
.into_string()
.expect("the post string result which sets file_name has broken")
.to_owned();
if file_name.contains(&download_name) {
file_path = format!("'{}{}'", &downloads_path, &file_name);
}
}
let len = file_path.len();
if file_path[len - 4..len - 1] == "dmg".to_string() {
let mount_command = ["hdiutil", "mount"];
let output = Command::new(&mount_command[0])
.arg(&mount_command[1])
.arg(&file_path)
.output()
.expect("failed to execute mount cmd");
if output.status.success() {
println!(
"command successful, returns: {}",
String::from_utf8_lossy(&output.stderr).into_owned()
);
} else {
println!(
"command failed, returns: {}",
String::from_utf8_lossy(&output.stderr).into_owned()
);
}
}
}
fn main() {
setup_downloads(&"StarUML".to_string());
}
Split your Command into a variable and print it using the debugging formatter after you have specified the arguments:
let mut c = Command::new(&mount_command[0]);
c
.arg(&mount_command[1])
.arg(&file_path);
println!("{:?}", c);
This outputs
"hdiutil" "mount" "\'/Users/shep/Downloads/StarUML.dmg\'"
Note that Command automatically provides quoting for each argument, but you have added your own set of single quotes:
format!("'{}{}'", &downloads_path, &file_name);
// ^ ^
Remove these single quotes.

How do I concatenate multiple files in Gradle?

Is there an easy way to concatenate multiple text files into a single one in Gradle? The build script should look something like this:
FileCollection jsDeps = files(
'file1.js',
'file2.js'
// other files here
)
task concatenate << {
// concatenate the files to file.js
}
I am using Gradle 2.3.
leftShift / "<<" is deprecated in gradle 3.4 You may use something like:
task concatenate {
doLast {
def toConcatenate = files("filename1", "filename2", ...)
def outputFileName = "output.txt"
def output = new File(outputFileName)
output.write('') // truncate output if needed
toConcatenate.each { f -> output << f.text }
}
You can also register the files as inputs/outputs to help with incremental builds. It's especially helpful with larger files.
something like this:
task 'concatenateFiles', {
inputs.files( fileTree( "path/to/dir/with/files" ) ).skipWhenEmpty()
outputs.file( "$project.buildDir/tmp/concatinated.js" )
doLast {
outputs.files.singleFile.withOutputStream { out ->
for ( file in inputs.files ) file.withInputStream { out << it << '\n' }
}
}
}
Instead of the fileTree, it can also be replaced with sourceset/sourceset output, specific files, outputs from a different task, etc.
Gradle doc on task inputs/output
Concatenating files in groovy
The following task should do the job:
task concatenate << {
def toConcatenate = files('f1', 'f2', 'f3')
def output = new File('output')
toConcatenate.each { f -> output << f.text }
}
(new File('test.js')).text = file('test1.js').getText() + file('test2.js').getText()
UPDATE:
For collections.
(new File('test.js')).text = files('test1.js', 'test2.js').collect{it.getText()}.join("\n")

Resources