I know the primary travis build logs are available on the web and with the logs command in the travis command line client, but I was wondering if there is a way to access other files generated as part of the build process, such as the file /home/travis/.rvm/log/1391454748_rbx-2.2.4/rubygems.install.log referenced in https://travis-ci.org/rspec/rspec-its/jobs/18148204
Those files are lost once the build is finished. If you want to read them, you should add a cat command to print out to the log you see.
before_script: cat /home/travis/.rvm/log/*_rbx-2.2.4/rubygems.install.log
If the install command is failing, then you should override install to install the gem for which the installation is failing:
install: gem install XXX || cat /home/travis/.rvm/log/*_rbx-2.2.4/rubygems.install.log
banzaiman's answer is good (it helped me!). But if you use:
install: gem install XXX || cat /home/travis/.rvm/log/*_rbx-2.2.4/rubygems.install.log
then the cat command will likely succeed, and so the line above will count as as success, and the build will continue. If you want the build to fail when the install fails, then you need to make sure the line has a non-zero exit status. So do something like this:
install: gem install XXX || { cat /home/travis/.rvm/log/*_rbx-2.2.4/rubygems.install.log && 1; }
The expression in curly braces will be run only if the gem install XXX fails (i.e., has a non-zero exit status). cat will presumably succeed, so the command after the && will be run. That 1 ensures a non-zero exit status for the whole line, causing the build to stop at that point.
Note the necessary whitespace around the curly braces.
Related
I have here an execute block that looks like this
execute 'uninstall_datadog' do
command 'sudo apt-get --purge remove datadog-agent -y'
only_if 'command -v datadog-agent'
end
So my understanding is if exit status of only_if is not 0 then this block will not execute. Is this true?
Yes.
However I see some problems with your example:
the built-in package resource is able to remove a package if it is installed.
package 'datadog-agent' do
action :purge
options '--yes' # not sure about this
end
If the package is not installed (anymore), chef will just skip it.
See https://docs.chef.io/resource_package.html for more details.
usually chef runs as root so the sudo command is not needed in execute commands.
only_if by default runs in the environment of the chef-client/chef-solo/chef-apply process. See https://docs.chef.io/resource_common.html#arguments you'll have to adjust environment variables like PATH when impersonating as another user
command is a bash builtin, looks like might not be executing as bash.
bash -c "command -v datadog-agent" would work, but could also use which datadog-agent instead.
I need to run a while loop to install Python dependencies. In the Python world recently there are 2 ways to install dependencies which have become established:
using conda (for some people this is the "robust/stable/desired way", provided by a "Python distribution" called Anaconda/Miniconda),
using pip (in the last few years included as the official way of Python itself).
The "pseudocode" should be:
try to install the dependency with the conda command
if it fails then install it with the pip command
In the Python world dependencies are specified in a requirements.txt file, usually exact versions (==) as one dependency per line with the pattern <MY_DEPENDENCY>==<MY_VERSION>.
The equivalent bash desired command is: while read requirement; do conda install --yes $requirement || pip install $requirement; done < requirements.txt, however this does not work in the GNU make/Makefile world for reasons that I don't completely get.
I've tried a few different flavors of that while loop - all unsuccessful. Basically once the conda command fails I am not able to go on with the pip attempt. I am not sure why this happens (as it works in "normal bash") and I can not find a way to manage some sort of low-level try/catch pattern (for those familiar with high level programming languages).
This is my last attempt which is not working because it stops when conda fails:
foo-target:
# equivalent to bash: conda install --yes $requirement || pip install $requirement;
while read requirement; do \
conda install --yes $requirement ; \
[ $$? != 0 ] || pip install $requirement; \
done < requirements.txt
How do I make sure I try to install each requirement inside requirements.txt first with conda, when conda fails then with pip?
Why is my code not working? I see people pointing to the differences between sh and bash, but I am not able to isolate the issue.
Edit:
I ended up working around using the bash command inside the Makefile, but I find this solution not ideal, because I need to maintain yet another chunk of code in a one-line bash script (see below), is there a way to keep all the stuff inside a Makefile avoiding bash at all?
The Makefile target:
foo-target:
bash install-python-dependencies.sh
The bash one line script:
#!/usr/bin/env bash
while read requirement; do conda install --yes $requirement || pip install $requirement; done < requirements.txt
I can run the script directly from the command line (bash), I can also run it from within the Makefile, but I would like to get rid of the bash script and always execute make foo-target without using bash (avoiding bash even inside the Makefile).
As shown above, your makefile will work as you expect, other than that you have to escape the $ in shell variables like $$requirement.
I couldn't reproduce your problem with a simplified example to emulate the behavior:
foo-target:
for i in 1 2 3; do \
echo conda; \
test $$i -ne 2; \
[ $$? -eq 0 ] || echo pip; \
done
gives the expected output:
$ make
conda
conda
pip
conda
Have you added the .POSIX: target to your makefile, that you don't show here? If I do that then I get the behavior you claim to see:
conda
make: *** [Makefile:2: foo-target] Error 1
The reason for this is described in the manual for .POSIX:
In particular, if this target is mentioned then recipes will be invoked as if the shell had been passed the '-e' flag: the first failing command in a recipe will cause the recipe to fail immediately.
If you want to keep .POSIX mode but not get this error the simplest way is to use the method you show in your first example; I don't know why you stopped using it:
foo-target:
while read requirement; do \
conda install --yes $$requirement || pip install $$requirement; \
done < requirements.txt
I'm writing a Pipeline script for Jenkins on Mac that needs to execute a couple of sh steps. Some of them involve RVM and Bundler ...
sh '#!/bin/bash -xl' +
' && rvm list' +
' && rvm use 2.3.1' +
' && gem install bundler' +
' && which bundler'
As you can see I have to use the hashbang to make RVM and Bundler to work, i.e having to be in a Login Shell but the problem is I don't see any log output for this in Jenkins anymore, even with -xl flag.
Does somebody know why log output is omitted and how to enable it for this?
UPDATE:
sh returnStdout: true, script: '#!/bin/bash -xl && rvm list && rvm use 2.3.1 && gem install bundler && which bundler'
Log Output:
[Pipeline] sh
[app_ios_test_automation] Running shell script
[Pipeline] echo
The "sh" function has optional parameters. If you call it like you're doing, you won't get the standard output of the script.
If you go into your pipeline job definition, where you specify the pipeline script itself, you should see a link labeled "Pipeline Syntax". This allows you to experiment with the pipeline steps that are enabled in your Jenkins instance. If you select "sh" from the dropdown and then click the "Advanced" button, you'll see the additional options you can set, including the "returnStdout" flag.
To expand on David Karr's answer, you're probably looking for something like this:
sh(returnStdout: true, script: "your script here")
I have a pretty straightforward setup..
- a Travis.yml file : https://github.com/openassistive/OpenATFrontEnd/blob/master/.travis.yml
which has this line:
before_script:
- go get -u -v github.com/spf13/hugo
but it fails - with
The command "eval go get -t -v ./..." failed. Retrying, 2 of 3.
(https://travis-ci.org/openassistive/OpenATFrontEnd/builds/166105574)
I can't figure it out. I see the language is set correctly - and looking at other SO posts the version number is correct. Is there a different version I should be using?
Read this, the go get .... is part of the default go build script on travis, if no makefile is found.
A simple solution may be to add a Makefile with an empty recipe
$ cat Makefile
target: ;
$ make && echo "ok"
make: « target » uptodate.
ok
So travis will set the default install step to true, which should avoid the got get
I'm writing a script that installs and configures Nagios to my requirements. It requires cpanm and some perl modules.
It's using the step/try/next function from here: https://stackoverflow.com/a/5196220
step "Downloading cpanm installer"
try `wget -q http://cpanmin.us -O $swrepo/cpanm.install`
next
step "Installing cpanm"
try echo '{ exec </dev/tty; cat $swrepo/cpanm.install | perl - App::cpanminus; }' | bash
# try bash -c "$(cat $swrepo/cpanm.install | perl - App::cpanminus)"
# try cat $swrepo/cpanm.install | perl - App::cpanminus
next
step "Installing Perl module Nagios Config"
try `cpanm Nagios::Config`
next
My problems here are:
whichever way I attempt to run the install for cpanminus, it fails the script, and won't install properly. I can't seem to make it function outside of the step/try/next functions (not that I want it to.)
The cpanm command fails too. If I isolate and run only this part of the script, it still fails, with "cpanm command not found." I can run it manually at the command line.
Any pointers for the slightly frustrated?
Update
I pulled the cpanm setup out to a separate file:
step "Installing cpanm"
try sh conf_cpanm.sh
next
Which works, and I'll probably try and pull it back in at a later date, but so far that functions. So it can stay.
However, doing the same for
try cpanm Nagios::Config
won't work. The file looks like this:
#!/bin/bash
cpanm Nagios::Config
...and if I run that by calling sh conf_nagcpanm.sh it works fine.
I think using backticks
try `cpanm Nagios::Config`
is a mistake. bash will take an expression in backticks, execute it, and substitute the output of the command for the expression. The output of cpanm is not going to be shell commands, so this will not work. It should simply be
try cpanm Nagios::Config