I have set up a gitlab runner on my windows. I have a build.sh script in which I am just echoing "Hello world". I have provided these lines in my gitlab-ci.yml file:
build:
stage: build
script:
- ./build.sh
The runner executes this job but does not print the echo command which I have mentioned in the build.sh file. But if I changed the extension to .bat it works and shows me the output. The gitlab-runner is set up for shell. What can be the possible reason? or I am missing something?
GitLab will output for anything that ends up written to STDOUT or STDERR. It's hard to say what's happening without seeing your whole script, but I imagine somehow you're not actually echoing to STDOUT. This is why the output isn't ending up in the CI output.
To test this I created a test project on GitLab.com. One difference in my test is my CI YAML script command was sh build.sh. This is because the script wasn't executable so it couldn't be executed with ./build.sh.
builds.sh file:
#!/bin/bash
echo "This is output from build.sh"
.gitlab-ci.yml file:
build:
stage: build
script:
- sh build.sh
The build output:
Running with gitlab-runner 12.3.0-rc1 (afb9fab4)
on docker-auto-scale 72989761
...
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/dblessing/ci-output-test/.git/
Created fresh repository.
Checking out cfe8a4ee as master...
Skipping Git submodules setup
$ sh build.sh
This is output from build.sh
Job succeeded
Related
I would like to add a Gitlab job:
Check if the generated files are different that the ones in git
Could you please help me?
You can use git diff --exit-code after running the generation script to assert whether files have changed.
For example, suppose you modify an existing file then run git diff --exit-code, the command will exit non-zero. In a gitlab job, that means the job will fail.
$ echo "foo" >> existing-file.txt
$ git diff --exit-code # exits non-zero (failure)
So, you could have a gitlab job that runs your generation script then checks if the files have changed. If the files don't change, the command exits 0 and the job passes.
check_openapi:
stage: .pre
# ...
script:
- ./mvnw verify # generate the openapi docs
- git diff --exit-code # fails if the files tracked by git have changed
It's important to note that git diff will only work on tracked git files. Therefore, if your generation code potentially adds new files, you should make sure that you run git add --intent-to-add for any newly created files. Otherwise, you may miss some cases because new files aren't tracked by git by default. You can add this to the CI job or you can incorporate it into your code generation script.
In example:
$ echo "foo" > newfile.txt
$ git diff --exit-code # exits 0 (success?!)
$ echo "foo" > newfile.txt
$ git add --intent-to-add ./newfile.txt
$ git diff --exit-code # exits non-zero (failure)
So, if your generation script doesn't run git add --intent-to-add as part of its process, a complete solution may look like this:
check_openapi:
stage: .pre
# ...
script:
- ./mvnw verify
- git add --intent-to-add . # make sure new files are tracked/diff'd
- git diff --exit-code
I am using gitlab-ci to automate build and release. In the last job, I want to upload the artifacts to a remote server using lftp.
$(pwd)/publish/ address is the artifacts that were generated in the previous job. And all variables declared in the gitlab Settings --> CI / CD.
This the job yaml code:
upload-job:
stage: upload
image: mwienk/docker-lftp:latest
tags:
- dotnet
only:
- master
script:
- lftp -e "open $HOST; user $FTP_USERNAME $FTP_PASSWORD; mirror -X .* -X .*/ --reverse --verbose --delete $(pwd)/publish/ wwwroot/; bye"
Note that lftp transfers my files, however, I'm not sure all of my files are transferred.
I added echo "All files Transfered." but it never runs.
There is no error and warning in the pipeline log but I got the following error:
I don't know what is it for. Have anyone faced the error and have found any solution?
Finally, I solved the problem by some changes in lft command parameters.
The point in troubleshooting ERROR: Job failed: exit code 1 is that to use commands with the verbose parameter that return sufficient log to enable you to troubleshoot the problem. Another important point is to know how to debug shell scripts such as bash, powershell, or etc.
Also, you can run and test commands directly in a shell, it is helpful.
The following links are helpful to troubleshoot command-line scripts:
How to debug a bash script?
5 Simple Steps On How To Debug a Bash Shell Script
Use the PowerShell Debugger to Troubleshoot Scripts
For lftp logging:
How to enable lftp protocol logging?
I am looking for a way to call main.go file from .gitlab-ci.yml File. Main.go accepts flag and returns boolean values.
From bash, I can execute below
go run ./cmd/main.go -timeout=10
It displays true/ false values.
Now from my .gitlab-ci.yml file, I want to call above program. We already have working set up (docker images) where all the GO/ SH binaries will be placed. So .gitlab-ci.yml file knows the location of above program. It is just that I am not finding way to call above binary in .gitlab-ci.yml inside scripts section. I should be able to capture return response of GO binary.
There is an easy way to debug this (this only works on self hosted runners)
Set in your .gitlab-ci.yml
script:
- tail -f /dev/null
- your commands
- your commands ...
After run this job, your runner will freeze and you will be able to connect inside your runner container
docker (or kubectl) exec -it <your runner container> bash
after this, you can debug and test all of your commands and you will know the reason of why your commands are not working and you can fix it later in your gitlab file
From the GitLab CI documentation the bash shell is supported on Windows.
Supported systems by different shells:
Shells Bash Windows Batch PowerShell
Windows ✓ ✓ (default) ✓
In my config.toml, I have tried:
[[runners]]
name = "myTestRunner"
url = xxxxxxxxxxxxxxxxxxx
token = xxxxxxxxxxxxxxxxxx
executor = "shell"
shell = "bash"
But if my .gitlab-ci.yml attempts to execute bash script, for example
stages:
- Stage1
testJob:
stage: Stage1
when: always
script:
- echo $PWD
tags:
- myTestRunner
And then from the folder containing the GitLab multi runner I right-click and select 'git bash here' and then type:
gitlab-runner.exe exec shell testJob
It cannot resolve $PWD, proving it is not actually using a bash executor. (Git bash can usually correctly print out $PWD on Windows.)
Running with gitlab-runner 10.6.0 (a3543a27)
Using Shell executor...
Running on G0329...
Cloning repository...
Cloning into 'C:/GIT/CI_dev_project/builds/0/project-0'...
done.
Checking out 8cc3343d as bashFromBat...
Skipping Git submodules setup
$ echo $PWD
$PWD
Job succeeded
The same thing happens if I push a commit, and the web based GitLab CI terminal automatically runs the .gitlab-ci script.
How do I correctly use the Bash terminal in GitLab CI on Windows?
Firstly my guess is that it is not working as it should (see the comment below your question). I found a workaround, maybe it is not what you need but it works. For some reason the command "echo $PWD" is concatenated after bash command and then it is executed in a Windows cmd. That is why the result is "$PWD". To replicate it execute the following in a CMD console (only bash is open):
bash && echo $PWD
The solution is to execute the command inside bash with option -c (it is not the ideal solution but it works). The .gitlab-ci.yml should be:
stages:
- Stage1
testJob:
stage: Stage1
when: always
script:
- bash -c "echo $PWD"
tags:
- myTestRunner
I have a circle.yml file like so:
dependencies:
override:
- meteor || curl https://install.meteor.com | /bin/sh
deployment:
production:
branch: "master"
commands:
- ./deploy.sh
When I push to Github, I get the error:
/home/ubuntu/myproject/deploy.sh returned exit code 126
bash: line 1: /home/ubuntu/myproject/deploy.sh: Permission denied Action failed: /home/ubuntu/myproject/deploy.sh
When I run the commands that live inside deploy.sh outside of the file (under commands) everything runs fine.
Everything in the circle.yml file seems to be in line with the examples in the CircleCI docs.. What am I doing wrong?
Several possible problems:
deploy.sh might not be marked as executable (chmod +x deploy.sh would fix this)
The first line of deploy.sh might not be a runnable shell...
If the first doesn't work, can we please see the contents of deploy.sh?
I was having the same issue. I added sh to the front of my commands section to get it to work.
deployment:
production:
branch: "master"
commands:
- sh ./deploy.sh
Hopefully that fix saves everyone sometime going forward.
Assuming you have already checked it in, use this command to flag it as executable to git:
git update-index --chmod=+x script.sh
reference:
https://www.pixelninja.me/make-script-committed-to-git-executable/
As #palfrey says the script is probably not marked as executable, and sometimes it seems to be marked wrong on deployment even when you have previously run chmod +x on your script at your local machine. (Why? I don't know. If someone does please enlighten me!)
Here is a general command to use to ensure your scripts are always marked as executable. This assumes they are all located in a /home/ubuntu/${CIRCLE_PROJECT_REPONAME}/scripts directory and all have a .sh extension. If your directory(s) is(are) different, edit to use your directory instead.
Since all my scripts source a shared script (shared.sh) at the top of each script that are called by circle.yml I add the following code to shared.sh which ensures all scripts are marked as executable:
SCRIPTS="/home/ubuntu/${CIRCLE_PROJECT_REPONAME}/scripts"
find "${SCRIPTS}" | grep "\.sh$" | xargs chmod +x
Works like a charm. :-)