I am trying to run a script with pre-commit hook. Here is my script:
build_script.sh
#! /bin/bash
echo "Stage 1: Preparing CMake configuration"
.pre-commit-config.yaml
fail_fast: false
- repo: local
hooks:
- id: Generate build
name: Generate build
entry: sh build_script.sh
language: system
always_run: true
pass_filenames: false
I can see when I run command git commit -m "message", the hook calls the script and Generate build will Pass. However, I do not see the echo on the terminal. I would like to see the message, "Stage 1: Preparing CMake configuration". What is wrong with this setup?
pre-commit takes a bit from unix philosophy here -- it is silent unless there is a problem (by default).
if your script fails then the output will be displayed
you can also use --verbose to show the full output (for debugging) and optionally set verbose: true on your hook itself. note: these are both debugging options and are not intended for use outside that -- we've found that when hooks are noisy contributors tend to ignore all of the output
disclaimer: I wrote pre-commit
Related
I am used to work with virtualenvs. However for some reason I am not able to activate an env in a github action job.
In order to debug I added this step:
- name: Activate virtualenv
run: |
echo $PATH
. .venv/bin/activate
ls /home/runner/work/<APP>/<APP>/.venv/bin
echo $PATH
On the action logs I can see
/opt/hostedtoolcache/Python/3.9.13/x64/bin:/opt/hostedtoolcache/Python/3.9.13/x64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[...] # Cut here because a lot of lines are displayed. My executables are present including the one I'm trying to execute : pre-commit.
/home/runner/work/<APP>/<APP>/.venv/bin:/opt/hostedtoolcache/Python/3.9.13/x64/bin:/opt/hostedtoolcache/Python/3.9.13/x64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
So it should work...
But the next steps which is
- name: Linters
run: pre-commit
Generates those error logs
Run pre-commit
pre-commit
shell: /usr/bin/bash -e {0}
env:
[...] # private
/home/runner/work/_temp/8e893c8d-5032-4dbb-8a15-59be68cb0f5d.sh: line 1: pre-commit: command not found
Error: Process completed with exit code 127.
I have no issue if I transform the step above this way :
- name: Linters
run: .venv/bin/pre-commit
For some reason bash is not able to find my executable while the folder containing it is referenced in $PATH.
I'm sure you know that activation of a virtualenv is not magic — it just prepends …/.venv/bin/ to $PATH. Now the problematic thing in Github Action is that every run is executed by a different shell and hence every run has a default PATH as if the virtualenv was deactivated.
I see 3 ways to overcome that. The 1st you already mentioned — just use .venv/bin/<command>.
The 2nd is to activate the venv in every step:
- name: Linters
run: |
. .venv/bin/activate
pre-commit
The 3rd is: activate it once and store $PATH in a file that Actions use to restore environment variables at every step. The file is described in the docs.
So your entire workflow should looks like this:
- name: Activate virtualenv
run: |
. .venv/bin/activate
echo PATH=$PATH >> $GITHUB_ENV
- name: Linters
run: pre-commit
I am using gitlab-ci to automate build and release. In the last job, I want to upload the artifacts to a remote server using lftp.
$(pwd)/publish/ address is the artifacts that were generated in the previous job. And all variables declared in the gitlab Settings --> CI / CD.
This the job yaml code:
upload-job:
stage: upload
image: mwienk/docker-lftp:latest
tags:
- dotnet
only:
- master
script:
- lftp -e "open $HOST; user $FTP_USERNAME $FTP_PASSWORD; mirror -X .* -X .*/ --reverse --verbose --delete $(pwd)/publish/ wwwroot/; bye"
Note that lftp transfers my files, however, I'm not sure all of my files are transferred.
I added echo "All files Transfered." but it never runs.
There is no error and warning in the pipeline log but I got the following error:
I don't know what is it for. Have anyone faced the error and have found any solution?
Finally, I solved the problem by some changes in lft command parameters.
The point in troubleshooting ERROR: Job failed: exit code 1 is that to use commands with the verbose parameter that return sufficient log to enable you to troubleshoot the problem. Another important point is to know how to debug shell scripts such as bash, powershell, or etc.
Also, you can run and test commands directly in a shell, it is helpful.
The following links are helpful to troubleshoot command-line scripts:
How to debug a bash script?
5 Simple Steps On How To Debug a Bash Shell Script
Use the PowerShell Debugger to Troubleshoot Scripts
For lftp logging:
How to enable lftp protocol logging?
I am trying to integrate CodeCov in CircleCI, but the command
bash <(curl -s https://codecov.io/bash)
returns
The '<' operator is reserved for future use. when executed in the pipeline.
I am following the documentation that I found for
CodeCov: https://docs.codecov.io/v4.3.0/docs/about-the-codecov-bash-uploader
CircleCI: https://circleci.com/docs/reference-2-1/#jobs
My config.yml has the following steps:
steps:
- checkout
- run:
name: "Run Unit Tests"
command: dotnet.exe test ./UnitTests --collect:"XPlat Code Coverage"
- run:
name: Upload CodeCov.io Data
command: bash <(curl -s https://codecov.io/bash)
when: always # Uploads code coverage results, pass or fail
I have also tried with just curl -s https://codecov.io/bash but in this way I get the error
Invoke-WebRequest : Cannot process command because of one or more missing mandatory parameters: Uri.
Has anyone of you made this integration?
Thank you
Update:
I had to add the .exe to the curl command:
- run:
name: Upload Coverage Results
command: curl.exe -s https://codecov.io/bash | bash -s --
when: "always"
Now it builds but can't find the report.
==> Circle CI detected.
project root: C:/Users/circleci/project
Yaml not found, that's ok! Learn more at http://docs.codecov.io/docs/codecov-yaml
==> Running gcov in C:/Users/circleci/project (disable via -X gcov)
FIND: Parameter format not correct
==> Python coveragepy not found
==> Searching for coverage reports in:
+ C:/Users/circleci/project
--> No coverage report found.
Please visit http://docs.codecov.io/docs/supported-languages
CircleCI received exit code 0
“The '<' operator is reserved for future use.”
Try to run your command as shown below:
cmd /c "<your command>"
I have set up a gitlab runner on my windows. I have a build.sh script in which I am just echoing "Hello world". I have provided these lines in my gitlab-ci.yml file:
build:
stage: build
script:
- ./build.sh
The runner executes this job but does not print the echo command which I have mentioned in the build.sh file. But if I changed the extension to .bat it works and shows me the output. The gitlab-runner is set up for shell. What can be the possible reason? or I am missing something?
GitLab will output for anything that ends up written to STDOUT or STDERR. It's hard to say what's happening without seeing your whole script, but I imagine somehow you're not actually echoing to STDOUT. This is why the output isn't ending up in the CI output.
To test this I created a test project on GitLab.com. One difference in my test is my CI YAML script command was sh build.sh. This is because the script wasn't executable so it couldn't be executed with ./build.sh.
builds.sh file:
#!/bin/bash
echo "This is output from build.sh"
.gitlab-ci.yml file:
build:
stage: build
script:
- sh build.sh
The build output:
Running with gitlab-runner 12.3.0-rc1 (afb9fab4)
on docker-auto-scale 72989761
...
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/dblessing/ci-output-test/.git/
Created fresh repository.
Checking out cfe8a4ee as master...
Skipping Git submodules setup
$ sh build.sh
This is output from build.sh
Job succeeded
From the GitLab CI documentation the bash shell is supported on Windows.
Supported systems by different shells:
Shells Bash Windows Batch PowerShell
Windows ✓ ✓ (default) ✓
In my config.toml, I have tried:
[[runners]]
name = "myTestRunner"
url = xxxxxxxxxxxxxxxxxxx
token = xxxxxxxxxxxxxxxxxx
executor = "shell"
shell = "bash"
But if my .gitlab-ci.yml attempts to execute bash script, for example
stages:
- Stage1
testJob:
stage: Stage1
when: always
script:
- echo $PWD
tags:
- myTestRunner
And then from the folder containing the GitLab multi runner I right-click and select 'git bash here' and then type:
gitlab-runner.exe exec shell testJob
It cannot resolve $PWD, proving it is not actually using a bash executor. (Git bash can usually correctly print out $PWD on Windows.)
Running with gitlab-runner 10.6.0 (a3543a27)
Using Shell executor...
Running on G0329...
Cloning repository...
Cloning into 'C:/GIT/CI_dev_project/builds/0/project-0'...
done.
Checking out 8cc3343d as bashFromBat...
Skipping Git submodules setup
$ echo $PWD
$PWD
Job succeeded
The same thing happens if I push a commit, and the web based GitLab CI terminal automatically runs the .gitlab-ci script.
How do I correctly use the Bash terminal in GitLab CI on Windows?
Firstly my guess is that it is not working as it should (see the comment below your question). I found a workaround, maybe it is not what you need but it works. For some reason the command "echo $PWD" is concatenated after bash command and then it is executed in a Windows cmd. That is why the result is "$PWD". To replicate it execute the following in a CMD console (only bash is open):
bash && echo $PWD
The solution is to execute the command inside bash with option -c (it is not the ideal solution but it works). The .gitlab-ci.yml should be:
stages:
- Stage1
testJob:
stage: Stage1
when: always
script:
- bash -c "echo $PWD"
tags:
- myTestRunner