Codecov bash uploader `eval error` on `alpine:edge` docker image - bash

I'm trying to upload coverage reports to codecov.io using the codecov-bash script provided by Codecov. The bash script fails to run on Gitlab CI running an alpine:edge docker image.
Below is the error:
$ /bin/bash <(curl -s https://codecov.io/bash)
/bin/sh: eval: line 107: syntax error: unexpected "("
And here is the relevant part of my .gitlab-ci.yml file:
after_script:
- apk -U add git curl bash findutils
- /bin/bash <(curl -s https://codecov.io/bash)
Line 107 of the script is inside the show_help() function, just under This is non-exclusive, use -s "*.foo" to match specific paths.:
show_help() {
cat << EOF
Codecov Bash $VERSION
Global report uploading tool for Codecov
Documentation at https://docs.codecov.io/docs
Contribute at https://github.com/codecov/codecov-bash
-h Display this help and exit
-f FILE Target file(s) to upload
-f "path/to/file" only upload this file
skips searching unless provided patterns below
-f '!*.bar' ignore all files at pattern *.bar
-f '*.foo' include all files at pattern *.foo
Must use single quotes.
This is non-exclusive, use -s "*.foo" to match specific paths.
-s DIR Directory to search for coverage reports.
Already searches project root and artifact folders.
-t TOKEN Set the private repository token
(option) set environment variable CODECOV_TOKEN=:uuid
-t #/path/to/token_file
-t uuid
-n NAME Custom defined name of the upload. Visible in Codecov UI
-e ENV Specify environment variables to be included with this build
Also accepting environment variables: CODECOV_ENV=VAR,VAR2
-e VAR,VAR2
-X feature Toggle functionalities
-X gcov Disable gcov
-X coveragepy Disable python coverage
-X fix Disable report fixing
-X search Disable searching for reports
-X xcode Disable xcode processing
-X network Disable uploading the file network
-X gcovout Disable gcov output
-X html Enable coverage for HTML files
-X recursesubs Enable recurse submodules in git projects when searching for source files
-N The commit SHA of the parent for which you are uploading coverage. If not present,
the parent will be determined using the API of your repository provider.
When using the repository provider's API, the parent is determined via finding
the closest ancestor to the commit.
-R root dir Used when not in git/hg project to identify project root directory
-F flag Flag the upload to group coverage metrics
-F unittests This upload is only unittests
-F integration This upload is only integration tests
-F ui,chrome This upload is Chrome - UI tests
-c Move discovered coverage reports to the trash
-Z Exit with 1 if not successful. Default will Exit with 0
-- xcode --
-D Custom Derived Data Path for Coverage.profdata and gcov processing
Default '~/Library/Developer/Xcode/DerivedData'
-J Specify packages to build coverage. Uploader will only build these packages.
This can significantly reduces time to build coverage reports.
-J 'MyAppName' Will match "MyAppName" and "MyAppNameTests"
-J '^ExampleApp$' Will match only "ExampleApp" not "ExampleAppTests"
-- gcov --
-g GLOB Paths to ignore during gcov gathering
-G GLOB Paths to include during gcov gathering
-p dir Project root directory
Also used when preparing gcov
-k prefix Prefix filepaths to help resolve path fixing: https://github.com/codecov/support/issues/472
-x gcovexe gcov executable to run. Defaults to 'gcov'
-a gcovargs extra arguments to pass to gcov
-- Override CI Environment Variables --
These variables are automatically detected by popular CI providers
-B branch Specify the branch name
-C sha Specify the commit sha
-P pr Specify the pull request number
-b build Specify the build number
-T tag Specify the git tag
-- Enterprise --
-u URL Set the target url for Enterprise customers
Not required when retrieving the bash uploader from your CCE
(option) Set environment variable CODECOV_URL=https://my-hosted-codecov.com
-r SLUG owner/repo slug used instead of the private repo token in Enterprise
(option) set environment variable CODECOV_SLUG=:owner/:repo
(option) set in your codecov.yml "codecov.slug"
-S PATH File path to your cacert.pem file used to verify ssl with Codecov Enterprise (optional)
(option) Set environment variable: CODECOV_CA_BUNDLE="/path/to/ca.pem"
-U curlargs Extra curl arguments to communicate with Codecov. e.g., -U "--proxy http://http-proxy"
-A curlargs Extra curl arguments to communicate with AWS.
-- Debugging --
-d Don't upload, but dump upload file to stdout
-q PATH Write upload file to path
-K Remove color from the output
-v Verbose mode
EOF
}
I've tried many things to solve the issue, but I can't find a solution. On their GitHub repo, there is this issue that seems linked but the proposed solution has not worked for me: Failing on busybox 1.26, incorrect flags passed to find.
You can find the full log of the job here, line 434: https://gitlab.com/gaspacchio/back-to-the-future/-/jobs/788303704

Based on KamilCuk's comment, below is the full line needed to properly upload code coverage reports to codecov:
bash -c '/bin/bash <(curl -s https://codecov.io/bash)'
As pointed out by KamilCuk, notice the closing '.
The -c flag is documented as such in the man pages for bash:
-c string
If the -c option is present, then commands are read from string. If there are arguments after the string, they are assigned to the positional parameters, starting with $0.
As of today, I don't know why this works. Feel free to edit this answer if you have any clues.

Related

Cleaning up file based variables. ERROR: Job failed: exit code 1

I am using gitlab-ci to automate build and release. In the last job, I want to upload the artifacts to a remote server using lftp.
$(pwd)/publish/ address is the artifacts that were generated in the previous job. And all variables declared in the gitlab Settings --> CI / CD.
This the job yaml code:
upload-job:
stage: upload
image: mwienk/docker-lftp:latest
tags:
- dotnet
only:
- master
script:
- lftp -e "open $HOST; user $FTP_USERNAME $FTP_PASSWORD; mirror -X .* -X .*/ --reverse --verbose --delete $(pwd)/publish/ wwwroot/; bye"
Note that lftp transfers my files, however, I'm not sure all of my files are transferred.
I added echo "All files Transfered." but it never runs.
There is no error and warning in the pipeline log but I got the following error:
I don't know what is it for. Have anyone faced the error and have found any solution?
Finally, I solved the problem by some changes in lft command parameters.
The point in troubleshooting ERROR: Job failed: exit code 1 is that to use commands with the verbose parameter that return sufficient log to enable you to troubleshoot the problem. Another important point is to know how to debug shell scripts such as bash, powershell, or etc.
Also, you can run and test commands directly in a shell, it is helpful.
The following links are helpful to troubleshoot command-line scripts:
How to debug a bash script?
5 Simple Steps On How To Debug a Bash Shell Script
Use the PowerShell Debugger to Troubleshoot Scripts
For lftp logging:
How to enable lftp protocol logging?

LD_PRELOAD not working on Heroku + jemalloc + quotaguard

TL;DR: update your bin/qgtunnel.
I've recently noticed an increase in my web dyno's memory usage. After digging a bit, I could see that the LD_PRELOAD variable that should be set with heroku-buildpack-jemalloc was not set correctly. I used a tiny script (bin/show_preload) that helped me debug that and trace which program was overriding LD_PRELOAD.
#!/usr/bin/env bash
echo "buildpack=foo preload='$LD_PRELOAD' at=start-app cmd='$#'"
$#
I introduced that in our Procfile:
web: bin/show_preload bin/qgtunnel bin/show_preload bin/start-nginx bin/show_preload bin/start-pgbouncer bin/show_preload bundle exec puma -C config/puma.rb
And when lauching on heroku I can see that bin/qgtunnel overrides our LD_PRELOAD configuration.
I created a tiny helper for the time being which makes sure I keep original value as well as what is added by bin/qgtunnel:
#!/usr/bin/env bash
after_qgtunnel_script=$(mktemp)
echo <<-BASH > $after_qgtunnel_script
# Retrieve previous LD_PRELOAD value
export LD_PRELOAD="\$LD_PRELOAD $LD_PRELOAD"
# Clean after usage
rm $after_qgtunnel_script
# Start following commands
$#
BASH
chmod +x $after_qgtunnel_script
bin/qgtunnel $after_qgtunnel_script $#
If you ever need this script use it in place of bin/qgtunnel
After reaching out to Quotaguard, they patched the qgtunnel binary and there is no error anymore:
curl https://quotaguard.s3.amazonaws.com/qgtunnel-2.4.1.tar.gz | tar xz
git add bin/qgtunnel vendor/nss_wrapper/libnss_wrapper.so
git commit -m "Update qgtunnel to fix LD_PRELOAD"
NOTE: new versions may occur since that one, see the related documentation

Installing/running JQ on git bash

I'm trying to implement a git hook that edits some JSON every time I push.
I have JQ installed on my Mac using homebrew "brew install jq", but when the git hook runs my .sh I get the error
jq: command not found
My latest attempts have been to use curl to download the jq library, point to it, and run jq that way:
jq=/usr/local/Cellar/jqz
curl -L -o $jq https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
Unfortunately, this is also returning the same 'command not found' error.
Sidenote: jq=/usr/bin/jq gives me a permission error when I try to write to it
jq=/usr/local/Cellar/jqz
curl -L -o $jq https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
It looks like you are storing the binary with the name jqz. No surprise that it cannot be executed as jq; you would have to invoke it as jqz.
I don't know if /usr/local/Cellar is part of your PATH?
The canonical way would be:
jq='/usr/local/bin/jq'
curl -L -o "$jq" https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
you could also store it in the bin directory of your home folder: `jq="$HOME/bin" which should be added automatically to your PATH on most installations (might require a logout & login).

Teamcity with Subversion post commit script on windows

We would like Teamcity to build our solutions on every commit into subversion.
Following the documentation, we are to create a .sh script :-
SERVER=https://buildserver-url
USER=buildserver-user
PASS="<password>"
LOCATOR=$1
# The following is one-line:
(sleep 10; curl --user $USER:$PASS -X POST "$SERVER/app/rest/vcs-root-instances/commitHookNotification?locator=$LOCATOR" -o /dev/null) >/dev/null 2>&1 <&1 &
exit 0
Subversion is running on a windows environment, and so the .sh file will fail.
We are trying to convert this into a .bat file of which we have :-
set SERVER=https://buildserver-url
set USER=buildserver
set PASS=password
LOCATOR=%1%
(timeout 10; curl --user %USER%:%PASS% -X POST "%SERVER%/app/rest/vcs-root-instances/commitHookNotification?locator=%LOCATOR%" -o /dev/null) >/dev/null 2>%1% <%1% &
exit 0
However, this is still failing when trying to execute with
"The system cannot find the path specified"
It seems that perhaps we havnt converted this correctly?
Are the programs you're referencing (such as curl and timeout.exe) in locations that are present in the $PATH/%PATH% variable? How about any other files you're referencing - are you specifying full paths
Side note: Did you install curl and timeout.exe on the Windows server?
Also, /dev/null does not exist on Windows; you need to redirect to NUL. You can't just change the file extension and some of your syntax and expect a bash script to work on Windows.
Were I in your shoes, I'd skip batch altogether and write the script in something modern and sane like Powershell.

SVN filtering commits by messages

I have to accept or not accept the commit on a particular repository based on the comments with the commit (using hooks). I don't know how to do it. I have to do it on a Windows device. I read somewhere that I should modify the pre-commit.tmpl file to accept just that word as the commit so I did modify this statement:
SVNLOOK=/usr/local/bin/svnlook
$SVNLOOK log -t "$TXN" "$REPOS" | \
grep ""[a-zA-Z0-9]"" > /dev/null || exit 1
into this:
SVNLOOK=/usr/local/bin/svnlook
$SVNLOOK log -t "$TXN" "$REPOS" | \
grep "^.*hello.*$" > /dev/null || exit 1
Also, it says to change the .tmpl extension for windows. But I don't know if a grep search is right also, what is the other alternative to doing the same task?
The exampels inside the .tmpl files are made for unix and using unix commands. You need to install the appropriate unix tools and adapt the scripts to your architecture(modifying paths etc..)
On windows you also need to rename the file to .bat so it is executable.
Note that no environment variables are available in hook scripts.
I would recommend to use python as a platform independent way of providing hook scripts. There are tons of python hook scripts available.

Resources