pre-commit for local hook gives error: "unrecognized arguments: .pre-commit-config.yaml" - continuous-integration

I have the following repo in pre-commit file .pre-commit-config.yaml
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements
args: ["--compare"]
But it keeps giving me the error:
error: unrecognized arguments: .pre-commit-config.yaml
As it passes the filename as an argument to my python script. How can I prevent this?

to clean up your example a little bit -- and use files to only run when the necessary files change:
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements --compare
files: ^requirements-dev.txt$
pass_filenames: false
note that I did a couple things:
args doesn't really make sense for local hooks, you can just put that in entry
pass_filenames (as you did) -- pre-commit is a framework based on passing filenames to executables, but you can turn that off
files: this will make it so the hook only gets triggered if requirements-dev.txt changes
alternatively (if you expect changes outside requirements-dev.txt to need to run this hook) you can drop files and use always_run: true
disclaimer: I'm the author of pre-commit

I spent quite some time figuring out what caused this and how to solve this. It's not documented well, eventually I fixed it by trial and error. We have to use pass_filenames: false in our hook:
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements
pass_filenames: false
args: ["--compare"]

Related

GoReleaser and ssh-agent Github Actions: Why could not read Username ... terminal prompts disabled?

I have been going around and around between instructions for GitHub Actions, GoReleaser, and Ssh-Agent and cannot get my simple release build script to work. My goal is simple... I have a go private repository containing a CLI application and its go.mod file has a dependency on another private repository that we've created. Building the application locally is successful.
The issue is that when I try to build this simple application in a GitHub Action, things become really complicated very quickly... repository secrets, deploy key, an a few other moving parts. As common as this use-case is, I failed to find a single example where someone has implemented a release build script for it... I am about ready to switch to a mono-repo out of frustration.
Details... The github build script works properly until the actual build using GoReleaser, which fails with the following:
"release failed after 6serror=hook failed: go mod tidy: exit status 1; output: go: downloading..."
and
"fatal: could not read Username for 'https://github.com': terminal prompts disabled"
From my understanding, Ssh-Agent should be setting up access using the SSH private key that I've configured in our account. Hence, GoReleaser should have no trouble accessing any repository that has a DEPLOY_KEY containing the SSH public key.
I would really appreciate your help in getting all of these moving parts to work together. I am sure that there are a lot of other folks wrangling with this issue, too.
Thanks for your time and interest
name: Release
on:
push:
tags:
- "v*.*.*"
jobs:
build:
name: Build Release Binaries
runs-on: ubuntu-latest
permissions:
contents: write
#packages: write
steps:
- name: Install SSH Client
uses: webfactory/ssh-agent#v0.5.4
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Configure Go 1.18
uses: actions/setup-go#v3
with:
go-version: 1.18
- name: Debug
run: |
pwd
echo ${HOME}
echo ${GITHUB_WORKSPACE}
echo ${GOPATH}
echo ${GOROOT}
- name: Debug2
run: go env
- name: Check out the code into the Go module directory.
uses: actions/checkout#v3
with:
repository: 'myorg/myrepo'
fetch-depth: 0 # See: https://goreleaser.com/ci/actions/
path: go/src/github.com/myorg/myrepo
- name: Run GoReleaser
uses: goreleaser/goreleaser-action#v3
with:
# either 'goreleaser' (default) or 'goreleaser-pro'
distribution: goreleaser
version: latest
args: release --rm-dist
workdir: ${{ github.workspace }}/go/src/github.com/myorg/myrepo
env:
GITHUB_TOKEN: ${{ secrets.TOKEN }}
Run goreleaser/goreleaser-action#v3
with:
distribution: goreleaser
version: latest
args: release --rm-dist
workdir: /home/runner/work/myrepo/myrepo/go/src/github.com/myorg/myrepo
install-only: false
env:
SSH_AUTH_SOCK: /tmp/ssh-HIEFX12pQLiS/agent.1733
SSH_AGENT_PID: 1734
APP_VERSION: v2.1.3
BUILD_TIME: Tue Jul 19 07:03:53 UTC 2022
GITHUB_TOKEN: ***
Downloading https://github.com/goreleaser/goreleaser/releases/download/v1.10.2/goreleaser_Linux_x86_64.tar.gz
Extracting GoReleaser
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/0d57d027-19c9-4eee-b395-8e6b3c534c98 -f /home/runner/work/_temp/2d3cd5e7-7087-4ff0-b2db-c036bb8c5bc8
GoReleaser latest installed successfully
Using /home/runner/work/myrepo/myrepo/go/src/github.com/myorg/myrepo as working directory
v2.1.3 tag found for commit 'b94e310'
/opt/hostedtoolcache/goreleaser-action/1.10.2/x64/goreleaser release --rm-dist
•starting release...
• loading config file file=.goreleaser.yaml
•loading environment variables
•getting and validating git state
• building... commit=b94e310435835d012155fce67176ef54a687326e latest tag=v2.1.3
•parsing tag
•setting defaults
•running before hooks
• running hook=go mod tidy
•took: 6s
⨯release failed after 6serror=hook failed: go mod tidy: exit status 1; output: go: downloading
I would suggest to Configure git for private modules in the Github action, adding one simple step in your workflow like:
- name: Configure git for private modules
env:
GITHUB_API_TOKEN: ${{ secrets.GH_API_TOKEN }}
run: git config --global url."https://x:${GITHUB_API_TOKEN}#github.com".insteadOf "https://github.com"
And add the GH_API_TOKEN secrets in the repo in order to be able to download the go modules during the go mod tidy command.

cloud build trigger based on github commits does not work

i have the following build.yaml to build an image for my project
steps:
# Build the container image with the Python version of our choice.
- name: gcr.io/cloud-builders/docker
args:
[ 'build'
, '--tag=gcr.io/$PROJECT_ID/$_IMAGE_NAME'
, '--file=./Dockerfile_pipeline'
, '.'
]
images: ['gcr.io/$PROJECT_ID/$_IMAGE_NAME']
substitutions:
_IMAGE_NAME: pipeline:latestRun
This works fine when kicked off in gcp console via
gcloud beta builds submit --verbosity debug --config build.yaml
However, when i create a cloudbuild trigger (based on commits on my repo) that kicks off my build.yaml i am getting the following error
starting build "4cc47363-32f5-4862-8533-90810c972c5a"
FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://github.com/mmistroni/GCP_Experiments
* branch 450bfce1f17042d3163e39a3ecad89d06922ea47 -> FETCH_HEAD
HEAD is now at 450bfce adding marketloader trigger
BUILD
Already have image (with digest): gcr.io/cloud-builders/docker
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /workspace/Dockerfile_pipeline: no such file or directory
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1
What am i missing?
kind regards
marco
i found the solution after fiddling for a while
When the trigger is kicked off from github commit, the root directory is the root my repo, GCP_Experiments
the dockerfile instead resides in GCP_Experiments\dataflow\pipeline\Dockerfile_pipeline
So i had to do two fixes
1 - put the full path of the docker file as --file argument
2 - specify, as a PATH parameter, the directory where file resides relative to root repo, aka ./dataflow/pipeline
rgds

terraform init failing in gitlab pipeline

I am a newbie for gitlab and I have defined this pipeline defined but keeps failing with error. Help is greatly appreciated
$ terraform init -backend=false
/bin/sh: eval: line 118: terraform: not found
Cleaning up file based variables
ERROR: Job failed: exit code 127
--see my pipeline
image:
name: hashicorp/terraform:light
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
stages:
- Test and Lint
Validate Terraform:
stage: Test and Lint
script:
- echo "Validate Terraform"
- cd infra/
- apk update && apk add bash
- terraform init -backend=false
- terraform validate
- terraform fmt -check
rules:
- if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^(dev|master|production)$/'
The image documentation states:
The light version of this container will copy the current stable version of the binary into the container, and set it for use as the default entrypoint.
You are overriding the container's entrypoint and it looks like you are overwriting the PATH variable, so the terraform executable that you are trying to execute is not found.
Do not override the entrypoint, or make sure that the terraform executable's directory is included in the PATH variable.

pre-commit on windows for terraform

trying to get pre-commit up and running on windows, trying a simple terraform fmt command, but dont many examples of how to run exe piecing it together i have the below:
my .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.1.0 # Use the ref you want to point at
hooks:
- id: detect-aws-credentials
- id: detect-private-key
- repo: local
hooks:
- id: terraform-fmt
name: terraform fmt
description: runs terraform fmt
entry: terraform fmt
args: [-recursive]
language: system
but im getting the error below from pre-commit run -a:
Detect AWS Credentials...................................................Passed
Detect Private Key.......................................................Passed
terraform fmt............................................................Failed
- hook id: terraform-fmt
- exit code: 1
The fmt command expects at most one argument.
Usage: terraform fmt [options] [DIR]
It then looks like its running terraform fmt multiple times as i keep getting the error in a loop. Any idea what im missing?
you may have better luck with https://github.com/antonbabenko/pre-commit-terraform
that said, you can get your example working by using the following I believe:
- repo: local
hooks:
- id: terraform-fmt
name: terraform fmt
description: runs terraform fmt
entry: terraform fmt -recursive
language: system
pass_filenames: false
note that I've done several things:
pass_filenames: false -- pre-commit normally works by passing filenames to hooks, this is why your thing is being invoked multiple times
I removed args (it's unnecessary and only really helpful for remote repositories) and combined it with entry
note that using this as a local hook will ~generally be worse than using the repository above because it will always run against all files instead of just the files you changed (usually making it much much slower!)
disclaimer: I'm the author of pre-commit

CircleCI version 2.1 - "Cannot find a definition for command named 'restore-cache'"

I'm currently attempting to use the commands feature available in CircleCI version 2.1, so that I can reuse some common commands. I'm testing using the CLI command:
circleci config process ./.circleci/config.latest.yaml > ./.circleci/config.yml
But I recieve the following error:
Error: Error calling workflow: 'main'
Error calling job: 'build'
Error calling command: 'build_source'
Cannot find a definition for command named restore-cache
It seems that restore-cache works just fine in a straight-up version 2 config file, but when I try and process a 2.1 file using process it kicks up a fuss.
Below is an edited version of my config.yaml file which should hopefully be of some use. Please let me know if there is any additional information that would be useful.
version: 2.1
defaults: &defaults
/**
* Unimportant stuff
*/
aliases:
- &restore-root-cache
keys:
- v1-deps-{{ .Branch }}-{{ checksum "package.json" }}
- v1-deps-{{ .Branch }}
- v1-deps
commands:
build_source:
description: 'Installs dependencies, then builds src, builds documentation, and runs tests'
steps:
- restore-cache: *restore-root-cache
- other-commands...
jobs:
build:
<<: *defaults
steps:
- checkout
- build_source
workflows:
version: 2.1
main:
jobs:
- build:
filters:
branches:
ignore: develop
The command is restore_cache (with an underscore), not restore-cache (with a dash) https://circleci.com/docs/2.0/configuration-reference/#restore_cache
It should work in commands.
restore cache is a special step that needs to be under a job. Not another command.

Resources