GoReleaser and ssh-agent Github Actions: Why could not read Username ... terminal prompts disabled? - go

I have been going around and around between instructions for GitHub Actions, GoReleaser, and Ssh-Agent and cannot get my simple release build script to work. My goal is simple... I have a go private repository containing a CLI application and its go.mod file has a dependency on another private repository that we've created. Building the application locally is successful.
The issue is that when I try to build this simple application in a GitHub Action, things become really complicated very quickly... repository secrets, deploy key, an a few other moving parts. As common as this use-case is, I failed to find a single example where someone has implemented a release build script for it... I am about ready to switch to a mono-repo out of frustration.
Details... The github build script works properly until the actual build using GoReleaser, which fails with the following:
"release failed after 6serror=hook failed: go mod tidy: exit status 1; output: go: downloading..."
and
"fatal: could not read Username for 'https://github.com': terminal prompts disabled"
From my understanding, Ssh-Agent should be setting up access using the SSH private key that I've configured in our account. Hence, GoReleaser should have no trouble accessing any repository that has a DEPLOY_KEY containing the SSH public key.
I would really appreciate your help in getting all of these moving parts to work together. I am sure that there are a lot of other folks wrangling with this issue, too.
Thanks for your time and interest
name: Release
on:
push:
tags:
- "v*.*.*"
jobs:
build:
name: Build Release Binaries
runs-on: ubuntu-latest
permissions:
contents: write
#packages: write
steps:
- name: Install SSH Client
uses: webfactory/ssh-agent#v0.5.4
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Configure Go 1.18
uses: actions/setup-go#v3
with:
go-version: 1.18
- name: Debug
run: |
pwd
echo ${HOME}
echo ${GITHUB_WORKSPACE}
echo ${GOPATH}
echo ${GOROOT}
- name: Debug2
run: go env
- name: Check out the code into the Go module directory.
uses: actions/checkout#v3
with:
repository: 'myorg/myrepo'
fetch-depth: 0 # See: https://goreleaser.com/ci/actions/
path: go/src/github.com/myorg/myrepo
- name: Run GoReleaser
uses: goreleaser/goreleaser-action#v3
with:
# either 'goreleaser' (default) or 'goreleaser-pro'
distribution: goreleaser
version: latest
args: release --rm-dist
workdir: ${{ github.workspace }}/go/src/github.com/myorg/myrepo
env:
GITHUB_TOKEN: ${{ secrets.TOKEN }}
Run goreleaser/goreleaser-action#v3
with:
distribution: goreleaser
version: latest
args: release --rm-dist
workdir: /home/runner/work/myrepo/myrepo/go/src/github.com/myorg/myrepo
install-only: false
env:
SSH_AUTH_SOCK: /tmp/ssh-HIEFX12pQLiS/agent.1733
SSH_AGENT_PID: 1734
APP_VERSION: v2.1.3
BUILD_TIME: Tue Jul 19 07:03:53 UTC 2022
GITHUB_TOKEN: ***
Downloading https://github.com/goreleaser/goreleaser/releases/download/v1.10.2/goreleaser_Linux_x86_64.tar.gz
Extracting GoReleaser
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/0d57d027-19c9-4eee-b395-8e6b3c534c98 -f /home/runner/work/_temp/2d3cd5e7-7087-4ff0-b2db-c036bb8c5bc8
GoReleaser latest installed successfully
Using /home/runner/work/myrepo/myrepo/go/src/github.com/myorg/myrepo as working directory
v2.1.3 tag found for commit 'b94e310'
/opt/hostedtoolcache/goreleaser-action/1.10.2/x64/goreleaser release --rm-dist
•starting release...
• loading config file file=.goreleaser.yaml
•loading environment variables
•getting and validating git state
• building... commit=b94e310435835d012155fce67176ef54a687326e latest tag=v2.1.3
•parsing tag
•setting defaults
•running before hooks
• running hook=go mod tidy
•took: 6s
⨯release failed after 6serror=hook failed: go mod tidy: exit status 1; output: go: downloading

I would suggest to Configure git for private modules in the Github action, adding one simple step in your workflow like:
- name: Configure git for private modules
env:
GITHUB_API_TOKEN: ${{ secrets.GH_API_TOKEN }}
run: git config --global url."https://x:${GITHUB_API_TOKEN}#github.com".insteadOf "https://github.com"
And add the GH_API_TOKEN secrets in the repo in order to be able to download the go modules during the go mod tidy command.

Related

Azure pipeline fails on building Kotlin Multiplatform shared framework usinig embedAndSignAppleFrameworkForXcode and fastlane

I'm working on a Kotlin Multiplatform project which is building fine locally but I can't get it to work on an Azure DevOps pipeline.
Some good things to know:
not using Cocoapods
using the embedAndSignAppleFrameworkForXcode gradlew command in Build Phases
all commands using fastlane work for multiple developers locally
we use custom configurations like: ProjectADebug/ProjectARelease but we defined KOTLIN_FRAMEWORK_BUILD_TYPE for all of them
I'm trying to get an Azure DevOps pipeline to build and upload to App Store Connect using fastlane. We are using match for signing, that works great. Archiving fails and it looks like it's failing on building the shared KMM framework.
Anybody with the same problems that could help me out? Or some tips how I can view those gym logs on the Azure VM because I assume there it says what actually went wrong instead of this general error.
▸ Running script 'Build Kotlin Common'
▸ Copying /Users/runner/Library/Developer/Xcode/DerivedData/Project-ffubndppzitzbxhibjgeavrhnzpw/Build/Intermediates.noindex/ArchiveIntermediates/Project/BuildProductsPath/ProjectRelease-iphoneos/Airship_AirshipCore.bundle
▸ Copying /Users/runner/Library/Developer/Xcode/DerivedData/Project-ffubndppzitzbxhibjgeavrhnzpw/Build/Intermediates.noindex/ArchiveIntermediates/Project/BuildProductsPath/Project Release-iphoneos/Airship_AirshipAutomation.bundle
** ARCHIVE FAILED **
The following build commands failed:
PhaseScriptExecution Build\ Kotlin\ Common /Users/runner/Library/Developer/Xcode/DerivedData/Project-ffubndppzitzbxhibjgeavrhnzpw/Build/Intermediates.noindex/ArchiveIntermediates/Project/IntermediateBuildFilesPath/Project.build/ProjectRelease-iphoneos/Project.build/Script-2F4970EC27CD16A000E32F91.sh (in target 'Project' from project 'Project')
(1 failure)
ERROR [2022-05-10 13:04:32.36]: Exit status: 65
ERROR [2022-05-10 13:04:32.53]: ⬆️ Check out the few lines of raw `xcodebuild` output above for potential hints on how to solve this error
WARN [2022-05-10 13:04:32.53]: 📋 For the complete and more detailed error log, check the full log at:
WARN [2022-05-10 13:04:32.53]: 📋 /Users/runner/Library/Logs/gym/Project-Project.log
This is the the lane in Fastfile:
lane :azure_beta do |options|
label = options[:label].capitalize
git_url = "someURL"
match(
type: "appstore",
readonly: true,
git_url: git_url,
keychain_name: ENV["MATCH_KEYCHAIN_NAME"],
keychain_password: ENV["MATCH_KEYCHAIN_PASSWORD"],
verbose: true
)
build_app(
project: "../Project/Project.xcodeproj",
configuration: "#{label}Release",
scheme: label
)
# fails on the build_app step...
changelog = changelog_from_git_commits(
pretty: "- (%ae) %s",
date_format: "short",
merge_commit_filtering: "exclude_merges"
)
upload_to_testflight(
changelog: changelog,
app_identifier: label == "Project" ? idsProjectA : idsProjectB,
skip_waiting_for_build_processing: true
)
version_number = get_version_number(
xcodeproj: "../Project/Project.xcodeproj",
target: "Project", #Hardcoded because we have multiple targets, label is specificed in build_app configuration
configuration: "#{label}Release"
)
add_git_tag(
includes_lane: false,
prefix: "ios-#{label.downcase}-#{version_number}-",
build_number: number_of_commits
)
delete_keychain(name: ENV["MATCH_KEYCHAIN_NAME"])
end
And this is my pipeline YAML:
pool:
vmImage: 'macos-latest'
variables:
- group: fastlane
jobs:
- job: testflight
steps:
- task: Bash#3
displayName: fastlane update
inputs:
targetType: 'inline'
script: |
gem update fastlane
fastlane --version
- task: JavaToolInstaller#0
inputs:
versionSpec: '11'
jdkArchitectureOption: 'x64'
jdkSourceOption: 'PreInstalled'
- task: Bash#3
displayName: 'Update Dependencies'
inputs:
targetType: 'inline'
script: HOMEBREW_NO_AUTO_UPDATE=1 brew bundle
- task: Bash#3
displayName: "Set build properties"
inputs:
targetType: 'inline'
script: |
echo "sdk.dir=/Users/runner/Library/Android/sdk"
echo "INCLUDE_MOCKER=false" >> local.properties
echo "INCLUDE_ANDROID=false" >> local.properties
echo "INCLUDE_TESTER=false" >> local.properties
echo "APP_LABEL=$(APP_LABEL)" >> local.properties
env:
APP_LABEL: $(APP_LABEL)
- task: Gradle#2
displayName: 'Clean label common'
inputs:
workingDirectory: ''
tasks: "common:cleanLabel"
env:
APP_LABEL: $(APP_LABEL)
- task: Bash#3
displayName: fastlane ios
env:
MATCH_PASSWORD: $(MATCH_PASSWORD)
FASTLANE_PASSWORD: $(FASTLANE_PASSWORD)
FASTLANE_SESSION: $(FASTLANE_SESSION)
FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD: $(FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD)
inputs:
targetType: 'inline'
script: |
sudo xcode-select -s /Applications/Xcode_13.2.app
cd ios/Project
fastlane azure_beta label:Project app_identifier:project.bundle.id itc_team_id:itc.team.id team_id:team.id git_match_branch:master username:me#myself.com
As it turned out there was an error in building the common KMM layer, I would have found it when doing a clean checkout probably but I found out by using a self-hosted agent on Azure Devops so I could navigate to the /Users/runner/Library/Logs/gym/Project-Project.log as Pylyp Dukhov suggested.

pre-commit for local hook gives error: "unrecognized arguments: .pre-commit-config.yaml"

I have the following repo in pre-commit file .pre-commit-config.yaml
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements
args: ["--compare"]
But it keeps giving me the error:
error: unrecognized arguments: .pre-commit-config.yaml
As it passes the filename as an argument to my python script. How can I prevent this?
to clean up your example a little bit -- and use files to only run when the necessary files change:
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements --compare
files: ^requirements-dev.txt$
pass_filenames: false
note that I did a couple things:
args doesn't really make sense for local hooks, you can just put that in entry
pass_filenames (as you did) -- pre-commit is a framework based on passing filenames to executables, but you can turn that off
files: this will make it so the hook only gets triggered if requirements-dev.txt changes
alternatively (if you expect changes outside requirements-dev.txt to need to run this hook) you can drop files and use always_run: true
disclaimer: I'm the author of pre-commit
I spent quite some time figuring out what caused this and how to solve this. It's not documented well, eventually I fixed it by trial and error. We have to use pass_filenames: false in our hook:
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements
pass_filenames: false
args: ["--compare"]

How to properly set up CircleCI plugin

I am trying to integrate CircleCI plugin with my spring-pet-clinic project. I was following the instruction on CircleCI web page. I have created .circleci folder inside my project root folder.
Inside .circleci I have added config.yml file and copy-pasted config from CircleCI page
My config was like this:
# Use the latest 2.1 version of CircleCI pipeline processing engine,
see https://circleci.com/docs/2.0/configuration-reference/
version: 2.1
# Use a package of configuration called an orb, see
https://circleci.com/docs/2.0/orb-intro/
orbs:
# Declare a dependency on the welcome-orb
welcome: circleci/welcome-orb#0.3.1
# Orchestrate or schedule a set of jobs, see
https://circleci.com/docs/2.0/workflows/
workflows:
# Name the workflow "Welcome"
Welcome:
# Run the welcome/run job in its own container
jobs:
- welcome/run
After I ran the project CircleCI has thrown an error. Especially this one: "Config Processing Error: Don't rerun"
$#!/bin/sh -eo pipefail
# No configuration was found in your project. Please refer to
https://circleci.com/docs/2.0/ to get started with your
configuration.
#
# -------
# Warning: This configuration was auto-generated to show you
the
message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1
Spin Up Environment looks like this
Build-agent version 1.0.10572-3ce00c85 (2019-04-
15T22:09:28+0000)
Docker Engine Version: 17.05.0-ce
Kernel Version: Linux b0a81c56acff 4.4.0-144-generic
#170~14.04.1-
Ubuntu SMP Mon Mar 18 15:02:05 UTC 2019 x86_64 Linux
Starting container bash:4.4.19
using image
bash#sha256:9f0a4aa3c9931bd5fdda51b1b2b74a0398a8eabeaf9519d807e010b9d9d41993
Using build environment variables
BASH_ENV=/tmp/.bash_env-5cbebf83d4b030000849b60f-0-build
CI=true
CIRCLECI=true
CIRCLE_BRANCH=master
CIRCLE_BUILD_NUM=5
CIRCLE_BUILD_URL=https://circleci.com/gh/sajmon2325/Spring-
Pet-
Clinic/5
CIRCLE_COMPARE_URL=
CIRCLE_JOB=Build Error
CIRCLE_NODE_INDEX=0
CIRCLE_NODE_TOTAL=1
CIRCLE_PREVIOUS_BUILD_NUM=4
CIRCLE_PROJECT_REPONAME=Spring-Pet-Clinic
CIRCLE_PROJECT_USERNAME=sajmon2325
CIRCLE_REPOSITORY_URL=git#github.com:sajmon2325/Spring-Pet-
Clinic.git
CIRCLE_SHA1=48f6db114b41c338e606de32d8648c64ba5119fd
CIRCLE_SHELL_ENV=/tmp/.bash_env-5cbebf83d4b030000849b60f-0-
build
CIRCLE_STAGE=Build Error
CIRCLE_USERNAME=sajmon2325
CIRCLE_WORKFLOW_ID=2789d93e-f1e4-4c81-93f1-846f7d38c107
CIRCLE_WORKFLOW_JOB_ID=670105ca-617e-445e-9b5e-6ac57f6af8da
CIRCLE_WORKFLOW_UPSTREAM_JOB_IDS=
CIRCLE_WORKFLOW_WORKSPACE_ID=2789d93e-f1e4-4c81-93f1-
846f7d38c107
CIRCLE_WORKING_DIRECTORY=~/project
Using environment variables from project settings and/or
contexts
CIRCLE_JOB=**REDACTED**
So at first I thought that I have only a skeleton of CircleCI configuration, that's why I have edited my config.yml file to look like this (the actual version)
# Java Maven CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-java/ for more details
#
version: 2
jobs:
build:
docker:
# specify the version you desire here
- image: circleci/openjdk:11-browsers-legacy
# Specify service dependencies here if necessary
# CircleCI maintains a library of pre-built images
# documented at https://circleci.com/docs/2.0/circleci-images/
# - image: circleci/postgres:9.4
working_directory: ~/repo
environment:
# Customize the JVM maximum heap limit
MAVEN_OPTS: -Xmx3200m
steps:
- checkout
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "pom.xml" }}
# fallback to using the latest cache if no exact match
is
found
- v1-dependencies-
- run: mvn install -DskipTests
- run: mvn dependency:go-offline
- save_cache:
paths:
- ~/.m2
key: v1-dependencies-{{ checksum "pom.xml" }}
# run tests!
- run: mvn integration-test
But even this is not working. I still have the same error:
$#!/bin/sh -eo pipefail
# No configuration was found in your project. Please refer to https://circleci.com/docs/2.0/ to get started with your configuration.
#
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1
I just need to successfully integrate CircleCi plugin with my project. If you need to see my repo, here is the link: https://github.com/sajmon2325/Spring-Pet-Clinic.git
The problem is that .circleci is not in the root of the repository. It is currently in sfg-pet-clinic/, and the CircleCI build process won't find it there.

CircleCI version 2.1 - "Cannot find a definition for command named 'restore-cache'"

I'm currently attempting to use the commands feature available in CircleCI version 2.1, so that I can reuse some common commands. I'm testing using the CLI command:
circleci config process ./.circleci/config.latest.yaml > ./.circleci/config.yml
But I recieve the following error:
Error: Error calling workflow: 'main'
Error calling job: 'build'
Error calling command: 'build_source'
Cannot find a definition for command named restore-cache
It seems that restore-cache works just fine in a straight-up version 2 config file, but when I try and process a 2.1 file using process it kicks up a fuss.
Below is an edited version of my config.yaml file which should hopefully be of some use. Please let me know if there is any additional information that would be useful.
version: 2.1
defaults: &defaults
/**
* Unimportant stuff
*/
aliases:
- &restore-root-cache
keys:
- v1-deps-{{ .Branch }}-{{ checksum "package.json" }}
- v1-deps-{{ .Branch }}
- v1-deps
commands:
build_source:
description: 'Installs dependencies, then builds src, builds documentation, and runs tests'
steps:
- restore-cache: *restore-root-cache
- other-commands...
jobs:
build:
<<: *defaults
steps:
- checkout
- build_source
workflows:
version: 2.1
main:
jobs:
- build:
filters:
branches:
ignore: develop
The command is restore_cache (with an underscore), not restore-cache (with a dash) https://circleci.com/docs/2.0/configuration-reference/#restore_cache
It should work in commands.
restore cache is a special step that needs to be under a job. Not another command.

CircleCI API behaving differently from github commit trigger?

I'm running the cypress-example-kitchen sink app on CircleCI.
This is my yaml config script:
version: 2.1
orbs:
cypress: cypress-io/cypress#1.0.1
workflows:
build:
jobs:
- cypress/install:
build: 'npm run build'
- cypress/run:
requires:
- cypress/install
start: 'npm start'
This kicks off and passes just fine when I make a commit to my fork of the repo above.
However, when I try to execute a CircleCI build programmatically, using
curl -X POST https://circleci.com/api/v1.1/project/github/Atticus29/cypress-example-kitchensink?circle-token=myApiToken, the build fails and the jobs dashboard on CircleCI tells me that something is wrong with my config file:
6 schema violations found required key [jobs] not found workflows:
5 schema violations found
workflows: minimum size: [2], found: 1
workflows: build: jobs: 4 schema violations found
workflows: build: jobs: 0: 0 subschemas matched instead of one
workflows: build: jobs: 0: expected type: String, found: Mapping
workflows: build: jobs: 0: install: extraneous key [build] is not permitted
workflows: build: jobs: 1: 0 subschemas matched instead of one
workflows: build: jobs: 1: expected type: String, found: Mapping
workflows: build: jobs: 1: run: extraneous key [start] is not permitted
And that something went wrong with my build:
Build-agent version 0.1.1216-48f80d08 (2018-12-07T16:01:40+0000)
Configuration errors: 2 errors occurred:
Configuration version 2.1 requires the "Enable Build Processing" project setting. Enable Build Processing under Project Settings ->
Advanced Settings. In order to retrigger build processing, you must
push a new commit.
Cannot find a job named build to run in the jobs: section of your configuration file. If you expected a workflow to run, check your
config contains a top-level key called 'workflows:'
I can confirm that Enable Build Processing is on.
None of these were problems when I ran the build in the usual way. Any advice?
Circle CI for some reason keeps on assuming that the projects are not set up for v2.0 despite config.yml being called the right thing and living in the right place in the repo. After a few commits, this issue seems to go away?
I ended up running a build programmatically with the following script:
#!/bin/bash
PERSONAL_TOKEN=myPersonalTokenHere
MOST_RECENT_BUILD=`curl -s "https://circleci.com/api/v1.1/recent-builds?circle-token=$PERSONAL_TOKEN&limit=1"| grep 'build_num'|grep -o '\d.'|sed 's/,//g'|sort -r -n|head -n1`
curl -X POST "https://circleci.com/api/v1.1/project/github/holmbergius/wildMeCypress/$MOST_RECENT_BUILD/retry?circle-token=$PERSONAL_TOKEN"

Resources