I have GitLab runner installed on a MacOS using homebrew. The runner configuration is located under ${HOME}/.gitlab-runner/config.toml and service configuration located under ${HOME}/Library/LaunchAgents/homebrew.mxcl.gitlab-runner.plist is the default.
Below is my gitlab-runner toml configuration file.
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "MY_RUNNER_NAME"
url = "https://gitlab.com/"
token = "MY_GITLAB_TOKEN"
executor = "shell"
shell = "bash"
environment = ["PATH=/usr/local/opt/openjdk#8/bin:/usr/local/opt/ruby#2.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin", "LC_ALL=en_US.UTF-8", "LANG=en_US.UTF-8"]
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
The runner is connected correctly to Gitlab.com, it executes the steps correctly, but it got stuck in the Uploading artifacts step until the build times out.
Below is my Uploading artifacts logs.
Uploading artifacts...
Runtime platform arch=amd64 os=darwin pid=16690 revision=775dd39d version=13.8.0
android/app/build/outputs/bundle/release/app-release.aab: found 1 matching files and directories
ERROR: Job failed: execution took longer than 1h0m0s seconds
As a debugging step, I tried to run the gitlab-runner artifacts-uploader locally to trace the behavior using this command gitlab-runner --debug --log-level debug artifacts-uploader --verbose --id MY_BUILD_ID --token MY_GITLAB_TOKEN --url https://gitlab.com/ --path android/app/build/outputs/bundle/release/app-release.aab --expire-in "1 week".
Below is my gitlab-runner artifacts-uploader logs.
Runtime platform arch=amd64 os=darwin pid=25259 revision=775dd39d version=13.8.0
android/app/build/outputs/bundle/release/app-release.aab: found 1 matching files and directories
Dialing: tcp gitlab.com:443 ...
It is obvious that the gitlab-runner artifacts-uploader got stuck connecting to gitlab.com:443, and now I am out of ideas on how to trace or solve this issue.
Related
Firstly I set up 1 worker for 1 job. Deploying my backend for the API.
I'm using "shell" as the executer. The .toml file is this structure:
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "Gitlab Runner Josere Backend"
url = "https://gitlab.com/"
token = "sOmEtOkeN1G0Tfr0mGitlab"
executor = "shell"
[runners.custom_build_dir]
[runners.cache]
[some mumbo jumbo about caching.. does it matter?]
With some struggle I got that to work fine with this .gitlab-ci.yml:
deploy-production:
stage: deploy
variables:
GIT_STRATEGY: clone
script:
- cd ./lumen/
- composer install
- sudo cp -r $CI_PROJECT_DIR/lumen/. /home/josere/public_html/api/
- sudo cp /home/josere/env/.env /home/josere/public_html/api
This is the execution output of the runner:
Running with gitlab-runner 15.2.1 (32fc1585)
on Gitlab Runner Josere backend 9JxGrMLz
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
Running on ####[my server]#####...
Getting source from Git repository
00:03
Fetching changes with git depth set to 50...
Initialized empty Git repository in /home/gitlab-runner/builds/9JxGrMLz/0/paspalas/josere/.git/
Created fresh repository.
... etc ...
In my frontend repo in Gitlab I went to the same runners settings. I can't really install a runner (its allready running I guess) but I can copy the token that is shown there.
Then I changed my .toml file according to this doc from gitlab (https://docs.gitlab.com/runner/fleet_scaling/):
concurrent = 2
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "Gitlab Runner Josere Backend"
url = "https://gitlab.com/"
token = "sOmEtOkeN1G0Tfr0mGitlab"
executor = "shell"
[runners.custom_build_dir]
[runners.cache]
[some mumbo jumbo about caching.. does it matter?]
[[runners]]
name = "Gitlab Runner Josere Frontend"
url = "https://gitlab.com/"
token = "TheOtherTokenThatIgotFromFrontendRepo!"
executor = "shell"
[runners.custom_build_dir]
[runners.cache]
[some mumbo jumbo about caching.. does it matter?]
notice I do keep the executor on "shell".
this is the script for .gitlab-ci.yml that goes in the root of the frontend repo:
deploy-production:
stage: deploy
variables:
GIT_STRATEGY: clone
script:
- npm install
- npm run build
- sudo cp -r $CI_PROJECT_DIR/public/. /home/josere/public_html/
But when I commit my frontend and check the (failing) log for the worker it writes this:
Running with gitlab-runner 15.4.0~beta.5.gdefc7017 (defc7017)
on green-1.shared.runners-manager.gitlab.com/default JLgUopmM
Preparing the "docker+machine" executor
00:06
Using Docker executor with image ruby:2.5 ...
Pulling docker image ruby:2.5 ...
Using docker image sha256:27d###mumbojumbo###2383b for ruby:2.5 with digest ruby#sha256:ecc3###mumbojumbo###444b ...
Preparing environment
00:00
Running on runner-jlguopmm-project-39467125-concurrent-0 via runner-jlguopmm-shared-1665674167-6adf45bf...
Getting source from Git repository
00:02
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/paspalas/josere-frontend/.git/
Created fresh repository.
Checking out c39e641c as materialui...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:01
Using docker image sha256:27d###mumbojumbo###3b for ruby:2.5 with digest ruby#sha256:ecc3e###mumbojumbo####44b ...
$ sudo npm install
/bin/bash: line 126: sudo: command not found
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
clearly it seems multiple things go wrong, to start with: why is it using docker while I explicitly tell it to be "shell"?
I fixed the issue. Even though the docs of GitLab differentiates between "runner" and "job", the gitlab-runner calls these "registrations" of a "runner". I did the (extra) registeration like so:
- gitlab-runner register
[filling in info]
- nano /etc/gitlab-runner/config.toml
[check if you have the additional runner]
- gitlab-runner run
[according to gitlab-runner help this is to fire up multiple runners]
- gitlab-runner list
[ now you can check if all "runners" (jobs) are running]
I am trying to containerize my application build, though, when running the build that uses bazel with bazel-gazelle inside a container I will get this error:
$ bazel run --spawn_strategy=local //:gazelle --verbose_failures
INFO: Analyzed target //:gazelle (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /home/workstation/.cache/bazel/_bazel_workstation/fb227af4c7b6aa39cc5b15d7fd9b737a/external/go_sdk/BUILD.bazel:43:15: GoToolchainBinary external/go_sdk/builder [for host] failed: (Exit 1): go failed: error executing command
(cd /home/workstation/.cache/bazel/_bazel_workstation/fb227af4c7b6aa39cc5b15d7fd9b737a/execroot/__main__ && \
exec env - \
GOROOT_FINAL=GOROOT \
external/go_sdk/bin/go tool link -o bazel-out/host/bin/external/go_sdk/builder bazel-out/host/bin/external/go_sdk/builder.a)
# Configuration: e0f1106e28100863b4221c55fca6feb935acec078da5376e291cf644e275dae5
# Execution platform: #local_config_platform//:host
/opt/go/pkg/tool/linux_amd64/link: mapping output file failed: invalid argument
Target //:gazelle failed to build
INFO: Elapsed time: 2.302s, Critical Path: 0.35s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
I tried to run it standalone:
$ /home/workstation/.cache/bazel/_bazel_workstation/fb227af4c7b6aa39cc5b15d7fd9b737a/external/go_sdk/bin/go tool link -o bazel-out/host/bin/external/go_sdk/builder bazel-out/host/bin/external/go_sdk/builder.a
/opt/go/pkg/tool/linux_amd64/link: mapping output file failed: invalid argument
and still got no success.
Never had this kind of link problem and the linker don't provide much more information. Tried to install all packages I could think of and still no luck.
For context:
Running Ubuntu 20.04 LTS
Docker 20.10.9
Bazel 4.2.2
Rules GO v0.31.0
Bazel Gazelle v0.25.0
Also tried to run it with the strace, though I don't think I am skilled enough to find meaningful information from the tool output.
#edit
For more context:
e$ /home/workstation/.cache/bazel/_bazel_workstation/fb227af4c7b6aa39cc5b15d7fd9b737a/external/go_sdk/bin/go tool link -v -o bazel-out/host/bin/external/go_sdk/builder bazel-out/host/bin/external/go_sdk/builder.a
HEADER = -H5 -T0x401000 -R0x1000
searching for runtime.a in /opt/go/pkg/linux_amd64/runtime.a
/opt/go/pkg/tool/linux_amd64/link: mapping output file failed: invalid argument
I have the following setup. Kotlin Multiplatform project with a Web App and Shared module. I also have included a build of a library that I had to modify myself as it wasn't working properly.
When trying to deploy to Heroku with these steps I get the following error
> Task :common:root:jsPackageJson
> Task :web-app:packageJson
> Task :web-app:testPackageJson
> Task :rootPackageJson
> Task :kotlinNpmInstall
warning workspace-aggregator-7524aa80-df12-48c3-95c5-de7144004dc0 > firebase-kotlin-sdk-firebase-app > webpack-dev-server > sockjs > uuid#3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
warning workspace-aggregator-7524aa80-df12-48c3-95c5-de7144004dc0 > firebase-kotlin-sdk-firebase-app > webpack-dev-server > url > querystring#0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
error https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001269.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, stat '/app/.cache/yarn/v6/npm-caniuse-lite-1.0.30001269-3a71bee03df627364418f9fd31adfc7aa1cc2d56-integrity/node_modules/caniuse-lite/LICENSE'"
> Task :firebase-kotlin-sdk:kotlinNpmInstall
error https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001269.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, stat '/app/.cache/yarn/v6/npm-caniuse-lite-1.0.30001269-3a71bee03df627364418f9fd31adfc7aa1cc2d56-integrity/node_modules/caniuse-lite/data/features/getcomputedstyle.js'"
> Task :firebase-kotlin-sdk:kotlinNpmInstall FAILED
> Task :kotlinNpmInstall FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':kotlinNpmInstall'.
> Process 'Resolving NPM dependencies using yarn' returns 1
yarn install v1.22.10
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':firebase-kotlin-sdk:kotlinNpmInstall'.
> Process 'Resolving NPM dependencies using yarn' returns 1
yarn install v1.22.10
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
BUILD FAILED in 5m 20s
18 actionable tasks: 18 executed
! ERROR: Failed to run Gradle!
We're sorry this build is failing. If you can't find the issue in application
code, please submit a ticket so we can help: https://help.heroku.com
You can also try reverting to the previous version of the buildpack by running:
$ heroku buildpacks:set https://github.com/heroku/heroku-buildpack-gradle#previous-version
Thanks,
Heroku
! Push rejected, failed to compile Gradle app.
! Push failed
Tried running kotlinNpmInstall locally on both the library and the whole project and it works properly, with no failures.
While I do not know the exact cause, it seems that multiple yarn process are spawned that corrupt each other's packages.
This can be solved with the mutex option.
To add yarn options to a kotlin/js project you can add a .yarnrc file to the root of your project.
My .yarnrc's content:
--install.mutex network
This makes yarn use a predefined port on the network for its mutex.
When I run a command mvn clean test -Dspring.profiles.active=GITLAB-CI-TEST in the GitLab CI CD it not loading properties file application-gitlab-ci-test.properties. It is loading only application.properties.
As file application-gitlab-ci-test.properties contains the different value for spring.datasource.url the pipeline is failing in the remote runners with error
The last packet sent successfully to the server was 0 milliseconds ago.
The driver has not received any packets from the server.
Of course, this error is expected as properties file application.properties refers to the localhost database.
Code which loading application-gitlab-ci-test.properties:
#Profile("GITLAB-CI-TEST")
#PropertySource("classpath:application-gitlab-ci-test.properties")
#Configuration
public class GitLabCiTestProfile {
}
When I try to run the same command locally it's working as expected and in logs, I see the following records:
2020-03-30 19:23:00.609 DEBUG 604 --- [ main]
o.s.b.c.c.ConfigFileApplicationListener : Loaded config file
'file:/G:/****/****/****/****/target/classes/application.properties'
(classpath:/application.properties)
2020-03-30 19:23:00.609 DEBUG 604 --- [ main]
o.s.b.c.c.ConfigFileApplicationListener : Loaded config file
'file:/G:/****/****/****/****/target/classes/application-GITLAB-CI-TEST.properties' (classpath:/application-GITLAB-CI-TEST.properties) for profile
GITLAB-CI-TEST
I noticed that remote runners missing the second line. This one which loading application-GITLAB-CI-TEST.properties.
I also tried mvn clean test --batch-mode -PGITLAB-CI-TEST and this one too failing in the remote host but in local run working as expected.
I found the workaround for this issue by using the command
mvn clean test --batch-mode -Dspring.datasource.url=jdbc:mysql://mysql-db:3306/*******?useSSL=false&allowPublicKeyRetrieval=true
Can you please help me to solve this issue as this workaround is not satisfying me?
I found the solution to this issue.
I changed the name of the profile from the upper case (GITLAB-CI-TEST) to lower case (gitlab-ci-test), to match the lower case of profile name in properties file - application-gitlab-ci-test.properties.
Now in the remote runner, I'm using the following command:
mvn clean test -Dspring.profiles.active=gitlab-ci-test
Spring doc - link
I have registred gitlab runner to the gitlab instance. My registered URL is as follows: http://azurestackgitlab1.southeastasia.cloudapp.azure.com/ci
However while running the build for that project; build is getting failed and giving the below message:
Running with gitlab-ci-multi-runner 1.11.1 (a67a225)
on java test (96d320b3)
WARNING: image is not supported by selected executor and shell
Using Shell executor...
Running on AzureStackPOCVM...
Cloning repository...
Cloning into '/home/gitlab-runner/builds/96d320b3/0/root/demoproject'...
fatal: unable to access 'http://gitlab-ci-token:xxxxxx#gitlab-ce.hxakzvpf0otezeojz3wqhme5wg.cx.internal.cloudapp.net/root/demoproject.git/': Could not resolve host: gitlab-ce.hxakzvpf0otezeojz3wqhme5wg.cx.internal.cloudapp.net
ERROR: Job failed: exit status 1
snippet of config.toml:
[[runners]]
name = "java test"
url = "http://azurestackgitlab1.southeastasia.cloudapp.azure.com/ci"
token = "96d320b33d3c69d706dad7f90df84e"
executor = "shell"
[runners.cache]
How to overcome this problem?
You will get this URL mentioned in etc/gitlab/gitlab.rb file. If you will change "hxakzvpf0otezeojz3wqhme5wg.cx.internal.cloudapp.net"
with your "http://azurestackgitlab1.southeastasia.cloudapp.azure.com"
It should work!
For more information you can refer : http://azurestackgitlab1.southeastasia.cloudapp.azure.com/help/administration/environment_variables.md