Unable to deploy pre built image in app engine standard environment (GCP) - spring-boot

My spring boot application was working fine in cloud build & deployed without any issue till September.
Now my trigger fails in gcloud app deploy.
Step #4: ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Deployment cannot use a pre-built image. Pre-built images are only allowed in the App Engine Flexible Environment.
app.yaml
runtime: java11
env: standard
service: service
handlers:
- url: /.*
script: this field is required, but ignored
cloudbuild.yaml
steps:
# backend deployment
# Step 1:
- name: maven:3-jdk-14
entrypoint: mvn
dir: 'service'
args: ["test"]
# Step 2:
- name: maven:3-jdk-14
entrypoint: mvn
dir: 'service'
args: ["clean", "install", "-Dmaven.test.skip=true"]
# Step 3:
- name: docker
dir: 'service'
args: ["build", "-t", "gcr.io/service-base/base", "."]
# Step 4:
- name: "gcr.io/cloud-builders/docker"
args: ["push", "gcr.io/service-base/base"]
# Step 5:
- name: 'gcr.io/cloud-builders/gcloud'
dir: 'service/src/main/appengine'
args: ['app', 'deploy', "--image-url=gcr.io/service-base/base"]
timeout: "30m0s"
# Step 6:
# dispatch.yaml deployment
- name: "gcr.io/cloud-builders/gcloud"
dir: 'service/src/main/appengine'
args: ["app", "deploy", "dispatch.yaml"]
timeout: "30m0s"
timeout: "100m0s"
images: ["gcr.io/service-base/base"]
Cloud build error
Thanks in advance. Im confused how my build was working fine before & what am i doing wrong now.

You can't deploy custom container on App Engine standard. You have to provide your code and the environment runtime. Then Buildpack is used to create a standard container on Google Side (for information, a new Cloud Build job is ran for this) and deployed on App Engine.
I recommend you to have a look to Cloud Run to use your custom container. It's very close to App Engine (and even better on many points!) and very customizable.

What your cloudbuild.yaml comment's refer to as Step 5 corresponds to the Step #4 in the error because system begins numbering steps from 0.
The error message is accurate; App Engine standard (!) differs from App Engine flexible in that the latter (flexible) permits container image deployments. App Engine standard deploys from sources.
See Google's example.
It's possible that something has changed Google's side that's causing the issue but, the env: standard in your app.yaml suggests the build file has changed.

Related

Google cloud build with pack and secrets manager not accessing environment variables

I'm using a standard gcr.io/k8s-skaffold/pack build function to build my app for google cloud run using google cloud build.
In my cloudbuild.yaml I load 2 secrets from google secrets manager and pass it to the build function. The google cloud build has access to those secrets, otherwise I would get an error message for this (I got this kind of error at the beginning when setting up the build, now it seems to have access).
However, it seems like the environment variables don't get set.
I think that it might be a syntactical problem of how I try to pass the variables.
This is the stripped down cloudbuild.yaml
steps:
- name: gcr.io/k8s-skaffold/pack
args:
- build
- '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
- '--builder=gcr.io/buildpacks/builder:v1'
- '--network=cloudbuild'
- '--path=.'
- '--env=SEC_A=$$SEC_A'
- '--env=SEC_B=$$SEC_B'
secretEnv: ['SEC_A', 'SEC_B']
id: Buildpack
entrypoint: pack
availableSecrets:
secretManager:
- versionName: projects/<pid>/secrets/SEC_A/versions/latest
env: SEC_A
- versionName: projects/<pid>/secrets/SEC_B/versions/latest
env: SEC_B
An Error message that I hacked into the build for checking shows me that the env var is empty during this build step.
I tried using $, $$ (as seen above), &&, ${...}, for substitution. But maybe the problem lies somewhere else.
Yes, it's a common issue and a trap on Cloud Build. In fact, your secrets can't be read if you use the args[] arrays to pass argument. you have to use the script mode, like that
steps:
- name: gcr.io/k8s-skaffold/pack
entrypoint: bash
args:
- -c
- |
pack build $_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA --builder=gcr.io/buildpacks/builder:v1 --network=cloudbuild --path=. --env=SEC_A=$$SEC_A --env=SEC_B=$$SEC_B
secretEnv: ['SEC_A', 'SEC_B']
id: Buildpack

Google Cloud Build - Custom machine types not working

I already set up custom machine types in the cloud build file.
But, google tall me a warning (Long build duration).
Custom machine types not applied!
What did I do wrong?
cloudbuild.yaml
steps:
- name: "gcr.io/cos-cloud/cos-customizer"
args:
[
"start-image-build",
"-image-project=cos-cloud",
"-image-name=${_BASE_IMAGE_NAME}",
"-gcs-bucket=${_CLOUD_BUILD_BUCKET_NAME}",
"-gcs-workdir=cloud-build-${BUILD_ID}",
]
- name: "gcr.io/cos-cloud/cos-customizer"
args:
[
"run-script",
"-env=_MIX_ENV=${_MIX_ENV},_RELEASE_NAME=${_RELEASE_NAME}",
"-script=cloudbuild.sh",
]
- name: "gcr.io/cos-cloud/cos-customizer"
args:
[
"finish-image-build",
"-zone=us-central1-a",
"-project=${PROJECT_ID}",
"-image-project=${PROJECT_ID}",
"-image-family=${_IMAGE_FAMILY_NAME}",
"-image-name=${_BASE_IMAGE_NAME}-${_IMAGE_FAMILY_NAME}-${SHORT_SHA}",
]
options:
machineType: "E2_HIGHCPU_8"
timeout: 3600s
Google's build duration warning
This is not a CloudBuild problem.
cos-customizer configures the machine type is impossible.
https://github.com/GoogleCloudPlatform/cos-customizer/issues/79

Percy not running in CircleCI orbs (w/ Cypress)

I'm trying to get Percy.io to take snapshots of a simple test written in Cypress, building in CircleCI. However, the 'builds' are showing up as failed in the Percy dashboard despite the test/build passing in CircleCI. In the Cypress test runner it is showing 'Percy not running' where my snapshots are placed.
I've followed the tutorials on the Percy and Cypress sites. I can get Percy to work locally, by running percy exec -- cypress run
but the CircleCI config doesn't run Cypress via the command cypress run, it runs it via the cypress orb.
It seems like the two orbs, Cypress and Percy, doesn't know the other exists.
Here's my CircleCI config file:
version: 2.1
orbs:
node: circleci/node#4.5.1
cypress: cypress-io/cypress#1.28.0
slack: circleci/slack#4.4.2
percy: percy/agent#0.1.3
workflows:
version: 2
commit-workflow:
jobs:
- cypress/run:
name: Smoke Tests
record: true
store_artifacts: true
spec: cypress/integration/E2E/*
post-steps:
- store_test_results:
path: test-results
- slack/notify:
channel: general
event: fail
template: basic_fail_1
mentions: '#Jac'
- slack/notify:
channel: general
event: pass
template: basic_success_1
mentions: '#Jac'
- percy/finalize_all:
requires:
- Smoke Tests
The Run Cypress Tests step doesn't make any mention of Percy, so I'm assuming it simply isn't running - that despite using the Percy orb, there's some sort of config I'm missing?
Apologies, I keep finding answers to my questions after posting to Stack
Overflow! I obviously don't know the properties of cypress/run well enough. But essentially, there's a custom command-prefix property that can be added for the purpose of amending the command used to run cypress. In fact, Percy is the example used in the Cypress docs.
Config now looks like:
version: 2.1
orbs:
node: circleci/node#4.5.1
cypress: cypress-io/cypress#1.28.0
slack: circleci/slack#4.4.2
percy: percy/agent#0.1.3
workflows:
version: 2
commit-workflow:
jobs:
- cypress/run:
name: Smoke Tests
record: true
store_artifacts: true
spec: cypress/integration/E2E/*
command-prefix: npx percy exec --
post-steps:
- store_test_results:
path: test-results
- slack/notify:
channel: general
event: fail
template: basic_fail_1
mentions: '#Jac'
- slack/notify:
channel: general
event: pass
template: basic_success_1
mentions: '#Jac'
- percy/finalize_all:
requires:
- Smoke Tests

Google Cloud Build - How to Cache Bazel?

I recently started using Cloud Build with Bazel.
So I have a basic cloudbuild.yaml
steps:
- id: 'run unit tests'
name: gcr.io/cloud-builders/bazel
args: ['test', '//...']
which runs all tests of my Bazel project.
But as you can see from this screenshot, every build takes around 4 minutes, although I haven't touched any code which would affect my tests.
Locally running the tests for the first time takes about 1 minute. But running the tests a second time, with the help of Bazels cache, it takes only a few seconds.
So my goal is to use the Bazel cache with Google Cloud Build
Update
As suggested by Thierry Falvo I'v looked into those recommendations. An thus I tried to the add the following to my cloudbuild.yaml:
steps:
- name: gcr.io/cloud-builders/gsutil
args: ['cp', 'gs://cents-ideas-build-cache/bazel-bin', 'bazel-bin']
- id: 'run unit tests'
name: gcr.io/cloud-builders/bazel
args: ['test', '//...']
- name: gcr.io/cloud-builders/gsutil
args: ['cp', 'bazel-bin', 'gs://cents-ideas-build-cache/bazel-bin']
Although I created the bucket and folder, I get this error:
CommandException: No URLs matched
I think that rather than cache discrete results (artifacts), you want to use GCS (cloud storage) as a bazel remote cache.
- name: gcr.io/cloud-builders/bazel
args: ['test', '--remote_cache=https://storage.googleapis.com/<bucketname>', '--google_default_credentials', '--test_output=errors', '//...']

CircleCI version 2.1 - "Cannot find a definition for command named 'restore-cache'"

I'm currently attempting to use the commands feature available in CircleCI version 2.1, so that I can reuse some common commands. I'm testing using the CLI command:
circleci config process ./.circleci/config.latest.yaml > ./.circleci/config.yml
But I recieve the following error:
Error: Error calling workflow: 'main'
Error calling job: 'build'
Error calling command: 'build_source'
Cannot find a definition for command named restore-cache
It seems that restore-cache works just fine in a straight-up version 2 config file, but when I try and process a 2.1 file using process it kicks up a fuss.
Below is an edited version of my config.yaml file which should hopefully be of some use. Please let me know if there is any additional information that would be useful.
version: 2.1
defaults: &defaults
/**
* Unimportant stuff
*/
aliases:
- &restore-root-cache
keys:
- v1-deps-{{ .Branch }}-{{ checksum "package.json" }}
- v1-deps-{{ .Branch }}
- v1-deps
commands:
build_source:
description: 'Installs dependencies, then builds src, builds documentation, and runs tests'
steps:
- restore-cache: *restore-root-cache
- other-commands...
jobs:
build:
<<: *defaults
steps:
- checkout
- build_source
workflows:
version: 2.1
main:
jobs:
- build:
filters:
branches:
ignore: develop
The command is restore_cache (with an underscore), not restore-cache (with a dash) https://circleci.com/docs/2.0/configuration-reference/#restore_cache
It should work in commands.
restore cache is a special step that needs to be under a job. Not another command.

Resources