I already set up custom machine types in the cloud build file.
But, google tall me a warning (Long build duration).
Custom machine types not applied!
What did I do wrong?
cloudbuild.yaml
steps:
- name: "gcr.io/cos-cloud/cos-customizer"
args:
[
"start-image-build",
"-image-project=cos-cloud",
"-image-name=${_BASE_IMAGE_NAME}",
"-gcs-bucket=${_CLOUD_BUILD_BUCKET_NAME}",
"-gcs-workdir=cloud-build-${BUILD_ID}",
]
- name: "gcr.io/cos-cloud/cos-customizer"
args:
[
"run-script",
"-env=_MIX_ENV=${_MIX_ENV},_RELEASE_NAME=${_RELEASE_NAME}",
"-script=cloudbuild.sh",
]
- name: "gcr.io/cos-cloud/cos-customizer"
args:
[
"finish-image-build",
"-zone=us-central1-a",
"-project=${PROJECT_ID}",
"-image-project=${PROJECT_ID}",
"-image-family=${_IMAGE_FAMILY_NAME}",
"-image-name=${_BASE_IMAGE_NAME}-${_IMAGE_FAMILY_NAME}-${SHORT_SHA}",
]
options:
machineType: "E2_HIGHCPU_8"
timeout: 3600s
Google's build duration warning
This is not a CloudBuild problem.
cos-customizer configures the machine type is impossible.
https://github.com/GoogleCloudPlatform/cos-customizer/issues/79
Related
I have 2 gcp projects pool-infra pool-dev. I use a github action to run a mvn command the configuration look like this...
- name: Authenticate with pure-infra project
uses: 'google-github-actions/auth#v0.8.1'
with:
service_account: my#pool-infra.iam.gserviceaccount.com
workload_identity_provider: projects/<pool-infra-id>/locations/global/workloadIdentityPools/....
token_format: 'access_token'
project_id: pure-platform-dev
- name: 'Set up Cloud SDK'
uses: 'google-github-actions/setup-gcloud#v1'
with:
project_id: pool-app
- name: Run Package
working-directory: my-service
run: |
gcloud config set project pool-app
gcloud config get project
mvn clean package jacoco:report
But I see an error that suggests the project ID is incorrect...
"errors": [
{
"domain": "usageLimits",
"message": "Cloud SQL Admin API has not been used in project <pool-infra-num> before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/sqladmin.googleapis.com/overview?project=<pool-infra-num> then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.",
"reason": "accessNotConfigured",
"extendedHelp": "https://console.developers.google.com"
}
],
I would expect those project nums to be for pool-app not infra. What am I missing? How do I properly set the project for the mvn project?
This is coming from the JDBC connection pool when it tries to connect.
My spring boot application was working fine in cloud build & deployed without any issue till September.
Now my trigger fails in gcloud app deploy.
Step #4: ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Deployment cannot use a pre-built image. Pre-built images are only allowed in the App Engine Flexible Environment.
app.yaml
runtime: java11
env: standard
service: service
handlers:
- url: /.*
script: this field is required, but ignored
cloudbuild.yaml
steps:
# backend deployment
# Step 1:
- name: maven:3-jdk-14
entrypoint: mvn
dir: 'service'
args: ["test"]
# Step 2:
- name: maven:3-jdk-14
entrypoint: mvn
dir: 'service'
args: ["clean", "install", "-Dmaven.test.skip=true"]
# Step 3:
- name: docker
dir: 'service'
args: ["build", "-t", "gcr.io/service-base/base", "."]
# Step 4:
- name: "gcr.io/cloud-builders/docker"
args: ["push", "gcr.io/service-base/base"]
# Step 5:
- name: 'gcr.io/cloud-builders/gcloud'
dir: 'service/src/main/appengine'
args: ['app', 'deploy', "--image-url=gcr.io/service-base/base"]
timeout: "30m0s"
# Step 6:
# dispatch.yaml deployment
- name: "gcr.io/cloud-builders/gcloud"
dir: 'service/src/main/appengine'
args: ["app", "deploy", "dispatch.yaml"]
timeout: "30m0s"
timeout: "100m0s"
images: ["gcr.io/service-base/base"]
Cloud build error
Thanks in advance. Im confused how my build was working fine before & what am i doing wrong now.
You can't deploy custom container on App Engine standard. You have to provide your code and the environment runtime. Then Buildpack is used to create a standard container on Google Side (for information, a new Cloud Build job is ran for this) and deployed on App Engine.
I recommend you to have a look to Cloud Run to use your custom container. It's very close to App Engine (and even better on many points!) and very customizable.
What your cloudbuild.yaml comment's refer to as Step 5 corresponds to the Step #4 in the error because system begins numbering steps from 0.
The error message is accurate; App Engine standard (!) differs from App Engine flexible in that the latter (flexible) permits container image deployments. App Engine standard deploys from sources.
See Google's example.
It's possible that something has changed Google's side that's causing the issue but, the env: standard in your app.yaml suggests the build file has changed.
Is there a way to provide a "display name" for steps that would be shown in the build details of the build history?
What is currently display is the "name" field which is used to indicate the container image and is not a very useful way to convey meaning.
The id field worked for me. I have a cloudbuild.yaml file as below:
steps:
- id: 'build-container-image'
name: 'gcr.io/cloud-builders/docker'
args:
- 'build'
- '--network=cloudbuild'
- '-t'
- '${_REGION_NAME}-docker.pkg.dev/$PROJECT_ID/${_REPO_NAME}/${_SERVICE_NAME}'
- '.'
- id: 'push-image-to-artifact-registry'
name: 'gcr.io/cloud-builders/docker'
args:
- 'push'
- '${_REGION_NAME}-docker.pkg.dev/$PROJECT_ID/${_REPO_NAME}/${_SERVICE_NAME}'
- id: 'deploy-image-to-cloud-run'
name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
entrypoint: gcloud
args:
- 'run'
- 'deploy'
- '${_SERVICE_NAME}'
- '--image'
- '${_REGION_NAME}-docker.pkg.dev/$PROJECT_ID/${_REPO_NAME}/${_SERVICE_NAME}'
- '--region'
- '${_REGION_NAME}'
- '--platform'
- 'managed'
- '--allow-unauthenticated'
substitutions:
_REGION_NAME: us-west1
_REPO_NAME: container-images
_SERVICE_NAME: public-web-cloud-run
images:
- '${_REGION_NAME}-docker.pkg.dev/$PROJECT_ID/${_REPO_NAME}/${_SERVICE_NAME}'
This is what will be shown in the build details without the id field
This is the build details after the id fields are added:
Link to the id field in the official doc
Found out after some tests that the "id" field is presented 'as is' in the build details for each step and can be used as a field that is presented in the console. Note though that substitutions (i.e. ${QQQ}) did not work for me in the 'id' field
I recently started using Cloud Build with Bazel.
So I have a basic cloudbuild.yaml
steps:
- id: 'run unit tests'
name: gcr.io/cloud-builders/bazel
args: ['test', '//...']
which runs all tests of my Bazel project.
But as you can see from this screenshot, every build takes around 4 minutes, although I haven't touched any code which would affect my tests.
Locally running the tests for the first time takes about 1 minute. But running the tests a second time, with the help of Bazels cache, it takes only a few seconds.
So my goal is to use the Bazel cache with Google Cloud Build
Update
As suggested by Thierry Falvo I'v looked into those recommendations. An thus I tried to the add the following to my cloudbuild.yaml:
steps:
- name: gcr.io/cloud-builders/gsutil
args: ['cp', 'gs://cents-ideas-build-cache/bazel-bin', 'bazel-bin']
- id: 'run unit tests'
name: gcr.io/cloud-builders/bazel
args: ['test', '//...']
- name: gcr.io/cloud-builders/gsutil
args: ['cp', 'bazel-bin', 'gs://cents-ideas-build-cache/bazel-bin']
Although I created the bucket and folder, I get this error:
CommandException: No URLs matched
I think that rather than cache discrete results (artifacts), you want to use GCS (cloud storage) as a bazel remote cache.
- name: gcr.io/cloud-builders/bazel
args: ['test', '--remote_cache=https://storage.googleapis.com/<bucketname>', '--google_default_credentials', '--test_output=errors', '//...']
I have a Google Cloud Build build that times out after 10 min, 3 sec. Is there a way to extend that timeout?
The build status is set to "Build failed (timeout)" and I'm okay with it taking longer than 10 minutes.
In cloudbuild.yaml you have to add something like timeout: 660s.
E.g.
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build', '-t', 'gcr.io/[PRODUCT_ID]/[CONTAINER_IMAGE]', '.' ]
images:
- 'gcr.io/[PRODUCT_ID]/[CONTAINER_IMAGE]'
timeout: 660s
If you defined your build using a cloudbuild.yaml, you can just set the timeout field; see the full definition of a Build Resource in the documentation.
If you are using the gcloud CLI, it takes a --timeout flag; try gcloud builds submit --help for details.
Example: gcloud builds submit --timeout=900s ...