ERROR: (gcloud.auth.activate-service-account) The .json key file is not in a valid format -- via impersonate-service-account - bash

Question

Is it possible to use short-lived credentials, with docker-compose, to run a bash scripted gcloud command?
Related posts that I attempted to use but they are 5+ years old and I've been led to believe that the gcloud auth command has changed during this time:
ERROR: (gcloud.auth.activate-service-account) Failed to activate the given service account. Please ensure provided key file is valid
gcloud auth activate-service-account [ERROR] Please ensure provided key file is valid
Setup
there is a lot going on but I've attempted to abbreviate to the relevant parts
Makefile
auth: ## commands for short lived auth
#gcloud config set project ${GCP_PROJECT}
#gcloud auth application-default login --impersonate-service-account="inst-dataflow-svc#${GCP_PROJECT}.iam.gserviceaccount.com"
#gcloud auth configure-docker $(REGION)-docker.pkg.dev
gcloud-flex-build: ## build & push base docker image
docker-compose build gcloud-build-flex-local
docker-compose run gcloud-build-flex-local
docker-compose.yaml
version: '3.4'
services:
gcloud-build-flex-local:
build:
dockerfile: docker/gcloud-build-flex-template.dockerfile
context: .
image: us-central1-docker.pkg.dev/gcp-project/dataflow-docker-registry/local-build/pubsub-to-gbq-build-flex-template
volumes:
- type: bind
source: ${HOME}/.config/gcloud/
target: /tmp
docker/gcloud-build-flex-template.dockerfile
FROM gcr.io/google.com/cloudsdktool/cloud-sdk:408.0.1
COPY docker/scripts/gcloud-build-flex-template.sh /app/gcloud-build-flex-template.sh
COPY dataflow/pubsub-to-gbq/pubsub-to-gbq-metadata /app/pubsub-to-gbq-metadata
WORKDIR /app
ENTRYPOINT "/app/gcloud-build-flex-template.sh"
/app/gcloud-build-flex-template.sh
#!/bin/bash
set -euo pipefail
SERVICE_ACCOUNT_EMAIL=inst-dataflow-svc#gcp-project.iam.gserviceaccount.com
GCP_PROJECT=gcp-project
export GOOGLE_APPLICATION_CREDENTIALS=/tmp/application_default_credentials.json
# debugging
echo $GOOGLE_APPLICATION_CREDENTIALS
ls -lah /tmp/
cat $GOOGLE_APPLICATION_CREDENTIALS
gcloud auth activate-service-account $SERVICE_ACCOUNT_EMAIL --project=$GCP_PROJECT --key-file=$GOOGLE_APPLICATION_CREDENTIALS
Execution
make auth
make gcloud-flex-build
Error
ERROR: (gcloud.auth.activate-service-account) The .json key file is not in a valid format.
make: *** [gcloud-flex-build] Error 1
stdout (abbreviated)
docker-compose build gcloud-build-flex-local
[+] Building 0.4s (9/9) FINISHED
...
docker-compose run gcloud-build-flex-local
drwxr-xr-x 17 root root 544 Dec 30 10:36 .
drwxr-xr-x 1 root root 4.0K Dec 30 10:40 ..
-rw------- 1 root root 591 Dec 30 10:36 application_default_credentials.json
{
"delegates": [],
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/inst-dataflow-svc#gcp-project.iam.gserviceaccount.com:generateAccessToken",
"source_credentials": {
"client_id": "alphanumeric string .apps.googleusercontent.com",
"client_secret": "alphanumeric string",
"refresh_token": "alphanumeric string",
"type": "authorized_user"
},
"type": "impersonated_service_account"
}
I can make it work via docker run by spoofing the credentials to include only the "source_credentials" object, passed in as a volume, but this same trick doesn't seem to work with docker-compose running a script inside a container...

Answer 1

There is a similar type of configuration mentioned in this document. This involves three major steps:
Create short-lived credentials for your service account and download
your service account keys.
Create the configuration files for making your docker environment up. Use
the above cred files for granting required permissions.
Once you have all the configuration files in place use your docker-compose
commands for making your environment up.
Follow this documentation for more details.