enviornment: 'dev'
acr-login: $(enviornment)-acr-login
acr-secret: $(enviornment)-acr-secret
dev-acr-login and dev-acr-secret are secrets stored in keyvault for acr login and acr secret.
In Pipeline, getting secrets with this task
- task: AzureKeyVault#1
inputs:
azureSubscription: $(connection)
KeyVaultName: $(keyVaultName)
SecretsFilter: '*'
This task will create task variables with name 'dev-acr-login' and 'dev-acr-secret'
Not if I want to login in docker I am not able to do that
Following code works and I am able to login into acr.
- bash: |
echo $(dev-acr-secret) | docker login \
$(acrName) \
-u $(dev-acr-login) \
--password-stdin
displayName: 'docker login'
Following doesnot work. Is there a way that I can use variable names $(acr-login) and $(acr-secret) rather than actual keys from keyvault?
- bash: |
echo $(echo $(acr-secret)) | docker login \
$(acrRegistryServerFullName) \
-u $(echo $(acr-login)) \
--password-stdin
displayName: 'docker login'
You could pass them as environment variables:
- bash: |
echo $(echo $ACR_SECRET) | ...
displayName: docker login
env:
ACR_SECRET: $(acr-secret)
But what is the purpose, as opposed to just echoing the password values as you said works in the other example? As long as the task is creating secure variables, they will be protected in logs. You'd need to do that anyway, since they would otherwise show up in diagnostic logs if someone enabled diagnostics, which anyone can do.
An example to do that:
- bash: |
echo "##vso[task.setvariable variable=acr-login;issecret=true;]$ACR_SECRET"
env:
ACR_SECRET: $($(acr-secret)) # Should expand recursively
See Define variables : Set secret variables for more information and examples.
Related
I am trying to pass a JWT token in between jobs but something prevents it to be passed correctly. According to the docs, if I want to pass variables between jobs I need to use outputs as explained here. What I am doing is the following:
name: CI
on:
pull_request:
branches:
- main
jobs:
get-service-url:
...does something not interesting to us...
get-auth-token:
runs-on: ubuntu-latest
outputs:
API_TOKEN: ${{ steps.getauthtoken.outputs.API_TOKEN }}
steps:
- name: Get Token
id: getauthtoken
run: |
API_TOKEN:<there is a full JWT token here>
echo -n "API_TOKEN=$API_TOKEN" >> $GITHUB_OUTPUT
use-token:
runs-on: ubuntu-latest
needs: [get-service-url,get-auth-token]
name: Run Tests
steps:
- uses: actions/checkout#v3
- name: Run tests
run: |
newman run ${{ github.workspace }}/tests/collections/my_collection.json --env-var "service_url=${{needs.get-service-url.outputs.service_URL}}" --env-var "auth_token=${{needs.get-auth-token.outputs.API_TOKEN}}"
So, during a run, in my output I see:
Run newman run /home/runner/work/my-repo/my-repo/tests/collections/my_collection.json --env-var "service_url=https://test.net" --env-var "auth_token="
At first I thought there was something wrong in passing the token itself between jobs. Hence I tried
to put a dummy token an export it in the output. In my get-auth-token job, the call to output it became:
echo -n "API_TOKEN=test" >> $GITHUB_OUTPUT
and in the log I saw it there:
--env-var "auth_token=test"
so the way I am passing it intra jobs is fine. Moreover, the token is there and is correct because I hard coded one to simplify my tests. Indeed if in my get-auth-token job I try to echo $API_TOKEN I see in the logs *** which makes me understand Github is correctly obfuscating it.
I then tried not to pass it in between jobs. So I created the same token, hardcoded, right before the newman run command and referenced it in the newman run directly and tada! The log now is:
Run newman run /home/runner/work/my-repo/my-repo/tests/collections/my_collection.json --env-var "service_url=https://test.net" --env-var "auth_token=***"
So the token is there! But I need it to be coming from another job. There is something preventing the token to be passed in between jobs and I don't know how to achieve that.
Found out a trick to make this happen. Consists on temporarily "obfuscating" the secret to the eyes of Github.
In the job where I retrieve the secret I encode it and export it to GITHUB_OUTPUT:
API_TOKEN_BASE64=`echo -n <my_secret> | base64 -w 0`
echo -n "API_TOKEN=$API_TOKEN_BASE64" >> $GITHUB_OUTPUT
In the job where I need the secret I decode it (and use where needed):
API_TOKEN=`echo -n ${{needs.get-auth-token.outputs.API_TOKEN}} | base64 --decode`
I'm currently working on a pipeline where I'm using some backend to ensure my terraform forms will run but the problem that I have is that I'm getting the following issue:
Too many command-line arguments. Did you mean to use -chdir?
Error: Terraform exited with code 1.
Error: Process completed with exit code 1.
My current command is the following:
- name: Terraform Init
run: |
terraform init -backend=true -input=false \
-backend-config=subscription_id=${{ secrets.AZURE_SUBS_ID}} \
-backend-config=resource_group_name={{ secrets.AZURE_RG }} \
-backend-config=storage_accname=${{ SECRETS.AZURE_AD_STORAGE_ACCOUNT }} \
-backend-config=container_name="tfstate" \
-backend-config=tenant_id=${{ secrets.AZURE_TENANT_ID }} \
-backend-config=client_id=${{ secrets.AZURE_CLIENT_ID }} \
-backend-config=client_secret=${{ secrets.AZURE_CSECRET }}
By using my terminal, I can use a file like this, and I'm able to set up all my environment variables on my local machine. However, when I'm using the pipelines it seems that I can't do that. Does anyone know what is the best thing to do?
This is most likely a YAML formatting issue here as the Terraform argument parsing logic is throwing the error. The | inserts newline characters at the end of each line. You need > instead. The \ are also unnecessary as this is a multi-line YAML string, and not a multi-line shell interpreter command:
- name: Terraform Init
run: >
terraform init -backend=true -input=false
-backend-config=subscription_id=${{ secrets.AZURE_SUBS_ID}}
-backend-config=resource_group_name={{ secrets.AZURE_RG }}
-backend-config=storage_accname=${{ SECRETS.AZURE_AD_STORAGE_ACCOUNT }}
-backend-config=container_name="tfstate"
-backend-config=tenant_id=${{ secrets.AZURE_TENANT_ID }}
-backend-config=client_id=${{ secrets.AZURE_CLIENT_ID }}
-backend-config=client_secret=${{ secrets.AZURE_CSECRET }}
tengo creado un pipeline de CI/CD con Bitbucket Pipelines. Dentro del archivo bitbucket-pipelines.yml defini un custom pipe. Ej:
custom:
manual-deploy:
- step:
name: Manuel Deploy
services:
- docker
caches:
- maven
script:
- echo "Deploy..."
I need to get the username or nickname of the user who triggered a custom step. How can I do this?
I read in the documentation that the variable BITBUCKET_STEP_TRIGGERER_UUID exists, but I don't know how to identify which user this UUID belongs to
You can use the $BITBUCKET_STEP_TRIGGERER_UUID to resolve the actual username the following way:
export BITBUCKET_TRIGGERER_USERNAME=$(curl -X GET -g "https://api.bitbucket.org/2.0/users/${BITBUCKET_STEP_TRIGGERER_UUID}" | jq --raw-output '.display_name')
I found the answer in one of the comments here: https://jira.atlassian.com/browse/BCLOUD-16711
I am trying to deploy a cloud function using cloudbuild.yaml. It works fine if I don't use any conditional statement. I am facing an error when I execute my cloudbuild.yaml file with if conditional statement. What is the correct way to write it. Below is my code:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
id: deploy
args:
- '-c'
- 'if [ $BRANCH_NAME != "xoxoxoxox" ]
then
[
'functions', 'deploy', 'groups',
'--region=us-central1',
'--source=.',
'--trigger-http',
'--runtime=nodejs8',
'--entry-point=App',
'--allow-unauthenticated',
'--service-account=xoxoxoxox#appspot.gserviceaccount.com'
]
fi'
dir: 'API/groups'
Where am I doing it wrong ?
From the github page, https://github.com/GoogleCloudPlatform/cloud-sdk-docker, the entrypoint is not set to gcloud. So you cannot specify the arguments like that.
Good practice for specifying directory is to start with /workspace
Also the right way to write the step should be
steps:
- name: 'gcr.io/cloud-builders/gcloud'
id: deploy
dir: '/workspace/API/groups'
entrypoint: bash
args:
- '-c'
- |
if [ $BRANCH_NAME != "xoxoxoxox" ]
then
gcloud functions deploy groups
--region=us-central1
--source=.
--trigger-http
--runtime=nodejs8
--entry-point=App
--allow-unauthenticated
--service-account=xoxoxoxox#appspot.gserviceaccount.com
fi
I'm not sure you can do this.
In my case, I use the branch selector in the Cloud build trigger to select which branch (or tag) I want to build from a pattern.
I wanted to delete the latest version of each service only if there were more than two previous versions. This was my solution.
args:
- "-c"
- |
if [[ $(gcloud app versions list --format="value(version.id)" --service=MY-SERVICE | wc -l) -ge 2 ]];
then
gcloud app versions list --format="value(version.id)" --sort-by="~version.createTime" --service=admin | tail -n -1 | xargs gcloud app versions delete --service=MY-SERVICE --quiet;
fi
I have several Bash scripts that invoke AWS CLI commands for which permissions have changed to require MFA, and I want to be able to prompt for a code generated by my MFA device in these scripts so that they can run with the necessary authentication.
But there seems to be no simple built in way to do this. The only documentation I can find involves a complicated process of using aws sts get-session-token and then saving each value in a configuration, which it is then unclear how to use.
To be clear what I'd like is that when I run one of my scripts that that contains AWS CLI commands that require MFA, I'm simply prompted for the code, so that providing it allows the AWS CLI operations to complete. Something like:
#!/usr/bin/env bash
# (1) prompt for generated MFA code
# ???
# (2) use entered code to generate necessary credentials
aws sts get-session-token ... --token-code $ENTERED_VALUE
# (3) perform my AWS CLI commands requiring MFA
# ....
It's not clear to me how to prompt for this when needed (which is probably down to not being proficient with bash) or how to use the output of get-session-token once I have it.
Is there a way to do what I'm looking for?
I've tried to trigger a prompt by specifying a --profile with a mfa_serial entry; but that doesn't work either.
Ok after spending more time on this script with a colleague - we have come up with a much simpler script. This does all the credential file work for you , and is much easier to read. It also allows for all your environments new tokens to be in the same creds file. The initial call to get you MFA requires your default account keys in the credentials file - then it generates your MFA token and puts them back in the credentials file.
#!/usr/bin/env bash
function usage {
echo "Example: ${0} dev 123456 "
exit 2
}
if [ $# -lt 2 ]
then
usage
fi
MFA_SERIAL_NUMBER=$(aws iam list-mfa-devices --profile bh${1} --query 'MFADevices[].SerialNumber' --output text)
function set-keys {
aws configure set aws_access_key_id ${2} --profile=${1}
aws configure set aws_secret_access_key ${3} --profile=${1}
aws configure set aws_session_token ${4} --profile=${1}
}
case ${1} in
dev|qa|prod) set-keys ${1} $(aws sts get-session-token --profile bh${1} --serial-number ${MFA_SERIAL_NUMBER} --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' --output text --token-code ${2});;
*) usage ;;
esac
Inspired by #strongjz and #Nick answers, I wrote a small Python command to which you can pipe the output of the aws sts command.
To install:
pip install sts2credentials
To use:
aws sts get-session-token \
--serial-number arn:aws:iam::123456789012:mfa/your-iam-user \
--token-code 123456 \
--profile=your-profile-name \
| sts2credentials
This will automatically add the access key ID, the secret access key, and the session token under a new "sts" profile in your ~/.aws/credentials file.
For bash you could read in the value, then set those values from the sts output
echo "Type the mfa code that you want to use (4 digits), followed by [ENTER]:"
read ENTERED_VALUE
aws sts get-session-token ... --token-code $ENTERED_VALUE
then you'll have to parse the output of the sts call which has the access key, secret and session token.
{
Credentials: {
AccessKeyId: "ASIAJPC6D7SKHGHY47IA",
Expiration: 2016-06-05 22:12:07 +0000 UTC,
SecretAccessKey: "qID1YUDHaMPet5xw/vpw1Wk8SKPilFihdiMSdSIj",
SessionToken: "FQoDYXdzEB4aDLwmzouEQ3eckfqJxyLOARbBGasdCaAXkZ7ABOcOCNx2/7sS8N7A6Dpcax/t2G8KNTcUkRLdxI0gTvPoKQeZrH8wUrL4UxFFP6kCWEasdVIBAoUfuhdeUa1a7H216Mrfbbv3rMGsVKUoJT2Ar3r0pYgsYxizOWzH5VaA4rmd5gaQvfSFmasdots3WYrZZRjN5nofXJBOdcRd6J94k8m5hY6ClfGzUJEqKcMZTrYkCyUu3xza2S73CuykGM2sePVNH9mGZCWpTBcjO8MrawXjXj19UHvdJ6dzdl1FRuKdKKeS18kF"
}
}
then set them
aws configure set aws_access_key_id default_access_key --profile NAME_PROFILE
aws configure set aws_secret_access_key default_secret_key --profile NAME_PROFILE
aws configure set default.region us-west-2 --profile
aws some_commmand --profile NAME_PROFILE
http://www.tldp.org/LDP/Bash-Beginners-Guide/html/sect_08_02.html
AWS STS API Reference
http://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html
AWS CLI STS Command
http://docs.aws.amazon.com/cli/latest/reference/sts/get-session-token.html
I wrote something very similar to what you are trying to in Go, here but this is for the sts assumerole not get-session-token.
I wrote a simple script to set the AWS credentials file for a profile called mfa. Then all bash scripts you write just need to have the "--profile mfa" added so they will just work. This also allows for multiple AWS accounts - as many of us have those these days. I'm sure this can be improved - but it was quick and dirty and does what you want and everything I need.
You will have to amend facts in the script to fit your account details - I have marked them clearly with chevrons < >. NB Obviously once you have populated the script with all your details it is not to be copied about - unless you want unintended consequences. This uses recursion within the credentials file - as the standard access keys are called each time to create the mfa security tokens.
#!/bin/bash
# Change for your username - would be /home/username on Linux/BSD
dir='/Users/<your-user-name>'
region=us-east-1
function usage {
echo "Must enter mfa token and then either dev/qa/prod"
echo "i.e. mfa-set-aws-profile.sh 123456 qa"
exit 2
}
if [[ $1 == "" ]]
then
echo "Must give me a token - how do you expect this to work - DOH :-)"
usage
exit 2
fi
# Write the output from sts command to a json file for parsing
# Just add accounts below as required
case $2 in
dev) aws sts get-session-token --profile dev --serial-number arn:aws:iam::<123456789>:mfa/<john.doe> --token-code $1 > $dir/mfa-json;;
qa) aws sts get-session-token --profile qa --serial-number arn:aws:iam::<123456789>:mfa/<john.doe> --token-code $1 > $dir/mfa-json;;
-h) usage ;;
*) usage ;;
esac
# Remove quotes and comma's to make the file easier to parse -
# N.B. gsed is for OSX - on Linux/BSD etc sed should be just fine.
/usr/local/bin/gsed -i 's/\"//g;s/\,//g' $dir/mfa-json
# Parse the mfa info into vars for use in aws credentials file
seckey=`cat $dir/mfa-json | grep SecretAccessKey | gsed -E 's/[[:space:]]+SecretAccessKey\: //g'`
acckey=`cat $dir/mfa-json | grep AccessKeyId | gsed 's/[[:space:]]+AccessKeyId\: //g'`
sesstok=`cat $dir/mfa-json | grep SessionToken | gsed 's/[[:space:]]+SessionToken\: //g'`
# output all the gathered info into your aws credentials file.
cat << EOF > $dir/.aws/credentials
[default]
aws_access_key_id = <your normal keys here if required>
aws_secret_access_key = <your normal keys here if required>
[dev]
aws_access_key_id = <your normal keys here >
aws_secret_access_key = <your normal keys here >
[qa]
aws_access_key_id = <your normal keys here >
aws_secret_access_key = <your normal keys here >
[mfa]
output = json
region = $region
aws_access_key_id = $acckey
aws_secret_access_key = $seckey
aws_session_token = $sesstok
EOF