Trying to save API responses in their original JSON format as env variables. Getting formatting errors.
echo "GIT_PR_OBJECT=$(curl -u USER:TOKEN https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.number }})" >> $GITHUB_ENV
Error from Github:
Error: Unable to process file command 'env' successfully.
Error: Invalid environment variable format ' "sha": "123456789",'
My plan is to access this object multiple times throughout the workflow, that way I only need to make one call to Github API rather than few. Here is how I access the data object saved in the env var.
echo "GIT_EMAIL=$(${{ env.GIT_COMMIT_OBJECT }} | jq -r '.commit.author.email')" >> $GITHUB_ENV
How can I save my curl response as an env variable in it's JSON format?
Why not save json content to file instead? Saving to environment variable, while sounds fancy, also smells like a can of worms you don't have to deal with in a first place.
Taking your example:
- run: |
curl -u USER:TOKEN https://api.github.com/repos/${{ github.repository }}/pulls/${{ github.event.number }} > $HOME/pr.json
echo "GIT_EMAIL=$(jq -r '.commit.author.email' $HOME/pr.json)" >> $GITHUB_ENV
shell: bash
While it is a working solution in my eyes, i fail to understand why it has to be environment variable, which is your main question here.
Related
I'm currently working on a pipeline where I'm using some backend to ensure my terraform forms will run but the problem that I have is that I'm getting the following issue:
Too many command-line arguments. Did you mean to use -chdir?
Error: Terraform exited with code 1.
Error: Process completed with exit code 1.
My current command is the following:
- name: Terraform Init
run: |
terraform init -backend=true -input=false \
-backend-config=subscription_id=${{ secrets.AZURE_SUBS_ID}} \
-backend-config=resource_group_name={{ secrets.AZURE_RG }} \
-backend-config=storage_accname=${{ SECRETS.AZURE_AD_STORAGE_ACCOUNT }} \
-backend-config=container_name="tfstate" \
-backend-config=tenant_id=${{ secrets.AZURE_TENANT_ID }} \
-backend-config=client_id=${{ secrets.AZURE_CLIENT_ID }} \
-backend-config=client_secret=${{ secrets.AZURE_CSECRET }}
By using my terminal, I can use a file like this, and I'm able to set up all my environment variables on my local machine. However, when I'm using the pipelines it seems that I can't do that. Does anyone know what is the best thing to do?
This is most likely a YAML formatting issue here as the Terraform argument parsing logic is throwing the error. The | inserts newline characters at the end of each line. You need > instead. The \ are also unnecessary as this is a multi-line YAML string, and not a multi-line shell interpreter command:
- name: Terraform Init
run: >
terraform init -backend=true -input=false
-backend-config=subscription_id=${{ secrets.AZURE_SUBS_ID}}
-backend-config=resource_group_name={{ secrets.AZURE_RG }}
-backend-config=storage_accname=${{ SECRETS.AZURE_AD_STORAGE_ACCOUNT }}
-backend-config=container_name="tfstate"
-backend-config=tenant_id=${{ secrets.AZURE_TENANT_ID }}
-backend-config=client_id=${{ secrets.AZURE_CLIENT_ID }}
-backend-config=client_secret=${{ secrets.AZURE_CSECRET }}
I need to use some gcloud commands in order to create a Redis instance on GCP as terraform does not support some options that I need.
I'm trying this:
terraform {
# Before apply, run script.
before_hook "create_redis_script" {
commands = ["apply"]
execute = ["REDIS_REGION=${local.module_vars.redis_region}", "REDIS_PROJECT=${local.module_vars.redis_project}", "REDIS_VPC=${local.module_vars.redis_vpc}", "REDIS_PREFIX_LENGHT=${local.module_vars.redis_prefix_lenght}", "REDIS_RESERVED_RANGE_NAME=${local.module_vars.redis_reserved_range_name}", "REDIS_RANGE_DESCRIPTION=${local.module_vars.redis_range_description}", "REDIS_NAME=${local.module_vars.redis_name}", "REDIS_SIZE=${local.module_vars.redis_size}", "REDIS_ZONE=${local.module_vars.redis_zone}", "REDIS_ALT_ZONE=${local.module_vars.redis_alt_zone}", "REDIS_VERSION=${local.module_vars.redis_version}", "bash", "../../../scripts/create-redis-instance.sh"]
}
The script is like this:
echo "[+]Creating IP Allocation Automatically using <$REDIS_VPC-network\/$REDIS_PREFIX_LENGHT>"
gcloud compute addresses create $REDIS_RESERVED_RANGE_NAME \
--global \
--purpose=VPC_PEERING \
--prefix-lenght=$REDIS_PREFIX_LENGHT \
--description=$REDIS_RANGE_DESCRIPTION \
--network=$REDIS_VPC
The error I get is:
terragrunt apply
5b35d0bf15d0a0d61b303ed32556b85417e2317f
5b35d0bf15d0a0d61b303ed32556b85417e2317f
5b35d0bf15d0a0d61b303ed32556b85417e2317f
ERRO[0002] Hit multiple errors:
Hit multiple errors:
exec: "REDIS_REGION=us-east1": executable file not found in $PATH
ERRO[0002] Unable to determine underlying exit code, so Terragrunt will exit with error code 1
I encountered the same issue and resigned myself to pass the values as parameters instead of environment variables.
It involves to modify the script and is a far less clearer declaration, but it works :|
I want to write a shell script to login and get bugs for a project. I want the dashboard values like bugs, Vulnerabilities, code smells and coverage.
The url of dashboard is: http://www.example.com/dashboard?id=example_project_name.
Here is what I tried:
curl GET -u username:password http://www.example.com/api/issues/search?project=example_project_name&types=BUG.
So, this prints all the data. I just need the value show in the below image:
Basically What I want to achieve is that I’m using a Sonarqube plugin in Jenkins, so I use extended email plugin to send email for job execution and in that email I want to give details like number of bugs in the repository after the build.
Is there any other way?
Finally after reading the documentation carefully, I got the values. Here is the script that I created.
#!/bin/bash
vul=$(curl -sX GET -u username:password 'http://www.example.com/api/issues/search?projectKeys=example_project_name&types=VULNERABILITY');
bug=$(curl -sX GET -u username:password 'http://www.example.com/api/issues/search?projectKeys=example_project_name&types=BUG');
no_vul=$(echo $vul | jq -r .total);
no_bug=$(echo $bug | jq -r .total);
echo "Total number of VULNERABILITIES are $no_vul"
echo "Total number of BUGS are $no_bug"
Here is the API documentation URL.
I've ran into a strange issue. I'm trying to script my router to collect usage stats and other stuff. I'm making one cURL to the auth URL to get a valid session id, then another using that session id to the page I need.
Here is my script:
SESSION_ID=$(curl --silent -D - -X POST http://10.0.0.1/login.cgi -d'admin_username=admin&admin_password=admin' | grep 'SESSION' | sed 's/Set-Cookie: SESSION=//' | sed 's/; path=\///')
echo $SESSION_ID # 1234567890
curl -v -H "Cookie: SESSION=$SESSION_ID" http://10.0.0.1/modemstatus_dslstatus.html
If I manually take SESSION_ID and insert it in place of '"$SESSION_ID"' everything is dandy. cURL shows the headers (via -v) and they are correct. Running the command while manually inserting the session id produces identical headers.
I'm sure it's something small. Please teach me something :)
Check for carriage returns \r in your variables which wouldn't appear with a simple echo in some cases.
I have a shell script for getting data from some servers, in ~/.bin/shellScript :
OLD="$(curl --silent http://someServer:12345/stats.json | json someKey)"
NEW="$(curl --silent http://otherServer:12345/stats.json | json someKey")
echo "OLD $OLD - NEW $NEW"
I want to echo the results for running it interactively, but I've been wanting to log the results collected too.
So crontab -e, and add */5 * * * * /home/user/.bin/shellScript >> /media/dump/scriptDump.
Running the script interactively works fine - I get OLD 123 - NEW 456, but when I look at what's been running from cron, I get OLD - NEW with no data being grabbed with curl.
As discovered in the comments, you need to add the full path of json when you are calling it. This is because crontab's limited environment.
So instead of
OLD="$(curl --silent http://someServer:12345/stats.json | json someKey)"
NEW="$(curl --silent http://otherServer:12345/stats.json | json someKey")
it has to be
OLD="$(curl --silent http://someServer:12345/stats.json | /path/to/json someKey)"
NEW="$(curl --silent http://otherServer:12345/stats.json | /path/to/json someKey)"
^^^^^^^^^^^^^
[[Note your second line had ") instead of )"]]
Otherwise, you can also add the json path into crontab, as indicated on How to get CRON to call in the correct paths:
PATH=/usr/local/sbin: ... :/path/of/json