I am trying to install the following Jelastic manifest:
type: update
name: Not working
onInstall:
- getPublicKey
- forEach(nodes.cp):
- api:
- method: environment.control.SetContainerEnvVars
params:
nodeId: ${#i.id}
vars:
SOME_SECRET: '{"type": "RS256", "key": "${response.out}"}'
actions:
getPublicKey:
- cmd [cp]: |
curl -s -H "Authorization: my-token" http://${nodes.auth.master.intIP}:9011/api/key | jq -r '.keys[] | select(.algorithm | contains("RS256")).publicKey'
The response output in the Jelastic console for the getPublicKey action reads:
cmd [cp: 113094].response: {"result":0,"errOut":"","nodeid":113094,"exitStatus":0,"out":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAprg9fCTMSfm6psfOnhfL\nUDdlrV39LU9p8msWsYUjO4M2E6m5GcADYPHkLdLV/c7W+vgDvpHLfxU8peP/21BS\nCVQVYEFpYCRY2NcOTsP2zLj7PTAFiw8wyOwK7u05EM7CgK7LS6rDotMIZFNzIPG5\nfJNz+hDyhvvNhWg56dcmPIrBxN26Piv+N6vtWqJDuVQNXKwEk/w4uUxiz9gNSEi/\nhJlLgHxTsSMh9YXUIyKn8QBACF4GQKmToBPW7ScEnX/Bm6y9g4JbYYIwWBTRwUfy\nkhbojk6mAPcKY+diWM2PE385pyjIWshKUgtBKcgPNJXDU3RPXAdzN0hQ1sJbNV5z\nzwIDAQAB\n-----END PUBLIC KEY-----"}
Here we can clearly see the newlines displayed as \n characters. For some reason, the newline characters \n are interpreted as true newlines in the environment variable SOME_SECRET, which now reads incorrectly (this is the result of the env command):
SOME_SECRET={"type": "RS256", "key": "-----BEGIN PUBLIC KEY-----
The variable value is displayed until the first newline character, which is not what I want.
How can I make sure the \n get outputted to the SOME_SECRET environment variable and that they are not replaced with a real new line? When I execute the env command on my ubuntu image, I want to see this:
SOME_SECRET={"type": "RS256", "key": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAprg9fCTMSfm6psfOnhfL\nUDdlrV39LU9p8msWsYUjO4M2E6m5GcADYPHkLdLV/c7W+vgDvpHLfxU8peP/21BS\nCVQVYEFpYCRY2NcOTsP2zLj7PTAFiw8wyOwK7u05EM7CgK7LS6rDotMIZFNzIPG5\nfJNz+hDyhvvNhWg56dcmPIrBxN26Piv+N6vtWqJDuVQNXKwEk/w4uUxiz9gNSEi/\nhJlLgHxTsSMh9YXUIyKn8QBACF4GQKmToBPW7ScEnX/Bm6y9g4JbYYIwWBTRwUfy\nkhbojk6mAPcKY+diWM2PE385pyjIWshKUgtBKcgPNJXDU3RPXAdzN0hQ1sJbNV5z\nzwIDAQAB\n-----END PUBLIC KEY-----"}
The following manifest does exactly what I want:
type: update
name: Working
onInstall:
- getPublicKey
- api:
- method: environment.control.AddContainerEnvVars
params:
nodeGroup: cp
vars:
SOME_SECRET: '{"type": "RS256", "key": "${response.out}"}'
actions:
getPublicKey:
- cmd [cp]: |
curl -s -H "Authorization: some-key" http://${nodes.auth.master.intIP}:9011/api/key | jq -r '.keys[] | select(.algorithm | contains("RS256")).publicKey' | sed ':a;N;$!ba;s/\n/\\n/g'
Note the use of sed in the getPublicKey to format the response output appropriately. Also, note the use of environment.control.AddContainerEnvVars instead of environment.control.SetContainerEnvVars. The latter removes all environment variables on the node group (except PATH), while the former really adds new environment variables (and overwrites existing ones with the new values).
Finally, note that it is necessary to have the getPublicKey action. The following manifest would not work (I don't know why):
type: update
name: Not Working
onInstall:
- cmd [cp]: |
curl -s -H "Authorization: some-key" http://${nodes.auth.master.intIP}:9011/api/key | jq -r '.keys[] | select(.algorithm | contains("RS256")).publicKey' | sed ':a;N;$!ba;s/\n/\\n/g'
- api:
- method: environment.control.AddContainerEnvVars
params:
nodeGroup: cp
vars:
SOME_SECRET: '{"type": "RS256", "key": "${response.out}"}'
Related
I'm attempting to pass an array of dynamically fetched data from one GitHub Action job to the actual job doing the build. This array will be used as part of a matrix to build for multiple versions. However, I'm encountering an issue when the bash variable storing the array is evaluated.
jobs:
setup:
runs-on: ubuntu-latest
outputs:
versions: ${{ steps.matrix.outputs.value }}
steps:
- id: matrix
run: |
sudo apt-get install -y jq && \
MAINNET=$(curl https://api.mainnet-beta.solana.com -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id":1, "method":"getVersion"}' | jq '.result["solana-core"]') && \
TESTNET=$(curl https://api.testnet.solana.com -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id":1, "method":"getVersion"}' | jq '.result["solana-core"]') && \
VERSIONS=($MAINNET $TESTNET) && \
echo "${VERSIONS[#]}" && \
VERSION_JSON=$(echo "${VERSIONS[#]}" | jq -s) && \
echo $VERSION_JSON && \
echo '::set-output name=value::$VERSION_JSON'
shell: bash
- id: debug
run: |
echo "Result: ${{ steps.matrix.outputs.value }}"
changes:
needs: setup
runs-on: ubuntu-latest
# Set job outputs to values from filter step
outputs:
core: ${{ steps.filter.outputs.core }}
package: ${{ steps.filter.outputs.package }}
strategy:
matrix:
TEST: [buy, cancel, create_auction_house, delegate, deposit, execute_sale, sell, update_auction_house, withdraw_from_fee, withdraw_from_treasury, withdraw]
SOLANA_VERSION: ${{fromJson(needs.setup.outputs.versions)}}
steps:
- uses: actions/checkout#v2
# For pull requests it's not necessary to checkout the code
- uses: dorny/paths-filter#v2
id: filter
with:
filters: |
core:
- 'core/**'
package:
- 'auction-house/**'
- name: debug
id: debug
working-directory: ./auction-house/program
run: echo ${{ needs.setup.outputs.versions }}
In the setup job above, the two versions are evaluated to a bash array (in VERSIONS) and converted into a JSON array to be passed to the next job (in VERSION_JSON). The last echo in the matrix step results in a print of [ "1.10.31", "1.11.1" ], but the debug step prints out this:
Run echo "Result: "$VERSION_JSON""
echo "Result: "$VERSION_JSON""
shell: /usr/bin/bash -e {0}
env:
CARGO_TERM_COLOR: always
RUST_TOOLCHAIN: stable
Result:
The changes job also results in an error:
Error when evaluating 'strategy' for job 'changes'.
.github/workflows/program-auction-house.yml (Line: 44, Col: 25): Unexpected type of value '$VERSION_JSON', expected type: Sequence.
It definitely seems like the $VERSION_JSON variable isn't actually being evaluated properly, but I can't figure out where the evaluation is going wrong.
For echo '::set-output name=value::$VERSION_JSON' you need to use double quotes or bash would not expand $VERSION_JSON.
set-output is not happy with multi-lined data. For your case, you can use jq -s -c so the output will be one line.
The below value of private_localaddress needs to passed to "range_start" in yaml file. how to do this ?
Here is the script:
#!/bin/bash
MAC=`curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/`
myarr=($MAC)
for val in ${myarr[#]}; do
interfaceindex=`curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/$val/device-number`
if [ $interfaceindex == 2 ];
then
private_localaddress=`curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/$val/local-ipv4s`
echo $private_localaddress
fi
done
Here is the YAML file :
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: worker-private-eth2
spec:
config: '{
"cniVersion": "0.3.1",
"ipam": {
"type": "whereabouts",
"range": "10.30.11.0/25",
"range_start": $private_localaddress,
"range_end": "10.30.11.127",
"routes": [
{ "dst": "10.93.123.0/24", "gw": "10.30.11.1" }
],
Ffffff
Use a proper yaml parser. Here are some examples:
Using mikefarah/yq:
value="$private_localaddress" yq \
'.spec.config |= (fromjson | .ipam.range_start = strenv(value) | tojson)'
Using kislyuk/yq:
yq -y --arg value "$private_localaddress" \
'.spec.config |= (fromjson | .ipam.range_start = $value | tojson)'
Using itchyny/gojq:
gojq --arg value "$private_localaddress" --yaml-input --yaml-output \
'.spec.config |= (fromjson | .ipam.range_start = $value | tojson)'
Perhaps the simplest would be to make private_localaddress an environment variable and use envsubst to apply the changes to your YAML.
However to end up with correct YAML code, you need to ensure that you have a quoted string to the right of the colon. The best would be if you can already ensure that your YAML is written as
"range_start": "$private_localaddress",
If this is not feasible, you have to add the quotes when creating the variable, i.e.
export private_localaddress=\"$(curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/$val/local-ipv4s)\"
Any example to set from and to path in $HOME/.dlv/config.yml ? I have tried setting:
case 1 ( with quotes in from and to)
substitute-path:
{"from":
"/private/var/tmp/_bazel/d3eb9a0ef06857aebc54b41ff425d2ee"
"to": "/Users/xxx/code/src"}
case 2: ( without quotes in from and to)
substitute-path:
{from:
"/private/var/tmp/_bazel/d3eb9a0ef06857aebc54b41ff425d2ee"
to: "/Users/xxx/code/src"}
case 2: ( with hyphan before '{from' )
substitute-path:
-{from:
"/private/var/tmp/_bazel/d3eb9a0ef06857aebc54b41ff425d2ee"
to: "/Users/xxx/code/src"}
case 2: ( with hyphan before '{from' )
substitute-path:
-{"from":
"/private/var/tmp/_bazel/d3eb9a0ef06857aebc54b41ff425d2ee"
"to": "/Users/xxx/code/src"}
all 4 cases fails with config. error.
any working sample to set the path?
I think you are missing a space. The quotes are relevant only if the values or keys have spaces in them, or some other non-printable, non-ascii characters.
The config here, without comments:
$ cat ~/.config/dlv/config.yml | sed '/^#/d; /^$/d'
aliases:
# command: ["alias1", "alias2"]
substitute-path:
- {from: /my/source/code/was/here, to: /but/now/its/here}
debug-info-directories: ["/usr/lib/debug/.build-id"]
Seems like valid yaml:
$ yq < ~/.config/dlv/config.yml
{
"aliases": null,
"substitute-path": [
{
"from": "/my/source/code/was/here",
"to": "/but/now/its/here"
}
],
"debug-info-directories": [
"/usr/lib/debug/.build-id"
]
}
The yq tool is a wrapper for jq.
$ yq --help | sed 8q
usage: yq [options] <jq filter> [input file...]
[jq_filter] [files [files ...]]
yq: Command-line YAML processor - jq wrapper for YAML documents
yq transcodes YAML documents to JSON and passes them to jq.
See https://github.com/kislyuk/yq for more information.
I am trying to implement a ci/cd pipeline using gitlab and i created a ci file with the following content.
stages:
- deploy
image:
name: "ubuntu:16.04"
first-pipeline:test:
stage: deploy
tags:
- executor:docker
only:
refs:
- branches
- schedules
script:
- export ANSIBLE_HOST_KEY_CHECKING=False
- echo "Job: $job_param"
- ansible-playbook -i production.ini -e "job_id=$job_param ansible_ssh_user=ubuntu" my-playbook.yml -l "10.37.23.230"
- apk add curl
- curl -X POST http://98.121.222.32:8080/api/v2/removejob -H 'Content-Type: application/json' -d "{"jobId": $job_param}"
- echo $query
- echo "Executed at= $now"
I keep running into the following error message : bad indentation of a mapping entry
24 | ...
25 | ... ubuntu" my-playbook.yml -l "10.37.23.230"
26 | ...
27 | ... application/json' -d "{"jobId": $job_param}"
-----------------------------------------^
28 | ...
29 | ...
Any suggestion on how to fix it? Any help will be appreciated. Thank you.
This worked :-
'curl -H "Content-Type: application/json" -X POST http://98.121.222.32:8080/api/v2/removejob -d "{"jobId": $job_param}"'
I am trying to launch an awx/tower job template passing a list of values in a variable but the task is getting executed only on one target host.
sample request
curl -H "Content-Type: application/json" -X POST -s -u admin:admin123 -d '{ "extra_vars": { "domain": "dom-cn-1", "targets": "dev-cn-c1", "targets": "dev-cn-c2", "fwcmd": "fw sam -v -J src 192.168.10.10" }}' -k https://172.16.102.4/api/v2/job_templates/10/launch/
The above command does not execute as expected and runs on a single host. However, this is working as expected when I run the same playbook via cli.
vars file snip
domain: dom-cn-1
targets:
- dev-cn-c1
- dev-cn-c2
Playbook file
- name: "Create output file"
check_point_mgmt:
command: run-script
parameters:
script-name: "Create output file"
script: "fw sam -v -J src 192.168.10.10"
targets: "{{ targets }}"
session-data: "{{login_response}}"
Let's extract your json in your curl command:
{
"extra_vars": {
"domain": "dom-cn-1",
"targets": "dev-cn-c1",
"targets": "dev-cn-c2",
"fwcmd": "fw sam -v -J src 192.168.10.10"
}
}
You are not passing a list but twice the same parameter with different values. You have to correct you json like this:
{
"extra_vars": {
"domain": "dom-cn-1",
"targets": ["dev-cn-c1", "dev-cn-c2"],
"fwcmd": "fw sam -v -J src 192.168.10.10"
}
}