Bash getting escaped variable key value with script - bash

I want to use jq to get value's out of a the Traefik API service. The output of the service looks like this:
{
"loadBalancer": {
"servers": [
{
"url": "http://172.18.0.19:3000"
},
{
"url": "http://172.18.0.17:3000"
},
{
"url": "http://172.20.0.3:3000"
}
],
"healthCheck": {
"path": "/health",
"interval": "10s",
"timeout": "10s",
"followRedirects": true
},
"passHostHeader": true
},
"status": "enabled",
"serverStatus": {
"http://172.18.0.17:3000": "UP",
"http://172.18.0.19:3000": "UP",
"http://172.20.0.3:3000": "DOWN"
},
"provider": "docker",
"type": "loadbalancer"
}
I want to get the value of the serverStatus dynamically. First assigning a variable with the correct IP address from docker inspect. I'm using curl with jq to check if the container is being serviced.
The problem is when getting it with a variable I cannot seem to escape the special characters in the url key. Nothing is returned.
This is the command I use is:
TRAEFIK_URL="http://someurl.com/"
TRAEFIK_URL="'Authorization: Basic hashkey"
NEW_IPADDRESS=$(docker container inspect some_container_name | jq -r .[].NetworkSettings.Networks.proxy.IPAddress)
NEW_ENV_URL="http://${NEW_IPADDRESS}:3000"
curl --location --request GET "$TRAEFIK_URL" --header "$TRAEFIK_AUTH_HEADER" | jq -r '.serverStatus["${NEW_ENV_URL}"]'
I've tried using this command and some other obvious options but all didn't work.
jq -r .serverStatus[$NEW_ENV_URL]
I do get the correct value when I'm not using a variable to get the status using:
jq -r '.serverStatus["http://172.18.0.17:3000"]'
Any help is welcome.

Use the option --arg to pass arguments to jq.
jq --arg url http://172.18.0.17:3000 -r '.serverStatus[$url]'
Always use this option to avoid security problems by injections.

One of the preferred ways to pass specific environment variables into jq is using the --arg command-line option; in your case:
jq -r --arg url "${NEW_ENV_URL}" '.serverStatus[$url]'

Related

Gerrit Checks plugin - How to create/update Checker for a gerrit change with API

My Gerrit server installed Checks plugin and I also try to create many checkers for my repo.
I can get information about Checkers of a gerrit change by admin account (configured Administrate Checkers and password was generated for "HTTP Credentials" in settings menu) with curl command:
url="http://my_gerrit_ip:8081"
curl -k -X GET \
-u admin:Jy7xK+oJ2tA2yzc+jG5n7yttsFUNmXDNKEDh+PoygQ \
--header "Content-Type: application/json; charset=UTF-8" \
--data gerrit-field-content.json \
$url/changes/66/revisions/1/checks/
The output looks like:
[
{
"repository": "test2",
"change_number": 66,
"patch_set_id": 1,
"checker_uuid": "a:a",
"state": "NOT_STARTED",
"created": "2020-10-28 02:56:55.000000000",
"updated": "2020-10-28 02:56:55.000000000",
"checker_name": "test2",
"checker_status": "ENABLED",
"blocking": [
"STATE_NOT_PASSING"
],
"submit_impact": {
"required": true
},
"checker_description": "test2"
},
{
"repository": "test2",
"change_number": 66,
"patch_set_id": 1,
"checker_uuid": "b:b",
"state": "NOT_STARTED",
"created": "2020-10-28 02:56:55.000000000",
"updated": "2020-10-28 02:56:55.000000000",
"checker_name": "test2_2",
"checker_status": "ENABLED",
"blocking": [],
"submit_impact": {},
"checker_description": "test2_2"
}
]
But now, i want to update the state of a Checker of my gerrit change via curl command too but it throw message "Authentication required".
I tried with:
curl -k -X POST \
-u admin:Jy7xK+oJ2tA2yzc+jG5n7yttsFUNmXDNKEDh+PoygQ \
--header "Content-Type: application/json; charset=UTF-8" \
--data gerrit-field-content.json \
$url/changes/66/revisions/1/checks/
The gerrit-field-content.json content:
{
"checker_uuid": "a:a",
"state": "SUCCESSFUL",
"url": "",
"started": ""
}
How can I update state of a checker via api? Or are there any better way to do this?
Result of checker state maybe look like a sample in this one https://gerrit-review.googlesource.com/c/gerrit/+/285580
Note: the it's not able update checker when i leave password for admin empty (curl -u admin:) because my Gerrit server is not require password for logging in.
Afraid I am just poking around myself, but in the zuul change it says:
* Access control for the `checks` API in Gerrit depends on a single
global administrative permission, ``administrateCheckers``. This is
required in order to use the `checks` API and can not be restricted
by project. This means that any system using the `checks` API can
interfere with any other.
Hope that is helpful, as I deploy this I'll update if I gain any other useful experience

How to read response of a JIRA REST API

Using the Atlassian documentation, I'm able to create a new JIRA ticket using curl command. I want to understand how to read the response of this command into a variable? Essentially i want to store the JIRA ticket ID into a variable that can be used in other scripts.
Reference Link: https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/
_SAMPLE_CODE_
curl -D- -u charlie:charlie -X POST --data '{
"fields": {
"project":{
"key": "TEST"
},
"summary": "REST ye merry gentlemen.",
"description": "Creating of an issue using project keys and issue type names using the REST API",
"issuetype": {
"name": "Bug"
}
}
}' -H "Content-Type: application/json" http://localhost:8080/rest/api/2/issue/
_SAMPLE_RESPONSE_
{
"id":"39000",
"key":"TEST-101",
"self":"http://localhost:8080/rest/api/2/issue/39000"
}
From the above log, I want to understand how can we fetch the "key" (i.e. JIRA number or JIRA ID) from response (essentially stdout) into a variable.
you can use jq command line tool to extract value from json:
#!/bin/bash
key=$(curl -s -u charlie:charlie -X POST --data '{
"fields": {
"project":{
"key": "TEST"
},
"summary": "REST ye merry gentlemen.",
"description": "Creating of an issue using project keys and issue type names using the REST API",
"issuetype": {
"name": "Bug"
}
}
}' -H "Content-Type: application/json" http://localhost:8080/rest/api/2/issue/|jq -r .key)
echo "key: $key"
jq command is not installed by default on linux systems.
you will need to install it manually:
sudo apt install jq
sudo yum install jq

SoftLayer_Virtual_Guest_Block_Device_Template_Group::editObject update datacenters

The createFromExternalSource method does not allow you to specify a list of datacenters to enable the image to be used. I can get the image imported, but then when I then go to edit the image template to augment the datacenters, I always get:
{"error":"Object does not exist to execute method on.
(SoftLayer_Virtual_Guest_Block_Device_Template_Group::editObject)","code":"SoftLayer_Exception"}
Does anyone have a proper example of using editObject for SoftLayer_Virtual_Guest_Block_Device_Template_Group using JSON not XMLRPC like the slcli does?
Preferred if some has a curl example which updates attributes on SoftLayer_Virtual_Guest_Block_Device_Template_Group object would be awesome.
To update the datacenters of an image template you have to use the method “setAvailableLocations” of the service SoftLayer_Virtual_Guest_Block_Device_Template_Group.
The method editObject just edit an image template group's name and note.
You can use this curl example to update the datacenters of an imagen template:
curl -d #post.json -k https://[username]:[apiKey]#api.softlayer.com/rest/v3/SoftLayer_Virtual_Guest_Block_Device_Template_Group/1722867/setAvailableLocations.json | python -mjson.tool
You have to add in this curl example the json file “#post.json”, where is the image template body.
The json file must contain the following data:
{
"parameters": [
[
{
"id": 265592
},
{
"id": 814994
},
{
"id": 138124
},
{
"id": 154820
},
{
"id": 449600
}
]
]
}
To get the datacenters ids you can use this curl command example:
curl -k "https://[username]:[apiKey]#api.softlayer.com/rest/v3/SoftLayer_Location/getDatacenters.json" | python -mjson.tool

How to filter unique values with jq?

I'm using the gcloud describe command to get metadata information about instances.What's the best way to filter the json response with jq to get the name of the instance - if it contains "kafka" as a key.
.name + " " + .metadata.items[]?.key | select(contains("kafka"))'
Basically if items contains kafka print name.This is just a small excerpt from the json file.
"metadata": {
"fingerprint": "xxxxx=",
"items": [
{
"key": "kafka",
"value": "xxx="
},
{
"key": "some_key",
"value": "vars"
}
],
"kind": "compute#metadata"
},
"name": "instance-name",
"networkInterfaces": [
{
"accessConfigs": [
{
"kind": "compute#accessConfig",
"name": "External NAT",
"natIP": "ip",
"type": "ONE_TO_ONE_NAT"
}
],
"kind": "compute#networkInterface",
"name": "",
"network": xxxxx
}
],
I'm sure this is possible with jq, but in general working with gcloud lists is going to be easier using the built-in formatting and filtering:
$ gcloud compute instances list \
--filter 'metadata.items.key:kafka' \
--format 'value(name)'
--filter tells you which items to pick; in this case, it grabs the instance metadata, looks at the items, and checks the keys for those containing kafka (use = instead to look for keys that are exactly kafka).
--format tells you to grab just one value() (as opposed to a table, JSON, YAML) from each matching item; that item will be the name of the instance.
You can learn more by running gcloud topic filters, gcloud topic formats, and gcloud topic projections.
Here is a simple jq solution using if and any:
if .metadata.items | any(.key == "kafka") then . else empty end
| .name

Bash Loop Through URLs Issue

I'm using cURL request to google safe browsing API to check for any of my requested site is malicious or not. When i'm issuing the following command
$curl -H "Content-Type: application/json" -X POST -d ' { "client": { "clientId": "Test", "clientVersion": "1.0.0" }, "threatInfo": { "threatTypes": ["MALWARE", "SOCIAL_ENGINEERING","THREAT_TYPE_UNSPECIFIED","UNWANTED_SOFTWARE","POTENTIALLY_HARMFUL_APPLICATION"], "platformTypes": ["WINDOWS","PLATFORM_TYPE_UNSPECIFIED","ANY_PLATFORM"], "threatEntryTypes": ["URL"], "threatEntries": [ {"url":"hxxp://bookmyroom.pk/assets/timepicker/f.exe"} ] } }' https://safebrowsing.googleapis.com/v4/threatMatches:find?key=AIzaSyD1IMgjaHEza6e9m_jwtjBgPmJX0IMKKIs
Caution: I have "xx"ed instead "tt" for the http string. This site might be a potential malicious site.So do not open it using browser.
I'm getting JSON response as
{
"matches": [
{
"threatType": "UNWANTED_SOFTWARE",
"platformType": "ANY_PLATFORM",
"threat": {
"url": "hxxp://bookmyroom.pk/assets/timepicker/f.exe"
},
"cacheDuration": "300.000s",
"threatEntryType": "URL"
}
]
}
Caution: I have "xx"ed instead "tt" for the http string. This site might be a potential malicious site.So do not open it using browser.
When i'm doing the same in loop
#check.sh
for i in $(cat malsite.txt); do
content="$(curl -H "Content-Type: application/json" -X POST -d ' { "client": { "clientId": "Test", "clientVersion": "1.0.0" }, "threatInfo": { "threatTypes": ["MALWARE", "SOCIAL_ENGINEERING","THREAT_TYPE_UNSPECIFIED","UNWANTED_SOFTWARE","POTENTIALLY_HARMFUL_APPLICATION"], "platformTypes": ["WINDOWS","PLATFORM_TYPE_UNSPECIFIED","ANY_PLATFORM"], "threatEntryTypes": ["URL"], "threatEntries": [ {"url":"$i"} ] } }' https://safebrowsing.googleapis.com/v4/threatMatches:find?key=AIzaSyD1IMgjaHEza6e9m_jwtjBgPmJX0IMKKIs)"
echo "$i"
echo "$content"
done
#malsite.txt
"hxxp://bookmyroom.pk/assets/timepicker/f.exe"
Caution: I have "xx"ed instead "tt"ing for the http string.This site might be a potential malicious site.So do not open it using browser.
I'm not getting any results. It just returns empty result.
#Result of /.check.sh :
{}
Not sure , Where i'm making mistakes. Any thoughts ?

Resources