Hi I need to extract the 1st key of the output json I've tried with different regex but didn't give expected results could you please let me to solve this.
LANGUAGES=`curl \
--request GET \
--header 'authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' \
--header 'content-type: application/string' \
--url 'https://api.github.com/repos/${{ github.repository }}/languages' \
`
echo "$LANGUAGES" | regex
outputs and keys will be dynamic
{
"HCL": 56543,
"Shell": 22986,
"Dockerfile": 307
}
Expected output : HCL
{
"Java": 56543,
"C++": 22986,
"C#": 307
}
Expected output : Java
{
"Python": 56543,
"SHELL": 22986,
"C": 307
}
Expected output : Python
echo "$LANGUAGES" | jq -r 'keys_unsorted | first'
Related
I need to send an HTML file as the body of an eamil to several customers. Our company will be using SendGrid for this, and I need to be able to send the email via API Curl Call.
The way that I'm doing it so far works for simple html or plain text:
curl -s --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SECRET_API_KEY" \
--header 'Content-Type: application/json' \
--data '{"personalizations":[{"to":[{"email":"my1#email.com"},{"email":"my2#email.com"}]}],"from":{"email":"info#somewhere.com"},"subject":"Testing sending emails via SendgridAPI","content":[{"type":"text\/html","value":"Test API Email From ME"}]}'
Now this works just fine. The problem is when I want to replace 'Test API Email From ME' with the contents of a rather large, complex HTML file. This has all the usual cli nightmares such as a mix of ' and " and new lines everywhere. I need to sanatize the HTML in order to accomplish three things:
The final result needs to be a valid command line string
The --data switch argument needs to remain a valid JSON enconded string
The HTML should not break.
What I do is I create the actual string command and the execute it using a scripting language. So I can perform any operation that I want on the html before inserting it in the value field of the content field. So my question is: what are the string operations that I should perform on the html so that I can send the email using this methodology?
Using jq and bash
I'll do it with static data, you may improve upon it
Define a JSON template for the API:
IFS='' read -r -d '' json_template <<'EOF'
{
"personalizations": [
{
"to": [
{ "email": "my1#email.com" },
{ "email": "my2#email.com" }
]
}
],
"from": { "email": "info#somewhere.com" },
"subject": "Testing sending emails via SendgridAPI",
"content": [
{
"type": "text/html",
"value": "Test API Email From ME"
}
]
}
EOF
Define the HTML content:
IFS='' read -r -d '' html_email <<'EOF'
<!doctype html>
<html>
<head>
title>Simple Email</title>
</head>
<body>
Test API Email From ME
</body
</html>
EOF
Replace the email content in the JSON with the HTML
json_data=$(
jq -c -n \
--arg html "$html_email" \
--argjson template "$json_template" \
'$template | .content[0].value = $html'
)
Send the query
curl -s --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SECRET_API_KEY" \
--header 'Content-Type: application/json' \
--data "$json_data"
Here is how you can compose a proper JSON data payload with jq so it can be sent to the API.
jq will ensure every values, recipients, from, subject and the html body will be respectively encoded to proper JSON data objects, arrays and strings before it is submitted as --data #- to curl:
I added comments everywhere, so it is very clear what is done at every step:
#!/usr/bin/env bash
recipients=(
'my1#email.com'
'my2#email.com'
)
from='info#somewhere.com'
subject='Testing sending emails via SendgridAPI'
# Streams null-delimited recipients array entries
printf '%s\0' "${recipients[#]}" |
# jq slurps the null-delimited recipients,
# read the raw html content into the jq $contentHTML variable
# and integrate it all as a proper JSON
jq --slurp --raw-input --rawfile contentHTML example.html \
--arg from "$from" \
--arg subject "$subject" \
'
# Fills the jq $recipient JSON array variable
# by splitting the null-delmited entries
# from the incoming stream
split( "\u0000") as $recipients |
{
"personalizations": [
{
# Uses the $recipients array that has been
# slurped from the input stream
"to": $recipients
}
],
"from": {
# Use the $from that has been passed as --arg
"email": $from
},
# Use the $subject that has been passed as --arg
"subject": $subject,
"content": [
{
"type": "text/html",
"value": $contentHTML
}
]
}
' |
# Get the resultant JSON piped into curl
# that will read the data from the standard input
# using --data #-
# rather than passing it as an argument, because
# the payload could exceed the maximum length of arguments
curl -s --request POST \
--url https://api.sendgrid.com/v3/mail/send \
--header "Authorization: Bearer SECRET_API_KEY" \
--header 'Content-Type: application/json' \
--data #-
I have got around 1000 contacts to import to Mailchimp. This is my company's old database, which we have exported from the CSM system, and we want every contact to confirm their subscription if they want to be on our subscription list.
When I try to import it through Mailchimp, I can't give the contact status pending.
So, I have managed how to do it with single contact through bash, but I will want to import the whole contact list.
I am not familiar with this scripting language that much, so can anybody advise me, is there a way to import the data from the CSV file and how can I do it?
Or maybe there is some other way to do it?
This is the code that is working for a single contact:
#!/bin/bash
set -euo pipefail
list_id="Add_LIST_ID"
user_email="Add_E_MAIL"
user_fname="Add_F_NAME"
user_lname="Add_L_NAME"
curl -sS --request POST \
--url "https://$API_SERVER.api.mailchimp.com/3.0/lists/$list_id/members" \
--user "key:$API_KEY" \
--header 'content-type: application/json' \
--data #- \
<<EOF | jq '.id'
{
"email_address": "$user_email",
"status": "pending",
"merge_fields": {
"FNAME": "$user_fname",
"LNAME": "$user_lname"
}
}
EOF
EDIT1
Okay, I have managed to load the data from csv file. The code is below.
while IFS=, read -r col1
do
{
#!/bin/bash
set -euo pipefail
list_id="LIST_ID"
echo "$col1"
curl -sS --request POST \
--url "https://$API_SERVER.api.mailchimp.com/3.0/lists/$list_id/members" \
--user "key:$API_KEY" \
--header 'content-type: application/json' \
--data #- \
<<EOF | jq '.id'
{
"email_address": "$(echo $col1)",
"status": "pending",
"merge_fields": {
"FNAME": "",
"LNAME": ""
}
}
EOF
}
done < mails.csv
I have put echo line after list_id to see if the data is imported correctly.
The code is working (no errors in the buildup), but I have managed to add a contact to the list only once (subscriber hash is the response). In other tries, I have got a "null" value in response. Does anybody know why?
I have a bash script that sends a curl request and displays the response.
#!/bin/bash
token=$(curl -k -X GET \
'https://v.mytesting.io/oauth/token?grant_type=password&username=user1&password=123' \
-H 'Authorization: Basic 12345678' \
-H 'Host: v.mytesting.io.io')
v=$( jq -r ".access_token" <<<"$token" )
ts=$(curl -k -X POST \
https://timeseries.mytimeseries.io/v5/time_series/query \
-H 'Authorization: Bearer '"$v" \
-H 'Content-Type: application/json' \
-H 'Host: timeseries.mytimeseries.io' \
-H 'tenant: 123-123-123' \
-d '{"operation" : "raw","responseFormat" : "kairosDB","startTime": "1d-ago","stopTime": "now","tagList" : [ {"tagId" : "V.S.23164117.AVG.10M"}]}')
p=$(jq '.queries[].sample_size, .queries[].results[].name' <<<"$ts")
echo "$p"
My current output is just a value and the name of the tagId.
My query only allows for 1 tagId ( you can see above )
I want to be able to set a list of tagId's.
Then when I run this script it should loop through the list of tagId's and execute the curl request replacing the V.S.23164117.AVG.10M with each value
in the list.
Then output the entire list of results into a file.
list would be like so - (I would love to be able to enter this list into a seperate file and the bash script calls that file. Sometimes this list can be a few hundred lines.
V.S.23164117.AVG.10M
V.S.23164118.AVG.10M
V.S.23164119.AVG.10M
V.S.23164115.AVG.10M
V.S.23164114.AVG.10M
output would like look so.
value tagId
value tagId
value tagId
100 V.S.23164117.AVG.10M
etc..
thank you for any help
You can loop over list of tags using a small script. I'm not 100% clean of the output format. You can change the 'echo' to match the required format.
Note minor change to quotes to allow variable expansion in the body.
The tags will be stored in a file, for examples, tags.txt
V.S.23164117.AVG.10M
V.S.23164118.AVG.10M
V.S.23164119.AVG.10M
And the script will be use the file
#! /bin/bash
# Use user defined list of tags
tags=tags.txt
token=$(curl -k -X GET \
'https://v.mytesting.io/oauth/token?grant_type=password&username=user1&password=123' \
-H 'Authorization: Basic 12345678' \
-H 'Host: v.mytesting.io.io')
v=$( jq -r ".access_token" <<<"$token" )
for tag in $(<$tags) ; do
ts=$(curl -k -X POST \
https://timeseries.mytimeseries.io/v5/time_series/query \
-H 'Authorization: Bearer '"$v" \
-H 'Content-Type: application/json' \
-H 'Host: timeseries.mytimeseries.io' \
-H 'tenant: 123-123-123' \
-d '{"operation" : "raw","responseFormat" : "kairosDB","startTime": "1d-ago","stopTime": "now","tagList" : [ {"tagId" : "'"$tag"'"}]}')
p=$(jq '.queries[].sample_size, .queries[].results[].name' <<<"$ts")
echo "$tag $p"
done
I have a curl request in below format
curl -v -H "Content-Type:application/json" -H "x-user-id:xxx" -H "x-api-key:yyy" --data '{"logs":"'"${TEST_OUTPUT}"'","pass":"true | false"}' https://razeedash.one.qqq.cloud.com/api/v1/clusters/zzz/api/test_results
This works fine while I do from my MAC terminal. But the same command throws
13:49:26 {
13:49:26 "status": "error",
13:49:26 "message": "Invalid credentials"
13:49:26 }
I saw this post but not sure how else would I send a json body without curly braces. I know that we can save it as a file.json and use the file as body.But for some reasons that cannot be implemented in my scenario
In general, you should avoid trying to build JSON using string interpolation. Use a tool like jq to handle any necessary quoting.
jq -n --argson o "$TEST_OUTPUT" '{logs: $o, pass: "true | false"}' |
curl -v -H "Content-Type:application/json" \
-H "x-user-id:xxx" \
-H "x-api-key:yyy" \
--data #- \
https://razeedash.one.qqq.cloud.com/api/v1/clusters/zzz/api/test_results
However, if you can manage to correctly generate your JSON as you are now, you can just replace the jq command with echo:
echo '{"logs": ...' | curl ...
The #- argument to --data says to read from standard input.
hello I have this script in bash:
first part : authentication against server - works fine.
second part: adds user - and this works fine when I define user directly in curl code - but when I want to add user from parameters.. then I got a message error:
"{"code":400,"reason":"Bad Request","message":"The request could not
be processed because the provided content is not valid
JSON","detail":"Unexpected character ('$' (code 36)): expected a valid
value (number, String, array, object, 'true', 'false' or 'null')\n at
[Source: org.apache.catalina.connector.CoyoteInputStream#3a248e6a;
line: 2, column: 18]"}"
Any ideas how can I pass values to this script ? :)
Regards
#!/bin/bash
adm_user="admin"
adm_pass="secret"
user_name=""
user_pass=""
auth_url="http://url/OpenAM-11.0.0/json/authenticate"
add_user_url="http://url/OpenAM-11.0.0/json/users/?_action=create"
session_id=$(curl \
--request POST --header "X-OpenAM-Username: $adm_user " \
--header "X-OpenAM-Password: $adm_pass" \
--header "Content-Type: application/json" \
--data "{}" $auth_url | cut -d"\"" -f4 )
sleep 1
curl \
--request POST \
--header "iplanetDirectoryPro: $session_id" \
--header "Content-Type: application/json" \
--data \
'{
"username": "$1",
"userpassword": "secret12",
"mail": "bjensen#example.com"
}' \
$add_user_url
OK, mates I got the answer:
'{
"username": "'"$1"'",
"userpassword": "'"$2"'" ,
"mail": "bjensen#example.com"
}' \