I have a bash script that user enter name of a database and then my script needs to loop through this database config file. my JSON looks like this:
{
"_id":1,
"db_name":"postgres",
"username":"postgres_user",
"password":"postgres_pass",
"daily" : {
"days_to_backup" : [1,2],
"delete_after" : 14,
"compressed" : true,
"path" : "localhost"
},
"monthly" : {
"days_to_backup" : [2,5,30],
"delete_after" : 7,
"compressed" : false,
"path" :" localhost"
},
"yearly" : {
"days_to_backup" : [100],
"delete_after" : 14,
"compressed" : true,
"path" : "localhost"
}
}
{
"_id":2,
"db_name":"test",
"username":"test_user",
"password":"test_pass",
"daily" : {
"days_to_backup" : [1,7],
"delete_after" : 14,
"compressed" : true,
"path" : "localhost"
},
"monthly" : {
"days_to_backup" : [2,5,25],
"delete_after" : 7,
"compressed" : false,
"path" : "localhost"
},
"yearly" : {
"days_to_backup" : [50],
"delete_after" : 14,
"compressed" : true,
"path" : "localhost"
}
}
Now once user run the script with particular database name, i want to loop through the "days_to_backup" numbers and for each day look it it's equal for today date. The problem is that i don't know how to fetch the daily/monthly/yealy.days_to_backup and then loop through the days.
I tried with jq :
#! /bin/sh
#monday is first day of the week
export Today=$(date +%d)
while getopts d: flag
do
case "${flag}" in
d) database=${OPTARG};;
esac
done
if jq .db_name file.json | grep $database
then
jq 'select(.db_name=="'$database'")' file.json
##this one returns the whole document of the specified database
##loop here for daily/monthly_yearly.days_to_backup???
else
echo "database not found"
fi
Edit: my input is the name of the database ( " postgres " ) and then i will iterate through the daily/monthly/yearly array and if the number is like current date ( today is 8 of the month) so the output will be - echo "backup today".
Edit2: sorry my bad. for daily it means that day of the week so (1,7) are sunday and saturday and if current date is sunday or saturday i want and echo "backup today". for monthly it means the day of the month, so if i have monthly (01,08) it will echo "backup today" because today is the 8th of the month. and for yearly i want days since the first day of the year.
Ok so I succeeded to loop through the array but the problem now is that it run over the array of the 2 big objects (id 1 + 2)
for i in $(jq -r ".daily.days_to_backup | .[]" file.json)
do
echo $i "im number"
done
how do I select that it will run only over _id:1 / db_name: postgres?
An efficient approach would be based on using any, e.g. along the lines of the following:
jq --arg db $database --argjson Today $Today '
select(.db_name==$db and
any(.daily[], .monthly[], .yearly[]; . == $Today) )
' file.json
Note that both parameters ($database and $Today) should be passed in as command-line arguments rather than provided using shell-based string interpolation. Using --argjson will ensure that any leading 0s in the shell variable $Today do not cause problems.
Note also that the above query will return a stream of 0 or more values. You might want to guarantee the names are distinct, e.g. by using the jq built-in filter unique (defined for arrays), or the stream-oriented filter uniques defined in the jq Cookbook.
Related
Objective: Trying to check if resource exist on azure with bash script
Code that I use :
status=$(az group list --query "[?name.contains(#,'test')]")
if [[ "$status" == null ]];
then
echo "not exist"
else
echo "exist"
fi
I have this resource in azure i.e it should return as "exist" however its says not exist
If I change to a non existant resource group name then time also its gives not exist.
do you see any syntax issue here ?
Instead of script if I execute at commmand line to check , below are results
user#ablab:~$ status=$(az group list --query "[?name.contains(#,'abcd')]")
user#ablab:~$ echo $status
[]
user#ablab:~$ status=$(az group list --query "[?name.contains(#,'test')]")
user#ablab:~$ echo $status
[ { "id": "/subscriptions/xxxx-xxxx-xxx--xxxxx3/resourceGroups/test1", "location": "westeurope", "managedBy": null, "name": "test1", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ]
Now I wanna use if condition, so that if its exist it should process set of flow else set of code..
Please let me know what wrong with my if statement.
The syntax is fine. However, I don't see from your example, that az would write the string null to stdout. In the first case, it prints, according to what you posted, the string []. To catch this case, you would have to test
if [[ $status == '[]' ]]
then
...
The quotes around the string tell bash to not interpret it as a glob pattern.
I'm confused. Why is the last item being echoed first?
echo -e "\n\nPROCESS: cik : $cik companyName : $companyName form : $form date : $dateStr textURL : $textURL ID : $ID"
OUTPUT:
ID : 6f89f534-2c81-4338-89ac-f691c842b8f1LION FINANCIAL CORP form : 40-17F2 date : 2017-01-05 textURL : edgar/data/1000209/0001193125-17-002593.txt
It does this for one other attempt to create a curl command.
echo curl -XPUT "$ES_HOST/edgar/filing/$ID" -H 'Content-Type: application/json' -d'{"cik":"'$cik'","companyName":"'$companyName'","form":"'$form'","date":"'$date'","textURL":"'$textURL'","data":"'$data'"}'
which gets printed as
-o /tmp/foo.txt.sec.gov/Archives/edgar/data/1000209/0001193125-17-002593.txt
WTH is going on?
My guess is that $textURL has a carriage return at the end of it, so the cursor is being moved back to the start of the line before the id is printed
I have a GroupMe bot that can send messages to the chat it is assigned to in this format:
curl -d '{"text" : "text", "bot_id" : "(REDACTED)"}' https://api.groupme.com/v3/bots/post
So, instead of typing up this monstrosity every time I wanted to send a message, I decided to create an alias for it.
Here is what I attempted to use in my .bash_profile for this:
alias groupmessage=groupmessagefunction
groupmessagefunction() { curl -d '{"text" : $1, "bot_id" : "(REDACTED)"}' https://api.groupme.com/v3/bots/post; }
Could anyone inform me what the correct formatting would be to get this to work? Thanks!
Update 1:
I now have my code as follows:
v="bot_id_string"
alias groupmessage=groupmessagefunction
groupmessagefunction() { curl -d '{"text" : '"$1"', "bot_id" : '"$v"'}' https://api.groupme.com/v3/bots/post; }
I would like to point out that what I am trying to accomplish is type in something like:
groupmessage "hello"
or
groupmessage hello
Then, it will send out the following command:
curl -d '{"text" : "hello", "bot_id" : "bot_id_string"}' https://api.groupme.com/v3/bots/post
I hope this helps clarify this question if any misunderstanding occurred.
Now, I hope this will be the solution for you:
v=bot_id
curl -d '{"text" : "Test", "'$v'" : "(REDACTED)"}' api.groupme.com/v3/bots/post
You have to use ' ' around $v --> '$v' because this will allow bash to evaluate $v inside curl -d ' ... ' .
Answer for your Update 1
As I see it would be the better way to use eval command:
v="bot_id_string"
alias groupmessage=groupmessagefunction
groupmessagefunction() {
a="curl -d '{\"text\" : \"$1\", \"bot_id\" : \"$v\"}' https://api.groupme.com/v3/bots/post"
echo "$a" # this echo just for checking
eval "$a"; }
groupmessage "Hello is it working now ?"
My document structure looks like this:
{"name":"John", "age":32, ...other fields}
All other fields need not be initialized, only name and age. I want to make a script that takes in name and number
./client.sh John 32
and in the script, it will do something like
db.client.insert({"name":$1,"age":$2});
how to achieve this?
Here is a simple example script that works
#!/bin/bash
if [ $# -lt 2 ]
then
echo "USAGE: $0 name age"
exit 1
fi
mongo <<EOF
use test
db.client.insert({"name":"$1", "age":$2})
db.client.find()
EOF
It assumes that mongo is installed and in your path and that you have the client collection in the test database.
A sample run below
hduser#localhost:~/temp$ ./mongo_ins.sh "overflow" 20
MongoDB shell version: 2.6.10
connecting to: test
switched to db test
WriteResult({ "nInserted" : 1 })
{ "_id" : ObjectId("56b35d134c24bf7c1190cfb3"), "name" : "Stack", "age" : 1 }
{ "_id" : ObjectId("56b35dabe23223802ea3fa61"), "name" : { }, "age" : 10 }
{ "_id" : ObjectId("56b35ebbd69a0abeeb817fe3"), "name" : "overflow", "age" : 20 }
bye
Hope this helps
I am trying to pass an associative array from one function to another, and am losing named index keys (e.g., filepath, search in example below) though the array passed in can access its elements correctly using indexes 0, 1. I must be doing something slightly wrong with bash syntax, but can't quite figure out where. Any help appreciated.
Using GNU bash, version 4.3.8 from Ubuntu 14.04
Just below is bash code example, and at bottom is output
#! /bin/bash
function test_function {
func_data=("${#}")
# without brackets above cannot access func_data[1]
# local ${!func_data}
# the above local statement does not seem to help either way
echo ""
for K in "${!func_data[#]}"; do echo $K; done
echo ""
echo "func_data : ${func_data}"
echo "func_data[filepath] : ${func_data[filepath]}"
echo "func_data[search] : ${func_data[search]}"
# all three echos above output first element of array,
# which is 'style "default" {' during first loop
# Can access array elements 0, 1 but no longer via filepath, search
echo "func_data[0] : ${func_data[0]}"
echo "func_data[1] : ${func_data[1]}"
echo "!func_data[#] : ${!func_data[#]}"
# echo above outputs '0 1' so indexes now are now zero based?
echo "func_data[#] : ${func_data[#]}"
# echo above outputs all array elements 'style "default" { ~/.gtkrc-2.0'
}
# In BASH, local variable scope is the current function and every
# child function called from it, so provide a function main to
# make it possible to utilize variable scope to fix issues
function main {
echo ""
declare -A gtkrc2=()
gtkrc2[filepath]="~/.gtkrc-2.0"
gtkrc2[search]="style \"default\" {"
echo "gtkrc2 filepath : ${gtkrc2[filepath]}"
echo "gtkrc2 search : ${gtkrc2[search]}"
test_function "${gtkrc2[#]}"
echo ""
declare -A gtkcss=()
gtkcss[filepath]="~/.config/gtk-3.0/gtk.css"
gtkcss[search]=".scrollbar {"
echo "gtkcss filepath : ${gtkcss[filepath]}"
echo "gtkcss search : ${gtkcss[search]}"
test_function "${gtkcss[#]}"
}
main
---------- OUTPUT ----------
gtkrc2 filepath : ~/.gtkrc-2.0
gtkrc2 search : style "default" {
func_data : style "default" {
func_data[filepath] : style "default" {
func_data[search] : style "default" {
func_data[0] : style "default" {
func_data[1] : ~/.gtkrc-2.0
!func_data[#] : 0 1
func_data[#] : style "default" { ~/.gtkrc-2.0
gtkcss filepath : ~/.config/gtk-3.0/gtk.css
gtkcss search : .scrollbar {
func_data : .scrollbar {
func_data[filepath] : .scrollbar {
func_data[search] : .scrollbar {
func_data[0] : .scrollbar {
func_data[1] : ~/.config/gtk-3.0/gtk.css
!func_data[#] : 0 1
func_data[#] : .scrollbar { ~/.config/gtk-3.0/gtk.css
This may or may not be the "correct" way to do this but this is the best I can figure out. Any suggestions from others are welcome:
function test_function {
arrname=$1
idxlist="$2"
echo ""
echo "Array passed=$arrname"
for idx in $idxlist; do
elemname=$arrname[$idx]
echo "idx=$idx, elem=${!elemname}"
done
}
# In BASH, local variable scope is the current function and every
# child function called from it, so provide a function main to
# make it possible to utilize variable scope to fix issues
function main {
echo ""
declare -A gtkrc2=()
gtkrc2[filepath]="~/.gtkrc-2.0"
gtkrc2[search]="style \"default\" {"
echo "gtkrc2 filepath : ${gtkrc2[filepath]}"
echo "gtkrc2 search : ${gtkrc2[search]}"
test_function gtkrc2 "${!gtkrc2[*]}"
echo ""
declare -A gtkcss=()
gtkcss[filepath]="~/.config/gtk-3.0/gtk.css"
gtkcss[search]=".scrollbar {"
echo "gtkcss filepath : ${gtkcss[filepath]}"
echo "gtkcss search : ${gtkcss[search]}"
test_function gtkcss "${!gtkcss[*]}"
}
main
In particular:
To pass each associative array to the function, we pass both the name of the array, and its list of indices
Inside the function, the array name and index list are taken from the positional parameters
We may then loop over the list of indices and obtain the corresponding value of each element. This is done by first generating the name of the element, and then using the ! indirection modifier to get the actual value.
This technique of indirection of arrays is described in this question, but only addresses indexed arrays, and not associative arrays; passing the index list is one way I can think of to get this to work for associative arrays.