I am trying to connection to Neptune DB and getting vertices details using CURL command. I have shell script for it. But somehow variable data is not going through it gremlin query. I have one Orgid.txt file where tenantid is present and my shell script reading the file and passing it to "name" variable
#!/bin/bash
i=1
rm VerticesCount1
while IFS= read -r line
do
name="$line"
#echo "Orgid name is "$i": $name"
curl -X POST https://<Neptune_endpoint>:<port>/gremlin -d '{"gremlin":"g.V().has(\"system.tenantId\",\"$name\").count()"}' >> VerticesCount1
#printf "\n"
echo >> VerticesCount1
((i=i+1))
done < Orgid.txt
As with your other question I tested with a simple data file and it works fine. However, note how I changed the type of quotes used by curl.
i=1
while IFS= read -r line
do
name="$line"
curl -X POST https://mydbcluster.cluster-xxxxxxxxxxxx.us-east-1.neptune.amazonaws.com:8182/gremlin -d \
"{\"gremlin\":\"g.V().has('code','$name').count()\"}"
((i=i+1))
done < values.txt
which produces
{"requestId":"4e3e80ed-efcb-40a7-b92b-366c6f391d4e","status":{"message":"","code":200,"attributes":{"#type":"g:Map","#value":[]}},"result":{"data":{"#type":"g:List","#value":[{"#type":"g:Int64","#value":1}]},"meta":{"#type":"g:Map","#value":[]}}}{"requestId":"6a269b5b-32f6-49d2-a31d-c51dd52eba29","status":{"message":"","code":200,"attributes":{"#type":"g:Map","#value":[]}},"result":{"data":{"#type":"g:List","#value":[{"#type":"g:Int64","#value":1}]},"meta":{"#type":"g:Map","#value":[]}}}
it is working fine with this code.
while IFS= read -r line
do
name="$line"
#echo "Orgid name is "$i": $name"
curl -X POST https://<Neptune_endpoint>:<port>/gremlin -d '{"gremlin":"g.V().has(\"system.tenantId\",\"'$name'\").count()"}' >> VerticesCount1
echo >> VerticesCount1
done < Orgid.txt
Related
I have a hash file containing several md5 hashes.
I want to create a bash script to curl virustotal to check if the hashes are known.
#!/bin/bash
for line in "hash.txt";
do
echo $line; curl -s -X GET --url 'https://www.virustotal.com/vtapi/v2/file/report?apikey=a54237df7c5c38d58d2240xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxcc0a0d7&resource='$line'';
done
but not working.
Could you help me please?
Better use a while loop. Your for loop would only run once, because bash interpret it as a value, not a file. Try this:
while read -r line; do
echo "$line"
curl -s -X GET --url "https://www.virustotal.com/vtapi/v2/file/report?apikey=a54237df7c5c38d58d2240xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxcc0a0d7&resource=$line"
done <"/path/to/hash.txt"
I'm building a .sh script to run curls based on the items (per line) placed in on a file fileWithItems.txt.
This is the script I built:
declare -a array
#assuming fileWithItems.txt contains one element per line to be used in the url is in the same folder as this .sh
mapfile -t array < fileWithItems.txt
host="localhost"
port="PORT"
i=0
while [ ${i} -lt ${#array[#]} ] ; do
curl -X PUT "$host:$port/path1/${array[$i]}/refresh" > log.txt
((i++))
done
Seem that the curl is not being built properly. How could it be optimized?
To elaborate further from my comments, you can do it like this:
host="localhost"
port="PORT"
while IFS= read -r line; do
curl -X PUT "$host:$port/path1/$line/refresh"
done < fileWithItems.txt > log.txt
Please note placement of > log.txt after done so that you don't overwrite same file every time.
Depending on certain conditions I want to use JWT else I want to provide path to certs. Thus in my shell script this is the code:
if /* some condition */
authorization='-H "'Authorization': 'Bearer' ${JWT}"'
else
authorization="--cert "${ADMIN_CERT_PATH}" --key "${ADMIN_KEY_PATH}""
Now the curl request should be:
curl -H "Authorization: Bearer 348129" for if condition
curl --cert /Users/.../admin_cert --key /Users/../admin_key .. for else path
In order to get that output I need to use the following format in my shell script for if condition
response_code="$(curl -s -o /dev/null -w "%{http_code}" "$authorization" "$status_url")"
and following format for else code:
response_code="$(curl -s -o /dev/null -w "%{http_code}" $authorization "$status_url")"
Note:
I need $authorization variable quoted in first case and unquoted in the else case.
I do not want to write 2 different curl commands instead reuse the authorization variable.
Thus, i need to modify the way I have declared my authorization variable such that I can write any one of the curl commands only once which works for both if and else cases.
curl supports a way to pass command line parameters in a file that I have used before when I have complex parameters. The idea is to place the complex command-line parameters into a simple text file and instruct curl to read parameters from it using --config parameter.
In this case the shell script would look something like the following.
#!/bin/sh
## "safely" create a temporary configuration file
curlctl=$(mktemp -q -t $(basename "$0"))
if test $? -ne 0
then
echo "$0: failed to create temporary file, exiting."
exit 75 # EX_TEMPFAIL
fi
trap 'rm "$curlctl"' 0
## write parameters used in all cases
cat>>"$curlctl" <<EOF
output = /dev/null
silent
write-out = %{http_code}
EOF
## append conditional parameters
if test "$some" = 'condition'
then
printf 'header = "Authorization: Bearer %s"\n' "$JWT" >> "$curlctl"
else
echo "cert = $ADMIN_CERT_PATH" >> "$curlctl"
echo "key = $ADMIN_KEY_PATH" >> "$curlctl"
fi
# uncomment to see what the config file looks like
# cat "$curlctl" | sed 's/^/curl config: /'
response_code=$(curl --config "$curlctl" http://httpbin.org/get)
echo "response code: $response_code"
The first few lines set up a temporary file that is deleted when the shell script exits. If you are already using trap then your cleanup will probably be more complex.
When you are using a shell that supports arrays, you can avoid the need for a temporary configuration file.
curl_opts=(-s -o /dev/null -w "%{http_code}")
if /* some condition */
curl_opts+=(-H "Authorization: Bearer $JWT")
else
curl_opts+=(--cert "$ADMIN_CERT_PATH" --key "$ADMIN_KEY_PATH")
fi
...
response_code="$(curl "${curl_opts[#]}" "$status_url")"
#!/bin/bash
# See if registry is set to expire updates
filename=hostnames
> test.log
PARAMETER=Updates
FILE=/etc/.properties
CODE=sudo if [ ! -f $FILE] && grep $PARAMETER $FILE; then echo "File found, parameter not found."
#CODE=grep $PARAMETER $FILE || sudo tee -a /etc/.properties <<< $PARAMETER
while read -r -a line
do
hostname=${line//\"}
echo $hostname":" >> test.log
#ssh -n -t -t $hostname "$CODE" >> test.log
echo $CODE;
done < "$filename"
exit
I want to set "Updates 30" in /etc/.properties on about 50 servers if:
The file exists (not all servers have the software installed)
The parameter "Updates" is not already set in the file (e.g. in case of multiple runs)
I am a little puzzled so far how, because I am not sure if this can be done in 1 line of bash code. The rest of the script works fine.
Ok, here's what i think would be a solution for you. Like explained in this article http://www.unix.com/shell-programming-scripting/181221-bash-script-execute-command-remote-servers-using-ssh.html
invoke the script which contains the commands that you want to be executed at the remote server
Code script 1:
while read -r -a line
do
ssh ${line} "bash -s" < script2
done < "$filename"
To replace a line in a text file, you can use sed (http://www.cyberciti.biz/faq/unix-linux-replace-string-words-in-many-files/)
Code script 2:
PARAMETER=Updates
FILE=/etc/.properties
NEWPARAMETER=Updates ###(What you want to write there)
if [ ! -f $FILE] && grep $PARAMETER $FILE; then exit
sed -i 's/$PARAMETER/$NEWPARAMETER/g' $FILE
So, I'm not certain this covers all your use case, I hope this helps you out if there is anything feel free to ask!
The following is iptable save file, which I modified by setting some variables like you see below.
-A OUTPUT -o $EXTIF -s $UNIVERSE -d $INTNET -j REJECT
I also have a bash script which is defining this variables and should call iptables-restore with the save file above.
#!/bin/sh
EXTIF="eth0"
INTIF="eth1"
INTIP="192.168.0.1/32"
EXTIP=$(/sbin/ip addr show dev "$EXTIF" | perl -lne 'if(/inet (\S+)/){print$1;last}');
UNIVERSE="0.0.0.0/0"
INTNET="192.168.0.1/24"
Now I need to use
/sbin/iptables-restore <the content of iptables save file>
in bash script and somehow insert the text file on top to this script, so the variables will be initialized. Is there any way to do that?
UPDATE: even tried this
/sbin/iptables-restore -v <<-EOF;
$(</etc/test.txt)
EOF
Something like this:
while read line; do eval "echo ${line}"; done < iptables.save.file | /sbin/iptables-restore -v
or more nicely formatted:
while read line
do eval "echo ${line}"
done < iptables.save.file | /sbin/iptables-restore -v
The eval of a string forces the variable expansion stuff.
Use . (dot) char to include one shell script to another:
#!/bin/sh
. /path/to/another/script
In your shell script:
. /path/to/variable-definitions
/sbin/iptables-restore < $(eval echo "$(</path/to/template-file)")
or possibly
/sbin/iptables-restore < <(eval echo "$(</path/to/template-file)")