I am trying to run a command using the AWS CLI and then set the output of that command to a variable, which was easy enough. I parsed the initial command with jq and the output was an array of the GroupNames which is exactly what I wanted. Now I am trying to run an if statement that says if $1 is in the list_groups: print the users in that group. My issue is two fold. I do not know if The output from the first command is readable because it is not in a specific format.
#!/bin/bash
list_groups=$(aws iam list-groups --output json | jq -r .'Groups[] | .GroupName' --output json)
echo $list_groups
if ${1} == $list_groups;
then
echo aws iam get-group --group-name $1 --output json
else
echo 'sorry'
fi
So I am trying to take the list_groups variable that I stored the output of that CLI command in and then with user input in $1 see if what I input is in that GroupName and if it is use that $1 to then print out the users inside that group show in the command below.
echo aws iam get-group --group-name $1 --output json
I am sorry as I cannot print my output here for the GroupNames I am getting back from the first command as it is sensitive but, here is an example.
Admin Users assets.dev assets.prod
As you can see there are no separating values and I am not sure if that is contributing to the issues.
If I wanted to run this command here is what I would enter
bash accounts_parse.sh Admin
The error I get is
command not found
Thank you all for your help and I will clarify if my explanation is sub par or more information is needed.
if ${1} == $list_groups;
This isn't how if statements in bash (and other POSIX shells) work.
The word that comes after if is a command to execute, and its exit status is evaluated, and if it's 0 the if-branch is taken.
That explains your "command not found" - it's trying to execute ${1} as a command, and you don't have an "Admin" command.
Use either
if [ "${1}" = "$list_groups" ]
or
if [[ "${1}" = "$list_groups" ]]
the latter is a bashism (with some improvements) and won't work in other otherwise compatible shells. "==" is also available as a bash extension but entirely equivalent to "=", so I'd advise to just use that.
What's important is that the [ is a command (typically a builtin, also accessible as "test"). It's not a special bit of syntax but a thing that is executed with arguments and returns a status. ([[ has some special handling and is more syntax-y, but conceptually it's still "a thing to execute")
You can use read builtin in bash:
read user _ < <(aws iam list-groups --output json | jq -r .'Groups[] | .GroupName' --output json)
if [[ $user == $1 ]]; then
echo aws iam get-group --group-name $1 --output json
else
echo 'sorry'
fi
read user _ will read first word from input into variable user and ignore remaining.
Related
This question already has answers here:
-z option inside if condition in shell script
(3 answers)
Closed 2 years ago.
I am trying to create a Bash script to create an AWS EC2 instance.
Goal: improve my Bash scripting skills
In order to improve my bash scripting skills, I wanted to practice creating if statements in my script.
As shown above, I created the if statement to check if the variable new_ami is empty then to echo "Did not find it" but if it is not empty, it will echo "Found AMI".
Here is my script
ami=$(aws ec2 describe-images --owners self amazon --filters "Name=name, Values=amzn2-*.0-x86_64-gp2" "Name=state, Values=available" --profile XXXXXX --output json | jq '.Images | sort_by(.CreationDate) | last(.[]).ImageId')
new_ami=$(echo "${ami}" | sed 's/"//g')
echo $new_ami
if test -z "$new_ami"
then
echo "Found AMI"
else
echo "Did not find it"
fi
When I run my script, this is the result, I got
ami-0ce1e3f77cd41957e
Did not find it
I have a problem:
The script echoes the variable new_ami which shows that he variable is not empty but the if statement fails to echo "Found AMI" instead it echoes "Did not find it" which means that the variable new_ami is empty.
How can this happen?
Why is my if statement behaving like this?
how do I fix it?
Thank you for all the help
#davidonstack if test -z "$new_ami" returns false, as it check for if variable has length.
So when its -z returns true, then expression will be executed, if false else expression will be executed.
Check more information in man test
-z string True if the length of string is zero.
-n string True if the length of string is nonzero.
Try to use -n as below
ami=$(aws ec2 describe-images --owners self amazon --filters "Name=name, Values=amzn2-*.0-x86_64-gp2" "Name=state, Values=available" --profile XXXXXX --output json | jq '.Images | sort_by(.CreationDate) | last(.[]).ImageId')
new_ami=$(echo "${ami}" | sed 's/"//g')
echo $new_ami
if test -n "$new_ami"
then
echo "Found AMI"
else
echo "Did not find it"
fi
I am trying to iterate through a line-by-line list of file ID strings (testIDs.txt) and pass them to an AWS command. The command should utilize the loop variable, and the output of the command should be stored in the "folder" variable. Whenever I run the script, I get blank spaces as output in the terminal.
#!bin/bash
while read p; do
folder=$(aws s3 ls s3://a-bucket/ --recursive | grep "${p}" | cut -c 32-)
echo "${folder}"
done < testIDs.txt
The output of the AWS command should be two strings, and I have checked that this is true by running the AWS line separately in the terminal and using a string instead of ${p}.
Note: Right now, I simply want to print folder, but later I will pass folder to another loop.
You may want to just use two for loops, (1) read contents of the bucket to temp (optional/could search it directly) , (2) Loop through IDs and (3) loop through each line of the bucket and look for the ID.
Example:
Read the folder structure to local temp, then for every ID look at the line of the file check for the testIDs and print them to a results file.
#!bin/bash
TMP="/tmp/s3-log-${RANDOM}.dat"
RESULTS="/tmp/s3-log-results.txt"
ID_FILE="testIDs.txt"
> "${RESULTS}"
aws s3 ls s3://a-bucket/ --recursive > "${TMP}"
while read p; do
while IFS= read -r line ;do
if ! [[ $line = *"${p}"* ]]; then
echo "id: ${p} => $(echo ${line} | cut -c 32-)" | tee -a "${RESULTS}"
fi
done < "${TMP}"
done < "${ID_FILE}"
This is probably what you're looking for; use the while loop if you don't know how many values you need to loop through:
while read -r -d $'\t' resourceName; do
echo "$resourceName"
done <<< "$(aws transfer list-servers --query='Servers[*].ServerId' --output text)"
In this case it's the -d $'\t' that influences the Bash word-splitting on the list of output AWS resources.
I have a list of users in a text file. I need to query our campus Active Directory to make sure these users are still "active" users (my machines are not part of the campus AD). The list of users is mapped to a number of identical attributes on the AD (name, cn, sAMAccountName, uid, gecos). I can successfully query the AD with ldapsearch for individual users, so what I am trying to figure out is:
a) how to use the file as input for the query
b) how to construct a shell script so the query will go line by line, and then output any "non-active" users so i can email it for notification.
Any help is much appreciated.
Something you could try to accomplish what you desire would be:
emailFile="/var/tmp/emailFile.txt"
for i in $(cat ${filename}) ; do
results=$(*INPUT LDAPSEARCH HERE REPLACING USERNAME WITH ${i}* | grep -i lastlogontimestamp)
if [[ ${results} != "" ]] ; then
echo "${i} appears to be an active user"
else
echo "${i} IS INACTIVE"
echo "${i}" >> "${emailFile}"
fi
done
This is NOT a working script, but this should give you a very good idea of how to get this to work in your environment. Couple things to note...
${filename} = location/filename of your list of names
INPUT LDAPSEARCH HERE REPLACING USERNAME WITH ${i} = You would enter in your command to run an ldap search here.
example $(ldapsearch -D "cn=manager" -w password -h server.example.com -b "dc=example,dc=com" "cn=${i}" | grep -i "lastlogontimestamp")
"lastlogontimestamp" would be equal to what part of the query you want to examine to determine if the user is active or not.
This would set the ${results} variable to a string you are looking for. This could be a datestamp or empty if users have never logged in.
The next "if" statement, would compare the results to a string/pattern you are expecting. My if statement above, states that if ${results} is not equal to nothing, then inform you that the user you searched is active. If the ${results} are indeed empty, then log the username.
You will have to modify the ldapsearch portion to fit your needs, and the if statement to match what you are expecting to see from your query if the user is not active. I hope this helps.
To answer you question you provided in the comments...
You could use nested if expressions. Example.
if [[ $(some command) = "Some expected result" ]] ; then
if [[ ${results} != "" ]] ; then
echo "${i} appears to be an active user"
else
echo "${i} IS INACTIVE"
echo "${i}" >> "${emailFile}"
fi
fi
done
You could add a second variable where you run a command in the for loop such as
userExists=$(some ldapsearch command to see if user exists)
results=$(*INPUT LDAPSEARCH HERE REPLACING USERNAME WITH ${i}* | grep -i lastlogontimestamp
if [[ ${userExists} = "Some expected result" ]] ; then
if [[ ${results} != "" ]] ; then
You also asked a question about using "grep" to look for more than one item. Example is below.
grep "514\|546"
Example 2.
Lets say I have a file called test.txt with 5 lines
one
two
three
four
five
I would run the following command to look inside the file to find "two" and "three"
cat test.txt | grep "two\|three"
I have a number of bash scripts, each doing its own thing merrily. Do note that while I program in other languages, I only use Bash to automate things, and am not very good at it.
I'm now trying to combine a number of them to create "meta" scripts, if you will, which use other scripts as steps. The problem is that I need to parse the output of each step to be able to pass a part of it as params to the next step.
An example:
stepA.sh
[...does stuff here...]
echo "Task complete successfuly"
echo "Files available at: $d1/$1"
echo "Logs available at: $d2/$1"
both the above are paths, such as /var/www/thisisatest and /var/log/thisisatest (note that files always start with /var/www and logs always start with /var/log ). I'm only interested in the files path.
steB.sh
[...does stuff here...]
echo "Creation of $d1 complete."
echo "Access with username $usr and password $pass"
all variables here are simple strings, that may contain special characters (no spaces)
What I'm trying to build is a script that runs stepA.sh, then stepB.sh and uses the output of each to do its own stuff. What I'm currently doing (both above scripts are symlinked to /usr/local/bin without the .sh part and made executable):
#!/bin/bash
stepA $1 | while read -r line; do
# Create the container, and grab the file location
# then pass it to then next pipe
if [[ "$line" == *:* ]]
then
POS=`expr index "$line" "/"`
PTH="/${line:$POS}"
if [[ "$PTH" == *www* ]]
then
#OK, have what I need here, now what?
echo $PTH;
fi
fi
done
# Somehow get $PTH here
stepB $1 | while read -r line; do
...
done
#somehow have the required strings here
I'm stuck in passing the PTH to the next step. I understand this is because piping runs it in a subshell, however all examples I've seen refer to to files and not commands, and I could not make this to work. I tried piping the echo to a "next step" such as
stepA | while ...
echo $PTH
done | while ...
#Got my var here, but cannot run stuff
done
How can I run stepA and have the PTH variable available for later?
Is there a "better way" to extract the path I need from the output than nested ifs ?
Thanks in advance!
Since you're using bash explicitly (in the shebang line), you can use its process substitution feature instead of a pipe:
while read -r line; do
if [[ "$line" == *:* ]]
.....
fi
done < <(stepA $1)
Alternately, you could capture the command's output to a string variable, and then parse that:
output="$(stepA $1)"
tmp="${output#*$'\nFiles available at: '}" # output with everything before the filepath trimmed
filepath="${tmp%%$'\n'*}" # trim the first newline and everything after it from $tmp
tmp="${output#*$'\nLogs available at: '}"
logpath="${tmp%%$'\n'*}"
the following script is working fine on one server but on the other it gives an error
#!/bin/bash
processLine(){
line="$#" # get the complete first line which is the complete script path
name_of_file=$(basename "$line" ".php") # seperate from the path the name of file excluding extension
ps aux | grep -v grep | grep -q "$line" || ( nohup php -f "$line" > /var/log/iphorex/$name_of_file.log & )
}
FILE=""
if [ "$1" == "" ]; then
FILE="/var/www/iphorex/live/infi_script.txt"
else
FILE="$1"
# make sure file exist and readable
if [ ! -f $FILE ]; then
echo "$FILE : does not exists. Script will terminate now."
exit 1
elif [ ! -r $FILE ]; then
echo "$FILE: can not be read. Script will terminate now."
exit 2
fi
fi
# read $FILE using the file descriptors
# $ifs is a shell variable. Varies from version to version. known as internal file seperator.
# Set loop separator to end of line
BACKUPIFS=$IFS
#use a temp. variable such that $ifs can be restored later.
IFS=$(echo -en "\n")
exec 3<&0
exec 0<"$FILE"
while read -r line
do
# use $line variable to process line in processLine() function
processLine $line
done
exec 0<&3
# restore $IFS which was used to determine what the field separators are
IFS=$BAKCUPIFS
exit 0
i am just trying to read a file containing path of various scripts and then checking whether those scripts are already running and if not running them. The file /var/www/iphorex/live/infi_script.txt is definitely present. I get the following error on my amazon server-
[: 24: unexpected operator
infinity.sh: 32: cannot open : No such file
Thanks for your helps in advance.
You should just initialize file with
FILE=${1:-/var/www/iphorex/live/infi_script.txt}
and then skip the existence check. If the file
does not exist or is not readable, the exec 0< will
fail with a reasonable error message (there's no point
in you trying to guess what the error message will be,
just let the shell report the error.)
I think the problem is that the shell on the failing server
does not like "==" in the equality test. (Many implementations
of test only accept one '=', but I thought even older bash
had a builtin that accepted two '==' so I might be way off base.)
I would simply eliminate your lines from FILE="" down to
the end of the existence check and replace them with the
assignment above, letting the shell's standard default
mechanism work for you.
Note that if you do eliminate the existence check, you'll want
to either add
set -e
near the top of the script, or add a check on the exec:
exec 0<"$FILE" || exit 1
so that the script does not continue if the file is not usable.
For bash (and ksh and others), you want [[ "$x" == "$y" ]] with double brackets. That uses the built-in expression handling. A single bracket calls out to the test executable which is probably barfing on the ==.
Also, you can use [[ -z "$x" ]] to test for zero-length strings, instead of comparing to the empty string. See "CONDITIONAL EXPRESSIONS" in your bash manual.