I am trying to read a file that was outputted with table format. I am able to read the file that has Volume description, snapshotId and TimeStarted list from an aws region. I am asking user to input a volume name and output the snapshotId for the volume entered. The list contains volumes0 through volumes30.
The issue is, when user enters Volume0, it outputs the snapshotId correctly but if the user enters Volume20, it will only output |. My guess is because the original file being read is in table format, it is doing this. Initially I though I could put in a condition, where if user enters Volume0, print snapshotId else if user enters Volume20 then print snapshotid.
I am looking for a better way to do this. How can I ignore table format when reading the file, should I convert it to text format? How? Or how can I ignore any format when reading? Here is my bash script:
readoutput() {
echo "Hello, please tell me what Volume you are searching for..(Volume?):"
read volSearch
echo "Searching for newest SnapshotIds from /Users/User/Downloads/GetSnapId for:" $volSearch
sleep 5
input="/Users/User/Downloads/GetSnapId"
if x=$(grep -m 1 "$volSearch" "$input")
then
echo "$x"
else
echo "$volSearch not found..ending search"
fi
extractSnap=$(echo "$x" | grep "snap-" | awk '{print $7}')
echo $extractSnap
}
readoutput
The issue that awk is not too smart, and tries to determine table separator automatically. In first row with Volume0 you have space before vertical line, so it things that both of space and vertical line are separators, but in next row, you have no space, so it takes wrong column.
Try next:
extractSnap=$(echo "$x" | cut -d'|' -f3 | cut -d'-' -f 2)
Related
Contents in data.csv
John.duff#example.com status
Hill.ref#example.com status
David.nelson#example.com status
Contents in data2.csv
John.duff#example.com unique_user_id
Hill.ref#example.com unique_user_id
David.nelson#example.com unique_user_id
My sample code:
While read c1.f1 c2.f1 c1.f2 c2.f2;
Do
Echo “$c2.f2 $c2.f1”
Done<paste data.csv data2.csv
This code will print me the values in 2nd columns of the both the files but I need some kind of validation before this like adding “if statement” to check whether email id’s of the users in first columns are equal in both files before printing 2nd columns. I want to print unique_user_id’s and status only by mapping the email id’s Can I do that?? Or can I do this better with the awk command and what is the best way to do this? Thank you
Here is a quick script I came up with:
#!/bin/bash
# using your method
paste data.csv data2.csv | while read file1_email file1_value file2_email file2_value
do
if [ "$file1_email" == "$file2_email" ]
then
echo "$file1_email, $file1_value, $file2_value"
fi
done
echo "------------------------------------------"
# this is mine
while read file1_email file1_value
do
file2_counter=$(grep -c $file1_email data2.csv)
if [ $file2_counter -gt 0 ]
then
file2_line=$(grep $file1_email data2.csv)
file2_value=$(echo $file2_line | awk '{print $2}')
echo "$file1_email, $file1_value, $file2_value"
fi
done <data.csv
The problem with your approach, it that it takes for granted that the address emails will always appear in the same lines in both files.
ex. if data2.csv has a different sorting applied, it could have all the same email addresses than in data2.csv. But the paste command will not sort anything, just display the data as it reads it.
That is why I came up with a different approach. Read the email address from data.csv, search in data2.csv if it is there, if yes, extract the value form it, and print all that data.
If you use my method, it will loop through every line of data.csv. But data.csv must be "clean" since the read assumes 2 fields. The first one being the email, the second one being the value. You should have a look at the data.csv content.
Then once in the while, it checks if the email read from data.csv is present in data2.csv. If it is not, nothing is done (there is no else to the if). But if you want to do something anyway, add an else and some code.
I have below record (and many other such records) in one file
9460 xyz abc (lmn):1027739543798. Taxpayer's identification number (INN): 123. For all IIB. 2016/02/03
I need to search for the keyword IIB. If it matches, then I need to take that entire record and write to another file.
Below is the code which already exists. This code is not working. Problem with this code is when it takes the full matched
record, it is ignoring the text which falls after ":" and writing to another file.
cat keyword.cfg | while read KwdName
do
echo "KEYWORD:"${KwdName} //This prints IIB
grep "^${KwdName}\|${KwdName}\|~${KwdName}~\|:${KwdName}$\|:${KwdName}~" ${mainFileWithListOfRecords} | awk -F ":" '{print $1}' >> ${destinationFile}
done
So, instead of writing below record to destination file
9460 xyz abc (lmn):1027739543798. Taxpayer's identification number (INN): 123. For all IIB. 2016/02/03
It is only writing,
9460 xyz abc (lmn)
cat -vte mainFileWithListOfRecords gives below output
9460^IMEZHPROMBANK^I^ICJSC ;IIB;~ Moscow, (lmn): 1027739543798. Taxpayer's identification number (INN): 123. For all IIB. 2016/02/031#msid=s1448434872350^IC1^I2000/12/28^I2015/11/26^I^I$
The short fix is replacing
awk -F ":" '{print $1}'
with
cut -d ":" -f2-
But what are you cutting? Maybe ${mainFileWithListOfRecords} is a variabele with a list of files. In that case grep will show the matching file in front of its matches. You can change that with the -h option.
The result is that you do not need to cut or awk:
grep -h "${KwdName}" ${mainFileWithListOfRecords} >> ${destinationFile}
(I changed the searchstring as well, with \|${KwdName}\| in your searchstring you will match KwdName in all combinations)
Of course, it cuts on the colon - you programmed it that way. In your code, you have | awk -F ":" '{print $1}', which basically means "throw away everything starting from the first colon".
If you don't want to do this, why do you explicitly request it? What was your original intention, when writing the awk command?
I am having some problems with using the first column ${1} as input to a script.
Currently the portions of the script looks like this.
#!/bin/bash
INPUT="${1}"
for NAME in `cat ${INPUT}`
do
SIZE="`du -sm /FAServer/na3250-a/homes/${NAME} | sed 's|/FAServer/na3250-a/homes/||'`"
DATESTAMP=`ls -ld /FAServer/na3250-a/homes/${NAME} | awk '{print $6}'`
echo "${SIZE} ${DATESTAMP}"
done
However, I want to modify the INPUT="${1}" to take the first {1} within a specific file. This is so I can run the lines above in another script and use a file that is previously generated as the input. Also to have the output go out to a new file.
So something like:
INPUT="$location/DisabledActiveHome ${1}" ???
Here's my full script below.
#!/bin/bash
# This script will search through Disabled Users OU and compare that list of
# names against the current active Home directories. This is to find out
# how much space those Home directories take up and which need to be removed.
# MUST BE RUN AS SUDO!
# Setting variables for _adm and storage path.
echo "Please provide your _adm account name:"
read _adm
echo "Please state where you want the files to be generated: (absolute path)"
read location
# String of commands to lookup information using ldapsearch
ldapsearch -x -LLL -h "REDACTED" -D $_adm#"REDACTED" -W -b "OU=Accounts,OU=Disabled_Objects,DC="XX",DC="XX",DC="XX"" "cn=*" | grep 'sAMAccountName'| egrep -v '_adm$' | cut -d' ' -f2 > $location/DisabledHome
# Get a list of all the active Home directories
ls /FAServer/na3250-a/homes > $location/ActiveHome
# Compare the Disabled accounts against Active Home directories
grep -o -f $location/DisabledHome $location/ActiveHome > $location/DisabledActiveHome
# Now get the size and datestamp for the disabled folders
INPUT="${1}"
for NAME in `cat ${INPUT}`
do
SIZE="`du -sm /FAServer/na3250-a/homes/${NAME} | sed 's|/FAServer/na3250-a/homes/||'`"
DATESTAMP=`ls -ld /FAServer/na3250-a/homes/${NAME} | awk '{print $6}'`
echo "${SIZE} ${DATESTAMP}"
done
I'm new to all of this so any help is welcome. I will be happy to clarify any and all questions you might have.
EDIT: A little more explanation because I'm terrible at these things.
The lines of code below came from a previous script are a FOR loop:
INPUT="${1}"
for NAME in `cat ${INPUT}`
do
SIZE="`du -sm /FAServer/na3250-a/homes/${NAME} | sed 's|/FAServer/na3250-a/homes/||'`"
DATESTAMP=`ls -ld /FAServer/na3250-a/homes/${NAME} | awk '{print $6}'`
echo "${SIZE} ${DATESTAMP}"
done
It is executed by typing:
./Script ./file
The FILE that is being referenced has one column of user names and no other data:
User1
User2
User3
etc.
The Script would take the file and look at the first users name, which is reference by
INPUT=${1}
then run a DU command on that user and find out what the size of their HOME drive is. That would be reported by the SIZE variable. It will do the same thing with the DATESTAMP in regards to when the HOME drive was created for the user. When it is done doing the tasks for that user, it would move on to the next one in the column until it is done.
So following that logic, I want to automate the entire process. Instead of doing this in two steps, I would like to make this all a one step process.
The first process would be to generate the $location/DisabledActiveHome file, which would have all of the disabled users names. Then to run the last portion to get the Size and creation date of each HOME drive for all the users in the DisabledActiveHome file.
So to do that, I need to modify the
INPUT=${1}
line to reflect the previously generated file.
$location/DisabledActiveHome
I don't understand your question really, but I think you want this. Say your file is called file.txt and looks like this:
1 99
2 98
3 97
4 96
You can get the first column like this:
awk '{print $1}' file.txt
1
2
3
4
If you want to use that in your script, do this
while read NAME; do
echo $NAME
done < <(awk '{print $1}' file.txt)
1
2
3
4
Or you may prefer cut like this:
while read NAME; do
echo $NAME
done < <(cut -d" " -f1 file.txt)
1
2
3
4
Or this may suit even better
while read NAME OtherUnwantedJunk; do
echo $NAME
done < file.txt
1
2
3
4
This last, and probably best, solution above uses IFS, which is bash's Input Field Separator, so if your file looked like this
1:99
2:98
3:97
4:96
you would do this
while IFS=":" read NAME OtherUnwantedJunk; do
echo $NAME
done < file.txt
1
2
3
4
INPUT="$location/DisabledActiveHome" worked like a charm. I was confused about the syntax and the proper usage and output
forgive me if this is painfully simple but I'm not a programmer so it's hard for me to tell what's easy and what's hard.
I have a bash script that I use (that someone else wrote) for finding out internal customer data where I basically run "info customername" and it searches our internal customer database for all customer records matching that customer name and outputs a list with their account numbers (which all have the same prefix of 11111xxxxxxxx), in the form of "Sample Customer - 111119382818873".
We have another bash script where you enter "extrainfo 11111xxxxxxxx", we get the plaintext data from their account, which we use for many things that are important to us.
The missing feature is that "extrainfo" cannot search by name, only number. So I'd like to bridge that gap. Ideally, I'd enter "extrainfo customername" and it would run a search using "info customername", generate a list of results as a menu, allow me to choose which customer I meant, and then run the "extrainfo 11111xxxxxxxxx" command of that customer. If there is only one match, it would automatically run the extrainfo command properly.
Here's what I have that works but only for the first result that "info customername" generates:
#!/bin/bash
key=`/usr/local/bin/info $1 | grep 11111 | awk '{print $NF}'`
/usr/local/bin/extrainfo $key
It's the menu stuff I'm having a hard time figuring out. I hope this was clear but again, I'm pretty dumb with this stuff so I probably left something important out. Thanks.
This might work for you:
#!/bin/bash
# Set the prompt for the select command
PS3="Type a number or 'q' to quit: "
# Create a list of customer names and numbers (fill gaps with underscores)
keys=$(/usr/local/bin/info $1 | sed 's/ /_/g')
# Show a menu and ask for input.
select key in $keys; do
if [ -n "$key" ]; then
/usr/local/bin/extrainfo $(sed 's/.*_11111/11111/' <<<"$key")
fi
break
done
Basically, this script reads all the customer info, finds all lines with the customer number prefix, and loads it into search.txt. Then it displays the file with line numbers in front of it, waits for you to choose a line number, and then strips out the customer name and spaces in front of the customer id. Finally, it runs my other script with just the customer id. It's hacky but functional.
#!/bin/bash
/usr/local/bin/info $1 | grep 11111 > search.txt
cat -n search.txt
read num
key=`sed -n ${num}p search.txt | awk '{print $NF}'`
/usr/local/bin/extrainfo $key
I have a script which uses grep to find lines in a text file (ics calendar to be specific)
My script finds a date match, then goes up and down a few lines to copy the summary and start time of the appointment into a separate variable. The problem I have is that I'm going to have multiple appointments at the same time, and I need to run through the whole process for each result in grep.
Example:
LINE=`grep -F -n 20130304T232200 /path/to/calendar.ics | cut -f1 d:`
And it outputs only the lines, such as
86 89
Then it goes on to capture my other variables, as such:
SUMMARYLINE=$(( $LINE + 5 ))
SUMMARY:`sed -n "$SUMMARYLINE"p /path/to/calendar.ics
my script runs fine with one output, but it obviously won't work with more than 1 and I need for it to. should I send the grep results into an array? a separate text file to read from? I'm sure I'll need a while loop in here somehow. Need some help please.
You can call grep from a loop quite easily:
while IFS=':' read -r LINE notused # avoids the use of cut
do
# First field is now in $LINE
# Further processing
done < <(grep -F -n 20130304T232200 /path/to/calendar.ics)
However, if the file is not too large then it might be easier to read the whole file into an array and more around that.
With your proposed solution, you are reading through the file several times. Using awk, you can do it in one pass:
awk -F: -v time=20130304T232200 '
$1 == "SUMMARY" {summary = substr($0,9)}
/^DTSTART/ {start = $2}
/^END:VEVENT/ && start == time {print summary}
' calendar.ics