reuse numerical value from a text entry in subsequent entries - bash

I'm trying to automate a procedure and to create the output file that I need. So far everything is going well, but I cannot figure out how to take the number at the end of one row and fill it in for rows after it.
I've tried solutions using awk, sed, etc, but so far, I can't seem to figure this out even with the helps of many googles.
POP-Test-01
10.10.10.10
User: User
Pass: Pass
POP-Test-02
10.10.10.11
User: User
Pass: Pass
POP-Test-03
10.10.10.12
User: User
Pass: Pass
But I want it to take the numerical values from the first line, and append it to the user and pass lines. But for the pass line, I'd like to add it twice.
POP-Test-01
10.10.10.10
User: User01
Pass: Pass0101
POP-Test-02
10.10.10.11
User: User02
Pass: Pass0202
POP-Test-03
10.10.10.12
User: User03
Pass: Pass0303
These are only examples, but the last two digits of the hostname(ex. POP-Test-03) will always be numerical digits.
Edit:
I've had some requests for more details. Sorry about that guys, so here is the skinny. Please note that all values are fake, these come from an input file. I just used the most generic values I could think of.
The input is a text file with only the values in the top example.
So I run a script that deploys instances and the only output I get from the deployment script is the Hostname and the IP address. I know it's not clean, but below is how I'm adding the user and password. But the user and pass are always the same between each batch of servers with only the number of the server being appended to the user and the password(twice). I'm doing this because an application we use requires the input of:
HOSTNAME
IP
User: (username)
Pass: (password)
(empty line)
Normally I have to enter these manually which is time consuming and I was hoping not necessary since I've started it with the commands below in a script I am trying to build to automate the process.
So I start with this:
POP-Test-01
10.10.10.10
And I'm trying to get to this:
POP-Test-01
10.10.10.10
User: User01
Pass: Pass0101
awk '/([0-9]{1,3}[\.]){3}[0-9]{1,3}/{print;print "User\:
Train";next}1' DeploymentDetails2.txt | tee DeploymentDetails2.txt<br/>
awk '/User/{print;print "Pass\: Training";next}1'
DeploymentDetails2.txt | tee DeploymentDetails2.txt<br/>
awk '/Pass/{print;print "\n";next}1' DeploymentDetails2.txt | tee
DeploymentDetails.txt<br/>

This appears to produce what you are looking for:
#!/bin/bash
HOST="POP-Test-"
IP=10
for i in {01..99} ; do
echo ${HOST}${i}
echo "10.0.10.${IP}"
echo "User: User${i}"
echo "Pass: Pass${i}${i}"
echo
let IP++
done
As you did not specify that there was an input file, I took the liberty of generating the ending number in the script, and reusing it in the user and pass lines.

Related

Filter Column in CSV and get the unique value

I am having three columns in a CSV: Client Name, save Set Name and Status. For some clients, we have two Status as Failed and Success both. So, I want to filter those clients only which have status as only Failed. Clients who are having two entries such as Failed and success also, I want to omit.
When I am using the listed command, it's giving me values whose status was successful also might be later on. I want values which are only Failed. Not successful even once
cat "$pwd"/Daily-Failed.csv|egrep -i 'failed|Interrupted'|awk -F',' '{print $2,$3,$9}'|sort -u > "$pwd"/Final-Failed/Failed.csv
(edit) Or with newlines:
cat "$pwd"/Daily-Failed.csv|
egrep -i 'failed|Interrupted'|
awk -F',' '{print $2,$3,$9}'|
sort -u > "$pwd"/Final-Failed/Failed.csv
enter image description here
Please find the input and desired output. Input Client Name, Save Set, Status
Star,D:/,Failed
Star,C:/,Failed
Moon,C:/,Failed
Galaxy,D:/,Failed
Sun,D:/,Failed
Star,C:/,Success
Sun,D:/,Success
Output "Client Name","Save Set",Status
Galaxy,D:/,Failed
Moon,C:/,Failed
Star,D:/,Failed
I want to filter those clients only which have status as only Failed. Clients who are having two entries such as Failed and success also, I want to omit.
I'm going to assume, looking at your sample input (Which really needs to be text in your question, not an image), that both the Client Name and Save Set columns matter - you have (Star, C:/) with both success and failure rows, and (Star, D:/) with just failure, and the latter shows up in your output, and that's the only way that would make sense given your stated goal. On the other hand you also have two (Sun, D:/) rows, one success, one failure, and that shows up in your output even though it doesn't meet your criteria any way you look at it...
Anyways, this sort of grouping and filtering of tabular data screams database, and I like to script sqlite to make it do all the work in such cases:
#!/bin/sh
filename=Daily-Failed.csv
sqlite3 -batch -csv -header <<EOF
.import '${filename}' tbl
SELECT *
FROM tbl
GROUP BY "Client Name", "Save Set"
HAVING count(*) = 1 AND Status = 'Failed'
EOF
after taking the data in your image and turning it into a CSV file Daily-Failed.csv looking like
Client Name,Save Set,Status
Star,D:/,Failed
Star,C:/,Failed
Moon,C:/,Failed
Galaxy,D:/,Failed
Sun,D:/,Failed
Star,C:/,Success
Sun,D:/,Success
that script will output
"Client Name","Save Set",Status
Galaxy,D:/,Failed
Moon,C:/,Failed
Star,D:/,Failed

Bash complete with "#"-sign

I am trying to create an autocomplete script for scp.
The script reads the user and hostname from my .ssh/config file
My .ssh/config file looks like:
Host host1
HostName host1
User userA
port 22
Host host2
HostName host2
User userB
port 22
Host host3
HostName host3
User userB
port 22
My .autocomplete_scp.sh file is:
# SSH
function _scp_completion() {
pcregrep -o -M 'HostName [a-zA-Z.]+[\n\t\s]+User [a-zA-Z]+'
$HOME/.ssh/config | awk 'NR % 2 == 1 { o=$2 ; next } { print $2"#"o}'
}
complete -W "$(_scp_completion)" scp
I source this file in my bashrc.
Now when I type userA and press Tab, the autocomplete function will give me userA#host1. When I type userB and hit Tab, the autocomplete function will give me userB#, but I am not able to get the full string (userB#host2 or userB#host3).
It also doesn't work when I type userA#h and hit the Tab button twice. So it seems to get stuck due to the # sign.
(When I remove the # sign from the _scp_completion function it works fine.)
Any ideas to fix this? Thanks!
The easy way to make it work is to remove '#' from $COMP_WORDBREAKS or Bash would handle # by itself. You can try like this:
COMP_WORDBREAKS=${COMP_WORDBREAKS//#}
complete -W 'userA#host1 userB#host2 userB#host3' scp
According to bash doc:
COMP_WORDBREAKS
The set of characters that the readline library treats as word separators when performing word completion.

Use multiple column variables in bash script to pull output from routers

I have a script that logs on to routers and pulls output that is named routerauto. I would like to use data from a text file to automatically populate required commands to pull required info from a large number of routers.
Ultimately I would like the script to move through each line of the text file, filling in the gaps with the output from the columns as below. The text file uses tab as separator.
routerauto VARIABLE1 "sh service id VARIABLE2 sap VARIABLE4 detail"
Example data:
hostnamei serv-id cct sap
london-officei 123456 No987654321 8/1/4:100
Example output:
routerauto london-office "sh service id 123456 sap 8/1/4:100 detail"
Here is a bash only solution:
#!/bin/bash
while read hostnamei servid cct sap; do
echo routerauto $hostnamei \"sh service id $servid sap $sap detail\"
done < <(tail -n +2 sample.data)
Producing given your sample file:
routerauto london-officei "sh service id 123456 sap 8/1/4:100 detail"
Please note this assume no space are allowed in your various data fields.

shell script display grep results

I need some help with displaying how many times two strings are found on the same line! Lets say I want to search the file 'test.txt', this file contains names and IP's, I want to enter a name as a parameter when running the script, the script will search the file for that name, and check if there's an IP-address there also. I have tried using the 'grep' command, but I don't know how I can display the results in a good way, I want it like this:
Name: John Doe IP: xxx.xxx.xx.x count: 3
The count is how many times this line was found, this is how my grep script looks like right now:
#!/bin/bash
echo "Searching $1 for the Name '$2'"
result=$(grep "$2" $1 | grep -E "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)")
echo $result
I will run the script like 'sh search test.txt John'.
I'm having trouble displaying the information I get from the grep command, maybe there's a better way to do this?
EDIT:
Okey, I will try to explain a little better, let's say I want to search a .log file, I want a script to search that file for a string the user enters as a parameter. i.e if the user enters 'sh search test.log logged in' the script will search for the string "logged in" within the file 'test.log'. If the script finds this line on the same line as a IP-address the IP address is printed, along with how many times this line was found.
And I simply don't know how to do it, I'm new to shell scripting, and was hoping I could use grep along with regular expressions for this! I will keep on trying, and update this question with an answer if I figure it out.
I don't have said file on my computer, but it looks something like this:
Apr 25 11:33:21 Admin CRON[2792]: pam_unix(cron:session): session opened for user 192.168.1.2 by (uid=0)
Apr 25 12:39:01 Admin CRON[2792]: pam_unix(cron:session): session closed for user 192.168.1.2
Apr 27 07:42:07 John CRON[2792]: pam_unix(cron:session): session opened for user 192.168.2.22 by (uid=0)
Apr 27 14:23:11 John CRON[2792]: pam_unix(cron:session): session closed for user 192.168.2.22
Apr 29 10:20:18 Admin CRON[2792]: pam_unix(cron:session): session opened for user 192.168.1.2 by (uid=0)
Apr 29 12:15:04 Admin CRON[2792]: pam_unix(cron:session): session closed for user 192.168.1.2
Here is a simple Awk script which does what you request, based on the log snippet you posted.
awk -v user="$2" '$4 == user { i[$11]++ }
END { for (a in i) printf ("Name: %s IP: %s count: %i\n", user, a, i[a]) }' "$1"
If the fourth whitespace-separated field in the log file matches the requested user name (which was passed to the shell script as its second parameter), add one to the count for the IP address (from field 11).
At the end, loop through all non-zero IP addresses, and print a summary for each. (The user name is obviously whatever was passed in, but matches your expected output.)
This is a very basic Awk script; if you think you want to learn more, I urge you to consult a simple introduction, rather than follow up here.
If you want a simpler grep-only solution, something like this provides the information in a different format:
grep "$2" "$1" |
grep -o -E '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)' |
sort | uniq -c | sort -rn
The trick here is the -o option to the second grep, which extracts just the IP address from the matching line. It is however less precise than the Awk script; for example, a user named "sess" would match every input line in the log. You can improve on that slightly by using grep -w in the first grep -- that still won't help against users named "pam" --, but Awk really gives you a lot more control.
My original answer is below this line, partly becaus it's tangentially useful, partially because it is required in order to understand the pesky comment thread below.
The following
result=$(command)
echo $result
is wrong. You need the second line to be
echo "$result"
but in addition, the detour over echo is superfluous; the simple way to write that is simply
command

How do I echo username, full name, and login time from finger into columns? I'm using bash on openSUSE13.1

Basically I have three users logged in to my machine right now. Test User1, Test User2, and Test User3.
I would like to use finger to get username, full name and the time they logged into the machine.
I would like to output the information like so:
Login Name Login Time
testuser1 Test User1 1300
testuser2 Test User2 1600
testuser3 Tesr User3 1930
I have two tabs in between Login and Name and three tabs between Login Time. The same goes for the user information below each header.
I cannot figure out how to pull this data from finger very well and I absolutely cannot figure out how to get the information into nice, neat, readable columns. Thanks in advance for any help!
This might not be perfect so you'll have to play around with substr starting and ending points. Should be good enough to get you started:
finger -s testuser1 testuser2 testuser3 | awk '{print substr($0,1,31),substr($0,46,14)}'
Try :r!finger. On my Mac, I get nice columns. YMMV.
:help :r!
Here's another way using awk:
finger -l | awk '{ split($1, a, OFS); print a[2], a[4], substr($3, 20, 6) }' FS="\n" RS= | column -t
The -l flag of finger produces a multi-line format (and is compatible with the -s flag). This is useful when fields like 'name' are absent. We can then process the records using awk in paragraph mode. In my example above, you can adjust the sub-string to suit the datespec of your choice. If you have gawk, then you'll have access to some time functions that may interest you if you wish to change the spec. Finally, you can print the fields of interest into column -t for pretty printing. HTH.

Resources