This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 6 years ago.
In bash script from the output below, I need to print the lines between "Device #0" and "Device #1", but as all that is part of a bigger script I should use variables for start/stop lines.
----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
Device #0
Device is a Hard drive
State : Online
Block Size : 512 Bytes
Supported : Yes
Programmed Max Speed : SATA 6.0 Gb/s
Transfer Speed : SATA 6.0 Gb/s
Reported Channel,Device(T:L) : 0,0(0:0)
Reported Location : Connector 0, Device 0
Vendor : ATA
Model :
Firmware : 003Q
Serial number : S2HTNX0H418779
World-wide name : 5002538C402805A4
Reserved Size : 265496 KB
Used Size : 897129 MB
Unused Size : 18327 MB
Total Size : 915715 MB
Write Cache : Enabled (write-back)
FRU : None
S.M.A.R.T. : No
S.M.A.R.T. warnings : 0
Power State : Full rpm
Supported Power States : Full power,Powered off
SSD : Yes
Temperature : 39 C/ 102 F
NCQ status : Enabled
----------------------------------------------------------------
Device Phy Information
----------------------------------------------------------------
Phy #0
PHY Identifier : 0
SAS Address : 30000D1701801803
Attached PHY Identifier : 3
Attached SAS Address : 50000D1701801800
----------------------------------------------------------------
Runtime Error Counters
----------------------------------------------------------------
Hardware Error Count : 0
Medium Error Count : 0
Parity Error Count : 0
Link Failure Count : 0
Aborted Command Count : 0
SMART Warning Count : 0
Model, SSD
----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
Device #1
Device is a Hard drive
State : Online
Block Size : 512 Bytes
Supported : Yes
Programmed Max Speed : SATA 6.0 Gb/s
Transfer Speed : SATA 6.0 Gb/s
Reported Channel,Device(T:L) : 0,0(0:0)
Reported Location : Connector 0, Device 0
Vendor : ATA
Model :
Firmware : 003Q
Serial number : S2HTNX0H418779
World-wide name : 5002538C402805A4
Reserved Size : 265496 KB
Used Size : 897129 MB
Unused Size : 18327 MB
Total Size : 915715 MB
Write Cache : Enabled (write-back)
FRU : None
S.M.A.R.T. : No
S.M.A.R.T. warnings : 0
Power State : Full rpm
Supported Power States : Full power,Powered off
SSD : Yes
Temperature : 39 C/ 102 F
NCQ status : Enabled
----------------------------------------------------------------
Device Phy Information
----------------------------------------------------------------
Phy #0
PHY Identifier : 0
SAS Address : 30000D1701801803
Attached PHY Identifier : 3
Attached SAS Address : 50000D1701801800
----------------------------------------------------------------
Runtime Error Counters
----------------------------------------------------------------
Hardware Error Count : 0
Medium Error Count : 0
Parity Error Count : 0
Link Failure Count : 0
Aborted Command Count : 0
SMART Warning Count : 0
Model, SSD
----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
Device #2
Device is a Hard drive
State : Online
Block Size : 512 Bytes
Supported : Yes
Programmed Max Speed : SATA 6.0 Gb/s
Transfer Speed : SATA 6.0 Gb/s
Reported Channel,Device(T:L) : 0,0(0:0)
Reported Location : Connector 0, Device 0
Vendor : ATA
Model :
Firmware : 003Q
Serial number : S2HTNX0H418779
World-wide name : 5002538C402805A4
Reserved Size : 265496 KB
Used Size : 897129 MB
Unused Size : 18327 MB
Total Size : 915715 MB
Write Cache : Enabled (write-back)
FRU : None
S.M.A.R.T. : No
S.M.A.R.T. warnings : 0
Power State : Full rpm
Supported Power States : Full power,Powered off
SSD : Yes
Temperature : 39 C/ 102 F
NCQ status : Enabled
----------------------------------------------------------------
Device Phy Information
----------------------------------------------------------------
Phy #0
PHY Identifier : 0
SAS Address : 30000D1701801803
Attached PHY Identifier : 3
Attached SAS Address : 50000D1701801800
----------------------------------------------------------------
Runtime Error Counters
----------------------------------------------------------------
Hardware Error Count : 0
Medium Error Count : 0
Parity Error Count : 0
Link Failure Count : 0
Aborted Command Count : 0
SMART Warning Count : 0
Model, SSD
In this case the output for Device #0 to Device #2 is the same, but it doesn't really matter for the test.
So trying with cat arcconf | awk '/Device #0/,/Device #1/' where the output above is stored in a file called arcconf works. But trying to use variables instead of 0 and 1 doesn't work at all:
MIN_INDEX=0
INDEX=1
cat arcconf | awk '/Device #"$MIN_INDEX"/,/Device #"$INDEX"/'
cat arcconf | sed -n -e "/Device #"$INDEX_MIN"$/,/Device #"$INDEX"$/{ /Device #"$INDEX_MIN"$/d; /Device #"$INDEX"$/d; p; }"
It doesn't display anything.
Could you please help.
Also as I am going to use the output from Device to Device lines several times, is it possible to store it in some new variable which I should use in the future?
Thanks,
Valentina
Bash variables are not expanded within single quotes, that's why the first command doesn't work. Replace single quotes with double quotes:
cat arcconf | awk "/Device #$MIN_INDEX/,/Device #$INDEX/"
The second command should work, but it's unnecessarily complicated.
You don't need to drop out of the double-quoted string for the sake of the variables, this will work fine:
cat arcconf | sed -n -e "/Device #$INDEX_MIN$/,/Device #$INDEX$/{ /Device #$INDEX_MIN$/d; /Device #$INDEX$/d; p; }"
In fact it's better this way, as now the variables are within a double-quoted string, which is a good habit, as unquoted variables containing spaces would cause problems.
You can send variables to awk via -v var=val:
awk \
-v start="Device #$MIN_INDEX" \
-v end="Device #$MAX_INDEX" \
'$0 ~ end { p=0 }
$0 ~ start { p=1 }
p' arcconf
Simply by moving around p; you can whether or not to include the start and end line:
$0 ~ end { p=0 }; p; $0 ~ start { p=1 } # Will not include start nor end
$0 ~ end { p=0 }; $0 ~ start { p=1 }; p # Will include start and end
$0 ~ start { p=1 }; p; $0 ~ end { p=0 } # Will include start but not end
$0 ~ end { p=0 }; p; $0 ~ start { p=1 } # Will include end but not start
You can try below sed command -
#MIN_INDEX=0
#INDEX=1
#sed -n "/Device\ #$MIN_INDEX/,/Device\ #$INDEX/p" kk.txt
And to set the output to a variable -
#sed -n "/Device\ #$MIN_INDEX/,/Device\ #$INDEX/w output.txt" kk.txt
#var=`cat output.txt`
#echo $var
Explanation
-n to remove duplicate when pattern match.
w is to write the output to file output.txt
p is to print. We need to use escape character \ to search space.
Related
i have the script that has output logfile.txt :
File_name1
Replay requests : 5
Replay responsee : 5
Replay completed : 5
--------------------
File_name2
Replay requests : 3
Replay responsee : 3
Replay completed : 3
--------------------
I need to check that counts at all 3 line were the same, and if one of the line mismatched move File_name to "echo".
I tried to grep with pattern file like cat logfile.txt | grep -f patternfile.ptrn with for loop, but there is no result, can`t find how to put first count in parameter that allow me to check it with next line, and how to check when there are many files_names at the logfile.
Pattern was :
Replay requests :
Replay responsee :
Replay completed :
--------------------
Its a correct idea or mb i`m moving in wrong way?
I need to check that counts at all 3 line were the same, and if one of the line mismatched move File_name to "echo".
Here is one approach/solution.
Given your input example.
File_name1
Replay requests : 5
Replay responsee : 5
Replay completed : 5
--------------------
File_name2
Replay requests : 3
Replay responsee : 3
Replay completed : 3
--------------------
The script.
#!/usr/bin/env bash
while mapfile -tn4 array && ((${#array[*]})); do
name="${array[0]}"
contents=("${array[#]:1}")
contents=("${contents[#]##* }")
for n in "${contents[#]:1}"; do
(( contents[0] != n )) &&
printf '%s\n' "$name" &&
break
done
done < <(grep -Ev '^-+$' file.txt)
It will not print anything (filename) but change just one value of count (assuming count is the last string per line which is a number) then it should print the filename.
Note that mapfile aka readarray is a bash4+ feature.
The script above assumes that there are 4 lines in between the dashes that separates the Filenames.
and how to check when there are many files_names at the logfile.
Not sure what that means. Clarify the question.
Here is a stating point for a script; I have not well understood the whole question and don't know what exact output is expected.
#! /bin/bash
declare -A dict
while read -a line ; do
test "${line[0]}" == "Replay" || continue
rep="${line[1]}"
num="${line[3]}"
if test "${dict[$rep]}" == "" ; then
dict[$rep]=$num
elif test "${dict[$rep]}" != "$num" ; then
echo "Value changed for $rep : ${dict[$rep]} -> $num"
fi
done < "logfile.txt"
If for instance the input is
File_name1
Replay requests : 5
Replay responsee : 3
Replay completed : 7
--------------------
File_name2
Replay requests : 2
Replay responsee : 3
Replay completed : 6
--------------------
the output will be :
Value changed for requests : 5 -> 2
Value changed for completed : 7 -> 6
Is it helpful?
Sample Input: (tab separated values in table format)
Vserver Volume Aggregate State Type Size Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
vs1 vol1 aggr1 online RW 2GB 1.9GB 5%
vs1 vol1_dr aggr0_dp online DP 200GB 160.0GB 20%
vs1 vol2 aggr0 online RW 150GB 110.3GB 26%
vs1 vol2_dr aggr0_dp online DP 150GB 110.3GB 26%
vs1 vol3 aggr1 online RW 150GB 120.0GB 20%
I've a task to find the volumes under an aggregate which has breached threshold so that they can be moved to a different aggregate.
Need your help to read the above table line by line, capture volume associated with a specific aggregate name (which will passed as an argument) and add the size of the volume to variable (say total). The next lines should be read till the variable, total is less than or equal to the size that should be moved (again which will passed as an argument)
Expected output if <aggr1> and <152GB> are passed as arguments
vol1 aggr1 2GB
vol3 aggr1 150GB
You want to read the file line by line, so you can use awk. You give arguments with the syntax -v aggr=<aggr>. You will enter that on command line:
awk -f script.awk -v aggr=aggr1 -v total=152 tabfile
here is an awk script:
BEGIN {
if ( (aggr == "") || (total == 0.) ) {
print "no <aggr> or no <total> arg\n"
print "usage: awk -f script.awk -v aggr=<aggr> -v total=<total> <file_data>"
exit 1;}
sum = 0;
}
$0 ~ aggr {
scurrent = $6; sub("GB","", scurrent);
sum += scurrent;
if (sum <= total) print $2 "\t" $3 "\t" $6;
else exit 0;
}
The BEGIN block is interpreted once, at the beginning! Here you initialize sum variable and you check the presence of mandatory arguments. If they are missing, their value is null.
The script will read the file line by line, and will process only lines containing aggr argument.
Each column is referred thanks to $ and its NUM; your volume size is in the column $6.
I keep getting this error when i execute the bash script (**./partion.sh: line 11: $'n\np\n1\n\nw\n': command not found
**):
./partion.sh: line 11: $'n\np\n1\n\nw\n': command not found
Checking that no-one is using this disk right now ... OK
Disk /dev/sdd: 3 GiB, 3221225472 bytes, 6291456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0ca4ca9d
Old situation:
>>>
New situation:
Here is the script:
#!/bin/bash
hdd="/dev/sdd"
for i in $hdd;do
echo
"n
p
1
w
"|sfdisk $i;done
I am still a newbie so really appreciate all the help i can get=)
I think it should be:
#!/bin/bash
hdd="/dev/sdd"
for i in $hdd;do
echo "n
p
1
w
"|sfdisk $i;done
Multi-line echo should start on the same line. Else, next line is treated as a command on its own.
You have an extra newline between the command echo and the string that you are telling the shell to echo, which is what is causing your error message.
But you are also sending interactive commands to sfdisk, which is not an interactive tool. Your code appears to be based on the top of this article, which #GregTarsa linked in his comment, but that is sending those commands to fdisk, not sfdisk. (You are also missing another newline between the l and w fdisk commands.)
The sfdisk program is designed to take a text description of the desired partitioning layout and apply it to a disk. It doesn't expect single-letter commands on input; it expects its input to look like this:
# partition table of /dev/sda
unit: sectors
/dev/sda1 : start= 2048, size= 497664, Id=83, bootable
/dev/sda2 : start= 501758, size=1953021954, Id= 5
/dev/sda3 : start= 0, size= 0, Id= 0
/dev/sda4 : start= 0, size= 0, Id= 0
/dev/sda5 : start= 501760, size=1953021952, Id=8e
Which is less generic than the fdisk commands, since it requires actual block numbers, but much easier to use (and for someone reading your script to understand), since you're not trying to remote-control an interactive program by echoing commands to it.
If you are going to remote-control an interactive program from the shell, I recommend looking into expect instead of doing blind echos.
Also, in general, if you are trying to print multiline data, echo is probably not the way to go. With lines this short, I would reach for printf:
printf '%s\n' n p l "" "" w | fdisk "$i"
But the more generally useful tool is the here-document:
fdisk "$i" <<EOF
n
p
l
w
EOF
A newline terminates a command. If you want to pass a multiline argument to echo, you need to move your quote. For example:
for i in $hdd; do
echo "n
p
1
w
" | sfdisk $i
done
It would probably look better to write:
for i in $hdd; do printf 'n\np\n1\n\nw\n' | sfdisk $i; done
A third option is to use a heredoc:
for i in $hdd; do
sfdisk $i << EOF
n
p
1
w
EOF
done
I always result in bites (4521564 b)
if the result is > 1 YB - print result in YB
if the result is > 1 ZB and < 1 YB - print result in ZB
if the result is > 1 EB and < 1 ZB - print result in EB
if the result is > 1 PB and < 1 EB - print result in PB
if the result is > 1 TB and < 1 PB - print result in TB
if the result is > 1 GB and < 1 TB - print result in GB
if the result is > 1 MB and < 1 GB - print result in MB
if the result is > 1 KB and < 1 MB - print result in KB
Do not know about the bash for which it can be calculated?
Using awk:
f.awk contents:
$ cat f.awk
function calc(num,i)
{
if (num>=1024)
calc(num/1024,i+1);
else
printf "%.2f %s\n", num,a[i+1];
}
BEGIN{
split("b KB MB GB TB PB EB ZB YB",a);
calc(val,0)
}
Run the above awk program like this:
$ awk -v val=4521564 -f f.awk
4.31 MB
The logic used here is to keep diving the number by 1024 till the original number becomes less than 1024. And during every division increment a count. The count is finally mapped to the units to get the appropriate unit. The function calc is called recursively.
Eg: I/p: 1000bytes: In this case, since no. is less than 1024, no division is done and the counter is 0. The counter maps to bytes index.
I/p : 2050 : Divided by 1024 and count is incremented by 1. After division, since the no. is less than 1024, it is printed with the unit pointed by the counter, which in this case is Kb.
shell doesn't do floating point without a helper so this shell function will round
byteme(){
v=$1
i=0
s=" KMGTPEZY"
while [ $v -gt 1024 ]; do
i=$((i+1))
v=$((v/1024))
done
echo $v${s:$i:1}b
}
byteme 1234567890
1Gb
I have a text file with the following contents:
QAM Mode : QAM-16
QAM Annex : Annex A
Frequency : 0 Hz
IF Frequency : 0 Hz
Fast Acquisition : 0
Receiver Mode : cable
QAM Lock : 1
FEC Lock : 1
Output PLL Lock : 0
Spectrum Inverted : 0
Symbol Rate : -1
Symbol Rate Error : 0
IF AGC Level (in units of 1/10 percent) : 260
Tuner AGC Level (in units of 1/10 percent) : 1000
Internal AGC Level (in units of 1/10 percent) : 0
SNR Estimate (in 1/100 dB) : 2260
**FEC Corrected Block Count (Since last tune or reset) : 36472114
FEC Uncorrected Block Count (Since last tune or reset) : 0
FEC Clean Block Count (Since last tune or reset) : 0**
Cumulative Reacquisition Count : 0
Uncorrected Error Bits Output From Viterbi (Since last tune or reset) : 0
Total Number Of Bits Output from Viterbi (Since last tune or reset) : 0
viterbi bit error rate (in 1/2147483648 th units) : 0
Carrier Frequency Offset (in 1/1000 Hz) : -2668000
Carrier Phase Offset (in 1/1000 Hz) : 0
**Good Block Count (Reset on read) : -91366870**
**BER Raw Count (Reset on read) : 0**
DS Channel Power (in 10's of dBmV units ) : -760
Channel Main Tap Coefficient : 11846
Channel Equalizer Gain Value in dBm : 9
**Post Rs BER : 2147483648
Post Rs BER Elapsed Time (in Seconds) : 0**
Interleave Depth : 1
I need to parse the numbers from the bolded lines using a bash script but I haven't been able to do this with the command set I have available. This is my first time every using BASH scripts and the searches I've found that could help used some grep, sed, and cut options that weren't available. The options I have are listed below:
grep
Usage: grep [-ihHnqvs] PATTERN [FILEs...]
Search for PATTERN in each FILE or standard input.
Options:
-H prefix output lines with filename where match was found
-h suppress the prefixing filename on output
-i ignore case distinctions
-l list names of files that match
-n print line number with output lines
-q be quiet. Returns 0 if result was found, 1 otherwise
-v select non-matching lines
-s suppress file open/read error messages
sed
BusyBox v1.00-rc3 (00:00) multi-call binary
Usage: sed [-efinr] pattern [files...]
Options:
-e script add the script to the commands to be executed
-f scriptfile add script-file contents to the
commands to be executed
-i edit files in-place
-n suppress automatic printing of pattern space
-r use extended regular expression syntax
If no -e or -f is given, the first non-option argument is taken as the sed
script to interpret. All remaining arguments are names of input files; if no
input files are specified, then the standard input is read. Source files
will not be modified unless -i option is given.
awk
BusyBox v1.00-rc3 (00:00) multi-call binary
Usage: awk [OPTION]... [program-text] [FILE ...]
Options:
-v var=val assign value 'val' to variable 'var'
-F sep use 'sep' as field separator
-f progname read program source from file 'progname'
Can someone please help me with this? Thanks!
AWK can do that for you:
awk '/^(FEC.*Block|Good Block|BER|Post)/{print $NF}' textfile
grep -e "^FEC " -e "^Good Block" -e "BER" file.txt | awk '{print $NF}'
grep: Match lines that: start with FEC or start with Good Block or contains BER
awk: Print the last space-separated field in each line
If you have the right grep, you can do this with grep alone, using a regex look-ahead:
$ /bin/grep -Po "(?<=Post Rs BER : )(.+)" data.txt
2147483648
$
I got the inspiration for this here
In addition, you can do this with a pure bash one-liner, no awk, sed, grep, or other helpers:
$ { while read line; do if [[ $line =~ "Post Rs BER : (.*)$" ]]; then echo ${BASH_REMATCH[1]}; fi; done; } < data.txt
2147483648
$
or
$ cat data.txt | { while read line; do if [[ $line =~ "Post Rs BER : (.*)$" ]]; then echo ${BASH_REMATCH[1]}; fi; done; }
2147483648
$