Multi line invert grep match using file - bash

I am trying to visualise all new sockets created after a save point in Debian Linux 5.14:
ss -a > state
ss -a | grep -v -f state
Expected output : Nothing
Observed output : The same as ss -a
I checked the content of the file and every line does properly end with a $ indicating it's multine.
Can't truely grasp why this happens, did anyone encounter this before ?

Use -F so the lines are treated as fixed strings and not regexes. This ensures that items like * and [::ffff:127.0.0.1] are not treated as wildcards and character classes.
For good measure, use -x to match whole lines.
$ ss -a > state
$ wc -l < state
1867
$ ss -a | grep -vxFf state | wc -l
56

Related

How To Save Every Line Of A Text File Into Different Variables

i want to make a script which will deauth automatically all mac address which has been saved before in a text file.
So my text file looks like this:
C8:xx:xx:AC:xx:xx
3A:xx:Xx:12:xx:xx
upto 20 lines.
I want each mac address either in a single text file or storing in a variable.
To run for example
aireplay-ng --deauth 10 -b $mac1
Thanks
Use xargs:
<input-file xargs -L 1 aireplay-ng --deauth 10 -b
To filter only valid mac addresses, use grep in the pipeline:
<input-file grep -o -E '(([[:xdigit:]]){2}:){5}([[:xdigit:]]){2}' |
xargs -L 1 aireplay-ng --deauth 10 -b
You can do this with a simple while loop, replace input.file with the file name.
while read -r line; do aireplay-ng --deauth 10 -b $line; done < input.file
One trick you could do to make sure that you only get MAC addresses (assuming no other lines have colons and lines with MAC addresses don't have other content):
while read -r line; do aireplay-ng --deauth 10 -b $line; done < cat input.file | grep :

bash how to get var from file with multiple lines

I have two files, the first contains a series of names :
CONTAINER1
CONTAINER2
CONTAINER3
CONTAINER4
And the second script runs when the sintaxis is ./script.sh CONTAINER1
CONT=$1
PID=`opmnctl status -noheaders -fmt %prt30%pid7R | grep -w $CONT | awk '{print $3}'`
netstat -anp | grep $PID | grep ESTABLISHED > $CONT-temp
Is it possible to run the second script by parsing each line of the first file?. I'm new at bash, any help would be appreciated.
Thanks in advance
You can use xargs for this: it reads words from its stdin and appends them to the given command. Normally it takes several parameters at a time so it invokes the given command as few times as possible, but you can control that with the -L option:
xargs -L 1 echo "name is:" < containers.txt
produces
name is: CONTAINER1
name is: CONTAINER2
name is: CONTAINER3
name is: CONTAINER4
You would write
xargs -L 1 ./script.sh < containers.txt

Use output of bash command (with pipe) as a parameter for another command

I'm looking for a way to use the ouput of a command (say command1) as an argument for another command (say command2).
I encountered this problem when trying to grep the output of who command but using a pattern given by another set of command (actually tty piped to sed).
Context:
If tty displays:
/dev/pts/5
And who displays:
root pts/4 2012-01-15 16:01 (xxxx)
root pts/5 2012-02-25 10:02 (yyyy)
root pts/2 2012-03-09 12:03 (zzzz)
Goal:
I want only the line(s) regarding "pts/5"
So I piped tty to sed as follows:
$ tty | sed 's/\/dev\///'
pts/5
Test:
The attempted following command doesn't work:
$ who | grep $(echo $(tty) | sed 's/\/dev\///')"
Possible solution:
I've found out that the following works just fine:
$ eval "who | grep $(echo $(tty) | sed 's/\/dev\///')"
But I'm sure the use of eval could be avoided.
As a final side node: I've noticed that the "-m" argument to who gives me exactly what I want (get only the line of who that is linked to current user). But I'm still curious on how I could make this combination of pipes and command nesting to work...
One usually uses xargs to make the output of one command an option to another command. For example:
$ cat command1
#!/bin/sh
echo "one"
echo "two"
echo "three"
$ cat command2
#!/bin/sh
printf '1 = %s\n' "$1"
$ ./command1 | xargs -n 1 ./command2
1 = one
1 = two
1 = three
$
But ... while that was your question, it's not what you really want to know.
If you don't mind storing your tty in a variable, you can use bash variable mangling to do your substitution:
$ tty=`tty`; who | grep -w "${tty#/dev/}"
ghoti pts/198 Mar 8 17:01 (:0.0)
(You want the -w because if you're on pts/6 you shouldn't see pts/60's logins.)
You're limited to doing this in a variable, because if you try to put the tty command into a pipe, it thinks that it's not running associated with a terminal anymore.
$ true | echo `tty | sed 's:/dev/::'`
not a tty
$
Note that nothing in this answer so far is specific to bash. Since you're using bash, another way around this problem is to use process substitution. For example, while this does not work:
$ who | grep "$(tty | sed 's:/dev/::')"
This does:
$ grep $(tty | sed 's:/dev/::') < <(who)
You can do this without resorting to sed with the help of Bash variable mangling, although as #ruakh points out this won't work in the single line version (without the semicolon separating the commands). I'm leaving this first approach up because I think it's interesting that it doesn't work in a single line:
TTY=$(tty); who | grep "${TTY#/dev/}"
This first puts the output of tty into a variable, then erases the leading /dev/ on grep's use of it. But without the semicolon TTY is not in the environment by the moment bash does the variable expansion/mangling for grep.
Here's a version that does work because it spawns a subshell with the already modified environment (that has TTY):
TTY=$(tty) WHOLINE=$(who | grep "${TTY#/dev/}")
The result is left in $WHOLINE.
#Eduardo's answer is correct (and as I was writing this, a couple of other good answers have appeared), but I'd like to explain why the original command is failing. As usual, set -x is very useful to see what's actually happening:
$ set -x
$ who | grep $(echo $(tty) | sed 's/\/dev\///')
+ who
++ sed 's/\/dev\///'
+++ tty
++ echo not a tty
+ grep not a tty
grep: a: No such file or directory
grep: tty: No such file or directory
It's not completely explicit in the above, but what's happening is that tty is outputting "not a tty". This is because it's part of the pipeline being fed the output of who, so its stdin is indeed not a tty. This is the real reason everyone else's answers work: they get tty out of the pipeline, so it can see your actual terminal.
BTW, your proposed command is basically correct (except for the pipeline issue), but unnecessarily complex. Don't use echo $(tty), it's essentially the same as just tty.
You can do it like this:
tid=$(tty | sed 's#/dev/##') && who | grep "$tid"

Redirect the output to two processes and merge

Background: sometimes I use ps aux | grep process_name to figure out some statistics of a process. But I can't remember the headers of ps output. So I need to do:
ps aux | tee 1.txt | grep process_name > 2.txt
cat 1.txt | head -1 | cat - 2.txt
So my questions is: is there a way to achieve this without the two temporary files, and preferably use one line of commands instead of two lines?
The simplest way is just to use something a bit smarter than grep. For example:
ps aux | perl -ne 'print if $. == 1 || m/process_name/'
will print the first line of ps aux's output, plus any line that matches process_name (which is a Perl regular expression — more powerful than POSIX BRE's, and therefore more complicated, but I don't think you'll find any surprises if you're just searching for a process name).
Edited to add: For that matter, since ps aux's headers are unlikely to change between runs, you could write:
ps aux | head -1 ; ps aux | grep process_name
though I don't know if that still counts as "one line of commands". :-)
More options: Since you don't like the above, here are some more. This one will read and print the first line directly, and then leave the rest for grep to read and process:
ps aux | ( ( read -r LINE ; echo "$LINE" ) ; grep process_name )
This one will cause every line to be written both to standard output (file descriptor 1) and to a custom file descriptor (3); then head gets standard output, while grep gets what was written to the custom file descriptor:
( ( ps aux | while read -r LINE ; do echo "$LINE" ; echo "$LINE" 1>&3 ; done | head -1 1>&4 ) 3>&1 | grep process_name ) 4>&1
That one is the most general way that I know of to do what you're asking for — at least, using only Bash builtins and features — but as you can see, it's quite unwieldy, which is why I recommend the simpler ways when you don't need the full power of this approach. Also, that last one no longer guarantees that headers come first; on my system, the output of head seems to consistently end up after the output of grep. (I suppose that could be addressed by swapping the positions of the head and grep commands, but that still wouldn't be reliable so far as I know, and it seemed to have weird effects when I tried it just now.)
Or, since I never remember perl or awk without looking it up:
ps aux | head -n 1 && ps aux |grep process_name
Another dirty trick is to memorize that the header starts with 'USER' and do
ps aux | grep -E 'USER | process_name'
This will also show the grep command itself :)
Or
ps aux | egrep "(^USER +PID)|^process_name"
sed will do exactly what you want:
ps aux | sed -n -e '1 p; /PROCESS_NAME/p'
The sed command line:
-n print only the specified lines
-e execute the following script
1 p print the first line
; command separator
/PROCESS_NAME/p print lines containing PROCESS_NAME
use the name(s) of processes with -C
ps u -C java -C totem
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
stefan 16878 0.1 10.5 429036 107868 pts/7 Sl+ 12:51 0:24 /opt/java/bin/java -Xmx256M -Xms32M -Xbootclasspa
stefan 25791 0.5 3.8 202800 38984 ? Sl 17:45 0:16 totem file:///home/stefan/Desktop/scala/dritte/dl
I hope this doesn't only work on Linux. :)
I'm always baffled to see, how many people don't know (or use) the options of ps - maybe there are too many? It's always ps | grep x. :)

Limit the output of the TOP command to a specific process name

If you call the top command, you get all the running processes. But how can I limit the output only to a certain process name like "java"?
I've tried this
top -l 2 | grep java
but in this way you get only snapshots and not a continuously updated list. And top -l 0 | grep java is not really clear.
I prefer the following so I can still use top interactively without having to look up the pids each time I run it:
top -p `pgrep process-name | tr "\\n" "," | sed 's/,$//'`
Of course if the processes change you'll have to re-run the command.
Explanation:
pgrep process-name returns a list of process ids which are separated by newlines
tr "\\n" "," translates these newlines into commas, because top wants a comma-separated list of process ids
sed is a stream editor, and sed 's/,$//' is used here to remove the trailing comma
Find the pids of the processes you want to monitor and then use the -p option which allows you to provide a list of pids to the top command.
Example:
top -p 18884 -p 18892 -p 18919
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
18884 user 25 0 672M 95M 9476 S 0.0 1.1 0:02 1 java
18892 user 25 0 2280M 123M 12252 S 0.0 1.5 0:05 1 java
18919 user 22 0 1492M 198M 28708 S 0.0 2.4 0:07 1 java
(I believe you can also pass in a comma-separated list.)
how about top -b | grep java
Expanding on #dogbane's answer, you can get all the PIDs for a named process with pgrep to do the following:
top -p "$(pgrep -d ',' java)"
Use the watch command
watch -d 'top -n1 | grep mysql'
Using the answer from here I was able to create a one liner
top -pid $(pgrep process_name | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/ -pid /g')
This works for me on MacOS 10.12 (Sierra)
I solved my problem using:
top -n1 -b | grep "proccess name"
in this case:
-n is used to set how many times top will what proccess
and -b is used to show all pids
it's prevents errors like :
top: pid limit (20) exceeded
The following code updates a list of processes every 5 seconds via the watch command:
watch -n 5 -t top -b -n 1 -p$(pgrep java | head -20 | tr "\\n" "," | sed 's/,$//')
I run it (eg.): top -b | egrep -w 'java|mysqld'
Suppose .. if we have more than 20 process running on the server with the same name ... this will not help
top -p pgrep oracle | head -n 20 | tr "\\n" "," | sed 's/,$//'
It will try to list and provide real time output of 20 process where we have good chance of missing other prcesses which consumes more resource ....
I am still looking for better option on this
A more specific case, like I actually was looking for:
For Java processes you can also use jps -q whereby jps is a tool from $JAVA_HOME/bin and hence should be in your $PATH.
I came here looking for the answer to this on OSX. I ended up getting what I wanted with bash and awk:
topfiltered() {
[[ -z "$1" ]] && return
dump="/tmp/top_dump"
rm -f "$dump"
while :; do
clear
[[ -s "$dump" ]] && head -n $(( $LINES - 1 )) "$dump"
top -l 1 -o cpu -ncols $(( $COLUMNS / 8 )) | awk -v p="$(pgrep -d ' ' $#)" '
BEGIN { split(p, arr); for (k in arr) pids[arr[k]]=1 }
NR<=12 || ($1 in pids)
' >"$dump"
done
}
I loop top in logging mode and filter it with awk, building an associative array from the output of pgrep. Awk prints the first 12 lines, where line 12 is the column headers, and then every line which has a pid that's a key in the array. The dump file is used for a more watchable loop.
just top -bn 1 | grep java will do the trick for you
Running the below will give continuous update in console:
bcsmc2rtese001 [~]$ echo $SHELL
/bin/bash
bcsmc2rtese001 [~]$ top | grep efare or watch -d 'top | grep efare' or top -p pid
27728 efare 15 0 75184 3180 1124 S 0.3 0.0 728:28.93 tclsh
27728 efare 15 0 75184 3180 1124 S 0.7 0.0 728:28.95 tclsh
Here's the only solution so far for MacOS:
top -pid `pgrep java | awk 'ORS=" -pid "' | sed 's/.\{6\}$//'`
though this will undesirably report invalid option or syntax: -pid if there are no java processes alive.
EXPLANATION
The other solutions posted here use the format top -p id1,id2,id3, but MacOS' top only supports the unwieldy format top -pid id1 -pid id2 -pid id3.
So firstly, we obtain the list of process ids which have process name "java":
pgrep java
and we pipe this to awk which joins the results with delimitor " -pid "
| awk 'ORS=" -pid "'
Alas, this leaves a trailing delimitor! For example, we may so far have obtained the string "123 -pid 456 -pid 789 -pid ".
We then just use sed to shave off the final 6 characters of the delimitor.
| sed 's/.\{6\}$//'`
We're ready to pass the results to top:
top -pid `...`
get pid of process:
# pidof <process>
tell top what process pid(s) to display
# top -p <pid1>,<pid2>, etc
example:
landis#linux-e6510:~>pidof konsole
1841 1709
landis#linux-e6510:~>top -p 1841,1709
Top will only display the 2 'konsole' processes.
This is useful on a busy server via ssh, not having to pipe thru grep.
You need to feed the output of pgrep {proc_name} to top -pid {pid_No}. The problem with top -pid is that it expects -pid before each pid you want to monitor.
On Mac in zsh I can handle this problem e.g. like that:
top `pgrep proc_name | awk '{printf " -pid %d",$1}'`
It's not ideal too, because pgrep looks for substrings in process names. E.g. pgrep dd can return results for icdd, hidd, cloudd etc. The option
pgrep -x should return the exact matches only (like grep -w). But it doesn't work for me in Mac Terminal, although it does in Ubuntu virtual machine.
Using the approach mentioned in the answer by Rick Byers:
top -p `pgrep java | paste -sd "," -`
but I had more than 20 processes running so following command can be helpful for someone who encounter a similar situation.
top -p `pgrep java | head -n 20 | paste -sd "," -`
pgrep gets the list of processes with given name - java in this case. head is used to get first 20 pids because top cannot handle more than 20 pids when using -p argument. Finally paste joins the list of pids with ','.
You can control the process name you are looking for in the above command and the number of processes with that name you are interested to watch. You can ignore the head -n 20 part if the number of your processes with the given name is less than 20.

Resources