I am trying to execute the one liner below as a part of bash script.
command1 |grep -ID|grep -v + | awk '{print "command2" $2}'|bash
The first part of the pipe prints the info below:
root#system:~# command1 |grep -v ID|grep -v +
| id | name | mac_address | fixed_ips |
| 00277225-34fa-48f5-9a2a-ee5f1c5b1dcb | dummy | fa:18:3e:c4:85:94 | {"subnet_id": "0cd4d824-4420-4049-87c3-ed33c3addbf5", "ip_address": "11.170.1.121"} |
:
:
| ff9a6ed5-9694-45bc-bf71-59565f96d809 | BAT-T0-A2-0-7-tport | fa:18:3e:62:70:fb | {"subnet_id": "f9ae81ed-3b1a-45a7-96fd-c417ed32
So, $2 in awk command2 is "00277225-34fa-48f5-9a2a-ee5f1c5b1dcb".
e.g.
command2 00277225-34fa-48f5-9a2a-ee5f1c5b1dcb
The whole purpose of this one-liner is to execute a number of "command2" instances with different parameter values from the printout above.
e.g.
command2 00277225-34fa-48f5-9a2a-ee5f1c5b1dcb
command2 ff9a6ed5-9694-45bc-bf71-59565f96d809
:
:
But I can not make the $2 recognized the way below
command1 |grep -ID|grep -v + | awk '{print "command2" $2}'|bash
I think I am missing few syntax tricks here (as newbie).
p.s: If I copy / paste the whole line in command line, it works fine.
You seem to be looking for
command1 | awk '!/ID/ && !/\+/ {print $2}' | xargs -n 1 command2
I refactored the ugly useless greps into the Awk script; but the real beef here is xargs. It reads parameters from standard input and passes them on to the command you supply in the positional parameters.
The option -n 1 says to only accept one additional argument at a time; but if command2 is a well-written standard Unix command, it can probably accept an arbitrary number of arguments, and will simply loop over them. In that case, removing -n 1 will be a lot more efficient.
Incidentally, your original attempt was fairly close; you should have added a space after command2 in the print statement. But I hope this solution will also help you see how to "think Unix".
Related
So, I have text file with a a bunch of numbers, one number per line to be specific, so I do:-
cat filename.txt|sort -n|head -1 to get the top number and I can do cat filename.txt|sort -n|tail -1 to get the bottom number.
Just to be sure is there a way to send cat filename.txt|sort -n| and its output to two different commands in one line and have the out put (the highest number and the lowest number next to each other)
You can do interesting things with tee and process substitutions, but the order of the output may not be stable (due to timing of processes)
sort -n filename.txt | tee >(tail -1 >/dev/tty) | head -1
In this case, I'd use sed to print the first and last line:
sort -n filename.txt | sed -n '1p; $p'
As #chepner suggests
... | sed -n '1p; $p' | paste - - # tab separated
or
... | awk 'NR == 1 {first = $0} END {print first, $0}' # space separated
There is a useful command tee syntax tee second.txt will output to second.txt
You can combine that with bash executive pipes eg tee >(wc),
So you can do 2 or more commands by eg tee >(wc) | head
I am running a command that spits out IPs, I need to feed it to another program at a specific location as it comes, how do I do it?
$ command1 | command2 -c configfile -i "$1" status
"$1" is where I want the result of command1 to go to.
Thanks.
xargs is your tool
$ command1 | xargs -I {} command2 -c configfile -i {} status
you can refer to the argument multiple times, for example
$ echo this | xargs -I {} echo {}, {}, and {}
this, this, and this
based on the last comment, perhaps you want to do something like this
$ var=$(command1) && command2 "$var" ... | command3 "$var" ...
To pass command2 a filename which will, when read, provide output from command1, the appropriate tool is process substitution:
command2 -c configfile -i <(command1) status
The <(...) syntax will be replaced with a filename -- on Linux, of the form /dev/fd/NN; on some other platforms a named pipe instead -- from which the output of command2 can be streamed.
There are probably 100 ways to do this in a bash shell. This one is quick and easy. We can use ifconfig, grep the IP address, use awk to pull it out and assign it to an environment variable. Then use the boolean operator to run the next command which uses the environment variable
IPADDR=`ifconfig | grep 172 | awk '{print $2}' | awk -F: '{print $2}'` && echo $IPADDR
here is what I want to do
while ["ps a | grep '[m]ono' | awk '{print $1}'" != ""] ; do
kill ps a | grep '[m]ono' | awk '{print $1}'
done
meaning if the grep returns nothing, don't try a kill.
The thing is I've always been lost with the expression evaluation in bash, sometime I use " " around something, sometime it's eval, sometime it's ''.
Could someone explain to me how to write the condition in my loop and explain the difference between the above? I'm used to find the working one with many tries and it feels like a huge loss of time.
Best choice: Do something else.
Utilities already exist for this purpose. Example:
killall mono
pkill mono
...or, even better, something targeted to the specific executable you want to terminate:
fuser -k /path/to/something.exe
...which would kill only programs with a file handle on that specific executable, rather than all applications running with mono on the machine.
...but, to explain the bugs:
There are two things wrong here: Missing command substitutions, and missing whitespace.
Missing whitespace:
["ps a | grep '[m]ono' | awk '{print $1}'" != ""]
...is literally trying to run a command with a name starting with [ps, as in, looking in the PATH for...
/bin/[ps\ a
/usr/bin/[ps\ a
...etc. [ is a command, and needs a space after its name like any other command. Thus:
[ "ps a | grep '[m]ono' | awk '{print $1}'" != "" ]
...fixes this problem (but leaves another one).
Missing command substitutions:
[ "ps a | grep '[m]ono' | awk '{print $1}'" != "" ]
...is comparing a string that starts with "ps a" to to ""; it does not compare the output of running a command that starts with ps a. To do that, you'd instead run:
[ "$(ps a | grep '[m]ono' | awk '{print $1}')" != "" ]
The content of $(...) is replaced with the output of the command within; thus, running your pipeline and comparing its output to an empty string.
I'm trying to create a shell script getting the process id of the Skype app on my Mac.
ps -clx | grep 'Skype' | awk '{print $2}' | head -1
The above is working fine, but there are two problems:
1)
The grep command would get all process if their name just contains "Skype". How can I ensure that it only get the result, if the process name is exactly Skype?
2)
I would like to make a shell script from this, which can be used from the terminal but the process name should be an argument of this script:
#!/bin/sh
ps -clx | grep '$1' | awk '{print $2}' | head -1
This isn't returning anything. I think this is because the $2 in the awk is treated as an argument too. How can I solve this?
Your ps -cl1 output looks like this:
UID PID PPID F CPU PRI NI SZ RSS WCHAN S ADDR TTY TIME CMD
501 185 172 104 0 31 0 2453272 1728 - S ffffff80145c5ec0 ?? 0:00.00 httpd
501 303 1 80004004 0 31 0 2456440 1656 - Ss ffffff8015131300 ?? 0:11.78 launchd
501 307 303 4004 0 33 0 2453456 7640 - S ffffff8015130a80 ?? 0:46.17 distnoted
501 323 303 40004004 0 33 0 2480640 9156 - S ffffff80145c4dc0 ?? 0:03.29 UserEventAgent
Thus, the last entry in each line is your command. That means you can use the full power of regular expressions to help you.
The $ in a regular expression means the end of the string, thus, you could use $ to specify that not only does the output must have Skype in it, it must end with Skype. This means if you have a command called Skype Controller, you won't pull it up:
ps -clx | grep 'Skype$' | awk '{print $2}' | head -1
You can also simplify things by using the ps -o format to just pull up the columns you want:
ps -eo pid,comm | grep 'Skype$' | awk '{print $1}' | head -1
And, you can eliminate head by simply using awk's ability to select your line for you. In awk, NR is your record number. Thus you could do this:
ps -eo pid,comm | grep 'Skype$' | awk 'NR == 1 {print $1}'
Heck, now that I think of it, we could eliminate the grep too:
ps -eo pid,comm | awk '/Skype$/ {print $1; exit}'
This is using awk's ability to use regular expressions. If the line contains the regular expression, 'Skype$', it will print the first column, then exit
The only problem is that if you had a command Foo Skype, this will also pick it up. To eliminate that, you'll have to do a bit more fancy footwork:
ps -eo pid,comm | while read pid command
do
if [[ "$command" = "Skype" ]]
then
echo $pid
break
fi
done
The while read is reading two variables. The trick is that read uses white space to divide the variables it reads in. However, since there are only two variables, the last one will contain the rest of the entire line. Thus if the command is Skype Controller, the entire command will be put into $command even though there's a space in it.
Now, we don't have to use a regular expression. We can compare the command with an equality.
This is longer to type in, but you're actually using fewer commands and less piping. Remember awk is looping through each line. All you're doing here is making it more explicit. In the end, this is actually much more efficient that what you originally had.
If pgrep is available on Mac, you can use pgrep '^Skype$'. This will list the process id of all processes called Skype.
You used the wrong quotes in your script:
ps -clx | grep "$1" | awk '{print $2}' | head -1
or
pgrep "^$1$"
The problem with your second example is that the $1 is in single quotes, which prevents bash from expanding the variable. There is already a utility that accomplishes what you want without manually parsing ps output.
pgrep "$1"
You can do this in AppleScript:
tell application "System Events"
set skypeProcess to the process "Skype"
set pid to the unix id of skypeProcess
pid
end tell
which means you can use 'osascript' to get the PID from within a shell script:
$ osascript -e "tell application \"System Events\"" -e "set skypeProcess to the process \"Skype\"" -e "set pid to the unix id of skypeProcess" -e "pid" -e "end tell"
3873
You can format the output of ps using the -o [field],... and list by process name using -C [command_name] ;however, ps will still print the column header, which can be removed by piping it through grep -v PID
ps -o pid -C "$1" |grep -v PID
where $1 would be the command name (in this case Skype)
I'd so something like:
ps aux | grep Skype | awk 'NR==1 {print $2}'
==== UPDATE ====
Use the parameter without quotes and use single quotes for awk
#!/bin/bash
ps aux | grep $1 | awk 'NR==1 {print $2}'
Method 1 - Use awk
I don't see any reason to use the -l flag (long format), I also don't see any reason to use grep and awk at the same time: awk has grep capability built in. Here is my plan: use ps and output just 2 columns: pid and command, then use awk to pick out what you want:
ps -cx -o pid,command | awk '$2 == "Skype" { print $1 }'
Method 2 - Use bash
This method has the advantage that if you already script in bash, you don't even need awk, which save one process. The solution is longer than the other method, but very straight forward.
#!/bin/bash
ps -cx -o pid,command | {
while read pid command
do
if [ "_$command" = "_$1" ]
then
# Do something with the pid
echo Found: pid=$pid, command=$command
break
fi
done
}
pgrep myAwesomeAppName
This works great under Catalina 10.15.2
Use double quotes to allow bash to perform variable substitution.
Single quotes disable bash variable substitution mechanism.
ps -clx | grep "$1" | awk "{print $2}" | head -1
I am using 'tail -f' to follow a log file as it's updated; next I pipe the output of that to grep to show only the lines containing a search term ("org.springframework" in this case); finally I'd like to make is piping the output from grep to a third command, 'cut':
tail -f logfile | grep org.springframework | cut -c 25-
The cut command would remove the first 25 characters of each line for me if it could get the input from grep! (It works as expected if I eliminate 'grep' from the chain.)
I'm using cygwin with bash.
Actual results: When I add the second pipe to connect to the 'cut' command, the result is that it hangs, as if it's waiting for input (in case you were wondering).
Assuming GNU grep, add --line-buffered to your command line, eg.
tail -f logfile | grep --line-buffered org.springframework | cut -c 25-
Edit:
I see grep buffering isn't the only problem here, as cut doesn't allow linewise buffering.
you might want to try replacing it with something you can control, such as sed:
tail -f logfile | sed -u -n -e '/org\.springframework/ s/\(.\{0,25\}\).*$/\1/p'
or awk
tail -f logfile | awk '/org\.springframework/ {print substr($0, 0, 25);fflush("")}'
On my system, about 8K was buffered before I got any output. This sequence worked to follow the file immediately:
tail -f logfile | while read line ; do echo "$line"| grep 'org.springframework'|cut -c 25- ; done
What you have should work fine -- that's the whole idea of pipelines. The only problem I see is that, in the version of cut I have (GNU coreutiles 6.10), you should use the syntax cut -c 25- (i.e. use a minus sign instead of a plus sign) to remove the first 24 characters.
You're also searching for different patterns in your two examples, in case that's relevant.