I am trying to execute my script, but the $ 1 argument is concatenated with the arguments of the last pipe, resulting in the following
killProcess(){
ps aux |grep $1 | tr -s " " " " | awk "{printf \"%s \",\$2}" | tr " " "\n" | xargs -l1 echo $1
}
$killProcess node
node 18780
node 965856
node 18801
node 909028
node 19000
node 1407472
node 19028
node 583620
node 837
node 14804
node 841
node 14260
but I just want the list of pids, without the node argument to be able to delete them, that only happens when I put it under a script, in command line it works normally for me because I don't pass any arguments to the script and it doesn't get concatenated.
The immediate problem is that you don't want the $1 at the end. In that context, $1 expands to the first argument to the function ("node", in your example), which then gets passed to xargs and treated as part of the command it should execute. That is, the last part of the pipeline expands to:
xargs -l1 echo node
...so when xargs receives "18780" as input, it runs echo node 18780, which of course prints "node 18780".
Solution: remove the $1, making the command just xargs -l1 echo, so when xargs receives "18780" as input, it runs echo 18780, which prints just "18780".
That'll fix it, but there's also a huge amount of simplification that can be done here. Many elements of the pipe aren't doing anything useful, or are working at cross purposes with each other.
Start with the last command in the pipe, xargs. It's taking in PIDs, one per line, and printing them one per line. It's not really doing anything at all (that I can see anyway), so just leave it off. (Unless, of course, you actually want to use kill instead of echo -- in that case, leave it on.)
Now look at the next two commands from the end:
awk "{printf \"%s \",\$2}" | tr " " "\n"`
Here, awk is printing the PIDs with a space after each one, and then tr is turning the spaces into newlines. Why not just have awk print each one with a newline to begin with? You don't even need printf for this, you can just use print since it automatically adds a newline. It's also simpler to pass the script to awk in single-quotes, so you don't have to escape the double-quotes, dollar sign, and (maybe) backslash. So any of these would work:
awk "{printf \"%s\\n\",\$2}"
awk '{printf "%s\n",$2}'
awk '{print $2}'
Naturally, I recommend the last one.
Now, about the command before awk: tr -s " " " ". This "squeezes" runs of spaces into single spaces, but that's not needed since awk treats runs of spaces as (single) field delimiters. So, again, leave that command out.
At this point, we're down to the following pipeline:
ps aux | grep $1 | awk '{print $2}'
There are two more things I'd recommend here. First, you should (almost) always have double-quotes around shell variable, parameter, etc references like $1. So use grep "$1" instead.
But don't do that, because awk is perfectly capable of searching; there's no need for both grep and awk. In fact, awk can be much more precise, searching only a specific field instead of the whole line. The downside is, it is a bit more complex to do, but knowing how to make awk do more complex things is useful. The best way to let awk work with a shell variable or parameter is to use its -v option to create an awk variable with the same value, and use that. You can then use the ~ to check for a regex match to the variable. Something like this:
awk -v proc="$1" '$11 ~ proc {print $2}'
Note: I'm assuming you want to search for $1 in the executable name, and that that's the 11th field of ps aux on your system. Searching that field only will keep it from matching in e.g. the username (killing all of a user's processes because their name contains some program name isn't polite). You might actually want to be even more specific, so that e.g. trying to kill node doesn't accidentally kill nodemon as well; that'll be a matter of using more specific search patterns.
So, here's the final result:
killProcess(){
ps aux | awk -v proc="$1" '$11 ~ proc {print $2}'
}
To actually kill the processes, add back xargs -l1 kill at the end.
Related
I run the command
df -gP /data1 /data2 | grep -v File | awk '{print $1}' |
awk -F/dev/ '$0=$2' | tr '\n' '
on the AIX shell (ksh) and it prints the output below:
lv_data01 lv_data02 root#testhost:/
However, I would like the output to be printed this way. Could someone help?
lv_data01 lv_data02
Using grep … | awk … | awk … is not necessary; a single awk could do the whole job. So could sed and it might even be easier. I'd be tempted to deal with the spacing by using:
x=$(df … | sed …); echo $x
The tr command, once corrected, replaces newlines with spaces, so the prompt follows without a newline before it. The ; echo suggestion adds the missing newline; the echo $x suggestion (note no double quotes) does too.
As for the sed command:
sed -n '/File/!{ s/[[:space:]].*//; s%^.*/dev/%%p; }'
Don't print anything by default
If the line doesn't match File (doing the work of grep -v):
remove the first space (blank or tab) and everything after it (doing the work of awk '{print $1}')
replace everything up to /dev/ with nothing and print (doing the work of awk -F/dev/ '{$0=$2}')
The command substitution and capture, followed by echo, deals with spaces and newlines.
So, my suggested solution is:
x=$(df -gP /data1 /data2 | sed -n '/File/!{ s/[[:space:]].*//; s%^.*/dev/%%p; }'); echo $x
You could add unset x after the echo if you are going to be using this directly in the shell and not in a shell script. If it'll be encapsulated in a shell script, you don't have to worry about it.
I'm blithely assuming the output from df -gP won't contain a path such as this, with two occurrences of /dev:
/who/knows/dev/lv_data01/dev/bin
If that's a real problem, you can fix the sed script, but I don't think it will be. It's one thing the second awk script in the question handles differently.
I need to use awk to see what users are logged in the computer, create a file with their names and inside that file print the pid of the process they're running. I've used this, but it does not work:
who | awk '{for(i = 0; i < NR; i++)
system("ps -u " $1 "| tail +2 | awk '{print $1}' >" $1".log")
}'
Is there any way to do this?
Thanks a lot!
To achieve your goal of using awk to create those files, I would start with ps rather than with who. That way, ps does more of the work so that awk can do less. Here is an example that might work for you. (No guarantees, obviously!)
ps aux | awk 'NR>1 {system("echo " $2 " >> " $1 ".txt")}'
Discussion:
The command ps aux prints a table describing each active process, one line at a time. The first column of each line contains the name of the process's user, the second column its PID. The line also contains lots of other information, which you can play with as you improve your script. That's what you pipe into awk. (All this is true for Linux and the BSDs. In Cygwin, the format is different.)
Inside awk, the pattern NR>1 gets rid of the first line of the output, which contains the table headers. This line is useless for the files you want awk to generate.
For all other lines in the output of ps aux, awk adds the PID of the current process (ie, $2) to the file username.txt, using $1 for username. Because we append with >> rather than overwriting with >, all PIDs run by the user username end up being listed, one line at a time, in the file username.txt.
UPDATE (Alternative for when who is mandatory)
If using who is mandatory, as noted in a comment to the original post, I would use awk to strip needless lines and columns from the output of who and ps.
for user in $(who | awk 'NR>1 {print $1}')
do
ps -u "$user" | awk 'NR>1' > "$user".txt
done
For readers who wonder what the double-quotes around $user are about : Those serve to guard against globbing (if $user contains asterisks (*)) and word splitting (if $user contains whitespace).
I will leave my original answer stand for the benefit of any readers with more freedom to choose the tools for their job.
Is that what you had in mind?
I am having trouble figuring out how to grep the characters between two single quotes .
I have this in a file
version: '8.x-1.0-alpha1'
and I like to have the output like this (the version numbers can be various):
8.x-1.0-alpha1
I wrote the following but it does not work:
cat myfile.txt | grep -e 'version' | sed 's/.*\?'\(.*?\)'.*//g'
Thank you for your help.
Addition:
I used the sed command sed -n "s#version:\s*'\(.*\)'#\1#p"
I also like to remove 8.x- which I edited to sed -n "s#version:\s*'8.x-\(.*\)'#\1#p".
This command only works on linux and it does not work on MAC. How to change this command to make it works on MAC?
sed -n "s#version:\s*'8.x-\(.*\)'#\1#p"
If you just want to have that information from the file, and only that you can quickly do:
awk -F"'" '/version/{print $2}' file
Example:
$ echo "version: '8.x-1.0-alpha1'" | awk -F"'" '/version/{print $2}'
8.x-1.0-alpha1
How does this work?
An awk program is a series of pattern-action pairs, written as:
condition { action }
condition { action }
...
where condition is typically an expression and action a series of commands.
-F "'": Here we tell awk to define the field separator FS to be a <single quote> '. This means the all lines will be split in fields $1, $2, ... ,$NF and between each field there is a '. We can now reference these fields by using $1 for the first field, $2 for the second ... etc and this till $NF where NF is the total number of fields per line.
/version/{print $2}: This is the condition-action pair.
condition: /version/:: The condition reads: If a substring in the current record/line matches the regular expression /version/ then do action. Here, this is simply translated as if the current line contains a substring version
action: {print $2}:: If the previous condition is satisfied, then print the second field. In this case, the second field would be what the OP requests.
There are now several things that can be done.
Improve the condition to be /^version :/ && NF==3 which reads _If the current line starts with the substring version : and the current line has 3 fields then do action
If you only want the first occurance, you can tell the system to exit immediately after the find by updating the action to {print $2; exit}
I'd use GNU grep with pcre regexes:
grep -oP "version: '\\K.*(?=')" file
where we are looking for "version: '" and then the \K directive will forget what it just saw, leaving .*(?=') to match up to the last single quote.
Try something like this: sed -n "s#version:\s*'\(.*\)'#\1#p" myfile.txt. This avoids the redundant cat and grep by finding the "version" line and extracting the contents between the single quotes.
Explanation:
the -n flag tells sed not to print lines automatically. We then use the p command at the end of our sed pattern to explicitly print when we've found the version line.
Search for pattern: version:\s*'\(.*\)'
version:\s* Match "version:" followed by any amount of whitespace
'\(.*\)' Match a single ', then capture everything until the next '
Replace with: \1; This is the first (and only) capture group above, containing contents between single quotes.
When your only want to look at he quotes, you can use cut.
grep -e 'version' myfile.txt | cut -d "'" -f2
grep can almost do this alone:
grep -o "'.*'" file.txt
But this may also print lines you don't want to: it will print all lines with 2 single quotes (') in them. And the output still has the single quotes (') around it:
'8.x-1.0-alpha1'
But sed alone can do it properly:
sed -rn "s/^version: +'([^']+)'.*/\1/p" file.txt
Lets say that I have a command list kittens that returns something in this multi line format in my terminal (in this exact layout):
[ 'fluffy'
'buster'
'bob1' ]
How can I fetch bob1 and assign to a variable for scripting use? Here's my non working try so far.
list kittens | grep "'([^']+)' \]"
I am not overly familiar with grepping on the cli and am running into issues of syntax with quotes and such.
If you know that bob1 will be in the last line, you can capture it like that:
myvar="$(list kittens | tail -n1 | grep -oP "'\K[^']+(?=')")"
This uses tail to find the last line and then grep with a lookahead and a lookbehind in the regular expression to extract the part inside the quotes.
Edit: The above assume that you are using GNU grep (for the -P mode). Here's an alternative with sed:
myvar="$(list kittens | tail -n1 | sed -e "s/^[^']*'//; s/'[^']*$//")"
Could be done by awk alone:
list kittens |awk 'END{gsub(/\047|[[:blank:]]|\]/,"");print $0}'
bob1
Example:
echo "$kit"
[ 'fluffy'
'buster'
'bob1' ]
echo "$kit" |awk 'END{gsub(/\047|[[:blank:]]|\]/,"");print $0}'
bob1
To Assign it to any variable:
var=$(list kittens |awk 'END{gsub(/\047|[[:blank:]]|\]/,"");print $0}'
Explanation:
END{}: End block is used to take data from last line as we are interested only for last line.
gsub: This is awk's inbuilt function for search and replacement tasks. Here white space and double quoted and single quotes are removed. Not that \047 is used for single quote replacement.
I have tried all kinds of filters using grep to try and solve this but just cannot crack it.
cpumem="$(ps aux | grep -v 'grep' | grep 'firefox-bin' | awk '{printf $3 "\t" $4}'
I am extracting the CPU and Memory usage for a process and when I run it from the command line, I get the 2 fields outputted correctly:
ps aux | grep -v 'grep' | grep 'firefox-bin' | awk '{printf $3 "\t" $4}'
> 1.1 4.4
but the same command executed from within the bash script produces this:
cpumem="$(ps aux | grep -v 'grep' | grep 'firefox-bin' | awk '{printf $3 "\t" $4}')"
echo -e cpumem
> 1.1 4.40.0 0.10.0 0.0
I am guessing that it is picking up 3 records, but I just don't know where from.
I am filtering out any other grep processes by using grep -v 'grep', can someone offer any suggestions or a more reliable way ??
Maybe you have 3 records because 3 firefox are running (or one is running, and it is threading itself).
You can avoid the grep hazzle by giving ps and option to select the processes. E.g. the -C to select processes by name. With ps -C firefox-bin you get only the firefox processes. But this does not help at all, when there is more than one process.
(You can also use the ps option to output only the columns you want, so your line would be like
ps -C less --no-headers -o %cpu,%mem
).
For the triple-record you must come up with a solution, what should happen, where more than one is running. In a multiuser environment with programms that are threading there can always be situations where you have more than one process of a kind. There are many possible solution where none can help you, as you dont say, way you are going to do with it. One can think of solutions like selecting only from one user, and only the one with the lowest pid, or the process-leader in case of groups, to change the enclosing bash-script to use a loop to handle the multiple values or make it working somehow different when ps returns multiple results.
I was not able to reproduce the problem, but to help you debug, try print $11 in your awk command, that will tell you what process it is talking about
cpumem="$(ps aux | grep -v 'grep' | grep 'firefox-bin' | awk '{printf $3 "\t" $4 "\t" $11 "\n"}')"
echo -e cpumem
It's actually an easy fix for the output display; In your echo statement, wrap the variable in double-quotes:
echo -e "$cpumem"
Without using double-quotes, newlines are not preserved by converting them to single-spaces (or empty values). With quotes, the original text of the variable is preserved when outputted.
If your output contains multiple processes (i.e. - multiple lines), that means your grep actually matched multiple lines. There's a chance a child-process is running for firefox-bin, maybe a plugin/container? With ps aux, the 11th column will tell you what the actual process is, so you can update your awk to be the following (for debugging):
awk '{printf $3 "\t" $4 "\t" $11}'