When invoking xargs with only -n1, xargs executes a separate echo command for every item:
$ echo 1 2 | xargs -n1
1
2
But when using -n1 with the -I option, which passes the string to be replaced to xargs, it passes all the arguments to a single echo command, effectively ignoring-n1:
$ echo 1 2 | xargs -n1 -I% echo %
1 2
My goal is to execute an arbitrary command with different arguments:
$ echo 1 2 | xargs -n1 -I% mycommand %
# What I want to achieve
mycommand 1
mycommand 2
but I'm quite baffled by the behavior I'm seeing, so:
Why xargs seemingly ignore -n1?
What is the correct way to do what I am trying to? Note that I don't want to deal with any files while doing so.
From xargs(1):
-I replace-str
Replace occurrences of replace-str in the initial-arguments with
names read from standard input. Also, unquoted blanks do not
terminate input items; instead the separator is the newline
character. Implies -x and -L 1.
$ echo $'1\n2' | xargs -n1 -I% echo %
1
2
$ echo $'1\n2' | xargs -n1 -I% echo '*' %
* 1
* 2
From a file I'm retrieving the last line using the following cmd;
tail -n 1 build.log
The output looks like this:
1477101542,,ui,say,--> amazon-ebs: AMIs were created:\n\nus-east-1: ami-63237174\nus-west-1: ami-21236841\nus-west-2: ami-27872347
I'm trying to fetch the string after us-east-1:, us-west-1: & us-west-2 using the following grep commands:
echo | tail -n 1 build.log | egrep -m1 -oe 'us-east-1: ami-.{8}' | egrep -m1 -oe 'ami-.{8}'
I run this cmd three times for each condition. Is there a better way to do this?
If the order in which the regions appear is fixed, you can simply do:
$ echo | tail -n 1 build.log | egrep -o 'ami-.{8}'
ami-63237174
ami-21236841
ami-27872347
If you want to extract the region names and you have GNU grep, try:
$ echo | tail -n 1 build.log | grep -Po 'us-[^:]+(?=: ami-.{8})'
us-east-1
us-west-1
us-west-2
To get both region names and associated values:
$ echo | tail -n 1 build.log | egrep -o 'us-[^:]+: ami-.{8}'
us-east-1: ami-63237174
us-west-1: ami-21236841
us-west-2: ami-27872347
I am searching a word in a file through grep command. Now I need to store the status in a variable V1 with 0 or 1. how can i do it?
tail -n 2 test.s | grep -q "FA|"$(date "+%m/%d/%Y")
tail -n 2 test1.s | grep -q "FA|"$(date "+%m/%d/%Y")
tail -n 2 test2.s | grep -q "FA|"$(date "+%m/%d/%Y")
If the above searching word is found then variable V1 value should be 0 else 1.
file content :
keytran|20160111|test.s
submKeyqwqwqw|NDM|Jan 11 01:34|test.s|6666666|sdgdh-RB|ltd.ET.CTS00.act
loadstatus|thunnnB|6666666|FA|01/16/2016|01:34:57|01/16/2016
|01:37:13|load|test.s
please suggest
Depending on your shell, after each command execution the status of the previous command is available in a special variable: bash family $?, csh family $status$:
#/bin/bash
tail -n 2 test.s | grep -q "FA|"$(date "+%m/%d/%Y")
V1=$?
or
#/bin/csh
tail -n 2 test.s | grep -q "FA|"$(date "+%m/%d/%Y")
set V1=$status
This is my shell script.
Given a directory, and a word, search the directory and print the absolute path of the file that has the maximum occurrences of the word and also print the number of occurrences.
I have written the following script
#!/bin/bash
if [[ -n $(find / -type d -name $1 2> /dev/null) ]]
then
echo "Directory exists"
x=` echo " $(find / -type d -name $1 2> /dev/null)"`
echo "$x"
cd $x
y=$(find . -type f | xargs grep -c $2 | grep -v ":0"| grep -o '[^/]*$' | sort -t: -k2,1 -n -r )
echo "$y"
else
echo "Directory does does not exists"
fi
result: scriptname directoryname word
output: /somedirectory/vtb/wordsearch : 4
/foo/bar: 3
Is there any option to replace xargs grep -c $2 ? Because grep -c prints the count=number of lines which contains the word but i need to print the exact occurrence of a word in the files in a given directory
Using grep's -c count feature:
grep -c "SEARCH" /path/to/files* | sort -r -t : -k 2 | head -n 1
The grep command will output each file in a /path/name:count format, the sort will numerically (-n) sort by the 2nd (-k 2) field as delimited by a colon (-t :) in reverse order (-r). We then use head to keep the first result (-n 1).
Try This:
grep -o -w 'foo' bar.txt | wc -w
OR
grep -o -w 'word' /path/to/file/ | wc -w
grep -Fwor "$word" "$dir" | sed "s/:${word}\$//" | sort | uniq -c | sort -n | tail -1
First off, I'm new to this. I have some experience with windows scripting and apple script but not much with bash. What I'm trying to do is grab the PID and %CPU of a specific process. then compare the %CPU against a set number, and if it's higher, kill the process. I feel like I'm close, but now I'm getting the following error:
[[: 0.0: syntax error: invalid arithmetic operator (error token is ".0")
what am I doing wrong? here's my code so far:
#!/bin/bash
declare -i app_pid
declare -i app_cpu
declare -i cpu_limit
app_name="top"
cpu_limit="50"
app_pid=`ps aux | grep $app_name | grep -v grep | awk {'print $2'}`
app_cpu=`ps aux | grep $app_name | grep -v grep | awk {'print $3'}`
if [[ ! $app_cpu -gt $cpu_limit ]]; then
echo "crap"
else
echo "we're good"
fi
Obviously I'm going to replace the echos in the if/then statement but it's acting as if the statement is true regardless of what the cpu load actually is (I tested this by changing the -gt to -lt and it still echoed "crap"
Thank you for all the help. Oh, and this is on a OS X 10.7 if that is important.
I recommend taking a look at the facilities of ps to avoid multiple horrible things you do.
On my system (ps from procps on linux, GNU awk) I would do this:
ps -C "$app-name" -o pid=,pcpu= |
awk --assign maxcpu="$cpu_limit" '$2>maxcpu {print "crappy pid",$1}'
The problem is that bash can't handle decimals. You can just multiply them by 100 and work with plain integers instead:
#!/bin/bash
declare -i app_pid
declare -i app_cpu
declare -i cpu_limit
app_name="top"
cpu_limit="5000"
app_pid=`ps aux | grep $app_name | grep -v grep | awk {'print $2'}`
app_cpu=`ps aux | grep $app_name | grep -v grep | awk {'print $3*100'}`
if [[ $app_cpu -gt $cpu_limit ]]; then
echo "crap"
else
echo "we're good"
fi
Keep in mind that CPU percentage is a suboptimal measurement of application health. If you have two processes running infinite loops on a single core system, no other application of the same priority will ever go over 33%, even if they're trashing around.
#!/bin/sh
PROCESS="java"
PID=`pgrep $PROCESS | tail -n 1`
CPU=`top -b -p $PID -n 1 | tail -n 1 | awk '{print $9}'`
echo $CPU
I came up with this, using top and bc.
Use it by passing in ex: ./script apache2 50 # max 50%
If there are many PIDs matching your program argument, only one will be calculated, based on how top lists them. I could have extended the script by catching them all and avergaing the percentage or something, but this will have to do.
You can also pass in a number, ./script.sh 12345 50, which will force it to use an exact PID.
#!/bin/bash
# 1: ['command\ name' or PID number(,s)] 2: MAX_CPU_PERCENT
[[ $# -ne 2 ]] && exit 1
PID_NAMES=$1
# get all PIDS as nn,nn,nn
if [[ ! "$PID_NAMES" =~ ^[0-9,]+$ ]] ; then
PIDS=$(pgrep -d ',' -x $PID_NAMES)
else
PIDS=$PID_NAMES
fi
# echo "$PIDS $MAX_CPU"
MAX_CPU="$2"
MAX_CPU="$(echo "($MAX_CPU+0.5)/1" | bc)"
LOOP=1
while [[ $LOOP -eq 1 ]] ; do
sleep 0.3s
# Depending on your 'top' version and OS you might have
# to change head and tail line-numbers
LINE="$(top -b -d 0 -n 1 -p $PIDS | head -n 8 \
| tail -n 1 | sed -r 's/[ ]+/,/g' | \
sed -r 's/^\,|\,$//')"
# If multiple processes in $PIDS, $LINE will only match\
# the most active process
CURR_PID=$(echo "$LINE" | cut -d ',' -f 1)
# calculate cpu limits
CURR_CPU_FLOAT=$(echo "$LINE"| cut -d ',' -f 9)
CURR_CPU=$(echo "($CURR_CPU_FLOAT+0.5)/1" | bc)
echo "PID $CURR_PID: $CURR_CPU""%"
if [[ $CURR_CPU -ge $MAX_CPU ]] ; then
echo "PID $CURR_PID ($PID_NAMES) went over $MAX_CPU""%"
echo "[[ $CURR_CPU""% -ge $MAX_CPU""% ]]"
LOOP=0
break
fi
done
echo "Stopped"
Erik, I used a modified version of your code to create a new script that does something similar. Hope you don't mind it.
A bash script to get the CPU usage by process
usage:
nohup ./check_proc bwengine 70 &
bwegnine is the process name we want to monitor 70 is to log only when the process is using over 70% of the CPU.
Check the logs at: /var/log/check_procs.log
The output should be like:
DATE | TOTAL CPU | CPU USAGE | Process details
Example:
03/12/14 17:11 |20.99|98| ProdPROXY-ProdProxyPA.tra
03/12/14 17:11 |20.99|100| ProdPROXY-ProdProxyPA.tra
Link to the full blog:
http://felipeferreira.net/?p=1453
It is also useful to have app_user information available to test whether the current user has the rights to kill/modify the running process. This information can be obtained along with the needed app_pid and app_cpu by using read eliminating the need for awk or any other 3rd party parser:
read app_user app_pid tmp_cpu stuff <<< \
$( ps aux | grep "$app_name" | grep -v "grep\|defunct\|${0##*/}" )
You can then get your app_cpu * 100 with:
app_cpu=$((${tmp_cpu%.*} * 100))
Note: Including defunct and ${0##*/} in grep -v prevents against multiple processes matching $app_name.
I use top to check some details. It provides a few more details like CPU time.
On Linux this would be:
top -b -n 1 | grep $app_name
On Mac, with its BSD version of top:
top -l 1 | grep $app_name