I have a bash script to control Linux perf. As you may know, perf takes core list which can be specified in 1 of the three ways.
-C1 #core 1 only
-C1-4 # core 1 through 4
-C1,3 # core 1 and 3
Currently, I have an environment variable CORENO which will control -C$CORENO.
However, I need to offset CORENO by a fix offset (I.e.2)
I could do ((CORENO+=2)) but that only work for case 1.
Is there a Linux/bash trick to allow me to apply fix offset to every number in a bash variable?
Since you're on Linux, here's some GNU sed:
addtwo() {
sed -re 's/[^0-9,-]//g; s/[0-9]+/$((\0+2))/g; s/^/echo /e;' <<< "$1"
}
addtwo "1"
addtwo "1-4"
addtwo "3,4,5"
It will output:
3
3-6
5,6,7
It works by replacing all numbers with $((number+2)) and evaluating the result as a shell command. A whitelisting of allowed characters is applied first to avoid any security issues.
Take a look at seq
for core in `seq 2 10`; do
echo CORENO=$core
done
I’ve upvoted the sed-based answer from #that other guy because I like it more than mine, which is a “pure bash” solution, consisting of a recursive function.
function increment () {
local current="$1" n=$(($2))
if [[ "$current" =~ ^[0-9]+$ ]]; then
echo $((current+n))
elif [[ $current == *,* ]]; then
echo $(increment ${current%%,*} $n),$(increment ${current#*,} $n)
elif [[ $current == *-*-* ]]; then
echo ERROR
elif [[ $current == *-* ]]; then
echo $(increment ${current%-*} $n)-$(increment ${current#*-} $n)
else
echo ERROR
fi
}
CORENO=3-5
CORENO=$(increment $CORENO 2)
echo $CORENO
increment 3-5,6-8 3
My function will print ERROR when given an illegal argument. The one from #that other guy is much more liberal...
Related
Below is code which I developed. I am passing four arguments in function and want to return variable output which I am passing to that function as a argument number four. I am getting below mentioned error.
test.sh
output='PASS'
A=(1.0,1.0,1.0,1.0,0.0,1.0)
T=(1,2,3,4,5,6)
function compare(){
j=0
for i in "$2"
do
if [[ "$3" = '1.0' ]]
then
"$4"="PASS"
echo -e "\nA:$3 and T:$i sec" >> "$1"
else
"$4"="FAIL"
echo -e "\nA:$3 and T:$i sec" >> "$1"
fi
j=$(( $j + 1 ))
done
return "$4"
}
result=compare $1 ${A[#]} ${T[#]} output
echo "result:$result"
When I call ./test.sh file.txt, I get the following error:
./test.sh: line 13: output=FAILED: command not found
./test.sh: line 18: return: output: numeric argument required
result=
Lots of problems here:
trying to assign a value to a variable value (this is the cause of the "output=FAIL" error you see)
passing arrays as first-class values
collecting output of the function
It's still not clear how A and T are related (settle down, techbros), but it looks like T contains the index you want to look up in A.
#!/bin/bash
# validate the bash version at least 4.4 here.
function compare(){
local file=$1
local -n arrayA=$2
local -n arrayT=$3
local -n resultVar=$4
resultVar=PASS
for i in "${arrayT[#]}"; do
if [[ "${arrayA[i]}" != '1.0' ]]; then
resultVar=FAIL
# normally I would return here, but you want to log the whole array
fi
printf "A:%f and T:%d sec\n" "${arrayA[i]}" "$i" >> "$file"
done
}
output='PASS'
T=(1 2 3 4 5 6)
A=([1]=1.0 1.0 1.0 1.0 0.0 1.0) # force A to start with index 1, corresponding to the first element of T
compare "$1" A T output
echo "result:$output"
I am writing a for loop. But the loop is dependent on the content of positional argument.
If the positional arguments are seq 2 1 10, the loop is for i in $(seq 2 1 10)
If the positional arguments are purely numbers such as 1 2 5 7 10, then the loop is for i in 1 2 5 7 10.
I tried this, but it didn't work:
test () {
if [[ $1 == seq ]]
then
for i in $(seq $2 $3 $4)
else
for i in $#
fi
do
echo $i
done
}
I also tried this:
test2 () {
if [[ $1 == seq ]]
then
sss="for i in $(seq $2 $3 $4)"
else
sss="for i in $#"
fi
$sss
do
echo $i
done
}
also doesn't work.
So my questions are:
I know I could write explicit two loop inside if. But if loop content is large, this is a waste of code space. Is there any better way?
In my second attempt, why doesn't $sss expand to a for sentence and get parsed properly by bash?
Save the list of numbers in an array.
test () {
if [[ $1 == seq ]]
then
numbers=($(seq "$2" "$3" "$4"))
else
numbers=("$#")
fi
for i in "${numbers[#]}"
do
echo $i
done
}
In my second attempt, why doesn't $sss expand to a for sentence and get parsed properly by bash?
A variable can be expanded into a command to run, but not into a flow control construct like a for loop or if statement. Those need to be written out directly, they can't be stored in variables. If you try, bash will attempt to run a command named for--that is, it will look in /bin, /usr/bin, etc., for a binary named for.
An alternative to using arrays as in John Kugelman's answer is to use set -- to change the positional parameters:
test ()
{
if [[ $1 == seq ]]; then
set -- $(seq $2 $3 $4)
fi
for i; do
echo $i
done
}
Note that for i is equivalent to for i in "$#".
John already mentioned why it didn't work - variable interpolation and splitting happens after control flow is parsed.
I have this script that should make sure that the users current PHP version is between a certain range, though it SHOULD work, there is a bug somewhere that makes it think that the version is out of range, could someone take a look and tell me what I can do to fix it?
function version { echo "$#" | gawk -F. '{ printf("%d.%d.%d\n", $1,$2,$3); }'; }
phpver=`php -v |grep -Eow '^PHP [^ ]+' |gawk '{ print $2 }'`
if [ $(version $phpver) > $(version 5.2.13) ] || [ $(version $phpver) < $(version 5.2.13) ]; then
echo "PHP Version $phpver must be between 5.2.13 - 5.3.15"
exit
fi
Here's how to compare versions.
using sort -V:
function version_gt() { test "$(printf '%s\n' "$#" | sort -V | head -n 1)" != "$1"; }
example usage:
first_version=5.100.2
second_version=5.1.2
if version_gt $first_version $second_version; then
echo "$first_version is greater than $second_version !"
fi
pro:
solid way to compare fancy version strings:
support any length of sub-parts (ie: 1.3alpha.2.dev2 > 1.1 ?)
support alpha-betical sort (ie: 1.alpha < 1.beta2)
support big size version (ie: 1.10003939209329320932 > 1.2039209378273789273 ?)
can easily be modified to support n arguments. (leaved as an exercise ;) )
usually very usefull with 3 arguments: (ie: 1.2 < my_version < 2.7 )
cons:
uses a lot of various calls to different programs. So it's not that efficient.
uses a pretty recent version of sort and it might not be available on your
system. (check with man sort)
without sort -V:
## each separate version number must be less than 3 digit wide !
function version { echo "$#" | gawk -F. '{ printf("%03d%03d%03d\n", $1,$2,$3); }'; }
example usage:
first_version=5.100.2
second_version=5.1.2
if [ "$(version "$first_version")" -gt "$(version "$second_version")" ]; then
echo "$first_version is greater than $second_version !"
fi
pro:
quicker solution as it only calls 1 subprocess
much more compatible solution.
cons:
quite specific, version string must:
have version with 1, 2 or 3 parts only. (excludes '2.1.3.1')
each parts must be numerical only (excludes '3.1a')
each part can't be greater than 999 (excludes '1.20140417')
Comments about your script:
I can't see how it could work:
as stated in a comment > and < are very special shell character and you should replace them by -gt and -lt
even if you replaced the characters, you can't compare version numbers as if they where integers or float. For instance, on my system, php version is 5.5.9-1ubuntu4.
But your function version() is quite cleverly written already and may help you by circumventing the classical issue that sorting alphabetically numbers won't sort numbers numerically ( alphabetically 1 < 11 < 2, which is wrong numerically). But be carefull: arbitrarily large numbers aren't supported by bash (try to keep under 32bits if you aim at compatibility with 32bits systems, so that would be 9 digit long numbers). So I've modified your code (in the second method NOT using sort -V) to force only 3 digits for each part of the version string.
EDIT: applied #phk amelioration, as it is noticeably cleverer and remove a subprocess call in the first version using sort. Thanks.
There is possibility for deb-distributions:
dpkg --compare-versions <version> <relation> <version>
For example:
dpkg --compare-versions "0.0.4" "gt" "0.0.3"
if [ $? -eq "0" ]; then echo "YES"; else echo "NO"; fi
It is doing a lexical comparison. Use one of these:
if [ $(version $phpver) -gt $(version 5.2.13) ] || [ $(version $phpver) -lt $(version 5.2.13) ]; then
if [[ $(version $phpver) > $(version 5.2.13) ]] || [[ $(version $phpver) < $(version 5.2.13) ]]; then
if (( $(version $phpver) > $(version 5.2.13) )) || (( $(version $phpver) < $(version 5.2.13) )); then
Or do it all in awk or some other tool. It is screaming for some optimisation. It also seems you're not producing numbers either, so you have a pretty odd design. Usually the version substrings are multiplied by 1000 and then all summed up to get a single comparable scalar.
Here's another solution that:
does not run any external command apart from tr
has no restriction on number of parts in version string
can compare version strings with different number of parts
Note that it's Bash code using array variables.
compare_versions()
{
local v1=( $(echo "$1" | tr '.' ' ') )
local v2=( $(echo "$2" | tr '.' ' ') )
local len="$(max "${#v1[*]}" "${#v2[*]}")"
for ((i=0; i<len; i++))
do
[ "${v1[i]:-0}" -gt "${v2[i]:-0}" ] && return 1
[ "${v1[i]:-0}" -lt "${v2[i]:-0}" ] && return 2
done
return 0
}
The function returns:
0 if versions are equal (btw: 1.2 == 1.2.0)
1 if the 1st version is bigger / newer
2 if the 2nd version is bigger / newer
However #1 -- it requires one additional function (but function min is quite usable to have anyway):
min()
{
local m="$1"
for n in "$#"
do
[ "$n" -lt "$m" ] && m="$n"
done
echo "$m"
}
However #2 -- it cannot compare version strings with alpha-numeric parts (though that would not be difficult to add, actually).
A much safer option for testing the PHP CLI version is to use PHP's own version_compare function.
#!/bin/bash
MIN_VERSION="7.0.0"
MAX_VERSION="7.1.4"
PHP_VERSION=`php -r 'echo PHP_VERSION;'`
function version_compare() {
COMPARE_OP=$1;
TEST_VERSION=$2;
RESULT=$(php -r 'echo version_compare(PHP_VERSION, "'${TEST_VERSION}'", "'${COMPARE_OP}'") ? "TRUE" : "";')
test -n "${RESULT}";
}
if ( version_compare "<" "${MIN_VERSION}" || version_compare ">" "${MAX_VERSION}" ); then
echo "PHP Version ${PHP_VERSION} must be between ${MIN_VERSION} - ${MAX_VERSION}";
exit 1;
fi
echo "PHP Version ${PHP_VERSION} is good!";
The following solution should more accurately addresses your exact need. It can be used to test whether the CURRENT version string falls between MIN and MAX. I am assuming that MIN and MAX are acceptable version numbers (i.e. MIN <= CURRENT <= MAX rather than MIN < CURRENT < MAX).
# Usage: version MIN CURRENT MAX
version(){
local h t v
[[ $2 = "$1" || $2 = "$3" ]] && return 0
v=$(printf '%s\n' "$#" | sort -V)
h=$(head -n1 <<<"$v")
t=$(tail -n1 <<<"$v")
[[ $2 != "$h" && $2 != "$t" ]]
}
For example...
if ! version 5.2.13 "$phpver" 5.3.15; then
echo "PHP Version $phpver must be between 5.2.13 - 5.3.15"
exit
fi
If you're on Bash 3 with an older version of sort (lookin at you macOS...), then I created the following helper script you can source in (can also be ran as a command):
https://github.com/unicorn-fail/version_compare
I wrote this inelegant function a while back for a similar problem. vers_limit will return 0 if arg1 is less than or equal to arg2, non-0 otherwise:
vers_limit()
{
VERNEW=$1
VERLMT=$2
CHKNEW=$VERNEW
CHKLMT=$VERLMT
PASSED=
while :
do
PARTNEW=${CHKNEW%%.*}
PARTLMT=${CHKLMT%%.*}
if [[ $PARTNEW -lt $PARTLMT ]]
then
PASSED=GOOD
break
elif [[ $PARTNEW -gt $PARTLMT ]]
then
PASSED=
break
else
NXTNEW=${CHKNEW#*.}
if [[ $NXTNEW == $CHKNEW ]]
then
if [[ $NXTNEW == $CHKLMT ]]
then
PASSED=GOOD
break
else
NXTNEW=0
fi
fi
NXTLMT=${CHKLMT#*.}
if [[ $NXTLMT == $CHKLMT ]]
then
NXTLMT=0
fi
fi
CHKNEW=$NXTNEW
CHKLMT=$NXTLMT
done
test "$PASSED"
}
This is not as compact as some of the other solutions here, nor does it provide 3-way status (i.e., less, equal, greater), though I believe one can order the arguments, interpret the pass/fail status, and/or call twice to accomplish any desired result. That said, vers_limit does have certain virtues:
No calls to external utilities such as sort, awk, gawk, tr, etc.
Handles numeric versions of arbitrary size (up to the shell's
limit for integer calculations)
Handles an arbitrary number of "dotted" parts.
v_min="5.2.15"
v_max="5.3.15"
v_php="$(php -v | head -1 | awk '{print $2}')"
[ ! "$v_php" = "$(echo -e "$v_php\n$v_min\n$v_max" | sort -V | head -2 | tail -1)" ] && {
echo "PHP Version $v_php must be between $v_min - $v_max"
exit
}
This puts v_min, v_max and v_php in version order and tests if v_php is not in the middle.
PURE BASH
I found my way to this page because I had the same problem. The other answers did not satisfy me, so I wrote this function.
So long as the 2 versions have the same number of periods in them this will compare the versions correctly.
It does a c style for loop, setting $i incrementally from 0 to # of numbers in the version string.
for each #:
if new - old is neg we know the first version is newer.
If new - old is pos we know the first version is older.
If new - old is 0 then it is the same and we need to continue checking.
We run false after to set exit status of the function for the case where $1 == $2 the versions are totally identical.
newver=1.10.1
installedver=1.9.25
#installedver=1.11.25
#installedver=1.10.1
checkupdate(){
# $1 = new version
# $2 = installed version
IFS='.' read -r -a nver <<< "$1"
IFS='.' read -r -a iver <<< "$2"
for ((i = 0 ; i < "${#nver[#]}" ; i++)) ;do
case "$((${nver[i]}-${iver[i]}))" in
-*) return 1 ;;
0) ;;
*) return 0 ;;
esac
false
done
}
checkupdate "$newver" "$installedver" && echo yes || echo no
Another method for SH
After I tried to implement my above function on Android I realized that I would not always have bash, so the above function did not work for me. Here is the version I wrote using awk to get around needing bash:
checkupdate(){
# $1 = new version
# $2 = installed version
i=1
#we start at 1 and go until number of . so we can use our counter as awk position
places=$(awk -F. '{print NF+1}' <<< "$1")
while (( "$i" < "$places" )) ;do
npos=$(awk -v pos=$i -F. '{print $pos}' <<< "$1")
ipos=$(awk -v pos=$i -F. '{print $pos}' <<< "$2")
case "$(( $npos - $ipos ))" in
-*) return 1 ;;
0) ;;
*) return 0 ;;
esac
i=$((i+1))
false
done
}
I'm new to bash and wondered if you guys could help - I have a list of files named things like
amp1_X_1
amp1_X_2
...
amp43_X_3
and have to extract the maximum values of the numbers which appear either side of X.
I've been looking at various links to help solve my problem, but when I try and tweak the examples to suit my purpose I get various errors, like "command not found" or %% operators remaining unevaluated.
How do I Select Highest Number From Series of <string>_# File Names in Bash Script
extract numbers from file names
For example, I tried something like
max=-1
for file in amp*_X_1
do
pattern=_X_*
num=${file}%%${pattern}
num=${num}##amp
echo "num is $num"
[[ $num -gt $max ]] && max=$num
done
echo "max is: $max"
(credit largely to ghostdog74 in the first link) which would in any case only work for one set of numbers, but it returns with the %% unevaluated. Have I missed something stupid/is there a better way to do this?
Thanks!
there are some errors in your script:
1)for file in amp*_X_1
if you want use the files in a specific folder you have to use commands like ls or find
i.e.
for file in $(ls -1 <your file>)
for file in ($find . -name "<your file>")
etc..
2) num=${file}%%${pattern}
use num=${file%%$pattern}
below you can find your script edited:
max=-1
for file in $(find . -name "amp*")
do
echo "file[$file]"
pattern=_X_*
num=${file%%$pattern}
num=$(echo $num|sed "s/^\.\///g" | sed "s/amp//g")
echo "num is $num"
[[ $num -gt $max ]] && max=$num
done
echo "max is: $max"
output:
sh-4.3$ bash -f main.sh
file[./amp1_X_1]
num is 1
file[./amp3_X_2]
num is 3
file[./amp3_X_1]
num is 3
file[./amp43_X_1]
num is 43
max is: 43
now this script is looking for the maximum between the numbers on the left side of X.
From your question isn't clear what's your expected result..
1) find maximum between all numbers on both X side?
2) find maximum between all numbers on left X side?
3) find maximum between all numbers on right X side?
if 1 is the case you have to add another variable, get the value, and check it.
something like
max=-1
for file in $(find . -name "amp*")
do
echo "file[$file]"
pattern=_X_*
num=${file%%$pattern}
num=$(echo $num|sed "s/^\.\///g" | sed "s/amp//g")
echo "num is $num"
pattern2=*_X_
num2=${file##$pattern2}
echo "num2 is $num2"
[[ $num -gt $max ]] && max1=$num
[[ $num2 -gt $max ]] && max2=$num2
if [[ $max1 -gt $max2 ]]; then
echo "max is: $max1"
else
echo "max is: $max2"
fi
done
#echo "max is: $max"
How would you achieve this in bash. It's a question I got asked in an interview and I could think of answers in high level languages but not in shell.
As I understand it, the real implementation of tail seeks to the end of the file and then reads backwards.
The main idea is to keep a fixed-size buffer and to remember the last lines. Here's a quick way to do a tail using the shell:
#!/bin/bash
SIZE=5
idx=0
while read line
do
arr[$idx]=$line
idx=$(( ( idx + 1 ) % SIZE ))
done < text
for ((i=0; i<SIZE; i++))
do
echo ${arr[$idx]}
idx=$(( ( idx + 1 ) % SIZE ))
done
If all not-tail commands are allowed, why not be whimsical?
#!/bin/sh
[ -r "$1" ] && exec < "$1"
tac | head | tac
Use wc -l to count the number of lines in the file. Subtract the number of lines you want from this, and add 1, to get the starting line number. Then use this with sed or awk to start printing the file from that line number, e.g.
sed -n "$start,\$p"
There's this:
#!/bin/bash
readarray file
lines=$(( ${#file[#]} - 1 ))
for (( line=$(($lines-$1)), i=${1:-$lines}; (( line < $lines && i > 0 )); line++, i-- )); do
echo -ne "${file[$line]}"
done
Based on this answer: https://stackoverflow.com/a/8020488/851273
You pass in the number of lines at the end of the file you want to see then send the file via stdin, puts the entire file into an array, and only prints the last # lines of the array.
The only way I can think of in “pure” shell is to do a while read linewise on the whole file into an array variable with indexing modulo n, where n is the number of tail lines (default 10) — i.e. a circular buffer, then iterate over the circular buffer from where you left off when the while read ends. It's not efficient or elegant, in any sense, but it'll work and avoids reading the whole file into memory. For example:
#!/bin/bash
incmod() {
let i=$1+1
n=$2
if [ $i -ge $2 ]; then
echo 0
else
echo $i
fi
}
n=10
i=0
buffer=
while read line; do
buffer[$i]=$line
i=$(incmod $i $n)
done < $1
j=$i
echo ${buffer[$i]}
i=$(incmod $i $n)
while [ $i -ne $j ]; do
echo ${buffer[$i]}
i=$(incmod $i $n)
done
This script somehow imitates tail:
#!/bin/bash
shopt -s extglob
LENGTH=10
while [[ $# -gt 0 ]]; do
case "$1" in
--)
FILES+=("${#:2}")
break
;;
-+([0-9]))
LENGTH=${1#-}
;;
-n)
if [[ $2 != +([0-9]) ]]; then
echo "Invalid argument to '-n': $1"
exit 1
fi
LENGTH=$2
shift
;;
-*)
echo "Unknown option: $1"
exit 1
;;
*)
FILES+=("$1")
;;
esac
shift
done
PRINTHEADER=false
case "${#FILES[#]}" in
0)
FILES=("/dev/stdin")
;;
1)
;;
*)
PRINTHEADER=true
;;
esac
IFS=
for I in "${!FILES[#]}"; do
F=${FILES[I]}
if [[ $PRINTHEADER == true ]]; then
[[ I -gt 0 ]] && echo
echo "==> $F <=="
fi
if [[ LENGTH -gt 0 ]]; then
LINES=()
COUNT=0
while read -r LINE; do
LINES[COUNT++ % LENGTH]=$LINE
done < "$F"
for (( I = COUNT >= LENGTH ? LENGTH : COUNT; I; --I )); do
echo "${LINES[--COUNT % LENGTH]}"
done
fi
done
Example run:
> bash script.sh -n 12 <(yes | sed 20q) <(yes | sed 5q)
==> /dev/fd/63 <==
y
y
y
y
y
y
y
y
y
y
y
y
==> /dev/fd/62 <==
y
y
y
y
y
> bash script.sh -4 <(yes | sed 200q)
y
y
y
y
Here's the answer I would give if I were actually asked this question in an interview:
What environment is this where I have bash but not tail? Early boot scripts, maybe? Can we get busybox in there so we can use the full complement of shell utilities? Or maybe we should see if we can squeeze a stripped-down Perl interpreter in, even without most of the modules that would make life a whole lot easier. You know dash is much smaller than bash and perfectly good for scripting use, right? That might also help. If none of that is an option, we should check how much space a statically linked C mini-tail would need, I bet I can fit it in the same number of disk blocks as the shell script you want.
If that doesn't convince the interviewer that it's a silly question, then I go on to observe that I don't believe in using bash extensions, because the only good reason to write anything complicated in shell script nowadays is if total portability is an overriding concern. By avoiding anything that isn't portable even in one-offs, I don't develop bad habits, and I don't get tempted to do something in shell when it would be better done in a real programming language.
Now the thing is, in truly portable shell, arrays may not be available. (I don't actually know whether the POSIX shell spec has arrays, but there certainly are legacy-Unix shells that don't have them.) So, if you have to emulate tail using only shell builtins and it's got to work everywhere, this is the best you can do, and yes, it's hideous, because you're writing in the wrong language:
#! /bin/sh
a=""
b=""
c=""
d=""
e=""
f=""
while read x; do
a="$b"
b="$c"
c="$d"
d="$e"
e="$f"
f="$x"
done
printf '%s\n' "$a"
printf '%s\n' "$b"
printf '%s\n' "$c"
printf '%s\n' "$d"
printf '%s\n' "$e"
printf '%s\n' "$f"
Adjust the number of variables to match the number of lines you want to print.
The battle-scarred will note that printf is not 100% available either. Unfortunately, if all you have is echo, you are up a creek: some versions of echo cannot print the literal string "-n", and others cannot print the literal string "\n", and even figuring out which one you have is a bit of a pain, particularly as, if you don't have printf (which is in POSIX), you probably don't have user-defined functions either.
(N.B. The code in this answer, sans rationale, was originally posted by user 'Nirk' but then deleted under downvote pressure from people whom I shall charitably assume were not aware that some shells do not have arrays.)