im have a script which puts the output of "df" into a string.
When the script runs once everything is fine, but when i put it in a while loop, it crashs in the second run with the following error.
cut: invalid field value ‘\n2\n3\n4\n5\n6’
Try 'cut --help' for more information.
./getdata.sh: line 248: 1
2
3
4
5
6%6: syntax error in expression (error token is "2
3
4
5
6%6")
Code snippet:
PARTITION_RETURN_STRING=""
PARTITION_TEMP_STRING=""
for i in $(df | awk '{ print $6 }')
do
if [ "$i" != "Mounted" ]; then
for abc in $(seq 1 6)
do
PARTITION_TEMP_STRING=$(df -m | awk -v bla=$i '$6 == bla' | tr -s ' ' | cut -d ' ' -f $abc)
PARTITION_RETURN_STRING="$PARTITION_RETURN_STRING$PARTITION_TEMP_STRING"
if [ "$(($abc%6))" = "0" ]; then
PARTITION_RETURN_STRING+="|"
else
PARTITION_RETURN_STRING+="#"
fi
done
fi
done
Seems like there is a value which is getting used in the first run and didnt resets for the second run, but i couldnt figure it out.
Related
I assign a keyword as variable, and need to awk from a file using this variable and loop. The file has millions of lines.
i have tried the code below.
DEVICE="DEV2"
while read -r line
do
echo $line
X_keyword=`echo $line | cut -d ',' -f 2 | grep -w "X" | cut -d '=' -f2`
echo $X_keyword
done <<< "$(grep -w $DEVICE $config)"
log="Dev2_PRT.log"
while read -r file
do
VALUE=`echo $file | cut -d '|' -f 1`
HEADER=`echo $VALUE | cut -c 1-4`
echo $file
if [[ $HEADER = 'PTR:' ]]; then
VALUE=`echo $file | cut -d '|' -f 4`
echo $VALUE
XCOORD+=($VALUE)
((X++))
fi
done <<< "awk /$X_keyword/ $log"
expected result:
the log files content lots of below:
PTR:1|2|3|4|X_keyword
PTR:1|2|3|4|Y_rest .....
Filter the X_keyword and get the field no 4.
Unfortunately your shell script is simply the wrong approach to this problem (see https://unix.stackexchange.com/q/169716/133219 for some of the reasons why) so you should set it aside and start over.
To demonstrate the solution, lets create a sample input file:
$ seq 10 | tee file
1
2
3
4
5
6
7
8
9
10
and a shell variable to hold a regexp that's a character list of the chars 5, 6, or 7:
$ var='[567]'
Now, given the above input, here is the solution for how to g/re/p pattern as variable and count how many results:
$ awk -v re="$var" '$0~re{print; c++} END{print "---" ORS c+0}' file
5
6
7
---
3
If that's not all you need then please edit your question to clarify your requirements and provide concise, testable sample input and expected output.
I am trying to do a simple thing:
To get the second number in the the line with the second occurence of the word TER and lower it by one and further process it. The tr -s ' ' is there because the file is not delimited by tabs, but by different amounts of whitespaces.
My script:
first_res_atombumb= grep 'TER' tata_sbox_cuda.pdb | head -n 2 | tail -1 |tr -s ' '| cut -f 2 -d ' '
echo $((first_res_atombumb-1))
but this only returnes:
255
-1
Of course I want to have 254.
adding | tr -d '\n' does not help either, what on earth is going on? I have already asked several people at work noone seems to know.
the lines in question look linke this
TER 128 DA3 4
TER 255 DA3 8
and if I apply grep 'TER' tata_sbox_cuda.pdb | head -n 2 | tail -1 | tr -s ' '| cut -f 2 -d ' ' in the command line i get what i expect, just 255
With bash, I'd write
n_ter=0
while read -a words; do
if [[ ${words[0]} == TER ]] && (( ++n_ter == 2 )); then
echo $(( ${words[1]} - 1 ))
fi
done < file
but I'd use awk
awk '$1 == "TER" && ++n == 2 {print $2 - 1}' file
The problem with your code: you forgot to use the $() command substitution syntax
first_res_atombumb= grep 'TER' tata_sbox_cuda.pdb | head -n 2 | tail -1 |tr -s ' '| cut -f 2 -d ' '
# .................^...............................................................................^
echo $((first_res_atombumb-1))
You're setting the variable to an empty string in the environment of the grep command. Then, since you're not capturing the output of that pipeline, "255" is printed to the terminal. Because the variable is unset in your current shell, you get echo $((-1))
All you need is:
first_res_atombumb=$(grep 'TER' tata_sbox_cuda.pdb | head -n 2 | tail -1 |tr -s ' '| cut -f 2 -d ' ')
# .................^^...............................................................................^
But I'd still use awk.
If I understand your problem correctly you can solve it using AWK:
awk 'BEGIN{v=0} $1 == "TER" {v++;if (v==2) {print $2-1 ;exit}}' tata_sbox_cuda.pdb
Explanation:
BEGIN{v=0} declaring and nulling the variable.
$1 == "TER" execute the command in {} only if it's the second occurence of TER.
{v++;if (v==2) {print $2-1 ;exit}}' increase the value of v and check if it's 2, in this case subtract 1 from the second field and display, exit afterwards (will make the processing faster and will skip unnecessary lines).
Do you know what is wrong with my script as I always get the error mesage:
position frac1 frac2
: command not found:
'/s1_met.sh: line 3: syntax error near unexpected token `do
'/s1_met.sh: line 3: `for lineF1 in $(cat $1); do
Code here:
export IFS=$'\n'
echo "position frac1 frac2";
for lineF1 in $(cat $1); do
if [ $(echo $lineF1 | cut -b 1-2) = "##" ]; then
echo "skip line" >&2;
else
startF1=$(echo $lineF1 | cut -f 4);
stopF1=$(echo $lineF1 | cut -f 5);
fracF1=$(echo $lineF1 | cut -f 9 | cut -d ";" -f 4 | cut -d "=" -f 2);
lineF2=$(grep "$startF1" $2);
if [ -z "$lineF2" ]; then
echo "position $startF1 cannot be found" >&2;
else
fracF2=$(echo $lineF2 | cut -f 9 | cut -d ";" -f 4 | cut -d "=" -f 2);
echo "$startF1 $fracF1 $fracF2";
fi;
fi
done;
There's nothing wrong with it, you must not be running it with BASH.
Edited to say you need to check your line endings, your comment below with the ^M means that you have extra characters on the line. See here.
https://stackoverflow.com/tags/bash/info
Try putting the "shebang" line in the script shebang docs
To do this, run which bash which will tell you something like /bin/bash. Your script should then be:
#!/bin/bash
echo "I'm running with bash!"
Try that, your syntax is OK.
I am trying to write a script in BASH that will take between 1 and 5 command line arguments from the user and report them back in reverse numerical order to standard output. The only command I know that would work similarly to this is the sort command, but this only works for files. Is there a similar command for sorting command line arguments? Here is what I have so far.
#!/bin/bash
if [ $# -lt 1 ] || [ $# -gt 5 ];
then echo "Incorrect number of arguments!"
else
sorted=sort -rn $*
echo "SORTED: $sorted"
fi
Try:
sorted=$( printf '%s\n' "$#" | sort -rn )
printf '%s\n' "${sorted//$'\n'/ }"
You can give the sort command values from standard input. It expects every value on its own line, which you can achieve by combining echo and tr:
sorted=$(echo $* | tr ' ' '\n' | sort -rn - | tr '\n' ' ')
The last invocation of tr is only necessary if you want the result to be space-delimited again and not newline-delimited.
#!/bin/bash
if [ $# -lt 1 ] || [ $# -gt 5 ];
then echo "Incorrect number of arguments!"
else
sorted=$(echo $* | tr ' ' '\n' | sort -rn | tr '\n' ' ')
echo "SORTED: $sorted"
fi
echo $* | tr ' ' '\n' | sort -rn | tr '\n' ' '
You need to use command substitution $(...) to capture the output of a command like that.
#!/bin/bash
if [ $# -lt 1 ] || [ $# -gt 5 ]; then
echo "Incorrect number of arguments!"
else
sorted=$(for var in "$#"; do echo "$var"; done | sort -rn | tr -d '\n')
echo "SORTED: $sorted"
fi
$ ./test 1 2 3 4 5
SORTED: 5 4 3 2 1
$ ./test 5 4 3 2 1
SORTED: 5 4 3 2 1
I have the following data in a Tab delimited file:
_ DATA _
Col1 Col2 Col3 Col4 Col5
blah1 blah2 blah3 4 someotherText
blahA blahZ blahJ 2 someotherText1
blahB blahT blahT 7 someotherText2
blahC blahQ blahL 10 someotherText3
I want to make sure that the data in 4th column of this file is always an integer. I know how to do this in perl
Read each line, Store value of 4th column in a variable
check if that variable is an integer
if above is true, continue the loop
else break out of the loop with message saying file data not correct
But how would I do this in a shell script using standard linux/unix filter? My guess would be to use grep, but I am not sure how?
cut -f4 data | LANG=C grep -q '[^0-9]' && echo invalid
LANG=C for speed
-q to quit at first error in possible long file
If you need to strip the first line then use tail -n+2 or you could get hacky and use:
cut -f4 data | LANG=C sed -n '1b;/[^0-9]/{s/.*/invalid/p;q}'
awk is the tool most naturally suited for parsing by columns:
awk '{if ($4 !~ /^[0-9]+$/) { print "Error! Column 4 is not an integer:"; print $0; exit 1}}' data.txt
As you get more complex with your error detection, you'll probably want to put the awk script in a file and invoke it with awk -f verify.awk data.txt.
Edit: in the form you'd put into verify.awk:
{
if ($4 !~/^[0-9]+$/) {
print "Error! Column 4 is not an integer:"
print $0
exit 1
}
}
Note that I've made awk exit with a non-zero code, so that you can easily check it in your calling script with something like this in bash:
if awk -f verify.awk data.txt; then
# action for success
else
# action for failure
fi
You could use grep, but it doesn't inherently recognize columns. You'd be stuck writing patterns to match the columns.
awk is what you need.
I can't upvote yet, but I would upvote Jefromi's answer if I could.
Sometimes you need it BASH only, because tr, cut & awk behave differently on Linux/Solaris/Aix/BSD/etc:
while read a b c d e ; do [[ "$d" =~ ^[0-9] ]] || echo "$a: $d not a numer" ; done < data
Edited....
#!/bin/bash
isdigit ()
{
[ $# -eq 1 ] || return 0
case $1 in
*[!0-9]*|"") return 0;;
*) return 1;;
esac
}
while read line
do
col=($line)
digit=${col[3]}
if isdigit "$digit"
then
echo "err, no digit $digit"
else
echo "hey, we got a digit $digit"
fi
done
Use this in a script foo.sh and run it like ./foo.sh < data.txt
See tldp.org for more info
Pure Bash:
linenum=1; while read line; do field=($line); if ((linenum>1)); then [[ ! ${field[3]} =~ ^[[:digit:]]+$ ]] && echo "FAIL: line number: ${linenum}, value: '${field[3]}' is not an integer"; fi; ((linenum++)); done < data.txt
To stop at the first error, add a break:
linenum=1; while read line; do field=($line); if ((linenum>1)); then [[ ! ${field[3]} =~ ^[[:digit:]]+$ ]] && echo "FAIL: line number: ${linenum}, value: '${field[3]}' is not an integer" && break; fi; ((linenum++)); done < data.txt
cut -f 4 filename
will return the fourth field of each line to stdout.
Hopefully that's a good start, because it's been a long time since I had to do any major shell scripting.
Mind, this may well not be the most efficient compared to iterating through the file with something like perl.
tail +2 x.x | sort -n -k 4 | head -1 | cut -f 4 | egrep "^[0-9]+$"
if [ "$?" == "0" ]
then
echo "file is ok";
fi
tail +2 gives you all but the first line (since your sample has a header)
sort -n -k 4 sorts the file numerically on the 4th column, letters will rise to the top.
head -1 gives you the first line of the file
cut -f 4 gives you the 4th column, of the first line
egrep "^[0-9]+$" checks if the value is a number (integers in this case).
If egrep finds nothing, $? is 1, otherwise it's 0.
There's also:
if [ `tail +2 x.x | wc -l` == `tail +2 x.x | cut -f 4 | egrep "^[0-9]+$" | wc -l` ] then
echo "file is ok";
fi
This will be faster, requiring two simple scans through the file, but it's not a single pipeline.
#OP, use awk
awk '$4+0<=0{print "not ok";exit}' file