I have been working on a KornShell (ksh) script where I am struck with an error with sed expression.
I have a file named abc with 100 entries and I want to assign every 8th argument in every line of abc file to a variable.
I have used something like this.
#!/bin/ksh
typeset -i x=1
while read line ; do
var1=$(sed -n '$xp' abc.txt | awk '{print $8}')
print $var1
x="$x+1"
done < abc.txt
exit
I want to refer to variable x as the line number, but I am getting an error with sed expression in referencing x variable. Please help me out.
Your quotes are wrong. Anything in single quotes is a verbatim string; if you want variable interpolation, you need to use double quotes (or, in very special circumstances, no quoting at all).
You might as well refactor everyting into Awk, too. Trivially,
var1=$(awk -v n="$x" 'NR==n{ print $8 }' abc.txt)
However, the main loop reading the whole file again just to get one line out of it is highly inefficient. Maybe you want something like
awk '{ print NR, $8 }' abc.txt |
while read x var1; do
print "$var1"
# presumably do something with $x too?
done
Related
I have a script named a.sh that produces an output - some lines of text.
I have another script named b.sh and I'd like to take the output of a.sh and hold it in a variable.
or even better to pipe it immidiately and remove all lines that are too short - meaning all lines that have less than X amount of words.
each word is seperated by a space(or multiple spaces)
how can I do that?
I would pipe the script awk and let it count words: awk '{ if (NF>4) { print }}'
Awk's default field separator separates the line into words. This means that if the number of fields (NF) is more than (>) 4 awk will prints the line.
It can be shortened to awk 'NF>4' since awk's default action is to print.
An alternative approach would be to use wc (since it literally stands for word count). You could use it in the b script like this:
while read line; do
if [[ $(wc -w <<< "$line") -gt 4 ]]
then
echo $line
fi
done
inside b.sh:
./a.sh "$1" "$2" "$3" | awk -v COUNT=$4 'NF>=COUNT'
I still haven't been able to hold the output in a variable but this worked for the parsing
Variable b has a string. Awk retrieves a substring which I want to assign to variable c. This is what I did:
#!/bin/bash
b=$(llsubmit multiple.cmd)
echo $b | c=$(awk '{
ret=match($0,".in.")
rwt=match($0,"\" has")
rqt=rwt-(ret+4)
subs=substr($0,(ret+4),rqt)
}')
... but I get a blank output for echo $c:
You can't pipe into an assignment.
c=$(echo "$b" | awk '{
ret=match($0,".in.")
rwt=match($0,"\" has")
rqt=rwt-(ret+4)
subs=substr($0,(ret+4),rqt)
}')
(Notice also the quoting around $b.)
But your Awk script looks rather complex. And it doesn't produce any output. Should it print something at the end? Without access to sample output from llsubmit this is mildly speculative, but I'm guessing something like this could work:
c=$(echo "b" | sed -n 's/.*\(\.in\.[^"]*\)" has .*/\1/p')
(Notice also the backslashes to make the dots match literally.)
You should properly then use double quotes in echo "$c" too (unless you are completely sure that the output cannot contain any shell metacharacters).
... And, of course, very often you don't want or need to store results in a variable in shell scripts if you can refactor your code into a pipeline. Perhaps you are really looking for something like
llsubmit multiple.cmd |
sed -n 's/.*\(\.in\.[^"]\)" has .*/p' |
while read -r job; do
: things with "$job"
done
It's hard to tell from your question since you didn't provide sample input and expected output but is this what you're trying to do:
$ b='foo .in.bar has'
$ c="${b% has*}"
$ c="${c#*.in.}"
$ echo "$c"
bar
I am trying to use one or two lines of Bash (that can be run in a command line) to read a folder-name and return the version inside of the name.
So if I have myfolder_v1.0.13 I know that I can use echo "myfolder_v1.0.13" | awk -F"v" '{ print $2 }' and it will return with 1.0.13.
But how do I get the shell to read the folder name and pipe with the awk command to give me the same result without using echo? I suppose I could always navigate to the directory and translate the output of pwd into a variable somehow?
Thanks in advance.
Edit: As soon as I asked I figured it out. I can use
result=${PWD##*/}; echo $result | awk -F"v" '{ print $2 }'
and it gives me what I want. I will leave this question up for others to reference unless someone wants me to take it down.
But you don't need an Awk at all, here just use bash parameter expansion.
string="myfolder_v1.0.13"
printf "%s\n" "${string##*v}"
1.0.13
You can use
basename "$(cd "foldername" ; pwd )" | awk -Fv '{print $2}'
to get the shell to give you the directory name, but if you really want to use the shell, you could also avoid the use of awk completetly:
Assuming you have the path to the folder with the version number in the parameter "FOLDERNAME":
echo "${FOLDERNAME##*v}"
This removes the longest prefix matching the glob expression "*v" in the value of the parameter FOLDERNAME.
How do i iterate through a file and print the first word only. The line is colon separated. example
root:01:02:toor
the file contains several lines. And this is what i've done so far but it does'nt work.
FILE=$1
k=1
while read line; do
echo $1 | awk -F ':'
((k++))
done < $FILE
I'm not good with bash-scripting at all. So this is probably very trivial for one of you..
edit: variable k is to count the lines.
Use cut:
cut -d: -f1 filename
-d specifies the delimiter
-f specifies the field(s) to keep
If you need to count the lines, just
count=$( wc -l < filename )
-l tells wc to count lines
awk -F: '{print $1}' FILENAME
That will print the first word when separated by colon. Is this what you are looking for?
To use a loop, you can do something like this:
$ cat test.txt
root:hello:1
user:bye:2
test.sh
#!/bin/bash
while IFS=':' read -r line || [[ -n $line ]]; do
echo $line | awk -F: '{print $1}'
done < test.txt
Example of reading line by line in bash: Read a file line by line assigning the value to a variable
Result:
$ ./test.sh
root
user
A solution using perl
%> perl -F: -ane 'print "$F[0]\n";' [file(s)]
change the "\n" to " " if you don't want a new line printed.
You can get the first word without any external commands in bash like so:
printf '%s' "${line%%:*}"
which will access the variable named line and delete everything that matches the glob :* and do so greedily, so as close to the front (that's the %% instead of a single %).
Though with this solution you do need to do the loop yourself. If this is the only thing you want to do with the variable the cut solution is better so you don't have to do the file iteration yourself.
I have a shell script that constructs an awk program as a string then pass that string to awk. This is because I want to use values of shell variables in the awk program.
My code looks like this:
awk_prog="'{if (\$4~/$shell_var/) print \$1,\$2}'"
echo $awk_prog
awk $awk_prog $FILENAME
However, when I pass the string to awk, I always get the error:
'{if ($4~/regex/) print $1,$2}'
awk: '{if
awk: ^ invalid char ''' in expression
What does that error message mean? I tried the -F: switch but it does not help. How can I settle this issue?
Thank you.
This is caused by shell quoting. The following will work:
awk_prog="{ if (\$4 ~ /$shell_var/) print \$1, \$2 }"
echo "$awk_prog"
awk "$awk_prog" $FILENAME
When you run awk '{ print }' foo from the command line, the shell interprets and removes the quotes around the program so awk receives two arguments - the first is the program text and the second is the filename foo. Your example was sending awk the program text '{if ...}' which is invalid syntax as far as awk is concerned. The outer quotes should not be present.
In the snippet that I gave above, the shell uses the quotes in the awk_prog= line to group the contents of the string into a single value and then assigns it to the variable awk_prog. When it executes the awk "$awk_prog"... line, you have to quote the expansion of $awk_prog so awk receives the program text as a single argument.
There's another way to get your shell variable into awk -- use awk's -v option:
awk -v pattern="$shell_var" '$4 ~ pattern {print $1, $2}' "$FILENAME"
Use -v multiple times if you have several variables to pass to awk.
If you truly want to hold your awk program in a shell variable, build it up using printf:
awk_script="$( printf '$4 ~ /%s/ {print $1, $2}' "$shell_var" )"
awk "$awk_script" "$FILENAME"
Note the use of quotes in the printf command: single quotes around the template to protect the dollar signs you want awk to interpret, double quotes for shell variables.
Another (IMO simpler) solution which (I think) addresses what you are intuitively trying to do is simply to use eval. You want the shell to behave as if you had literally typed:
awk '{if ($4~/foo/) print $1,$2}' path
(where foo and path are the literal contents of $shell_var and $FILENAME). To make that happen, just slap an eval on the front of your last line (and perhaps quotes for good measure, but they aren't necessary in this case) so that your last line is:
eval "awk $awk_prog $FILENAME"