Here's my issue, i know how to count the files using the following two strategies but i have a problem with each one.
I am using '.sh' extension.
First:
count=`echo $2 | awk -F, {'print NF'}`
causes my program to throw an error at me: awk: cannot execute - No such file or directory
Secondly:
count=`echo $2 | tr -cd , | wc -c`
Works if you have multiple values separated by commas, however, it will not work if the input is a single item with no commas.
Like i said, this was previously working with the awk but for some reason when i ran it on the physical device instead of the virtual machine it gave me that error.
any ideas?
Thing I know are NOT the issue:
Version of shell is the same.
Try count=$(echo ${2} | awk -F, '{print NF}') instead - you have your braces and quotes inside-out.
Although, it seems your bigger problem is that awk appears to not be executable... You might try which awk and ls -l $(which awk) to see what's up with that...
Related
I run the command
df -gP /data1 /data2 | grep -v File | awk '{print $1}' |
awk -F/dev/ '$0=$2' | tr '\n' '
on the AIX shell (ksh) and it prints the output below:
lv_data01 lv_data02 root#testhost:/
However, I would like the output to be printed this way. Could someone help?
lv_data01 lv_data02
Using grep … | awk … | awk … is not necessary; a single awk could do the whole job. So could sed and it might even be easier. I'd be tempted to deal with the spacing by using:
x=$(df … | sed …); echo $x
The tr command, once corrected, replaces newlines with spaces, so the prompt follows without a newline before it. The ; echo suggestion adds the missing newline; the echo $x suggestion (note no double quotes) does too.
As for the sed command:
sed -n '/File/!{ s/[[:space:]].*//; s%^.*/dev/%%p; }'
Don't print anything by default
If the line doesn't match File (doing the work of grep -v):
remove the first space (blank or tab) and everything after it (doing the work of awk '{print $1}')
replace everything up to /dev/ with nothing and print (doing the work of awk -F/dev/ '{$0=$2}')
The command substitution and capture, followed by echo, deals with spaces and newlines.
So, my suggested solution is:
x=$(df -gP /data1 /data2 | sed -n '/File/!{ s/[[:space:]].*//; s%^.*/dev/%%p; }'); echo $x
You could add unset x after the echo if you are going to be using this directly in the shell and not in a shell script. If it'll be encapsulated in a shell script, you don't have to worry about it.
I'm blithely assuming the output from df -gP won't contain a path such as this, with two occurrences of /dev:
/who/knows/dev/lv_data01/dev/bin
If that's a real problem, you can fix the sed script, but I don't think it will be. It's one thing the second awk script in the question handles differently.
I have written the following script that extracts a number from an rss file.
#!/bin/sh
wget -O selic https://conteudo.bcb.gov.br/api/feed/pt-br/PAINEL_INDICADORES/juros
line=$(grep 'dailyratevalue' selic)
index=$(awk -v var=$line 'BEGIN {print index(var, "dailyratevalue") }')
end=$((index+21))
echo $line | cut -c $index-$end | tail -c 4 | tr ',' '.' > selic
In zsh it works perfectly, but i need it to work in bash, too. I have tried running it on bash but i get the following error
awk: cmd. line:1: <content
awk: cmd. line:1: ^ syntax error
The error pattern <content comes from the line that is being fed as a parameter to awk, which makes no sense to me, since awk is just supposed to get me the position of the pattern i want.
What could this be?
index=$(awk -v var="$line" 'BEGIN {print index(var, "dailyratevalue") }')
should fix it.
awk can do all of the extra steps. You can just
wget -qO - https://conteudo.bcb.gov.br/api/feed/pt-br/PAINEL_INDICADORES/juros | \
awk -F '&[gl]t;' '/dailyratevalue/ {sub(",", ".", $25); print $25;}'
and obtain the value you want.
This is setting the FS and getting the field you want for the line that matches dailyratevalue.
#DiegoTorresMilano's answer is probably better overall, but if you want to do it in bash, the main thing you need to do is double-quote your variable references. Without double-quotes around them, bash (and most shells other than zsh) splits variables into "words", and also expands anything that looks like a wildcard expression into a list of matching filenames. You almost never want this, so use double-quotes. In your case, there are two places they're needed: around $line here:
index=$(awk -v var="$line" 'BEGIN {print index(var, "dailyratevalue") }')
and here:
echo "$line" | cut -c $index-$end | tail -c 4 | tr ',' '.' > selic
Note that you don't need double-quotes around the $( ) expressions, because they're on the right side of an assignment statement, and that isn't subject to word splitting and wildcard expansion. If they occurred elsewhere, you'd probably want double-quotes around them too.
BTW, shellcheck.net is really good at pointing out common mistakes like this, so I recommend running your scripts through it (even when they seem to be working correctly).
I have tried all kinds of filters using grep to try and solve this but just cannot crack it.
cpumem="$(ps aux | grep -v 'grep' | grep 'firefox-bin' | awk '{printf $3 "\t" $4}'
I am extracting the CPU and Memory usage for a process and when I run it from the command line, I get the 2 fields outputted correctly:
ps aux | grep -v 'grep' | grep 'firefox-bin' | awk '{printf $3 "\t" $4}'
> 1.1 4.4
but the same command executed from within the bash script produces this:
cpumem="$(ps aux | grep -v 'grep' | grep 'firefox-bin' | awk '{printf $3 "\t" $4}')"
echo -e cpumem
> 1.1 4.40.0 0.10.0 0.0
I am guessing that it is picking up 3 records, but I just don't know where from.
I am filtering out any other grep processes by using grep -v 'grep', can someone offer any suggestions or a more reliable way ??
Maybe you have 3 records because 3 firefox are running (or one is running, and it is threading itself).
You can avoid the grep hazzle by giving ps and option to select the processes. E.g. the -C to select processes by name. With ps -C firefox-bin you get only the firefox processes. But this does not help at all, when there is more than one process.
(You can also use the ps option to output only the columns you want, so your line would be like
ps -C less --no-headers -o %cpu,%mem
).
For the triple-record you must come up with a solution, what should happen, where more than one is running. In a multiuser environment with programms that are threading there can always be situations where you have more than one process of a kind. There are many possible solution where none can help you, as you dont say, way you are going to do with it. One can think of solutions like selecting only from one user, and only the one with the lowest pid, or the process-leader in case of groups, to change the enclosing bash-script to use a loop to handle the multiple values or make it working somehow different when ps returns multiple results.
I was not able to reproduce the problem, but to help you debug, try print $11 in your awk command, that will tell you what process it is talking about
cpumem="$(ps aux | grep -v 'grep' | grep 'firefox-bin' | awk '{printf $3 "\t" $4 "\t" $11 "\n"}')"
echo -e cpumem
It's actually an easy fix for the output display; In your echo statement, wrap the variable in double-quotes:
echo -e "$cpumem"
Without using double-quotes, newlines are not preserved by converting them to single-spaces (or empty values). With quotes, the original text of the variable is preserved when outputted.
If your output contains multiple processes (i.e. - multiple lines), that means your grep actually matched multiple lines. There's a chance a child-process is running for firefox-bin, maybe a plugin/container? With ps aux, the 11th column will tell you what the actual process is, so you can update your awk to be the following (for debugging):
awk '{printf $3 "\t" $4 "\t" $11}'
I am trying to manipulate a text file. I've got most of it figured out myself, but I'm stumped with why sed seems to go into infinite loop mode. The text file can be downloaded from census.gov.
At the moment, I just want a list of states that I can throw into a for loop to do some state-specific processing. So far, I've got this. (I'm not a bash expert, suggestions are welcome.
sed 1d tables/ansi.csv | awk -F "," '{print $1}' | uniq | tr \n : | sed s/:/" "/g
I want to put this into $() to use the output in a for loop, but for some reason, sed is hanging up and not exiting. I actually need to add a couple of things to the final sed command, to properly format things, but I want to get this running correctly before I go any further.
In the end - I want something that looks like (just showing the first few):
"AL" "AK" "AZ" "AR" "CA" "CO" ....
Right now, sed returns more of less what I expect and returns (just showing the last few)
...."MP" "PR" "UM" "VI" "
But, rather than exiting, sed hangs and I have to Ctrl-C out of the script. If I remove the final sed statement, the little script runs as I would expect, without hanging.
So, why on earth is this hanging?
I would suggest putting the sed script inside quotes:
sed 1d tables/ansi.csv | awk -F "," '{print $1}' | uniq | tr '\n' : | sed 's/:/" "/g'
The reason that sed seems to "hang" may be that tr has removed the final newline which sed requires. By the way, the newline argument to tr needs to be quoted.
However, the whole thing can be done in AWK:
awk -F, 'NR > 1 {a[$1]=$1} END { delim=":"; num=asort(a); for (i=1;i<=num;i++) printf "\"%s\" ",a[i]; printf "\n"}' tables/ansi.csv
awk -F"," 'NR>1 && (!($1 in a)){print $1;a[$1]}' file|sort|awk '{printf "\"%s\" ",$1}'
I have a file with fields separated by pipe characters and I want to print only the second field. This attempt fails:
$ cat file | awk -F| '{print $2}'
awk: syntax error near line 1
awk: bailing out near line 1
bash: {print $2}: command not found
Is there a way to do this?
Or just use one command:
cut -d '|' -f FIELDNUMBER
The key point here is that the pipe character (|) must be escaped to the shell. Use "\|" or "'|'" to protect it from shell interpertation and allow it to be passed to awk on the command line.
Reading the comments I see that the original poster presents a simplified version of the original problem which involved filtering file before selecting and printing the fields. A pass through grep was used and the result piped into awk for field selection. That accounts for the wholly unnecessary cat file that appears in the question (it replaces the grep <pattern> file).
Fine, that will work. However, awk is largely a pattern matching tool on its own, and can be trusted to find and work on the matching lines without needing to invoke grep. Use something like:
awk -F\| '/<pattern>/{print $2;}{next;}' file
The /<pattern>/ bit tells awk to perform the action that follows on lines that match <pattern>.
The lost-looking {next;} is a default action skipping to the next line in the input. It does not seem to be necessary, but I have this habit from long ago...
The pipe character needs to be escaped so that the shell doesn't interpret it. A simple solution:
$ awk -F\| '{print $2}' file
Another choice would be to quote the character:
$ awk -F'|' '{print $2}' file
Another way using awk
awk 'BEGIN { FS = "|" } ; { print $2 }'
And 'file' contains no pipe symbols, so it prints nothing. You should either use 'cat file' or simply list the file after the awk program.