I have written the following script that extracts a number from an rss file.
#!/bin/sh
wget -O selic https://conteudo.bcb.gov.br/api/feed/pt-br/PAINEL_INDICADORES/juros
line=$(grep 'dailyratevalue' selic)
index=$(awk -v var=$line 'BEGIN {print index(var, "dailyratevalue") }')
end=$((index+21))
echo $line | cut -c $index-$end | tail -c 4 | tr ',' '.' > selic
In zsh it works perfectly, but i need it to work in bash, too. I have tried running it on bash but i get the following error
awk: cmd. line:1: <content
awk: cmd. line:1: ^ syntax error
The error pattern <content comes from the line that is being fed as a parameter to awk, which makes no sense to me, since awk is just supposed to get me the position of the pattern i want.
What could this be?
index=$(awk -v var="$line" 'BEGIN {print index(var, "dailyratevalue") }')
should fix it.
awk can do all of the extra steps. You can just
wget -qO - https://conteudo.bcb.gov.br/api/feed/pt-br/PAINEL_INDICADORES/juros | \
awk -F '&[gl]t;' '/dailyratevalue/ {sub(",", ".", $25); print $25;}'
and obtain the value you want.
This is setting the FS and getting the field you want for the line that matches dailyratevalue.
#DiegoTorresMilano's answer is probably better overall, but if you want to do it in bash, the main thing you need to do is double-quote your variable references. Without double-quotes around them, bash (and most shells other than zsh) splits variables into "words", and also expands anything that looks like a wildcard expression into a list of matching filenames. You almost never want this, so use double-quotes. In your case, there are two places they're needed: around $line here:
index=$(awk -v var="$line" 'BEGIN {print index(var, "dailyratevalue") }')
and here:
echo "$line" | cut -c $index-$end | tail -c 4 | tr ',' '.' > selic
Note that you don't need double-quotes around the $( ) expressions, because they're on the right side of an assignment statement, and that isn't subject to word splitting and wildcard expansion. If they occurred elsewhere, you'd probably want double-quotes around them too.
BTW, shellcheck.net is really good at pointing out common mistakes like this, so I recommend running your scripts through it (even when they seem to be working correctly).
Related
I run the command
df -gP /data1 /data2 | grep -v File | awk '{print $1}' |
awk -F/dev/ '$0=$2' | tr '\n' '
on the AIX shell (ksh) and it prints the output below:
lv_data01 lv_data02 root#testhost:/
However, I would like the output to be printed this way. Could someone help?
lv_data01 lv_data02
Using grep … | awk … | awk … is not necessary; a single awk could do the whole job. So could sed and it might even be easier. I'd be tempted to deal with the spacing by using:
x=$(df … | sed …); echo $x
The tr command, once corrected, replaces newlines with spaces, so the prompt follows without a newline before it. The ; echo suggestion adds the missing newline; the echo $x suggestion (note no double quotes) does too.
As for the sed command:
sed -n '/File/!{ s/[[:space:]].*//; s%^.*/dev/%%p; }'
Don't print anything by default
If the line doesn't match File (doing the work of grep -v):
remove the first space (blank or tab) and everything after it (doing the work of awk '{print $1}')
replace everything up to /dev/ with nothing and print (doing the work of awk -F/dev/ '{$0=$2}')
The command substitution and capture, followed by echo, deals with spaces and newlines.
So, my suggested solution is:
x=$(df -gP /data1 /data2 | sed -n '/File/!{ s/[[:space:]].*//; s%^.*/dev/%%p; }'); echo $x
You could add unset x after the echo if you are going to be using this directly in the shell and not in a shell script. If it'll be encapsulated in a shell script, you don't have to worry about it.
I'm blithely assuming the output from df -gP won't contain a path such as this, with two occurrences of /dev:
/who/knows/dev/lv_data01/dev/bin
If that's a real problem, you can fix the sed script, but I don't think it will be. It's one thing the second awk script in the question handles differently.
sed terminating early because of escape characters in variables. Hoping awk can do what I need but can't see how!
# Main section ==========================================╕
LASTIP=`grep -E '[0-2]{0,1}[0-9]{0,9}[0-9]{0,1}\.[0-2]{0,1}[0-9]{0,9}[0-9]{0,1}\.[0-2]{0,1}[0-9]{0,9}[0-9]{0,1}\.[0-2]{0,1}[0-9]{0,9}[0-9]{0,1}' $SRCDIR/$IPLOGFILE | tail -1|awk -F'\t' '{print$3}'`
if [ "$CURRENTIP" == "$LASTIP" ]; then
# Still using old IP===================================╕
FIRSTDETECTED=`grep $LASTIP $SRCDIR/$IPLOGFILE | tail -1|awk -F'\t' '{print$1}'`
LASTDETECTED=`grep $LASTIP $SRCDIR/$IPLOGFILE | tail -1|awk -F'\t' '{print$2}'`
OLDLINE=$(printf "$FORMAT" "$FIRSTDETECTED" "$LASTDETECTED" "$LASTIP")
AMENDEDLINE=$(printf "$FORMAT" "$FIRSTDETECTED" "$TIMESTAMP" "$LASTIP")
sed -i "s/'$OLDLINE'/'$AMENDEDLINE'/g" $SRCDIR/$IPLOGFILE
This works fine apart from the last sed, which terminates because $OLDLINE and/or $AMENDEDLINE contains escape chars. I thought I could do a direct substitution for awk to solve the issue but the more I thought about it the more I thought the whole section could be done much more efficiently with awk - maybe in one line of awk? Trouble is I don't know where to start. Am I fooling myself about simplifying it or is there a way? If there is, you may have to help me out, as I find this stuff 'warps my fragile little mind'*
*courtesy of Cartman ;P
I've snipped out the section but can supply the rest of the script if that helps?
You can perhaps try like that
sed -i 's/'"$OLDLINE"'/'"$AMENDEDLINE"'/g' "$SRCDIR"/"$IPLOGFILE"
I think '$OLDLINE' and '$AMENDEDLINE' are not expanded
I am new to AWK and trying to write some code where I can delete all files in a directory apart from the newest N number.
My code works if I use a hard coded number instead of a variable.
Works:
delete=`ls -t | awk 'NR>3'`
echo $delete
Does Not Work:
amount=3
delete=`ls -t | awk 'NR>$amount'`
echo $delete
I know the problem lies somewhere with the bash variable not being recognised as an awk variable however I do not know how to correct.
I have tried variations of code to fix this such as below, however I am still at a loss.
amount=3
delete=`ls -t | awk -v VAR=${amount} 'NR>VAR'`
echo $delete
Could you please advise what the correct code is ?
Shells don't expand anything inside single quotes.
Either:
amount=3
delete=$(ls -t | awk "NR>$amount")
or:
amount=3
delete=$(ls -t | awk -v amount=$amount 'NR > amount')
Be aware that parsing the output of ls is fraught if your file names are not limited to the portable file name character set. Spaces, newlines, and other special characters in the file name can wreck the parsing.
The simplest fix is to use double quotes instead of single. Single quotes prevent the shell from interpolating the variable $amount in the quoted string.
amount=3
ls -t | awk "NR>$amount"
I would not use a variable to capture the result. If you do use one, you need to quote it properly when interpolating it.
amount=3
delete=$(ls -t | awk -v VAR="$amount" 'NR>VAR')
echo "$delete"
Note that this is basically identical to your second attempt, which should have worked, modulo the string quoting issues.
When I use AWK print command outside shell it is working perfectly. Below is content of the file (sample.txt) which is comma separated.
IROG,1245,OUTO,OTUG,USUK
After, executing below command outside shell I get IROG as output.
cat sample.txt | awk -F, '{print $1}' > data.txt
Below is inside the shell script
my $HOME ='home/tmp/stephen';
my $DATA ="$HOME/data.txt";
my $SAMPLE ="$HOME/sample.txt";
`cat $SAMPLE | awk -F, '{print $1}' > $DATA`;
But here i get the same content as in original file instead of 1st column.
output is IROG,1245,OUTO,OTUG,USUK
but I expect only IROG. Can someone advise where I am wrong here?
The $1 inside your backticks expression is being expanded by perl before being executed by the shell. Presumably it has no value, so your awk command is simply {print }, which prints the whole record. You should escape the $ to prevent this from happening:
`awk -F, '{print \$1}' "$SAMPLE" > "$DATA"`;
Note that I have quoted your variables and also removed your useless use of cat.
If you mean to use a shell script, as opposed to a perl one (which is what you've currently got), you can do this:
home=home/tmp/stephen
data="$home/data.txt"
sample="$home/sample.txt"
awk -F, '{print $1}' "$sample" > "$data"
In the shell, there must be no spaces in variable assignments. Also, it is considered bad practice to use UPPERCASE variable names, as you risk overwriting the ones used internally by the shell. Furthermore, it is considered good practice to use double quotes around variable expansions to prevent problems related to word splitting and glob expansion.
There are a few ways that you could trim the leading whitespace from your first field. One would be to use sub to remove it:
awk -F, '{sub(/^ */, ""); print $1}'
This removes any space characters from the start of the line. Again, remember to escape the $ if doing this within backticks in perl.
Here's my issue, i know how to count the files using the following two strategies but i have a problem with each one.
I am using '.sh' extension.
First:
count=`echo $2 | awk -F, {'print NF'}`
causes my program to throw an error at me: awk: cannot execute - No such file or directory
Secondly:
count=`echo $2 | tr -cd , | wc -c`
Works if you have multiple values separated by commas, however, it will not work if the input is a single item with no commas.
Like i said, this was previously working with the awk but for some reason when i ran it on the physical device instead of the virtual machine it gave me that error.
any ideas?
Thing I know are NOT the issue:
Version of shell is the same.
Try count=$(echo ${2} | awk -F, '{print NF}') instead - you have your braces and quotes inside-out.
Although, it seems your bigger problem is that awk appears to not be executable... You might try which awk and ls -l $(which awk) to see what's up with that...