Is there a way to process 2 different files passed as arguments to a bash script which uses awk.
Script signature:
./statistical_sig.sh path_to_reviews_folder hotel_1 hotel_2
I tried the following but only the first argument got processed.
hotel1="$2";
hotel2="$3";
dos2unix -U $hotel1 | dos2unix -U $hotel2 | echo "$hotel1" "$hotel2" | xargs | awk -v hotel1="$hotel1" -v hotel2="$hotel2" { .. code ..}
You don't need all these pipes to run awk.
Either you will use something like this if you plan to read with awk some other files and use hotel1 and hotel2 somehow inside your awk code:
awk -v hotel1=$(dos2unix -U "$hotel1") -v hotel2=$(dos2unix -U "$hotel2") { awk code ..} file1 file2
Or you will use this if you plan to read and process contents of files hotel1 and hotel2:
awk { awk code ..} <(dos2unix -U "$hotel1") <(dos2unix -U "$hotel2")
Alternativelly you can modify your code like this, but this is less efficient :
hotel1=$(dos2unix "$hotel1") && hotel2=$(dos2unix "$hotel2") && echo "$hotel1 $hotel2" | awk '{your code here}'
If you explain better your question advising what is the awk code and what you are trying to achieve, you will get better advises.
Related
Problem
I have this comand:
sed $((SS - default_scripts))!d customScripts.txt
and it gives me Foo Bar.
I want to convert this to lowercase.
Attempt
When I tried using the | awk '{print tolower($0)}' command on it it returned nothing:
$($(sed $((SS - default_scripts))!d customScripts.txt) | awk '{print tolower($0)}')
Final
Please enlighten me on my typo, or recommend me another POSIX way of converting a whole string to lowercase in a compact manner. Thank you!
The pipe to awk should be inside the same command substitution as sed, so that it processes the output of sed.
$(sed $((SS - default_scripts))!d customScripts.txt | awk '{print tolower($0)}')
You don't need another command substitution around both of them.
Your typo was wrapping everything in $(...) and so first trying to execute the output of just the sed part and then trying to execute the output of the sed ... | awk ... pipeline.
You don't need sed commands nor shell arithmetic operations when you're using awk. If I understand what you're trying to do with this:
$(sed $((SS - default_scripts))!d customScripts.txt) | awk '{print tolower($0)}'
correctly then it'd be just this awk command:
awk -v s="$SS" -v d="$default_scripts" 'BEGIN{n=s-d} NR==n{print tolower($0); exit}' customScripts.txt
I have a bash script that basically does some math with awk and then exports the results to a file on the machine, but it doesn't create the file or even modify the file if I create it for it. It works fine on my Mac, but can't seem to get it working on Ubuntu. Below is my code.
awk -v a="$topnum" -v b="${allArray[2]}" 'BEGIN { if (a==b) print 2 >"/home/skyler/Documents/SkyMine/arp_nr.var" }'
Try to separate awk and get the result into a new file using tee command.
Option 1. Using tee command.
awk -v a="$topnum" -v b="${allArray[2]}" 'BEGIN { if (a==b) print 2 }' | tee "/home/skyler/Documents/SkyMine/arp_nr.var"
Option 2. Using the output (STDOUT) redirection
awk -v a="$topnum" -v b="${allArray[2]}" 'BEGIN { if (a==b) print 2 }' > "/home/skyler/Documents/SkyMine/arp_nr.var"
PS. You should post your data file example, it would be easier to understand and help you.
I am not sure why i am getting the unexpected syntax '( err
#!/bin/bash
DirBogoDict=$1
BogoFilter=/home/nikhilkulkarni/Downloads/bogofilter-1.2.4/src/bogofilter
echo "spam.."
for i in 'cat full/index |fgrep spam |awk -F"/" '{if(NR>1000)print$2"/"$3}'|head -500'
do
cat $i |$BogoFilter -d $DirBogoDict -M -k 1024 -v
done
echo "ham.."
for i in 'cat full/index | fgrep ham | awk -F"/" '{if(NR>1000)print$2"/"$3}'|head -500'
do
cat $i |$BogoFilter -d $DirBogoDict -M -k 1024 -v
done
Error:
./score.bash: line 7: syntax error near unexpected token `('
./score.bash: line 7: `for i in 'cat full/index |fgrep spam |awk -F"/" '{if(NR>1000)print$2"/"$3}'|head -500''
Uh, because you have massive syntax errors.
The immediate problem is that you have an unpaired single quote before the cat which exposes the Awk script to the shell, which of course cannot parse it as shell script code.
Presumably you want to use backticks instead of single quotes, although you should actually not read input with for.
With a fair bit of refactoring, you might want something like
for type in spam ham; do
awk -F"/" -v type="$type" '$0 ~ type && NR>1000 && i++<500 {
print $2"/"$3 }' full/index |
xargs $BogoFilter -d $DirBogoDict -M -k 1024 -v
done
This refactors the useless cat | grep | awk | head into a single Awk script, and avoids the silly loop over each output line. I assume bogofilter can read file name arguments; if not, you will need to refactor the xargs slightly. If you can pipe all the files in one go, try
... xargs cat | $BogoFilter -d $DirBogoDict -M -k 1024 -v
or if you really need to pass in one at a time, maybe
... xargs sh -c 'for f; do $BogoFilter -d $DirBogoDict -M -k 1024 -v <"$f"; done' _
... in which case you will need to export the variables BogoFilter and DirBogoDict to expose them to the subshell (or just inline them -- why do you need them to be variables in the first place? Putting command names in variables is particularly weird; just update your PATH and then simply use the command's name).
In general, if you find yourself typing the same commands more than once, you should think about how to avoid that. This is called the DRY principle.
The syntax error is due to bad quoting. The expression whose output you want to loop over should be in command substitution syntax ($(...) or backticks), not single quotes.
I have a problem with bash. I have big log file and I must check only a part of all log. In this purpose I use those expressions:
cat 29.log | grep -A 999 "15/02/06-22:30"
or
awk '$1>="15/02/06-22:30" {print$0}' 29.log
I want to change "15/02/06-22:30" at "date +%y/%m/d-%H:M" but when I use command
awk '$1>="date +%y/%m/d-%H:M" {print$0}' 29.log
or
awk '$1>='date +%y/%m/d-%H:M' {print$0}' 29.log
nothing happens.
Any ideas?
I need this in one command, not a script
You can pass shell variables to AWK using the -v flag:
awk -v d="$(date '+%y/%m/%d-%H:%M')" '$1>=d' 29.log
grep -A 999 `date '+%y/%m/d-%H:M'` 29.log
I have a chain of commands that can execute all at once, however I wish to put it inside of a bash script. The problem is that I have no clue how to. My command is like so:
/usr/bin/sort -n db | /usr/bin/awk -F: '{print $1; print $2}' | db5.1_load -T -t hash newdb
How can I convert the above into a bash script?
This should normally be as simple as putting the shell command into a text file, and putting the Unix shebang on the first line of the file, which defines which program to use to run the script (in this case, /bin/bash). So this would look like:
#!/bin/bash
/usr/bin/sort -n db | /usr/bin/awk -F: '{print $1; print $2}' | db5.1_load -T -t hash newdb