I'm attempting to parse a make -n output to make sure only the programs I want to call are being called. However, awk tries to interpret the contents of the output and run (?) it. Errors are something like awk: fatal: Cannot find file 'make'. I have gotten around this by saving the output as a temporary file and then reading that into awk. However, I'm sure there's a better way; any suggestions?
EDIT: I'm using the output later in my script and would like to avoid saving a file to increase speed if possible.
Here's what isn't working:
my_input=$(make -n file)
my_lines=$(echo $my_input | awk '/bin/ { print $1 }') #also tried printf and cat
Here's what works but obviously takes longer than it has to because of writing the file:
make -n file > temp
my_lines=$(awk '/bin/ { print $1 }' temp)
Many thanks for your help!
You can directly parse the output when it is generated by the following command and save the result in a file.
make -n file | grep bin > result.out
If you really want to go for an overkill awk solution, change your second line in the following way:
my_lines="$(awk '/bin/ { print }' temp)"
Related
I have a file .txt with some informations, i need to grep the "Report:" line and save each line in a different .txt file!
it should result something like this in the end:
case1.txt
case2.txt
case3.txt
I tried to
cat cases.txt| grep Report: | while read Report; do echo $Report | > /home/kali/Desktop/allcases/case.txt done
but it didnt work and just created one file called case.txt containing the last grepped "Report:"
I dont know if i was very clear then i'll show this screenshot:
cases
I wanted to split all theses reports in a different .txt file for each report!
These case informations are from a game, so dont worry!
awk would be better suited than grep and a while loop. If acceptable, you can try;
awk '/^Report/{cnt++;close(report); report="case"cnt".txt"}/./{print > report}' file.txt
perl -ne '++$i && `printf "$_" > case$i.txt` if /Report:/' cases.txt
This is looping over cases.txt and shelling out printf "$_" > case$i.txt if the line matches /Report:/
Because it's perl there's some syntax and precedence tricks in here to make it terse and confusing.
I am trying to add a few headers to occasionally empty CSV file (so sometimes the file will be empty and sometimes it will have data) via bash. So far I have tried using sed and awk.
Sed:
sed -i '1liAge,Sex,Location' $nameOfFile
Awk:
awk 'BEGIN{print "Age,Sex,Location"}1' $nameOfFile
Both of these do not appear to work since no headers are added. When I echo $nameOfFile, csvInfo.csv is printed so I don't believe there's anything wrong with $nameOfFile. Not sure where i'm going wrong here.
Your sed command requires there to already be at least one line in the file, so I don't think you can make that work reliably.
The Awk command you have should work as such, but it writes its results to standard output. If you have GNU Awk, you can use -i inplace to get behavior similar to sed -i; but the portable solution is to write the results to a temporary file, then move back on top of the old file.
awk 'BEGIN{print "Age,Sex,Location"}1' "$nameOfFile" >tmp
mv tmp "$nameOfFile"
Notice also When to wrap quotes around a shell variable.
If you want to avoid adding a header when it's already there, you can keep the BEGIN block, but suppress printing of the old header, if present:
awk 'BEGIN{print "Age,Sex,Location"}
NR>1 || $0 !~ /^Age,Sex,Location$/' "$nameOfFile" >tmp
mv tmp "$nameOfFile"
Proper hygiene around temporary files is somewhat more involved for a production script, but if this is just for your personal use, let's leave it at this. For more information, perhaps review Removing created temp files in unexpected bash exit
I am trying to to provide a file for my shell as an input which in return should test if the file contains a specific word and decide what command to execute. I am not figuring out yet where the mistake might lie. Please find the shell script that i wrote:
#!/bin/(shell)
input_file="$1"
output_file="$2"
grep "val1" | awk -f ./path/to/script.awk $input_file > $output_file
grep "val2" | sh ./path/to/script.sh $input_file > $output_file
when I input the the file that uses awk everything get executed as expected, but for the second command I don't even get an output file. Any help is much appreciated
Cheers,
You haven't specified this in your question, but I'm guessing you have a file with the keyword, e.g. file cmdfile that contains x-g301. And then you run your script like:
./script "input_file" "output_file" < cmdfile
If so, the first grep command will consume the whole cmdfile on stdin while searching for the first pattern, and nothing will be left for the second grep. That's why the second grep, and then your second script, produces no output.
There are many ways to fix this, but choosing the right one depends on what exactly you are trying to do, and how does that cmdfile look like. Assuming that's a larger file with other things than just the command pattern, you could pass that file as a third argument to your script, like this:
./script "input_file" "output_file" "cmdfile"
And have your script handle it like this:
#!/bin/bash
input_file="$1"
output_file="$2"
cmdfile="$3"
if grep -q "X-G303" "$cmdfile"; then
awk -f ./mno/script.awk "$input_file" > t1.json
fi
if grep -q "x-g301" "$cmdfile"; then
sh ./mno/tm.sh "$input_file" > t2.json
fi
Here I'm also assuming that your awk and sh scripts don't really need the output from grep, since you're giving them the name of the input file.
Note the proper way to use grep for existence search is via its exit code (and the muted output with -q). Instead of the if we could have used shortcircuiting (grep ... && awk ...), but this way is probably more readable.
awk '/^nameserver/ && !modif { printf("nameserver 127.0.0.1\n"); modif=1 } {print}' testfile.txt
It is displaying output but I want to write the output to same file. In my example testfile.txt.
Not possible per se. You need a second temporary file because you can't read and overwrite the same file. Something like:
awk '(PROGRAM)' testfile.txt > testfile.tmp && mv testfile.tmp testfile.txt
The mktemp program is useful for generating unique temporary file names.
There are some hacks for avoiding a temporary file, but they rely mostly on caching and read buffers and quickly get unstable for larger files.
Since GNU Awk 4.1.0, there is the "inplace" extension, so you can do:
$ gawk -i inplace '{ gsub(/foo/, "bar") }; { print }' file1 file2 file3
To keep a backup copy of original files, try this:
$ gawk -i inplace -v INPLACE_SUFFIX=.bak '{ gsub(/foo/, "bar") }
> { print }' file1 file2 file3
This can be used to simulate the GNU sed -i feature.
See: Enabling In-Place File Editing
Despite the fact that using a temp file is correct, I don't like it because :
you have to be sure not to erase another temp file (yes you can use mktemp - it's a pretty usefull tool)
you have to take care of deleting it (or moving it like thiton said) INCLUDING when your script crash or stop before the end (so deleting temp files at the end of the script is not that wise)
it generate IO on disk (ok not that much but we can make it lighter)
So my method to avoid temp file is simple:
my_output="$(awk '(PROGRAM)' source_file)"
echo "$my_output" > source_file
Note the use of double quotes either when grabbing the output from the awk command AND when using echo (if you don't, you won't have newlines).
Had to make an account when seeing 'awk' and 'not possible' in one sentence. Here is an awk-only solution without creating a temporary file:
awk '{a[b++]=$0} END {for(c=1;c<=b;c++)print a[c]>ARGV[1]}' file
You can also use sponge from moreutils.
For example
awk '!a[$0]++' file|sponge file
removes duplicate lines and
awk '{$2=10*$2}1' file|sponge file
multiplies the second column by 10.
Try to include statement in your awk file so that you can find the output in a new file. Here total is a calculated value.
print $total, total >> "new_file"
This inline writing worked for me. Redirect the output from print back to the original file.
echo "1" > test.txt
awk '{$1++; print> "test.txt"}' test.txt
cat test.txt
#$> 2
awk '/<ul>/ {ul++} ul == 6 { getline } 1' /var/www/html/INFOSEC/english/test.html
if i run this line of code, the shell will not help me to modify the file, instead, it only output the result in the shell. Can any one help???thx
The simplest solution is to just send the output to a file; you might want to copy the file beforehand so you don't have to overwrite the file you're reading (which might otherwise lead to undesired behavior).
cp test.html test.html.orig
awk 'your awk script here' test.html.orig >test.html
# and then optionally remove the copy:
rm test.html.orig