i know it is very basic question but im total new in shell scripting
i a txt file called 'berkay' and content of it is like
03:05:16 debug blablabla1
03:05:18 error blablablablabla2
05:42:14 degub blabblablablabal
06:21:24 debug balbalbal1
I want to print the lines whose second column is error so the output will be
03:05:18 error blablablablabla2
I am thinking about something like " if nawk { $2}" but i need help.
With this for example:
$ awk '$2=="error"' file
03:05:18 error blablablablabla2
Why is this working? Because when the condition is true, awk automatically performs its default behaviour: {print $0}. So there is no need to explicitly write it.
Related
I have a file that has data as follows
a==1 b==2 c==9 x==4 d==5 ...etc each of them are in new line I need to get the c==9 from a file and c==9 could be anywhere in the file so getting the value from using line number is not possible. I am looking to use bash as possible solution.
What exactly do you want and what you have tried? Simplest way is to use awk:
~$ cat test
a==1
b==1
dskjfhjks
kjfhkhd
c==1
kjfds
~$ awk '/c==1/ { print }' test
c==1
NOTE: I am a noob at bash scripts and the awk command - please excuse any dumb mistakes I make.
I am unable to substitute shell variables into my awk pattern. I am trying to scan through a file, find the first occurence of a specific string in the file, and print each line that succeed it in order until it hits an empty string/line.
I don't know the string I am searching for in advance, and I would like to substitute in that variable.
When I run this with the string directly specified (e.g "< main>:"), it works perfectly. I've already searched on how awk patterns work, and how to substitute in variables. I've tried using the -v flag for awk, directly using the shell variable - nothing works.
funcName="<${2}>:"
awk=`awk -v FN="$funcName" '/FN/,/^$/' "$ofile"`
rfile=search.txt
echo -e "$awk" > "$rfile"
The error is just that nothing prints. I want to print all the lines between my desired string and the next empty line.
Could you please try following, haven't tested it because no clear samples but should work.
funcName="<${2}>:"
awk_result=$(awk -v FN="$funcName" 'index($0,FN){found=1} found; /^$/{found=""}' "$ofile")
rfile=search.txt
echo -e "$awk_result" > "$rfile"
Things fixed in OP's attempt:
NEVER keep same name of a variable as a binary's name or on a keyword's name so changed awk variable name to awk_result.
Use of backticks is depreciated now, so always wrap your variable for having values in var=$(......your commands....) fixed it for awk_result variable.
Now let us talk about awk code fix, I have used index method which checks if value of variable FN is present in a line then make a FLAG(a variable TRUE) and make it false till line is empty as per OP's ask.
I have been working on this little script at work to free up my own time and am currently stuck on part of it. The script is supposed to pull some content from a JSON, modify the content, and then re-upload it. The modification part is the portion that doesn't work.
An example of what the content looks like after being extracted from the JSON is:
<p>App1_v1.0_20160911_release.apk</p<p>App2_v2.0_20160915_beta.apk</p><p>App3_v3.0_20150909_VendorRelease.apk</p>
The modification function is supposed to update the list with the newer app filenames in the same location. I've tried using both SED and AWK to get this to work but I haven't gotten anywhere fast.
Here are examples of both commands and the parameters for the substitution I am trying to run on the example file:
old_name=App1_.*_release.apk
new_name=App1_v1.0_20160920_1152_release.apk
sed "s/$old_name/$new_name/" body > upload
awk -v oldname="$old_name" -v newname="$new_name" '{sub(oldname, newname)}1' body > upload
What ends up happening is the substitution will change the correct part of the list, but then nuke everything between that point and the end of the list.
Thank you for any and all help.
PS: If I didn't explain something correctly or you feel some information is missing, please comment and let me know so I can better explain the problem.
There are SO many possible values of oldname, newname, and your input data that could cause either of the commands you wrote to fail - don't use that "replace a regexp with a backreference-enabled-string" approach in any command, use string operations instead (which means you can't use sed since sed doesn't support strings)
This modifies your sample input as you say you want:
$ awk -v new='App1_v1.0_20160920_1152_release.apk' 'BEGIN{RS="</p>\n?"; FS=OFS="<p>"} NR==1{$2=new} {printf "%s%s", $0, RT}' file
<p>App1_v1.0_20160920_1152_release.apk<p>App2_v2.0_20160915_beta.apk</p><p>App3_v3.0_20150909_VendorRelease.apk</p>
If that's not adequate then edit your question to better explain your requirements and provide more truly representative sample input/output.
The above uses GNU awk for multi-char RS and RT.
I have a CSV file with several thousand lines, and I need to take some of the columns in that file to create another CSV file to use for import to a database.
I'm not in shape with shell scripting anymore, is there anyone who can help with pointing me in the correct direction?
I have a bash script to read the source file but when I try to print the columns I want to a new file it just doesn't work.
while IFS=, read symbol tr_ven tr_date sec_type sec_name name
do
echo "$name,$name,$symbol" >> output.csv
done < test.csv
Above is the code I have. Out of the 6 columns in the original file, I want to build a CSV with "column6, column6, collumn1"
The test CSV file is like this:
Symbol,Trading Venue,Trading Date,Security Type,Security Name,Company Name
AAAIF,Grey Market,22/01/2015,Fund,,Alternative Investment Trust
AAALF,Grey Market,22/01/2015,Ordinary Shares,,Aareal Bank AG
AAARF,Grey Market,22/01/2015,Ordinary Shares,,Aluar Aluminio Argentino S.A.I.C.
What am I doing wrong with my script? Or, is there an easier - and faster - way of doing this?
Edit
These are the real headers:
Symbol,US Trading Venue,Trading Date,OTC Tier,Caveat Emptor,Security Type,Security Class,Security Name,REG_SHO,Rule_3210,Country of Domicile,Company Name
I'm trying to get the last column, which is number 12, but it always comes up empty.
The snippet looks and works fine to me, maybe you have some weird characters in the file or it is coming from a DOS environment (use dos2unix to "clean" it!). Also, you can make use of read -r to prevent strange behaviours with backslashes.
But let's see how can awk solve this even faster:
awk 'BEGIN{FS=OFS=","} {print $6,$6,$1}' test.csv >> output.csv
Explanation
BEGIN{FS=OFS=","} this sets the input and output field separators to the comma. Alternatively, you can say -F=",", -F, or pass it as a variable with -v FS=",". The same applies for OFS.
{print $6,$6,$1} prints the 6th field twice and then the 1st one. Note that using print, every comma-separated parameter that you give will be printed with the OFS that was previously set. Here, with a comma.
I'm using a bash script to grep out some lines from a log file. The basic format of this log file is:
field1: value1, field2=value2, field3=value3,
field4=value4,value5,value6, field5=value7
Sometimes there will be lines in which field1: value1 is identical, but some of the other information is either the same or different. I'd like to filter those lines out, so that I only grep out the first instance of anything that has the same "field1: value1" tuple.
I'd prefer a nice command-line one-liner if you can find something especially simple. I definitely want to keep it in the bash script. This is on linux, so we've got all the command-line tools available.
Thanks!
Using awk:
awk -F, '!arr[$1]++ { print }' LOGFILE
The awk program uses an array to keep a count of the number of times a particular 'field1: value1` string is seen, but only prints the incoming line the first time.