I have a configuration(conf.file) with list of variables and its values generated from shell script
cat conf.file
export ORA_HOME=/u01/app/12.1.0
export ORA_SID=test1
export ORA_LOC=
export TW_WALL=
export TE_STAT=YES
I want to find any variable has null value after equal(=) symbol, if so, then report the message as Configuration file has following list of null variables
You can use awk for this:
awk -F"[= ]" '$3=="" && NF==3 {print $2}' conf.file
That will split each record by a space or an equal sign, then test the third field in each row. If it's empty, it will print the second field (the variable).
UPDATE: Added in a test for Number of Fields (NF) equal to 3 to avoid null rows.
try:
awk -F"=" '$2' Input_file
As you need after = a field shouldn't be empty so making = as a field separator and checking if 2nd field is not empty then no action defined in my code so default print action will happen for any line which satisfy this condition. Let me know if this helps.
EDIT: Above will give only those values whose values are NULL after =, thanks to JNevill for letting me know that requirement is exactly opposite, following may help now in same.
awk -F"=" '!$2{gsub(/.* |=/,"",$1);print $1}' Input_file
Related
I have generated 2 .csv files, one containing the original md5sums of some files in a directory and one containing the md5sums calculated at a specific moment.
md5_original.csv
----------
$1 $2 $3
7815696ecbf1c96e6894b779456d330e,,s1.txt
912ec803b2ce49e4a541068d495ab570,,s2.txt
040b7cf4a55014e185813e0644502ea9,,s64.txt
8a0b67188083b924d48ea72cb187b168,,b43.txt
etc.
md5_$current_date.csv
----------
$1 $2 $3
7815696ecbf1c96e6894b779456d330e,,s1.txt
4d4046cae9e9bf9218fa653e51cadb08,,s2.txt
3ff22b3585a0d3759f9195b310635c29,,b43.txt
etc.
* some files could be deleted when calculating current md5sums
I am looking to iterate over the values of column $3 in md5_$current_date.csv and, for each value of that column, to check if it exists in the md5_original.csv and if so, finally to compare its value on $1.
Output should be:
s2.txt hash changed from 912ec803b2ce49e4a541068d495ab570 to 4d4046cae9e9bf9218fa653e51cadb08.
b43.txt hash changed from 8a0b67188083b924d48ea72cb187b168 to 3ff22b3585a0d3759f9195b310635c29.
I have written the script for building this two .csv files, but I am struggling to the awk part where I have to do what I have asked above. I don't know if there is a better way to do this, I am a newbie.
I would use GNU AWK for this task following way, let md5_original.csv content be
7815696ecbf1c96e6894b779456d330e {BLANK_COLUMN} s1.txt
912ec803b2ce49e4a541068d495ab570 {BLANK_COLUMN} s2.txt
040b7cf4a55014e185813e0644502ea9 {BLANK_COLUMN} s64.txt
8a0b67188083b924d48ea72cb187b168 {BLANK_COLUMN} b43.txt
and md5_current.csv content be
7815696ecbf1c96e6894b779456d330e {BLANK_COLUMN} s1.txt
4d4046cae9e9bf9218fa653e51cadb08 {BLANK_COLUMN} s2.txt
3ff22b3585a0d3759f9195b310635c29 {BLANK_COLUMN} b43.txt
then
awk 'FNR==NR{arr[$3]=$1;next}($3 in arr)&&($1 != arr[$3]){print $3 " hash changed from " arr[$3] " to " $1}' md5_original.csv md5_current.csv
output
s2.txt hash changed from 912ec803b2ce49e4a541068d495ab570 to 4d4046cae9e9bf9218fa653e51cadb08
b43.txt hash changed from 8a0b67188083b924d48ea72cb187b168 to 3ff22b3585a0d3759f9195b310635c29
Explanation: FNR is number of row in current file, NR is number of row globally, these are equal only when processing 1st file. When processing 1st file I create array arr so keys are filenames and values are corresponding hash values, next cause GNU AWK to go to next line i.e. no other action is undertaken, so rest is applied only for all but first file. ($3 in arr) is condition: is current $3 one of keys of arr? If it does hold true I print concatenation of current $3 (that is filename) hash changed from string value for key $3 from array arr (that is old hash value) to string $1 (current hash value). If given filename is not present in array arr then no action is undertaken.
Edit: added exclusion for hash which not changed as suggested in comment.
(tested in gawk 4.2.1)
So I have a file named testingFruits.csv with the following columns:
name,value_id,size
apple,1,small
mango,2,small
banana,3,medium
watermelon,4,large
I also have an associative array that stores the following data:
fruitSizes[apple] = xsmall
fruitSizes[mango] = small
fruitSizes[banana] = medium
fruitSizes[watermelon] = xlarge
Is there anyway I can update the 'size' column within the file based on the data within the associative array for each value in the 'name' column?
I've tried using awk but I had no luck. Here's a sample of what I tried to do:
awk -v t="${fruitSizes[*]}" 'BEGIN{n=split(t,arrayval,""); ($1 in arrayval) {$3=arrayval[$1]}' "testingFruits.csv"
I understand this command would get the bash defined array fruitSizes, do a split on all the values, then check if the first column (name) is within the fruitSizes array. If it is, then it would update the third column (size) with the value found in fruitSizes for that specific name.
Unfortunately this gives me the following error:
Argument list too long
This is the expected output I'd like in the same testingFruits.csv file:
name,value_id,size
apple,1,xsmall
mango,2,small
banana,3,medium
watermelon,4,xlarge
One edge case I'd like to handle is the presence of duplicate values in the name column with different values for the value_id and size columns.
If you want to stick to an awk script, pass the array via stdin to avoid running into ARG_MAX issues.
Since your array is associative, listing only the values ${fruitSizes[#]} is not sufficient. You also need the keys ${!fruitSizes[#]}. pr -2 can pair the keys and values in one line.
This assumes that ${fruitSizes[#]} and ${!fruitSizes[#]} expand in the same order, and your keys and values are free of the field separator (, in this case).
printf %s\\n "${!fruitSizes[#]}" "${fruitSizes[#]}" | pr -t -2 -s, |
awk -F, -v OFS=, 'NR==FNR {a[$1]=$2; next} $1 in a {$3=a[$1]} 1' - testingFruits.csv
However, I'm wondering where the array fruitSizes comes from. If you read it from a file or something like that, it would be easier to leave out the array altogether and do everything in awk.
Suppose we have a file test_file.csv:
"261718"|"2017-08-21"|"ramesh_1"|"111"
"261719"|"2017-08-23"|"suresh_1"|"112"
required modified test_file.csv should be :
"261718"|"2017-08-21"|"ramesh"|"111"
"261719"|"2017-08-23"|"suresh"|"112"
How would I find and replace the third column with the required values passed as parameters? It should be within an iteration.
You can save your arguments as comma separated values and store it in a variable args.
Pass this variable to awk using -v option. overwrite the third column $3 with nth array element where n is the current row number
args='"ramesh","suresh"'
awk -F "|" -v args=$args '
BEGIN {
split(args,arr,",")
}
{
$3=arr[NR];OFS=FS;print
}' test_file.csv
Output:
"261718"|"2017-08-21"|"ramesh"|"111"
"261719"|"2017-08-23"|"suresh"|"112"
I trying to put a list into a column by using
set stats = awk '{if(NR>=1) print $1}' $STATFILE
but keep getting the error message "variable name must begin with letter" Any suggestions?
tcsh/csh is expecting no whitespace around the = mark. This would work better:
set stats=awk '{if(NR>=1) print $1}' $STATFILE
However, it still is not workable, since there is more than one token on the right-side of =. You may have meant something lik
set stats=`awk '{if(NR>=1) print $1}' $STATFILE`
using backtic ` to get the output of the awk command. Or you could have meant just a string:
set stats="awk '{if(NR>=1) print $1}' $STATFILE"
Either way, some work is needed.
lets say I have a file looking somewhat like this:
X NeedThis1 KEYWORD
.
.
NeedThis2 X KEYWORD
And I need to combine the two lines into one like this:
NeedThis2 NeedThis1 KEYWORD
It needs to be done for every line in that file that contains the same KEYWORD but it can't combine two lines that look like this (two X's at the first|second position)
X NeedThis1 KEYWORD
X NeedThis2 KEYWORD
I am considering myself bash-noob so any advice if it can be done with something like awk or sed would be appreciated.
awk '
{if ($1 == "X") end[$3] = $2; else start[$3] = $1}
END {for (kw in start) if (kw in end) print start[kw], end[kw], kw}
' file
Try this:
awk '
$1=="X" {key = $NF; value = $2; next}
$2=="X" && $NF==key {print value, $1, key}' file
Explanation:
When a line where first field is X, store the last field as key and second field as value.
Look for the next line where second field is X and last field matches the key stored from pervious action.
When found, print the value of last matched line along with first field of the current line and the key.
This will most definitely break if your data does not match the sample you have shown (if it has more spaces or fields in between), so feel free to adjust as per your needs.
I won't give you the full answer, but if you have some way to identify "KEYWORD" (not in your problem statement), then use a BASH associative array:
declare -A keys
while IFS= read -u3 -r line
do
set -- $line
eval keyword=\$$#
keys[$keyword]+=${line%$keyword}
done
you'll certainly have to do some more fiddling, but your problem statement is incomplete and some of the work needs to be an exercise for the reader.