modularize awk script to mask sensitive data in delimited file - bash

Question

I have a delimited file in the below format:
text1|12345|email#email.com|01-01-2020|1
Considering all the fields are sensitive data, i had written the following awk command to mask the first field with random data.
awk -F'|' -v cmd="strings /dev/urandom | tr -dc '0-9' | fold -w 5" 'BEGIN {OFS=FS} {cmd | getline a;$1=a;print}' source.dat > source_masked.dat
If i want to mask additional fields I add the following.
awk -F'|' -v cmd1="strings /dev/urandom | tr -dc '0-9' | fold -w 5" -v cmd2="strings /dev/urandom | tr -dc 'A-Za-z0-9' | fold -w 7" 'BEGIN {OFS=FS} {cmd | getline a; cmd2 | getline b;$2=b}' source.dat > source_masked.dat
How do i scale it if i want to mask 100s of columns with different datatypes?
Basically, i want to take the following from config file:
column number, datatype, length
and use it in the awk to generate the commands and the replacement script dynamically.
Could you please advice on the same.
I rewrote the same accepted answer on awk as it took a long time to mask larger files using bash.
The code for the same is:
function mask(datatype, precision) {
switch (datatype) {
case "string":
command = "strings /dev/urandom | tr -dc '[:alpha:]' | fold -w "
precision
break
case "alphaNumeric":
command = "strings /dev/urandom | tr -dc '[:alnum:]' | fold -w "
precision
break
case "number":
command = "strings /dev/urandom | tr -dc '[:digit:]' | fold -w "
precision
break
default:
command = "strings /dev/urandom | tr -dc '[:alnum:]' | fold -w "
precision
}
command | getline v
return v
}
BEGIN {
while ((getline line < "properties.conf") > 0) {
split(line, a, ",")
col = a[1]
type = a[2]
len = a[3]
masks[col] = type " "
len
}
IFS = "|"
OFS = "|"
} {
for (i = 1; i <= NF; i++) {
if (masks[i] != "") {
split(masks[i], m, " ")
$i = mask(m[1], m[2])
}
}
print
}

Answer 1

One approach is to read the mask configuration file into an array indexed by column number.
Then, read the data file line by line. put each field in a second array. Then, for each element of the mask array, randomize the appropriate data field. When all fields are updated, output the new line and move on to the next line.
Does this have to be done in awk? It might be easier/quicker to do it in native bash:
#!/bin/bash
declare mask_file=masks.conf
declare input_file=input.dat
declare output_file=output.dat
function create_mask() {
# ${1} is type, ${2} is length
case ${1} in
string ) ;;
date ) ;;
number ) ;;
* ) ;;
esac
}
while read column type length; do
masks[${column}]="${type} ${length}"
done < ${mask_file}
IFS='|'
while read -a data; do
for column in ${!masks[#]}; do
data[${column}]=$(create_mask ${masks[${column}]})
done
echo "${data[*]}" # Uses IFS as output separator.
done < ${input_file} > ${output_file}
I have not included the full contents of the create_mask() function, as I do not know what types you plan to support or format you want for each type.

Answer 2

You can use the built-in rand function instead to generate a random number.
Define an associative array with the list of fields that you want to mask.
E.g. here is a sample code that will mask field 1 & 4
awk -F\| '
BEGIN {
A_mask_field[1]
A_mask_field[4]
}
{
for ( i = 1; i <= NF; i++ )
{
if ( i in A_mask_field )
$i = sprintf( "%d", rand() * length($i) * 100000 )
}
}
1
' OFS=\| file