This is what I have so far, tried multiple ways but can't get it just right.
My goal is to sanitize the input to prevent problems while inputting to mysql from text file
cat 'file.txt' | awk '{gsub(/'"'"'/, '.') ; print $0}' > 'file_sanitized.txt'
What you want is:
awk '{gsub(/\047/,".")}1' file
See http://awk.freeshell.org/PrintASingleQuote.
Since you requested "how to escape single quote using awk" here's a solution using awk:
awk '{ gsub("\x27", ".");print $0}'
where \x27 is the escape sequence representation of the hexadecimal value 27 (a single quote).
For a list of all escape sequences see https://www.gnu.org/software/gawk/manual/html_node/Escape-Sequences.html
I'm not sure if I got the problem correctly. If you want to replace all occurrences of a single quote by a dot, use tr:
tr "'" "." < file.txt > sanitized.txt
If you want to escape the single quote with a backslash use sed like this:
sed "s/'/\\\'/g" file.txt > sanitized.txt
Note: Please take the advice from CharlesDuffy seriously. This is far from a stable and safe solution to escape values for an SQL import.
You have used wrong quoting:
awk '{gsub(/'"'"'/, "."); print}'
Related
I need to write a bash script to list values from an sql database.
I've got so far but now I need to get the rest of the way.
The string so far is
10.255.200.0/24";i:1;s:15:"10.255.207.0/24";i:2;s:14:"192.168.0.0/21
I now need to delete everything between the speech marks and send it to a new line.
desired output:
10.255.200.0/24
10.255.207.0/24
192.168.0.0/21
any help would be greatly appreciated.
$ tr '"' '\n' <<< $string | awk 'NR%2'
10.255.200.0/24
10.255.207.0/24
192.168.0.0/21
You could use :
echo 'INPUT STRING HERE' | sed $'s/"[^"]*"/\\\n/g'
Explanation :
sed 's/<PATTERN1>/<PATTERN2/g' : we substitute every occurrence of PATTERN1 by PATTERN2
[^"]*: any character that is not a ", any number of time
\\\n: syntax for newline in sed (reference here)
Considering that your Input_file is same as shown sample then could you please try following.
awk '
{
while(match($0,/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\/[0-9]+/)){
print substr($0,RSTART,RLENGTH)
$0=substr($0,RSTART+RLENGTH)
}
}' Input_file
This might work for you (GNU sed):
sed 's/"[^"]*"/\n/g' file
Or using along side Bash:
sed $'/"[^"]*"/\\n/g' file
Or using most other sed's:
sed ':a;/"[^"]*"\(.*\)\(.\|$\)/{G;s//\2\1/;ba}' file
This uses the feature that an unadulterated hold space contains a newline.
Need help in Bash to filter records based on a multicolumn delimiter.
Delimiter is |^^|
Sample record
xyz#ATT.NET|^^|xyz|^^|307
Awk runs file when used with single character delimiter but not with multi character.
awk -F"|^^|" "NF !=3 {print}" file.txt
Any suggestions?
The issue is that every character in your delimiter is a regexp metacharacter so you need to escape them when appropriate so awk knows you want them treated literally. This might be overkill:
awk -F'\\|\\^\\^\\|' 'NF!=3' file.txt
but I can't test it since you only provided one line of input, not the selection of lines some of which do/don't match that'd be required to test the script.
awk -F "<regex>" ...
It is not a multicolumn delimiter, is is a regular expression
simple regex,
such as match this single char are what you get use to,
but not all there is.
One way is to escape all the regex characters as #Ed Morton answered.
Alternatively,
you can replace all |^^| with a single character which never shows in your file content, here let's say a comma
sed 's/|^^|/,/g' file.txt
xyz#ATT.NET,xyz,307
The command would be
sed 's/|^^|/,/g' file.txt | awk -F, 'NF != 3'
I'm having a hard time finishing my script since there's this part which doesn't function the way I wanted it to be.
I have this line in my script:
cat /home/tmp/temp1.txt | awk '{gsub("~",RS);gsub("*",RS);print}' > /home/tmp/temp.txt
It works fine, yes.
But when I do something like this:
cat /home/tmp/temp1.txt | awk '{gsub("|",RS);print}' > /home/tmp/temp.txt
It's not working at all.
I wanted to change all my vertical bars into new line and yet I can't achieve it. Please help me with this. Thanks
You can do all the replacements in a single awk like this:
awk '{gsub(/[*~|]/, RS)} 1' /home/tmp/temp1.txt
Pipe is otherwise used for regex alternation that needs escaping. However inside the character class [...] pipe or asterisk need not be escaped as shown above.
It is also better to use /.../ regex literal in gsub function instead of quoted string.
If you really only want to replace vertical bars with newlines, you can do that much more succinctly with tr which translates characters:
echo "hi|there|my|friend" | tr '|' '\n'
hi
there
my
friend
Or, if you are using a file:
tr '|' '\n' < /home/tmp/temp.txt
Below is my sample data, in a file called 'tt':
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$"
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$"
I want to replace "HISTOGRAMBIN_#####002__UDBDI02$" with "HISTOGRAMBIN_#####002__UDBDI02$_MIG"
Why does this awk command intended to do a substitution not work?
awk '/HISTOGRAMBIN_#####002__UDBDI02\$/ {sub("'HISTOGRAMBIN_#####002__UDBDI02\$'","'HISTOGRAMBIN_#####002__UDBDI02\$_MIG'")} {print}' tt
EDIT: The awk command here is used to replace the object name picked from another file. $ can be anywhere in the name, but i want _mig to be at the end. My actual awk command which i use in my script is below:
awk '/"'${TAB_NAME}'"/{if (M==""){sub("'${TAB_NAME}'","'${TAB_NAME}_${TAB_EXT}'");M=1}}{print}' filename
I have to use "" to expand awk to use shell variables and I just want to replace the first occurrence.
in your awk line, the quotes were not correctly used. do
awk '{sub(/H....\$/, "replacement")}7' file
You could use sed for this purpose and you don't need to replace the whole string just replacing $ with $_MG is enough,
$ sed -r 's/^([^$]*\$)(.*)$/\1_MG\2/g' file
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$_MG"
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$_MG"
OR
A simpler one,
$ sed 's/\$/$_MG/g' file
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$_MG"
"UDBPEM1 "."HISTOGRAMBIN_#####002__UDBDI02$_MG"
awk '{sub(/\$/,"$_MG")}1' file
sed 's/\$/$_MG/' file
When you wrote:
awk '...sub("'HISTOGRAMBIN_#####002__UDBDI02\$'",...)...' file
The single quotes withing your sub() were causing your shell script to exit and re-enter awk the awk script specification so the HISTOGRAMBIN_#####002__UDBDI02\$ was actully being interpreted by shell first and THEN because you had the RE enclosed in double quotes instead of RE delimiters (/) the resulting string was being interpreted by awk twice, once when the script was read and then again when it was executed. So - you'd have had to escape that $ at least 3 times before awk stood a chance of it still being escaped when the script was executed.
Thanks Guys,
I got my awk command working
awk '/'${TAB_NAME}'/ {if (M==""){sub(/'$TAB_NAME'/,"'${TAB_NAME}_${TAB_EXT}'");M=1}}{print}' tt
Thanks kent, Ed Morton and avinash-raj for help.
How to use awk variable in search?
Name="jony"
awk -v name="$Name" '/name/ {print $0}' file
this will search for string name, not for $Name which is actually jony.
Correct, awk won't recogize variables in / /. You can do:
Name="jony"
awk -v name="$Name" '$0 ~ name' file
Since print is awk's default behavior we can avoid using it here.
Hope I understood problem correctly:
Why wont you try following one:
awk '/'"$Name"'/ { print } ' testfile
When writing an AWK one-liner, you could quote the script with either the single quotes or double quotes. In the latter case the shell does all the substitution directly so that you do not need to pass the variable into the script via -v option:
Name="jony"
awk "/$Name/" file
# this works. after shell has performed substitutions, the line looks like
awk "/jony/" file
[bad!] Or even without quotes if the name does not contain spaces:
awk /$Name/ file
All the simplicity vanishes as soon as you want to use $ in the script, including awk special variables that use $0, $1, etc, because you will have to escape the dollar sign to prevent shell variable expansion.
awk "/$Name/ {print \$0}"
In addition you will have to escape the double quotes to add literal text to the script. Looks clumsy:
awk "/$Name/ {print \"Found in: \" \$0}"
To crown it all, negating regular expression with double quotes will cause a shell error:
awk "!/$Name/"
#error> ... event not found ...
The error will happen if $Name itself contains ! sign. This makes using double quotes unreliable.
So, to be on the safe side, prefer single quotes :)