I have a file with million records and each line ends with SYSTEM;\N.
I want to delete all occurrences of ;\N from file. How can I approach this?
You can use the sed command to replace all the occurrences of the ';\N' from the file and replace it with ''.
sed -i 's/original/new/g' file.txt
Explanation:
sed = Stream EDitor
-i = in-place (i.e. save back to the original file)
The command string:
s = the substitute command
original = a regular expression describing the word to replace (or just the word itself)
new = the text to replace it with
g = global (i.e. replace all and not just the first occurrence)
file.txt = the file name
This worked finally sed -i '' 's/;\\N//g' test112.csv
Related
I got a text like this:
TOKEN = decrypt_aes(
"189272123124aqephkiz3")
And I want to change it into:
TOKEN = "189272123124aqephkiz3"
How can I make this?
I can do this when its in a single line with following command:enter code here`
sed -i "s/decrypt_aes(\(.*\))/\1/g"
But I don`t no how to do when its in multi line
If your file contains only the two lines you showed, then this might help with GNU sed:
sed -i 'N; s/decrypt_aes(\n *//; s/)//' file
From man sed:
N: Append the next line of input into the pattern space.
You can use
sed -i '/.*decrypt_aes($/{N;s/^\([^=]*=\).*\(".*"\))$/\1 \2/}' file
Details:
/.*decrypt_aes($/ - matches a line that ends with decrypt_aes( substring, and if it matches, the block that follows is executed
N - append a newline and the next line to the pattern space
s/^\([^=]*=\).*\(".*"\))$/\1 \2/ - replaces
^\([^=]*=\).*\(".*"\))$ - start of string in the pattern space (^), any zero or more chars other than = and then a = are captured into Group 1 (\1), then any text (.*), then a "..." substring (captured into Group 2 (\2)) and then a ) at the end of string
\1 \2 is the replacement pattern, which is Group 1 value + space + Group 2 value.
See the online demo:
#!/bin/bash
s='TOKEN = decrypt_aes(
"189272123124aqephkiz3")'
sed '/.*decrypt_aes($/{N;s/^\([^=]*=\).*\(".*"\))$/\1 \2/}' <<< "$s"
Output:
TOKEN = "189272123124aqephkiz3"
This might work for you (GNU sed):
sed 'N;s/decrypt_aes.*\(".*"\).*/\1/;P;D' file
Append the following line and if the combined lines matches decrpty_aes followed by "...", replace the match by the string "...".
N.B. If no match is found, each line will be printed as is.
I have a sed expression in a function that I pass parameters to.
insert_after_new_line() {
if ! grep -q "${2}" "${3}"; then
sed -i "/${1}/{N;N;a ${2}}" "${3}"
fi
}
insert_after_new_line search insert textfile.txt
I am trying to have a blank line inserted below the search string and the insert string inserted after.
so
text
text
search
text
becomes
text
text
search
insert
text
but I keep getting the error
sed: -e expression #1, char 0: unmatched `{'
I tested this. works in command line
sed -i '/search/a \\ninsert' file
sed really delimiters commands by a newline. There is a ; but it does not work for all commands, mostly these which take file name as an argument. ; does not work for r R or for example a. Sed will read everything after a command, so sed interprets is like a ${2}} as a single command, in result it does not find enclosing }, cause it's been eaten by a command. You need a newline:
sed -i "/${1}/{N;N;a ${2}
}" "${3}"
or
sed -i "/${1}/{N;N;a ${2}"$'\n'"}" "${3}"
this should work:
sed -i '/search/{G;ainsert
}' file
You can replace the text by shell variable, but replace the single quotes by double quotes too.
I have files that need to be removed from comments and white space until keyword . Line number varies . Is it possible to limit multiple continued sed substitutions based on Keyword ?
This removes all comments and white spaces from file :
sed -i -e 's/#.*$//' -e 's/;.*$//' -e '/^$/d' file
For example something like this :
# string1
# string2
some string
; string3
; string4
####
<Keyword_Keep_this_line_and_comments_white_space_after_this>
# More comments that need to be here
; etc.
sed -i '1,/keyword/{/^[#;]/d;/^$/d;}' file
I would suggest using awk and setting a flag when you reach your keyword:
awk '/Keyword/ { stop = 1 } stop || !/^[[:blank:]]*([;#]|$)/' file
Set stop to true when the line contains Keyword. Do the default action (print the line) when stop is true or when the line doesn't match the regex. The regex matches lines whose first non-blank character is a semicolon or hash, or blank lines. It's slightly different to your condition but I think it does what you want.
The command prints to standard output so you should redirect to a new file and then overwrite the original to achieve an "in-place edit":
awk '...' input > tmp && mv tmp input
Use grep -n keyword to get the line number that contains the keyword.
Use sed -i -e '1,N s/#..., when N is the line number that contains the keyword, to only remove comments on the lines 1 to N.
i want to replace a special string inside a file.txt. my strings are like this :
-> Old String
tech=/lsf/dfg/a.v,/ldf/fgh/b.v
-> New String
tech=$var
i have tried following
sed -i 's/tech=/lsf/dfg/a.v,/ldf/fgh/b.v/tech=$var/g' file.txt
it doesnt work.
sed -i 's#tech=/lsf/dfg/a.v,/ldf/fgh/b.v#tech=$var#g' file.txt
Just replace the delimiters '/' for the sed expression with '#' (or another character that is not in the string you are trying to match and replace).
It took me a while to figure out how to do this, so posting in case anyone else is looking for the same.
For adding a newline after a pattern, you can also say:
sed '/pattern/{G;}' filename
Quoting GNU sed manual:
G
Append a newline to the contents of the pattern space, and then append the contents of the hold space to that of the pattern space.
EDIT:
Incidentally, this happens to be covered in sed one liners:
# insert a blank line below every line which matches "regex"
sed '/regex/G'
This sed command:
sed -i '' '/pid = run/ a\
\
' file.txt
Finds the line with: pid = run
file.txt before
; Note: the default prefix is /usr/local/var
; Default Value: none
;pid = run/php-fpm.pid
; Error log file
and adds a linebreak after that line inside file.txt
file.txt after
; Note: the default prefix is /usr/local/var
; Default Value: none
;pid = run/php-fpm.pid
; Error log file
Or if you want to add text and a linebreak:
sed -i '/pid = run/ a\
new line of text\
' file.txt
file.txt after
; Note: the default prefix is /usr/local/var
; Default Value: none
;pid = run/php-fpm.pid
new line of text
; Error log file
A simple substitution works well:
sed 's/pattern.*$/&\n/'
Example :
$ printf "Hi\nBye\n" | sed 's/H.*$/&\nJohn/'
Hi
John
Bye
To be standard compliant, replace \n by backslash newline :
$ printf "Hi\nBye\n" | sed 's/H.*$/&\
> John/'
Hi
John
Bye
sed '/pattern/a\\r' file name
It will add a return after the pattern while g will replace the pattern with a blank line.
If a new line (blank) has to be added at end of the file use this:
sed '$a\\r' file name
Another possibility, e.g. if You don't have an empty hold register, could be:
sed '/pattern/{p;s/.*//}' file
Explanation:
/pattern/{...} = apply sequence of commands, if line with pattern found,
p = print the current line,
; = separator between commands,
s/.*// = replace anything with nothing in the pattern register,
then automatically print the empty pattern register as additional line)
The easiest option -->
sed 'i\
' filename