I have sql file with this strings :
(17, 14, '2015-01-20 10:38:40', 211, 'Just text\n\nFrom: Support <support#domain.com>\n Send: 20 Jan 2015 year. 10:33\n To: Admin\n Theme: [TST #0000014] Just text \n\nJust text: Text\n Test text test text\n\nJust text:\n Text\n\n-- \n Test\n Text.\n Many text words 0.84.2', 0, 2);
I want remove all text between symbols \n\ and ', 0, 2);
I want get this result:
(17, 14, '2015-01-20 10:38:40', 211, 'Just text', 0, 2);
How I can do it via sed?
I try use this example - cat file | sed 's/<b>.*</b>//g'. I changed <b> to \n\ and </b> to ', 0, 2); But it dont work, I get error in console
Thanks in advance!
You can try this command
sed 's/\\n\\.*\('\'', 0, 2);\)/\1/g' FileName
Output :
(17, 14, '2015-01-20 10:38:40', 211, 'Just text', 0, 2);
You have to escape the single quotes like '\'' as well as back slash \\
If you can find it, you can replace it with nothing.
So, depending on what you mean by \n and what you need to escape, you want something like sed 's/\\n\\.*'//g'.
Obviously, take care that this is really what you want to replace on every line. It might be worth searching for the target \\n\\.*' first, to make sure it doesn't accidentally grab too much on an unexpected line.
Related
I'm trying to remove a large amount of text from a file before inserting a few new lines. I can delete everything after the word 'CParticleSystemDefinition' with a single line of code like this
perl -0777 -pi -we "s/CParticleSystemDefinition\x22\K.*/\n}/s" "D:\Steam\steamapps\common\dota 2 beta\content\dota_addons\custom\particles\generic_gameplay\winter_effects_creep.vpcf"
But when I try to change the code slightly so that it adds a few new lines like this, it doesn't work
perl -0777 -pi -we "s/CParticleSystemDefinition\x22\K.*/\n m_Children = \n [\n {\n m_ChildRef = resource:\x22particles/generic_gameplay/winter_effects_breath.vpcf\x22\n },\n ]\n}/s" "D:\Steam\steamapps\common\dota 2 beta\content\dota_addons\custom\particles\generic_gameplay\winter_effects_creep.vpcf"
So, basically, what I want to do is make this file
{
_class = "CParticleSystemDefinition"
m_bShouldHitboxesFallbackToRenderBounds = false
m_nMaxParticles = 24
m_flConstantRadius = 15.000000
m_flConstantLifespan = 0.500000
m_ConstantColor =
[
212,
170,
145,
255,
]
m_bShouldSort = false
m_Renderers =
[
{
_class = "C_OP_RenderSprites"
m_nSequenceCombineMode = "SEQUENCE_COMBINE_MODE_USE_SEQUENCE_0"
m_bMod2X = true
m_nOrientationType = 3
m_hTexture = resource:"materials/particle/footprints/footprints_generic.vtex"
m_flAnimationRate = 1.000000
},
]
m_Emitters =
[
{
_class = "C_OP_ContinuousEmitter"
m_flEmitRate = 10.000000
m_flStartTime = 0.500000
m_nScaleControlPoint = 5
},
]
}
look like this
{
_class = "CParticleSystemDefinition"
m_Children =
[
{
m_ChildRef = resource:"particles/generic_gameplay/winter_effects_breath.vpcf"
},
]
}
Do it in two steps -- clear the rest of the file after that phrase, then add the desired text
perl -0777 -i.bak -wpe"s{Definition\x22\K.*}{}s; $_ .= qq(\n\tm_Children...)" file
where I've used ellipses to indicate the rest, for clarity. I added .bak to keep a backup file, until this is tested well enough.
Adding a string in the replacement part is fine as well of course -- I don't readily see what fails (and how?) in your code. Breaking it up into two steps simply makes it easier to review and organize it better but one can also run that code in the replacement part, using /e modifier
perl -0777 -i.bak -wpe"
s{Definition\x22\K.*}{
# any valid Perl code, what it evaluates to is used as replacement
qq(\n\tm_Children...)
}es;
" file
If you don't want tabs, which may or may not get expanded depending on various settings and on what's done with this, can prepare and use a string of spaces instead. Then we might as well build the replacement more systematically
perl -0777 -i.bak -wpe"
s{Definition\x22\K.*}{}s;
$s4 = q( ) x 4; # four spaces
$_ .= qq(\n${s4}m_Children =\n$s4) . join qq(\n$s4),
q([),
q({),
qq($s4).q(m_ChildRef = ...) # etc
qq(\n)
" file
Now one can either make this into a better system (adding a suitable programming construct for each new level of indentation for example, like map over such lines so to add indentation to all in one statement), if there is a lot -- or condense it if there's really just a few lines.
Again, this can run inside the regex's replacement side, with the additional /e modifier.
This can be done line-by-line in one pass as well, using the read-write (+<) mode for open
perl -MPath::Tiny -wE"
$f = shift // die qq(Need a filename);
open $fh, qq(+<), $f or die qq(Cant open $f: $!);
while (<$fh>) { last if /Definition\x22$/ }; # past the spot to keep
truncate $fh, tell($fh); # remove the rest
say qq(File now:\n), path($f)->slurp; # (just to see it now)
say $fh $_ for # add new content
qq(new line 1),
qq(new line 2)
" filename
(Carefully with read-write modes. Please read the docs with care first.)
I have a string containing a Python stack trace like this (with some irrelevant text before and after):
Traceback (most recent call last):
File "/workspace/r111.py", line 232, in test_assess
exec(code)
File "a111.py", line 17, in
def reset(self):
File "/workspace/r111.py", line 123, in failed
raise AssertionError(msg)
AssertionError: Dein Programm funktioniert nicht. Python sagt:
Traceback (most recent call last):
File "a111.py", line 6, in
File "/workspace/r111.py", line 111, in runcaptured
exec(c, variables)
File "", line 1, in
ZeroDivisionError: division by zero
Now I want to extract the line in which the error occurred (1 extracted from File "", line 1) using a multiline RegEx (in Ruby).
/File ".*", line ([0-9]+)/ works nicely, but matches all occurrences. I only want the last. Iterating over the matches in the target environment is not a valid solution, as I can't change the business logic there.
You may use
/(?m:.*)(?-m:File ".*", line ([0-9]+))/
Details
(?m:.*) - a modifier group where the multiline flag is on and the dot matches any char including line break chars that matches any zero or more chars as many as possible up to the last occurrence of the subsequent subpatterns
(?-m:File ".*", line ([0-9]+)) - another modifier group where the multiline flag is off and the dot now matches any char but line break chars:
File - a literal substring with a space after it
".*" - a double quote, any zero or mmore chars other than linebreaks and then another double quote
, line - comma, space, "line" substring
([0-9]+) -Group 1 capturing one or more digits.
There are dozens and dozens of posts in a lot of forums dealing with the command "sed".
Unfortunately, no one tutorial, no one post solved my very basic problem, I hope I will finally find a solution here :)
Here it is, I have hundreds of lines like this one :
INSERT [dbo].[TABLE_ONE] ([TABLE_ONE_ID], [TABLE_ONE_MNEMO], [TABLE_ONE_DESC], [AUTO_ANALYSIS], [SCOPE_LEVEL], [IN_ABV], [TABLE_ONE_ORDER], [REPORT_ETE]) VALUES (204, N'PERFO TEST', N'PERFO TEST', N'N', 1, 0, 999, N'70')
I just want to remove the string 'dbo' and the N prefix, so I would like to replace :
204, N'PERFO TEST', N'PERFO TEST', N'N', 1, 0, 999, N'70'
by :
204, 'PERFO TEST', 'PERFO TEST', 'N', 1, 0, 999, '70'
For the 'dbo' string, I found a solution but I don't like it because if there is another d, another b or another o in the text they also will be removed.
sed -i 's/[dbo]//g' file
Now, if I follow these instructions, according to a tutorial (http://www.theunixschool.com/2014/08/sed-examples-remove-delete-chars-from-line-file.html), it should work but it doesn't :
sed -i 's/dbo//g' file
It just... doesn't do anything ! So if you have a better solution, it would be great !
Now, for the N prefix, I just want to delete all the letters N that are after the coma, just like in the example above. If you have the solution, please do not hesitate to share it, I tried a looooot of different syntaxes and I just could delete all the N of the file haha :)
Thank you for your time !
Try this one sed 's/dbo//g' | sed 's/, N/, /g':
➜ ~ echo "INSERT [dbo].[TABLE_ONE] ([TABLE_ONE_ID], [TABLE_ONE_MNEMO], [TABLE_ONE_DESC], [AUTO_ANALYSIS], [SCOPE_LEVEL], [IN_ABV], [TABLE_ONE_ORDER], [REPORT_ETE]) VALUES (204, N'PERFO TEST', N'PERFO TEST', N'N', 1, 0, 999, N'70')" | sed 's/dbo//g' | sed 's/, N/, /g'
I have control file with following option...
OPTIONS(DIRECT=TRUE,ROWS=100,BINDSIZE=209700000,readsize=209700000)
load data
infile 'd:\test.DH'
"str '\n'"
append
into table name
FIELDS TERMINATED by '!'
OPTIONALLY ENCLOSED by '"'
trailing nullcols
sample or records, the terminator is "!"
9334!376!15950!9109!0!29109!109!0!!10003!05.02.2015 03:51:27!05.02.2015 03:51:46!05.02.2015 03:51:27!0!0!0!S!00c08309ed178b3f!005683540!6829109!079015!0!0!!!0!F!299!!!0!0!!0!-1, 0, -1, 1423075906663, 0, 0, 0!{, 1, 24307, 3000-12-31 23:59:59, 0}!!{60200103, 0, 0, 0, 0, 0, 2, 2, 1, 0, 0}!!!!!!!!!!!!!!!!=2, =0, =0, =0, =255, =829109, =510, , =1!1!00067!!!F,079015,,2993007,290009,5,02993007,005683540,6,6829109,,,,010006743081,0,10006743081,5,F,,,,290009,2079015,2079015,829109,93007,079015,2079015,829109,0,,0,07000,,,0,,,,,'00c08309ed178b3fH',,,,,,,0,0,0,0,0,0,0,299,0,,a2040000005b6424,7205,36899550,
338!8376!11230!333777!0!33777!333777!0!!10003!05.02.2015 03:51:04!05.02.2015 03:51:14!05.02.2015 03:51:04!0!0!0!S!6d!004382577!3333777!3407582!0!0!!!0!F!299!!!0!0!!0!-1, 0, -1, 1423075874285, 0, 0, 0!{, 1, 24927, 3000-12-31 23:59:59, 0}!!{60200103, 0, 0, 0, 0, 0, 2, 2, 1, 0, 0}!!!!!!!!!!!!!!!!=2, =0, =0, =0, =255, =33777, =600, , =1!10595!02020!!!F,3407582,,993001,20000,5,993001,004382577,6,3333777,,,,010595,0,0202010595,5,F,,,,220000,407582,407582,33777,993001,,407582,03407582,3333777,0,,0,5874000,,,0,,,,,'6dH',,,,,,,0,0,0,0,0,0,0,299,0,_1281,a2820000005d213d,7205,36899550,
when I run this in I get exceed
Record 1: Rejected - Error on table name, column logs.
Field in data file exceeds maximum length
this field is the last column of the record... the column is 3000Byte... I know it' snot the issue of the length of the record as I tried importing the same file with [b]navicate [/b]and it loaded all without any issue... there something wrong with [b]str[/b]... and it try to load all data on column [b]logs[/b]
I tried
"str '\t'"
"str '\r'"
"str '\n'"
and none of them helped me... thanks for your time
thanks
thanks I found the issue, I had to mentioned number of character in control file infront of the field i.e
OPTIONS(DIRECT=TRUE,ROWS=100,BINDSIZE=209700000,readsize=209700000)
load data
infile 'd:\test.DH'
"str '\n'"
append
into table name
FIELDS TERMINATED by '!'
OPTIONALLY ENCLOSED by '"'
trailing nullcols
(log char(1500))
Thanks everyone for readying :D
I would like to turn this string
"P07091 MMCNEFFEG
P06870 IVGGWECEQHS
SP0A8M0 VVPVADVLQGR
P01019 VIHNESTCEQ"
into an array that looks like in ruby.
["P07091 MMCNEFFEG", "P06870 IVGGWECEQHS", "SP0A8M0 VVPVADVLQGR", "P01019 VIHNESTCEQ"]
using split doesn't return what I would like because of the line breaks.
This is one way to deal with blank lines:
string.split(/\n+/)
For example,
string = "P07091 MMCNEFFEG
P06870 IVGGWECEQHS
SP0A8M0 VVPVADVLQGR
P01019 VIHNESTCEQ"
string.split(/\n+/)
#=> ["P07091 MMCNEFFEG", "P06870 IVGGWECEQHS",
# "SP0A8M0 VVPVADVLQGR", "P01019 VIHNESTCEQ"]
To accommodate files created under Windows (having line terminators \r\n) replace the regular expression with /(?:\r?\n)+/.
I like to use this as a pretty generic method for handling newlines and returns:
lines = string.split(/\n+|\r+/).reject(&:empty?)
string = "P07091 MMCNEFFEG
P06870 IVGGWECEQHS
SP0A8M0 VVPVADVLQGR
P01019 VIHNESTCEQ"
Using CSV::parse
require 'csv'
CSV.parse(string).flatten
# => ["P07091 MMCNEFFEG", "P06870 IVGGWECEQHS", "SP0A8M0 VVPVADVLQGR", "P01019 VIHNESTCEQ"]
Another way using String#each_line :-
ar = []
string.each_line { |line| ar << line.strip unless line == "\n" }
ar # => ["P07091 MMCNEFFEG", "P06870 IVGGWECEQHS", "SP0A8M0 VVPVADVLQGR", "P01019 VIHNESTCEQ"]
Building off of #Martin's answer:
lines = string.split("\n").reject(&:blank?)
That'll give you only the lines that are valued
Split can take a parameter in the form of the character to use to split, so you can do:
lines = string.split("\n")
I think it should be noted that in some situations, line breaks can include not only newlines (\n) but also carriage returns (\r) and that there could potentially be any combination or quantity thereof. Let's take the following string for example:
str = "Useful Line 1 ....
Useful Line 2
Useful Line 3
Useful Line 4... \n
Useful Line 5\r \n
Useful Line 6\n\r
Useful Line 7\n\r\n\r
Useful Line 8 \r\n\r\n
Useful Line 9\r\r\r Useful Line 10\n\n\n\n\nUseful Line 11 \r Useful Line 12"
To deal with all instances of \n and \r, I would do the following to replace all instances of \r with \n using gsub, and then I would combine all consecutive instances of \n using squeeze(arg):
str.gsub("\r", "\n").squeeze("\n")
which would result in :
#=>
"Useful Line 1 ....
Useful Line 2
Useful Line 3
Useful Line 4...
Useful Line 5
Useful Line 6
Useful Line 7
Useful Line 8
Useful Line 9
Useful Line 10
Useful Line 11
Useful Line 12"
...which brings me to our next issue. Sometimes those extra line breaks contain unwanted whitespace and not truly blank or empty lines. To deal with not only line breaks but also unwanted empty lines, I would add the each_line, reject, and strip method like so:
str.gsub("\r", "\n").squeeze("\n").each_line.reject{|x| x.strip == ""}.join
which would result in the desired string:
#=>
Useful Line 1 ....
Useful Line 2
Useful Line 3
Useful Line 4...
Useful Line 5
Useful Line 6
Useful Line 7
Useful Line 8
Usefule Line 9
Useful Line 10
Useful Line 11
Useful Line 12
Now more specifically to the OP, we could then simply use split("\n") to finish it all off (as was already mentioned by others):
str.gsub("\r", "\n").squeeze("\n").each_line.reject{|x| x.strip == ""}.join.split("\n")
or we could simply skip straight to the desired array by replacing each_line with map and leaving off the unnecessary join like so:
str.gsub("\r", "\n").squeeze("\n").split("\n").map.reject{|x| x.strip == ""}
both of which would result in:
#=>
["Useful Line 1 ....", " Useful Line 2", "Useful Line 3", " Useful Line 4... ", "Useful Line 5", " Useful Line 6", "Useful Line 7", " Useful Line 8 ", "Usefule Line 9", " Useful Line 10", "Useful Line 11 ", " Useful Line 12"]
NOTE:
You may also want to strip off leading and trailing whitespace from each line in which case we could replace .join.split("\n") with .map(&:strip) like so:
str.gsub("\r", "\n").squeeze("\n").each_line.reject{|x| x.strip == ""}.map(&:strip)
or
str.gsub("\r", "\n").squeeze("\n").split("\n").map.reject{|x| x.strip == ""}.map(&:strip)
which would both result in:
#=>
["Useful Line 1 ....", "Useful Line 2", "Useful Line 3", "Useful Line 4...", "Useful Line 5", "Useful Line 6", "Useful Line 7", "Useful Line 8", "Usefule Line 9", "Useful Line 10", "Useful Line 11", "Useful Line 12"]