sed command to remove some text - bash

There are dozens and dozens of posts in a lot of forums dealing with the command "sed".
Unfortunately, no one tutorial, no one post solved my very basic problem, I hope I will finally find a solution here :)
Here it is, I have hundreds of lines like this one :
INSERT [dbo].[TABLE_ONE] ([TABLE_ONE_ID], [TABLE_ONE_MNEMO], [TABLE_ONE_DESC], [AUTO_ANALYSIS], [SCOPE_LEVEL], [IN_ABV], [TABLE_ONE_ORDER], [REPORT_ETE]) VALUES (204, N'PERFO TEST', N'PERFO TEST', N'N', 1, 0, 999, N'70')
I just want to remove the string 'dbo' and the N prefix, so I would like to replace :
204, N'PERFO TEST', N'PERFO TEST', N'N', 1, 0, 999, N'70'
by :
204, 'PERFO TEST', 'PERFO TEST', 'N', 1, 0, 999, '70'
For the 'dbo' string, I found a solution but I don't like it because if there is another d, another b or another o in the text they also will be removed.
sed -i 's/[dbo]//g' file
Now, if I follow these instructions, according to a tutorial (http://www.theunixschool.com/2014/08/sed-examples-remove-delete-chars-from-line-file.html), it should work but it doesn't :
sed -i 's/dbo//g' file
It just... doesn't do anything ! So if you have a better solution, it would be great !
Now, for the N prefix, I just want to delete all the letters N that are after the coma, just like in the example above. If you have the solution, please do not hesitate to share it, I tried a looooot of different syntaxes and I just could delete all the N of the file haha :)
Thank you for your time !

Try this one sed 's/dbo//g' | sed 's/, N/, /g':
➜ ~ echo "INSERT [dbo].[TABLE_ONE] ([TABLE_ONE_ID], [TABLE_ONE_MNEMO], [TABLE_ONE_DESC], [AUTO_ANALYSIS], [SCOPE_LEVEL], [IN_ABV], [TABLE_ONE_ORDER], [REPORT_ETE]) VALUES (204, N'PERFO TEST', N'PERFO TEST', N'N', 1, 0, 999, N'70')" | sed 's/dbo//g' | sed 's/, N/, /g'

Related

How do I add new lines after deleting a large amount of text in Perl windows?

I'm trying to remove a large amount of text from a file before inserting a few new lines. I can delete everything after the word 'CParticleSystemDefinition' with a single line of code like this
perl -0777 -pi -we "s/CParticleSystemDefinition\x22\K.*/\n}/s" "D:\Steam\steamapps\common\dota 2 beta\content\dota_addons\custom\particles\generic_gameplay\winter_effects_creep.vpcf"
But when I try to change the code slightly so that it adds a few new lines like this, it doesn't work
perl -0777 -pi -we "s/CParticleSystemDefinition\x22\K.*/\n m_Children = \n [\n {\n m_ChildRef = resource:\x22particles/generic_gameplay/winter_effects_breath.vpcf\x22\n },\n ]\n}/s" "D:\Steam\steamapps\common\dota 2 beta\content\dota_addons\custom\particles\generic_gameplay\winter_effects_creep.vpcf"
So, basically, what I want to do is make this file
{
_class = "CParticleSystemDefinition"
m_bShouldHitboxesFallbackToRenderBounds = false
m_nMaxParticles = 24
m_flConstantRadius = 15.000000
m_flConstantLifespan = 0.500000
m_ConstantColor =
[
212,
170,
145,
255,
]
m_bShouldSort = false
m_Renderers =
[
{
_class = "C_OP_RenderSprites"
m_nSequenceCombineMode = "SEQUENCE_COMBINE_MODE_USE_SEQUENCE_0"
m_bMod2X = true
m_nOrientationType = 3
m_hTexture = resource:"materials/particle/footprints/footprints_generic.vtex"
m_flAnimationRate = 1.000000
},
]
m_Emitters =
[
{
_class = "C_OP_ContinuousEmitter"
m_flEmitRate = 10.000000
m_flStartTime = 0.500000
m_nScaleControlPoint = 5
},
]
}
look like this
{
_class = "CParticleSystemDefinition"
m_Children =
[
{
m_ChildRef = resource:"particles/generic_gameplay/winter_effects_breath.vpcf"
},
]
}
Do it in two steps -- clear the rest of the file after that phrase, then add the desired text
perl -0777 -i.bak -wpe"s{Definition\x22\K.*}{}s; $_ .= qq(\n\tm_Children...)" file
where I've used ellipses to indicate the rest, for clarity. I added .bak to keep a backup file, until this is tested well enough.
Adding a string in the replacement part is fine as well of course -- I don't readily see what fails (and how?) in your code. Breaking it up into two steps simply makes it easier to review and organize it better but one can also run that code in the replacement part, using /e modifier
perl -0777 -i.bak -wpe"
s{Definition\x22\K.*}{
# any valid Perl code, what it evaluates to is used as replacement
qq(\n\tm_Children...)
}es;
" file
If you don't want tabs, which may or may not get expanded depending on various settings and on what's done with this, can prepare and use a string of spaces instead. Then we might as well build the replacement more systematically
perl -0777 -i.bak -wpe"
s{Definition\x22\K.*}{}s;
$s4 = q( ) x 4; # four spaces
$_ .= qq(\n${s4}m_Children =\n$s4) . join qq(\n$s4),
q([),
q({),
qq($s4).q(m_ChildRef = ...) # etc
qq(\n)
" file
Now one can either make this into a better system (adding a suitable programming construct for each new level of indentation for example, like map over such lines so to add indentation to all in one statement), if there is a lot -- or condense it if there's really just a few lines.
Again, this can run inside the regex's replacement side, with the additional /e modifier.
This can be done line-by-line in one pass as well, using the read-write (+<) mode for open
perl -MPath::Tiny -wE"
$f = shift // die qq(Need a filename);
open $fh, qq(+<), $f or die qq(Cant open $f: $!);
while (<$fh>) { last if /Definition\x22$/ }; # past the spot to keep
truncate $fh, tell($fh); # remove the rest
say qq(File now:\n), path($f)->slurp; # (just to see it now)
say $fh $_ for # add new content
qq(new line 1),
qq(new line 2)
" filename
(Carefully with read-write modes. Please read the docs with care first.)

Create all combinations of 4 elements out of a larger list

I want to create all possible combinations of 4 chemical elements out of a list of 9 and use them to create folders named after these combinations.
The desired list looks something like this:
{Cr, Hf, Mo, Nb, Ta, Ti, V, W, Zr}
What I want to get out of it, would be:
CrHfMoNb
CrHfMoTa
CrHfMoTi
CrHfMoV
...
TiVWZr
and so on for all 126 possible arrangements, stored in a list or something similar so that I can use it as input for creating the folders. These combinations should be ordered alphabetically, so that Hf always comes after Cr and before Ti for example.
I can use both Bash and Python, I prefer the simpler method. If the method could easily be adapted to a different number like combinations of 5 that's a big plus.
Python has "itertools" which includes a function to perform these kind of combinations for you.
combinations('ABCD', 2) returns AB AC AD BC BD CD
so you could do something like...
#!/usr/bin/python3.5
import itertools
output = []
for i in itertools.combinations(['Cr', 'Hf', 'Mo', 'Nb', 'Ta', 'Ti', 'V', 'W', 'Zr'], 4):
output.append("".join(i))
print(sorted(output))
Which would produce all 126 combinations and sort them for you.
['CrHfMoNb', 'CrHfMoTa', 'CrHfMoTi', 'CrHfMoV', 'CrHfMoW', 'CrHfMoZr', 'CrHfNbTa', 'CrHfNbTi', 'CrHfNbV', 'CrHfNbW', 'CrHfNbZr', 'CrHfTaTi', 'CrHfTaV', 'CrHfTaW', 'CrHfTaZr', 'CrHfTiV', 'CrHfTiW', 'CrHfTiZr', 'CrHfVW', 'CrHfVZr', 'CrHfWZr', 'CrMoNbTa', 'CrMoNbTi', 'CrMoNbV', 'CrMoNbW', 'CrMoNbZr', 'CrMoTaTi', 'CrMoTaV', 'CrMoTaW', 'CrMoTaZr', 'CrMoTiV', 'CrMoTiW', 'CrMoTiZr', 'CrMoVW', 'CrMoVZr', 'CrMoWZr', 'CrNbTaTi', 'CrNbTaV', 'CrNbTaW', 'CrNbTaZr', 'CrNbTiV', 'CrNbTiW', 'CrNbTiZr', 'CrNbVW', 'CrNbVZr', 'CrNbWZr', 'CrTaTiV', 'CrTaTiW', 'CrTaTiZr', 'CrTaVW', 'CrTaVZr', 'CrTaWZr', 'CrTiVW', 'CrTiVZr', 'CrTiWZr', 'CrVWZr', 'HfMoNbTa', 'HfMoNbTi', 'HfMoNbV', 'HfMoNbW', 'HfMoNbZr', 'HfMoTaTi', 'HfMoTaV', 'HfMoTaW', 'HfMoTaZr', 'HfMoTiV', 'HfMoTiW', 'HfMoTiZr', 'HfMoVW', 'HfMoVZr', 'HfMoWZr', 'HfNbTaTi', 'HfNbTaV', 'HfNbTaW', 'HfNbTaZr', 'HfNbTiV', 'HfNbTiW', 'HfNbTiZr', 'HfNbVW', 'HfNbVZr', 'HfNbWZr', 'HfTaTiV', 'HfTaTiW', 'HfTaTiZr', 'HfTaVW', 'HfTaVZr', 'HfTaWZr', 'HfTiVW', 'HfTiVZr', 'HfTiWZr', 'HfVWZr', 'MoNbTaTi', 'MoNbTaV', 'MoNbTaW', 'MoNbTaZr', 'MoNbTiV', 'MoNbTiW', 'MoNbTiZr', 'MoNbVW', 'MoNbVZr', 'MoNbWZr', 'MoTaTiV', 'MoTaTiW', 'MoTaTiZr', 'MoTaVW', 'MoTaVZr', 'MoTaWZr', 'MoTiVW', 'MoTiVZr', 'MoTiWZr', 'MoVWZr', 'NbTaTiV', 'NbTaTiW', 'NbTaTiZr', 'NbTaVW', 'NbTaVZr', 'NbTaWZr', 'NbTiVW', 'NbTiVZr', 'NbTiWZr', 'NbVWZr', 'TaTiVW', 'TaTiVZr', 'TaTiWZr', 'TaVWZr', 'TiVWZr']
If you want them "neatly" just use...
#!/usr/bin/python3.5
import itertools
output = []
for i in itertools.combinations(['Cr', 'Hf', 'Mo', 'Nb', 'Ta', 'Ti', 'V', 'W', 'Zr'], 4):
output.append("".join(i))
while output:
print(output.pop(0))
which gives...
CrHfMoNb CrHfMoTa CrHfMoTi CrHfMoV CrHfMoW CrHfMoZr CrHfNbTa CrHfNbTi
CrHfNbV CrHfNbW CrHfNbZr CrHfTaTi CrHfTaV CrHfTaW CrHfTaZr CrHfTiV
CrHfTiW CrHfTiZr CrHfVW CrHfVZr CrHfWZr CrMoNbTa CrMoNbTi CrMoNbV
CrMoNbW CrMoNbZr CrMoTaTi CrMoTaV CrMoTaW CrMoTaZr CrMoTiV CrMoTiW
CrMoTiZr CrMoVW CrMoVZr CrMoWZr CrNbTaTi CrNbTaV CrNbTaW CrNbTaZr
CrNbTiV CrNbTiW CrNbTiZr CrNbVW CrNbVZr CrNbWZr CrTaTiV CrTaTiW
CrTaTiZr CrTaVW CrTaVZr CrTaWZr CrTiVW CrTiVZr CrTiWZr CrVWZr HfMoNbTa
HfMoNbTi HfMoNbV HfMoNbW HfMoNbZr HfMoTaTi HfMoTaV HfMoTaW HfMoTaZr
HfMoTiV HfMoTiW HfMoTiZr HfMoVW HfMoVZr HfMoWZr HfNbTaTi HfNbTaV
HfNbTaW HfNbTaZr HfNbTiV HfNbTiW HfNbTiZr HfNbVW HfNbVZr HfNbWZr
HfTaTiV HfTaTiW HfTaTiZr HfTaVW HfTaVZr HfTaWZr HfTiVW HfTiVZr HfTiWZr
HfVWZr MoNbTaTi MoNbTaV MoNbTaW MoNbTaZr MoNbTiV MoNbTiW MoNbTiZr
MoNbVW MoNbVZr MoNbWZr MoTaTiV MoTaTiW MoTaTiZr MoTaVW MoTaVZr MoTaWZr
MoTiVW MoTiVZr MoTiWZr MoVWZr NbTaTiV NbTaTiW NbTaTiZr NbTaVW NbTaVZr
NbTaWZr NbTiVW NbTiVZr NbTiWZr NbVWZr TaTiVW TaTiVZr TaTiWZr TaVWZr
TiVWZr
Not as short as the python method, but still straightforward. Create a shell array and cycle through that in four for loops yielding the desired 126 lines:
ELARR=(Cr Hf Mo Nb Ta Ti V W Zr)
for ((i=0; i<${#ELARR[#]}; i++))
do for ((j=i+1; j<${#ELARR[#]}; j++))
do for ((k=j+1; k<${#ELARR[#]}; k++))
do for ((l=k+1; l<${#ELARR[#]}; l++))
do echo ${ELARR[i]}${ELARR[j]}${ELARR[k]}${ELARR[l]}
done
done
done
done
CrHfMoNb
CrHfMoTa
CrHfMoTi
CrHfMoV
CrHfMoW
.
.
.
TaTiWZr
TaVWZr
TiVWZr
Will be even shorter if you assign the array's element count to a variable, and use that, and mayhap use a shorter array name...

sed how to delete text and symbols between

I have sql file with this strings :
(17, 14, '2015-01-20 10:38:40', 211, 'Just text\n\nFrom: Support <support#domain.com>\n Send: 20 Jan 2015 year. 10:33\n To: Admin\n Theme: [TST #0000014] Just text \n\nJust text: Text\n Test text test text\n\nJust text:\n Text\n\n-- \n Test\n Text.\n Many text words 0.84.2', 0, 2);
I want remove all text between symbols \n\ and ', 0, 2);
I want get this result:
(17, 14, '2015-01-20 10:38:40', 211, 'Just text', 0, 2);
How I can do it via sed?
I try use this example - cat file | sed 's/<b>.*</b>//g'. I changed <b> to \n\ and </b> to ', 0, 2); But it dont work, I get error in console
Thanks in advance!
You can try this command
sed 's/\\n\\.*\('\'', 0, 2);\)/\1/g' FileName
Output :
(17, 14, '2015-01-20 10:38:40', 211, 'Just text', 0, 2);
You have to escape the single quotes like '\'' as well as back slash \\
If you can find it, you can replace it with nothing.
So, depending on what you mean by \n and what you need to escape, you want something like sed 's/\\n\\.*'//g'.
Obviously, take care that this is really what you want to replace on every line. It might be worth searching for the target \\n\\.*' first, to make sure it doesn't accidentally grab too much on an unexpected line.

How to substitute the first line and split the first string on the following line:

I am just new to scripting and I need some help. I have something like a bazillion files that look like this.
Assign F2 Height
3IleN 2.34025e+07
4PheN 2.05028e+07
6LysN 1.43672e+07
7ThrN 1.49120e+07
8LeuN 1.30838e+07
9ThrN 1.44298e+07
And i want it to look like this + save it in another file with the same name as the previous file however, with a "MOD" written at the beginning.
Number AA Height
3 IleN 6.20756e+07
4 PheN 5.26499e+07
7 ThrN 3.00216e+07
8 LeuN 3.26377e+07
9 ThrN 4.03901e+07
10 GlyN 2.73659e+07
12 ThrN 3.16319e+07
13 IleN 5.94604e+07
If you could please describe and explain the parameters used, that would be of great help.
Thanks!
The following should work for you:
sed 's/^\([0-9]*\)/\1 /' filename

alphanumeric sort in VIM

Suppose I have a list in a text file which is as follows -
TaskB_115
TaskB_19
TaskB_105
TaskB_13
TaskB_10
TaskB_0_A_1
TaskB_17
TaskB_114
TaskB_110
TaskB_0_A_5
TaskB_16
TaskB_12
TaskB_113
TaskB_15
TaskB_103
TaskB_2
TaskB_18
TaskB_106
TaskB_11
TaskB_14
TaskB_104
TaskB_112
TaskB_107
TaskB_0_A_4
TaskB_102
TaskB_100
TaskB_109
TaskB_101
TaskB_0_A_2
TaskB_0_A_3
TaskB_116
TaskB_1_A_0
TaskB_111
TaskB_108
If I sort in vim with command %sort, it gives me output as -
TaskB_0_A_1
TaskB_0_A_2
TaskB_0_A_3
TaskB_0_A_4
TaskB_0_A_5
TaskB_10
TaskB_100
TaskB_101
TaskB_102
TaskB_103
TaskB_104
TaskB_105
TaskB_106
TaskB_107
TaskB_108
TaskB_109
TaskB_11
TaskB_110
TaskB_111
TaskB_112
TaskB_113
TaskB_114
TaskB_115
TaskB_116
TaskB_12
TaskB_13
TaskB_14
TaskB_15
TaskB_16
TaskB_17
TaskB_18
TaskB_19
TaskB_1_A_0
TaskB_2
But I would like to have the output as follows -
TaskB_0_A_1
TaskB_0_A_2
TaskB_0_A_3
TaskB_0_A_4
TaskB_0_A_5
TaskB_1_A_0
TaskB_2
TaskB_10
TaskB_11
TaskB_12
TaskB_13
TaskB_14
TaskB_15
TaskB_16
TaskB_17
TaskB_18
TaskB_19
TaskB_100
TaskB_101
TaskB_102
TaskB_103
TaskB_104
TaskB_105
TaskB_106
TaskB_107
TaskB_108
TaskB_109
TaskB_110
TaskB_111
TaskB_112
TaskB_113
TaskB_114
TaskB_115
TaskB_116
Note I just wrote this list to demonstrate the problem. I could generate the list in VIM. But I want to do it for other things as well in VIM.
With [n] sorting is done on the first decimal number
in the line (after or inside a {pattern} match).
One leading '-' is included in the number.
try this command:
sor n
and you don't need the %, sort sorts all lines if no range was given.
EDIT
as commented by OP, if you have:
TaskB_0_A_1
TaskB_0_A_2
TaskB_0_A_4
TaskB_0_A_3
TaskB_0_A_5
TaskB_1_A_0
you could try:
sor n /.*_\ze\d*/
or
sor nr /\d*$/
EDIT2
for newly edited question, this line may give you expected output based on your example data:
sor nr /\d*$/|sor n

Resources