sed in Vista - how to delete all symbols between? - windows

I have a bat file that I should use to delete a part of one file and save into another one. I need to delete all the symbols between text "[aaa bbb]" and "[ccc ddd]". That is if I have the text:
[aaa bbb]
1
2
3
[ccc ddd]
I should have as output:
[aaa bbb]
[ccc ddd]
Thank you
EDIT: I would like to clarify the question. I should delete all the symbols between marker1 and marker2. Marker1 and marker2 are just some words or parts of text but not obligatory lines. For example I would have:
[aaa bbb] [ccc]
1
2
3
4
5
[www yyy]
If I want to delete the text between [aaa bbb] and [www yyy] I should have as output:
[aaa bbb]
[www yyy]

Take a look at the section "Delete between marker 1 and marker2" on this sed hints page
Applying it on your example. clean.sed:
/^\[aaa bbb\]$/,/^\[ccc ddd\]$/{
/^\[aaa bbb\]$/!{
/^\[ccc ddd\]$/!d
}
}
Run using:
sed -f clean.sed inputfile.txt
To edit the input file "in place", use the -i option to sed:
sed -i.bak -f clean.sed datafile.txt
A backup copy of the file with the name "datafile.txt.bak" is saved before editing the original.
EDIT: Since the assumption that the markers where always on a line of their own was wrong, heres a script that can handle markers in the middle of a line:
/\[aaa bbb\]/,/\[ccc ddd\]/{
s/\[aaa bbb\].*/[aaa bbb]/
s/.*\[ccc ddd\]/[ccc ddd]/
/\[aaa bbb\]$/!{
/^\[ccc ddd\]/!d
}
}
For this input:
foo[aaa bbb]1
2
3
4
5[ccc ddd]bar
foo
[aaa bbb]
1
2
3
[ccc ddd]
bar
It produces:
foo[aaa bbb]
[ccc ddd]bar
foo
[aaa bbb]
[ccc ddd]
bar
Note! It can't handle files where the markers can appear on the same line.
EDIT again: If the input format for marker 1 is such that you can always count on it being on a line of its own you can simplify the script some:
/^\[aaa bbb\]$/,/\[ccc ddd\]/{
s/.*\[ccc ddd\]/[ccc ddd]/
/^\[aaa bbb\]$/!{
/^\[ccc ddd\]/!d
}
}
(Anchoring marker 1 at the beginning and end of a line and skipping the trimming of the marker 1 line.)

Note that sed is available for Windows, along with a whole bunch of other GNU utilities. I'm not sure if you're asking whether there's an equivalent, or how to actually do it once you've got the tool.

D:\tmp\sed.exe -f sedscript.sed D:\tmp\test.txt >c:\tmp\test2.txt
/^\[Product Feature\]$/,/^\[Dm$/{
/^\[Product Feature\]$/!{
/^\[Dm$/!d
}
}

I looked at cmd and at power shell - can't find anything useful. Get yourself ActivePerl?

If you trust the VB script "sed-like" of this answer...
sed.vbs:
Dim pat, patparts, rxp, inp
pat = WScript.Arguments(0)
patparts = Split(pat,"/")
Set rxp = new RegExp
rxp.Global = True
rxp.Multiline = False
rxp.Pattern = patparts(1)
Do While Not WScript.StdIn.AtEndOfStream
inp = WScript.StdIn.ReadLine()
WScript.Echo rxp.Replace(inp, patparts(2))
Loop
You can type
cscript /Nologo sed.vbs s/^\d+\s*$/ < in.txt (in.txt being your initial text)
and you will obtain the expected output...
^\d+\s*$
Would target any line beginning with one or more digit, followed by 0 or more spaces within one line.
That is not the best "pure sed" solution and it can not actually delete lines, but this is a native "vista-compliant" solution...
Actually, the following hack deliberately interpreting the "d sed-command" could be able to 'delete' lines:
Dim pat, patparts, rxp, inp
pat = WScript.Arguments(0)
patparts = Split(pat,"/")
Set rxp = new RegExp
rxp.Global = True
rxp.Multiline = False
rxp.Pattern = patparts(1)
Do While Not WScript.StdIn.AtEndOfStream
inp = WScript.StdIn.ReadLine()
out = rxp.Replace(inp, patparts(2))
if not patparts(2)="d" or not out="d" Then
WScript.Echo out
end if
Loop
cscript /Nologo sed.vbs s/^\d+\s*$/ < in.txt would actually produce:
[aaa bbb]
[ccc ddd]
In a .bat, you could have a sed.bat:
cscript /Nologo sed.vbs %1 < %2
and then execute that .bat like this:
C:\prog\sed>sed.bat s/^\d+\s*$/d in.txt

Related

what is this line means $runCmd = "cmexec $node1 echo \"\" > ".$logfile;

Code snippet:
my $node = shift;
my $runCmd = "cmviewcl -v -f line -p ".$package_name." | awk -F \"[:|=]\" \'(\$1 == \"script_log_file\") { print \$2 }\'";
my $logfile = $output[0];
chomp $logfile;
#DC1_list = utils::getDC1Host($hash_ref);
#DC2_list = utils::getDC2Host($hash_ref);
foreach $node1 (#DC1_list) {
$runCmd = "cmexec $node1 echo \"\" > ".$logfile;
Please let me know the what's this line means:
$runCmd = "cmexec $node1 echo \"\" > ".$logfile;
it was written before as:
$runCmd = "cmexec $node1 rm -rf ".$logfile;
which probably means remove the file in logfile variable forced recursive, but later changed to the above. so
what's it's doing?
Remove a file is different than an empty file.
The first option keep the file but override the content with "" (2x double quote), the second one remove the file.
Maybe your application need the file exist, because of this you cannot remove it.
If you have really copied this line verbatim, it is pretty nonsense.
Let's assume that the variables mentioned here have the folllowing values:
runCmd has value FOO
node1 has value BAR
logfile has value BAZ
After parameter expansion and making the quoting a bit more legible, this leaves you with a line equivalent to
FOO = 'cmexec BAR echo "" >' .BAZ
This means that a command named FOO is invoked. It must either be an executable file in the PATH, or a function. This command gets three parameters:
First parameter : a lonely equal sign
Second parameter: The string cmexec BAR echo "" >
Third paramete : the string .BAZ
I don't believe that anybody would seriously write such a command; my guess is that you made a typo, or error when doing a copy&paste of this command.

Figuring out pipes in python

i am currently writing a program in python and i am stuck. So my questtion is:
I have a program that reads a file and prints some lines to stdout like this:
#imports
import sys
#number of args
numArgs = len(sys.argv)
#ERROR if not enough args were committed
if numArgs <= 1:
sys.exit("Not enough arguments!")
#naming input file from args
Input = sys.argv[1]
#opening files
try:
fastQ = open(Input , 'r')
except IOError, e:
sys.exit(e)
#parsing through file
while 1:
#saving the lines
firstL = fastQ.readline()
secondL = fastQ.readline()
#you could maybe skip these lines to save ram
fastQ.readline()
fastQ.readline()
#make sure that there are no blank lines in the file
if firstL == "" or secondL == "":
break
#edit the Header to begin with '>'
firstL = '>' + firstL.replace('#' , '')
sys.stdout.write(firstL)
sys.stdout.write(secondL)
#close both files
fastQ.close()
Now i want to rewrite this program so that i can run a command line like : zcat "textfile" | python "myprogram" > "otherfile". So i looked around and found subprocess, but can't seem to figure out how to do it. thanks for your help
EDIT:
Now, if what you are doing is trying to write a Python script to orchestrate the execution of both zcat and myprogram, THEN you may need subprocess. – rchang
The intend is to have the "textfile" and the program on a cluster, so i dont need to copy any files from the cluster. i just want to login on the cluster and use the command:zcat "textfile" | python "myprogram" > "otherfile", so that the zcat and the program do their thing and i end up with "otherfile" on the cluster. hope you understand what i want to do.
Edit #2:
my solution
#imports
import sys
import fileinput
# Counter, maybe there is a better way
count = 0
# Iterieration over Input
for line in fileinput.input():
# Selection of Header
if count == 0 :
#Format the Header
newL = '>' + line.replace('#' , '')
# Print the Header without newline
sys.stdout.write(newL)
# Selection of Sequence
elif count == 1 :
# Print the Sequence
sys.stdout.write(line)
# Up's the Counter
count += 1
count = count % 4
THX
You could use fastQ = sys.stdin to read the input from stdin instead of the file or (more generally) fastQ = fileinput.input() to read from stdin and/or files specified on the command-line.
There is also fileinput.hook_compressed so that you don't need zcat and read the compressed file directly instead:
$ myprogram textfile >otherfile

mixing file.write and system calls that write to same file

I have code that invokes two system commands that write to a file and I need to add something else from my code in between these calls:
File.open('test.txt', 'w') {|f|
`echo 1 > #{f.path}`
f.write '2'
`echo 3 >> #{f.path}`
}
as the result the file contains just
2
3
the first line is missing. I am sure there is a simple solution, but I cannot find it.
You are opening the file in "write" mode which is clobbering the first echo. Instead use 'append' mode. Additionally, you're not flushing after you write "2" so it'll be out of order when you read it back. Remember, f.write doesn't append a newline, so you probably need that too.
irb(main):020:0> File.open('asdf', 'a') do |f|
irb(main):021:1* `echo 1 > asdf`
irb(main):022:1> f.write("2\n") and f.flush
irb(main):023:1> `echo 3 >> asdf`
irb(main):024:1> end
=> ""
irb(main):025:0> File.read('asdf')
=> "1\n2\n3\n"
irb(main):026:0> puts File.read('asdf')
1
2
3
The File.open(name, 'a') is the important part. It means append to this file instead of overwriting it. See http://www.ruby-doc.org/core-2.0/IO.html#method-c-new and What are the Ruby File.open modes and options? for descriptions of file open modes.
If it's important to delete any existing file, the first echo will implicitly take care of that (since it's a single >). Or you can do it in ruby explicitly:
File.delete('asdf') if File.exists?('asdf')
to answer my own question:
File.open('test.txt', 'w') {|f|
`echo 11 > #{f.path}`
f.seek(0, IO::SEEK_END)
f.write "2\n"
f.flush
`echo 33 >> #{f.path}`
}

How to combine multiple lines in a single text file into one line, in Windows?

I have a multiple standard text files that follow this format, with varying numbers of lines in each file:
Line1
Line2
Line3
Line4
I want to merge every line into one, with a space in between each set of characters, so the text file would look as such:
Line1 Line2 Line3 Line3
...and so on. This needs to work with any given number of lines, due to the fact that each text file contains a different number of lines. My intention is not to merge the lines in the text files; I want each text file to remain separate. All the solutions I have found online either don't quite fit this or work exclusively with UNIX. I am running Windows 7. This can be done in Powershell, VBS, Batch, a particular program, doesn't matter, it just needs to work with Windows.
Much appreciated!
#ECHO OFF
setlocal
(SET var=)
FOR /f "delims=" %%x IN (list.txt) DO (
CALL SET var=%%var%% %%x
)
SET var=%var:~1%
echo var=%var%=
Where list.txt is the file containing your lines and var is the variable into which you want the lines concatenated.
Using batch:
for /f "usebackqdelims=" %%i in ("infile.txt") do #<nul set /p"=%%i ">>"outfile.txt"
>>"outfile.txt" echo.
Using PowerShell give this a try and see if it's what you want:
$my_file = "C:\file.txt"
$out_file = "C:\out.txt"
(Get-Content -Path $my_file) -join " " | Set-Content -Path $out_file
For the sake of completeness here's another solution in vbscript:
Set fso = CreateObject("Scripting.FileSystemObject")
Set infile = fso.OpenTextFile("C:\infile.txt")
Set outfile = fso.OpenTextFile("C:\outfile.txt", 2, True)
If Not infile.AtEndOfStream Then outfile.Write infile.ReadLine
Do Until infile.AtEndOfStream
outfile.Write " " & infile.ReadLine
Loop
infile.Close
outfile.Close
Install git-scm, cygwin or something else that contains bash, then you can do
cat *.txt | tr "\n" " "
Something like this?
(gc C:\test.txt) -join " "

Puppet: How can I wrap a command into two line if >80 characters?

In puppet, if define command is > 80 characters, how can I wrap into two line to do it?
exec { 'create_domain':
command => "some command exceed 80 character...........................................................how to do how to do?.......",
}
It's sort of ugly, but if the last character in a string is a \ followed by a newline, then the string is continued on the next line. My sample.pp manifest is below:
exec { 'wrapped_string_example':
command => "/bin/echo 12345678901234567890123456789012345678901234567890\
wrapped > /var/tmp/test.txt";
}
Running this with puppet apply sample.pp gives the following output
$ puppet apply sample.pp
notice: /Stage[main]/Exec[wrapped_string_example]/returns: executed successfully
notice: Finished catalog run in 0.10 seconds
And catting the created file shows the lines have wrapped:
$ cat /var/tmp/test.txt
12345678901234567890123456789012345678901234567890wrapped
See https://github.com/puppetlabs/puppet/blob/9fbb36de/lib/puppet/parser/lexer.rb#L537 (as of Puppet v2.7.0)
Also this is sort of a known issue: http://projects.puppetlabs.com/issues/5022
For big chunks of data, heredocs are the best way of dealing with long lines in Puppet manifests. The /L interpolation option is particularly useful. /L causes \ at the end of a line to remove newlines. For example, the following does what you'd expect, stripping indentation and newlines, including the trailing newline.
sshkey { 'example.com':
ensure => present,
type => 'ssh-rsa',
key => #(KEY/L),
RfrXBrU1T6qMNllnhXsJdaud9yBgWWm6OprdEQ3rpkTvCc9kJKH0k8MNfKxeBiGZVsUn435q\
e83opnamtGBz17gUOrzjfmpRuBaDDGmGGTPcO8Dohwz1zYuir93bJmxkNldjogbjAWPfrX10\
8aoDw26K12sK61lOt6GTdR9yjDPdG4zL5G3ZjXCuDyQ6mzcNHdAPPFRQdlRRyCtG2sQWpWan\
3AlYe6h6bG48thlo6vyNvOD8s9K0YBnwl596DJiNCY6EsxnSAhA3Uf9jeKqlVqqrxhEzHufx\
07iP1nXIXCMUV
|-KEY
target => '/home/user/.ssh/authorized_keys',
}
Or to keep the final newline, leave out the - before the end text:
exec { 'create_domain':
command => #(CMD/L),
/bin/echo 123456789012345678901234567890123456789012345678901234567890123456\
wrapped > /var/tmp/test.txt
| CMD
}
As of Puppet 3.5 you have a couple of options that i have used. Ruby allows you to concat strings over a couple of lines.
string = "line #1"\
"line #2"\
"line #3"
p string # => "line #1line #2line #3"
Another option, as of Puppet 3.5 they have added HereDoc functionality. This will allow you to put the string in a section of a source code file that is treated as if it were a separate file.
$mytext = #(EOT)
This block of text is
visibly separated from
everything around it.
| EOT
The puppet documentation is here: https://docs.puppet.com/puppet/4.9/lang_data_string.html#heredocs
If you really care about the 80cols limit you can always abuse a template to achieve that goal
exec {'VeryLongExec':
command => template("${module}/verylongexec")
}
Then put the actual command in that template file
Credits should go to Jan Vansteenkiste to figure

Resources