Windows Command to detect and remove text in a file - windows

I have an ascii file and in there somewhere is the line:
BEGIN
and later on the line:
END
I'd like to be able to remove those two lines and everything in between from a command line call in windows. This needs to be completely automated.
EDIT: See sed in Vista - how to delete all symbols between? for details on how to use sed to do this (cygwin has sed).
EDIT: I am finding that SED could be working but when I pipe the output to a file, the carriage returns have been removed. How can I keep these? Using this sed regex:
/^GlobalSection(TeamFoundationVersionControl) = preSolution$/,/^EndGlobalSection$/{
/^GlobalSection(TeamFoundationVersionControl) = preSolution$/!{
/^EndGlobalSection$/!d
}
}
.. where the start section is 'GlobalSection(TeamFoundationVersionControl) = preSolution' and the end section is 'EndGlobalSection'. I'd also like to delete these lines as well.
EDIT: I am now using something simpler for sed:
/^GlobalSection(TeamFoundationVersionControl) = preSolution$/,/^EndGlobalSection$/d
The line feeds are still an issue though

Alternately, what I use these days is a scripting language that plays nicely with windows like Ruby or Python for such tasks. Ruby is easy to install in windows and makes problems like this child's play.
Here's a script you could use like:
cutBeginEnd.rb myFileName.txt
sourcefile = File.open(ARGV[0])
# Get the string and do a multiline replace
fileString = sourceFile.read()
slicedString = fileString.gsub(/BEGIN.*END\n/m,"")
#Overwrite the file
sourcefile.pos = 0
sourcefile.print slicedString
sourcefile.truncate(f.pos)
This does a pretty good job, allows for a lot of flexiblity, and is possibly more readable than sed.

Here is a 1-line Perl command that does what you want (just type it from the Command Prompt window):
perl -i.bak -ne "print unless /^BEGIN\r?\n/ .. /^END\r?\n/" myfile.txt
Carriage returns and line feeds will be preserved properly. The original version of myfile.txt will be saved as myfile.txt.bak.
If you don't have Perl installed, get ActivePerl.

Here's how to delete the entire GlobalSection(TeamFoundationVersionControl) = preSolution section using a C# regular expression:
// Create a regex to match against an entire GlobalSection(TeamFoundationVersionControl) section so that it can be removed (including preceding and trailing whitespace).
// The symbols *, +, and ? are greedy by default and will match everything until the LAST occurrence of EndGlobalSection, so we must use their non-greedy counterparts, *?, +?, and ??.
// Example of string to match against: " GlobalSection(TeamFoundationVersionControl) ...... EndGlobalSection "
Regex _regex = new Regex(#"(?i:\s*?GlobalSection\(TeamFoundationVersionControl\)(?:.|\n)*?EndGlobalSection\s*?)", RegexOptions.Compiled);

Related

SED command line utility (for Windows): How to add a new text line after specified line

Please give me suggestion. I need a working solution for sed.exe command line utility (SED build for Windows, version 4.1.5.4013). Or just give me a working UNIX-solution and i'll try to adapt it for my sed (for Win OS).
How to add the new (pretty sophisticated, to be honest) text line after specified line in a text file?
I have input file that contains following text (two lines):
[Pagebar Item Popup Menu]
Feature !Private, Item, MI_IDM_MENU_PAGEBAR_NEW_PAGE = New Page, 1,,,"New item menu"
I need to insert following line between first and second lines (above mentioned ):
Item, "Load tab" = Enable Javascript,,,,"Remove from Start Page.rtl" & Enable plugins & Enable display images & Reload
Is it possible to replace(but not to add new line every time when sed's command running!) an existing first line with two lines (replaced line + the new added string). Or, for an example, to make check if "Item, "Load tab"....." line already exists, avoid undesirable duplicates of the new line which is being inserted?
What I've tried but it didn't work:
sed -i "s/"\[Pagebar Item Popup Menu\]"/"\[Pagebar Item Popup Menu\]"\n"^Item,"Load tab"=Enable Javascript,,,,"Remove from Start Page\.rtl" & Enable plugins & Enable display images & Reload[ 0-9]\{1,\}[.,][0-9]\{3\}$/g" standard_menu.ini
sed -e "/pattern/ {p;s/.*/line_to_insert/;}" input_file > output_file
When pattern matches sed will do the following:
p: print pattern space: in this case this is the matching line
s/.*/line_to_insert/: changes pattern space to line_to_insert
If sed was called without -n then pattern space will be written out
If sed was called with -n then an additional p; command will be needed:
sed -e "/pattern/ {p;s/.*/line_to_insert/;p;}" input_file > output_file
For a unix solution under windows have you considered cygwin?
https://www.cygwin.com/
Secondly, your sed command looks a bit funky - have you tried making it easier on the eyes by using a different set of quotes to surround the expression?
finally, I think you might want cygwin as I don't believe sed alone will do what you want.
How to insert a line in a file between two blocks of known lines (if not already inserted previously), using bash?
This example uses grep as well
Thanks,
//P

search a pattern in each line and append it at the end of that line

I have a file with the following entries:
folder1/a_b.csv folder1/generated/
folder2/folder3/a_b1.csv folder12/generated/
folder4/b_c.csv folder123/generated/
folder5/d.csv folder1/new_folder/generated/
folder6/12.csv folder/anotherfolder/morefolder/evenmorefolder/generated/
I want to copy the csv file name from each line, paste them at the end of that line and append it with ".org". Hence, the changed file would look like
folder1/a_b.csv folder1/generated/a_b.csv.org
folder2/folder3/a_b1.csv folder12/generated/a_b1.csv.org
folder4/b_c.csv folder123/generated/b_c.csv.org
folder5/d.csv folder1/new_folder/generated/d.csv.org
folder6/12.csv folder/anotherfolder/morefolder/evenmorefolder/generated/12.csv.org
Basically, I am looking for a command in vim or sed using which I can search a pattern in each line and append it at the end of that line. Is it possible?
Thanks in advance.
Vim
Here's how to do this in Vim:
:%s/\([^/]*\.csv\)\( .*\)/&\1.org/
This global (:%) substitution matches the filename (characters that don't contain /, ending in .csv), and captures \(...\) it. It then matches the rest of the line, and captures that, too.
As a replacement, first keep the original match & (or \0), then append the first capture (\1) with the additional suffix.
sed
Though the regular expression syntax is somewhat different than in Vim, the identical expression can be used with sed:
sed -e 's/\([^/]*\.csv\)\( .*\)/&\1.org/' input
Alternatives
It looks like you want to do file renaming in batches. On Linux, the mmv command-line tool is well suited for that; you'll probably find many similar tools on the web, too.
This might work for you (GNU sed):
sed -r 's|/([^ ]*) .*|&\1.org|' file

Replace last line of XML file

Looking for help creating a script that will replace the last line of an XML file with a tag. I have a few hundred files so I'm looking for something that will process them in a loop. I've managed to rename the files sequentially like this:
posts1.xml
posts2.xml
posts3.xml
etc...
to make it easier to loop through. But I have no idea how to write a script to do this. I'm open to using either Linux or Windows (but i would guess that Linux is better for this kind of task).
So if you want to append a line to every file:
sed -i '$a<YOUR_SHINY_NEW_TAG>' *xml
To replace the last line:
sed -i '$s/.*/<YOUR_SHINY_NEW_TAG>/' *xml
But do note, sed is not the ideal tool to modify xml.
XMLStarlet is a command-line toolkit for performing XML parsing and manipulations. Note that as an XML-aware toolkit, it'll respect XML structure, character encoding and entity substitution.
Check out the ed command to see how to modify documents. You can wrap this in a standard bash loop.
e.g. in a doc consisting of a chain of <elem>s, you can add a following <added>5</added>:
mkdir new
for x in *.xml; do
xmlstarlet ed -a "//elem[count(//elem)]" -t elem -n added -v 5 $x > new/$x
done
Linux way using sed:
To edit the last line of the file in place, you can use sed:
sed -i '$s_pattern_replacement_' filename
To change the whole line to "replacement" use $s_.*_replacement_. Be sure to escape any _'s in replacement with a \.
To loop over files, just use for:
for f in /path/posts*.xml; do sed -i '$s_.*_replacement_' $f; done
This, however, is a dirty way as it's not aware of the XML structure, whereas the XML structure is not affected by newlines. You have to be sure the last line of the files contains exactly what you expect it to.
It makes little to no difference whether you're on Linux, Windows or MacOS
The question is what language do you want to use?
The following is an example in c# (not optimized, but read it as speudocode):
string rootDirectory = #"c:\myfiles";
var files = Directory.GetFiles(rootDirectory, "*.xml");
foreach (var file in files)
{
var lines = File.ReadAllLines(file);
lines[lines.Length - 1] = "whatever you want here";
File.WriteAllLines(file, lines);
}
You can compile this and run it on Windows, Linux, etc..
Or you could do the same in Python.
Of course this method does not actually parse the XML,
but you just wanted to replace the last line right?

Why can't my Perl script find the file when I run it from Windows?

I have a Perl Script which was built on a Linux platform using Perl 5.8 . However now I am trying to run the Perl Script on a Windows platform command prompt with the same Perl version.
I am using this command perl rgex.pl however it gives me one whole chunk of errors which looks to me like it has already been resolved in the script itself. The weird thing is I am able to run another Perl script without problem consisting of simple functions such as print, input etc.
The Code:
#!/usr/bin/perl
use warnings;
use strict;
use Term::ANSIColor;
my $file = "C:\Documents and Settings\Desktop\logfiles.log";
open LOG, $file or die "The file $file has the error of:\n => $!";
my #lines = <LOG>;
close (LOG);
my $varchar = 0;
foreach my $line ( #lines ) {
if ( $line =~ m/PLLog/ )
{
print("\n\n\n");
my $coloredText = colored($varchar, 'bold underline red');
print colored ("POS :: $coloredText\n\n", 'bold underline red');
$varchar ++;
}
print( $line );
}
When I run on the windows command prompt it gives me errors such as:
Unrecognized escape \D passed through at rgex.pl line 7.
=> No such file or directory at rgex.pl line 8.
Please give some advice on the codes please. Thanks.
A \ in a Perl string enclosed in double quotes marks the beginning of an escape sequence like \n for newline, \t for tab. Since you want \ to be treated literally you need to escape \ like \\ as:
my $file = "C:\\Documents and Settings\\Desktop\\logfiles.log";
Since you are not interpolating any variables in the string it's better to use single quotes:
my $file = 'C:\Documents and Settings\Desktop\logfiles.log';
(Inside single quotes, \ is not special unless the next character is a backslash or single quote.)
These error messages are pretty clear. They tell you exactly which lines the problems are on (unlike some error messages which tell you the line where Perl first though "Hey, wait a minute!").
When you run into these sorts of problems, reduce the program to just the problematic lines and start working on them. Start with the first errors first, since they often cascade to the errors that you see later.
When you want to check the value that you get, print it to ensure it is what you think it is:
my $file = "C:\\D....";
print "file is [$file]\n";
This would have shown you very quickly that there was a problem with $file, and once you know where the problem is, you're most of the way to solving it.
This is just basic debugging technique.
Also, you're missing quite a bit of the basics, so going through a good Perl tutorial will help you immensely. There are several listed in perlfaq2 or perlbook. Many of the problems that you're having are things that Learning Perl deals with in the first couple of chapters.

bash templating

i have a template, with a var LINK
and a data file, links.txt, with one url per line
how in bash i can substitute LINK with the content of links.txt?
if i do
#!/bin/bash
LINKS=$(cat links.txt)
sed "s/LINKS/$LINK/g" template.xml
two problem:
$LINKS has the content of links.txt without newline
sed: 1: "s/LINKS/http://test ...": bad flag in substitute command: '/'
sed is not escaping the // in the links.txt file
thanks
Use some better language instead. I'd write a solution for bash + awk... but that's simply too much effort to go into. (See http://www.gnu.org/manual/gawk/gawk.html#Getline_002fVariable_002fFile if you really want to do that)
Just use any language where you don't have to mix control and content text. For example in python:
#!/usr/bin/env python
links = open('links.txt').read()
template = open('template.xml').read()
print template.replace('LINKS', links)
Watch out if you're trying to force sed solution with some other separator - you'll get into the same problems unless you find something disallowed in urls (but are you verifying that?) If you don't, you already have another problem - links can contain < and > and break your xml.
You can do this using ed:
ed template.xml <<EOF
/LINKS/d
.r links.txt
w output.txt
EOF
The first command will go to the line
containing LINKS and delete it.
The second line will insert the
contents of links.txt on the current
line.
The third command will write the file
to output.txt (if you omit output.txt
the edits will be saved to
template.xml).
Try running sed twice. On the first run, replace / with \/. The second run will be the same as what you currently have.
The character following the 's' in the sed command ends up the separator, so you'll want to use a character that is not present in the value of $LINK. For example, you could try a comma:
sed "s,LINKS,${LINK}\n,g" template.xml
Note that I also added a \n to add an additional newline.
Another option is to escape the forward slashes in $LINK, possibly using sed. If you don't have guarantees about the characters in $LINK, this may be safer.

Resources