Merge CSV file using Perl - windows

I have a CSV file that I need to merge using the following Perl code, but I am not able to run it. It is supposed to out put multiple text files, but it is not working.
#!/usr/local/bin/perl
#
$template_file_name="rtr-template.txt";
while(<>) {
($location, $name, $lo0ip, $frameip, $framedlci, $eth0ip, $x)
= split (/,/);
open(TFILE, "< $template_file_name") || die "config template file $template_file_name:
$!\n";
$ofile_name = $name . ".txt";
open(OFILE, "> $ofile_name") || die "output config file $ofile_name: $!\n";
while (<TFILE>) {
s/##location##/$location/;
s/##rtrname##/$name/;
s/##eth0-ip##/$eth0ip/;
s/##loop0-ip##/$lo0ip/;
s/##frame-ip##/$frameip/;
s/##frame-DLCI##/$framedlci/;
printf OFILE $_;
}
}
The CSV file looks like this
Toronto, Router1, 172.25.15.1, 172.25.16.6,101, 172.25.100.1
And this the rtr-template.txt file
!
version 12.1
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname ##rtrname##
!
enable password cisco
enable secret cisco
!
interface Loopback0
ip address ##loop0-ip## 255.255.255.255
!
interface Serial0/0
description Frame-Relay Circuit
no ip address
encapsulation frame-relay
ip route-cache policy
frame-relay lmi-type ansi
no shutdown
!
interface Serial0/0.1 point-to-point
ip address ##frame-ip## 255.255.255.252
frame-relay interface-dlci ##frame-DLCI##
!
interface FastEthernet0/1
description User LAN Segment
ip address ##eth0-ip## 255.255.255.0
no shutdown
!
router eigrp 99
network 172.25.0.0
!
snmp-server location ##location##
!
line con 0
password cisco
login
transport input none
line aux 0
password cisco
login
line vty 0 4
password cisco
login
transport input telnet
!
end

The main problem is that you're running your program by double-clicking on its file name in Windows Explorer.
The way <> works is that it will read from any files that you specify on the command line (that appear in the #ARGV array) or, if that array is empty, then it will read from STDIN — usually the keyboard.
Double-clicking the file gives it no command-line parameters, so it waits for you to type input in the black window that appears. That means you've entered <RTR-DATA.CSV as input to your while loop and Perl has tried to split it on commas, giving only a single field, so it sets $location to <RTR-DATA.CSV. Not what you wanted!
So, if you run your program from the cmd window by entering
create-configs.pl RTR-DATA.CSV
then within the program #ARGV will contain RTR-DATA.CSV and the <> will automatically read from that file
Here are some further notes on your code
There is no need for the #! line on a Windows system, which will normally have the .pl file extension tied to the perl executable
You must always use strict and use warnings at the top of every Perl program you write, and declare all your variables at their first point of use. That would have given some very strong clues about the nature of your problem
You should normally chomp each line read from a file, as it will have a newline character at the end that can cause problems if you leave it at the end of the last field returned by split
In this case you should also probably add optional whitespace either side of the comma in the pattern you are splitting on, so as to remove leading and trailing spaces from the fields it returns
You should always use lexical file handles ($out_fh instead of OFILE) with the three-parameter form of open
And here's a rewrite of your code that takes into account all of those points. I hope it helps
use strict;
use warnings;
my $template_file = 'rtr-template.txt';
while ( <> ) {
chomp;
my ($location, $name, $lo0ip, $frameip, $framedlci, $eth0ip) = split /\s*,\s*/;
open my $t_fh, '<', $template_file
or die qq{Unable to open "$template_file" for input: $!};
my $out_file = "$name.txt";
open my $out_fh, '>', $out_file
or die qq{Unable to open "$out_file" for output: $!};
while (<$t_fh>) {
s/##location##/$location/g;
s/##rtrname##/$name/g;
s/##eth0-ip##/$eth0ip/g;
s/##loop0-ip##/$lo0ip/g;
s/##frame-ip##/$frameip/g;
s/##frame-DLCI##/$framedlci/g;
printf $out_fh $_;
}
}

Use Text::CSV to parse the CSV file and Template Toolkit or similar to do the templating. Don't reinvent the wheel.

Related

modifying shell stdout in real time

Ok so bear with me as I am not a professional, this is a proof of concept project to learn more about my shell, programming and just basic bash scripting.
So WHAT I WANT TO DO is: whenever anything is printed out in my terminal, be it the result of a command or an error message from the shell I want to apply some "filters" to what is being displayed so for example if I input "ls -a" in the terminal I would like to get the list of folders that the command returns but apply a TIME DELAY to the characters so that it seems like the list is being typed in real time.
More SPECIFICALLY I'd like for the script to take every alphanumerical character in STDOUT and spend a specific amount of time (say 100 milliseconds) iterating through random characters (these can be accessed randomly from a list) before finally stopping at the original value of the character.
WHAT I KNOW:
not much, I am new to programming in general so also the bash language but I can read some code and browsing through I found this http://brettterpstra.com/2012/09/15/matrixish-a-bash-script-with-no-practical-application/ script that plays with tput. This shows me the visual effect I'd like to accomplish can be accomplished...now to make it happen orderly and individually for each character printed to STDOUT...that is what I can't figure out.
WHAT I THINK:
in my mind I know I could take the STDOUT and pipe it to a file in which through any language (let's say python!) I can do all kinds of string manipulation and then return the output to STDOUT but I'd like for the characters to be manipulated in realtime so if for example the code was
cool_chars="£ ア イ ウ エ オ カ キ ク ケ コ サ シ ス "
stdout=whatever module works to grab STDOUT from shell as string
stdout = stdout.split(" ")
for word in stdout:
for letter in word:
n=0
while (n<10):
#print the following iteration in real time # shell but how????
print random.choice(cool_chars)
#finally stop at correct character
print letter
n++
Anyway, I've read a little about curses and ncurses and how you can create new windows with whatever specified parameters, I wonder if it'd be just a matter of creating a terminal with the specified parameters with the curses libraries and then making a link so that each new terminal instance opens my modified curses shell or if I can just do a bash shell script or if it'd be easiest to use something like python. I know all of the above can be options but I'm looking for the simplest, not necessarily most resource efficient answer.
Any help, comments, pointers etc is appreciated.
This does not answer you question fully, but it does print any input as if it was being type in real time:
perl -MTime::HiRes -F -ane '$|=1;$old=""; foreach $char(#F){Time::HiRes::sleep(0.1); print "\r${old}${char}"; $old.=$char}' /etc/hosts
instead of file, STDIN can be used:
echo -e "abc\ndef\nghi" | perl -MTime::HiRes -F -ane '$|=1;$old=""; foreach $char(#F){Time::HiRes::sleep(0.1); print "\r${old}${char}"; $old.=$char}'
We can make it shorter using shell's sleep:
perl -F -ane '$|=1;$old=""; foreach $char(#F){`sleep 0.1`; print "\r${old}${char}"; $old.=$char}'
EDIT:
The script below should fully solve your problem:
#!/usr/bin/perl
use strict;
use utf8;
binmode(STDOUT, ":utf8");
our $cols=`tput cols`;
our $|=1;
our $cursor="";
sub reset_line {
print "\r" . " "x$cols . "\r";
}
sub pick_cursor {
my #c = split (//,"£アイウエオカキクケコサシス");
$cursor=$c[int(rand(1+#c))];
}
while (<>) {
my $line="";
my #a=split //;
foreach my $char (#a) {
`sleep 0.1`;
reset_line;
pick_cursor;
if ( $char eq "\n" || $char =~ /\s/) {
print "${line}${char}";
}else {
print "${line}${char}${cursor}";
}
$line .= $char;
}
}

Error reading file in tainted mode

I am a trying to use taint mode. I want to open a file based on user input and open a file to read data. Below is my code
#!/usr/bin/perl -w
use strict;
use warnings;
my $name = $ARGV[0];
my $file = "/Desktop/data/$name";
open MYFILE, "$file" or die $!;
while (<MYFILE>) {
chomp;
print "$_\n";
}
close(MYFILE);
case 1) When I run file using
perl -w filename.pl input.txt
I am able to read data from the file.
case 2) When I change the
#!/usr/bin/perl -w
to
#!/usr/bin/perl -T
and run the file using
perl -T filename.pl input.txt
I am still able to read the data.
case 3)When I change file to open in write mode and run in tainted mode I get correct output as,
Insecure dependency in open while running with -t switch at test1.pl line 8.
What might be issue with case two scenarios? Or is that a correct behavior?
Is it allowed to open a file in taint mode for reading?
This is correct behaviour for taint mode. The documentation specifies:
You may not use data derived from outside your program to affect something else outside your program--at least, not by accident.
[...]
$arg = shift; # $arg is tainted
[...]
If you try to do something insecure, you will get a fatal error saying something like "Insecure dependency" or "Insecure $ENV{PATH}".
(edit: missed some stuff):
Tainted data may not be used directly or indirectly in any command that invokes a sub-shell, nor in any command that modifies files, directories, or processes, with the following exceptions:
Arguments to print and syswrite are not checked for taintedness.
(This is why the read-mode example doesn't complain about the file data.)
Command-line arguments are potentially insecure, and so are tainted until specified otherwise.
To determine whether data is tainted:
To test whether a variable contains tainted data, and whose use would thus trigger an "Insecure dependency" message, you can use the tainted() function of the Scalar::Util module, available in your nearby CPAN mirror, and included in Perl starting from the release 5.8.0.
To untaint data:
[...]the only way to bypass the tainting mechanism is by referencing subpatterns from a regular expression match. Perl presumes that if you reference a substring using $1, $2, etc., that you knew what you were doing when you wrote the pattern. That means using a bit of thought--don't just blindly untaint anything, or you defeat the entire mechanism. It's better to verify that the variable has only good characters (for certain values of "good") rather than checking whether it has any bad characters. That's because it's far too easy to miss bad characters that you never thought of.
(with a warning for use locale):
If you are writing a locale-aware program, and want to launder data with a regular expression containing \w, put no locale ahead of the expression in the same block. See SECURITY in perllocale for further discussion and examples.
This prevents the following from wiping out your hard drive:
perl script.pl '| rm -rf /'
Solution: Use the form of open that only accepts a file name.
open(my $fh, '<', $ARGV[0])

How do I write a file whose *filename* contains utf8 characters in Perl?

I am struggling creating a file that contains non-ascii characters.
The following script works fine, if it is called with 0 as parameter but dies when called with 1.
The error message is open: Invalid argument at C:\temp\filename.pl line 15.
The script is started within cmd.exe.
I expect it to write a file whose name is either (depending on the paramter) äöü.txt or äöü☺.txt. But I fail to create the filename containing a smiley.
use warnings;
use strict;
use Encode 'encode';
# Text is stored in utf8 within *this* file.
use utf8;
my $with_smiley = $ARGV[0];
my $filename = 'äöü' .
($with_smiley ? '☺' : '' ).
'.txt';
open (my $fh, '>', encode('cp1252', $filename)) or die "open: $!";
print $fh "Filename: $filename\n";
close $fh;
I am probably missing something that is obvious to others, but I can't find, so I'd appreciate any pointer towards solving this.
First of all, saying "UTF-8 character" is weird. UTF-8 can encode any Unicode character, so the UTF-8 character set is the Unicode character set. That means you want to create file whose name contain Unicode characters, and more specifically, Unicode characters that aren't in cp1252.
I've answered this on PerlMonks in the past. Answer copied below.
Perl treats file names as opaque strings of bytes. That means that file names need to be encoded as per your "locale"'s encoding (ANSI code page).
In Windows, code page 1252 is commonly used, and thus the encoding is usually cp1252.* However, cp1252 doesn't support Tamil and Hindi characters [or "☺"].
Windows also provides a "Unicode" aka "Wide" interface, but Perl doesn't provide access to it using builtins**. You can use Win32API::File's CreateFileW, though. IIRC, you need to still need to encode the file name yourself. If so, you'd use UTF-16le as the encoding.
Aforementioned Win32::Unicode appears to handle some of the dirty work of using Win32API::File for you. I'd also recommend starting with that.
* — The code page is returned (as a number) by the GetACP system call. Prepend "cp" to get the encoding.
** — Perl's support for Windows sucks in some respects.
The following runs on Windows 7, ActiveState Perl. It writes "hello there" to a file with hebrew characters in its name:
#-----------------------------------------------------------------------
# Unicode file names on Windows using Perl
# Philip R Brenan at gmail dot com, Appa Apps Ltd, 2013
#-----------------------------------------------------------------------
use feature ":5.16";
use Data::Dump qw(dump);
use Encode qw/encode decode/;
use Win32API::File qw(:ALL);
# Create a file with a unicode name
my $e = "\x{05E7}\x{05EA}\x{05E7}\x{05D5}\x{05D5}\x{05D4}".
"\x{002E}\x{0064}\x{0061}\x{0074}\x{0061}"; # File name in UTF-8
my $f = encode("UTF-16LE", $e); # Format supported by NTFS
my $g = eval dump($f); # Remove UTF ness
$g .= chr(0).chr(0); # 0 terminate string
my $F = Win32API::File::CreateFileW
($g, GENERIC_WRITE, 0, [], OPEN_ALWAYS, 0, 0); # Create file via Win32API
say $^E if $^E; # Write any error message
# Write to the file
OsFHandleOpen(FILE, $F, "w") or die "Cannot open file";
binmode FILE;
print FILE "hello there\n";
close(FILE);
no need to encode the filename (at least not on linux). This code works on my linux system:
use warnings;
use strict;
# Text is stored in utf8 within *this* file.
use utf8;
my $with_smiley = $ARGV[0] || 0;
my $filename = 'äöü' .
($with_smiley ? '?' : '' ).
'.txt';
open my $fh, '>', $filename or die "open: $!";
binmode $fh, ':utf8';
print $fh "Filename: $filename\n";
close $fh;
HTH, Paul

How do you create unicode file names in Windows using Perl

I have the following code
use utf8;
open($file, '>:encoding(UTF-8)', "さっちゃん.txt") or die $!;
print $file "さっちゃん";
But I get the file name as ã•ã£ã¡ã‚ƒã‚“.txt
I was wondering if there was a way of making this work as I would expect (meaning I have a unicode file name) this without resorting to Win32::API, Win32API::* or moving to another platform and using a Samba share to modify the files.
The intent is to ensure we do not have any Win32 specific modules that need to be loaded (even conditionally).
Perl treats file names as opaque strings of bytes. They need to be encoded as per your "locale"'s encoding (ANSI code page).
In Windows, this is is usually cp1252. It is returned by the GetACP system call. (Prepend "cp"). However, cp1252 doesn't support Japanese characters.
Windows also provides a "Unicode" aka "Wide" interface, but Perl doesn't provide access to it using builtins*. You can use Win32API::File's CreateFileW, though. IIRC, you need to still need to encode the file name yourself. If so, you'd use UTF-16le as the encoding.
* — Perl's support for Windows sucks in some respects.
Use Encode::Locale:
use utf8;
use Encode::Locale;
use Encode;
open($file, '>:encoding(UTF-8)', encode(locale_fs => "さっちゃん.txt") ) or die $!;
print $file "さっちゃん";
The following produces a unicoded file name on Windows 7 using Activestate Perl.
#-----------------------------------------------------------------------
# Unicode file names on Windows using Perl
# Philip R Brenan at gmail dot com, Appa Apps Ltd, 2013
#-----------------------------------------------------------------------
use feature ":5.16";
use Data::Dump qw(dump);
use Encode qw/encode decode/;
use Win32API::File qw(:ALL);
# Create a file with a unicode name
my $e = "\x{05E7}\x{05EA}\x{05E7}\x{05D5}\x{05D5}\x{05D4}".
"\x{002E}\x{0064}\x{0061}\x{0074}\x{0061}"; # File name in UTF-8
my $f = encode("UTF-16LE", $e); # Format supported by NTFS
my $g = eval dump($f); # Remove UTF ness
$g .= chr(0).chr(0); # 0 terminate string
my $F = Win32API::File::CreateFileW
($g, GENERIC_WRITE, 0, [], OPEN_ALWAYS, 0, 0); # Create file via Win32API
say $^E if $^E; # Write any error message
# Write to the file
OsFHandleOpen(FILE, $F, "w") or die "Cannot open file";
binmode FILE;
print FILE "hello there\n";
close(FILE);

Why can't my Perl script find the file when I run it from Windows?

I have a Perl Script which was built on a Linux platform using Perl 5.8 . However now I am trying to run the Perl Script on a Windows platform command prompt with the same Perl version.
I am using this command perl rgex.pl however it gives me one whole chunk of errors which looks to me like it has already been resolved in the script itself. The weird thing is I am able to run another Perl script without problem consisting of simple functions such as print, input etc.
The Code:
#!/usr/bin/perl
use warnings;
use strict;
use Term::ANSIColor;
my $file = "C:\Documents and Settings\Desktop\logfiles.log";
open LOG, $file or die "The file $file has the error of:\n => $!";
my #lines = <LOG>;
close (LOG);
my $varchar = 0;
foreach my $line ( #lines ) {
if ( $line =~ m/PLLog/ )
{
print("\n\n\n");
my $coloredText = colored($varchar, 'bold underline red');
print colored ("POS :: $coloredText\n\n", 'bold underline red');
$varchar ++;
}
print( $line );
}
When I run on the windows command prompt it gives me errors such as:
Unrecognized escape \D passed through at rgex.pl line 7.
=> No such file or directory at rgex.pl line 8.
Please give some advice on the codes please. Thanks.
A \ in a Perl string enclosed in double quotes marks the beginning of an escape sequence like \n for newline, \t for tab. Since you want \ to be treated literally you need to escape \ like \\ as:
my $file = "C:\\Documents and Settings\\Desktop\\logfiles.log";
Since you are not interpolating any variables in the string it's better to use single quotes:
my $file = 'C:\Documents and Settings\Desktop\logfiles.log';
(Inside single quotes, \ is not special unless the next character is a backslash or single quote.)
These error messages are pretty clear. They tell you exactly which lines the problems are on (unlike some error messages which tell you the line where Perl first though "Hey, wait a minute!").
When you run into these sorts of problems, reduce the program to just the problematic lines and start working on them. Start with the first errors first, since they often cascade to the errors that you see later.
When you want to check the value that you get, print it to ensure it is what you think it is:
my $file = "C:\\D....";
print "file is [$file]\n";
This would have shown you very quickly that there was a problem with $file, and once you know where the problem is, you're most of the way to solving it.
This is just basic debugging technique.
Also, you're missing quite a bit of the basics, so going through a good Perl tutorial will help you immensely. There are several listed in perlfaq2 or perlbook. Many of the problems that you're having are things that Learning Perl deals with in the first couple of chapters.

Resources