Fastest way to populate a hash in perl - performance

Question

I am trying filling a hash in perl from a file of around 564k lines, and the code is taking like 1.6~2.1 seconds to execute, while the equivalent in C# takes around 0.8 seconds to finish. Is there any better way to do it in Perl ?
I have tried so far :
# 1 - this version take ~ +1.6 seconds to fill the hash from file with ~ 564000
my %voc;
open(F,"<$voc_file");
while(defined(my $line=<F>)) {
chomp($line);
$voc{$line} = 1;
}
close(F);
and this
# 2 - this version take ~ +2.1 seconds to fill the hash from file with ~ 564000
my %voc;
my #voc_keys;
my #array_of_ones;
open(F,"<$voc_file");
my $voc_keys_index = 0;
while(defined(my $line=<F>)) {
chomp($line);
$voc_keys[$voc_keys_index] = $line;
$array_of_ones[$voc_keys_index] = 1;
$voc_keys_index ++;
}
#voc{#voc_keys} = #array_of_ones;
close(F);
In c#, I am using :
var voc = new Dictionary<String, int>();
foreach (string line in File.ReadLines(pathToVoc_file))
{
var trimmedline = line.TrimEnd(new char[] { '\n' });
voc[trimmedline] = 1;
}
And it takes only 700~800 ms

Answer 1

Definitely avoiding storing 1's as the data and using exists can save time and memory. You can eke out even more by removing the block from the loop:
my %voc;
open(F,"<$voc_file");
chomp, undef $voc{$_} while <F>;
close(F);
Benchmark results (using 20 character lines):
Benchmark: running ikegami, original, statementmodifier, statementmodifier_undef for at least 10 CPU seconds...
ikegami: 10 wallclock secs ( 9.54 usr + 0.46 sys = 10.00 CPU) # 2.10/s (n=21)
original: 10 wallclock secs ( 9.62 usr + 0.45 sys = 10.07 CPU) # 2.09/s (n=21)
statementmodifier: 10 wallclock secs ( 9.61 usr + 0.48 sys = 10.09 CPU) # 2.18/s (n=22)
statementmodifier_undef: 11 wallclock secs ( 9.85 usr + 0.48 sys = 10.33 CPU) # 2.23/s (n=23)
Benchmark:
use strict;
use warnings;
use Benchmark 'timethese';
my $voc_file = 'rand.txt';
sub check {
my ($voc) = #_;
unless (keys %$voc == 564000) {
warn "bad number of keys ", scalar keys %$voc;
}
chomp(my $expected_line = `head -1 $voc_file`);
unless (exists $voc->{$expected_line}) {
warn "bad data";
}
return;
}
timethese(-10, {
'statementmodifier' => sub {
my %voc;
open(F,"<$voc_file");
chomp, $voc{$_} = 1 while <F>;
close(F);
#check(\%voc);
return;
},
'statementmodifier_undef' => sub {
my %voc;
open(F,"<$voc_file");
chomp, undef $voc{$_} while <F>;
close(F);
#check(\%voc);
return;
},
'original' => sub {
my %voc;
open(F,"<$voc_file");
while(defined(my $line=<F>)) {
chomp($line);
$voc{$line} = 1;
}
close(F);
#check(\%voc);
return;
},
'ikegami' => sub {
my %voc;
open(F,"<$voc_file");
while(defined(my $line=<F>)) {
chomp($line);
undef $voc{$line};
}
close(F);
#check(\%voc);
return;
},
});
(Original incorrect answer replaced with this.)

Answer 2

Of course C# is going to be faster.
You could save a little time and some memory by replacing
$voc{$line} = 1; ... if ($voc{$key}) { ... } ...
with
undef $voc{$line}; ... if (exists($voc{$key})) { ... } ...