next »

[dirk]

Files are normally processed using the for loops below:

Code:

open my $INPUT, '<', $filename
    or croak "Can't open '$filename': $OS_ERROR";
my @lines = <$INPUT>;
close $INPUT;

foreach my $line (@lines) {
    # processing
}

Code:

open my $INPUT, '<', $filename
    or croak "Can't open '$filename': $OS_ERROR";

for my $line (<$INPUT>) {
    # processing
}

close $INPUT;

If the file is a large one, for example with 100 MB or more the script could stop with a memory allocation failure: Out of memory!

The reason is that the array @lines in the first example or the temporary array in the second example will require too much memory.

Therefor a while loop should be used which reads and processes only one line at a time:

Code:

open my $INPUT, '<', $filename
    or croak "Can't open '$filename': $OS_ERROR";

while ( my $line = <$INPUT> ) {
    # processing
}

close $INPUT;

So a file with 2 GB can be processed without any memory problems.

[perkiset]

Such important stuff Dirk - lots of n00bs do not consider streaming data anymore - the PHP function file_get_contents() makes it so easy to grab the whole file into a string variable that people are afraid of the fopen, fread, fseek and fclose style functions - which if you're an oldie (like you'n me  ) you were raised on...

Thanks! Nice one
/p