The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. September 18, 2019, 12:37:05 PM

Login with username, password and session length


Pages: [1]
  Print  
Author Topic: 3 Reasons Not To Read File into @array  (Read 5628 times)
Bompa
Administrator
Lifer
*****
Offline Offline

Posts: 564


Where does this show?


View Profile
« on: August 12, 2007, 06:57:59 AM »

1. CPU resourecs.
2. Memory
3. Bulky, cluttered code.



I did this for years:

open(IN, 'myfile.txt');
@array = <IN>;
close IN;



When I got more clever, I did this:

open(IN, 'myfile.txt');
chomp(@array = <IN>);
close IN;

This way I don't have to chomp the line endings off later in my script.  Weeeeeee!


As I grew in perl, so did myfile.txt grow in size, and in fact, I had several of them.
myfile_1.txt
myfile_2.txt
myfile_3.txt
etc. etc.

Each file was quite large and I was opening them all, reading them into arrays then closing them.

It seemed an okay method to me, except it was quite a hassle when I wanted to modify an element
in any one of the arrays and save it to file.


I had a lot of these all through my code:

open(OUT, '>myfile_1.txt');
print OUT @array1;
close OUT;


Somehow I came accross Tie::File.  Whoa!  Powerful and afaik it comes with Perl core.

Now I don't open files for read or write.



I just put this near the top of my code:

use Tie::File;


Then I just do this for each file/array:

tie @array1, 'Tie::File', 'myfile_1.txt';

Thereafter, whatever my code does to any element in any array, automatically is done
to the file AND the files are not loaded into memory so they can be gigantic.

That's it.
Bompa

« Last Edit: August 12, 2007, 01:32:53 PM by Bompa » Logged

"The most beautiful and profound emotion we can experience is the sensation of the mystical..." - Albert Einstein
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #1 on: August 12, 2007, 08:24:36 PM »

So TIE essentially makes a file accessible like an array... yes? The the PERL processor, knowing that you have a file rather than memory associated with the reference understands to read the [nth] element of the file rather than the array?
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
Bompa
Administrator
Lifer
*****
Offline Offline

Posts: 564


Where does this show?


View Profile
« Reply #2 on: August 13, 2007, 05:54:42 AM »

So TIE essentially makes a file accessible like an array... yes? The the PERL processor, knowing that you have a file rather than memory associated with the reference understands to read the [nth] element of the file rather than the array?

Exactly.

Logged

"The most beautiful and profound emotion we can experience is the sensation of the mystical..." - Albert Einstein
JasonD
Expert
****
Offline Offline

Posts: 100


View Profile
« Reply #3 on: August 13, 2007, 06:42:25 AM »

Bomps.

Thanks mate. I've been working with Perl for years and this is the 1st time I have come across that module.

Excellent Cheesy
Logged
proton
n00b
*
Offline Offline

Posts: 5



View Profile
« Reply #4 on: August 13, 2007, 06:54:18 AM »

could turn out to be very useful - thanks for the tip Wink
Logged

No links in signatures please
dirk
Global Moderator
Expert
*****
Offline Offline

Posts: 416


View Profile
« Reply #5 on: August 13, 2007, 03:50:17 PM »

A tie can also be very useful for hashes:

tie (%hash, SDBM_File, $filename, O_RDWR|O_CREAT, 0644) || die;

If you have a hash with about 10 MB content it will require normally about 1 GB RAM (factor 100).

Therefor we often use tie with a hash to minimize RAM.

Logged
Pages: [1]
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!