The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. September 19, 2019, 05:34:57 AM

Login with username, password and session length


Pages: [1]
  Print  
Author Topic: function to remove files older than X  (Read 3987 times)
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« on: November 20, 2008, 07:00:20 PM »

Anyone have a fast function to remove files recursively within a directory tree that have an access date older than a specified time?

I figure its the stat() function in php, but that would mean scanning every single file in the tree tree, 1 by 1, to find if it is older than X. is there a more efficient way?

the reason is to deal with inode overload with my spam network... it's getting a bit... large... and is pissing off 1 of my hosts, with the rest soon to follow I am sure.
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
Bompa
Administrator
Lifer
*****
Offline Offline

Posts: 564


Where does this show?


View Profile
« Reply #1 on: November 20, 2008, 07:46:39 PM »

Anyone have a fast function to remove files recursively within a directory tree that have an access date older than a specified time?

I figure its the stat() function in php, but that would mean scanning every single file in the tree tree, 1 by 1, to find if it is older than X. is there a more efficient way?

the reason is to deal with inode overload with my spam network... it's getting a bit... large... and is pissing off 1 of my hosts, with the rest soon to follow I am sure.

Don't you already have some sort of code that is accessing/delivering those files?


Logged

"The most beautiful and profound emotion we can experience is the sensation of the mystical..." - Albert Einstein
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #2 on: November 20, 2008, 07:57:47 PM »

These sound like cache files NBs, yes? And you need to get rid of things that have not been accessed for (x) amount of time?

There's no quick way to do it if it's a full directory tree... you'll need a tree walker and grab/analyze each file in the dir. I'd recommend a reentrant function something like below. Unfortunately, been a bad 2 days at the PerkPlace, so I'm pretty ...erm... relaxed endeavoring to forget it all. The following code will NOT WORK on it's own - I only have conceptual brains left ATM, there's no finish in me right now. But I think it'll push you where you've got to go.

Code:
<?php

$cutoff 
date('Y-m-d H:i:s'time() - (86400 30));
delOldCacheFiles('/www/sites');

function 
delOldCacheFiles($theDir)
{
global $cutoff;

$arr getDirs($theDir)
foreach($arr as $dirdelOldCacheFiles($dir);

$arr getFiles($theDir)
foreach($arr as $file)
{
$attribs stat($file);
if ($attribs['atime'] < $cutoffunlink($file);
}
}

function 
getDirs($inDir)
{
        
$out = array();
        
$h opendir($inDir);
        while (! 
is_bool($file=readdir($h))) {
                if (! 
is_dir($inDir '/' $file)) { continue; }
                if (( 
$file=='.') || ($file=='..')) {continue;}
                
$out[] = $file;
        }
        
closedir($h);
        
sort($out);
        return 
$out;
}
function 
getFiles($inDir)
{
        
$out = array();
        if (!
is_dir($inDir)) { die("ERR-01: Invalid Directory '$inDir'"); }
        
$h opendir($inDir);
        while (! 
is_bool($file=readdir($h))) {
                if (
is_dir($inDir '/' $file)) {continue;}
if ($file[0] == '.') { continue; }
                
$out[] = $file;
        }
        
closedir($h);
        
sort($out);
        return 
$out;
}

?>

Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Offline Offline

Posts: 1669



View Profile
« Reply #3 on: November 21, 2008, 07:15:44 AM »

In the shell, or using shell_exec/backtick:

Code:
find . -atime +2 -print

To break it down:
. = current directory and subdirectories (find is recursive by default)
-atime = locate files based on time last accessed
+2 = number of days ago - this is based on 24 hour periods so fractional portions are rounded up. Think of +2 as "three days ago or more".
-print = print the list to the screen. For your purposes, you can just do -delete and auto-delete all files that it finds.
Logged

hai
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #4 on: November 21, 2008, 07:31:50 AM »

hot stuff VS ... I didn't think you could use find to grab atimes - i though only mtimes.
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #5 on: November 21, 2008, 11:23:31 AM »

@V

so from PHP how would I run that?
Hopefully I have permissions to do it. otherwise I need to go the pure PHP route.
I am seriously multitasking and getting myself all confused with the 5 other things I am cranking on.

@perk
yep caching stuff. unfortunately 500,000 inodes seems to be about where a $3.99 host gets pissy... LOL
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #6 on: November 21, 2008, 12:07:58 PM »

Nuts -

all shell scripts can be run from either the exec() function (if you don't care about the results) or shell_exec() which will return to you whatever was returned in the shell.

So if you di this:
$buff = shell_exec('find /www/sites/nutballsSites/ -atime +30');
$lines = explode(chr(10), $buff);

would give you an array of files that have not bee accessed in 30 days. Parse or simply unlink to your hearts content from then on.

WAY WAY better, good eye VSloathe.
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Offline Offline

Posts: 1669



View Profile
« Reply #7 on: November 21, 2008, 12:22:46 PM »

So if you di this:
$buff = shell_exec('find /www/sites/nutballsSites/ -atime +30');
$lines = explode(chr(10), $buff);

I'm a fan of the backtick, but it's exactly the same as a call to shell_exec. e.g.

Code:
<?php
$buff 
= `find /www/sites/nutballsSites/ -atime +30 -print`;
?>


Don't forget an option on *what to do* with your output. By default find does not blit to stdout, so you need to add "-print". Or you could make it all nice and encapsulated, like this:

Code:
<?php
$buff 
= `find /www/sites/nutballsSites/ -atime +30 -delete`;
?>


No need to do anything with the output of the above, as you're just having the shell delete the files. Or you can examine the output to see if it throws an error, or add -print as a trailing arg to show the list of files deleted back to you.

Quote
WAY WAY better, good eye VSloathe.

Thanks sir. I try not to take 2 keystrokes when 1 would suffice when I'm writing code, if I can help it. Smiley
Logged

hai
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #8 on: November 21, 2008, 12:32:00 PM »

very cool. yeay it works. quick and simple. hopefully that will help with my inode overload...
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #9 on: November 21, 2008, 12:44:01 PM »

The VS comes through with another fine patch, the -delete flag is nice.

But I personally am *not* a fan of the backtick, because it falls so far away from the essential syntax of PHP and adds more exceptional structures - it starts looking more like PERL and less like C. From a readability standpoint, if I was looking quickly through some back code, I might miss that they are backticks and blow over the line - so I prefer functionalizing things like this so that my intentions are really, really clear.

Unlike you VS, with a young, nimble brain, I find that I struggle to remember what I coded 6 months ago and need all the hints I can get Wink
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Offline Offline

Posts: 1669



View Profile
« Reply #10 on: November 21, 2008, 12:50:20 PM »

Fair enough. I have vim set to highlight strings encased in backticks in a color that's distinct from any other type of string encapsulation, so it stands out fine for me.
Logged

hai
dink
Expert
****
Offline Offline

Posts: 349


View Profile
« Reply #11 on: November 23, 2008, 12:12:55 PM »

You doods rock.   Praise   Outstanding VS.   Applause

Getting rid of mega tons of trash on a shared host (with absolutely to rights) has been an ongoing prob. 

Going to play trash man now. 
Logged

[quote Nutballs]
the universe has a giant fist, and its got enough whoop ass for everyone.
[/quote]
Pages: [1]
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!