The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. September 16, 2019, 09:10:15 PM

Login with username, password and session length


Pages: [1]
  Print  
Author Topic: Paged sitemap help  (Read 3894 times)
gnarlyhat
Journeyman
***
Offline Offline

Posts: 51



View Profile WWW
« on: November 14, 2007, 03:02:16 AM »

I have a sitemap creator which spits out a static sitemap with all the pages. I would like to split them into ... say 50 urls per page. How do I go about doing this? Instead of just spitting out one file, I want it to split the sitemap into multiple pages with links to each one. I tried a loop and end up having multiple files of the same content Sad Thank you in advance Smiley

Here's the source of the original file.

Code:
<B><CENTER><font color="green" name="arial" size="4">Main Sitemap</font></CENTER></B>
<?
$path = ".";
$URL = "http://";
$URL .= $HTTP_SERVER_VARS["HTTP_HOST"];

$var = "<B><CENTER><font color=\"green\" name=\"arial\" size=\"4\">Main Sitemap</font></CENTER></B>";

//using the opendir function
$dir_handle = @opendir($path) or die("Unable to open $path");

//running the while loop
while ($file = readdir($dir_handle))
{
   if(is_dir($file) && $file != "." && $file != "..")
{
   $checkpath = $path."/".$file."/include/conf.php";
   $checkpath1 = $path."/".$file."/sitemap.php";
if (file_exists($checkpath1))
{
$sitename = ucwords(str_replace("-", " ", $file));
$dir_handle2 = @opendir($file) or die("Unable to open $file");

//running the while loop
while ($file2 = readdir($dir_handle2))
{
   if(!is_dir($file2) && strrchr($file2,'.') == ".php")
{
$sitename = ucwords(str_replace("-", " ", $file2));
$sitename = str_replace(".php", "", $sitename);

echo "<br><a href=$URL/$file/$file2>$sitename</a><br>\n\n";
$var .= "<br><a href=$URL/$file/$file2>$sitename</a><br>\n\n";

}
}
}
if (file_exists($checkpath))
{
$sitename = ucwords(str_replace("-", " ", $file));
$var .= "<br><a href=$URL/$file/>$sitename</a><br>\n\n";
$lines = file($file."/admin/keywords.txt");

foreach ($lines as $line_num => $line) {
$line = trim($line);
$link = trim(str_replace(" ", "-", $line));
$link = $link.".htm";
$var .= "<br><a href=$URL/$file/$link>$line</a><br>\n\n";
}

}
}
}
//closing the directory
closedir($dir_handle);

$fp = fopen("fullsitemap.htm", "w");
fwrite($fp, $var, strlen($var));
fclose($fp);

?>

<B><CENTER><font color="green" name="arial" size="4">PROCESS COMPLETED...</font></CENTER></B>
Logged
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #1 on: November 14, 2007, 09:57:40 AM »

Without looking at your code, I can offer that probably no one here's going to write that one for you... this is a logical exercise that you'll need to go through.

Some thoughts tho:
  • You'll want to build a generic page that can handle any subset of the data that you want to see. For example, either a GET parameter or cookie or something would contain the variables about where you currently are - like at record 1 or 51 and such.
  • You'll need a routine that looks at these variables, loads the result set and then extracts the 50 records you want to show and blends them with the generic page.
  • Done optimally, this code would NOT have a header/footer framework around it, the "theme" of the page would come from yet another file so that you could use this code in any site that you chose to use it in.

Some baby steps towards doing this would be to re-write this page so that it only shows 50 records, based on a $var that is the starting place - once you've got that working then you'd tackle the variable issue - perhaps you have a master /sitemap.php that points to /sitemapdetail.php?start=51 ... although this is not effective if you're doing this for spiders. If you are doing this for easy-easy scraping, then you'll want your URLs to look more like this: /sitemaps/51/index.html - in this case you'll need a mod_rewrite in Apache to make this work.

Hope this pushes you off in the right direction,
/p
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
gnarlyhat
Journeyman
***
Offline Offline

Posts: 51



View Profile WWW
« Reply #2 on: November 14, 2007, 06:24:21 PM »

Thanks perk. I wasn't expecting anyone to write it for me but just to point me to the right direction. As this is producing a static html paged sitemap, I think I can only take in half of your advice. I wasn't thinking right when I tried to insert some kind of loop and counter where it's supposed to spit out the link. That got me nowhere and bad results. I've thought over it while sleeping and I think it's just best to play with the $var at the end of the code, when it's all filled up. Read write 50 links plus the link to next page and loop it till it's done. I will post the code when I'm done with it ... hopefully Smiley
Logged
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #3 on: November 14, 2007, 06:28:57 PM »

Here's a couple little functions that I use to great affect. If you wrote a little reentrant directory walker, these two functions could be used to create a complete tree of your site very quickly.

Code:
<?php
function getDirs($inDir)
{
        
$out = array();
        
$h opendir($inDir);
        while (! 
is_bool($file=readdir($h))) {
                if (! 
is_dir($inDir '/' $file)) { continue; }
                if (( 
$file=='.') || ($file=='..')) {continue;}
                
$out[] = $file;
        }
        
closedir($h);
        
sort($out);
        return 
$out;
}
function 
getFiles($inDir)
{
        
$out = array();
        if (!
is_dir($inDir)) { die("ERR-01: Invalid Directory '$inDir'"); }
        
$h opendir($inDir);
        while (! 
is_bool($file=readdir($h))) {
                if (
is_dir($inDir '/' $file)) {continue;}
if ($file[0] == '.') { continue; }
                
$out[] = $file;
        }
        
closedir($h);
        
sort($out);
        return 
$out;
}
?>

Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
gnarlyhat
Journeyman
***
Offline Offline

Posts: 51



View Profile WWW
« Reply #4 on: November 14, 2007, 11:30:59 PM »

Thanks perk. I'm sure this will come in handy in time to come Smiley  Praise Praise
Logged
gnarlyhat
Journeyman
***
Offline Offline

Posts: 51



View Profile WWW
« Reply #5 on: November 16, 2007, 02:06:37 AM »

It took me a long time and I was even dreaming of this simple task at night but finally got it. Well ... at least part of it. Here's my problem now. After running it on my setup, the script found 246 links to be processed. Since I put 50 links per page, I should have 5 pages. The problem now is I only have 4 pages. Where did I go wrong? TIA  Praise

Code:
<?
$path = ".";
$URL = "http://";
$URL .= $HTTP_SERVER_VARS["HTTP_HOST"];

//using the opendir function
$dir_handle = @opendir($path) or die("Unable to open $path");

//running the while loop
while ($file = readdir($dir_handle))
{
   if(is_dir($file) && $file != "." && $file != "..")
{
   $checkpath1 = $path."/".$file."/sitemap.php";
if (file_exists($checkpath1))
{
$sitename = ucwords(str_replace("-", " ", $file));
$dir_handle2 = @opendir($file) or die("Unable to open $file");

//running the while loop
while ($file2 = readdir($dir_handle2))
{
   if(!is_dir($file2) && strrchr($file2,'.') == ".php" && $file2 != ".php")
{
$sitename = ucwords(str_replace("-", " ", $file2));
$sitename = str_replace(".php", "", $sitename);
$var .= "<a href=\"$URL/$file/$file2\">$sitename</a><br>\n";

}
}
}
}
}
//closing the directory
closedir($dir_handle);

$links = explode("\n", $var);
echo count($links)." links processed<br>";
echo "<a href=\"sitemap1.htm\">Main Sitemap</a>";
$page = 1;
$url = "<html><title>Sitemap ".$page."</title><body>\n\n<h1>Sitemap ".$page."</h1>\n\n";

foreach ($links as $link)
{
$count++;
$url .= $count;
$url .= $link."\n";

if ($count==50)
{
$url .= "<br><a href=\"sitemap".($page+1).".htm>Next Page</a><br>\n</body></html>";

$fp = fopen("sitemap".$page.".htm", "w");
fwrite($fp, $url, strlen($url));
fclose($fp);

$count = 0;
$page++;
$url = "<html><title>Sitemap ".$page."</title><body>\n\n<h1>Sitemap ".$page."</h1>\n\n";
}
}

?>
Logged
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #6 on: November 16, 2007, 01:01:26 PM »

...the script found 246 links to be processed. Since I put 50 links per page, I should have 5 pages. The problem now is I only have 4 pages. Where did I go wrong? TIA  Praise
Well somewhere in there, your math is wrong.  ROFLMAO

Here endeth the lesson Wink
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
gnarlyhat
Journeyman
***
Offline Offline

Posts: 51



View Profile WWW
« Reply #7 on: November 18, 2007, 05:31:03 AM »

You're wrong Perk ....

My maths is correct. My problem solving/logic is wrong. This endeth the lesson Smiley Mucho gracias.

Code:
<?
$path = ".";
$URL = "http://";
$URL .= $HTTP_SERVER_VARS["HTTP_HOST"];

//using the opendir function
$dir_handle = @opendir($path) or die("Unable to open $path");

//running the while loop
while ($file = readdir($dir_handle))
{
   if(is_dir($file) && $file != "." && $file != "..")
{
   $checkpath1 = $path."/".$file."/sitemap.php";
if (file_exists($checkpath1))
{
$sitename = ucwords(str_replace("-", " ", $file));
$dir_handle2 = @opendir($file) or die("Unable to open $file");

//running the while loop
while ($file2 = readdir($dir_handle2))
{
   if(!is_dir($file2) && strrchr($file2,'.') == ".php" && $file2 != ".php")
{
$sitename = ucwords(str_replace("-", " ", $file2));
$sitename = str_replace(".php", "", $sitename);
$var .= "<a href=\"$URL/$file/$file2\">$sitename</a><br>\n";

}
}
}
}
}
//closing the directory
closedir($dir_handle);

$links = explode("\n", $var);
echo count($links)." links processed<br>";
echo "<a href=\"sitemap1.htm\">Main Sitemap</a>";
$page = 1;
$url = "<html><title>Sitemap ".$page."</title><body>\n\n<h1>Sitemap ".$page."</h1>\n\n";

foreach ($links as $link)
{
$count++;
$url .= $link."\n";

if ($count==50)
{
$url .= "<br><a href=\"sitemap".($page+1).".htm>Next Page</a><br>\n</body></html>";

$fp = fopen("sitemap".$page.".htm", "w");
fwrite($fp, $url, strlen($url));
fclose($fp);

$count = 0;
$page++;
$url = "<html><title>Sitemap ".$page."</title><body>\n\n<h1>Sitemap ".$page."</h1>\n\n";
}
}

$url .= "\n<br><a href=\"sitemap1.htm>First Page</a><br>\n</body></html>";

$fp = fopen("sitemap".$page.".htm", "w");
fwrite($fp, $url, strlen($url));
fclose($fp);
?>

Logged
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #8 on: November 18, 2007, 12:20:55 PM »

ROFLMAO

Good for you lad. Someday when you are a father you may realize the benefit of a throwing of the guantlet and dismissal as the most powerful and empowering lesson of all Wink
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
gnarlyhat
Journeyman
***
Offline Offline

Posts: 51



View Profile WWW
« Reply #9 on: November 19, 2007, 08:23:53 PM »

I am a father Smiley An 8 month old one though ... still a loooooooooooong way to go.
Logged
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #10 on: November 19, 2007, 09:44:43 PM »

CONGRATS. It's a great journey, you're gonna love it.
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
Pages: [1]
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!