next »

[svakanda]

I am trying to entice google into indexing my NEW pages...I submitted a sitemap with 107 of them, and it's been a week now, and only three are indexed.  How do you guys go about getting basic backlinks for indexing?  I used to use joseph tierney's bloggergenerator, and i must say, it worked bloody awesome.  I have bloggers from nearly a year ago that are still up.  But the program seems to have died since then.

Anyone have suggestions for me?  I need this site up and searchable, it's my Real Business.  thanks !

[vsloathe]

Take a text file and populate it full of your new page links and their anchor text, one per line like:

http://www.mysite.com/shoes.html,shoes
http://www.mysite.com/dogs.html,dogs
http://www.mysite.com/cats.html,cats

Then create a little module that you're going to place all over your site. Something like this (coded in this window so my apologies if there are errors):

Code:

<?php
$arrData = file('myLinks.txt');
shuffle($arrData);
foreach($arrData as $line){
   $arrParts = explode(',', $line);
   echo '<a href="'.$arrParts[0].'">'.$arrParts[1].'</a>';
}

If you want to get fancier and add more factors into the equation, the sky's the limit pretty much. You just need to link from these new pages from every page on your site so that they get indexed, and make sure that you remove links very slowly.

[Keebler]

Take a text file and populate it full of your new page links and their anchor text, one per line like:

http://www.mysite.com/shoes.html,shoes
http://www.mysite.com/dogs.html,dogs
http://www.mysite.com/cats.html,cats

Then create a little module that you're going to place all over your site. Something like this (coded in this window so my apologies if there are errors):

Code:

<?php
$arrData = file('myLinks.txt');
shuffle($arrData);
foreach($arrData as $line){
   $arrParts = explode(',', $line);
   echo '<a href="'.$arrParts[0].'">'.$arrParts[1].'</a>';
}

If you want to get fancier and add more factors into the equation, the sky's the limit pretty much. You just need to link from these new pages from every page on your site so that they get indexed, and make sure that you remove links very slowly.

I am putting together a site that is going to have upwards of 10 million pages. Would the same thing work?

[vsloathe]

Sure, but I'd use a database instead of a flat file in that case.

OH! One important thing to note:

You will want to cache the set of links that you show on a particular page. You can do a shuffle() on them so that it's always 10, 20, 50 random links (whatever # you choose) per page, but you will want to cache that set for that page so that when a spider returns, it doesn't see a completely different set. That would be BAD.

[Keebler]

Hmm. I guess ill have to generate the links when I make the pages. I had it set up so that I had a footer.php included in each page. But I wouldnt know how to make it remember what to link to with caching.

So ill have to generate 5 random links each time and save them in the php.

[Keebler]

Sure, but I'd use a database instead of a flat file in that case.

OH! One important thing to note:

You will want to cache the set of links that you show on a particular page. You can do a shuffle() on them so that it's always 10, 20, 50 random links (whatever # you choose) per page, but you will want to cache that set for that page so that when a spider returns, it doesn't see a completely different set. That would be BAD.

I still need help on this.

How would I cache it?. and what do you mean by using a database

[vsloathe]

Just save the set of links for each page to a file or whatever.

[perkiset]

Essentially, you want the entire page image stored in either RAM (preferable) or your database. This then makes presentation of the page very fast, because rather than re-interpreting all the PHP to create the page, it is simply grabbed as a blob of text and spit back out to the user.

It is easiest to do this if you have a single "main.php" (this is what I do) - where all requests come through and the you create a page and send it back. If you don't, it's just a little more elbowgreasy.

In either case, you'll want to store the page image under the name of the current URL. Then, at the front end of processing, ask the cache (RAM or DB) if there is an image already stored. If so, grab it, echo it and quit. If you do not use a main.php methodology, then you'll probably want to include your caching code in the front of every page. The included code would look something like this pseudocode:

if ($buff = $db->singleAnswer("select pagebuff from cachetable where url='{$_SERVER['REQUEST_URI']}'"))
{
   echo $buff
   exit;
}
// since there's no cached version, create the page here...

// now here, push the cached version to the database...
$db->query("replace into cachetable values(url, pagebuff) values('{$_SERVER['REQUEST_URI']}', '$theCreatedPageBuff')");

With this kind of system, you'd also want a cache expiration mechanism, perhaps based on time or usage. There's a bit of work here, but it's an excellent exercise.