next »

[jammaster82]

(asssuming i have no robots.txt file)

so when 'they' come in to my index.html

they are gonna follow all those links
and then die, right?

so they couldnt reach for example a
hidden directory i had on a public
site unless it was linked to from
outside somewhere else or unless
i link to it myself  in index.html
and it finds me on the crawl ( Yes / No / Cancel ) ?

[m0nkeymafia]

Pretty much. lol.
They may not follow all links straight away, depends how much "they" [which I love btw] like your site.

-> ( Yes )
-> throw new Exception("They got me");

[dink]

Tain't necessarily true.  If you have any of the 'thingies' attached to your browser,like toolbars, or whatever, and you or someone else with the bad stuff, visit(s) the hidden stuff....they'll get there unless you gots em banned.

Best bet is to put up an htaccess in the hidden directory.  Ban everyone but you and the server. 

EOF