The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. September 18, 2019, 07:52:54 PM

Login with username, password and session length


Pages: [1]
  Print  
Author Topic: Botbase missing some yahoo?  (Read 3795 times)
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« on: November 01, 2008, 02:20:27 PM »

Hey perk, are you by any chance seeing a bunch of new Slurps coming through that are busting not being IDd as bots from the botbase?

72.30.161.224
72.30.142.245
72.30.79.114
72.30.79.52
72.30.142.186
72.30.142.216

and many more
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #1 on: November 02, 2008, 12:11:58 PM »

ok so now I have a question.

Since these are missing from the BotBase, it made me wonder something

why not just do the entire class B, which is yahoo? 72.30.*.*

Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
mightycitizen
Rookie
**
Offline Offline

Posts: 27


View Profile
« Reply #2 on: November 02, 2008, 03:59:03 PM »

ok so now I have a question.

Since these are missing from the BotBase, it made me wonder something

why not just do the entire class B, which is yahoo? 72.30.*.*



 Ditto
Logged

No links in signatures please
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #3 on: November 02, 2008, 04:26:16 PM »

I've not been watching Nuts - if you've followed my attitude at Syndk8, I am less and less satisfied with cloaking in general and as such, have not paid as much attention to the botbase. However, when I was watching this stuff hourly I'd pass info like that on to Dirk DOUBLE extra quick. I found a new data center in JP at one point and narrowly missed getting bagged by it.

Dirk is responsive and should let you know what he finds re. those pretty quick. I know that he does not just simply ID a complete Class C unless he has evidence to do so because there are often 1/2 Cs and such that are dedicated to human reviewers and such. Their premise has always been to only ID verified spiders, and if an address is used for human activity then it will not be listed. How he ascertains this is not something I've pried him for.

All that said, I have blueprinted my new mechanism for watching IPs and their behavior on my sites, in an attempt to re-verify the botbase as well as watch for boneheads scraping me. I'll let you in on the technique when I deploy, if you're interested.
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #4 on: November 02, 2008, 08:00:22 PM »

i would be interested of course. I actually think I am going to use my JS redirector on top of the botbase stuff.
I am cloaking redirects, so in the end, I want anyone who is not going to buy shit, to see my content pages. lol. A little different than cloaking for different content.
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #5 on: November 02, 2008, 09:00:04 PM »

10-4. I think you'd need a slightly different approach than I am looking at, however the telemetry piece would probably apply nicely for you to ascertain when you are getting validly spidered and when you're just getting slammed or scraped.
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
dirk
Global Moderator
Expert
*****
Offline Offline

Posts: 416


View Profile
« Reply #6 on: November 04, 2008, 06:56:09 PM »

Hi nutballs,

just to let you know that we have added the above IPs.

And thanks for the input!

Generally we don't add entire IP ranges because the fantomas spiderSpy(TM) botBase
should contain only IPs of bots and there are lots of Yahoo employees using Yahoo's IPs
for surfing.

If we would add those IPs we would decloak stealth pages.

Dirk
Logged
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #7 on: November 04, 2008, 08:23:14 PM »

I figured you'd chime in sooner or later.

Yea i figured as much. I guess from what perk said, it made me realize i use the botbase for a different purpose. your target is to prevent revealing a cloak. My target is ANY search engine spider or even possible spider.

So i get it now.

by the way, since I have a large network of sites, I will notice new bots. often. Do you want to me send them to you and if so, where?
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
dirk
Global Moderator
Expert
*****
Offline Offline

Posts: 416


View Profile
« Reply #8 on: November 04, 2008, 08:33:57 PM »

If you would send them directly to me that would be great.

You can use the email address from my profile.

And please include - if available - also the UserAgent.
Logged
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #9 on: November 04, 2008, 09:02:19 PM »

will do.
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
Pages: [1]
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!