next »

[perkiset]

Just use the berkley DB.
It will be much faster then any of the alternatives mentioned.
And it will be simplest and most portable.

If u are very concerned about speed, there are faster alternatives to the berkley db.
But you will have to make ur own C binding.

You're great Nop. You're the kind of guy that doesn't trust a doctor to give you the best answer so you build your own chest XRay machine and interpret it as well huh? 

[nop_90]

@perks u could just use gdbm which probably has a php binding.
It will be 10x quicker then anything else, besides maybe an in-memory hash lookup
I had used on when i made a markov with like 200K keys. And it used 2 words for lookup.
Obviously it was too large to put in memory and if memory serves correctly it could generate 1000 words in less then 1 sec.

But there was a russian who make a dbm, but it was based on btrees (if i remember correctly)
Prices was it was slower to insert a new entry. But when db was made it was like 10x faster to look up.
It was in portable C, so u would need to make binding for that.

Anyway just using gdbm would be tons quicker then using any DB.

[nop_90]

for my own reference
http://qdbm.sourceforge.net/
another gdbm like package
with references to others
i believe tinydbm is one i was referring to before.

Benchmarks for all of them are like 10-20K/queries per second with 1 million entries

[perkiset]

The requirement for SDBM or NDBM is an Apache thing Nop - not my preference at all. I tried to bind a couple different types of database access to the Apache Mod_Rewrite system and the SDBM was the only one that I could get to work reliably.

The essence of my example is that the DB is cached at the Apache level (in the Apache daemon itself) so it is "pre-you" ie., the cost of processor is borne by the ISP not the user, ergo there is no trackable processor weight for using that sort of system. It is also wicked fast and makes a very clean system on the PHP side, because you can literally have 2 separate website systems on the same box without regard for cloaking decisions... Apache does it all for you. So it's not really about DB speed nearly so much as appropriate tool placement, in that example anyway.

But back to real efficiency: my current system does all spider & surfer tracking, telemetry updates and cloak decisions with this tiny bit of code:

$spiderName = $db->singleAnswer("select surfer_New('{$_SERVER['REMOTE_ADDR']}')");

if $spiderName is !null then I have a spider... else it's a surfer. In any case, all the tracking tables and such are automatically updated. It takes approximately 0.0006 seconds to execute. I am satisfied with my performance, but appreciate your thoughts

[nop_90]

@perks
I think u misunderstand what gdbm does.
gdbm or the bdm has not requirement for apache or what ever.

point of discussion was fastest way to lookup IP.
gdbm does not reside in memory, but on disk. mysql is built upon bdm.

your system does not fit into the discussion
my system which finds places of intrest, and puts things which is intresting, can be modified at runtime if i desire, or i can make it recover at runtime from errors.
also it has a JIT compiler, that compiles functions into machine code.
probably it is the fastest of its type.
then again point of discussion is not about my system

[perkiset]

@perks
I think u misunderstand what gdbm does.
gdbm or the bdm has not requirement for apache or what ever.

point of discussion was fastest way to lookup IP.
gdbm does not reside in memory, but on disk. mysql is built upon bdm.

your system does not fit into the discussion

I'm afraid it's you that doesn't understand my friend... it's not that GDBM requires Apache, it's that Apache requires a DBM that it can bind to. In the instance that I am talking about (an Apache 1.4) it will only bind to SDBM reliably. I have read that Apache 2 can bind to GDBM tables, which means that my method will work wonderfully with GDBM and be up to date and fast as anything.

Here's the deal: if you are in a handling language (PHP/PERL, no matter), then you open a DBM table, then do a lookup, you pay for all that activity before you get the answer. The problem is that most page request handlers are not persistent, so they can't have always-open file handles... they must reopen for every call. This, of course, could be fixed by a persistent language like Python or any number of languages that will compile a shared object to hang off Apache, but that is WAY more complex than you need to go.

By having Apache bind to a DBM table the handle is cached and open all the time and the lookup is done at blazing speed within apache before the request is even dispatched to the handling script. Outside of a persistent language/connection, this is the fastest way to go, and an excellent and brilliant system if I do say so myself 

[nop_90]

Here's the deal: if you are in a handling language (PHP/PERL, no matter), then you open a DBM table, then do a lookup, you pay for all that activity before you get the answer. The problem is that most page request handlers are not persistent, so they can't have always-open file handles... they must reopen for every call. This, of course, could be fixed by a persistent language like Python or any number of languages that will compile a shared object to hang off Apache, but that is WAY more complex than you need to go.

Ok it is not the language, it is apache itself. It is like coding with 1 arm tied behind ur back. It is a POS but it is the best POS out there .
I will go back to my happy world where these problems do not exist

Ponders what would happen if u would take something allegroserve and then have a module which loads a shared library which is the PHP language.
That way u could install shit like wordpress etc on it.
Biggest problem is that allegroserve requires allegro lisp and that costs like $10K a bang. I wonder who thier clients, i think they must be military etc, because when i talked to a sales rep he did not really seem to be intrested in selling anything

With a system like that u fire up the lisp process. Then using something like slime and a secure shell you can compile and load code, that modifies even running programs.
But again it is outside this discussion .

[dbrown]

Thanks for the insight... I went with a mysql method. The inode method rocks, but hosts will bitch. It would be fine on my own servers, but I dont want to give up those class c's on the 3rd party host.