next »

[vsloathe]

The example at the bottom of this page:

http://www.php.net/manual/en/mongo.tutorial.php

Should get you further than us explaining it.

[nutballs]

ah ok, so its basically like a big ass array in file.
You of course could do relationships, just not by way of joins and such.

so multi lookup vs trying to do everything at once.

[rcjordan]

Shit meets fan over at reddit. Good writeup, thought VS might enjoy gleaning through it. Way over my head re specifics but having lived through parallel events elsewhere in the space-time continuum, I can empathize.

Here are some of the lessons we have learnt about Cassandra so far:

    * It scales up nicely, but doesn't scale down nearly as well. At only three nodes we weren't able to take advantage of most of Cassandra's safeguards for ending up in this situation.
    * After bootstrapping a new node, the old ones don't delete old data until you run the cleanup, which can unbalance your cluster.
    * The pending queue is a good thing to monitor
    * It's significantly preferable to be over-provisioned than underprovisioned, and you need good statistics on your growth to project the number of nodes you'll need to handle your load as it increases so that you don't fall behind the curve. We've learnt that this is true of all things in our system, but our monitoring was skewed by memcached masking the load issue.

http://blog.reddit.com/2010/05/reddits-may-2010-state-of-servers.html

[vsloathe]

Yep I read that this morning.

Particularly pertinent to my current situation as tech management is pushing for us to use Cassandra on a high-throughput implementation that we're currently developing, but all us gear heads really like MongoDB.

[isthisthingon]

MongoDB is fast - written in C++.  The nix community has really been rallying around MongoDB lately.  Last two issues of Linux Journal featured it, for what that's worth 

[kurdt]

MongoDB is fast - written in C++.  The nix community has really been rallying around MongoDB lately.  Last two issues of Linux Journal featured it, for what that's worth 

It's as fast as 8000000 operations per second. It was done with 25 Large EC2 instances. But as somebody pointed to me at Twitter, that's so fast that payload couldn't have been very big and latency was like non-existing so localhost type of benchmark. But it demonstrates that MongoDB is really, really fast. It actually even beats Redis that's mem-based solution. I'm interested to see how Redis 2.0 benchmarks. Though between MongoDB and Redis you'll need to remember that when you utilize MongoDB you are using HD where every read takes little life out of your harddrives but with Redis you are using RAM where you don't have any moving parts. That's of course if you have your own servers. With EC2 or similar cloud server you probably don't give a shit about harddrive life

[nutballs]

Put mongo on ssd then?

[rcjordan]

Seen this NB?  Maybe it'll help you wrap around the noSQL logic.

http://www.rackspacecloud.com/blog/2010/05/12/cassandra-by-example/

[kurdt]

Put mongo on ssd then?

Doesn't make any sense in a big scale. It will be cheaper just to deal with harddrive failures and wait for memristors to come.

[nutballs]

but speed would increase to the point of the CPU being the bottleneck potentially, instead of the drives.

[vsloathe]

Actually the network's your biggest bottleneck. Any node can answer when you come a'knockin, so waiting on reads and writes to sync across the wire would be what stops you up.

[isthisthingon]

Seen this NB?  Maybe it'll help you wrap around the noSQL logic.
http://www.rackspacecloud.com/blog/2010/05/12/cassandra-by-example/

Nice find - thx rc.  In terms of hardware entropy it's a critical part of the plan from the beginning.  The constant planned failures in these massive data architectures permits scaling inexpensively on commodity hardware.

[nutballs]

Actually the network's your biggest bottleneck. Any node can answer when you come a'knockin, so waiting on reads and writes to sync across the wire would be what stops you up.

assuming a network is involved. I always assume those speed tests have no net in place and it is local processing code and database, on 1 machine. real world would never reach it of course.

[kurdt]

Actually the network's your biggest bottleneck. Any node can answer when you come a'knockin, so waiting on reads and writes to sync across the wire would be what stops you up.

That's true. And big latency in any point of the net will make any server look slow.

assuming a network is involved. I always assume those speed tests have no net in place and it is local processing code and database, on 1 machine. real world would never reach it of course.

Actually it was done with 25 EC2 instances. I doubt they were are in one machine. Also one of the points of that 8mil test was that it was demonstration on how well MongoDB sharding works and that you can just throw more machines in when you run out of steam and MongoDB can use them fully without big penalties. But I do doubt also that any network outside EC2 cloud was involved.

[rcjordan]

Casandra (partially) to blame for Digg's current problems. Or scapegoat.

"The problems with the new architecture are so bad that VP of Engineering John Quinn is now gone"

http://techcrunch.com/2010/09/07/digg-struggles-vp-engineering-door/