next »

[m0nkeymafia]

Hi Guys,
Before I get into the nutz of this I thought I may as well say in some detail what I already had done to optimize my site, so then if someone else is having the same problem hopefully they can have a lot more info to use.

OK So I have this site, bout 3-4 months ago we started hitting 10k hits a day, 25-50 concurrent users at a time.  Each page can call 2-3ajax requests a minute [on average] so it caned the server a bit.

I am looking for ways to further optimize my site, as such I will list everything Ive done [and that i can remember ]

What ive done
Went throughslow pages on my site identifying slow pages / queries and subsequently optimised them as much as i could.
Restructured code and mysql calls to use less data where appropriate, and be called less.
Turned mysql 4 cached on
Went through queries again trying to make larger queries smaller so they could fit in mysql cache.
Removed less used features / options
Put in systems to remove old data [i.e. old private messages etc]
Passing large variables by reference into functions etc
And done any obvious loop optimisation

What I thought about doing
Currently using php 4 - thought about moving to php5 and use mysqli [meant to have significant performance increase]
Using memcache, currently hosts being nobs and not isntalling it!

Whats next?
its come to the point where i have a lot going on and each bit only takes a small amount of time, so its increasingly hard to get times down.

Anyone have any cool methods im missing out?
Besides the obivous what else can i do?

[perkiset]

Memcached is a great system, but I think you'd do better with APC. With APC you get a twofer automatically: first, all code is kept in RAM in a compiled state - no more touching the OS, no more recompilation of code. When you get there, you suddenly realize that you can create considerably more complicated classes and functions because once their compiled you don't pay the price ever again (at least until they time out). This alone will be a huge increase in your servers capability and decrease in total page turnaround time. The second benefit is the user cache space (much like memcached)

The Good: with APC you store things in their native PHP format ie., you simply store arrays - no need to serialize or anything - they're just stored. You also have complete control and access to everything in the cache - so you can query and flush as you need to. I find this enormously helpful and completely absent in the PHP implementation of memcache. It's also extremely fast. Just simply blows the competition away. Since it is direct memory access and there's no TCP (like memcache) or OS access (like some other so-called caches) it just rocks.

Personal favorite trick with APC: Every URL that comes into every one of my systems is translated internally, post Apache. All my external URLs (at least the ones that I want Google to see) are simple .HTML files (at least they look that way) but in fact may be very complicated translations with lots of GET params before I get around to processing the page. One of these get params is "cached=true" - if I see this, then after the page is processed I store it into APC with the name being the URL that called it. When another call comes in, right after I get past my tracking and ID portion, I look to see if there is anything in the cache with that exact URL - if so, I grab it, spit it out and exit. Cuts page processing down to a fraction (my sites are highly dynamic and this saves more than 70% on average of the processing time needed per page).

The Bad: APC stores everything as is - and you have to load the entire contents of an entity back into <this process' memory> to make use of it. For example: if you had a table of all IP addresses for all known spiders and wanted to cache it, that's great: but if you store that all as a single array in the cache, you'll need to move a copy of the entire array back into a local variable before you can ask it any questions. In memcache, you can post queryies for a certain address and it will hash out the answer (or not) and simply return what you were looking for, not the entire thing. If you have lots of disparate queries against your cache that you need to do at page load time, this might make sense. This of course is offset by the fact that you must TCP over to the memcache daemon, but that might make sense. In my case (I've eliminated memcached from all but 2 of my systems and they're not long for this world...) I chose to store individual entries in the APC cache for each address. There were arguments that having so many elements in the cache was counter productive and speed was decreased, but with the latest several rounds of APC updates there are users running into the 100s of thousands of entries without describing any added latency... so I am satisfied with this.

The Ugly: Memcache is a separate process - you can access it from anywhere, with any language, anytime. It is persistent, so if you kill Apache it does not go away. APC is a heap that is a child to the Apache process that PHP is running in ie., if you kill Apache, your cache goes away as well. This is not entirely horrible, but the next part is: Since it is a child of Apache, running PHP scripts in a shell cannot get to the cache. The only way a PHP script run from the shell can access the cache is by throwing a web request into Apache! That I really dislike, but I've learned to deal with it.

Given what you posted I feel that the next great thing you could do is learn APC, rethink your page processing steps imagining the notion of "persistent memory" between script calls and implement it. I think you'll be delightfully surprised.

/p

[thedarkness]

Great synopsis perk, well worth the read, I learn much here.

Cheers,
td

[m0nkeymafia]

Wanted to say thanks for the APC info
My host has APC on so goning to get jiggy with some APC later \o/

[perkiset]

Wonderful news! Ping if you wanna bounce ideas... it's powerful in a variety of ways.

/p

[m0nkeymafia]

Dude noooo, APC is incompatible with Zend?
Or so my host says after they tried to install it, also I checked on the net and it seems true.
Got any work arounds?! Man I was well lookin forward to getting it working

[perkiset]

The Zend Accelerator is a PHP cache as well... and most probably why APC got it's name, "Alternative PHP Cache."

I have no experience with it because it costs money  and with so many free options out there I just couldn't justify the purchase. I have no idea if, since zend is on the machine, what you need to do/can do/should do/must do to interface with it. I also do not know if there is a user cache space - but it *probably* does.

I'd wager that it's equivalent and that the things we talk about in reference to APC would work the same in Zend... I hope so! Lets us know how that goes - a little research and you might be really happy again...

/p

[m0nkeymafia]

TBH I have no idea what zend does apart from some optimization in your php when its being parsed
Ill look into it dude, and report here if i find anything, woop to happiness and caching lol