next »

[KaptainKrayola]

The Kaptain is working on something and wants to make it as portable as possible.  The original idea was to use a MySQL DB to hold the data and just query from there to populate what needs to be populated but that cuts down on portability (slightly) and adds a hand full of extra scripts to be written to make it manageable (in case we want to edit the data).  So, the other idea is to create a hand full of txt files that we can upload to the server and just have a script read them in and use the data from them.  The data will be read only so there won't be any writing to the txt files. 

So the question is: how efficient is it to read in txt files instead of a query to a DB?  ColdFusion sucks at interacting with the file system so we want to make sure the PHP isn't going to be the same way.

[dink]

I've had the same type of situation.  In my case, it's because of the restrictions on some to the el cheapo hosting that I love to hate.

The text file solution works well for me.  I don't just leave it as read only, tho.  I also have some scripts that write results to text files.

The smart guys can prolly tell you why I'm fullashiite.  My experience with PHP working with text files is a positive one.  Fast enough for what I want done, and as portable as it gets.

  

[m0nkeymafia]

Files are generally slower than myqsl
But reading off them isnt that expensive, especially if you read in a lot at a time
You should probably be looking to read off [if u can choose?] 32k at least per read and process that then the overheads will come down.

Generally speaking you want to hold a fair bit in each text file as obviously opening and closing text files will incur some overhead.

But my experience is C++ files so it may not convert to php, but as will be using lower level languages to read the file I should suspect all will be good.

[perkiset]

Kapn - if you can, compile PHP with APC - then in the front of your code so something like this:

if (!$myBuff = apc_fetch('theStoreBufferName'))
{
   $myBuff = file_get_contents('/aPath/aPath/aFile.txt');
   apc_store('theStoreBufferName', $myBuff);
}

... and on you go. You'll then pay for the data load once only.

(Note - I'm using a little PHP efficiency trick here which helps compile/run time but decreases readability - in the IF expression, the first part executed is the apc_fetch, which then drops it's result (or nothing) into $myBuff, which is evaluated as true if it contains ANYTHING or false if nothing, so the IF will be true if nothing is loaded into the variable $myBuff)


<PerkProgrammingRant>
It is really important IMO to move the weight of file/TCP/MySQL access to the EDITOR perspective of the data, not the web surfer. Too often, people make choices about how to load stuff because it is easy to code, rather than for efficiency. If it takes twice as long to CRUD your management files (Create Read Update Delete) when in an editing mode, but it is lightening fast when your browsing then you win. In the scenario listed above, the very first surfer will pay the price to load the files, but from then on it will just SCREAM. The code above will give you a performance increase in the few-100 times zone. No lie. APC is no more difficult to get working than MySQL and in a case where you have less control, it may actually be easier.
</PerkProgrammingRant>


Good luck, ping if you nn help,
/p

[dink]

Great response Perk.

Is APC available on some/most shared enviroments?   //don't have a clue what APC is.

Not positive how this helps in the 'portable' area either.

[perkiset]

You need to say "--enable-apc" in the compile string and also download the PEAR APC module - although I am not sure if you need to do the compile switch anymore.

APC is the Alternative PHP Cache - it caches both the compiled opcode as well as giving you a shared memory space for caching variables. It's fantastically great. I've posted on it quite a bit, ping back if you need more information.

From a portability standpoint, APC is the same regardless of PHP implementation - if you get PHP to run with APC you're golden. I have no idea how widespread it is because I don't use anyone else's hosting.

/p

[dink]

Thanks.  I'll look into it.

[dink]

Looks like APC likely will not be available on shared hosting plans.  I found the following in one of the install docs:

"add the following line to your php.ini extension=apc.so  ...restart apache, and apc should be enabled."

No access to php.ini and stopping or starting apache is verbotten.

One server actually has the eAccelerator module installed, but have caching and optimizer disabled. 

Looks like flat files for these servers.

[thedarkness]

<?php phpinfo(); ?> Put that in a file and call it to see what's "in" php or from the command line do "php -i" then look for "apc"

Kapitano, have been itching for a chance to try this http://pear.php.net/manual/en/package.caching.cache-lite.php now might be the time?

Not sure what the performance figures are on it but it may be worth a shot?

Cheers,
td

[m0nkeymafia]

APC is on my shared account [p.s. thanks perk for the code snippet, im gonna apc it up tonight or tomorrow with what else u wrote in the other thread]

Secondly, can I query something with you re your code, not saying your wrong just wondering how it works?
FYI im not trying to be smart - u rox0r, im simply wondering how it all works

Assignment has the lowest operating precedent doesnt it? [bar crap stuff like commas]
So surely the !$myBuff should fire off before the = apc_fetch() ?

if (!$myBuff = apc_fetch('theStoreBufferName'))

Should execute as:

if ((!$myBuff) = apc_fetch())
Ergo it wouldnt execute as you think it would??

Also re the efficiency of the statement.
Isnt the only thing being omitted is a test [to check if $myBuff contains anything or not?]

So if you were to write it out fully itd be
$myBuff = apc_fetch();
if (!$myBuf) { .... }

[perkiset]

Assignment has the lowest operating precedent doesnt it? [bar crap stuff like commas]
So surely the !$myBuff should fire off before the = apc_fetch() ?

if (!$myBuff = apc_fetch('theStoreBufferName'))

Should execute as:

if ((!$myBuff) = apc_fetch())
Ergo it wouldnt execute as you think it would??

You may be correct... depends on when the ! is evaluated in relation to the assignment - but if you walk through the evaluation of that statement, it's either going to evaluate on whether the $myBuff has anything OR the apc_fetch - and in either case, the result would be what I am looking for. To make it absolutely explicit it should prolly be written like this:

if (!($myBuff = apc_fetch())) { ...

to force the evaluation of the whole thing... but it works perfectly in both cases because the whether the NOT is applied against myBuff or apc_fetch (which it would be in your question) the net result of the IF is the same.

Excellent question though.

Also re the efficiency of the statement.
Isnt the only thing being omitted is a test [to check if $myBuff contains anything or not?]

So if you were to write it out fully itd be
$myBuff = apc_fetch();
if (!$myBuf) { .... }

Absolutely. In fact I write interchangeably that way. The only diff is another line of code. I read an article a long time ago that the way the PHP opcodes work, if you can limit line calls you are more efficient - ergo the reason for the horrific lines in the SMF code base that look like this:

Code:

echo '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"', $context['right_to_left'] ? ' dir="rtl"' : '', '><head>
<meta http-equiv="Content-Type" content="text/html; charset=', $context['character_set'], '" />
<meta name="description" content="', $context['page_title'], '" />', empty($context['robot_no_index']) ? '' : '
<meta name="robots" content="noindex" />', '
<meta name="keywords" content="PHP, MySQL, bulletin, board, free, open, source, smf, simple, machines, forum" />
<script language="JavaScript" type="text/javascript" src="', $settings['default_theme_url'], '/script.js?fin11"></script>
<script language="JavaScript" type="text/javascript"><!-- // --><![CDATA[
var smf_theme_url = "', $settings['theme_url'], '";
var smf_images_url = "', $settings['images_url'], '";
var smf_scripturl = "', $scripturl, '";
var smf_iso_case_folding = ', $context['server']['iso_case_folding'] ? 'true' : 'false', ';
var smf_charset = "', $context['character_set'], '";
// ]]></script>
<title>', $context['page_title'], '</title>';

... where they're using the comma in an echo statement to keep lots of output contained in one line. NEt-net it really doesn't make that much difference in a small app, but in a larger one with lots of processing before a page is delivered it may very well make a difference. Note the embedded immediate-ifs in that line as well... just horrible to read.

Really, great qqs. Hope my answers rose to the task!

/p

[m0nkeymafia]

Nice one dude, see im from the C++ school of optimisation, so good to know where PHP differs.
So doing more on one line [to an extent] helps efficiency? Commas become more useful by the day lol.

Cheers for reply dude!