So, when using perks webrequest class, I add gzip to the encode types, because I am trying to work in as small a footprint as possible.

scrape a site.
site sends html gzipped.
store that gzipped HTML in a var string, not a file.
un-gzip (gunzip?)
echo human readable to screen.

obviously i got all that, except the unzip part. most of the stuff i find online is file based, and doesnt seem to translate to strings in vars.

pointers?

Start here:

http://us.

php

 

.net

 /manual/en/function.gzuncompress.

php

 

yea i been playing withall those, and I keep getting "dataerror" as an error on gzinflate and gzuncompress, but I get an undefined function with gzdecode. do i need to prepare the "raw response" somehow?

i don't get it. compression is something I have never dealt with, ever, so i know nothing at all about it. but it seems pretty straight forward.

The reason i need to do gzip though is because some of the pages I am "borrowing from" are freaking huge, and i am maxing out the mem on my allocation with my shared host. This eventually wont be an issue for me, but I still would like to be efficient and have the option of running this almost anywhere.

I had a problem with huge pages and instead, modified the webRequest class to append each successive packet to a file and NOT load it into the final string... it's not a tough mod and is not heavy on memory at all... that'll also kill your gzip problem.

Do it in the getChunk() function - right after the fread, append to a file and set $thisBuff (or whatever it is) to '' so that you're not adding anything to the input buffer. You might also want to echo a period or something so that you can see it download - unless you're calling this via a webpage, in which case perhaps you want to append to a log file as well or something. but if you just check the size of <the buffer file> while you're downloading, you should see it growing.

Gotta run, off for a few hours,
/p