The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. September 16, 2019, 10:06:30 PM

Login with username, password and session length


Pages: [1]
  Print  
Author Topic: gzip - how to I UN-gzip a string?  (Read 2943 times)
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« on: February 15, 2008, 11:55:59 AM »

So, when using perks webrequest class, I add gzip to the encode types, because I am trying to work in as small a footprint as possible.

scrape a site.
site sends html gzipped.
store that gzipped HTML in a var string, not a file.
un-gzip (gunzip?)
echo human readable to screen.

obviously i got all that, except the unzip part. most of the stuff i find online is file based, and doesnt seem to translate to strings in vars.

pointers?
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #1 on: February 15, 2008, 12:03:51 PM »

Start here:

http://us.php.net/manual/en/function.gzuncompress.php


Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
nutballs
Administrator
Lifer
*****
Offline Offline

Posts: 5627


Back in my day we had 9 planets


View Profile
« Reply #2 on: February 15, 2008, 12:19:19 PM »

yea i been playing withall those, and I keep getting "dataerror" as an error on gzinflate and gzuncompress, but I get an undefined function with gzdecode. do i need to prepare the "raw response" somehow?

i don't get it. compression is something I have never dealt with, ever, so i know nothing at all about it. but it seems pretty straight forward.

The reason i need to do gzip though is because some of the pages I am "borrowing from" are freaking huge, and i am maxing out the mem on my allocation with my shared host. This eventually wont be an issue for me, but I still would like to be efficient and have the option of running this almost anywhere.
Logged

I could eat a bowl of Alphabet Soup and shit a better argument than that.
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 10096



View Profile
« Reply #3 on: February 15, 2008, 04:21:20 PM »

I had a problem with huge pages and instead, modified the webRequest class to append each successive packet to a file and NOT load it into the final string... it's not a tough mod and is not heavy on memory at all... that'll also kill your gzip problem.

Do it in the getChunk() function - right after the fread, append to a file and set $thisBuff (or whatever it is) to '' so that you're not adding anything to the input buffer. You might also want to echo a period or something so that you can see it download - unless you're calling this via a webpage, in which case perhaps you want to append to a log file as well or something. but if you just check the size of <the buffer file> while you're downloading, you should see it growing.

Gotta run, off for a few hours,
/p
Logged

It is now believed, that after having lived in one compound with 3 wives and never leaving the house for 5 years, Bin Laden called the U.S. Navy Seals himself.
Pages: [1]
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!