The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. January 08, 2009, 12:23:15 AM

Login with username, password and session length


Pages: [1] 2
  Print  
Author Topic: Rudimentary scraper in QT4  (Read 2030 times)
thedarkness
Global Moderator
Lifer
*****
Offline Offline

Posts: 581



View Profile
« on: September 11, 2008, 07:44:54 PM »

Said I'd post some examples so I will.....

Code:
// QtScrape.h

#include <QCoreApplication>
#include <QNetworkAccessManager>
#include <QNetworkRequest>
#include <QNetworkReply>
#include <QNetworkCookieJar>
#include <QDebug>

using namespace Qt;

class grabber : public QNetworkAccessManager // subclass (inherit) QNetworkAccessManager
{
Q_OBJECT

public:
grabber( QCoreApplication* theApp ) : QNetworkAccessManager()
{
// See http://doc.trolltech.com/4.4.0/signalsandslots.html
connect ( this, SIGNAL ( finished( QNetworkReply* ) ), this, SLOT ( replyFinished( QNetworkReply* ) ) );
connect ( this, SIGNAL ( completed() ), theApp, SLOT ( quit() ) );
}

void grab()
{
// See http://doc.trolltech.com/4.4.0/qnetworkaccessmanager.html#details
QUrl url = "http://www.google.com";
// In case we need session support etc.
QNetworkCookieJar* cookieJar = new QNetworkCookieJar();
setCookieJar( cookieJar );
QNetworkRequest request;
request.setUrl( url );
request.setRawHeader( "User-Agent", "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.0.1) Gecko/2008070206 Firefox/3.0.1" );
get( request );
}

private slots:
void replyFinished( QNetworkReply* reply )
{
qDebug()<<"Status: "<<reply->attribute( QNetworkRequest::HttpStatusCodeAttribute ).toInt();
qDebug()<<"Url: "<<reply->url();
qDebug()<<"Body: "<<reply->readAll();
emit completed();
}

signals:
void completed();
};

Code:
// QtScrape.cpp

#include "QtScrape.h"

int main(int argc, char *argv[])
{
QCoreApplication app(argc, argv);
grabber theGrabber( &app );
theGrabber.grab();
app.exec();  // Enter the event loop which will terminate when the grabber emits completed()
qDebug()<<"Finished!";
}

You may need to add
Code:
QT += network
to your .pro file.

That's it, Windows (with Visual C++ Pro, express, or mingw32), Linux or Mac.

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Online Online

Posts: 679



View Profile
« Reply #1 on: September 11, 2008, 07:54:57 PM »

<borat>Very nice, very nice. How much?</borat>
Logged

perkiset
Olde World Hacker
Administrator
Lifer
*****
Online Online

Posts: 5324


:sniffle: Humor was so much easier before.


View Profile
« Reply #2 on: September 11, 2008, 08:05:11 PM »

 ROFLMAO
Logged

If I can't be Mr. Root then I don't want to play.
thedarkness
Global Moderator
Lifer
*****
Offline Offline

Posts: 581



View Profile
« Reply #3 on: September 11, 2008, 08:06:57 PM »

If you are creating "Open Source" software it's free (as in beer).

This response is posted under the assumption that that was a serious question which is by no means certain  Tongue

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Online Online

Posts: 679



View Profile
« Reply #4 on: September 11, 2008, 08:18:22 PM »

 ROFLMAO ROFLMAO ROFLMAO

Serious?

 ROFLMAO ROFLMAO ROFLMAO ROFLMAO
Logged

perkiset
Olde World Hacker
Administrator
Lifer
*****
Online Online

Posts: 5324


:sniffle: Humor was so much easier before.


View Profile
« Reply #5 on: September 11, 2008, 08:18:48 PM »

I've always been baffled by the difference between Free (as in beer) and Free (as in lint). Makes me crazy.
Logged

If I can't be Mr. Root then I don't want to play.
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Online Online

Posts: 679



View Profile
« Reply #6 on: September 11, 2008, 08:25:29 PM »

Free lint?
Logged

thedarkness
Global Moderator
Lifer
*****
Offline Offline

Posts: 581



View Profile
« Reply #7 on: September 11, 2008, 08:32:13 PM »

I am so glad I took the time to post this  Roll Eyes

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
dink
Expert
****
Offline Offline

Posts: 332


View Profile
« Reply #8 on: September 11, 2008, 08:37:24 PM »

I am so glad I took the time to post this  Roll Eyes

Cheers,
td

:sucker.gif:   ROFLMAO

@perk . . . no lint in your beer.  It would be like sand in your KY Jelly.
No good, free or not.
Logged

[quote Nutballs]
the universe has a giant fist, and its got enough whoop ass for everyone.
[/quote]
deregular
Journeyman
***
Offline Offline

Posts: 67


View Profile
« Reply #9 on: September 11, 2008, 08:50:08 PM »

Quote
@perk . . . no lint in your beer.  It would be like sand in your KY Jelly.
No good, free or not.
yeah perk you remember that time when you and dink ..... sandy...  jelly ... oops wrong forum  Grin

@td hey dude long time no hear
Logged
thedarkness
Global Moderator
Lifer
*****
Offline Offline

Posts: 581



View Profile
« Reply #10 on: September 11, 2008, 08:54:16 PM »

@td hey dude long time no hear

Hey dereg,

How goes?

IM me some time.

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
perkiset
Olde World Hacker
Administrator
Lifer
*****
Online Online

Posts: 5324


:sniffle: Humor was so much easier before.


View Profile
« Reply #11 on: September 11, 2008, 09:17:40 PM »

F'reals, nice post TD. Thanks for it, and thanks for being seen back around. Seems like it's been a while.
Logged

If I can't be Mr. Root then I don't want to play.
thedarkness
Global Moderator
Lifer
*****
Offline Offline

Posts: 581



View Profile
« Reply #12 on: September 11, 2008, 10:05:45 PM »

F'reals, nice post TD. Thanks for it, and thanks for being seen back around. Seems like it's been a while.

It has, been super busy, still am. Flat out making sweet FA but I can't stay away from you guys, even though I know you're bad for me  ROFLMAO

I'll try and post some more stuff in the near future.
 
Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
nutballs
Administrator
Lifer
*****
Online Online

Posts: 3525


View Profile
« Reply #13 on: September 11, 2008, 10:10:13 PM »

i'll ask the obvious question.

WTF is QT4? lol
Logged
thedarkness
Global Moderator
Lifer
*****
Offline Offline

Posts: 581



View Profile
« Reply #14 on: September 11, 2008, 10:22:36 PM »

i'll ask the obvious question.

WTF is QT4? lol

nuts, how's living under that rock workin' for ya?  ROFLMAO

http://trolltech.com/products/qt

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
Pages: [1] 2
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!