The Cache: Technology Expert's Forum
 
*
Welcome, Guest. Please login or register. July 29, 2010, 07:55:55 PM

Login with username, password and session length


Pages: [1] 2
  Print  
Author Topic: Rudimentary scraper in QT4  (Read 4542 times)
thedarkness
Lifer
*****
Offline Offline

Posts: 585



View Profile
« on: September 11, 2008, 07:44:54 PM »

Said I'd post some examples so I will.....

Code:
// QtScrape.h

#include <QCoreApplication>
#include <QNetworkAccessManager>
#include <QNetworkRequest>
#include <QNetworkReply>
#include <QNetworkCookieJar>
#include <QDebug>

using namespace Qt;

class grabber : public QNetworkAccessManager // subclass (inherit) QNetworkAccessManager
{
Q_OBJECT

public:
grabber( QCoreApplication* theApp ) : QNetworkAccessManager()
{
// See http://doc.trolltech.com/4.4.0/signalsandslots.html
connect ( this, SIGNAL ( finished( QNetworkReply* ) ), this, SLOT ( replyFinished( QNetworkReply* ) ) );
connect ( this, SIGNAL ( completed() ), theApp, SLOT ( quit() ) );
}

void grab()
{
// See http://doc.trolltech.com/4.4.0/qnetworkaccessmanager.html#details
QUrl url = "http://www.google.com";
// In case we need session support etc.
QNetworkCookieJar* cookieJar = new QNetworkCookieJar();
setCookieJar( cookieJar );
QNetworkRequest request;
request.setUrl( url );
request.setRawHeader( "User-Agent", "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.0.1) Gecko/2008070206 Firefox/3.0.1" );
get( request );
}

private slots:
void replyFinished( QNetworkReply* reply )
{
qDebug()<<"Status: "<<reply->attribute( QNetworkRequest::HttpStatusCodeAttribute ).toInt();
qDebug()<<"Url: "<<reply->url();
qDebug()<<"Body: "<<reply->readAll();
emit completed();
}

signals:
void completed();
};

Code:
// QtScrape.cpp

#include "QtScrape.h"

int main(int argc, char *argv[])
{
QCoreApplication app(argc, argv);
grabber theGrabber( &app );
theGrabber.grab();
app.exec();  // Enter the event loop which will terminate when the grabber emits completed()
qDebug()<<"Finished!";
}

You may need to add
Code:
QT += network
to your .pro file.

That's it, Windows (with Visual C++ Pro, express, or mingw32), Linux or Mac.

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Offline Offline

Posts: 1598



View Profile
« Reply #1 on: September 11, 2008, 07:54:57 PM »

<borat>Very nice, very nice. How much?</borat>
Logged

perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 8649



View Profile
« Reply #2 on: September 11, 2008, 08:05:11 PM »

 ROFLMAO
Logged

It's the things you learn after you know it all that really count.
-John Wooden
thedarkness
Lifer
*****
Offline Offline

Posts: 585



View Profile
« Reply #3 on: September 11, 2008, 08:06:57 PM »

If you are creating "Open Source" software it's free (as in beer).

This response is posted under the assumption that that was a serious question which is by no means certain  Tongue

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Offline Offline

Posts: 1598



View Profile
« Reply #4 on: September 11, 2008, 08:18:22 PM »

 ROFLMAO ROFLMAO ROFLMAO

Serious?

 ROFLMAO ROFLMAO ROFLMAO ROFLMAO
Logged

perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 8649



View Profile
« Reply #5 on: September 11, 2008, 08:18:48 PM »

I've always been baffled by the difference between Free (as in beer) and Free (as in lint). Makes me crazy.
Logged

It's the things you learn after you know it all that really count.
-John Wooden
vsloathe
vim ftw!
Global Moderator
Lifer
*****
Offline Offline

Posts: 1598



View Profile
« Reply #6 on: September 11, 2008, 08:25:29 PM »

Free lint?
Logged

thedarkness
Lifer
*****
Offline Offline

Posts: 585



View Profile
« Reply #7 on: September 11, 2008, 08:32:13 PM »

I am so glad I took the time to post this  Roll Eyes

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
dink
Expert
****
Offline Offline

Posts: 332


View Profile
« Reply #8 on: September 11, 2008, 08:37:24 PM »

I am so glad I took the time to post this  Roll Eyes

Cheers,
td

:sucker.gif:   ROFLMAO

@perk . . . no lint in your beer.  It would be like sand in your KY Jelly.
No good, free or not.
Logged

[quote Nutballs]
the universe has a giant fist, and its got enough whoop ass for everyone.
[/quote]
deregular
Expert
****
Offline Offline

Posts: 169


View Profile
« Reply #9 on: September 11, 2008, 08:50:08 PM »

Quote
@perk . . . no lint in your beer.  It would be like sand in your KY Jelly.
No good, free or not.
yeah perk you remember that time when you and dink ..... sandy...  jelly ... oops wrong forum  Grin

@td hey dude long time no hear
Logged
thedarkness
Lifer
*****
Offline Offline

Posts: 585



View Profile
« Reply #10 on: September 11, 2008, 08:54:16 PM »

@td hey dude long time no hear

Hey dereg,

How goes?

IM me some time.

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
perkiset
Olde World Hacker
Administrator
Lifer
*****
Offline Offline

Posts: 8649



View Profile
« Reply #11 on: September 11, 2008, 09:17:40 PM »

F'reals, nice post TD. Thanks for it, and thanks for being seen back around. Seems like it's been a while.
Logged

It's the things you learn after you know it all that really count.
-John Wooden
thedarkness
Lifer
*****
Offline Offline

Posts: 585



View Profile
« Reply #12 on: September 11, 2008, 10:05:45 PM »

F'reals, nice post TD. Thanks for it, and thanks for being seen back around. Seems like it's been a while.

It has, been super busy, still am. Flat out making sweet FA but I can't stay away from you guys, even though I know you're bad for me  ROFLMAO

I'll try and post some more stuff in the near future.
 
Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
nutballs
Administrator
Lifer
*****
Online Online

Posts: 5296


DAMN YOU!!!!!!!


View Profile
« Reply #13 on: September 11, 2008, 10:10:13 PM »

i'll ask the obvious question.

WTF is QT4? lol
Logged
thedarkness
Lifer
*****
Offline Offline

Posts: 585



View Profile
« Reply #14 on: September 11, 2008, 10:22:36 PM »

i'll ask the obvious question.

WTF is QT4? lol

nuts, how's living under that rock workin' for ya?  ROFLMAO

http://trolltech.com/products/qt

Cheers,
td
Logged

"I want to be the guy my dog thinks I am."
 - Unknown
Pages: [1] 2
  Print  
 
Jump to:  

Perkiset's Place Home   Best of The Cache   phpMyIDE: MySQL Stored Procedures, Functions & Triggers
Politics @ Perkiset's   Pinkhat's Perspective   
cache
mart
coder
programmers
ajax
php
javascript
Powered by MySQL Powered by PHP Powered by SMF 1.1.2 | SMF © 2006-2007, Simple Machines LLC
Seo4Smf v0.2 © Webmaster's Talks


Valid XHTML 1.0! Valid CSS!