jubegnx

hey its me again, i need some help with the following...


#!/usr/bin/

perl

  -w

use WWW::Mechanize;
use HTTP::Cookies;

my $cookiefile = 'cookies.txt';
my $cjar = HTTP::Cookies->new(file => $cookiefile, autosave => 1, ignore_discard => 1);
my $mech = WWW::Mechanize->new(cookie_jar => $cjar, autocheck => 1);


$mech->get("https://www.google.com/accounts/ServiceLoginBox?service=blogger&continue=http%3A%2F%2Fwww.blogger.com%2Floginz%3Fd%3Dhttp%253A%252F%252Fwww.blogger.com%252Fcreate-blog.g&passive=true&alwf=true&uilel=3&skipvpage=true&rm=false&naui=8&showra=1&fpui=2&hl=en&nui=1&alinsu=1");

$mech->submit_form(
fields => { 'Email' => 'email',
    'Passwd' => 'password', } );

$mech->get("https://www.google.com/accounts/CheckCookie?continue=http%3A%2F%2Fwww.blogger.com%2Floginz%3Fd%3Dhttp%253A%252F%252Fwww.blogger.com%252Fcreate-blog.g&service=blogger&hl=en&chtml=LoginDoneHtml&skipvpage=true&alinsu=1&naui=8");

$mech->save_content("test.html");


now when i open test.html the following is in there...

    <html> <head> <title>  Redirecting  </title> <meta content="0; url='http://www.blogger.com/loginz?d=http%3A%2F%2Fwww.blogger.com%2Fcreate-blog.g&amp;auth=DQAAAGUAAAAAy9PgJmCt88c7CV9EHi8MUzBbwiw3gd8OnVdaXAkfCxaAUHg-AqC1u2MdTphIcMXBNFr0u-DzFgxXtqay_q_qSIks-OKJPdrcp2U2m6Uzo1uMymMhEyTHHV-r12ShsW7sjkUikx7Egl4PrFjVRwys'" http-equiv="refresh">  </head> <body alink="#ff0000" text="#000000" vlink="#551a8b" link="#0000cc" bgcolor="#ffffff"> <script type="text/

javascript

 " language="

javascript

 "><!--

    location.replace("http://www.blogger.com/loginz?du003dhttp%3A%2F%2Fwww.blogger.com%2Fcreate-blog.g&authu003dDQAAAGUAAAAAy9PgJmCt88c7CV9EHi8MUzBbwiw3gd8OnVdaXAkfCxaAUHg-AqC1u2MdTphIcMXBNFr0u-DzFgxXtqay_q_qSIks-OKJPdrcp2U2m6Uzo1uMymMhEyTHHV-r12ShsW7sjkUikx7Egl4PrFjVRwys"Applause
  //--> </script> </body> </html>


so now how can i catch that redirect and follow it?
if i use mech->find_link it finds that link but when it follows it, it goes to the google homepage. and when i manually copy and paste the link into the browser it takes me to the right page.

dirk

Have you tested the $mech->redirect_ok() option?

Bompa

Yah, like Dirk is trying to say, mech should follow redirects for you.  You just gotta
enable it or find the right thingamabobbie.


Bompa

jubegnx

how would i use the redirect_ok option?

dirk

You could add the line:

$mech->redirect_ok();

jubegnx

i tried that and i got the following --> Can't call method "request" on an undefined value at /usr/lib/

perl

 5/site_

perl

 /5.8.8/LWP/UserAgent.pm line 537.

i read the lwp user agent manual and tried this also --> push @{ $mech->requests_redirectable }, 'POST'; but still no luck...

jubegnx

sometimes i feel like drop kicking me in the face... the reason the find_link didn't work is because it had the ' character enclosing it all i had to do was remove that lol...

dirk

Great! So we can skip the 'Can't call method "request"' problem.

Bompa

quote author=jubegnx link=topic=338.msg2381#msg2381 date=1182653037

sometimes i feel like drop kicking me in the face... the reason the find_link didn't work is because it had the ' character enclosing it all i had to do was remove that lol...


Glad to hear it.  However, you are still *manually* following a redirect which means lots
more code for you, which means more bugs.  Sooner or later you will come accross more
than one redirect; it sucks to do that manually.  Last week I was logging on to a webmail
account and it has FIVE redirects!  But with redirect enabled (in LWP), I didn't have to do
shit to follow them.

Bompa

jubegnx

bompa, how did you enable the redirects to follow them automatically, can you show me for the lwp and i'l figure it out for mechanize like that time with the cookies. its not as simple as $mech->redirect_ok(1)

Bompa

quote author=jubegnx link=topic=338.msg2389#msg2389 date=1182711312

bompa, how did you enable the redirects to follow them automatically, can you show me for the lwp and i'l figure it out for mechanize like that time with the cookies. its not as simple as $mech->redirect_ok(1)



Sure. 

With LWP, POST is not normally redirectable, so it needs to be added.

To keep it in context, here's a block of the code that I was talking about hitting five redirects:

$ua = LWP::UserAgent->new(requests_redirectable => [ 'GET', 'HEAD', 'POST' ] );
$ua->credentials(
    'mysite.com:2095',
    'WebMail',
    'myusername' => 'mypassword'
  );
$ua->max_redirect( 10 );
$ua->timeout('30');
$ua->agent("Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1");
$ua->cookie_jar( {} );

----------------------------------

Normally, that $ua line would just look like this:
$ua = LWP::UserAgent->new();



Here is where I first

learn

 ed it, (credit nop_90 for the link, he gave it to me).

www.

perl

 monks.org/?node_id=380264


Bompa

jubegnx

thanks for the help bompa, but i think i'm screwed as far as mech following redirects automatically goes... from the

perl

  monks site

"WWW::Mechanize automatically follows HTTP redirects, if not, you can subclass WWW::Mechanize to override the redirect_ok() subroutine. Also, since WWW::Mechanize is itself a subclass of LWP::UserAgent, it might help to set max_redirect() to some higher value, and/or set request_redirectable(). see the LWP::UserAgent documentation.

However, your website might use META tags or

javascript

  for "redirects". If so, you have to do the redirection yourself. Depending on the exact format of the page, you could get away with a simple

regex

 , or maybe you need an HTML:Applausearser to find the redirection URL."

i used the request_redirectable() like you said, i tried the redirect_ok and set it to return true always, but it did exactly the same if i just didn't set any of those things. and blogger has the meta refresh so i guess i have to do it manually  Applause

btw: for anyone that ends up reading this thread and is stuck here i found a thread for this topic "WWW::Mechanize follow meta refreshes" --> http://

perl

 monks.org/?node_id=447314

Bompa

quote author=jubegnx link=topic=338.msg2396#msg2396 date=1182801520

thanks for the help bompa, but i think i'm screwed as far as mech following redirects automatically goes... from the

perl

  monks site


well, that sucks, but anyways, you're getting the link and parsing it with a

regex

  now?

then just do another mech call to follow it?


Bompa


Perkiset's Place Home   Politics @ Perkiset's