
![]() |
jubegnx
hey its me again, i need some help with the following...
#!/usr/bin/ perl-wuse WWW::Mechanize; use HTTP::Cookies; my $cookiefile = 'cookies.txt'; my $cjar = HTTP::Cookies->new(file => $cookiefile, autosave => 1, ignore_discard => 1); my $mech = WWW::Mechanize->new(cookie_jar => $cjar, autocheck => 1); $mech->get("https://www.google.com/accounts/ServiceLoginBox?service=blogger&continue=http%3A%2F%2Fwww.blogger.com%2Floginz%3Fd%3Dhttp%253A%252F%252Fwww.blogger.com%252Fcreate-blog.g&passive=true&alwf=true&uilel=3&skipvpage=true&rm=false&naui=8&showra=1&fpui=2&hl=en&nui=1&alinsu=1"); $mech->submit_form( fields => { 'Email' => 'email', 'Passwd' => 'password', } ); $mech->get("https://www.google.com/accounts/CheckCookie?continue=http%3A%2F%2Fwww.blogger.com%2Floginz%3Fd%3Dhttp%253A%252F%252Fwww.blogger.com%252Fcreate-blog.g&service=blogger&hl=en&chtml=LoginDoneHtml&skipvpage=true&alinsu=1&naui=8"); $mech->save_content("test.html"); now when i open test.html the following is in there... <html> <head> <title> Redirecting </title> <meta content="0; url='http://www.blogger.com/loginz?d=http%3A%2F%2Fwww.blogger.com%2Fcreate-blog.g&auth=DQAAAGUAAAAAy9PgJmCt88c7CV9EHi8MUzBbwiw3gd8OnVdaXAkfCxaAUHg-AqC1u2MdTphIcMXBNFr0u-DzFgxXtqay_q_qSIks-OKJPdrcp2U2m6Uzo1uMymMhEyTHHV-r12ShsW7sjkUikx7Egl4PrFjVRwys'" http-equiv="refresh"> </head> <body alink="#ff0000" text="#000000" vlink="#551a8b" link="#0000cc" bgcolor="#ffffff"> <script type="text/ javascript" language="javascript"><!--location.replace("http://www.blogger.com/loginz?du003dhttp%3A%2F%2Fwww.blogger.com%2Fcreate-blog.g&authu003dDQAAAGUAAAAAy9PgJmCt88c7CV9EHi8MUzBbwiw3gd8OnVdaXAkfCxaAUHg-AqC1u2MdTphIcMXBNFr0u-DzFgxXtqay_q_qSIks-OKJPdrcp2U2m6Uzo1uMymMhEyTHHV-r12ShsW7sjkUikx7Egl4PrFjVRwys" ![]() //--> </script> </body> </html> so now how can i catch that redirect and follow it? if i use mech->find_link it finds that link but when it follows it, it goes to the google homepage. and when i manually copy and paste the link into the browser it takes me to the right page. dirk
Have you tested the $mech->redirect_ok() option?
Bompa
Yah, like Dirk is trying to say, mech should follow redirects for you. You just gotta
enable it or find the right thingamabobbie. Bompa jubegnx
how would i use the redirect_ok option?
dirk
You could add the line:
$mech->redirect_ok(); jubegnx
i tried that and i got the following --> Can't call method "request" on an undefined value at /usr/lib/
perl5/site_perl/5.8.8/LWP/UserAgent.pm line 537.i read the lwp user agent manual and tried this also --> push @{ $mech->requests_redirectable }, 'POST'; but still no luck... jubegnx
sometimes i feel like drop kicking me in the face... the reason the find_link didn't work is because it had the ' character enclosing it all i had to do was remove that lol...
dirk
Great! So we can skip the 'Can't call method "request"' problem.
Bompa
quote author=jubegnx link=topic=338.msg2381#msg2381 date=1182653037 sometimes i feel like drop kicking me in the face... the reason the find_link didn't work is because it had the ' character enclosing it all i had to do was remove that lol... Glad to hear it. However, you are still *manually* following a redirect which means lots more code for you, which means more bugs. Sooner or later you will come accross more than one redirect; it sucks to do that manually. Last week I was logging on to a webmail account and it has FIVE redirects! But with redirect enabled (in LWP), I didn't have to do shit to follow them. Bompa jubegnx
bompa, how did you enable the redirects to follow them automatically, can you show me for the lwp and i'l figure it out for mechanize like that time with the cookies. its not as simple as $mech->redirect_ok(1)
Bompa
quote author=jubegnx link=topic=338.msg2389#msg2389 date=1182711312 bompa, how did you enable the redirects to follow them automatically, can you show me for the lwp and i'l figure it out for mechanize like that time with the cookies. its not as simple as $mech->redirect_ok(1) Sure. With LWP, POST is not normally redirectable, so it needs to be added. To keep it in context, here's a block of the code that I was talking about hitting five redirects: $ua = LWP::UserAgent->new(requests_redirectable => [ 'GET', 'HEAD', 'POST' ] ); $ua->credentials( 'mysite.com:2095', 'WebMail', 'myusername' => 'mypassword' ); $ua->max_redirect( 10 ); $ua->timeout('30'); $ua->agent("Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1"); $ua->cookie_jar( {} ); ---------------------------------- Normally, that $ua line would just look like this: $ua = LWP::UserAgent->new(); Here is where I first learned it, (credit nop_90 for the link, he gave it to me).www. perlmonks.org/?node_id=380264Bompa jubegnx
thanks for the help bompa, but i think i'm screwed as far as mech following redirects automatically goes... from the
perlmonks site"WWW::Mechanize automatically follows HTTP redirects, if not, you can subclass WWW::Mechanize to override the redirect_ok() subroutine. Also, since WWW::Mechanize is itself a subclass of LWP::UserAgent, it might help to set max_redirect() to some higher value, and/or set request_redirectable(). see the LWP::UserAgent documentation. However, your website might use META tags or javascriptfor "redirects". If so, you have to do the redirection yourself. Depending on the exact format of the page, you could get away with a simpleregex, or maybe you need an HTML:![]() i used the request_redirectable() like you said, i tried the redirect_ok and set it to return true always, but it did exactly the same if i just didn't set any of those things. and blogger has the meta refresh so i guess i have to do it manually ![]() btw: for anyone that ends up reading this thread and is stuck here i found a thread for this topic "WWW::Mechanize follow meta refreshes" --> http:// perlmonks.org/?node_id=447314Bompa
quote author=jubegnx link=topic=338.msg2396#msg2396 date=1182801520 thanks for the help bompa, but i think i'm screwed as far as mech following redirects automatically goes... from the perlmonks sitewell, that sucks, but anyways, you're getting the link and parsing it with a regexnow?then just do another mech call to follow it? Bompa |

Thread Categories

![]() |
![]() |
Best of The Cache Home |
![]() |
![]() |
Search The Cache |
- Ajax
- Apache & mod_rewrite
- BlackHat SEO & Web Stuff
- C/++/#, Pascal etc.
- Database Stuff
- General & Non-Technical Discussion
- General programming, learning to code
- Javascript Discussions & Code
- Linux Related
- Mac, iPhone & OS-X Stuff
- Miscellaneous
- MS Windows Related
- PERL & Python Related
- PHP: Questions & Discussion
- PHP: Techniques, Classes & Examples
- Regular Expressions
- Uncategorized Threads