Pidea

First post here . . .

I've developed a proxy checking and rating system that basically hits a known URL directly and then via the proxy that I'm testing.  I time both so can work out how much slower the proxy is compared to direct access and this is all stored in a database along with the overall reliability (number of tests passed in the last 24 hours).

The problem I have is sometimes a proxy that I have previously tested causes my script to completely hang.  When I access the proxy directly it's because the admin has introduced authentication.  Obviously I want to detect that the proxy now requires authentication so that I can mark it as inactive but how can I do this ?

Here's the code that I'm using to use the proxy:


function useProxy($url, $proxy, $port) {
// Function to retrieve page content using the CURL library compiled into

PHP

 

// Build proxy string
$proxyAddress=$proxy . ":" . $port;

$ch = curl_init ();
// We're  going to use a proxy
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0); // Note 0=on, 1=off

// Specify proxy location
curl_setopt($ch, CURLOPT_PROXY, $proxyAddress);

// Specify that we want output rather than report on success or failure
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);

// Define URL to get
curl_setopt ($ch, CURLOPT_URL, $url);

// Follow any headers that redirect us
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);

// Set timeout in case we don't get anything back
curl_setopt ($ch, CURLOPT_TIMEOUT, 30);

// Masquerade as IE6 on Windows XP :-)
curl_setopt ($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1;

.NET

  CLR 1.1.4322;

.NET

  CLR 1.0.3705)');

// Go and get it !
$content = curl_exec ($ch);
curl_close ($ch);

return $content;
}


Although I'm fairly au-fait with

PHP

 , this bloody cURL stuff is baffling !  Is there an option that I should set to catch the authentication request ?  Because the script hangs $content doesn't return any value so I can't scan it looking for any tell tale clues ?

Or should I dump cURL and use something else ?  Thanks in advance !

perkiset

So the timeout has no effect, because the problem is not a hang, but there is an auth request? Is that the issue?

Also, have you looked at CURLOPT_HTTPAUTH and such  - although I have not used that stuff, I would assume that an "invalid auth" response would at least handle the hang...

Pidea

quote

So the timeout has no effect, because the problem is not a hang, but there is an auth request? Is that the issue?


Absolutely right.

quote

Also, have you looked at CURLOPT_HTTPAUTH and such  - although I have not used that stuff, I would assume that an "invalid auth" response would at least handle the hang...


From what I can work out CURLOPT_HTTPAUTH is used to specify a username and password to use.  I've not figured out how catch the response.

perkiset

I guess what I'm suggesting, is if you send an invalid username and password, can you get it to bomb - so that the response would at least give you a failure or something?

Other than that, I have not seen what an auth request looks like in HTTP... what you might try, is to ask for headers only and see what you get - I am guessing that passing your initial request but seeing an auth request (or whatever it is) in the headers would let you know that you can't go any further. I'm personally in uncharted waters, but that's the angle I'd take to pull the challenge apart.

Bompa

I'm suprised that you don't get a 401 status code.



Bompa

Pidea

quote

I guess what I'm suggesting, is if you send an invalid username and password, can you get it to bomb - so that the response would at least give you a failure or something?


Good idea, I'll try that now

quote

I'm suprised that you don't get a 401 status code.


What I should get is a 407 and if I switch on LiveHeaders in Firefox that's what I see.  I'll try perkiset suggestion and will let you know how I get on.

Thanks guys

Bompa

quote author=Pidea link=topic=417.msg2725#msg2725 date=1185526621

quote

I guess what I'm suggesting, is if you send an invalid username and password, can you get it to bomb - so that the response would at least give you a failure or something?


Good idea, I'll try that now

quote

I'm suprised that you don't get a 401 status code.


What I should get is a 407 and if I switch on LiveHeaders in Firefox that's what I see.  I'll try perkiset suggestion and will let you know how I get on.

Thanks guys


Ok, 407.  So, you *are* getting that with Firefox, but not with your code?

In

perl

  we just use like $status = $res->status_line;

Why doesn't your code return the same status code as Firefox?

Bompa

mrsdf

The easy way is to check with invalid user/password. The other way is to actually check the headers, this may be useful when you just want to get something out of the headers.

After looking at the curl error codes I didn't see any 407 reference, and the curl authors don't really care about making the lib useful and easy to use at the same time
( http://curl.haxx.se/mail/lib-2006-09/0272.html ) , here's how to actually check the headers for the error.

php

  curl_setopt  ->> CURLOPT_HEADERFUNCTION
quote from the manual:
quote

The name of a callback function where the callback function takes two parameters. The first is the cURL resource, the second is a string with the header data to be written. The header data must be written when using this callback function. Return the number of bytes written.

I know it says 'written' and not 'read', but this function actually checks the headers receiver from the server.

The headerfunction allows you to check for some text in the headers, but unfortunately the only way (I know) to get data out of it is to use a global var. I've been using something like this to get the cookies out of the header into an array. So here is what it should look like:

function headerfunc($ch,$data)
{
        global $somevar;
        //replace function with stristr 407 or preg_match or something
        if(something_looks_wrong_in_headers($data))
        {
              $somevar=TRUE;
        }
        return strlen($data);
}

Initialize $somevar with FALSE just before doing curl_exec and check its value afterwards.

$ch=curl_init();
.....
.....
curl_setopt($ch,CURLOPT_HEADERFUNCTION,'headerfunc');
$somevar=FALSE;
$response=curl_exec($ch);
if($somevar)
{
  do_something();
}


Perkiset's Place Home   Politics @ Perkiset's