next »

[mashal]

hi im scraping this site http://www.cleartrip.com/ with following code but it giving me not data
$target ='http://www.cleartrip.com/flights/results?from=CCU&to=DEL&depart_date=22/06/2012&adults=1&childs=0&infants=0&dep_time=0&class=Economy&airline=&carrier=&x=57&y=16&flexi_search=yes&tb=n';

   $data=file_get_contents($target);
   echo $data;

[perkiset]

I'll have to look from a real machine (not my pad) in the morning, but it looks as though the results are being delivered via AJAX or another out of band method. If this is the case then the page (pulled as you have) would never show results.

I'll check it out tomorrow and comment further. Might have an idea how to get around it.

[mashal]

ok thx alot for the help i try with a differnt method too all other websites that i scrap it give me required result but this website cleartip.com isnt giving me anything im new to scrapping world so im confused tried with <textarea> too but no success

[perkiset]

The textarea tip in this case will not help you.

Are you familiar with AJAX?

[Bompa]

Just a thought to use their mobile page:

If this is the data you need:

Code:

Get instant access to Cleartrip on your BlackBerry. Download the Cleartrip App »
Cleartrip logo
Kolkata - New Delhi
Fri, 22 Jun | 1 adult
modify | filter
sort by: price | time
Select a flight (1-10 of 120)
IndiGo
07:05 - 09:20
2h 15m, non-stop , 6E 212
Rs. 7,841
Select »
SpiceJet
07:20 - 09:30
2h 10m, non-stop , SG 608
Rs. 7,841
Select »
IndiGo
08:45 - 10:55
2h 10m, non-stop , 6E 228
Rs. 7,841
Select »
IndiGo
11:45 - 14:00
2h 15m, non-stop , 6E 236
Rs. 7,841
Select »
SpiceJet
12:00 - 14:10
2h 10m, non-stop , SG 255
Rs. 7,841
Select »
GoAir
14:35 - 16:45
2h 10m, non-stop , G8 712
Rs. 7,841
Select »
IndiGo
16:40 - 18:55
2h 15m, non-stop , 6E 206
Rs. 7,841
Select »
SpiceJet
17:10 - 19:20
2h 10m, non-stop , SG 219
Rs. 7,841
Select »
IndiGo
20:35 - 22:45
2h 10m, non-stop , 6E 224
Rs. 7,841
Select »
Air India
07:00 - 09:05
2h 5m, non-stop , AI 763
Rs. 8,072
Select »
1 - 10 of 120 flights « prev | next »
Home | Flights | Trains | Trips
Sign in to your Cleartrip account
Cleartrip's desktop site »
© 2006 - 2012 Cleartrip
Just a moment...

All I did was add "android" to my usual user agent string.


Bompa

[mashal]

yeah i want this info but im not geting how u getng it?

[mashal]

@Bompa can you please tell me where you add android? i added the user agent in firefox? how can i add android to string?

[mashal]

@ bompa i also get this thing but problem is i want to get this info through php code not by any user agent

[Bompa]

What is useragent?
http://www.google.com/search?hl=en&safe=off&site=&source=hp&q=user+agent+string&oq=user+agent+string&aq=0&aqi=g10&aql=&gs_l=hp.1.0.0l10.1195.1195.0.2561.1.1.0.0.0.0.237.237.2-1.1.0...0.0.5YvS6A7KLaQ


How to change useragent?
http://www.google.com/search?hl=en&safe=off&site=&source=hp&q=how+to+change+useragent+string&oq=how+to+change+useragent+string&aq=f&aqi=g-l3g-lK7&aql=&gs_l=hp.3..0i13l3j0i13i30l7.1932.8383.0.9026.34.25.2.7.7.1.338.2860.9j15j0j1.25.0...0.0.4VoF_NsTVPw

Afaik, To do this with php code, is not as easy as just getting a page like before.

If you really need it, you have to research "scraping with curl"

http://www.google.com/search?hl=en&safe=off&q=scraping+with+curl&oq=scraping+with+curl&aq=f&aqi=g1g-m1g-mK5g-bK2&aql=&gs_l=serp.3..0j0i5j0i5i30l5j0i8i30l2.289288.292623.0.293104.18.17.0.0.0.0.349.2299.8j6j2j1.17.0...0.0.FnkrFJRDOFk


Bompa

PS: It is most important for you to use Google.  It answers immediately,
which is better than waiting for us.

[mashal]

i tried with this code
$url = "http://www.cleartrip.com/flights/results?from=CCU&to=DEL&depart_date=22/06/2012&adults=1&childs=0&infants=0&dep_time=0&class=Economy&airline=&carrier=&x=57&y=16&flexi_search=no&tb=n";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page ;

but 2nd page info about flights are not displayed evry thing else are being displayed i dnt know wts the matter is???

P.S i am not new to computer so i use to google my problem but im new to scraping thats why em bothering you

[Bompa]

Try this:

Code:

$url = "http://www.cleartrip.com/m/flights/results?from=CCU&to=DEL&depart_date=22/06/2012&adults=1&childs=0&infants=0&dep_time=0&class=Economy&airline=&carrier=&x=57&y=16&flexi_search=no&tb=n";
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "android Mozilla/5.0 (Linux; U; Android 0.5; en-us)");
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page ;

$fh = fopen( "SCRAPEDCONTENT.HTML", 'w');
fwrite($fh, "$curl_scraped_page\n");
fclose($fh);

[mashal]

thx alot......................................getng evry thing you are genious

[mashal]

i realy appreciate about the help but im struck with this http://www.expedia.co.in/  website it have no mobile version plus it is also java and ajax based
i tried previous code that you suggested it work ecellent for cleartip.com but with this website no help

[mashal]

one thing more if you reply that in cleartrip.com wiht this
"https://www.cleartrip.com/m/flights/results?from=CCU&to=DEL&depart_dd=22&depart_mmyyyyy=06%2F2012&return_dd=24&return_mmyyyyy=06%2F2012&adults=1&childs=0&infants=0&mobile=true&class=Economy&carrier=&dep_time=0&ret_time=0&airline_codes=ALL"
 address im not geting the roundtrip full information..............isnt any way to get data from these ajax or java based websites?? please help me out....

[mashal]

no answer..................................
im stuck with post method of CURL................................

$url = "https://www.cleartrip.com/m/flights/itinerary/68f7a81c4b-6cb8-4ebe-889d-250c956b399e/info";

            $sid='a8680776-7db7-4b9d-970c-83d7b8ebd260';
            $rnd_one='O';
            $from='DEL';
            $to='AMD';
            $depart_date='22/06/2012';
            $adults='1';
            $childs='0';
         $infants='0';
         $dep_time='0';
         $class='Economy';
         $airline='';
         $carrier='';
         $timestamp='';
         $companyid='110340';
         $source='MOBILE';
         $fromCityName='';
         $toCityName='';
         $preferred_flights='ALL,';
         $preferred_time='';
         $BIZ_ACTION_MODE='VIEW_ORDER_CAPTURE';
         $out_fare_key='supp_AMADEUS|si-a8680776-7db7-4b9d-970c-83d7b8ebd260|fk_S2_4793_1340366100000_M2SIP,V2IPJK_true_fk_9W-K_2514_1340379300000_M2SIP,V2IPJK_true_true';
         $out_price='9698';
         $out_no_legs='2';
         $out_leg_aircode_1='S2';
         $out_leg_aircode_2='9W-K';
     

//url-ify the data for the POST
//$fields_string='';
//foreach($fields as $key=>$value)
//{ $fields_string .= $key.'='.$value.'&'; }
//rtrim($fields_string,'&');

$ch = curl_init($url);
//print_r($ch);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($ch, CURLOPT_POST, 1);
 curl_setopt($ch, CURLOPT_POSTFIELDS    ,$sid.$rnd_one. $from.$to. $depart_date.$adults.$childs.$infants.$dep_time.$class.$airline.$carrier.$timestamp.$companyid.$source.$fromCityName.$toCityName.$preferred_flights.$preferred_time.$BIZ_ACTION_MODE.$out_fare_key.$out_price.$out_no_legs.$out_leg_aircode_1.$out_leg_aircode_2);
curl_setopt($ch, CURLOPT_USERAGENT, "android Mozilla/5.0 (Linux; U; Android 0.5; en-us)");
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page ;
page shows empty..........................