Php cURL Web Scraping

Question

I want to scrape the Price of the mobile phone from the website url : http://www.flipkart.com/apple-iphone-5s/p/itmdv6f75dyxhmt4?pid=MOBDPPZZDX8WSPAT

If you view the code, the price is placed in the following SPAN

<div class="pricing line">
        <div class="prices" itemprop="offers" itemscope="" itemtype="http://schema.org/Offer">
                    <div>
                        <span class="selling-price omniture-field" data-omnifield="eVar48" data-eVar48="37500">Rs. 37,500</span> // Fetch this price
                    </div>
                    <span class="sticky-message">Selling Price</span>
            <meta itemprop="price" content="37,500"> 
            <meta itemprop="priceCurrency" content="INR">
        </div>
</div>

My code to fetch this so far is :

<?php
$curl = curl_init('http://www.flipkart.com/apple-iphone-5s/p/itmdv6f75dyxhmt4?pid=MOBDPPZZDX8WSPAT');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);

$page = curl_exec($curl);

if(!empty($curl)){ //if any html is actually returned

    $pokemon_doc->loadHTML($curl);
    libxml_clear_errors(); //remove errors for yucky html

    $pokemon_xpath = new DOMXPath($pokemon_doc);

    //get all the h2's with an id
    $pokemon_row = $pokemon_xpath->query('//h2[@id]');

    if($pokemon_row->length > 0){
        foreach($pokemon_row as $row){
            echo $row->nodeValue . "<br/>";
        }
    }
}

else
    print "Not found";
?>

This shows an error :

Fatal error: Call to a member function loadHTML() on a non-object in D:\xampp\htdocs\jiteen\php-scrape\phpScrape.php on line 9

What shall I do, I can't trace the error

May I suggest simplehtmldom.sourceforge.net Believe me, its awesome. And very easy to use too. — wandering-geek
– wandering-geek, Commented Jan 27, 2015 at 11:13
Hello @kkaosninja, I appreciate your help and time. But to be honest, I am not very much able to fulfill my requirement from that (may be because I didn't read the document thoroughly). Can you suggest me a short and faster way. ALso the code is tough to understand the file that I downloaded from there : simple_html_dom.html. — Jiteen
– Jiteen, Commented Jan 27, 2015 at 11:26

Kevin · Accepted Answer · 2020-01-28 02:36:33Z

7

First, you forgot to instantiate the DOMDocument class, (at least on the code you have in this question).

$curl = curl_init('http://www.flipkart.com/apple-iphone-5s/p/itmdv6f75dyxhmt4?pid=MOBDPPZZDX8WSPAT');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');

$page = curl_exec($curl);

if(!empty($curl)) { //if any html is actually returned

    $pokemon_doc = new DOMDocument;
    libxml_use_internal_errors(true);
    $pokemon_doc->loadHTML($page);
    libxml_clear_errors();

    $pokemon_xpath = new DOMXPath($pokemon_doc);

    $price = $pokemon_xpath->evaluate('string(//div[@class="prices"]/meta[@itemprop="price"]/@content)');
    echo $price;

    $rupees = $pokemon_xpath->evaluate('string(//div[@class="prices"]/div/span)');
    echo $rupees;
}
else {
    print "Not found";
}

Sample Output

edited Jan 28, 2020 at 2:36

answered Jan 27, 2015 at 11:13

Kevin

41.9k12 gold badges57 silver badges72 bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

Jiteen Over a year ago

Thanks for your time and advice. It removed the error, but now its a Blank page and I do not have the desired output i.e the Price from the URL. Please help me in that as well.

Kevin Over a year ago

@Jiteen which price is it anyway? the rupees?

Jiteen Over a year ago

Yes, it is in Rs. I beleive it was US DOllar we could have searched on the basis of '$' sign, I guess. As you may have noticed, I just want to fetch the value in the following line : ** <span class="selling-price omniture-field" data-omnifield="eVar48" data-eVar48="37500">Rs. 37,500</span>**

Kevin Over a year ago

@Jiteen oh okay, just as i thought, check out my revision, there's also a sample output

Jiteen Over a year ago

This works GREAT. Sorry to bother you again, I want to fetch the price of SnapDeal as well. Check this link: snapdeal.com/product/apple-iphone-5s-16-gb/… The Price lies in : <span id="selling-price-id" itemprop="price">36920</span> I want to know how shall I change the $price = $pokemon_xpath->evaluate('string(//div[@class="prices"]/meta[@itemprop="price"]/@content)'); to fetch the desired data.

|

Collectives™ on Stack Overflow

Php cURL Web Scraping

1 Answer 1

11 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

11 Comments

Your Answer

Sign up or log in

Post as a guest

Related