php curl script to get an aspx page's content

Question

I have been trying for about a day now to get an xml feed into my application. Not usually an issue and I have two other feeds coming into the application using the simpleXml method which I have attached below.

My issue is that this other page is an aspx page and it seems to have some sort of redirect or probably just using an aspx framework that is using a clean url.

This is the script I used for the other two data pulls that works fine.

$grb_feed_url = 'http://www.grb.uk.com/rss.php';
$grb_jobs = simplexml_load_file($grb_feed_url, 'SimpleXMLElement', LIBXML_NOCDATA);

That is great but when I try it for the url http://www.milkround.com/rss.aspx it returns nothing.

I then tried a cURL script, this one works fine for the godaddy example but return nothing for the Milkround url. Also strange is that if I remove the CURLOPT_FOLLOWLOCATION line or set it to 0 it returns with "object moved to here".

function get_data($url)
{
  $ch = curl_init();
  $timeout = 5;
  curl_setopt($ch,CURLOPT_URL,$url);
  curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
  curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,$timeout);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
  $data = curl_exec($ch);
  curl_close($ch);
  return $data;
}

$returned_content = get_data('http://www.milkround.com/rss');
print_r($returned_content);

/* example of a url that works using this script */
/* $returned_content = get_data('http://www.godaddy.com/hosting/website-builder.aspx'); */

MrCode · Accepted Answer · 2012-03-15 11:57:42Z

4

You need to include a User-Agent header in the cURL request, otherwise the site produces a 501 error:

curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/4");

answered Mar 15, 2012 at 11:57

MrCode

64.6k10 gold badges95 silver badges114 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user1271418 Over a year ago

Absolutely right, I can get the data with that addition. Legend!

Collectives™ on Stack Overflow

php curl script to get an aspx page's content

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related