My python code output wrong html data

Question

I have this code snippet as part of python code to crawl a particular website (see the code below). But to my surprise the output code is not an html. I am using python 3.4

   import urllib.request as ur
   user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
   headers = { 'User-Agent' : user_agent }

   s = ur.urlopen('http://www.nairaland.com')
   pl = s.read()
   print(pl)

My output from this code is:

b''

rather than the expected html code. Please guide me towards getting this code work. I need the html code in another part of the code. Thanks in advance.

Robᵩ · Accepted Answer · 2015-04-19 22:08:22Z

1

The excellent requests library returns the correct HTML:

import requests
s = requests.get('http://www.nairaland.com')
pl = s.text
print(pl)

answered Apr 19, 2015 at 22:08

Robᵩ

170k20 gold badges251 silver badges323 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

My python code output wrong html data

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related