Python code only working for title tag not for table

Question

In regex when writing <title>(.+?)</title> it is working but when this title tag is change to <table>(.+?)</table> it gives '[]' (square brackets) as output. My code is :

import urllib
import re

urls = ["http://physics.iitd.ac.in/content/list-faculty-members", "http://www.iitkgp.ac.in/commdir3/list.php?division=3&deptcode=ME","http://www.iitkgp.ac.in/commdir3/list.php?division=3&deptcode=CE"]
i = 0
regex = '<table>(.+?)</table>'
pattern = re.compile(regex)

while i< len(urls):
    htmlfile = urllib.urlopen(urls[i])
    htmltext = htmlfile.read()
    tables  = re.findall(pattern,htmltext)

    print tables
    i+=1

mechanical_meat · Accepted Answer · 2016-07-18 16:16:40Z

1

Use BeautifulSoup:

import urllib
import re

from BeautifulSoup import BeautifulSoup as bs

urls = ["http://physics.iitd.ac.in/content/list-faculty-members", 
        "http://www.iitkgp.ac.in/commdir3/list.php?division=3&deptcode=ME", 
        "http://www.iitkgp.ac.in/commdir3/list.php?division=3&deptcode=CE"]
i = 0

while i < len(urls):
    htmlfile = urllib.urlopen(urls[i])
    htmltext = htmlfile.read()
    soup = bs(htmltext)
    tables = soup.find_all('table')

    print tables
    i+=1

edited Jul 18, 2016 at 16:16

answered Jul 18, 2016 at 16:11

mechanical_meat

170k25 gold badges237 silver badges231 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Disguise Over a year ago

Thank you it is working perfectly. :) One more question: What if i want the downloaded data from table should be displayed as in table structure only, can that happen?

mechanical_meat Over a year ago

Can you elaborate on what you need?

Disguise Over a year ago

yeah, what i want is the above link consists of professors data in table form and on running this code i get all their details but it cannot be understood by someone who don't know this. So i want if it could be downloaded in the same table format on cmd?

Disguise Over a year ago

Although i have posted this as another question with code that i have tried and the code you have helped with. May i send you the link of question? @bernie

mechanical_meat Over a year ago

Sure you may. I will answer that one

Collectives™ on Stack Overflow

Python code only working for title tag not for table

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related