Get HTML table into pandas Dataframe, not list of dataframe objects

Question

I apologize if this question has been answered elsewhere but I have been unsuccessful in finding a satisfactory answer here or elsewhere.

I am somewhat new to python and pandas and having some difficulty getting HTML data into a pandas dataframe. In the pandas documentation it says .read_html() returns a list of dataframe objects, so when I try to do some data manipulation to get rid of the some samples I get an error.

Here is my code to read the HTML:

df = pd.read_html('http://espn.go.com/nhl/statistics/player/_/stat/points/sort/points/year/2015/seasontype/2', header = 1)

Then I try to clean it up:

df = df.dropna(axis=0, thresh=4)

And I received the following error:

Traceback (most recent call last): File "module4.py", line 25, in
<module> df = df.dropna(axis=0, thresh=4) AttributeError: 'list'
object has no attribute 'dropna'

How do I get this data into an actual dataframe, similar to what .read_csv() does?

iamzeid · Accepted Answer · 2021-10-27 09:07:37Z

27

From https://pandas.pydata.org/pandas-docs/version/0.17.1/io.html#io-read-html, read_html returns a list of DataFrame objects, even if there is only a single table contained in the HTML content".

So df = df[0].dropna(axis=0, thresh=4) should do what you want.

edited Oct 27, 2021 at 9:07

iamzeid

1141 gold badge2 silver badges18 bronze badges

answered Jul 20, 2016 at 17:21

Laurent S

4,3563 gold badges30 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Aritesh · Accepted Answer · 2018-04-26 12:33:52Z

11

pd.read_html returns you a list with one element and that element is the pandas dataframe, i.e.

df = pd.read_html(url) ###<-- List

df[0] ###<-- Pandas DataFrame

edited Apr 26, 2018 at 12:33

Aritesh

2,1031 gold badge17 silver badges18 bronze badges

answered Apr 26, 2018 at 11:10

MiLe

2133 silver badges7 bronze badges

Collectives™ on Stack Overflow

Get HTML table into pandas Dataframe, not list of dataframe objects

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related