From the course: pandas Essential Training

Unlock the full course today

Join today to access over 24,900 courses taught by industry experts.

Working with duplicates

Working with duplicates

From the course: pandas Essential Training

Start my 1-month free trial Buy for my team

Working with duplicates

“

- [Instructor] Have you ever been at a magic show where the magician pulls the same rabbit out of the hat multiple times? Well, having duplicate data in your pandas DataFrame is kind of like that, except it's not nearly as funny or entertaining. So let's take a look at our DataFrame nw, and you can see that we've got 10 columns and 2048 rows. Now, if we look at the duplicated method, and we sum them. Actually, before we do that, let's just take a look at what the duplicated method is all about. And you can see that the duplicated method returns a boolean series denoting the duplicate rows. And so, you can see that we've got six duplicates here. And so if I want to go ahead and display them, these are the six rows that are duplicates. You can see we've got identical entries for every single row. So we've got 2048 rows and 10 columns. So if we go ahead and drop the duplicates, we should find that we're now at 2042 rows, which is exactly what we have here. Now, let's say I want to use…

Contents