Keep only an element of a webpage while web-scraping

Question

I am trying to extract a table from a webpage with python. I managed to get all the contents inside of that table, but since I am very new to webscrapping I don't know how to keep only the elements that I am looking for.

I know that I should look for this class in the code: <a class="_3BFvyrImF3et_ZF21Xd8SC", which specify the items in the table.

So how can I keep only those classes to then extract the title of them?

<a class="_3BFvyrImF3et_ZF21Xd8SC" title="r/Python" href="/r/Python/">r/Python</a>
<a class="_3BFvyrImF3et_ZF21Xd8SC" title="r/Java" href="/r/Java/">r/Java</a>

I miserably failed in writing a code for that. I don't know how I could extract only these classes, so any inputs will be highly appreciated.

Are you trying to find elements by class name? Try - stackoverflow.com/questions/30002313/… — xprilion
– xprilion, Commented Feb 2, 2023 at 21:35
What library are you using to scrape the webpage and transform the HTML? — nokko
– nokko, Commented Feb 2, 2023 at 21:48

undetected Selenium · Accepted Answer · 2023-02-02 21:49:14Z

1

To extract the value of title attributes you can use list comprehension and you can use either of the following locator strategies:

Using CSS_SELECTOR:

print([my_elem.get_attribute("title") for my_elem in driver.find_elements(By.CSS_SELECTOR, "a._3BFvyrImF3et_ZF21Xd8SC[title]")])

Using XPATH:

print([my_elem.get_attribute("title") for my_elem in driver.find_elements(By.XPATH, "//a[@class='_3BFvyrImF3et_ZF21Xd8SC' and @title]")])

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

answered Feb 2, 2023 at 21:49

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Raphael Borges · Accepted Answer · 2023-02-02 21:48:51Z

1

Okay, I have made a very simple thing that worked.

Basically I pasted the code on VSCODE and the selected all the occurrences of that class. Then I just had to copy and paste in another file. Not sure why the shortcut CTRL + Shift + L did not work, but I have managed to get what I needed.

Select all occurrences of selected word in VSCode

answered Feb 2, 2023 at 21:48

Raphael Borges

491 gold badge1 silver badge8 bronze badges

Collectives™ on Stack Overflow

Keep only an element of a webpage while web-scraping

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related