Python Selenium - Find element by class and text

Question

I'm trying to paginate through the results of this search: Becoming Amazon search. I get a 'NoSuchElementException'..'Unable to locate element: < insert xpath here >

Here is the html:

<div id="pagn" class="pagnHy">
    <span class="pagnLink">
        <a href="/s/ref=sr_pg_2?rh=...">2</a>
    </span>
</div>

Here are the xpaths I've tried:

driver.find_element_by_xpath('//*[@id="pagn" and @class="pagnLink" and text()="2"]')

driver.find_element_by_xpath('//div[@id="pagn" and @class="pagnLink" and text()="2"]')

driver.find_element_by_xpath("//*[@id='pagn' and @class='pagnLink' and text()[contains(.,'2')]]")

driver.find_element_by_xpath("//span[@class='pagnLink' and text()='2']")

driver.find_element_by_xpath("//div[@class='pagnLink' and text()='2']")

If I just use find_element_by_link_text(...) then sometimes the wrong link will be selected. For example, if the number of reviews is equal to the page number I'm looking for (in this case, 2), then it will select the product with 2 reviews, instead of the page number '2'.

Andersson · Accepted Answer · 2018-12-03 20:44:34Z

2

You're trying to mix attributes and text nodes from different WebElements in the same predicate. You should try to separate them as below:

driver.find_element_by_xpath('//div[@id="pagn"]/span[@class="pagnLink"]/a[text()="2"]')

answered Dec 3, 2018 at 20:44

Andersson

52.8k18 gold badges83 silver badges132 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mariah Akinbi Over a year ago

this works! what are the attributes and text nodes in this example? Do you have any good resources for learning this?

Andersson Over a year ago

Attributes in this case are id and class: nodes that appears inside tag e.g. <tag_name attr_1="foo" attr_2="bar">...</tag_name>. In XPath you can refer to attributes as @attr_name or attribute:attr_name. Text node in your case is "2" - the value outside tag, e.g. <tag_name>parent_text_1<child_tag>child_text</child_tag>parent_text_2</tag_name>. You can use this cheat sheet to create well-formed XPath expressions (or CSS-selectors)

user10740899 · Accepted Answer · 2018-12-03 21:20:49Z

2

Sometimes it might be better to take a intermediate step and first to get the element which contains the results. Afterwards you just search within this element. Doing it this way you simplify your search terms.

from selenium import webdriver

url = 'https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&fieldkeywords=becoming&rh=i%3Aaps%2Ck%3Abecoming'
driver = webdriver.Firefox()
resp = driver.get(url)
results_list_object = driver.find_element_by_id('s-results-list-atf')
results = results_list_object.find_elements_by_css_selector('li[id*="result"]')

for number, article in enumerate(results):
    print(">> article %d : %s \n" % (number, article.text))

answered Dec 3, 2018 at 21:20

user10740899

Comments

cody · Accepted Answer · 2018-12-03 20:45:23Z

1

When I look at the markup, I'm seeing the following:

<span class="pagnLink">
    <a href="/s/ref=sr_pg_2?rh=...">2</a>
</span>

So you want to find a span with class pagnLink that has a child a element with the text 2, or:

'//*[@class="pagnLink"]/a[text()="2"]'

answered Dec 3, 2018 at 20:45

cody

11.2k3 gold badges27 silver badges39 bronze badges

Collectives™ on Stack Overflow

Python Selenium - Find element by class and text

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related