136

I have a problem; I am using the Selenium (Firefox) web driver to open a webpage, click a few links, etc., and then capture a screenshot.

My script runs fine from the CLI, but when run via a cron job it is not getting past the first find_element() test. I need to add some debug, or something to help me figure out why it is failing.

Basically, I have to click a 'log in' anchor before going to the login page. The construct of the element is:

<a class="lnk" rel="nofollow" href="/login.jsp?destination=/secure/Dash.jspa">log in</a>

I am using the find_element By LINK_TEXT method:

login = driver.find_element(By.LINK_TEXT, "log in").click()

A) How do I check that the link is actually being picked up by Python? Should I use try/catch block?

B) Is there a better/more reliable way to locate the DOM element than by LINK_TEXT? E.g., in jQuery, you can use a more specific selector, $('a.lnk:contains(log in)').do_something();

I have solved the main problem and it was just finger trouble. I was calling the script with incorrect parameters - a simple mistake.

I'd still like some pointers on how to check whether an element exists. Also, an example/explanation of implicit / explicit Waits instead of using a crappy time.sleep() call.

Improve this question
2
  • You can also find elements by CSS locator or xpath. Tends to be less brittle than by text contents. Commented Mar 5, 2012 at 15:15
  • I believe that both - using content text and XPATH/CSS - are fragile to minor changes in design rather than application logic. A better way would be to find element by either id, class_name or name. Commented May 3, 2017 at 13:30

14 Answers 14

Reset to default
189

For a):

from selenium.common.exceptions import NoSuchElementException
def check_exists_by_xpath(xpath):
    try:
        webdriver.find_element_by_xpath(xpath)
    except NoSuchElementException:
        return False
    return True

For b): Moreover, you can take the XPath expression as a standard throughout all your scripts and create functions as above mentions for universal use.

I recommend to use CSS selectors. I recommend not to mix/use "by id", "by name", etc. and use one single approach instead.

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

You should also pass object: browser = webdriver.Firefox() to your custom function
In general, I have rarely come across html pages which conform to the idea of uniform matching. Is there a way to work around mix-matches of the locator methods on such pages?
@KshitijSaraogi Not sure completely understand your question. If you are staying that it's impossible to have only 'css selectors' in your webdriver tests - it's actually quite possible. I've been doing just that for the last 6 years.
@AlexOkrushko What are your reasons for recommending css selectors over xpath?
@BoZenKhaa, at least 2 reason: familiarity of css selectors to the devs, which contributes to the readability of the code; xpath had some terrible performance in IE.
|
128

You can grab a list of elements instead of a single element. An empty list in python is falsey. Example:

if driver.find_elements(By.CSS_SELECTOR, '#element'):
    print "Element exists!"

You can also use By.ID and By.NAME, but that just turns your id or name into a css selector anyway. Source

Improve this answer

14 Comments

Is this faster than the above method of Exceptions handling?
@GalBracha Good question, I'm assuming it would probably depend on the version and platform, so if the speed difference is that important to you here, I would recommend testing it on your target platform for the best results. However, there is a point to be made here about pre-optimizations and the code readability might be more important than the negligable performance improvement in this case
The find-element (singular) form throws an exception when the element is not found. The find-elements (plural with 's') will return an empty list. In my humble opinion all the other examples that are using try-catch to see if an element exist are bad programming style. Try-catch should only be used to catch unexpected situations. If you think the element may not exist your code should be written to handle that situation.
@BillWorthington agree 100%. I'm not a Python dev so I don't know what's normal around here, but I wouldn't write things that way in any other languages
@BrianLeishman I was trying to point out your example is the RIGHT way to do it and the OTHER examples were not good programming style.
|
71

A) Yes. The easiest way to check if an element exists is to simply call find_element inside a try/catch.

B) Yes, I always try to identify elements without using their text for two reasons:

  1. the text is more likely to change and;
  2. if it is important to you, you won't be able to run your tests against localized builds.

The solution is either:

  1. You can use XPath to find a parent or ancestor element that has an ID or some other unique identifier and then find its child/descendant that matches or;
  2. you could request an ID or name or some other unique identifier for the link itself.

For the follow-up questions, using try/catch is how you can tell if an element exists or not and good examples of waits can be found here: http://seleniumhq.org/docs/04_webdriver_advanced.html

Improve this answer

5 Comments

One use I have for finding text is to see whether a successful/unsuccessful flash message is displayed. Not sure if there's a better way to do it?
Those are absolutely excellent reasons to avoid selecting by text
This was helpful, note that you can now use is_displayed(): selenium.dev/documentation/webdriver/elements/information
The link is (effectively) broken. It redirects to a generic page.
@Raoul: Wouldn't that run into the problem of a thrown exception?
29

A solution without try&catch and without new imports:

if len(driver.find_elements_by_id('blah')) > 0: # Pay attention: find_element*s*
    driver.find_element_by_id('blah').click # Pay attention: find_element
Improve this answer

7 Comments

You're finding it twice, it would certainly be more efficient to store the collection, and if it's truthy, click the first, e.g. e = driver.find_elements_by_id('blah') if e: e[0].click
@BrianLeishman I agree yours way it's more efficient, but I think mine more easy to read and understand
Good solution, but I agree with, Brian, regarding you're finding twice.
From a comment: "find_element_by_* and find_elements_by_* are removed in Selenium 4.3.0. Use find_element instead.".
And from Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found."
|
7

The same as Brian, but add to this answer from tstempko:

I tried and it works quickly:

driver.implicitly_wait(0)

if driver.find_element_by_id("show_reflist"):
    driver.find_element_by_id("show_reflist").find_element_by_tag_name("img").click()

After this, I restore my default value:

driver.implicitly_wait(30)
Improve this answer

4 Comments

yes! if you know the page is already loaded, you dont want to wait around for 30 seconds to get your "doesnt exist return". this returns immediately, just what I needed.
Wouldn't that throw an exception if something doesn't exist (the whole point of this question)? And thus not really work?
OK, the OP has left the building: "Last seen more than 1 year ago". Perhaps somebody else can chime in?
From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found.". Will driver.implicitly_wait(0) prevent the exception from being thrown?
6

You could also do it more concisely using

driver.find_element_by_id("some_id").size != 0
Improve this answer

3 Comments

This will return an error. Check Lopi Dani for the reason and the correct way to do it.
Please note that size will return you the height and width of element.
Lopi Dani's answer.
2

When you know the element could not be there, the implicit wait could be a problem. I've created a simple context manager to avoid those waiting times

class PauseExplicitWait(object):
    """
    Example usage:

    with PauseExplicitWait(driver, 0):
        driver.find_element(By.ID, 'element-that-might-not-be-there')
    """
    def __init__(self, driver, new_wait=0):
        self.driver = driver
        self.original_wait = driver.timeouts.implicit_wait
        self.new_wait = new_wait
      
    def __enter__(self):
        self.driver.implicitly_wait(self.new_wait)
  
    def __exit__(self, exc_type, exc_value, exc_tb):
        self.driver.implicitly_wait(self.original_wait)
Improve this answer

Comments

2

driver.find_element_by_id("some_id").size() is a class method.

We need:

driver.find_element_by_id("some_id").size which is a dictionary, so:

if driver.find_element_by_id("some_id").size['width'] != 0:
   print 'button exist'
Improve this answer

3 Comments

this is not always right, the width can be zero if element exist also. el = driver.find_element_by_class_name('gNO89b') el.size this is a valid element in google search page but the width is 0
I don't think this will work. find_element_by_id() probably throws an exception if the element does not exist (the whole point of this question).
From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found."
1

You can find elements by available methods and check response array length if the length of an array equal the 0 element not exist.

element_exist = False if len(driver.find_elements_by_css_selector('div.eiCW-')) > 0 else True
Improve this answer

1 Comment

From a comment: "find_element_by_* and find_elements_by_* are removed in Selenium 4.3.0. Use find_element instead.". Though that would throw an exception if the element doesn't exist.
1

With the latest Selenium, you can now use you can now use .is_displayed():

https://www.selenium.dev/documentation/webdriver/elements/information/

Improve this answer

4 Comments

This answer helped me. I have written same answer with example and part A) and B) to give some pointers for both part of the questions because Raoul's suggested edit queue is full.
Can you elaborate? What is the "latest Selenium"? What version? When was it released? What version did you test it with? On what system? What is the first version that introduced this feature? Please respond by editing (changing) your question/answer, not here in comments (*************** without *************** "Edit:", "Update:", or similar - the question/answer should appear as if it was written today).
Wouldn't that throw an exception if the element doesn't exist (before is_displayed() gets a chance to be called)?
From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found."
1

You could use is_displayed() like below:

res = driver.find_element_by_id("some_id").is_displayed()
assert res, 'element not displayed!'
Improve this answer

2 Comments

Wouldn't that throw an exception if it doesn't exist (the whole point of this question)?
From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found."
0

You can check by find_elements. If the result is null, that element does not exist.

if driver.find_elements(By.SOMETHING, "#someselector") == []:
    continue   # That element does not exist
Improve this answer

1 Comment

From a comment: "find_element_by_* and find_elements_by_* are removed in Selenium 4.3.0. Use find_element instead.". Though that would throw an exception if the element doesn't exist.
0

A) Use .is_displayed() as explained at Information about web elements:

# After navigating to the URL,
# get the Boolean value for if this element is displayed
is_button_visible = driver.find_element(By.CSS_SELECTOR, "[name='login']").is_displayed()

Continue your Visual Studio Code logic using "is_button_visible" for branching.

B) Refer to Sam Woods's answer

Improve this answer

2 Comments

Wouldn't that throw an exception if the element doesn't exist (before is_displayed() gets a chance to be called) - the whole point of this question?
From Bill Worthington's comment: "The find-element (singular) form throws an exception when the element is not found."
-1 
el = WebDriverWait(driver, timeout=3).until(lambda d: d.find_element(By.TAG_NAME,"p"))

doc

Improve this answer

1 Comment

Please provide a short explanation with answers to describe how and why it works. Sharing a link to the documentation is good, but the answer should be able to stand on its own.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.