Extract text from span element using selenium in python

Question

I'm trying to extract the contact number (XXXXX-XXXXXX) from the following HTML code using Selenium webdriver's find_element_by_xpath method.

 <div id="contact-info">
                    <div class="a">
                        <div class="b">
                            <table class="c">
                                <tr>
                                    <td class="d">
                                        <div class="e">Contact Info</span></div>
                                    </td>
                                    <td class="f">
                                        <div></div>
                                    </td>
                                </tr>
                            </table>
                        </div>
                        <div class="g">
                            <div class="h">
                                <table>
                                    <tr>
                                        <td>
                                            <div class="i"><span class="j">Mobile</span></div>
                                        </td>
                                        <td class="k">
                                            <div class="random_class_name"><span><span dir="ltr">XXXXX-XXXXXX</span></span>
                                            </div>
                                        </td>
                                    </tr>
                                </table>
                            </div>

This is my code for extracting the required data-

print(browser.find_element_by_xpath('//div[@id="contact-info"]/div[1]/div[2]/div[1]/div[2]/span[1]/span[1]').text)

But I'm thrown with an exception saying the element can't be located. What am I possibly doing wrong here? How can I fix this?

KunduK · Accepted Answer · 2020-04-08 16:54:11Z

1

To get the Mobile-Number you can take the reference of the text Mobile and find the next sibling.

Induce WebDriverWait() and wait for visibility_of_element_located() and following xpath.

print(WebDriverWait(browser,10).until(EC.visibility_of_element_located((By.XPATH,"//td[.//div[contains(.,'Mobile')]]/following-sibling::td[1]/div"))).text)

You need to import following libraries.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

edited Apr 8, 2020 at 16:54

answered Apr 8, 2020 at 16:45

KunduK

33.4k5 gold badges19 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Abdullah Shahriar Over a year ago

I'm getting a TimeoutException this time

KunduK Over a year ago

@AbdullahShahriar : The xpath is fine.I have tested before posted as an answer.Could you just check if there any iframe above your table element?

Abdullah Shahriar Over a year ago

Sorry, I was scraping the wrong file. The solution you posted works fine! Thanks! :D

E.Wiest · Accepted Answer · 2020-04-08 16:40:04Z

1

Correct your XPath with :

print(browser.find_element_by_xpath('//span[@dir="ltr"]').text)

answered Apr 8, 2020 at 16:40

E.Wiest

5,9152 gold badges9 silver badges12 bronze badges

Comments

SeleniumUser · Accepted Answer · 2020-04-08 18:26:04Z

1

contactNumber = WebDriverWait(browser, 30).until(
            EC.element_to_be_clickable((By.XPATH, "//div[@class='random_class_name']//span[1]//span")))
print contactNumber.text

or

contactNumber = WebDriverWait(browser, 30).until(
    EC.element_to_be_clickable((By.XPATH, "//span[contains(text(), 'XXXXX-XXXXXX')]")))
print contactNumber.text

Note : add below imports to your solution :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

Upddated Section: Check your table is in the iframe if so then use below code to swiutch to iframe ;

iframe=driver.find_element_by_tag_name("iframe")
driver.switch_to.frame(iframe)

edited Apr 8, 2020 at 18:26

answered Apr 8, 2020 at 16:47

SeleniumUser

4,1973 gold badges15 silver badges45 bronze badges

6 Comments

SeleniumUser Over a year ago

check if your table in the iframe if so you need to switch to iframe before using above code: Please refer updated section to switch to iframe

Abdullah Shahriar Over a year ago

There's no iframe above the table element. But thanks for pointing it out. I was scraping the wrong file and so the error was raised. Your solution works completely fine!

Abdullah Shahriar Over a year ago

I have another question in mind, what if the class name random_class_name was not known, how can I access the element?

SeleniumUser Over a year ago

you mean same contact number element ? if so then you can directly use //span[contains(text(), 'XXXXX-XXXXXX')]

Abdullah Shahriar Over a year ago

No, assume 'XXXXX-XXXXXX' is also unknown

|

Collectives™ on Stack Overflow

Extract text from span element using selenium in python

3 Answers 3

3 Comments

Comments

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related