Here is the link: "https://btlaesthetics.com/en/find-a-physician"
I have searched a zip code '6292' in location search box by using selenium (as shown in code below). And it searches and shows results. Now it clicks on "Show details" of each clinic(section) 1 by 1, and it does so, but the main issue I'm facing is that, it scrapes data(telephone, email, website) of 1st section, each time it clicks on each section's detail, rather than scraping each section's own data. Here is the code:
from selenium import webdriver
from bs4 import BeautifulSoup
import time
options = webdriver.ChromeOptions()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(executable_path='D:\python\chromedriver\chromedriver.exe', options=options)
driver.get("https://btlaesthetics.com/en/find-a-physician")
driver.maximize_window()
element = driver.find_element_by_xpath("//input[@class='inp-text filter-address-input pac-target-input']")
element.send_keys("6292")
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
element.send_keys(u'\ue007')
time.sleep(3)
html = driver.page_source
soup = BeautifulSoup(html, 'html.parser')
item_provider_near = soup.find_all('div', class_='item-provider')
for each in range(len(item_provider_near)):
each = "(//a[@class='toggle-more'])[" + str(each+1) + "]"
driver.find_element_by_xpath(each).click()
time.sleep(3)
phone = soup.find('span',class_='provider-phone').text
email = soup.find('span',class_='provider-email').text
website = soup.find('span',class_='provider-website').text
if phone:
print(phone)
else:
pass
if email:
print(email)
else:
pass
if website:
print(website)
else:
pass
driver.quit()
At the end, after scraping, it shows error: selenium.common.exceptions.elementnotinteractableexception message element not interactable
