Python Selenium not staying within parent element when searching for child element

Issue

I am currently trying to scrape this yelp page ‘https://www.yelp.com/biz/taste-of-lebanon-west-springfield’. I first grab all the divs that contain their services so I can know if they offer takeout, delivery, etc. Then after I loop through to determine if they contain the green checkkmark span but anytime I search it does not stay within the parent element even though I am using that element as my search driver.

I am trying to find all the spans that contain this class ‘icon–24-checkmark-v2 css-yyirv3’ but whenever my script reaches the loop instead of staying within the parent element when searching the find_element_by_xpath will search the entire page so it always comes up as true even though there will not be a span with a matching class in that element. How do I get my search to stay within the parent element when trying to find a child element by xpath?

driver.get('https://www.yelp.com/biz/taste-of-lebanon-west-springfield')
try:
    el = WebDriverWait(driver, 30).until(EC.visibility_of_element_located((By.XPATH, "//div[@class=' photoHeader__373c0__YdvQE border-color--default__373c0__2oFDT']")))
    print("Page is ready!")
except TimeoutException:
    print("Loading took too much time!")

businessName = driver.find_element_by_xpath("//h1[@class='css-11q1g5y']").text
services = driver.find_elements_by_xpath("//div[@class=' display--inline-block__373c0__2de_K margin-r3__373c0__r37sx margin-b1__373c0__1khoT border-color--default__373c0__2oFDT']")
for service in services:
    try:
        checkMark = service.find_element_by_xpath("//span[@class='icon--24-checkmark-v2 css-yyirv3']")
        print("Checkmark found")  
        
    except NoSuchElementException:
        print("No checkmark found")  

Solution

Change this:

checkMark = service.find_element_by_xpath("//span[@class='icon--24-checkmark-v2 css-yyirv3']")

To this:

checkMark = service.find_element_by_xpath(".//span[@class='icon--24-checkmark-v2 css-yyirv3']")

The period . is needed to stay within that specific section of the given element service

Page is ready!
Checkmark found
Checkmark found
No checkmark found
Checkmark found
Checkmark found
Checkmark found
No checkmark found
No checkmark found
No checkmark found
No checkmark found
Checkmark found
Checkmark found
Checkmark found
Checkmark found
Checkmark found
Checkmark found
Checkmark found
Checkmark found
Checkmark found

Answered By – JD2775

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published