How can I extract the URLs from a website?


I am trying to get the list of URLs of each restaurant from this website . So far this is the code that I am trying to implement

Reproducible example

from selenium import webdriver
from import expected_conditions as EC
from import By
from selenium.webdriver.common.keys import Keys

driver_path = '/Users/driverpath'

from import Service
from import ChromeDriverManager
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))


cards = driver.find_elements(By.CSS_SELECTOR, "")
urls = [card.get_attribute('href') for card in cards]


I am getting None as result and I wonder if I am selecting in the wrong way the CSS of the div.


You need to get the a element inside the div with the restaurant-info class before you can get the href attribute from it:

from selenium import webdriver
from import By

driver = webdriver.Chrome()

infos = driver.find_elements(By.CLASS_NAME, "restaurant-info")
for info in infos:
    print(info.find_element(By.CSS_SELECTOR, "a").get_attribute("href"))


Answered By – Ann Zen

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published