Scroll to the end of the infinite loading page using selenium Python


I’m scraping the follower names from the twitter using Selenium and that page is infinite, whenever I scroll down I can see new followers.
Somehow I want to go to the bottom of the page so that I can scrape all the followers.

while number != 5:
   number = number + 1

usernames = driver.find_elements_by_class_name(
for username in usernames:

Right now code is scrolling 5 times. I have put a static value but I don’t know how much scrolls are needed to reach the bottom of the page.


Use below code for infinite loading. It will keep scrolling until new elements are getting loaded i.e. page size is changing.

# Get scroll height after first time page load
last_height = driver.execute_script("return document.body.scrollHeight")
while True:
    # Scroll down to bottom
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    # Wait to load page
    # Calculate new scroll height and compare with last scroll height
    new_height = driver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
    last_height = new_height

Answered By – rahul rai

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published