Selenium proxy with authentication

Issue

I have to use selenium and proxy with authentication.
I have a few constraints

1. I can’t use selenium-wire (only pure selenium allowed)
2. I have to use headless mode (e.g. chrome_options.add_argument("--headless"))

I read this answer Python proxy authentication through Selenium chromedriver but it doesn’t work for headless mode.

Is it possible to use proxy with authentication in my case? (browser(Chrome, Firefox) is not important)

---

I need python function to create selenium webdriver object authenticate to proxy

Solution

you can’t because you need a GUI to handle it with selenium in your case
so I would recommend using a virtual display like Xvfb display server

You can use PyVirtualDisplay (a Python wrapper for Xvfb) to run headless.

for Linux

sudo apt-get install firefox xvfb

install virtual display for python

pip install pyvirtualdisplay

then

from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(800, 600))
display.start()

# now Firefox will run in a virtual display. 
# you will not see the browser.
browser = webdriver.Firefox()
browser.get('http://www.google.com')
print browser.title
browser.quit()

display.stop()

in this case, you do not need to add options.add_argument("--headless") argument and you can follow the answers commented above as a solution or doing it your way but I think this is the best solution for using pure selenium for proxy

Answered By – CatChMeIfUCan

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0