DEV Community

Cover image for Web Browser Automation with Python
codesharedot
codesharedot

Posted on

Web Browser Automation with Python

#selenium #python

The selenium module is used to control web browsers. You can control Chrome, Firefox, Chrome Mobile and many other browsers directly from Python code. You should know the basics of Python.

Install

Install the selenium module and web driver. Use the python package manager pip to install the module.

pip3 install selenium
Enter fullscreen mode Exit fullscreen mode

Each browser has a specific web driver. You can find a list of drivers on the selenium website.

For Chrome you need chromedriver. For Firefox you need geckodriver. The version of the driver needs to match the browser version on your pc.

Selenium

After the driver and module are installed, you can fire up a browser. First thing to do is import the selenium module and the time module.

Then start a browser instance 

#!/usr/bin/python3 browser=webdriver.Firefox()
Enter fullscreen mode Exit fullscreen mode

Get the webpage:

#!/usr/bin/python3 browser.get("https://twitter.com")
Enter fullscreen mode Exit fullscreen mode

So you have this code

#!/usr/bin/python3 from selenium import webdriver import time # start web browser browser=webdriver.Firefox() # get source code browser.get("https://twitter.com") html = browser.page_source time.sleep(2) print(html) # close web browser browser.close()
Enter fullscreen mode Exit fullscreen mode

This then opens the browser and shows the HTML. You can interact with the browser just as you would normally do: click elements, type, scroll and much more.

Related links:

Top comments (2)

Collapse
 
davidhullster profile image
spongyBrain •

Nice quick intro to Selenium! I like the brevity.
It didn't occur to me that I can run Selenium from Python. Derp!

If anybody else gets this error 'FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver'', there's a nice StackOverflow article here: StackOverflow that explains how to get the geckodriver binary for your OS.

Collapse
 
programingtube profile image
Big Box •

Any best alternative please