Feature and motivation

TL/DR on current Browser Version Caching:

• Is only done by one driver manager
• Provides very little benefit
• Can cause failures
• Can cause confusion

Recommendation:

• Keep driver version caching and driver caching and remove default browser version caching.
• Allow users (and bindings) to "opt-in" to browser caching if they feel it's important.

Cost vs Benefit

Driver caching

• Pro - reduces number of http calls sent (saves ~0.1 seconds each)
• Pro - allows using working driver even when not connected to internet
• Con - might not get the latest point/patch release for ~1 day
• Con - if user needs a patch release, they need to know to specify the full version in command (relatively obvious), or to clear the cache, or to wait 24 hours (both less obvious).

Conclusion: small gains, but very small tradeoff makes it worth it.

Browser caching

• Pro - reduces number of command line executions (this saves 0.03 seconds)
• Con - might use the wrong driver for ~1 hour
• Con - if the user is getting the wrong driver, they need to know how to clear the cache (not obvious)

Conclusion: very small time savings; essentially a rounding error in the latency from one http call, at the expense of potential errors and confusion.

Comparison of Existing Driver Managers

No Driver Caching

By default these implementations do not check browser version, and download the latest driver version. There is an optional toggle to get the latest based on the installed browser. They do not cache drivers:

• https://github.com/rosolko/WebDriverManager.Net
• https://www.npmjs.com/package/chromedriver

This implementation caches driver versions list from Mozilla, but not the drivers themselves:

• https://www.npmjs.com/package/geckodriver

This one gets latest version, but does no driver caching:

• https://github.com/seleniumbase/SeleniumBase

Driver version caching

These cache the driver themselves to minimize the number of http calls that need to be made:

• https://github.com/SergeyPirogov/webdriver_manager
• https://github.com/titusfortner/webdrivers/

Browser version caching

Driver & Driver version caching makes sense to minimize the number of http calls being sent unnecessarily.
There is only one implementation that also caches the browser / browser version, and that's the java webdriver manager.

Failure example:

Chromium based drivers work for version n & n+1.
If you have v105 and v106 gets installed, v105 driver will still work. This is fine if you are running tests regularly.
If you haven't run a test on a machine with a browser that hasn't been updated in a while, it can cause a failure as seen below:

SL-1495:~ titusfortner$ /Applications/Microsoft\ Edge.app/Contents/MacOS/Microsoft\ Edge --version Microsoft Edge 102.0.1245.30 SL-1495:~ titusfortner$ code/selenium/common/manager/macos/selenium-manager --browser edge INFO	/Users/titusfortner/.cache/selenium/msedgedriver/mac64/102.0.1245.44/msedgedriver SL-1495:~ titusfortner$ /Applications/Microsoft\ Edge.app/Contents/MacOS/Microsoft\ Edge --version Microsoft Edge 107.0.1418.28 SL-1495:~ titusfortner$ code/selenium/common/manager/macos/selenium-manager --browser edge INFO	/Users/titusfortner/.cache/selenium/msedgedriver/mac64/102.0.1245.44/msedgedriver 

Confusion Scenario:

• User runs tests and sees an old version or posts logs and someone tells them they have an old version and to update
• They update and re-run
• Even if it passes, they still get the old version of the driver

Options

1. Do nothing - very minor performance improvement, potential problems;
2. Remove browser caching entirely - very minor performance hit, no problems
3. Opt-in - Users who know what they are doing and understand tradeoffs can use it; Also Bindings can still get the performance benefit by storing the cache for the duration of the execution (all tests within a build/suite(s))
4. Opt-out - Allow the bindings to opt-out of the caching for tests run after the initial caching.

I recommend option 3.