Google Git
Sign in
chromium / chromium / web-tests / ef3a15493718bab0583d8772d59dc1ec35ff3a4a
commitef3a15493718bab0583d8772d59dc1ec35ff3a4a[log] [tgz]
authorCharles William Dick <cwd@google.com>Wed Oct 08 00:36:47 2025
committerCharles Dick <cwd@google.com>Wed Oct 08 01:29:27 2025
tree3497224bc644fef6e8a9b448098623a5dbad40a2
parent05af5717ed0e1956edff23a0c1256fb9685af184 [diff]
Roll third_party/crossbench/ 1584e1021..6874c8656 (33 commits) https://chromium.googlesource.com/crossbench/+log/1584e10215b3..6874c86567d9 $ git log 1584e1021..6874c8656 --date=short --no-merges --format='%ad %ae %s' 2025-10-07 victorvianna Ignore 2 formatting commits in `git blame` 2025-10-07 chromium-autoroll Roll Buildtools from fb3a6ce2c0af to a9da2cb478f2 (1 revision) 2025-10-06 kalutes Fix quoting for inline hjson textproto 2025-10-06 victorvianna Reformat one more JS file 2025-10-06 victorvianna Fix inconsistency between isort and `git cl format` 2025-10-06 victorvianna Reformat python files 2025-10-06 victorvianna Reformat JS files 2025-10-06 khokhlov [LoadLine 2] A no-tracing version of the benchmark 2025-10-06 chromium-autoroll Roll Buildtools from 2325cd98715b to fb3a6ce2c0af (1 revision) 2025-10-03 chromium-autoroll Roll Buildtools from 02f9e66e820b to 2325cd98715b (1 revision) 2025-10-03 victorvianna Skip test_set_main_screen_brightness on all M1/M2 macs, not only in CQ 2025-10-03 cbruni Export pyi for protobuf files for better code completion 2025-10-03 cbruni Adb: Use AnyPath for apks 2025-10-03 cbruni Fix TraceProcessorProbe order warning 2025-10-02 chromium-autoroll Roll Buildtools from 674b131e2bf5 to 02f9e66e820b (1 revision) 2025-10-02 khokhlov [LoadLine 2] Move perfetto configs to separate files 2025-10-01 chromium-autoroll Roll Buildtools from c79eaa6eb273 to 674b131e2bf5 (1 revision) 2025-10-01 khokhlov Support invocation of JS probe via actions 2025-10-01 cbruni Move probe validation to Runner 2025-10-01 cbruni Refactor trace_processor and perfetto probe 2025-10-01 cbruni Remove pyfakefs hacks 2025-10-01 cbruni Display probe example configs in help 2025-09-30 cwd Dump heap platform method and probe 2025-09-30 chromium-autoroll Roll Buildtools from 3e6e75d1961a to c79eaa6eb273 (1 revision) 2025-09-30 stevensd Fix potential type error in webdriver timeout 2025-09-30 cbruni Use more Final and object type annotations 2025-09-29 cwd retry bond credentials, more logging 2025-09-29 chromium-autoroll Roll Buildtools from fe8567e14316 to 3e6e75d1961a (1 revision) 2025-09-29 cbruni Improve binary resolution 2025-09-29 cbruni Clean up platform temp dirs 2025-09-28 chromium-autoroll Roll Buildtools from 88a3274e7b0c to fe8567e14316 (1 revision) 2025-09-27 chromium-autoroll Roll Buildtools from 6b7117dcf645 to 88a3274e7b0c (1 revision) 2025-09-26 chromium-autoroll Roll Buildtools from 7ffbee862d9d to 6b7117dcf645 (1 revision) Created with: roll-dep third_party/crossbench Change-Id: Ic18c89b52fd7f221a19f5ea9889901b637ef3010 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/web-tests/+/7018610 Reviewed-by: Kameron Lutes <kalutes@google.com>
4 files changed
tree: 3497224bc644fef6e8a9b448098623a5dbad40a2
  1. cuj/
  2. docs/
  3. synthetic/
  4. third_party/
  5. tools/
  6. .gitignore
  7. .gitmodules
  8. .prettierignore
  9. .prettierrc.yml
  10. .pylintrc
  11. .style.yapf
  12. .vpython3
  13. DEPS
  14. eslint.config.js
  15. LICENSE
  16. mypy.ini
  17. OWNERS
  18. PRESUBMIT.py
  19. README.md
README.md

web-tests

web-tests contains:

  • Definitions for CUJs implemented using crossbench's loading benchmark
  • Configuration files for running benchmarks that are built in to crossbench (such as speedometer)
  • Metric definitions and queries for CUJs and benchmarks.

Setup

Do not git clone web-tests. Use the fetch command included with depot_tools.

mkdir src cd src gcloud auth login fetch web-tests cd web-tests

Don't forget to run gclient sync every time you pull new changes from origin.

Poetry

web-tests uses poetry to manage python dependencies.

sudo apt install python3.11 python3-poetry

Alternatively, install poetry in a python venv:

python3 -m venv web-tests-venv source web-tests-venv/bin/activate pip install poetry

Check that you have poetry on your path and make sure you have the right $PATH settings.

poetry --help || echo "Please update your \$PATH to include poetry bin location"; # Depending on your setup, add one of the following to your $PATH: echo "`python3 -m site --user-base`/bin"; python3 -c "import sysconfig; print(sysconfig.get_path('scripts'))";

Install the necessary dependencies from the lock file using poetry:

cd cuj/crossbench/runner export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring poetry env use 3.11 poetry install

Setting PYTHON_KEYRING_BACKEND to keyring.backends.null.Keyring disables keyring and prevents poetry install from getting stuck waiting for user input in the GUI.

Running Tests

Android

Before running a test against an android target, make sure your device is connected through adb:

adb devices > List of devices attached > 192.168.20.194:5555 device

Replace <DEVICE ID> below with the actual device ID from adb devices:

cd cuj/crossbench/runner poetry run python run.py --platform adb --device <DEVICE ID>

ChromeOS

Before running a test against a ChromeOS target, make sure passwordless SSH is available to the device. Either an IP address or a SSH host is supported as the device id.

Replace <DEVICE> below with the IP address or hostname of your device:

cd cuj/crossbench/runner poetry run python run.py --platform cros --device <DEVICE>

Local

Running against a local browser on linux is minimally supported, but may require manual changes to test configuration.

cd cuj/crossbench/runner poetry run python run.py --platform local

Specifying Tests and Variants

The minimal invocation of the runner will attempt to run all benchmarks, CUJs, and corresponding variants in series.

To run a subset of tests, use the --tests flag. --tests supports Python regex format for matching the test names.

poetry run python run.py --platform adb --device <DEVICE ID> --tests speedometer.*

To specify only certain variants of a test, you can use the --variants flag. --variants also supports Python regex format for matching variants.

poetry run python run.py --platform adb --device <DEVICE ID> --tests local-conference --variants 16p

Specifying Browsers

By default the runner will use ‘Chrome’ if the --browser flag is not specified. The format accepted by the --browser flag depends on the platform. For Android, use the package name of the installed browser. For ChromeOS or local, use the path to the browser executable.

Secrets

Some CUJs require secrets to perform privileged actions (such as a test account username/password, or auth tokens for the Google Meet Bond API). Place your secrets in secrets.hjson and pass the file to the runner:

poetry run python run.py --platform adb --device <DEVICE ID> --secrets /home/me/secrets.hjson --tests docs

Looping Tests

Tests can be repeated for a number of iterations or for a specified amount of time using the --playback flag. This flag is supported by crossbench and will iterate the post-setup sections of a CUJ and collect metrics for the entire invocation (instead of splitting metrics by iteration).

poetry run python run.py --platform adb --device <DEVICE ID> --tests tab-stress --variants blank-tab --playback 50x 
poetry run python run.py --platform adb --device <DEVICE ID> --tests tab-stress --variants blank-tab --playback 2h

Contributing

To get started contributing to web-tests, refer to the docs.