gh-125669: Do not run -ugui tests by default on make test

[sobolevn]

Basically this PR changes how --fast-ci is defined and also disables gui tests with it. Because make tests spawns a lot of gui windows locally and distrurb the development workflow.

Most likely this won't affect any buildbots (if they correctly use --slow-ci). But, correct me if I am wrong.

Regular make test run now:

» make test + ./python.exe -u -W default -bb -E -m test --fast-ci --timeout= --dont-add-python-opts == resources: all,-cpu,-gui 

With TESTOPTS="-u gui" (or EXTRATESTOPS) passed:

» make test TESTOPTS="-u gui" + ./python.exe -u -W default -bb -E -m test --fast-ci --timeout= -u gui --dont-add-python-opts == resources: all,-cpu 

• Issue: Some weird make test behavior #125669

[vstinner]

Most likely this won't affect any buildbots (if they correctly use --slow-ci). But, correct me if I am wrong.

Unix buildbots run make buildbottest which uses --slow-ci. Windows buildbots run Tools\buildbot\test.bat which also uses --slow-ci.

Basically this PR changes how --fast-ci is defined and also disables gui tests with it. Because make tests spawns a lot of gui windows locally and distrurb the development workflow.

Why not only changing the make test command rather than changing the --fast-ci option for everybody? You should be able to pass -u -gui to make test.

[vstinner]

Something like:

diff --git a/Makefile.pre.in b/Makefile.pre.in index fb6f22d5739..20621257575 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -2097,7 +2097,7 @@ cleantest: all # Similar to buildbottest, but use --fast-ci option, instead of --slow-ci. .PHONY: test test: all - $(TESTRUNNER) --fast-ci --timeout=$(TESTTIMEOUT) $(TESTOPTS) + $(TESTRUNNER) --fast-ci -u-gui --timeout=$(TESTTIMEOUT) $(TESTOPTS) # Run the test suite for both architectures in a Universal build on OSX. # Must be run on an Intel box.

[vstinner]

See also the doc: https://docs.python.org/dev/using/configure.html#make-test

[vstinner]

Please add a NEWS entry in the Tests category.

[erlend-aasland]

CI master @hugovk, are you ok with this PR?

[vstinner]

Buildbots run Tools/buildbot/test.bat which runs PCbuild/rt.bat. I'm not sure that this change is correct. Maybe it will disable GUI tests on buildbots.

[sobolevn]

Yes, thanks for this question. I was not also sure about that. Since I don't ever use windows and not quite sure about how it should work. I will revert this.

[vstinner]

LGTM. Thanks for the different updates.

[hugovk]

CI master @hugovk, are you ok with this PR?

Looks good, thanks all! 👍

[sobolevn]

Thanks everyone! 👏

[warsaw]

Thanks for fixing this! One ask:

I don't think that we should backport this change

Please reconsider. It's painful for the 3.13 branch and changing it would be a net win.