We've had a couple of related issue, but I couldn't find this specific case reported before.

With the current git main:

platform linux -- Python 3.12.3, pytest-8.2.0.dev99+g1d322ffb4, pluggy-1.5.0 

doing something like:

def test_x(): assert False, dir("")

results in:

 def test_x(): > assert False, dir("") E AssertionError: ['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', ...] E assert False

and even with -vv, the output never seems to get more verbose.