Pytest trying to check if custom argument is a file crashes due to filename being too long #10169
Description
I have a custom flag defined in conftest.py, and when I try to assign it to a value that is too long pytest crashes before ever getting to my code. This reproduces even if the flag isn't defined, and even if the current working directory is /.
Failing example:
/> pytest --xxxxx_flags=" --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx" Traceback (most recent call last): File "/home/ANT.AMAZON.COM/jdckmz/.local/bin/pytest", line 8, in <module> sys.exit(console_main()) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 188, in console_main code = main() File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 146, in main config = _prepareconfig(args, plugins) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 325, in _prepareconfig config = pluginmanager.hook.pytest_cmdline_parse( File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 55, in _multicall gen.send(outcome) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/helpconfig.py", line 102, in pytest_cmdline_parse config: Config = outcome.get_result() File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result raise ex[1].with_traceback(ex[2]) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall res = hook_impl.function(*args) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1013, in pytest_cmdline_parse self.parse(args) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1301, in parse self._preparse(args, addopts=addopts) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1203, in _preparse self.hook.pytest_load_initial_conftests( File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__ return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall return outcome.get_result() File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result raise ex[1].with_traceback(ex[2]) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall res = hook_impl.function(*args) File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1080, in pytest_load_initial_conftests self.pluginmanager._set_initial_conftests( File "/home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 525, in _set_initial_conftests if anchor.exists(): # we found some file object File "/usr/lib/python3.8/pathlib.py", line 1407, in exists self.stat() File "/usr/lib/python3.8/pathlib.py", line 1198, in stat return self._accessor.stat(self) OSError: [Errno 36] File name too long: '/--xxxxx_flags= --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx'If I reduce the length of the flag, I get the expected behavior for my project, and this different and expected error from my pytest MVP:
/> pytest --xxxxx_flags=" --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxx" =========================================================================== test session starts ============================================================================ platform linux -- Python 3.8.10, pytest-7.0.0, pluggy-1.0.0 rootdir: / plugins: flaky-3.7.0, colcon-core-0.10.0, cov-2.8.1 collected 0 items ============================================================================= warnings summary ============================================================================= home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/cacheprovider.py:433 /home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: could not create cache path /.pytest_cache/v/cache/nodeids config.cache.set("cache/nodeids", sorted(self.cached_nodeids)) home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/stepwise.py:52 /home/ANT.AMAZON.COM/jdckmz/.local/lib/python3.8/site-packages/_pytest/stepwise.py:52: PytestCacheWarning: could not create cache path /.pytest_cache/v/cache/stepwise session.config.cache.set(STEPWISE_CACHE_DIR, []) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================================================================== 2 warnings in 0.01s ============================================================================ ERROR: file or directory not found: --xxxxx_flags= --xxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxx --xxxxxxxxxxxxxxxxxxxxxxI did a little digging into my version of pytest (7.0.0) to make sure I wasn't doing something wrong, but it looks like there is a blind call to pathlib.Path.exists() with a path constructed from the argument in __init__.py:
# # Internal API for local conftest plugin handling. # def _set_initial_conftests( self, namespace: argparse.Namespace, rootpath: Path ) -> None: ... testpaths = namespace.file_or_dir foundanchor = False for testpath in testpaths: path = str(testpath) i = path.find("::") if i != -1: path = path[:i] anchor = absolutepath(current / path) if anchor.exists(): # this throws OSError which is never caughtIt seems to me like there should be a try or something here, since in cases like mine the argument may not be a file at all, and that can cause OS level errors.
Operating System: Ubuntu 20.04 LTS
> pytest --version pytest 7.0.0 > python3 --version Python 3.8.10 > pip list /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead from cryptography.utils import int_from_bytes Package Version ----------------------------- -------------------- aiohttp 3.8.1 aiosignal 1.2.0 alabaster 0.7.12 apturl 0.5.2 argcomplete 1.8.1 astroid 2.9.3 async-timeout 4.0.2 atomicwrites 1.4.0 attrs 21.4.0 autobahn 17.10.1 Automat 0.8.0 aws-requests-auth 0.4.3 awscli 1.22.52 awscrt 0.13.0 awsiotsdk 1.9.0 Babel 2.9.1 bcrypt 3.2.0 beautifulsoup4 4.8.2 black 22.1.0 blinker 1.4 boto3 1.20.52 botocore 1.23.52 Brlapi 0.7.0 cached-property 1.5.1 catkin-pkg-modules 0.5.2 cbor 1.0.0 certifi 2021.10.8 cffi 1.15.0 chardet 4.0.0 charset-normalizer 2.0.11 click 8.0.3 cmakelang 0.6.13 cmakelint 1.4.2 colcon-argcomplete 0.3.3 colcon-bash 0.4.2 colcon-cd 0.1.1 colcon-cmake 0.2.26 colcon-common-extensions 0.3.0 colcon-core 0.10.0 colcon-defaults 0.2.6 colcon-devtools 0.2.3 colcon-library-path 0.2.1 colcon-metadata 0.2.5 colcon-notification 0.2.13 colcon-output 0.2.12 colcon-package-information 0.3.3 colcon-package-selection 0.2.10 colcon-parallel-executor 0.2.4 colcon-pkg-config 0.1.0 colcon-powershell 0.3.7 colcon-python-setup-py 0.2.7 colcon-recursive-crawl 0.2.1 colcon-ros 0.3.23 colcon-test-result 0.3.8 colcon-zsh 0.4.0 colorama 0.4.3 command-not-found 0.3 constantly 15.1.0 control 0.9.1 cov-core 1.15.0 coverage 4.5.2 cryptography 36.0.1 cupshelpers 1.0 cycler 0.11.0 Cython 0.29.14 dbus-python 1.2.16 defer 1.0.6 distlib 0.3.4 distro 1.4.0 distro-info 0.23ubuntu1 docker 5.0.3 docker-compose 1.25.0 dockerpty 0.4.1 docopt 0.6.2 docutils 0.15.2 duplicity 0.8.12.0 EasyCluster 0.22.2 empy 3.3.2 entrypoints 0.3 evdev 1.3.0 fasteners 0.14.1 filelock 3.7.1 filemagic 1.6 flake8 3.7.9 flaky 3.7.0 fonttools 4.29.1 frozenlist 1.3.0 future 0.18.2 gitdb 4.0.9 gitdb2 4.0.2 github.py 0.5.0 GitPython 3.1.26 gpg 1.13.1-unknown greenlet 1.1.2 html5lib 1.0.1 httplib2 0.14.0 hyperlink 19.0.0 idna 3.3 ifcfg 0.18 imagesize 1.3.0 importlib-metadata 4.10.1 incremental 16.10.1 influxdb 5.3.1 iniconfig 1.1.1 isort 5.10.1 Jinja2 3.0.3 jmespath 0.10.0 jsonschema 3.2.0 keyring 18.0.1 keyrings.alt 3.4.0 kiwisolver 1.3.2 language-selector 0.1 lark-parser 0.8.1 launchpadlib 1.10.13 lazr.restfulclient 0.14.2 lazr.uri 1.0.3 lazy-object-proxy 1.7.1 lockfile 0.12.2 louis 3.12.0 lxml 4.5.0 lz4 3.0.2+dfsg macaroonbakery 1.3.1 Mako 1.1.0 MarkupSafe 2.0.1 matplotlib 3.5.1 mccabe 0.6.1 mock 3.0.5 monotonic 1.5 more-itertools 8.12.0 mpi4py 3.0.3 msgpack 1.0.3 multi-key-dict 2.0.3 multidict 6.0.2 mypy-extensions 0.4.3 netifaces 0.10.4 nose2 0.9.1 notify2 0.3 numpy 1.22.2 oauthlib 3.1.0 olefile 0.46 packaging 21.3 pandas 1.4.0 paramiko 2.9.2 pathspec 0.9.0 pbr 5.8.1 pexpect 4.8.0 Pillow 9.0.1 pip 22.1.2 pipenv 2022.6.7 platformdirs 2.5.0 pluggy 1.0.0 protobuf 3.19.4 psutil 5.8.0 ptyprocess 0.7.0 py 1.11.0 py-ubjson 0.14.0 pyasn1 0.4.8 pyasn1-modules 0.2.1 pybind11 2.8.0 pycairo 1.16.2 pycodestyle 2.8.0 pycparser 2.21 pycrypto 2.6.1 pycups 1.9.73 pydocstyle 2.1.1 pydot 1.4.1 pyelftools 0.28 pyflakes 2.1.1 Pygments 2.11.2 PyGObject 3.36.0 PyHamcrest 1.9.0 PyJWT 1.7.1 pylint 2.12.2 pymacaroons 0.13.0 PyNaCl 1.5.0 pyOpenSSL 19.0.0 pyparsing 3.0.7 pypng 0.0.20 PyQRCode 1.2.1 PyQt5 5.14.1 pyquaternion 0.9.9 pyRFC3339 1.1 pyrsistent 0.15.5 pyserial 3.5 pytest 7.0.0 pytest-cov 2.8.1 python-apt 2.0.0+ubuntu0.20.4.7 python-dateutil 2.8.2 python-debian 0.1.36ubuntu1 python-dotenv 0.19.2 python-jenkins 1.7.0 python-magic 0.4.16 python-snappy 0.5.3 PyTrie 0.2 pytz 2021.3 pyxdg 0.26 PyYAML 5.3.1 reportlab 3.5.34 requests 2.27.1 requests-unixsocket 0.2.0 roman 2.0.0 rosdistro-modules 0.9.0 rospkg-modules 1.4.0 rplidar 0.9.2 rsa 4.7.2 s3transfer 0.5.1 scipy 1.8.0 screen-resolution-extra 0.0.0 SecretStorage 2.3.1 service-identity 18.1.0 setproctitle 1.1.10 setuptools 45.2.0 simplejson 3.16.0 sip 4.19.21 six 1.16.0 smmap 5.0.0 smmap2 3.0.1 snowballstemmer 2.2.0 soupsieve 1.9.5 Sphinx 4.4.0 sphinx-autoapi 1.8.4 sphinxcontrib-applehelp 1.0.2 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-dotnetdomain 0.4 sphinxcontrib-golangdomain 0.2.0.dev0 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 sphinxcontrib-websupport 1.2.4 SQLAlchemy 1.4.35 ssh-import-id 5.10 tensorrt 8.0.1.6 texttable 1.6.2 toml 0.10.2 tomli 2.0.1 tripy 1.0.0 Twisted 18.9.0 txaio 2.10.0 typed-ast 1.5.2 typing_extensions 4.0.1 u-msgpack-python 2.1 ubuntu-advantage-tools 27.9 ubuntu-drivers-common 0.0.0 ufw 0.36 unattended-upgrades 0.1 Unidecode 1.3.2 urllib3 1.26.8 usb-creator 0.3.7 virtualenv 20.14.1 virtualenv-clone 0.5.7 wadllib 1.3.3 wcwidth 0.1.8 webencodings 0.5.1 websocket-client 1.2.3 wheel 0.34.2 wrapt 1.13.3 wsaccel 0.6.2 xdot 1.1 xkit 0.0.0 xmltodict 0.12.0 yarl 1.7.2 zipp 3.7.0 zope.interface 4.7.1 zstandard 0.17.0 - a detailed description of the bug or problem you are having
- output of
pip listfrom the virtual environment you are using - pytest and operating system versions
- minimal example if possible