Skip to content

traceback.extract_stack raises exception if source filename is pyc and exists on disk #92336

Closed
@huwcbjones

Description

@huwcbjones

Bug report
If you compile a python file with the cfile as a pyc and it exists on disk, then if you extract the stack when this module is in the stack, linecache will try to load the file as utf-8 and explode.

Simple proof of concept below:

import py_compile import sys import traceback from tempfile import NamedTemporaryFile with NamedTemporaryFile(suffix=".py") as tmp_f: tmp_f.write(b"""\ def call(func):  return func() """) tmp_f.flush() py_compile.compile(tmp_f.name, dfile="/tmp/crash.pyc", cfile="/tmp/crash.pyc") sys.path.insert(0, "/tmp") import crash crash.call(traceback.print_stack)

results in a traceback of

Traceback (most recent call last): File "/home/huw/poc.py", line 16, in crash.call(traceback.print_stack) File "/tmp/crash.pyc", line 2, in call File "/usr/lib/python3.9/traceback.py", line 190, in print_stack print_list(extract_stack(f, limit=limit), file=file) File "/usr/lib/python3.9/traceback.py", line 211, in extract_stack stack = StackSummary.extract(walk_stack(f), limit=limit) File "/usr/lib/python3.9/traceback.py", line 366, in extract f.line File "/usr/lib/python3.9/traceback.py", line 288, in line self._line = linecache.getline(self.filename, self.lineno).strip() File "/usr/lib/python3.9/linecache.py", line 30, in getline lines = getlines(filename, module_globals) File "/usr/lib/python3.9/linecache.py", line 46, in getlines return updatecache(filename, module_globals) File "/usr/lib/python3.9/linecache.py", line 137, in updatecache lines = fp.readlines() File "/usr/lib/python3.9/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd3 in position 9: invalid continuation byte 
$ hd /tmp/crash.pyc 00000000 61 0d 0d 0a 00 00 00 00 3e d3 73 62 22 00 00 00 |a.......>.sb"...| ^ position 9 

Your environment
macOS 11.6.1 (x64): 3.7.4, 3.8.1, 3.9.1, 3.10.0
Debian Buster (x64): 3.7.3, 3.9.2
Debian Bullseye (x64): 3.9.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions