Open
Description
Bug report
Bug description:
dbm backends were previously thread safe, but I think dbm.sqlite introduced in #114481 is not.
I am not sure if the resolution should be to add a doc comment to shelve/dbm or some other way to fix it, say specifying a preferred backend or multithreading argument in shelve.open as an argument. (Or if there is a way to fix this in dbm.sqlite itself)
Example code:
from concurrent.futures import ThreadPoolExecutor import shelve CACHE = shelve.open('test') def check_set_cache(value): if 'value' in CACHE: print(CACHE['value']) CACHE['value'] = value return CACHE['value'] jobs = list(range(1, 100)) executor = ThreadPoolExecutor(max_workers=5) entries = list(executor.map(check_set_cache, jobs)) print(entries)Log
Traceback (most recent call last): File "/usr/lib64/python3.13/dbm/sqlite3.py", line 79, in _execute return closing(self._cx.execute(*args, **kwargs)) ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140036133836608 and this is thread id 140035884230336. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/megh/pybug/reproduce.py", line 14, in <module> entries = list(executor.map(check_set_cache, jobs)) File "/usr/lib64/python3.13/concurrent/futures/_base.py", line 619, in result_iterator yield _result_or_cancel(fs.pop()) ~~~~~~~~~~~~~~~~~^^^^^^^^^^ File "/usr/lib64/python3.13/concurrent/futures/_base.py", line 317, in _result_or_cancel return fut.result(timeout) ~~~~~~~~~~^^^^^^^^^ File "/usr/lib64/python3.13/concurrent/futures/_base.py", line 449, in result return self.__get_result() ~~~~~~~~~~~~~~~~~^^ File "/usr/lib64/python3.13/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/usr/lib64/python3.13/concurrent/futures/thread.py", line 59, in run result = self.fn(*self.args, **self.kwargs) File "/home/megh/pybug/reproduce.py", line 7, in check_set_cache if 'value' in CACHE: ^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.13/shelve.py", line 102, in __contains__ return key.encode(self.keyencoding) in self.dict ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen _collections_abc>", line 817, in __contains__ File "/usr/lib64/python3.13/dbm/sqlite3.py", line 89, in __getitem__ with self._execute(LOOKUP_KEY, (key,)) as cu: ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.13/dbm/sqlite3.py", line 81, in _execute raise error(str(exc)) dbm.sqlite3.error: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140036133836608 and this is thread id 140035884230336. (Observed here beancount/beanprice#91 )
CPython versions tested on:
3.13
Operating systems tested on:
Linux
Linked PRs
Metadata
Metadata
Assignees
Projects
Status
No status