Skip to content

Make multiprocessing import local to support pyodide #7519

@twiecki

Description

@twiecki

Description

We almost have PyMC working natively under pyodide. But upon import we get an error importing multiprocessing which isn't included in pyodide:

--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) Cell In[5], line 1 ----> 1 import pymc as pm 2 pm.__version__ File /lib/python3.12/site-packages/pymc/__init__.py:72 70 from pymc.pytensorf import * 71 from pymc.sampling import * ---> 72 from pymc.smc import * 73 from pymc.stats import * 74 from pymc.step_methods import * File /lib/python3.12/site-packages/pymc/smc/__init__.py:16 1 # Copyright 2024 The PyMC Developers 2 # 3 # Licensed under the Apache License, Version 2.0 (the "License"); (...) 12 # See the License for the specific language governing permissions and 13 # limitations under the License. 15 from pymc.smc.kernels import IMH, MH ---> 16 from pymc.smc.sampling import sample_smc 18 __all__ = ("sample_smc",) File /lib/python3.12/site-packages/pymc/smc/sampling.py:21 18 import warnings 20 from collections import defaultdict ---> 21 from concurrent.futures import ProcessPoolExecutor, wait 22 from typing import Any 24 import cloudpickle File /lib/python312.zip/concurrent/futures/__init__.py:44, in __getattr__(name) 41 global ProcessPoolExecutor, ThreadPoolExecutor 43 if name == 'ProcessPoolExecutor': ---> 44 from .process import ProcessPoolExecutor as pe 45 ProcessPoolExecutor = pe 46 return pe File /lib/python312.zip/concurrent/futures/process.py:55 52 # This import is required to load the multiprocessing.connection submodule 53 # so that it can be accessed later as `mp.connection` 54 import multiprocessing.connection ---> 55 from multiprocessing.queues import Queue 56 import threading 57 import weakref File /lib/python312.zip/multiprocessing/queues.py:23 19 import errno 21 from queue import Empty, Full ---> 23 import _multiprocessing 25 from . import connection 26 from . import context ModuleNotFoundError: No module named '_multiprocessing' 

If we made that optional, it will work out of the box.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions