- Notifications
You must be signed in to change notification settings - Fork 477
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
(windows10 / python3.6 / conda:latest / all-other-packages:latest)
There is a memory leak somewhere in the multiprocessing section:
| with multiprocessing.Pool(n_worker) as pool: |
Running 8 workers on a system with 16GB of ram will lead to a crash within ~1-2mins and a nasty pagefile increase to 40GB! I re-wrote the script using the older Process() api and no shared variables between instances which fixed the issue so its probably something to do with data passing.
All of my tests were done on the full celebahq dataset.
Janspiry
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request