This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author pierreglaser
Recipients pablogsal, pierreglaser, pitrou
Date 2019-05-11.15:58:49
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1557590329.54.0.795308311699.issue36867@roundup.psfhosted.org>
In-reply-to
Content
Shared memory segments are now tracked by the brand new resource_tracker! Thanks Antoine for the review. Does anyone have an opinion on introducing a public API for users to make the resource_tracker track resources of their choice? What We have in mind is: - making public the existing resource_tracker.register/unregister - adding a new function, resource_tracker.make_trackable(resource_type, cleanup_func), where: * resource_type is a string (an identifier that will be used each time a resource of the type resource_type needs tracking, via the call resource_tracker.register(resource_name, resource_type) * cleanup_func must be a callable taking a single string as argument (the name of the resource that needs tracking). This function will be called after the end of a process for reach resource of type resource_type the process did not clean properly Under the hood, make_trackable simply populates resource_tracker._CLEANUP_FUNCS with new items. Here is a simple example:	import os	import resource_tracker	import shutil	from multiprocessing import util	class ClassCreatingAFolder: """Class where each instance creates a temporary folder. Each temporary folder is supposed to exist for the duration of the instance that created it. """ def __init__(self, folder_name):	self.folder_name = folder_name	os.mkdir(folder_name)	# any instance normally garbage-collected should remove its folder, and	# notice the resource_tracker that its folder was correctly removed.	util.Finalize(self, ClassCreatingAFolder.cleanup, args=(folder_name,))	# If this session quits abruptly, the finalizer will not be called for	# the instances of ClassCreatingAFolder that were still alive	# before the shutdown. The resource_tracker comes into play, and removes	# the folders associated to each of these resources.	resource_tracker.register( folder_name, # argument to shutil.rmtree "ClassCreatingAFolder") @staticmethod def cleanup(folder_name):	resource_tracker.unregister(folder_name, "ClassCreatingAFolder")	shutil.rmtree(folder_name)	# Tell the resource_tracker how to cleanup resources created by	# ClassCreatingAFolder instances	resource_tracker.make_trackable("ClassCreatingAFolder", shutil.rmtree) Typical resources that can be made trackable include memmaped objects, temporary folders. Our use-case is joblib that has its own mmap type that we would like to track using the semaphore_tracker. Any thoughts?
History
Date User Action Args
2019-05-11 15:58:49pierreglasersetrecipients: + pierreglaser, pitrou, pablogsal
2019-05-11 15:58:49pierreglasersetmessageid: <1557590329.54.0.795308311699.issue36867@roundup.psfhosted.org>
2019-05-11 15:58:49pierreglaserlinkissue36867 messages
2019-05-11 15:58:49pierreglasercreate