It is better if the function annotation(PEP 3107) can be supported by built-in function and C function writtin in extension module, just like the __doc__ attribute.

Extension modules can use PyFunction_GetAnnotations() to access and modify the annotations dictionary. In addition, PyFunction_SetAnnotations() can be used to add annotations. I added some documentation for these functions in r64934.

Sorry I haven't state the issue clearly. For this issue I mean the built-in function should able to define an __annotations__ attribute, just like the __doc__ attribute, but not to access it in extension module.

PyCFunctionObject has indeed no way to store annotations. This could be useful for extension module writers. The PyMethodDef structure could grow a "ml_annotations" member. A patch is welcome!

By considering the implementing, some problems emerged. First of all, as we know, all CFunctionObject and their attributes are imutable, but the __annotations__ attribute should be a dict, and dict is mutable. So how to solve this? Secondly, the annotation value can be abitrary expression, and then, for extension module, would it be reasonable to restrict these value to string? Thanks!

- A immmutable object may contain mutable members. Try with a tuple containing a list. Then, I don't think that something says that CFunctionObjects are immutable. They don't have any modifiable attribute, until today! - (Did I say "string"?) The new PyMethodDef::ml_annotations would not be a char*, but a PyObject* member. If it is not possible to set it in the static array, one could update the array in the module init function. Anyway, for a SWIG module I think the best is to set the __annotations__ in the shadow python file. It seems more practical to build the dict there.

I think there is reason that CFunctionObjects are immutable: single CFunctionObject is shared by mutiple Python interpreters, so any change of CFunctionObject would affect other Python interpreters. Is that right? If it should be immutable, then we should use something like static array to assign annotations to CFunctionObject, and the value also should be immutable, that means the value couldn't be abitrary PyObject. (by value I mean the value of every __annotations__ dict items.) For SWIG, there's a way to bypass the Python side proxy, eg. for a simple C function, in the shadow module we directly let 'func=_cmod.func', where _cmod is the C DLL module. So the annotation information would be lost if we can't directly assign annotation to C function.

There never should be multiple Python interpreters running in the same process, though.

As I understand, at least C extension modules, which built as shared library, would be shared among Python interpreter in different process space. Is that correct?

On Wed, Jul 23, 2008 at 8:44 PM, Haoyu Bai <report@bugs.python.org> wrote: > > Haoyu Bai <divinekid@gmail.com> added the comment: > > As I understand, at least C extension modules, which built as shared > library, would be shared among Python interpreter in different process > space. Is that correct? The operating system should provide memory protection between processes. > > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue3208> > _______________________________________ >

Shared libraries share code, not memory. But were you talking about sub-interpreters? http://docs.python.org/dev/c-api/init.html#Py_NewInterpreter mod_python uses them, but see the "Caveats" section of the doc.

I found the explanation of why buitl-ins are immutable: For the curious: there are two reasons why changing built-in classes is disallowed. First, it would be too easy to break an invariant of a built-in type that is relied upon elsewhere, either by the standard library, or by the run-time code. Second, when Python is embedded in another application that creates multiple Python interpreters, the built-in class objects (being statically allocated data structures) are shared between all interpreters; thus, code running in one interpreter might wreak havoc on another interpreter, which is a no-no. (From http://www.python.org/download/releases/2.2.3/descrintro/) Is the statement still valid for current version of Python?

The "First" argument does not apply here, we could just say "annotations are not a function invariant", but the "Second" argument is valid to me. A solution would be a global (or interpreter-local if we really want to support sub-interpreters) registry that stores annotations. The index could not be the PyCFunctionObject (since it is different for every bound method), but the address of the PyMethodDef entry.

I don't think it's reasonable not to support multiple interpreters in a single process - they're quite widely used by mod_python and mod_wsgi, and probably by others. I'm not sure whether that's a problem here or not, though. If we need to allow function annotations to be arbitrary PyObjects, these PyObject pointers can't (in general) refer to statically allocated python objects, so some extension modules will have to allocate them in the module initialisation function (and presumably deallocate them again when the module is unloaded). I would have thought that any such PyObjects are going to be valid only from within a single interpreter. Perhaps I'm wrong. Certainly it would be unpleasant if a change to one of the objects in one interpreter was reflected in other interpreters, but if that didn't risk causing a crash due to the memory allocation going wrong, or something equally nasty, it might be acceptable.

What is the status of this bug?

Awaiting a patch.

This issue has been addressed by the PEP 436 (Argument Clinic) which supports annotation per parameter and annotation on the return type. This PEP has been implemented in Python 3.4. I suggest to close the issue, but I would prefer that Larry closes the issue instead of me, he wrote the PEP.

Argument Clinic theoretically could support annotations for builtins, though it's never been tested. I don't know if it makes sense to close this bug yet.