basic support for coreference #68
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
- require Python 3.6+ due to f-strings - Travis test with Python 3.6-3.9 - expect pip older than 2016, which knows python_requires
- If no head is specified, the first word from mention_words will be used instead. - If mention_words are provided, they must contain the head. - docstring
and store it in CorefCluster, not CorefMention
- EmptyNode is a subclass of Node - root is stored in empty nodes, not computed (because empty nodes have parent=None and root may be unreachable) - node.create_empty_child(deprel='...') deprel is now required - it is the enhanced UD deprel to be stored in DEPS - argument `after` specifies the position (ord) of the newly created empty node - root.create_empty_child() is a faster version, which does not set `deps` and `ord`.
In Python, empty attributes should be None (it is more memory efficient than '_'), the underscore is just a matter of CoNLL-U serialization.
so the setter can be overriden in EmptyNode
Benchmark "NewTreex" shows 0.5% more memory, but total time 6% faster (rehanging, i.e. changing a parent is 21% faster, next_node 65% faster): experiment|TIME |MAXMEM |load |save |iterN|rehang|remove|add |reorder| ----------|-----:|------:|-----:|----:|----:|-----:|-----:|----:|------:| udapi |40.507|832.815|17.066|4.521|0.715|3.502 |2.348 |2.868|4.174 | udapi_new |37.958|837.302|16.720|3.846|0.251|2.766 |2.284 |2.699|4.106 |
reading+writing support for multiple clusters in one node stored using layered attributes
…dTuple root.create_empty_child() should not sort the empty nodes because they may not have ord filled. In contrast, node.create_empty_child() should sort the empty nodes.
* l.append(x) is faster than l += [x] * l.sort(); l2=l; is faster than l2 = sorted(l)
The main trick is to prevent creating new lists (and memory allocations) wherever possible. E.g. `node.children` creates a new `ListOfNodes` object with a copy of the list of children and `node.children(add_self=True)` creates one more copy (and the previous copy is thrown away for gc). Thus internally, we can call node._children (which currently does not guarantee sorted result), which creates no extra list. `node.children(add_self=True)` was changed so it creates just a single new list. Further speedup is possible in future. Very minor speedup is due to direct usage of attributes instead of overloaded properties, e.g. node._ord instead of node.ord. This is not worth the effort in user blocks, but internally in the core API it makes a (small) difference in total.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
No description provided.