As we discussed before, shared_ptr is preferred to avoid in our refactoring code for simplifying our design and making lifecycle management clearer.

The shared_ptr of OperatorBase should be removed. We design our operator lifecycle like below:

• all Python API will return an OperatorBase* or NetOp*. We let Python side to destroy operators.
• If we destroy a NetOp or RnnOp (which contains step_net), all operator belongs to that net will be destroyed, too.
• If two NetOp want to share same Operator, or NetOp add an Operator* but that Operator* is held by Python, too. We use Operator::Clone() to create a new Operator pointer. i.e. One operator instance is avoided to be shared by multiple NetOp.
  • Since our operator design is very lightweight(only variable names and attrs are stored), and topology is constructed before whole training started, Clone an operator will simplify our design without costing too much in time and space.

The whole things come to following steps:

1. Add Clone method to OperatorBase
2. Remove shared_ptr inside C++ and Python
3. Expose CreateOpAndAddItToSelf method for NetOp in Python(to avoid Clone every time).