Skip to content

Conversation

rundiwu
Copy link
Member

@rundiwu rundiwu commented Jul 8, 2019

Checklist

  • I've tested that my changes are compatible with the latest version of Tensorflow.
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.

Motivation and Context

  1. Support nested layer customization.
  2. Enable better 'in_channels' exception raise in dynamic model definition.

Description

I briefly list the design below just for a reminder.

  • Nested layer customization
    • Logic
      Each layer maintain a private attribute self._layers to store possible nested layer. When a Layer is registered as an attribute of another Layer, it is automatically added to the second layer's self._layers. Weights are collected recursively.
      Inner layer set to be unaware of LayerNode concept because it does not need to be involved in the model node graph. Inner layer is only regarded as the outer layer‘s 'private' attribute, which is not known by the model.
    • Example
       class MyLayer(tl.layers.Layer): def __init__(self, name=None): super(MyLayer, self).__init__(name=name) self.input_layer = tl.layers.Dense(n_units=20) # it's ok not to pass `in_channels` here self.build(None) self._built = True def build(self, inputs_shape=None): self.W = self._get_weights('weights', shape=(20, 10)) def forward(self, inputs): inputs = self.input_layer(inputs) output = tf.matmul(inputs, self.W) return output
      MyLayer would contain three weights: W of MyLayer, W and b of the inner Dense.
  • 'in_channels' exception in dynamic model
    Now it is checked when setting Layer as an attribute of dynamic Model. If in_channels not provided, there will be an exception raised, e.g.
    The registered layer `dense_4` should be built in advance. Do you forget to pass the keyword argument 'in_channels'? 
@rundiwu rundiwu requested review from JingqingZ and zsdonghao July 8, 2019 00:28
@rundiwu rundiwu merged commit 1477d57 into tensorlayer:master Jul 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants