PaddlePaddle simplifies the way it parses the network configuration by introducing some global variables in this PR Fix V2 API #2288.

optimizer is used to set the default values for some parameters, including L2 regularization, gradient_clipping_threshold and so on. These settings are stored in some global variables, so before parsing the network topology, these global variables should be correctly initialized.

This means the definition of optimizer must be called before the definition of network topology to enable the global parameter settings.

The following codes are wrong because gradient_clipping_threshold and L2 regularization will not work.

cost = seq2seq_net(source_dict_dim, target_dict_dim) parameters = paddle.parameters.create(cost) optimizer = paddle.optimizer.RMSProp( learning_rate=1e-3, gradient_clipping_threshold=10.0, regularization=paddle.optimizer.L2Regularization(rate=8e-4))

This will be right:

optimizer = paddle.optimizer.RMSProp( learning_rate=1e-3, gradient_clipping_threshold=10.0, regularization=paddle.optimizer.L2Regularization(rate=8e-4)) cost = seq2seq_net(source_dict_dim, target_dict_dim) parameters = paddle.parameters.create(cost)

This is very dangerous for users because no error is reported.