Background

@seiriosPlus find a bug in the new Trainer high-level API when writing a model with learning rate decay. The optimizer is created in a different program than the trainer use.

Reason and solution

In the design of Trainer API, train_program is a function call, it will be called inside Trainer under a with scope

But the init parameter optimizer is an object, it will be created outside the with scope, this is not right, so we should make optimizer also a function call that return an optimizer boject

the interface should be

class Trainer(object): def __init__(self, train_func, optimizer_func, param_path=None, place=None, parallel=False):