Skip to content

Add probability distribution transformation APIs #40536

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Mar 31, 2022
Merged

Add probability distribution transformation APIs #40536

merged 8 commits into from
Mar 31, 2022

Conversation

@cxxly
Copy link
Contributor

@cxxly cxxly commented Mar 14, 2022 •
edited
Loading

PR types

New features

PR changes

APIs

Describe

Adds 13 transformation APIs and 2 distribution APIs :

new transformation APIs:

  1. Transform
  2. AbsTransform
  3. AffineTransform
  4. ChainTransform
  5. ExpTransform
  6. IndependentTransform
  7. PowerTransform
  8. ReshapeTransform
  9. SigmoidTransform
  10. SoftmaxTransform
  11. StackTransform
  12. StickBreakingTransform
  13. TanhTransform

new distribution APIs:

  1. Independent
  2. TransformedDistribution

Examples:

import paddle x = paddle.to_tensor([1., 2.]) affine = paddle.distribution.AffineTransform(paddle.to_tensor(0.), paddle.to_tensor(1.)) print(affine.forward(x)) # Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True, # [1., 2.]) print(affine.inverse(affine.forward(x))) # Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True, # [1., 2.]) print(affine.forward_log_det_jacobian(x)) # Tensor(shape=[1], dtype=float32, place=Place(gpu:0), stop_gradient=True, # [0.])
@paddle-bot-old
Copy link

paddle-bot-old bot commented Mar 14, 2022 •
edited
Loading

✅ This PR's description meets the template requirements!
Please wait for other CI results.
@paddle-bot-old
Copy link

paddle-bot-old bot commented Mar 14, 2022

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.
@cxxly cxxly force-pushed the distribution-transform branch from e7850b7 to 141c642 Compare March 28, 2022 10:25
@cxxly cxxly force-pushed the distribution-transform branch from 141c642 to ae14d10 Compare March 28, 2022 11:22
@cxxly cxxly force-pushed the distribution-transform branch from ae14d10 to 540de48 Compare March 28, 2022 12:38
@cxxly cxxly force-pushed the distribution-transform branch from 540de48 to b14dae1 Compare March 28, 2022 17:03
@cxxly cxxly force-pushed the distribution-transform branch from f37b54c to 9fcd9e8 Compare March 29, 2022 05:36
@cxxly cxxly force-pushed the distribution-transform branch from 9fcd9e8 to 983c1fb Compare March 29, 2022 06:36
@cxxly cxxly changed the title Add probability distribution transformation APIs for Paddle Add probability distribution transformation APIs Mar 29, 2022
@cxxly cxxly force-pushed the distribution-transform branch from 983c1fb to b96f2e1 Compare March 29, 2022 16:08
@cxxly cxxly force-pushed the distribution-transform branch from b96f2e1 to 3421235 Compare March 30, 2022 01:45
@cxxly cxxly force-pushed the distribution-transform branch from 3421235 to 6985966 Compare March 30, 2022 02:56
iclementine
iclementine reviewed Mar 30, 2022
View reviewed changes
python/paddle/distribution/transform.py Outdated
Copy link

@iclementine iclementine Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo
Copy link
Contributor Author

@cxxly cxxly Mar 30, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated
iclementine
iclementine previously approved these changes Mar 30, 2022
View reviewed changes
TCChenlong
TCChenlong previously approved these changes Mar 30, 2022
View reviewed changes
Copy link
Contributor

@TCChenlong TCChenlong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
TODO：Add Chinese documentation
jeff41404
jeff41404 reviewed Mar 30, 2022
View reviewed changes
python/paddle/distribution/transform.py Outdated
Copy link
Contributor

@jeff41404 jeff41404 Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"power" means paddle.distribution.PowerTransform？
Copy link
Contributor Author

@cxxly cxxly Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已更新，粘贴错误，粘贴了一个旧的代码示例
@jeff41404
Copy link
Contributor

jeff41404 commented Mar 30, 2022

Examples in Describe above: print(affine.inverse(power.forward(x))), "power" means paddle.distribution.PowerTransform? or should be affine?
@cxxly
Copy link
Contributor Author

cxxly commented Mar 30, 2022 

import paddle x = paddle.to_tensor([1., 2.]) affine = paddle.distribution.AffineTransform(paddle.to_tensor(0.), paddle.to_tensor(1.)) print(affine.forward(x)) # Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True, # [1., 2.]) print(affine.inverse(affine.forward(x))) # Tensor(shape=[2], dtype=float32, place=Place(gpu:0), stop_gradient=True, # [1., 2.]) print(affine.forward_log_det_jacobian(x)) # Tensor(shape=[1], dtype=float32, place=Place(gpu:0), stop_gradient=True, # [0.])

Examples in Describe above: print(affine.inverse(power.forward(x))), "power" means paddle.distribution.PowerTransform? or should be affine?

是 ''affine''，粘贴错误，文档和PR描述均已更新
jeff41404
jeff41404 reviewed Mar 30, 2022
View reviewed changes
python/paddle/distribution/distribution.py Outdated
Copy link
Contributor

@jeff41404 jeff41404 Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we add some abstract methods in base class, e.g. mean, variance and rsample
Copy link
Contributor Author

@cxxly cxxly Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已添加，这样能保持不同子类持有的方法一致；存量 normal, uniform, categorical目前是NotImplementedError，按照设计文档计划后续统一更新
@cxxly cxxly force-pushed the distribution-transform branch from 6985966 to 176b413 Compare March 30, 2022 04:39
@cxxly cxxly force-pushed the distribution-transform branch from 176b413 to 4af8502 Compare March 30, 2022 06:11
@cxxly cxxly force-pushed the distribution-transform branch from 4af8502 to 723a279 Compare March 30, 2022 09:26
@cxxly cxxly force-pushed the distribution-transform branch from 723a279 to c8ef422 Compare March 30, 2022 14:24
jeff41404
jeff41404 approved these changes Mar 31, 2022
View reviewed changes
Copy link
Contributor

@jeff41404 jeff41404 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
XiaoguangHu01
XiaoguangHu01 approved these changes Mar 31, 2022
View reviewed changes
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
python/paddle/distribution/__init__.py
from paddle.distribution.kl import kl_divergence, register_kl
from paddle.distribution.multinomial import Multinomial
from paddle.distribution.normal import Normal
from paddle.distribution.transform import * # noqa: F403
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 Mar 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里需要import *的原因是什么呢？
Copy link
Contributor Author

@cxxly cxxly Mar 31, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

distribution/transform.py 文件中定义了需要公开API的__ALL__列表，在distribution/__init__中用import * 全部导出，并添加到__init__.py 的__all__列表中， 可以通过paddle.distribution.xxx访问，访问路径和竞品保持一致
@iclementine iclementine merged commit 6735a37 into PaddlePaddle:develop Mar 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

5 participants

@cxxly @jeff41404 @TCChenlong @iclementine @XiaoguangHu01