Skip to content

Commit ec5dc5e

Browse files
authored
Update with more acknowledgements, include Lanczos
1 parent 21122f7 commit ec5dc5e

File tree

1 file changed

+15
-4
lines changed

1 file changed

+15
-4
lines changed

README.md

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# pytorch-hessian-eigenthings
22

3-
The `hessian-eigenthings` module provides an efficient way to compute the eigendecomposition of the Hessian for an arbitrary PyTorch model. It uses PyTorch's Hessian-vector product and stochastic power iteration with deflation to compute the top eigenvalues and eigenvectors of the Hessian.
3+
The `hessian-eigenthings` module provides an efficient (and scalable!) way to compute the eigendecomposition of the Hessian for an arbitrary PyTorch model. It uses PyTorch's Hessian-vector product and your choice of (a) the Lanczos method or (b) stochastic power iteration with deflation in order to compute the top eigenvalues and eigenvectors of the Hessian.
44

55
## Installation
66

@@ -29,12 +29,23 @@ eigenvals, eigenvecs = compute_hessian_eigenthings(model, dataloader,
2929

3030
This also includes a more general power iteration with deflation implementation in `power_iter.py`.
3131

32+
## Citing this work
33+
If you find this repo useful and would like to cite it in a publication, here is a BibTeX entry:
34+
35+
@misc{hessian-eigenthings,
36+
author = {Noah Golmant, Zhewei Yao, Amir Gholami, Michael Mahoney, Joseph Gonzalez},
37+
title = {{pytorch-hessian-eigentings: efficient PyTorch Hessian eigendecomposition}},
38+
month = oct,
39+
year = 2018,
40+
version = {1.0},
41+
url = {https://github.com/noahgolmant/pytorch-hessian-eigenthings}
42+
}
43+
44+
3245
## Acknowledgements
3346

34-
This code was written in collaboration with Zhewei Yao, Amir Gholami, and Michael Mahoney in UC Berkeley's [RISELab](https://rise.cs.berkeley.edu).
47+
This code was written in collaboration with Zhewei Yao, Amir Gholami, Michael Mahoney, and Joseph Gonzalez in UC Berkeley's [RISELab](https://rise.cs.berkeley.edu).
3548

3649
The deflated power iteration routine is based on code in the [HessianFlow](https://github.com/amirgholami/HessianFlow) repository recently described in the following paper: Z. Yao, A. Gholami, Q. Lei, K. Keutzer, M. Mahoney. "Hessian-based Analysis of Large Batch Training and Robustness to Adversaries", *NIPS'18* ([arXiv:1802.08241](https://arxiv.org/abs/1802.08241))
3750

3851
Stochastic power iteration with acceleration is based on the following paper: C. De Sa, B. He, I. Mitliagkas, C. Ré, P. Xu. "Accelerated Stochastic Power Iteration", *PMLR-21* ([arXiv:1707.02670](https://arxiv.org/abs/1707.02670))
39-
40-

0 commit comments

Comments
 (0)