Skip to content

Conversation

@KumoLiu
Copy link
Contributor

@KumoLiu KumoLiu commented Apr 20, 2023

part of Project-MONAI/MONAI#6156

Description

Show lazy-resampling benchmark results on 3d spleen segmentation. The results include the following two main parts.

  • Time spent on each transform and the total amount of data preparation in the lazy and non-lazy mode.
  • End-to-end time comparison

Checks

  • Avoid including large-size files in the PR.
  • Clean up long text outputs from code cells in the notebook.
  • For security purposes, please check the contents and remove any sensitive info such as user names and private key.
  • Ensure (1) hyperlinks and markdown anchors are working (2) use relative paths for tutorial repo files (3) put figure and graphs in the ./figure folder
  • Notebook runs automatically ./runner.sh -t <path to .ipynb file>
Signed-off-by: KumoLiu <yunl@nvidia.com>
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

pre-commit-ci bot and others added 11 commits April 20, 2023 07:00
Signed-off-by: KumoLiu <yunl@nvidia.com>
Signed-off-by: KumoLiu <yunl@nvidia.com>
Signed-off-by: KumoLiu <yunl@nvidia.com>
Signed-off-by: KumoLiu <yunl@nvidia.com>
Signed-off-by: KumoLiu <yunl@nvidia.com>
Signed-off-by: KumoLiu <yunl@nvidia.com>
Signed-off-by: KumoLiu <yunl@nvidia.com>
@KumoLiu KumoLiu marked this pull request as ready for review April 26, 2023 02:50
@KumoLiu
Copy link
Contributor Author

KumoLiu commented Apr 26, 2023

Hi @wyli, @Nic-Ma, since I was using latest MONAI version to do the benchmark, can I just use MONAI 1.2.0 in the tutorial or should I use 1.2.0rc5 and updated it once 1.2.0 release?

@KumoLiu KumoLiu requested a review from Nic-Ma April 26, 2023 02:52
@wyli
Copy link
Contributor

wyli commented Apr 26, 2023

Hi @wyli, @Nic-Ma, since I was using latest MONAI version to do the benchmark, can I just use MONAI 1.2.0 in the tutorial or should I use 1.2.0rc5 and updated it once 1.2.0 release?

The main goal is to make the numbers reported reproducible as much as possible, so I think we just report what has been used in the experiments. 1.2.0rc5. (and pytorch 1.14.0? I couldn't find it)

@KumoLiu
Copy link
Contributor Author

KumoLiu commented Apr 26, 2023

Hi @wyli, @Nic-Ma, since I was using latest MONAI version to do the benchmark, can I just use MONAI 1.2.0 in the tutorial or should I use 1.2.0rc5 and updated it once 1.2.0 release?

The main goal is to make the numbers reported reproducible as much as possible, so I think we just report what has been used in the experiments. 1.2.0rc5. (and pytorch 1.14.0? I couldn't find it)

Hi @wyli, but I'm not test on 1.2.0rc5 because we have some new changes such as Project-MONAI/MONAI#6412, and the pytorch version is "1.14.0a0+410ce96". Maybe I should change both version to the stable one and retrain it?

@wyli
Copy link
Contributor

wyli commented Apr 26, 2023

Hi @wyli, @Nic-Ma, since I was using latest MONAI version to do the benchmark, can I just use MONAI 1.2.0 in the tutorial or should I use 1.2.0rc5 and updated it once 1.2.0 release?

The main goal is to make the numbers reported reproducible as much as possible, so I think we just report what has been used in the experiments. 1.2.0rc5. (and pytorch 1.14.0? I couldn't find it)

Hi @wyli, but I'm not test on 1.2.0rc5 because we have some new changes such as Project-MONAI/MONAI#6412, and the pytorch version is "1.14.0a0+410ce96". Maybe I should change both version to the stable one and retrain it?

sure, you can also just report the monai version from python -c "import monai; monai.config.print_config()" I see in the notebook there is 1.2.0rc4+9.ge18097d0 and 1.13.0a0+d0d6b1f these are good enough for reproducing the results.

Signed-off-by: KumoLiu <yunl@nvidia.com>
@wyli wyli merged commit 97d0af4 into Project-MONAI:main Apr 26, 2023
@KumoLiu KumoLiu deleted the lazy-benchmark branch April 26, 2023 09:50
boneseva pushed a commit to boneseva/MONAI-tutorials that referenced this pull request Apr 21, 2024
part of Project-MONAI/MONAI#6156 ### Description Show lazy-resampling benchmark results on 3d spleen segmentation. The results include the following two main parts. - Time spent on each transform and the total amount of data preparation in the lazy and non-lazy mode. - End-to-end time comparison ### Checks <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Avoid including large-size files in the PR. - [ ] Clean up long text outputs from code cells in the notebook. - [ ] For security purposes, please check the contents and remove any sensitive info such as user names and private key. - [ ] Ensure (1) hyperlinks and markdown anchors are working (2) use relative paths for tutorial repo files (3) put figure and graphs in the `./figure` folder - [ ] Notebook runs automatically `./runner.sh -t <path to .ipynb file>` --------- Signed-off-by: KumoLiu <yunl@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants