Stop raising RuntimeError when setting download=True (#8637) #8638
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8638
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
| Hi @imkasen! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
| return check_integrity(self._size_root.with_suffix(".tgz"), self._md5) | ||
| | ||
| def _download(self): | ||
| if self._check_exists(): |
There was a problem hiding this comment.
Thanks for the PR @imkasen .
@pmeier do you remember why we're erroring instead of just doing an early return like in other datasets e.g. clevr ?
vision/torchvision/datasets/clevr.py
Lines 81 to 83 in 838ad6c
There was a problem hiding this comment.
The general idea was to raise on download=True if we can't properly verify the integrity. For example, after a folder is extracted, there is no way for us to know if the images inside this folder have not been tempered with.
Contrast this with datasets like MNIST or CIFAR, where we have checksums for each file on record. Here we can properly check the integrity and thus there is no need to raise on download=True.
Now, both CLEVRClassification and Imagenette fall into the first category. I don't know why we didn't follow our own "rule" for the former, but it is likely too late to change it now. However, I would treat "relaxing" the rule for the latter with caution. I don't think anything has changed that warrants walking back on our previous decision. However, if you do, you probably want to do it for all datasets to keep things consistent.
4 participants
I fixed the problem (#8637) after reading the source code of
torchvision.datasets.CIFAR10(). RuntimeError is no longer thrown when downloading, and the downloaded compressed package will be verified.