Skip to content

Conversation

@ysiraichi
Copy link
Collaborator

@ysiraichi ysiraichi commented Feb 28, 2024

This PR implements small fixes introduced by past PRs:

@miladm

@zpcore
Copy link
Member

zpcore commented Feb 28, 2024

Thanks, I think you forgot to import exists.

@zpcore
Copy link
Member

zpcore commented Feb 28, 2024

There are also other issues I see in recent check-ins:

model_name in config_data()["dont_change_batch_size"])
, need self.model_name.
For the batch_size, I can't find DEFAULT_EVAL_BSIZE and DEFAULT_TRAIN_BSIZE to be use anywhere.

@ysiraichi ysiraichi changed the title [benchmarks] Fix find_near_file function. [benchmarks] Small fixes for benchmarking script. Feb 29, 2024
@ysiraichi
Copy link
Collaborator Author

ysiraichi commented Feb 29, 2024

@zpcore @vanbasten23 @cota I have updated this PR fixing another small issues with the benchmarking script, as pointed out by @zpcore. I have also tested it by running BERT_pytorch. Could you take a look at it, again, when you have some time?

Copy link
Member

@zpcore zpcore left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixing!

@ysiraichi ysiraichi merged commit 4cde625 into master Feb 29, 2024
@zpcore
Copy link
Member

zpcore commented Feb 29, 2024

Hi @ysiraichi , do you know where the default batch size from torchbench was used in the code?

@ysiraichi
Copy link
Collaborator Author

Do you mean where do we set the default batch size? As far as I understand, torchbench does it automatically when we instantiate a Model. Specifically, here:

return self.benchmark_cls()(
test=self.benchmark_experiment.test,
device=device,
batch_size=batch_size,
)

Which ends up in this function.

@zpcore
Copy link
Member

zpcore commented Feb 29, 2024

Got it, thanks! Now the batch_size setting looks clear to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6 participants