Cleanup for new leaderboard version

.github/workflows/check_pr_template.yml

@@ -150,10 +150,13 @@ jobs:
  // Create submission_info.yml
  try {
  let submissionPath;
+ // Remove any leading/trailing slashes and potential duplicate paths
+ const cleanFolder = data.submission_folder.replace(/^\/+|\/+$/g, '').replace(/^(external_tuning|self_tuning)\//, '');
+
  if (data.ruleset === 'external') {
- submissionPath = `submissions/external_tuning/${data.submission_folder}`;
+ submissionPath = `submissions/external_tuning/${cleanFolder}`;
  } else if (data.ruleset === 'self-tuning') {
- submissionPath = `submissions/self_tuning/${data.submission_folder}`;
+ submissionPath = `submissions/self_tuning/${cleanFolder}`;
  } else {
  core.setFailed(`Invalid ruleset value: ${data.ruleset}. Must be "external" or "self-tuning".`);
  return;

previous_leaderboards/README.md

@@ -1,3 +1,6 @@
-# Previouls AlgoPerf Leaderboards
+# Previous AlgoPerf Leaderboards
 
-This directory will contain previous AlgoPerf leaderboards. The most recent leaderboard can be found in the [README.md](../README.md) file.
+This directory contains previous AlgoPerf leaderboards.
+The most recent leaderboard can be found in the [README.md](../README.md) file.
+
+- [**v0.5**](./algoperf_v05/README.md) The inaugural leaderboard from the first AlgoPerf competition as described in the [ICLR 2025 paper](https://arxiv.org/abs/2502.15015).

submissions/external_tuning/baseline/submission_info.yml

@@ -0,0 +1,9 @@
+submission_name: Baseline
+submission_folder: submissions/external_tuning/baseline
+authors: ""
+affiliations: ""
+version: "1.0"
+ruleset: external
+framework: JAX
+description: >-
+ Baseline using NadamW ([Dozat, 2016](https://openreview.net/forum?id=OM0jvwB8jIp57ZJjtNEZ); [Loshchilov & Hutter, 2019](https://openreview.net/forum?id=Bkg6RiCqY7)) and a linear learning rate warmup followed by a cosine decay ([Dahl et al., 2023](https://arxiv.org/abs/2306.07179)).

submissions/external_tuning/schedule_free_adamw_v2/submission_info.yml

@@ -0,0 +1,10 @@
+submission_name: Schedule-Free AdamW
+submission_folder: submissions/external_tuning/schedule_free_adamw_v2
+authors: >-
+ Alice Yang, Aaron Defazio, Konstantin Mishchenko
+affiliations: Meta AI, Samsung AI
+version: "2.0"
+ruleset: external
+framework: PyTorch
+description: >-
+ An externally tuned version of Schedule Free AdamW ([Defazio et al., 2024](https://openreview.net/forum?id=0XeNkkENuI)) with a list of five hyperparameter configurations. Version 2.0 uses slightly different hyperparameter configurations. It also makes use of the batch-norm fixes made after the competition deadline, and which follows the published version of Schedule-Free in a few small details for consistency.

submissions/external_tuning/shampoo/submission_info.yml

@@ -0,0 +1,11 @@
+submission_name: Distributed Shampoo
+submission_folder: submissions/external_tuning/shampoo
+authors: >-
+ Hao-Jun Shi, Tsung-Hsien Lee, Anna Cai, Shintaro Iwasaki, Wenyin Fu, Yuchen
+ Hao, Mike Rabbat
+affiliations: Meta
+version: "1.0"
+ruleset: external
+framework: PyTorch
+description: >-
+ Based on the Distributed Shampoo algorithm of [Anil et al. (2020)](https://arxiv.org/abs/2002.09018) with an implementation tailored to leverage PyTorch performance optimizations. See [Shi et al. (2023)](https://arxiv.org/abs/2309.06497) for details. The submission uses a list of five hyperparameter settings.

submissions/external_tuning/weighted_schedule_free_adamw/submission_info.yml

@@ -0,0 +1,10 @@
+submission_name: Schedule-Free AdamW Weighted
+submission_folder: submissions/external_tuning/weighted_schedule_free_adamw
+authors: >-
+ Alice Yang, Aaron Defazio, Konstantin Mishchenko
+affiliations: Meta AI, Samsung AI
+version: "1.0"
+ruleset: external
+framework: PyTorch
+description: >-
+ An externally tuned version of Schedule Free AdamW ([Defazio et al., 2024](https://openreview.net/forum?id=0XeNkkENuI)) with a list of five hyperparameter configurations. It is a weighted dual-averaging version of Schedule-Free, which shows improved convergence over the unweighted version on some workloads, particularly ViT.

submissions/self_tuning/baseline/submission_info.yml

@@ -0,0 +1,9 @@
+submission_name: Baseline
+submission_folder: submissions/self_tuning/baseline
+authors: ""
+affiliations: ""
+version: "1.0"
+ruleset: self-tuning
+framework: JAX
+description: >-
+ Baseline using NadamW, a linear learning rate warmup followed by a cosine decay, and a single hyperparameter point ([Dahl et al., 2023](https://arxiv.org/abs/2306.07179)).

submissions/self_tuning/schedule_free_adamw_v2/submission_info.yml

@@ -0,0 +1,10 @@
+submission_name: Schedule-Free AdamW
+submission_folder: submissions/self_tuning/schedule_free_adamw_v2
+authors: >-
+ Alice Yang, Aaron Defazio, Konstantin Mishchenko
+affiliations: Meta AI, Samsung AI
+version: "2.0"
+ruleset: self-tuning
+framework: PyTorch
+description: >-
+ A self-tuning version of Schedule Free AdamW ([Defazio et al., 2024](https://openreview.net/forum?id=0XeNkkENuI)) using a single hyperparameter configuration. Version 2.0 makes use of the batch-norm fixes made after the competition deadline, and which follows the published version of Schedule-Free in a few small details for consistency.