Skip to content

Conversation

@developer0hye
Copy link
Contributor

Overview

This PR updates the sigmoid_focal_loss function to explicitly raise a ValueError when alpha > 1.
In the focal loss formulation, alpha is meant to balance positive and negative examples and should
generally lie within the range (0, 1) or be set to -1 to ignore the balancing factor.
Allowing alpha to exceed 1 can lead to unexpected behaviors.

Changes

  • Added a condition to check if alpha > 1. If true, it raises a ValueError.
  • Clarified in the docstring that valid values of alpha are within (0, 1) or -1 for ignoring.

Reasoning

  • According to the focal loss paper and its implementations, alpha in the range of (0, 1) (or -1)
    is standard practice.
  • Ensuring alpha does not exceed 1 prevents users from accidentally passing invalid parameters,
    thus avoiding silent errors or unstable training behavior.

Additional Notes

  • If there is a special use case for greater, we can discuss supporting that explicitly.
    However, for most applications, restricting alpha to (0, 1) or -1 remains consistent with the
    original focal loss approach.
- Restrict alpha to (0, 1) or -1 for ignore, as per the original focal loss settings. - Add a check to raise ValueError when alpha exceeds 1. - This helps prevent invalid alpha values from silently causing unexpected behaviors.
@pytorch-bot
Copy link

pytorch-bot bot commented Jan 26, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8882

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @developer0hye

@NicolasHug NicolasHug merged commit e8e8af5 into pytorch:main Feb 19, 2025
1 check passed
@developer0hye developer0hye deleted the focalloss-warning branch February 19, 2025 15:04
facebook-github-bot pushed a commit that referenced this pull request Jul 9, 2025
…8882) Reviewed By: scotts Differential Revision: D77997070 fbshipit-source-id: be9289a4b3fab0447d17ae18ceeddd617cc9ab75 Co-authored-by: Nicolas Hug <contact@nicolas-hug.com> Co-authored-by: Nicolas Hug <nh.nicolas.hug@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment