Show "experimental" badges around ML functionality #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| This is a benchmark review for experiment This pull request was cloned from Experiment configurationreview_config: # User configuration for the review # - benchmark - use the user config from the benchmark reviews # - <value> - use the value directly user_review_config: enable_ai_review: true enable_rule_comments: false enable_complexity_comments: benchmark enable_security_comments: benchmark enable_tests_comments: benchmark enable_comment_suggestions: benchmark enable_pull_request_summary: benchmark enable_review_guide: benchmark enable_approvals: false base_branches: [base-sha.*] ai_review_config: # The model responses to use for the experiment # - benchmark - use the model responses from the benchmark reviews # - llm - call the language model to generate responses model_responses: comments_model: benchmark comment_validation_model: benchmark comment_suggestion_model: benchmark complexity_model: benchmark security_model: benchmark tests_model: benchmark pull_request_summary_model: benchmark review_guide_model: benchmark overall_comments_model: benchmark # The pull request dataset to run the experiment on pull_request_dataset: # CodeRabbit - https://github.com/neerajkumar161/node-coveralls-integration/pull/5 - https://github.com/gunner95/vertx-rest/pull/1 - https://github.com/Altinn/altinn-access-management-frontend/pull/1427 - https://github.com/theMr17/github-notifier/pull/14 - https://github.com/bearycool11/AI_memory_Loops/pull/142 # Greptile - https://github.com/gumloop/guMCP/pull/119 - https://github.com/autoblocksai/python-sdk/pull/335 - https://github.com/grepdemos/ImageSharp/pull/6 - https://github.com/grepdemos/server/pull/61 - https://github.com/websentry-ai/pipelines/pull/25 # Graphite - https://github.com/KittyCAD/modeling-app/pull/6648 - https://github.com/KittyCAD/modeling-app/pull/6628 - https://github.com/Varedis-Org/AI-Test-Repo/pull/2 - https://github.com/deeep-network/bedrock/pull/198 - https://github.com/Metta-AI/metta/pull/277 # Copilot - https://github.com/hmcts/rpx-xui-webapp/pull/4438 - https://github.com/ganchdev/quez/pull/104 - https://github.com/xbcsmith/ymlfxr/pull/13 - https://github.com/tinapayy/B-1N1T/pull/36 - https://github.com/coder/devcontainer-features/pull/6 # Questions to ask to label the review comments review_comment_labels: [] # - label: correct # question: Is this comment correct? # Benchmark reviews generated by running # python -m scripts.experiment benchmark <experiment_name> benchmark_reviews: [] |
Reviewer's GuideThis PR introduces an 'experimental' status for features, primarily targeting ML functionality, controlled by a new global constant File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Hellebore commented May 2, 2025
There was a problem hiding this comment.
Hey @Hellebore - I've reviewed your changes - here's some feedback:
- Consider extracting the 'Experimental' badge UI (icon and text) into a reusable component, as it's implemented similarly in multiple places (Toolbar tooltips, Home button tooltip, CommandComboBox, ActionButtonDropdown).
- The logic checking if an item status is 'available' or 'experimental' is repeated in several locations within Toolbar.tsx; consider abstracting this into a helper function or consolidating the check.
Here's what I looked at during the review
- 🟡 General issues: 1 issue found
- 🟢 Security: all looks good
- 🟢 Testing: all looks good
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| {displayHotkeys(hotkey)} | ||
| </kbd> | ||
| {status === 'experimental' && ( | ||
| <div className="text-xs flex justify-center item-center gap-1 pb-1 border-b border-chalkboard-50"> |
There was a problem hiding this comment.
issue (typo): Possible typo in flex alignment class.
Tailwind's class is 'items-center', not 'item-center'. Please update it.
Suggested change
| <div className="text-xs flex justify-center item-center gap-1 pb-1 border-b border-chalkboard-50"> | |
| <div className="text-xs flex justify-center items-center gap-1 pb-1 border-b border-chalkboard-50"> |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
We want users to have realistic expectations about our ML-powered features, so we're going to add a little UI tag to remind them that we're working to improve the model and it will get better over time.
Summary by Sourcery
Add "experimental" badges and status to ML-powered features to set user expectations about ongoing improvements
New Features:
Enhancements:
Chores: