Add test_min_stack_version_supported testcase

[shashank-elastic]

Pull Request

Issue link(s): NA

Summary

Reported Issue

• We have some rules with older min_stack_version

Investigation

• We trimmed back ports at Prep of 8.19/9.1 via the PR, and the Trim Version Lock command was also executed. The PR was merged on Jul 7
• The minstack version addition for the said sample rule WS IAM Assume Role Policy Update came via PR [Rule Tunings] Reduce Usage of Flattened Fields in AWS Rules #4892 which was merged on Jul 19.
• It appears that we can add min_stack without a check to active versions in detection_rules/etc/stack-schema-map.yaml or even .github/workflows/lock-versions.yml

Comprehensively affected rules with minstack lesser than 8.18.0 are below

• AWS IAM Customer-Managed Policy Attached to Role by Rare User --> Updated part of PR [Rule Tunings] Reduce Usage of Flattened Fields in AWS Rules #4892 which was merged on Jul 19.
• AWS IAM Assume Role Policy Update --> Investigation Pinned above
• Microsoft Entra ID Sign-In Brute Force Activity --> Part of PR [Rule Tuning] ESQL Query Field Dynamic Field Standardization #4912 merged on Jul 16 which is also post version trimming
• Windows User Account Creation --> This is a development rule which means not in final artifact so trim-version-lock command would have skipped dropping the versions

Solution

• Added test_min_stack_version_supported test case
• Rule Tuning for production rules to drop older non supported minstack versions - Tune Rules that have unsupported versions in min_stack_version #5079

How To Test

• Unit Test should Fail on rules that have min_stack_versions older than supported versions - Job

• Unit Test Local Execution

 f"{self.rule_str(rule)} min_stack_version={min_stack_version} < supported={min_supported}"  )    if failures:  fail_msg = "The following rules have min_stack_version lower than the minimum supported in stack-schema-map.yaml:\n" > self.fail(fail_msg + "\n".join(failures)) E AssertionError: The following rules have min_stack_version lower than the minimum supported in stack-schema-map.yaml: E f6d07a70-9ad0-11ef-954f-f661ea17fbcd - AWS IAM Customer-Managed Policy Attached to Role by Rare User -> min_stack_version=8.16.5 < supported=8.18.0 E a60326d7-dca7-4fb7-93eb-1ca03a1febbd - AWS IAM Assume Role Policy Update -> min_stack_version=8.16.5 < supported=8.18.0 E cca64114-fb8b-11ef-86e2-f661ea17fbce - Microsoft Entra ID Sign-In Brute Force Activity -> min_stack_version=8.17.0 < supported=8.18.0 tests/test_all_rules.py:1066: AssertionError ========================================================================== short test summary info ========================================================================== FAILED tests/test_all_rules.py::TestRuleMetadata::test_min_stack_version_supported - AssertionError: The following rules have min_stack_version lower than the minimum supported in stack-schema-map.yaml: ======================================================================= 1 failed in 64.43s (0:01:04) ======================================================================== detection-rules on  test_min_stack_version_supported [$!+?] is 📦 v1.3.32 via 🐍 v3.12.8 (.venv) on ☁️ shashank.suryanarayana@elastic.co took 1m6s 

• Version Semantic Comparison in Debug Mode

• Unit Test should pass , once tuning is merged -> Tune Rules that have unsupported versions in min_stack_version #5079

Additional Context

More Details in thread - https://elastic.slack.com/archives/C07V87YPS3F/p1757442466614199

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[tradebot-elastic]

⛔️ Test failed

Results

• ❌ Windows User Account Creation (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS IAM Assume Role Policy Update (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS IAM Customer-Managed Policy Attached to Role by Rare User (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[github-actions]

Enhancement - Guidelines

These guidelines serve as a reminder set of considerations when addressing adding a feature to the code.

Documentation and Context

• Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
• Include additional context or screenshots.
• Ensure the enhancement includes necessary updates to the documentation and versioning.

Code Standards and Practices

• Code follows established design patterns within the repo and avoids duplication.
• Ensure that the code is modular and reusable where applicable.

Testing

• New unit tests have been added to cover the enhancement.
• Existing unit tests have been updated to reflect the changes.
• Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
• Validate that any rules affected by the enhancement are correctly updated.
• Ensure that performance is not negatively impacted by the changes.
• Verify that any release artifacts are properly generated and tested.
• Conducted system testing, including fleet, import, and create APIs (e.g., run make test-cli, make test-remote-cli, make test-hunting-cli)

Additional Checks

• Verify that the enhancement works across all relevant environments (e.g., different OS versions).
• Confirm that the proper version label is applied to the PR patch, minor, major.

[Mikaayenson]

Can you double check the repo for existing precedence?

[Mikaayenson]

We probably do not need the development rule. We also should think about mixing rule / code changes in the same pr and how it'll affect backports. It might be better to tune the rules separately.

[shashank-elastic]

Moving to draft Until I check and address the review comments

[shashank-elastic]

We probably do not need the development rule. We also should think about mixing rule / code changes in the same pr and how it'll affect backports. It might be better to tune the rules separately.

• Since we were dropping min_stack_versions and aligning the rules to minimum supported versions, this was a fairly safe operation during back ports.
• However to adhere to the best practise I have split the PR and the tuning PR is here Tune Rules that have unsupported versions in min_stack_version #5079

cc @Mikaayenson

[Mikaayenson]

lgtm!

[eric-forte-elastic]

🟢 Minor nit otherwise looks good to me! 👍