[Security Solution][Detection Engine] adds DOES NOT MATCH capability to IM rule type #227084
+2,282 −666
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
vitaliidm added backport:skip 
# Conflicts: # x-pack/solutions/security/plugins/security_solution/server/lib/detection_engine/rule_types/indicator_match/threat_mapping/build_threat_mapping_filter.ts # x-pack/solutions/security/plugins/security_solution/server/lib/detection_engine/rule_types/indicator_match/threat_mapping/get_signals_map_from_threat_index.ts
… corner case scenario
…kibana into de_9_2/im-not-matches
- const threatListItem = getThreatListSearchResponseMock().hits.hits[0]; +const threatListItem = getThreatListItemMock();
…/detection_engine/rule_types/indicator_match/threat_mapping/utils.test.ts Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Contributor Author
| @marshallmain, thanks for the review. I have addressed the comments |
…mon/components/threat_match/translations.ts Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
nastasha-solomon approved these changes Aug 5, 2025
Contributor
nastasha-solomon left a comment
nastasha-solomon left a comment There was a problem hiding this comment.
Just the one small tweak. Otherwise, lgtm for now!
| message: i18n.translate( | ||
| 'xpack.securitySolution.detectionEngine.ruleManagement.threatMappingField.singleNotMatchClauseError', | ||
| { | ||
| defaultMessage: 'Conditions in AND clauses must have at least one MATCHES entry.', |
Contributor
There was a problem hiding this comment.
Suggested change
| defaultMessage: 'Conditions in AND clauses must have at least one MATCHES entry.', | |
| defaultMessage: 'Conditions with AND clauses must have at least one MATCHES entry.', |
marshallmain approved these changes Aug 5, 2025
Contributor
marshallmain left a comment
marshallmain left a comment There was a problem hiding this comment.
LGTM, thanks for addressing my comments 🚀
Understanding the changes to validateCompleteThreatMatches was by far the hardest part, and I hope we can drastically simplify that logic by adjusting the way we name query clauses. The remaining unresolved comments are for future (soon) work.
| return false; | ||
| }); | ||
| Object.values(threatQueriesMap).forEach((threatQueriesPerId) => { | ||
| const matchedThreatQueriesForAndGroup: ThreatMatchNamedQuery[] = []; |
Contributor
There was a problem hiding this comment.
This const can go inside the inner loop as well?
Contributor
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
Async chunks
Page load bundle
Unknown metric groupsAPI count
ESLint disabled in files
Total ESLint disabled count
History
cc @vitaliidm |
gergoabraham pushed a commit to gergoabraham/kibana that referenced this pull request Aug 7, 2025
…to IM rule type (elastic#227084) ## Summary - addresses elastic/security-team#13022 - allows selecting DOES NOT MATCH condition, it would create alert for any document that has at least one matching field and not matching. Single DOES NOT MATCH is **not allowed**, to prevent creation of large number of alerts. User would be able to select match first and then tune it with DOES NOT MATCH if needed - added `negate: boolean` value to ThreatMap entry schema to indicate this mapping should use as DOES NOT MATCH. Field name `negate` was chosen as it already used in Kibana filters to indicate field does not match some value. - Enrichments for DOES NOT MATCH field are omitted - If one of the DOES NOT MATCH fields is empty, we create an alert. If both empty - no alert should be created - I fixed issue within `validateCompleteThreatMatches`, where False Positive alert can be created if there partial matches across AND group in multiple threats - Removed `x-pack/solutions/security/packages/kbn-securitysolution-io-ts-alerting-types/src/threat_mapping/index.ts` in favour of OpenAPI schema ### Feature Flag ```yml xpack.securitySolution.enableExperimental: - doesNotMatchForIndicatorMatchRuleEnabled ``` ### UI 1. Having help text under label and for the first entry, DOES NOT MATCH option should be disabled <img width="1112" height="214" alt="Screenshot 2025-07-28 at 10 47 26" src="https://github.com/user-attachments/assets/740c8170-45ae-4773-a93b-ac2e0bf12c9c" /> 2. User can add DOES NOT MATCH as second AND condition <img width="1050" height="320" alt="Screenshot 2025-07-28 at 10 47 08" src="https://github.com/user-attachments/assets/56b1f26e-6951-47e7-b7a6-4ff62f5b44a5" /> 3. If user deletes first MATCH we would show validation error <img width="1040" height="241" alt="Screenshot 2025-07-28 at 10 47 43" src="https://github.com/user-attachments/assets/a61274b3-087d-450d-97cc-5ac2b40079db" /> 4. user cannot setup a match and not match condition for the same field, validation error <img width="1064" height="282" alt="Screenshot 2025-07-28 at 10 48 02" src="https://github.com/user-attachments/assets/0ae790e1-7e3d-4fb7-8550-fb3d92b9d678" /> ### Flaky test runner Cypress: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8886 FTR: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8885 ### Docs issue - elastic/docs-content#2295 ### Test plan - elastic/security-team#13367 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: florent-leborgne <florent.leborgne@elastic.co> Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
NicholasPeretti pushed a commit to NicholasPeretti/kibana that referenced this pull request Aug 18, 2025
…to IM rule type (elastic#227084) ## Summary - addresses elastic/security-team#13022 - allows selecting DOES NOT MATCH condition, it would create alert for any document that has at least one matching field and not matching. Single DOES NOT MATCH is **not allowed**, to prevent creation of large number of alerts. User would be able to select match first and then tune it with DOES NOT MATCH if needed - added `negate: boolean` value to ThreatMap entry schema to indicate this mapping should use as DOES NOT MATCH. Field name `negate` was chosen as it already used in Kibana filters to indicate field does not match some value. - Enrichments for DOES NOT MATCH field are omitted - If one of the DOES NOT MATCH fields is empty, we create an alert. If both empty - no alert should be created - I fixed issue within `validateCompleteThreatMatches`, where False Positive alert can be created if there partial matches across AND group in multiple threats - Removed `x-pack/solutions/security/packages/kbn-securitysolution-io-ts-alerting-types/src/threat_mapping/index.ts` in favour of OpenAPI schema ### Feature Flag ```yml xpack.securitySolution.enableExperimental: - doesNotMatchForIndicatorMatchRuleEnabled ``` ### UI 1. Having help text under label and for the first entry, DOES NOT MATCH option should be disabled <img width="1112" height="214" alt="Screenshot 2025-07-28 at 10 47 26" src="https://github.com/user-attachments/assets/740c8170-45ae-4773-a93b-ac2e0bf12c9c" /> 2. User can add DOES NOT MATCH as second AND condition <img width="1050" height="320" alt="Screenshot 2025-07-28 at 10 47 08" src="https://github.com/user-attachments/assets/56b1f26e-6951-47e7-b7a6-4ff62f5b44a5" /> 3. If user deletes first MATCH we would show validation error <img width="1040" height="241" alt="Screenshot 2025-07-28 at 10 47 43" src="https://github.com/user-attachments/assets/a61274b3-087d-450d-97cc-5ac2b40079db" /> 4. user cannot setup a match and not match condition for the same field, validation error <img width="1064" height="282" alt="Screenshot 2025-07-28 at 10 48 02" src="https://github.com/user-attachments/assets/0ae790e1-7e3d-4fb7-8550-fb3d92b9d678" /> ### Flaky test runner Cypress: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8886 FTR: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8885 ### Docs issue - elastic/docs-content#2295 ### Test plan - elastic/security-team#13367 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: florent-leborgne <florent.leborgne@elastic.co> Co-authored-by: Marshall Main <55718608+marshallmain@users.noreply.github.com>
Merged
5 tasks
qn895 pushed a commit to qn895/kibana that referenced this pull request Sep 2, 2025
…H operator (elastic#232996) ## Summary - follow up for elastic#227084 - enables FF `doesNotMatchForIndicatorMatchRuleEnabled`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Summary
negate: booleanvalue to ThreatMap entry schema to indicate this mapping should use as DOES NOT MATCH.Field name
negatewas chosen as it already used in Kibana filters to indicate field does not match some value.validateCompleteThreatMatches, where False Positive alert can be created if there partial matches across AND group in multiple threatsx-pack/solutions/security/packages/kbn-securitysolution-io-ts-alerting-types/src/threat_mapping/index.tsin favour of OpenAPI schemaFeature Flag
UI
Flaky test runner
Cypress: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8886
FTR: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8885
Docs issue
Test plan