Skip to content

[Bug] Fix test_os_and_platform_in_query test and rules #3695

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 20, 2024
Merged

[Bug] Fix test_os_and_platform_in_query test and rules #3695

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
308335b
[Bug] Fix test_os_and_platform_in_query test and rules
brokensound77 May 18, 2024
d1d491f
Merge branch 'main' into fix-host-os-type-test-and-rules
brokensound77 May 18, 2024
3e9f637
Merge branch 'main' into fix-host-os-type-test-and-rules
brokensound77 May 20, 2024
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions rules/linux/execution_shell_via_child_tcp_utility_linux.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ integration = ["endpoint"]
maturity = "production"
min_stack_comments = "New fields added: required_fields, related_integrations, setup"
min_stack_version = "8.3.0"
updated_date = "2023/11/02"
updated_date = "2024/05/17"

[rule]
author = ["Elastic"]
Expand Down Expand Up @@ -61,12 +61,13 @@ tags = [
type = "eql"
query = '''
sequence by host.id, process.entity_id with maxspan=5s
[network where event.type == "start" and event.action in ("connection_attempted", "connection_accepted") and
process.name : ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish", "socat") and destination.ip != null and
not cidrmatch(destination.ip, "127.0.0.0/8", "169.254.0.0/16", "224.0.0.0/4", "::1")]
[process where event.type == "start" and event.action == "exec" and
process.name in ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish") and (
(process.args : ("-i", "-l")) or (process.parent.name == "socat" and process.parent.args : "*exec*")
[network where event.type == "start" and host.os.type == "linux" and
event.action in ("connection_attempted", "connection_accepted") and
process.name : ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish", "socat") and destination.ip != null and
not cidrmatch(destination.ip, "127.0.0.0/8", "169.254.0.0/16", "224.0.0.0/4", "::1")]
[process where event.type == "start" and host.os.type == "linux" and event.action == "exec" and
process.name in ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish") and (
(process.args : ("-i", "-l")) or (process.parent.name == "socat" and process.parent.args : "*exec*")
)]
'''

Expand Down
15 changes: 8 additions & 7 deletions rules/linux/execution_shell_via_tcp_cli_utility_linux.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ integration = ["endpoint"]
maturity = "production"
min_stack_comments = "New fields added: required_fields, related_integrations, setup"
min_stack_version = "8.3.0"
updated_date = "2023/11/02"
updated_date = "2024/05/17"

[rule]
author = ["Elastic"]
Expand Down Expand Up @@ -59,12 +59,13 @@ tags = [
type = "eql"
query = '''
sequence by host.id with maxspan=5s
[network where event.type == "start" and event.action in ("connection_attempted", "connection_accepted") and
process.name : ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish", "socat") and destination.ip != null and
not cidrmatch(destination.ip, "127.0.0.0/8", "169.254.0.0/16", "224.0.0.0/4", "::1")] by process.entity_id
[process where event.type == "start" and event.action in ("exec", "fork") and
process.name in ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish") and (
(process.args : ("-i", "-l")) or (process.parent.name == "socat" and process.parent.args : "*exec*")
[network where event.type == "start" and host.os.type == "linux" and
event.action in ("connection_attempted", "connection_accepted") and
process.name : ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish", "socat") and destination.ip != null and
not cidrmatch(destination.ip, "127.0.0.0/8", "169.254.0.0/16", "224.0.0.0/4", "::1")] by process.entity_id
[process where event.type == "start" and host.os.type == "linux" and event.action in ("exec", "fork") and
process.name in ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish") and (
(process.args : ("-i", "-l")) or (process.parent.name == "socat" and process.parent.args : "*exec*")
)] by process.parent.entity_id
'''

Expand Down
11 changes: 6 additions & 5 deletions rules/linux/persistence_kworker_file_creation.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ integration = ["endpoint"]
maturity = "production"
min_stack_comments = "New fields added: required_fields, related_integrations, setup"
min_stack_version = "8.3.0"
updated_date = "2024/02/21"
updated_date = "2024/05/17"

[transform]
[[transform.osquery]]
Expand Down Expand Up @@ -175,10 +175,11 @@ tags = [
timestamp_override = "event.ingested"
type = "eql"
query = '''
file where event.action in ("creation", "file_create_event") and process.name : "kworker*" and not (
(process.name : "kworker*kcryptd*") or
(file.path : ("/var/log/*", "/var/crash/*", "/var/run/*", "/var/lib/systemd/coredump/*", "/var/spool/*"))
)
file where host.os.type == "linux" and event.action in ("creation", "file_create_event") and
process.name : "kworker*" and not (
(process.name : "kworker*kcryptd*") or
(file.path : ("/var/log/*", "/var/crash/*", "/var/run/*", "/var/lib/systemd/coredump/*", "/var/spool/*"))
)
'''

[[rule.threat]]
Expand Down
6 changes: 3 additions & 3 deletions rules/linux/persistence_message_of_the_day_execution.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ integration = ["endpoint"]
maturity = "production"
min_stack_comments = "New fields added: required_fields, related_integrations, setup"
min_stack_version = "8.3.0"
updated_date = "2024/02/21"
updated_date = "2024/05/17"

[transform]
[[transform.osquery]]
Expand Down Expand Up @@ -157,8 +157,8 @@ tags = [
timestamp_override = "event.ingested"
type = "eql"
query = '''
process where event.type == "start" and event.action : ("exec", "exec_event") and
process.parent.executable : ("/etc/update-motd.d/*", "/usr/lib/update-notifier/*") and (
process where event.type == "start" and host.os.type == "linux" and event.action : ("exec", "exec_event") and
process.parent.executable : ("/etc/update-motd.d/*", "/usr/lib/update-notifier/*") and (
(process.name in ("bash", "dash", "sh", "tcsh", "csh", "zsh", "ksh", "fish") and (
(process.args : ("-i", "-l")) or (process.parent.name == "socat" and process.parent.args : "*exec*"))) or
(process.name : ("nc", "ncat", "netcat", "nc.openbsd") and process.args_count >= 3 and
Expand Down
4 changes: 2 additions & 2 deletions rules/linux/persistence_suspicious_file_opened_through_editor.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ creation_date = "2023/07/25"
maturity = "production"
min_stack_comments = "New fields added: required_fields, related_integrations, setup"
min_stack_version = "8.3.0"
updated_date = "2024/02/22"
updated_date = "2024/05/17"
integration = ["endpoint"]

[rule]
Expand Down Expand Up @@ -36,7 +36,7 @@ tags = [
timestamp_override = "event.ingested"
type = "eql"
query = '''
file where event.action in ("creation", "file_create_event") and file.extension == "swp" and
file where host.os.type == "linux" and event.action in ("creation", "file_create_event") and file.extension == "swp" and
file.path : (
/* common interesting files and locations */
"/etc/.shadow.swp", "/etc/.shadow-.swp", "/etc/.shadow~.swp", "/etc/.gshadow.swp", "/etc/.gshadow-.swp",
Expand Down
4 changes: 2 additions & 2 deletions rules/linux/privilege_escalation_potential_bufferoverflow_attack.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ creation_date = "2023/12/11"
maturity = "production"
min_stack_comments = "New fields added: required_fields, related_integrations, setup"
min_stack_version = "8.3.0"
updated_date = "2023/12/11"
updated_date = "2024/05/17"

[rule]
author = ["Elastic"]
Expand Down Expand Up @@ -41,7 +41,7 @@ tags = [
timestamp_override = "event.ingested"
type = "threshold"
query = '''
kibana.alert.rule.rule_id:5c81fc9d-1eae-437f-ba07-268472967013 and event.kind:signal
kibana.alert.rule.rule_id:"5c81fc9d-1eae-437f-ba07-268472967013" and host.os.type:linux and event.kind:signal
'''

[[rule.threat]]
Expand Down
4 changes: 2 additions & 2 deletions rules/macos/credential_access_promt_for_pwd_via_osascript.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ integration = ["endpoint"]
maturity = "production"
min_stack_comments = "New fields added: process.Ext.effective_parent"
min_stack_version = "8.7.0"
updated_date = "2024/02/06"
updated_date = "2024/05/17"

[rule]
author = ["Elastic"]
Expand Down Expand Up @@ -54,7 +54,7 @@ timestamp_override = "event.ingested"
type = "eql"

query = '''
process where event.action == "exec" and
process where event.action == "exec" and host.os.type == "macos" and
process.name : "osascript" and process.args : "-e" and process.command_line : ("*osascript*display*dialog*password*", "*osascript*display*dialog*passphrase*") and
not (process.parent.executable : "/usr/bin/sudo" and process.command_line : "*Encryption Key Escrow*") and
not (process.command_line : "*-e with timeout of 3600 seconds*" and user.id == "0" and process.parent.executable : "/bin/bash") and
Expand Down
2 changes: 1 addition & 1 deletion rules/macos/credential_access_suspicious_web_browser_sensitive_file_access.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ timestamp_override = "event.ingested"
type = "eql"

query = '''
file where event.action == "open" and process.executable != null and
file where event.action == "open" and host.os.type == "macos" and process.executable != null and
file.name : ("cookies.sqlite",
"key?.db",
"logins.json",
Expand Down
4 changes: 2 additions & 2 deletions rules/macos/defense_evasion_attempt_del_quarantine_attrib.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ integration = ["endpoint"]
maturity = "production"
min_stack_comments = "New fields added: required_fields, related_integrations, setup"
min_stack_version = "8.3.0"
updated_date = "2024/02/06"
updated_date = "2024/05/17"

[rule]
author = ["Elastic"]
Expand Down Expand Up @@ -55,7 +55,7 @@ timestamp_override = "event.ingested"
type = "eql"

query = '''
file where event.action == "extended_attributes_delete" and process.executable != null and
file where event.action == "extended_attributes_delete" and host.os.type == "macos" and process.executable != null and
(process.code_signature.trusted == false or process.code_signature.exists == false) and not
process.executable : ("/usr/bin/xattr",
"/System/*",
Expand Down
4 changes: 2 additions & 2 deletions rules/macos/initial_access_suspicious_mac_ms_office_child_process.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ integration = ["endpoint"]
maturity = "production"
min_stack_comments = "New fields added: process.Ext.effective_parent"
min_stack_version = "8.11.0"
updated_date = "2024/05/10"
updated_date = "2024/05/17"

[rule]
author = ["Elastic"]
Expand Down Expand Up @@ -52,7 +52,7 @@ timestamp_override = "event.ingested"
type = "eql"

query = '''
process where event.action == "exec" and
process where event.action == "exec" and host.os.type == "macos" and
process.parent.name: (
"Microsoft Word",
"Microsoft Outlook",
Expand Down
5 changes: 4 additions & 1 deletion tests/test_specific_rules.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,10 @@ def test_os_and_platform_in_query(self):
err_msg = f"{self.rule_str(rule)} missing required field for endpoint rule"
if "host.os.type" not in fields:
# Exception for Forwarded Events which contain Windows-only fields.
if rule.path.parent.name == "windows" and not any(field.startswith("winlog.") for field in fields):
if rule.path.parent.name == "windows":
if not any(field.startswith("winlog.") for field in fields):
self.assertIn("host.os.type", fields, err_msg)
else:
Comment on lines +54 to +57
Copy link
Contributor

@Mikaayenson Mikaayenson May 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

By the time we get here, we've already checked "host.os.type" not in fields:, so we probably dont need to assertIn again. Something like self.assertTrue(any(field.startswith("winlog.") for field in fields), "some more appropriate message about winlog and host.os.type" might be better.

Then in the else, self.fail(err_msg)
Copy link
Contributor Author

@brokensound77 brokensound77 May 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're reading it wrong. If winlog fields are present, the check is abandoned.

I initially had it collapsed more but it was much more readable this way.
Copy link
Contributor

@Mikaayenson Mikaayenson May 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean compare line 52 & 56 for example or 52 & 58.
self.assertIn("host.os.type", fields, err_msg)

# going to bypass this for now
Expand Down