Citrix ADC Integration Grok Fix

[Linu-Elias]

Proposed commit message

SDH issue on Citrix-ADC logs not being ingested properly.
In citrix_adc.log datastream, the grok pattern has timezone which is not optional hence below log formats are failing

<123> 10/08/2024:09:38:41 SYSLOGHOST 0-PPE-1 : default TCP CONN_TERMINATE 6715345 0 : Source 127.1.2.1:80 - Destination 127.1.1.2:20714 - Start Time 10/08/2024:09:37:54 - End Time 10/08/2024:09:38:41 - Total_bytes_send 1 - Total_bytes_recv 1 \n

Fix -

• Created a .json test file to incorporate the customer’s log format into the JSON file.
• Modified the grok pattern by updating %{WORD: event.timezone} optional in native.yaml file

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

• [ ]

How to test this PR locally

elastic-package test pipeline -v -g

--- Test results for package: citrix_adc - START --- ╭────────────┬─────────────┬───────────┬───────────────────────────────────────────────────────┬────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├────────────┼─────────────┼───────────┼───────────────────────────────────────────────────────┼────────┼──────────────┤ │ citrix_adc │ interface │ pipeline │ (ingest pipeline warnings test-interface-metrics.log) │ PASS │ 367.483032ms │ │ citrix_adc │ interface │ pipeline │ test-interface-metrics.log │ PASS │ 54.415927ms │ │ citrix_adc │ lbvserver │ pipeline │ (ingest pipeline warnings test-lbvserver-metrics.log) │ PASS │ 298.890111ms │ │ citrix_adc │ lbvserver │ pipeline │ test-lbvserver-metrics.log │ PASS │ 77.981971ms │ │ citrix_adc │ log │ pipeline │ (ingest pipeline warnings test-citrix-native.json) │ PASS │ 312.586091ms │ │ citrix_adc │ log │ pipeline │ (ingest pipeline warnings test-citrix-waf-cef.log) │ PASS │ 289.473836ms │ │ citrix_adc │ log │ pipeline │ (ingest pipeline warnings test-citrix-waf-native.log) │ PASS │ 348.825644ms │ │ citrix_adc │ log │ pipeline │ test-citrix-native.json │ PASS │ 85.157763ms │ │ citrix_adc │ log │ pipeline │ test-citrix-waf-cef.log │ PASS │ 210.569743ms │ │ citrix_adc │ log │ pipeline │ test-citrix-waf-native.log │ PASS │ 939.570883ms │ │ citrix_adc │ service │ pipeline │ (ingest pipeline warnings test-service-metrics.log) │ PASS │ 438.963898ms │ │ citrix_adc │ service │ pipeline │ test-service-metrics.log │ PASS │ 61.287781ms │ │ citrix_adc │ system │ pipeline │ (ingest pipeline warnings test-system-metrics.log) │ PASS │ 393.483888ms │ │ citrix_adc │ system │ pipeline │ test-system-metrics.log │ PASS │ 68.261458ms │ │ citrix_adc │ vpn │ pipeline │ (ingest pipeline warnings test-vpn-metrics.log) │ PASS │ 494.980998ms │ │ citrix_adc │ vpn │ pipeline │ test-vpn-metrics.log │ PASS │ 49.469335ms │ ╰────────────┴─────────────┴───────────┴───────────────────────────────────────────────────────┴────────┴──────────────╯ --- Test results for package: citrix_adc - END --- Done 

Related issues

Closes #10782

Screenshots

[elasticmachine]

🚀 Benchmarks report

Package citrix_adc 👍(3) 💚(1) 💔(2)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
interface	5780.35	3861	-1919.35 (-33.2%)	💔
system	6289.31	5319.15	-970.16 (-15.43%)	💔

To see the full report comment with /test benchmark fullreport

[ishleenk17]

Suggested change

	- description: Timezone field has been updated to optional in the native header Grok pattern.
	- description: Timezone field made optional for the citrix_adc log messages

[ishleenk17]

@Linu-Elias : There are couple more ingest pipeline as well in citrix_adc other than native.yml
Do we need to take care of timezone there as well ?

Tagging @elastic/security-external-integrations for review as well since they implemented the log part of the Citrix ADC.

[ShourieG]

LGTM

[ishleenk17]

@Linu-Elias Please check if we need the change on other ingest pipelines, otherwise looks good

[efd6]

Should this have a terminal \n literal? That doesn't look right.

[Linu-Elias]

@efd6 This is in the format of the sample logs that the customer provided in the SDH isuue

[andrewkroh]

There's a difference between this line and what are in the event.original samples from the user. The user's samples have an actual newline at the end of the event.original (encoded to "\n" in JSON). What's in in this .log file is a slash followed by an "n" to be literal. You can see that in the expected.json file where it contains "Total_bytes_recv 1 \\n" instead of "Total_bytes_recv 1 \n".

To match a test case to the user's samples, I think you will need to use the a .json test file instead of .log so that you can encode the newline into the message.

[Linu-Elias]

@andrewkroh Thanks, I have now used a .json file intead of .log. In this context, the expected.json file contains "Total_bytes_recv 1 \n" intead of "Total_bytes_recv 1 \\n".

[Linu-Elias]

@Linu-Elias Please check if we need the change on other ingest pipelines, otherwise looks good

@ishleenk17 I don't think we need to change other ingest pipelines since they all are passing with this change.

The requested change has been made. See #10778 (comment)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 07c4792

History

• 💚 Build #14818 succeeded e2ce0a2
• 💚 Build #14639 succeeded 705b111
• 💚 Build #14623 succeeded 79a95d8

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

Package citrix_adc - 1.7.1 containing this change is available at https://epr.elastic.co/search?package=citrix_adc