zscaler_zia: Sanitize unwanted characters in firewall.

[kcreddy]

Proposed commit message

Sanitize unwanted characters in firewall.

Character \u0000 appears unescaped in oipcat field inside
firewall logs causing json parsing errors. ZIA documentation suggests
adding \" to Feed Escape Character to escape special characters
such as \. But this character still appear unescaped.

Adding a gsub processor to replace these unwanted characters.

Reference: https://help.zscaler.com/zia/general-guidelines-nss-feeds-and-feed-formats

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Added a new test file (.json) with unescaped \u0000 character inside oipcat. The characters are removed in the final document.

cd packages/zscaler_zia && eval "$(elastic-package stack shellinit)" && elastic-package test pipeline --generate -v --data-streams=firewall

--- Test results for package: zscaler_zia - START --- ╭─────────────┬─────────────┬───────────┬────────────────────────────────────────────────────────────┬────────┬──────────────╮ │ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │ ├─────────────┼─────────────┼───────────┼────────────────────────────────────────────────────────────┼────────┼──────────────┤ │ zscaler_zia │ firewall │ pipeline │ (ingest pipeline warnings test-firewall-http-endpoint.log) │ PASS │ 481.934375ms │ │ zscaler_zia │ firewall │ pipeline │ (ingest pipeline warnings test-firewall.log) │ PASS │ 424.45325ms │ │ zscaler_zia │ firewall │ pipeline │ (ingest pipeline warnings test-unicode.json) │ PASS │ 486.630042ms │ │ zscaler_zia │ firewall │ pipeline │ test-firewall-http-endpoint.log │ PASS │ 79.703084ms │ │ zscaler_zia │ firewall │ pipeline │ test-firewall.log │ PASS │ 74.329917ms │ │ zscaler_zia │ firewall │ pipeline │ test-unicode.json │ PASS │ 56.771ms │ ╰─────────────┴─────────────┴───────────┴────────────────────────────────────────────────────────────┴────────┴──────────────╯ --- Test results for package: zscaler_zia - END --- Done 

[elasticmachine]

🚀 Benchmarks report

Package zscaler_zia 👍(6) 💚(1) 💔(1)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
audit	4149.38	2754.82	-1394.56 (-33.61%)	💔

To see the full report comment with /test benchmark fullreport

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[chrisberkhout]

Just needs a version bump :-)

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: cd4236c

History

• 💚 Build #14937 succeeded fe7b286

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

Package zscaler_zia - 3.2.2 containing this change is available at https://epr.elastic.co/search?package=zscaler_zia