Currently the parsing of system messages for Juniper SRX is quite new, however there is a few main issues that has to be resolved.

1. It is possible for the initial grok created for traffic structured in default.yml to hit the system-structured data as well, this causes issues. The grok should be rewritten into 3 components instead of the current 2:

• One grok to handle structured traffic data, should have a custom pattern that lists all the possible juniper process names (the field used to decide on which pipeline to send the data).
• One grok to handle structured system data, should re-use the custom pattern as a negative lookahead, to support mapping the process name but ignore any of the traffic ones.
• One grok to handle unstructured system data, this exists today, but some has too many optional values, structured and unstructured should be split into multiple.

2. The grok processor in system.yml has issues with certain data formats, here as well we should have more explicit grok patterns for structured and unstructured data, as unstructured data is still hitting the pattern for structured.

• Create one pattern for structured (they are always inside square brackets).
• One pattern for unstructured.
• One catchall pattern that we have today.

3. Currently the _temp_.to_be_parsed field is not actually parsed anywhere, the TAG field at the start of the grok also support only values without spaces, so things like "IKE negotiation failed with error" does not work.

Dissect for this is not working because of missing tag, its in the test data:
<27>1 2023-05-04T15:19:33.984+10:00 AB1234-A-AB-AB01C-ABC kmd 9159 asd2 asd IKE negotiation failed with error: Timed out. IKE Version: 1, VPN: IPSEC-AAAAA-AAA1-PROD-VPN Gateway: IKE-AAAAA-AAA1-GW, Local: 89.160.20.112/500, Remote: 67.43.156.1/500, Local IKE-ID: Not-Available, Remote IKE-ID: Not-Available, VR-ID: 5: Role: Initiator

Other sample data that hits traffic-structured on default.yml rather than the system pattern:
<37>1 2023-05-10T00:10:24.232+10:00 AB1234-A-AB-AB01C-ABC snmpd 8959 SNMPD_AUTH_FAILURE [junos@1111.1.1.1.1.111 function-name="nsa_log_community" message="unauthorized SNMP community" source-address="216.160.83.56" destination-address="89.160.20.128" index1="j5Cx6eSkKF7A"]