I have a Java application in a pod that is spitting out logs straight to stdout. These logs are fetched by the log-operator which runs a combination of fluent bit 3.1.8 and fluentd 1.16.1. The operator is kube-logging v 4.10 (https://kube-logging.dev/search/).
If I run an output crd to splunk all log entries come in single lines. also stack traces.
if I add the following filters
filters: - parser: key_name: event parse: type: json remove_key_name_field: true reserve_data: true - grep: exclude: - key: message pattern: ^$ - concat: flush_interval: 30 key: message multiline_start_regexp: /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}/ stream_identity_key: kubernetes.pod_name first one parses the event to an actual json blob from this
{ "host": "rancher-logging-root-fluentd-0", "time": "1760103161.9110134", "event": "{\"time\":\"2025-10-10T15:32:41.911013406+02:00\",\"stream\":\"stdout\",\"message\":\"2025-09-30T14:58:00.000+0000 ERROR 1 --- [ task-01] Exception ..." } { "host": "rancher-logging-root-fluentd-0", "time": "1760103161.9110134", "event": "{\"time\":\"2025-10-10T15:32:41.911013406+02:00\",\"stream\":\"stdout\",\"message\":\" at ...." } Parsing the event data then the exclude tag strips all empty messages. then the concat filter finds the start dates and concats these.
All goes well until there are no messages for the timeout period (flush_interval) if there are no messages; the last log entry disappears.
I really need that last log entry because the Java app processes things in bulk then pauses for a bit so every bulk action we lose the last line.
I've tried playing with the flush intervals of both flow and output crd. Changed the regexpr also tried to fiddle with the partial keys but then everything stops being passed along.
I expect that if the flush interval is reached that all things remaining in the buffer get sent out to splunk anyway even if there is no new start of multiline.
