All Products
Search
Document Center

Simple Log Service:Native plug-in: Data parsing in delimiter mode

Last Updated:Aug 22, 2025

The Logtail plug-in for delimiter mode parsing uses a separator to structure log content. This plug-in parses a log into multiple key-value pairs.

Entry point

If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Overview.

Configuration description

Parameter

Description

Original Field

The field that stores the original log content before parsing. The default value is content.

Separator

Select a separator based on your log content, such as a vertical bar (|).

Note

To specify a Non-printable Character as the separator, find its hexadecimal value in the ASCII table and enter it in the 0x<hexadecimal_value> format. For example, a character with an ASCII value of 1 is represented as 0x01.

Quote

If a log field contains the separator, you can specify a quote to enclose the field. Simple Log Service then parses the content enclosed in a pair of quotes as a complete field. You can select a quote based on your log format.

Note

To specify a Non-printable Character as the quote, find its hexadecimal value in the ASCII table and enter it in the 0x<hexadecimal_value> format. For example, a character with an ASCII value of 1 is represented as 0x01.

Extracted Field

  • If you provide a sample log, Simple Log Service automatically extracts the log content based on the sample and the specified separator. The extracted content is defined as values, and you must specify a key for each value.

  • If you do not provide a sample log, no values are listed. You must enter the keys based on your logs and the separator.

A key can contain only letters, digits, and underscores (_) and must start with a letter or an underscore (_). A key can be up to 128 bytes in length.

Allow Missing Field

Select Allow Missing Field to upload a log to Simple Log Service if the number of extracted values is less than the number of keys.

For example, a log is 11|22|33|44, the separator is a vertical bar (|), and the keys are A, B, C, D, and E.

  • If you select Allow Missing Field, the value of the E field is empty, and the log is uploaded to Simple Log Service.

  • If you do not select Allow Missing Field, the log is discarded.

    Note

    Logtail for Linux 1.0.28 or later and Logtail for Windows 1.0.28.0 or later support the Allow Missing Field parameter for delimiter mode.

Processing Method For Extra Fields

Specifies the method for processing extra values when the number of extracted values is greater than the number of keys.

  • Expand: Retains the extra values and adds them to fields in the __column$i__ format. $i specifies the ordinal number of the extra field, starting from 0. For example, __column0__ and __column1__.

  • Retain: Retains the extra values and adds them to a field named __column0__.

  • Discard: Discards the extra values.

Retain Original Field If Parsing Fails

Select Retain Original Field If Parsing Fails to preserve the original field if parsing is unsuccessful.

Retain Original Field If Parsing Succeeds

To retain the original field after successful parsing, select Retain Original Field If Parsing Succeeds.

Renamed Original Field

After you select Retain Original Field If Parsing Fails or Retain Original Field If Parsing Succeeds, you can rename the original field that stores the original log content.

Appendix

The Logtail plug-in for delimiter mode parsing supports single-character and multi-character separators.

Single character

The following is an example of a single-character log.

05/May/2022:13:30:28,10.10.*.*,"POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=****************&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=******************************** HTTP/1.1",200,18204,aliyun-sdk-java 05/May/2022:13:31:23,10.10.*.*,"POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=****************&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=******************************** HTTP/1.1",401,23472,aliyun-sdk-java

In single-character mode, you must specify a separator. You can also specify a quote.

  • Separator: Splits logs using a single-character separator, such as a tab character (\t), vertical bar (|), space, comma (,), semicolon (;), or a non-printable character. A double quotation mark (") cannot be used as a separator.

    A double quotation mark (") can be used as a quote. It can appear at a field border or within the field content. If a double quotation mark (") appears in the content, it must be escaped as "". Simple Log Service automatically reverts "" to " during parsing. For example, assume that the separator is a comma (,) and the quote is a double quotation mark ("). If a log field contains both a double quotation mark (") and a comma (,), you must enclose the field in quotes and escape the double quotation mark within the field as "". The log 1999,Chevy,"Venture ""Extended Edition, Very Large""","",5000.00 is parsed into five fields: 1999, Chevy, Venture "Extended Edition, Very Large", an empty field, and 5000.00.

  • Quote: If a log field contains the separator, you can specify a quote to enclose the field. Simple Log Service then parses the content enclosed in a pair of quotes as a complete field.

    You can set the quote to a single character, such as a tab character (\t), vertical bar (|), space, comma (,), semicolon (;), or a non-printable character.

    For example, if the separator is a comma (,) and the quote is a double quotation mark ("), the log 1997,Ford,E350,"ac, abs, moon",3000.00 is parsed into five fields: 1997, Ford, E350, ac, abs, moon, and 3000.00.

Multi-character

The following code provides examples of logs that can be parsed using a multi-character separator.

05/May/2022:13:30:28&&10.200.**.**&&POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=****************&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=******************************** HTTP/1.1&&200&&18204&&aliyun-sdk-java 05/May/2022:13:31:23&&10.200.**.**&&POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=****************&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=******************************** HTTP/1.1&&401&&23472&&aliyun-sdk-java

In multi-character mode, the separator contains two or three characters, such as ||, &&&, or ^_^. Logs are parsed based on separator matching. You do not need to use quotes to enclose log fields.

Important

Ensure that the log field content does not contain a full match for the separator. Otherwise, the field may be split incorrectly.

For example, if the separator is &&, the log 1997&&Ford&&E350&&ac&abs&moon&&3000.00 is parsed into five fields: 1997, Ford, E350, ac&abs&moon, and 3000.00.

References