You can use the Logtail time parsing plugin to parse the time field from a log. The plugin sets the parsed result as the value of the __time__
field in the log.
Entry point
If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Overview.
Configuration description
Parameter | Description |
Original Field | The original field that stores the log content before parsing. The default value is content. |
Time Format | The time format. Set a time format based on the time in a log. For example, if the time in a log is 10/Sep/2023:12:36:49, the time conversion format is %d/%b/%Y:%H:%M:%S. For more information, see Common log time formats. |
Time Zone | The time zone of the log time field. If you do not configure this parameter, the time zone of the environment where the Logtail process resides is used by default. |
Common log time formats
The following table describes the time formats that Logtail supports.
By default, timestamps in Simple Log Service are accurate to the second. You only need to configure a time format that is also accurate to the second.
If the time field in a raw log has a higher precision, such as millisecond, microsecond, or nanosecond, and you want to retain this precision in Simple Log Service, see Parse nanosecond-precision timestamps from raw logs when you use Logtail to collect logs.
On a Linux server, Logtail supports all time formats that the strftime function provides.
Time format | Description | Example |
%a | The abbreviated name of the day of the week. | Fri |
%A | The full name of the day of the week. | Friday |
%b | The abbreviated name of the month. | Jan |
%B | The full name of the month. | January |
%d | The day of the month in decimal format. Valid values: 01 to 31. | 07, 31 |
%f | The fractional part of a second, such as millisecond, microsecond, or nanosecond. | 123 |
%h | The abbreviated name of the month. This is equivalent to %b. | Jan |
%H | The hour in 24-hour format. | 22 |
%I | The hour in 12-hour format. | 11 |
%m | The month in decimal format. Valid values: 01 to 12. | 08 |
%M | The minute in decimal format. Valid values: 00 to 59. | 59 |
%n | A line feed. | line feed |
%p | AM or PM. | AM, PM |
%r | The time in 12-hour format. This is equivalent to %I:%M:%S %p. | 11:59:59 AM |
%R | The time in hours and minutes. This is equivalent to %H:%M. | 23:59 |
%S | The second in decimal format. Valid values: 00 to 59. | 59 |
%t | A tab character. | None |
%y | The two-digit year in decimal format. Valid values: 00 to 99. | 04, 98 |
%Y | The four-digit year in decimal format. | 2004, 1998 |
%C | The century in decimal format. Valid values: 00 to 99. | 16 |
%e | The day of the month in decimal format. Valid values: 1 to 31. A single-digit day is preceded by a space. | 7, 31 |
%j | The day of the year in decimal format. Valid values: 001 to 366. | 365 |
%u | The day of the week in decimal format. Valid values: 1 to 7. The value 1 indicates Monday. | 2 |
%U | The week number of the year. Sunday is the first day of the week. Valid values: 00 to 53. | 23 |
%V | The week number of the year. Monday is the first day of the week. Valid values: 01 to 53. If the first week of a year has four or more days in January, it is considered week 1. Otherwise, the next week is considered week 1. | 24 |
%w | The day of the week in decimal format. Valid values: 0 to 6. The value 0 indicates Sunday. | 5 |
%W | The week number of the year. Monday is the first day of the week. Valid values: 00 to 53. | 23 |
%c | The standard date and time. | Tue Nov 20 14:12:58 2020 |
%x | The standard date without the time. | Tue Nov 20 2020 |
%X | The standard time without the date. | 11:59:59 |
%s | The UNIX timestamp. | 1476187251 |
Examples
The following table describes common time standards and time expressions, and provides examples.
Example | Time expression | Time standard |
2017-12-11 15:05:07 | %Y-%m-%d %H:%M:%S | Custom |
[2017-12-11 15:05:07.012] | [%Y-%m-%d %H:%M:%S | Custom |
2017-12-11 15:05:07.123 | %Y-%m-%d %H:%M:%S.%f | Custom |
02 Jan 06 15:04 MST | %d %b %y %H:%M | RFC822 |
02 Jan 06 15:04 -0700 | %d %b %y %H:%M | RFC822Z |
Monday, 02-Jan-06 15:04:05 MST | %A, %d-%b-%y %H:%M:%S | RFC850 |
Mon, 02 Jan 2006 15:04:05 MST | %A, %d %b %Y %H:%M:%S | RFC1123 |
2006-01-02T15:04:05Z07:00 | %Y-%m-%dT%H:%M:%S | RFC3339 |
2006-01-02T15:04:05.999999999Z07:00 | %Y-%m-%dT%H:%M:%S | RFC3339Nano |
1637843406 | %s | Custom |
1637843406123 | %s | Custom (Simple Log Service processes the time with second-level precision) |
References
Configure a Logtail pipeline using API operations:
GetLogtailPipelineConfig - Get a Logtail pipeline configuration
ListLogtailPipelineConfig - List Logtail pipeline configurations
CreateLogtailPipelineConfig - Create a Logtail pipeline configuration
DeleteLogtailPipelineConfig - Delete a Logtail pipeline configuration
UpdateLogtailPipelineConfig - Update a Logtail pipeline configuration
Configure a processor plugin in the console:
Collect container logs from a Kubernetes cluster using a CRD (stdout/file)