Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
a307db2
ES|QL support: ESQL executor implementation, response type to accept …
mashhurs Apr 8, 2025
9c35f22
Merge with upstream, warn if query doesn't include METADATA which DSL…
mashhurs Apr 9, 2025
6f99055
Run unit tests with the LS version which actually supports the ES|QL.
mashhurs Apr 10, 2025
086a592
Add query type to the agent. DRY of supported ES/LS versions.
mashhurs Apr 10, 2025
e30e0f9
Remove query type from user-agent since it is useless, put back accid…
mashhurs Apr 10, 2025
7746c14
Initial docs added for ES|QL.
mashhurs Apr 10, 2025
76303d8
Update query to include condition with string.
mashhurs Apr 11, 2025
1fb29f7
Tested escaped chars cases, uses orignal query.
mashhurs Apr 12, 2025
5d47f2f
Integration tests added.
mashhurs Apr 14, 2025
c291e24
Skip the ESQL test if LS with the ES client which doesn't support ESQ…
mashhurs Apr 14, 2025
22e72e9
Add comments on response type and query params about ES|QL acceptance…
mashhurs Apr 14, 2025
af6e24a
Update spec/inputs/integration/elasticsearch_esql_spec.rb
mashhurs Apr 21, 2025
4ce6fa4
Integration test skip condition correction.
mashhurs Apr 21, 2025
4ed69ff
Introduce query_params option to accept drop_null_columns, set defaul…
mashhurs Apr 24, 2025
0725f98
Fix the failed integration test.
mashhurs Apr 25, 2025
cfb36f3
Request dropping null columns and filter out null values. Consider se…
mashhurs May 1, 2025
a92a71e
Apply suggestions from code review
mashhurs May 7, 2025
d4f559d
Apply code review suggestions: to use decorator as a proc call, doc s…
mashhurs May 7, 2025
65eb675
Rename warning msg field name to avoid conflicts. Generate a target a…
mashhurs May 8, 2025
789f467
Ignore sub-fields with warninigs and keep only parent.
mashhurs May 8, 2025
fefe6a0
Introduce at high level which other params such as , etc.. follow it…
mashhurs May 26, 2025
e108c87
Add a tech preview fior the ESQL.
mashhurs May 28, 2025
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Apply suggestions from code review
Co-authored-by: Rye Biesemeyer <yaauie@users.noreply.github.com> Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
  • Loading branch information
3 people authored May 7, 2025
commit a92a71ef94c23f6717c9b7f1a7bc86fe13d6b10d
16 changes: 6 additions & 10 deletions docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -234,18 +234,12 @@ The next scheduled run:
==== ES|QL support
{es} Query Language (ES|QL) provides a SQL-like interface for querying your {es} data.

To utilize the ES|QL feature with this plugin, the following version requirements must be met:
[cols="1,2",options="header"]
|===
|Component |Minimum version
|{es} |8.11.0 or newer
|{ls} |8.17.4 or newer
|This plugin |4.23.0+ (4.x series) or 5.2.0+ (5.x series)
To use {esql}, this plugin needs to be installed in {ls} 8.17.4 or newer, and must be connected to {es} 8.11 or newer.
|===

To configure ES|QL query in the plugin, set the `response_type` to `esql` and provide your ES|QL query in the `query` parameter.
To configure {esql} query in the plugin, set the `response_type` to `esql` and provide your {esql} query in the `query` parameter.

IMPORTANT: We recommend understanding https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-limitations.html[ES|QL current limitations] before using it in production environments.
IMPORTANT: {esql} is evolving and may still have limitations with regard to result size or supported field types. We recommend understanding https://www.elastic.co/guide/en/elasticsearch/reference/current/esql-limitations.html[ES|QL current limitations] before using it in production environments.

The following is a basic scheduled ES|QL query that runs hourly:
[source, ruby]
Expand Down Expand Up @@ -322,6 +316,8 @@ To illustrate the situation with example, assuming your mapping has a time `time
}

The ES|QL result will contain all three fields but the plugin cannot map them into {ls} event.

This a common occurence if your template or mapping follows the pattern of always indexing strings as "text" (`field`) + " keyword" (`field.keyword`) multi-field. In this case it's recommended to do `KEEP field` if the string is identical and there is only one subfield as the engine will optimize and retrieve the keyword, otherwise you can do `KEEP field.keyword | RENAME field.keyword as field` .
To avoid this, you can use the `RENAME` keyword to rename the `time` parent field to get all three fields with unique fields.
[source, ruby]
...
Expand Down Expand Up @@ -622,7 +618,7 @@ contents of the `aggregations` object of the query's response. In this case the
0 regardless of the default or user-defined value set in this plugin.

When using the `esql` setting, the query must be a valid ES|QL string.
When this setting is active, `target`, `size`, `slices` and `search_api` parameters are ignored.
When this setting is active, `index`, `size`, `slices`, `search_api`, `docinfo`, `docinfo_target` and `docinfo_fields` parameters are not allowed.

[id="plugins-{type}s-{plugin}-request_timeout_seconds"]
===== `request_timeout_seconds`
Expand Down
Loading