Skip to content

Build EQL Query Parser and planner on top of reusable xQL components #49997

@costin

Description

@costin

Meta issue describing the rough steps for reusing the SQL
common core inside EQL.

Parsing & AST

  • Make EQL parser take advantage of the new Node and Source packages
  • Create EQL specific exceptions based on the common ones to properly catch
  • Hook parser to the Expression tree and map WHERE query to a Filter

Session & Analysis

  • Create dedicated Analyzer for EQL using a subset of rules from SQL. In particular those around CTE and missing references should not be needed. It's worth starting with a small set and expand later if needed.
  • Pay special attention to the index resolution. As oppose to SQL, the index is resolved before the query so the index information EsIndex is already available.
    In practice this translates to a simpler PlanExecutor, that is the entry point in converting the query into an actual plan.
  • Start defining a Session and Configuration class - while it shares a lot of things, for the most part this can be EQL specific since the risk of duplication is low (quantity wise as well).
  • Add dedicated Verifier - just like with the Analyzer this will likely have a smaller set of rule. As a tangent, it is a good opportunity to potentially add some generic ones such as verifying that the input of a node is at most a subset of its child.
    This can act as a good deterrent in catching potential bugs that work across nodes.

Optimizer

  • Add EQL Optimizer - for the most part it will contain the common rules around Expression as most LogicalPlan ones are not useful at this stage.
  • It might be worth adding a basic second Verifier to check that the Optimizer hasn't introduced any bugs

Query translator

  • Move querying AST in QL
  • Extract basic rules for translating search queries (not aggregations) to QL

Query Planner

  • As the first release handles only basic queries, a trivial/basic query planner will be used (there's no planning required).

Query Executor

  • Create result representation across different response types
  • Reuse SearchHitExtractor from QL for parsing result set
  • Introduce search_after querying. This will be useful in the future for sequence/joins. Investigate whether serialization is required for the first stage.

Relates to #49773

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions