Skip to content

Commit 872418f

Browse files
committed
[DOCS] Significantly improve SQL docs
Introduce SQL commands Move reserved keywords into an appendix Add section on security Introduce concepts section
1 parent 68ec958 commit 872418f

19 files changed

+506
-204
lines changed
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
include::syntax-reserved.asciidoc[]

x-pack/docs/en/sql/language/reserved.asciidoc renamed to x-pack/docs/en/sql/appendix/syntax-reserved.asciidoc

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
1-
[[sql-spec-reserved]]
2-
=== Reserved Keywords
1+
[appendix]
2+
[[sql-syntax-reserved]]
3+
= Reserved Keywords
34

45
Table with reserved keywords that need to be quoted. Also provide an example to make it more obvious.
56

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
[[sql-concepts]]
2+
== Conventions and Terminology
3+
4+
For clarity, it is important to establish the meaning behind certain words as, the same wording might convey different meanings to different readers depending on one's familiarity with SQL versus {es}.
5+
6+
NOTE: This documentation while trying to be complete, does assume the reader has _basic_ understanding of {es} and/or SQL. If that is not the case, please continue reading the documentation however take notes and pursue the topics that are unclear either through the main {es} documentation or through the plethora of SQL material available in the open (there are simply too many excellent resources here to enumerate).
7+
8+
As a general rule, {es-sql} as the name indicates provides a SQL interface to {es}. As such, it follows the SQL terminology and conventions first, whenever possible. However the backing engine itself is {es} for which {es-sql} was purposely created hence why features or concepts that are not available, or cannot be mapped correctly, in SQL appear
9+
in {es-sql}.
10+
Last but not least, {es-sql} tries to obey the https://en.wikipedia.org/wiki/Principle_of_least_astonishment[principle of least suprise], though as all things in the world, everything is relative.
11+
12+
=== Mapping concepts across SQL and {es}
13+
14+
While SQL and {es} have different terms for the way the data is organized (and different semantics), essentially their purpose is the same.
15+
16+
So let's start from the bottom; these roughly are:
17+
18+
[cols="1,1,5", options="header"]
19+
|===
20+
|SQL
21+
|{es}
22+
|Description
23+
24+
|`column`
25+
|`field`
26+
|In both cases, at the lowest level, data is stored in in _named_ entries, of a variety of <<sql-data-types, data types>>, containing _one_ value. SQL calls such an entry a _column_ while {es} a _field_.
27+
Notice that in {es} a field can contain _multiple_ values of the same type (esentially a list) while in SQL, a _column_ can contain _exactly_ one value of said type.
28+
{es-sql} will do its best to preserve the SQL semantic and, depending on the query, reject those that return fields with more than one value.
29+
30+
|`row`
31+
|`document`
32+
|++Column++s and ++field++s do _not_ exist by themselves; they are part of a `row` or a `document`. The two have slightly different semantics: a `row` tends to be _strict_ (and have more enforcements) while a `document` tends to be a bit more flexible or loose (while still having a structure).
33+
34+
|`table`
35+
|`index`
36+
|The target against which queries, whether in SQL or {es} get executed against.
37+
38+
|`schema`
39+
|_implicit_
40+
|In RDBMS, `schema` is mainly a namespace of tables and typically used as a security boundary. {es} does not provide an equivalent concept for it. However when security is enabled, {es} automatically applies the security enforcement so that a role sees only the data it is allowed to (in SQL jargon, its _schema_).
41+
42+
|`catalog` or `database`
43+
|`cluster` instance
44+
|In SQL, `catalog` or `database` are used interchangebly and represent a set of schemas that is, a number of tables.
45+
In {es} the set of indices available are grouped in a `cluster`. The semantics also differ a bit; a `database` is essentially yet another namespace (which can have some implications on the way data is stored) while an {es} `cluster` is a runtime instance, or rather a set of at least one {es} instance (typically running distributed).
46+
In practice this means that while in SQL one can potentially have multiple catalogs inside an instance, in {es} one is restricted to only _one_.
47+
48+
|`cluster`
49+
|`cluster` (federated)
50+
|Traditionally in SQL, _cluster_ refers to a single RDMBS instance which contains a number of ++catalog++s or ++database++s (see above). The same word can be reused inside {es} as well however its semantic clarified a bit.
51+
52+
While RDBMS tend to have only one running instance, on a single machine (_not_ distributed), {es} goes the opposite way and by default, is distributed and multi-instance.
53+
54+
Further more, an {es} `cluster` can be connected to other ++cluster++s in a _federated_ fashion thus `cluster` means:
55+
56+
single cluster::
57+
Multiple {es} instances typically distributed across machines, running within the same namespace.
58+
multiple clusters::
59+
Multiple clusters, each with its own namespace, connected to each other in a federated setup (see <<modules-cross-cluster-search, Cross cluster Search>>).
60+
61+
|===
62+
63+
As one can see while the mapping between the concepts are not exactly one to one and the semantics somewhat different, there are more things in common than differences. In fact, thanks to SQL declarative nature, many concepts can move across {es} transparently and the terminology of the two likely to be used interchangebly through-out the rest of the material.

x-pack/docs/en/sql/endpoints/cli.asciidoc

Lines changed: 0 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
[role="xpack"]
21
[[sql-cli]]
32
== SQL CLI
43

@@ -37,18 +36,3 @@ James S.A. Corey |Leviathan Wakes |561 |1306972800000
3736
--------------------------------------------------
3837
// TODO it'd be lovely to be able to assert that this is correct but
3938
// that is probably more work then it is worth right now.
40-
41-
[[sql-cli-permissions]]
42-
[NOTE]
43-
===============================
44-
If you are using Security you need to add a few permissions to
45-
users so they can run SQL. To run SQL using the CLI a user needs
46-
`read`, `indices:admin/get`, and `cluster:monitor/main`. The
47-
following example configures a role that can run SQL in the CLI
48-
for the `test` and `bort` indices:
49-
50-
["source","yaml",subs="attributes,callouts,macros"]
51-
--------------------------------------------------
52-
include-tagged::{sql-tests}/security/roles.yml[cli_jdbc]
53-
--------------------------------------------------
54-
===============================

x-pack/docs/en/sql/endpoints/jdbc.asciidoc

Lines changed: 6 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
[role="xpack"]
21
[[sql-jdbc]]
32
== SQL JDBC
43

@@ -36,11 +35,11 @@ from `artifacts.elastic.co/maven` by adding it to the repositories list:
3635
[float]
3736
=== Setup
3837

39-
The driver main class is `org.elasticsearch.xpack.sql.jdbc.jdbc.JdbcDriver`. Note the driver
40-
also implements the JDBC 4.0 +Service Provider+ mechanism meaning it is registerd automatically
38+
The driver main class is `org.elasticsearch.xpack.sql.jdbc.jdbc.JdbcDriver`.
39+
Note the driver implements the JDBC 4.0 +Service Provider+ mechanism meaning it is registerd automatically
4140
as long as its available in the classpath.
4241

43-
Once registered, the driver expects the following syntax as an URL:
42+
Once registered, the driver understands the following syntax as an URL:
4443

4544
["source","text",subs="attributes"]
4645
----
@@ -120,12 +119,12 @@ Query timeout (in seconds). That is the maximum amount of time waiting for a que
120119

121120
To put all of it together, the following URL:
122121

123-
["source","text",subs="attributes"]
122+
["source","text"]
124123
----
125124
jdbc:es://http://server:3456/timezone=UTC&page.size=250
126125
----
127126

128-
Opens up a {es-jdbc} connection to `server` on port `3456`, setting the JDBC timezone to `UTC` and its pagesize to `250` entries.
127+
Opens up a {es-sql} connection to `server` on port `3456`, setting the JDBC connection timezone to `UTC` and its pagesize to `250` entries.
129128

130129
=== API usage
131130

@@ -175,20 +174,4 @@ connection. For example:
175174
["source","java",subs="attributes,callouts,macros"]
176175
--------------------------------------------------
177176
include-tagged::{jdbc-tests}/SimpleExampleTestCase.java[simple_example]
178-
--------------------------------------------------
179-
180-
[[sql-jdbc-permissions]]
181-
[NOTE]
182-
===============================
183-
If you are using Security you need to add a few permissions to
184-
users so they can run SQL. To run SQL a user needs `read` and
185-
`indices:admin/get`. Some parts of the API require
186-
`cluster:monitor/main`. The following example configures a
187-
role that can run SQL in JDBC querying the `test` and `bort`
188-
indices:
189-
190-
["source","yaml",subs="attributes,callouts,macros"]
191-
--------------------------------------------------
192-
include-tagged::{sql-tests}/security/roles.yml[cli_jdbc]
193-
--------------------------------------------------
194-
===============================
177+
--------------------------------------------------

x-pack/docs/en/sql/endpoints/rest.asciidoc

Lines changed: 0 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -186,17 +186,3 @@ or fewer results though. `time_zone` is the time zone to use for date
186186
functions and date parsing. `time_zone` defaults to `utc` and can take
187187
any values documented
188188
http://www.joda.org/joda-time/apidocs/org/joda/time/DateTimeZone.html[here].
189-
190-
[[sql-rest-permissions]]
191-
[NOTE]
192-
===============================
193-
If you are using Security you need to add a few permissions to
194-
users so they can run SQL. To run SQL a user needs `read` and
195-
`indices:admin/get`. The following example configures a role
196-
that can run SQL against the `test` and `bort` indices:
197-
198-
["source","yaml",subs="attributes,callouts,macros"]
199-
--------------------------------------------------
200-
include-tagged::{sql-tests}/security/roles.yml[rest]
201-
--------------------------------------------------
202-
===============================

x-pack/docs/en/sql/endpoints/translate.asciidoc

Lines changed: 0 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
[role="xpack"]
21
[[sql-translate]]
32
== SQL Translate API
43

@@ -57,18 +56,3 @@ the normal <<search-request-body,search>> API.
5756

5857
The request body accepts all of the <<sql-rest-fields,fields>> that
5958
the <<sql-rest,SQL REST API>> accepts except `cursor`.
60-
61-
[[sql-translate-permissions]]
62-
[NOTE]
63-
===============================
64-
If you are using Security you need to add a few permissions to
65-
users so they can run translate SQL. To translate SQL a user
66-
needs `read` and `indices:admin/get`. The following example
67-
configures a role that can run SQL against the `test` and
68-
`bort` indices:
69-
70-
["source","yaml",subs="attributes,callouts,macros"]
71-
--------------------------------------------------
72-
include-tagged::{sql-tests}/security/roles.yml[rest]
73-
--------------------------------------------------
74-
===============================

x-pack/docs/en/sql/functions/index.asciidoc

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -348,6 +348,25 @@ include-tagged::{sql-specs}/datetime.csv-spec[minuteOfHour]
348348
include-tagged::{sql-specs}/datetime.csv-spec[secondOfMinute]
349349
--------------------------------------------------
350350

351+
* Extract
352+
353+
As an alternative, one can support `EXTRACT` to extract fields from datetimes.
354+
You can run any <<sql-functions-datetime,datetime function>>
355+
with `EXTRACT(<datetime_function> FROM <expression>)`. So
356+
357+
["source","sql",subs="attributes,callouts,macros"]
358+
--------------------------------------------------
359+
include-tagged::{sql-specs}/datetime.csv-spec[extractDayOfYear]
360+
--------------------------------------------------
361+
362+
is the equivalent to
363+
364+
["source","sql",subs="attributes,callouts,macros"]
365+
--------------------------------------------------
366+
include-tagged::{sql-specs}/datetime.csv-spec[dayOfYear]
367+
--------------------------------------------------
368+
369+
351370
[[sql-functions-aggregate]]
352371
=== Aggregate Functions
353372

x-pack/docs/en/sql/index.asciidoc

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,11 @@ indices and return results in tabular format.
1919
<<sql-overview, Overview>>::
2020
Overview of {es-sql} and its features.
2121
<<sql-getting-started, Getting Started>>::
22-
Start using SQL right away in {es}
22+
Start using SQL right away in {es}.
23+
<<sql-concepts, Concepts and Terminology>>::
24+
Language conventions across SQL and {es}.
25+
<<sql-security,Security>>::
26+
Securing {es-sql} and {es}.
2327
<<sql-rest,REST API>>::
2428
Accepts SQL in a JSON document, executes it, and returns the
2529
results.
@@ -34,15 +38,17 @@ indices and return results in tabular format.
3438
<<sql-functions,Functions and Operators>>::
3539
List of functions and operators supported.
3640
<<sql-spec,SQL Language>>::
37-
Overview of the {es-sql} language, such as data types, syntax and
38-
reserved keywords.
39-
41+
Overview of the {es-sql} language, such as supported data types, commands and
42+
syntax.
4043
--
4144

4245
include::overview.asciidoc[]
4346
include::getting-started.asciidoc[]
47+
include::concepts.asciidoc[]
48+
include::security.asciidoc[]
4449
include::endpoints/index.asciidoc[]
4550
include::functions/index.asciidoc[]
4651
include::language/index.asciidoc[]
52+
include::appendix/index.asciidoc[]
4753

4854
:jdbc-tests!:

x-pack/docs/en/sql/language/data-types.asciidoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[[sql-data-types]]
2-
=== Data Types
2+
== Data Types
33

44
Most of {es} <<mapping-types, data types>> are available in {es-sql}, as indicated below:
55

@@ -42,7 +42,7 @@ uses the data type _particularities_ of the former over the latter as ultimately
4242

4343
[[sql-multi-field]]
4444
[float]
45-
==== SQL and multi-fields
45+
=== SQL and multi-fields
4646

4747
A core concept in {es} is that of an `analyzed` field, that is a full-text value that is interpreted in order
4848
to be effectively indexed. These fields are of type <<text, `text`>> and are not used for sorting or aggregations as their actual value depends on the <<analyzer, `analyzer`>> used hence why {es} also offers the <<keyword, `keyword`>> type for storing the _exact_

0 commit comments

Comments
 (0)