Merge branch 'main' into feature/query-iso-week-year

Author: ansoboleva

3.10/administration-cluster.md

@@ -118,9 +118,12 @@ UPDATE "123" WITH { … } IN sharded_collection
 ```
 
 If custom shard keys are used, you can no longer specify the primary key value
-for a new document, but must let the server generated one automatically. This
+for a new document, but must let the server generate one automatically. This
 restriction comes from the fact that ensuring uniqueness of the primary key
 would be very inefficient if the user could specify the document key.
+If custom shard keys are used, trying to store documents with the primary key value
+(`_key` attribute) set will result in a runtime error ("must not specify _key
+for this collection").
 
 Unique indexes on sharded collections are only allowed if the fields used to 
 determine the shard key are also included in the list of attribute paths for the index:

3.10/indexing-index-basics.md

@@ -23,8 +23,17 @@ by ArangoDB, without the user being required to create extra indexes for them.
 `_id` and `_key` are covered by a collection's primary key, and `_from` and `_to`
 are covered by an edge collection's edge index automatically.
 
-Using the system attribute `_id` in user-defined indexes is not possible, but 
-indexing `_key`, `_rev`, `_from`, and `_to` is.
+You cannot use the `_id` system attribute in user-defined indexes, but indexing
+`_key`, `_rev`, `_from`, and `_to` is possible.
+
+You cannot index fields that contain `.` in their attribute names because dots
+are interpreted as paths of nested attributes. For example, `fields: ["foo.bar"]`
+indexes the value of the nested attribute `bar` under the top-level attribute
+`foo` (`{ "foo": { "bar": "value" } }`) and not a top-level attribute
+`foo.bar` (`{ "foo.bar": "value" }`).
+
+You can also not index fields whose names either start or end with `:`, for
+example, `fields: ["foo:"]`. This notation is reserved for internal use.
 
 Creating new indexes is by default done under an exclusive collection lock. The collection is not
 available while the index is being created. This "foreground" index creation can be undesirable, 

3.11/administration-cluster.md

@@ -118,9 +118,12 @@ UPDATE "123" WITH { … } IN sharded_collection
 ```
 
 If custom shard keys are used, you can no longer specify the primary key value
-for a new document, but must let the server generated one automatically. This
+for a new document, but must let the server generate one automatically. This
 restriction comes from the fact that ensuring uniqueness of the primary key
 would be very inefficient if the user could specify the document key.
+If custom shard keys are used, trying to store documents with the primary key value
+(`_key` attribute) set will result in a runtime error ("must not specify _key
+for this collection").
 
 Unique indexes on sharded collections are only allowed if the fields used to 
 determine the shard key are also included in the list of attribute paths for the index:

3.11/indexing-index-basics.md

@@ -23,8 +23,17 @@ by ArangoDB, without the user being required to create extra indexes for them.
 `_id` and `_key` are covered by a collection's primary key, and `_from` and `_to`
 are covered by an edge collection's edge index automatically.
 
-Using the system attribute `_id` in user-defined indexes is not possible, but 
-indexing `_key`, `_rev`, `_from`, and `_to` is.
+You cannot use the `_id` system attribute in user-defined indexes, but indexing
+`_key`, `_rev`, `_from`, and `_to` is possible.
+
+You cannot index fields that contain `.` in their attribute names because dots
+are interpreted as paths of nested attributes. For example, `fields: ["foo.bar"]`
+indexes the value of the nested attribute `bar` under the top-level attribute
+`foo` (`{ "foo": { "bar": "value" } }`) and not a top-level attribute
+`foo.bar` (`{ "foo.bar": "value" }`).
+
+You can also not index fields whose names either start or end with `:`, for
+example, `fields: ["foo:"]`. This notation is reserved for internal use.
 
 Creating new indexes is by default done under an exclusive collection lock. The collection is not
 available while the index is being created. This "foreground" index creation can be undesirable, 

3.11/release-notes-api-changes311.md

@@ -29,7 +29,14 @@ integrations for ArangoDB 3.11.
 
 ### Endpoints augmented
 
+#### Restriction of indexable fields
 
+It is now forbidden to create indexes that cover fields whose attribute names
+start or end with `:` , for example, `fields: ["value:"]`. This notation is
+reserved for internal use.
+
+Existing indexes are not affected but you cannot create new indexes with a
+preceding or trailing colon using the `POST /_api/index` endpoint.
 
 ### Endpoints moved
 

3.11/release-notes-upgrading-changes311.md

@@ -10,3 +10,12 @@ upgrading to ArangoDB 3.11, and adjust any client programs if necessary.
 
 The following incompatible changes have been made in ArangoDB 3.11:
 
+Restriction of indexable fields
+-------------------------------
+
+It is now forbidden to create indexes that cover fields whose attribute names
+start or end with `:` , for example, `fields: ["value:"]`. This notation is
+reserved for internal use.
+
+Existing indexes are not affected but you cannot create new indexes with a
+preceding or trailing colon.

3.8/administration-cluster.md

@@ -118,9 +118,12 @@ UPDATE "123" WITH { … } IN sharded_collection
 ```
 
 If custom shard keys are used, you can no longer specify the primary key value
-for a new document, but must let the server generated one automatically. This
+for a new document, but must let the server generate one automatically. This
 restriction comes from the fact that ensuring uniqueness of the primary key
 would be very inefficient if the user could specify the document key.
+If custom shard keys are used, trying to store documents with the primary key value
+(`_key` attribute) set will result in a runtime error ("must not specify _key
+for this collection").
 
 Unique indexes (hash, skiplist, persistent) on sharded collections are
 only allowed if the fields used to determine the shard key are also

3.8/indexing-index-basics.md

@@ -23,8 +23,14 @@ by ArangoDB, without the user being required to create extra indexes for them.
 `_id` and `_key` are covered by a collection's primary key, and `_from` and `_to`
 are covered by an edge collection's edge index automatically.
 
-Using the system attribute `_id` in user-defined indexes is not possible, but 
-indexing `_key`, `_rev`, `_from`, and `_to` is.
+You cannot use the `_id` system attribute in user-defined indexes, but indexing
+`_key`, `_rev`, `_from`, and `_to` is possible.
+
+You cannot index fields that contain `.` in their attribute names because dots
+are interpreted as paths of nested attributes. For example, `fields: ["foo.bar"]`
+indexes the value of the nested attribute `bar` under the top-level attribute
+`foo` (`{ "foo": { "bar": "value" } }`) and not a top-level attribute
+`foo.bar` (`{ "foo.bar": "value" }`).
 
 Creating new indexes is by default done under an exclusive collection lock. The collection is not
 available while the index is being created. This "foreground" index creation can be undesirable, 

3.8/release-notes-known-issues38.md

@@ -32,6 +32,7 @@ AQL
 | **Date Added:** 2018-09-05 <br> **Component:** AQL <br> **Deployment Mode:** Cluster <br> **Description:** In a very uncommon edge case there is an issue with an optimization rule in the cluster. If you are running a cluster and use a custom shard key on a collection (default is `_key`) **and** you provide a wrong shard key in a modifying query (`UPDATE`, `REPLACE`, `DELETE`) **and** the wrong shard key is on a different shard than the correct one, a `DOCUMENT NOT FOUND` error is returned instead of a modification (example query: `UPDATE { _key: "123", shardKey: "wrongKey"} WITH { foo: "bar" } IN mycollection`). Note that the modification always happens if the rule is switched off, so the suggested workaround is to [deactivate the optimizing rule](aql/execution-and-performance-optimizer.html#turning-specific-optimizer-rules-off) `restrict-to-single-shard`. <br> **Affected Versions:** 3.4.x, 3.5.x, 3.6.x, 3.7.x, 3.8.x <br> **Fixed in Versions:** - <br> **Reference:** [arangodb/arangodb#6399](https://github.com/arangodb/arangodb/issues/6399){:target="_blank"} |
 | **Date Added:** 2021-04-19 <br> **Component:** AQL <br> **Deployment Mode:** All <br> **Description:** User-defined variable names in AQL cannot start with an underscore, in contrast to what the AQL documentation claimed. <br> **Affected Versions:** 3.5.x, 3.6.x, 3.7.x, 3.8.x <br> **Fixed in Versions:** 3.9.0 <br> **Reference:** [arangodb/arangodb#13513](https://github.com/arangodb/arangodb/issues/13513){:target="_blank"} |
 | **Date Added:** 2021-05-25 <br> **Component:** AQL <br> **Deployment Mode:** All <br> **Description:** Global and per-query memory limits are not enforced for `SHORTEST_PATH` and `K_SHORTEST_PATHS` AQL graph traversals. Such queries may thus cause out-of-memory issues as they will not be killed despite exceeding the configured `--query.memory-limit` or Cursor API `memoryLimit`. <br> **Affected Versions:** 3.5.x, 3.6.x, 3.7.x, 3.8.x <br> **Fixed in Versions:** 3.8.1 <br> **Reference:** [BTS-411](https://arangodb.atlassian.net/browse/BTS-411){:target="_blank"} (internal) |
+| **Date Added:** 2022-08-12 <br> **Component:** AQL <br> **Deployment Mode:** Cluster <br> **Description:** Since ArangoDB 3.8, there was a loophole for creating duplicate keys in the same collection. This was only possible in cluster and needed an AQL query that copied over data from a collection (source) into another (target). The target collection must have more than one shard and must use a custom shard key. Inserting documents into the target collection must have happened via an AQL query like `FOR doc IN source INSERT doc INTO target`. In this particular situation, the document keys (`_key` attribute) from the source collection were used as-is for insertion into the target collection. However, as the target collection is not sharded by `_key` and uses a custom shard key, it is actually not allowed to specify user-defined values for `_key`. Versions before 3.8 failed with the error "must not specifiy _key for this collection" when running such an AQL query, but versions between 3.8.0 and 3.8.7 did not. <br> **Affected Versions:** 3.8.0 - 3.8.7 <br> **Fixed in Versions:** 3.8.8 <br> **Reference:** [arangodb/arangodb#16764](https://github.com/arangodb/arangodb/issues/16764){:target="_blank"} |
 
 Upgrading
 ---------

3.9/administration-cluster.md

@@ -118,9 +118,12 @@ UPDATE "123" WITH { … } IN sharded_collection
 ```
 
 If custom shard keys are used, you can no longer specify the primary key value
-for a new document, but must let the server generated one automatically. This
+for a new document, but must let the server generate one automatically. This
 restriction comes from the fact that ensuring uniqueness of the primary key
 would be very inefficient if the user could specify the document key.
+If custom shard keys are used, trying to store documents with the primary key value
+(`_key` attribute) set will result in a runtime error ("must not specify _key
+for this collection").
 
 Unique indexes on sharded collections are only allowed if the fields used to 
 determine the shard key are also included in the list of attribute paths for the index: