Skip to content

Remove support for unrecognized keywords #1512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gregsdennis merged 2 commits into from
Jun 21, 2024
Merged

Remove support for unrecognized keywords #1512

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4cfe452
remove support for unrecognized keywords
gregsdennis May 23, 2024
e3a5aa2
update unrecognized section to include all keywords which could be kn…
gregsdennis Jun 19, 2024
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
remove support for unrecognized keywords
  • Loading branch information
@gregsdennis
gregsdennis committed Jun 21, 2024
commit 4cfe452450dbeb485d68cbb607eabeabacb44aac
42 changes: 6 additions & 36 deletions jsonschema-core.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,7 @@ identified in a meta-schema.

JSON Schema can be extended either by defining additional vocabularies, or less
formally by defining additional keywords outside of any vocabulary. Unrecognized
individual keywords simply have their values collected as annotations, while the
behavior with respect to an unrecognized vocabulary can be controlled when
declaring which vocabularies are in use.
individual keywords are not supported.
Comment on lines 68 to +70
Copy link
Member

@jdesrosiers jdesrosiers Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the purpose of this paragraph is to describe extension mechanisms. The x- convention is an extension mechanism, so I think it should be mentioned. I don't think it's necessary to say here that unrecognized keywords aren't allowed. This section is just an introduction. It doesn't have to be exhaustive. I think it's enough to briefly mention the extension mechanisms without needing to say what isn't an extension mechanism.
Copy link
Member Author

@gregsdennis gregsdennis Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The x- convention is an extension mechanism...

I'm curious of your definition of an extension mechanism. The x- behavior is defined by the spec, so in my eyes, these keywords are not extensions. This is clarified by #1518.

(This paragraph is going to change in #1510 anyway, so I'm not too concerned about getting it 100% right here.)
Copy link
Member

@jdesrosiers jdesrosiers Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we agree that the vocabulary system is an extension mechanism. It allows you to use custom keywords in your schemas. But, we decided that the vocabulary system was too cumbersome for users who just wanted to use simple annotation custom keywords, so we introduced x- as an alternative for users to define simple custom keywords without the overhead of the vocabulary system. Both are extension mechanisms for using custom keywords. One is complicated and powerful and the other is simple and highly constrained, but they're serving the same purpose.

x- is so constrained that I can see how you can view it as one thing defined by the spec, but I think there's more to it. x-foo and x-bar have the same annotation collection and validation behavior, but they aren't the same keyword. They have different semantics given to them by the schema author, not by the spec. I see the spec defined behavior of x- as constraints that x- extension keywords must adhere to, but it's not a complete definition. It take the same author to define the semantics for it to be a complete keyword.
Copy link
Member Author

@gregsdennis gregsdennis Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. Are you happy, then, to have x- keywords clarified as being "recognized" (using that word) in #1518?
Copy link
Member

@jdesrosiers jdesrosiers Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would work, but I just posted in the other tread an alternative that I think would be better. It avoids the problem altogether.

This document defines a core vocabulary that MUST be supported by any
implementation, and cannot be disabled. Its keywords are each prefixed with a
Expand Down Expand Up @@ -216,8 +214,8 @@ adjacent keywords.
Extension keywords, meaning those defined outside of this document and its
companions, are free to define other behaviors as well.

A JSON Schema MAY contain properties which are not schema keywords or are not
recognized as schema keywords. The behavior of such keywords is governed by
A JSON Schema MUST NOT contain properties which are not schema keywords or are
not recognized as schema keywords. The behavior of such keywords is governed by
{{unrecognized}}.

An empty schema is a JSON Schema with no properties.
Expand Down Expand Up @@ -440,10 +438,8 @@ vocabularies MUST NOT define any keywords which begin with this prefix.

#### Handling of unrecognized or unsupported keywords {#unrecognized}

Implementations SHOULD treat keywords they do not recognize, or that they
recognize but do not support, as annotations, where the value of the keyword is
the value of the annotation. Whether an implementation collects these
annotations or not, they MUST otherwise ignore the keywords.
Implementations MUST refuse to process schemas which contain keywords they do
not recognize, or that they recognize but do not support.
Copy link
Member

@jdesrosiers jdesrosiers Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wording here feels contradictory although I get that it isn't and why it isn't. I get that x- keywords are considered "recognized", but I'm not sure we're making that clear. Maybe we need to be more clear what is considered "recognized" or maybe we need to say it different way that's harder to misunderstand.
Copy link
Member Author

@gregsdennis gregsdennis Jun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Relequestual had this same concern in Slack. It's addressed in #1518.
Copy link
Member

@jdesrosiers jdesrosiers Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced the rewording in #1518 really addresses this concern. Not enough anyway. I think that rewording here is probably a better approach. Something like,

Implementations MUST refuse to evaluate schemas which contain keywords they do not know how to process.

I think that's clear, covers all the bases, and avoids confusion about whether x- counts as a recognized keyword. Implementations know how to process x- keywords, so it's clear this statement isn't referring to those keywords.
Copy link
Member Author

@gregsdennis gregsdennis Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That also addresses implementations which choose to support custom keywords outside of vocabs.
Copy link
Member Author

@gregsdennis gregsdennis Jun 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated with one tweak.

## Keyword Behaviors

Expand Down Expand Up @@ -831,11 +827,6 @@ re-use. These keywords do not affect validation or annotation results. Their
purpose in the core vocabulary is to ensure that locations are available for
certain purposes and will not be redefined by extension keywords.

While these keywords do not directly affect results, as explained in
{{non-schemas}} unrecognized extension keywords that reserve locations for
re-usable schemas may have undesirable interactions with references in certain
circumstances.

### Loading Instance Data

While none of the vocabularies defined as part of this or the associated
Expand Down Expand Up @@ -1008,13 +999,7 @@ refuse to process that schema.

Implementations that do not support a vocabulary that is optionally used by a
schema SHOULD proceed with processing the schema. The keywords will be
considered to be unrecognized keywords as addressed by {{unrecognized}}. Note
that since the recommended behavior for such keywords is to collect them as
annotations, vocabularies consisting only of annotations will have the same
behavior when used optionally whether the implementation supports them or not.
This allows annotation-only vocabularies to be supported without custom code,
even in implementations that do not support providing custom code for extension
vocabularies.
considered to be unrecognized keywords as addressed by {{unrecognized}}.

##### Vocabularies are schema resource-scoped

Expand Down Expand Up @@ -1580,21 +1565,6 @@ that take one or more subschemas as a value. These keywords may be `$defs` and
the standard applicators from this document, or extension keywords from a known
vocabulary, or implementation-specific custom keywords.

Multi-level structures of unknown keywords are capable of introducing nested
subschemas, which would be subject to the processing rules for `$id`. Therefore,
having a reference target in such an unrecognized structure cannot be reliably
implemented, and the resulting behavior is undefined. Similarly, a reference
target under a known keyword, for which the value is known not to be a schema,
results in undefined behavior in order to avoid burdening implementations with
the need to detect such targets.[^10]

[^10]: These scenarios are analogous to fetching a schema over HTTP but
receiving a response with a Content-Type other than `application/schema+json`.
An implementation can certainly try to interpret it as a schema, but the origin
server offered no guarantee that it actually is any such thing. Therefore,
interpreting it as such has security implication and may produce unpredictable
results.

Note that single-level custom keywords with identical syntax and semantics to
`$defs` do not allow for any intervening `$id` keywords, and therefore will
behave correctly under implementations that attempt to use any reference target
Expand Down