Skip to content

Conversation

@milseman
Copy link
Member

@milseman milseman commented Dec 12, 2021

Track source locations for Atoms, quantifier sub-values (like ranges and modifiers), and other refactoring and changes.

Naming convention:

  • Source.Position: A dimensionless point in the input
    • Corresponds to an index in the input
  • Source.Location: The region of the input from which a parsed entity came
    • Corresponds to an index-range in the input, but separate type to hold API
    • SourceLocation typealias
  • AST.Loc / Source.Loc: a located value, that is a value with a location.
    • Name is short because it's ubiquitous

AST builders and faking:

  • Builder global functions internalized and sunk into the _StringProcessing module
  • _fake globals removed in favor of static members or inits on SourceLocation
  • atom_m makes the custom char set member variant instead of the AST variant

Changes to AST or parser functionality:

  • Trivia now records the contents as a String
  • Source is no longer a collection, and dropped the Peekable protocol.
  • Atoms tracked
  • Quantifier values (like the ranges or kinds) tracked

After this PR, there's only a handful of minor things to track:

  • The | in alternations
  • Potentially, even finer-grained pieces:
    • The - in ranges inside custom character classes
    • Such as the closing ) tracked separately, though that's derivable from the group's range
    • Quote's \Q and \E distinct from the contents of the quote
    • The location of the ^ for inverting Unicode properties inside a \p{^...}
    • The portion covering the name in named back references (groups do have it though)
    • The location of the comma in a quantification a{3,7}
    • ...
@milseman milseman marked this pull request as ready for review December 13, 2021 02:09

/// Record source loc before processing and return
/// or throw the value/error with source locations.
mutating func recordLoc<T>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
mutating func recordLoc<T>(
mutating func recordLocation<T>(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this shouldn't even be internal. I'll make it fileprivate to lexical analysis. Within fileprivate usage, I think the abbreviation is appropriate.

@milseman
Copy link
Member Author

@swift-ci please test linux platform

@milseman milseman merged commit 67c4de8 into swiftlang:main Dec 13, 2021
@milseman milseman deleted the locus_sourcus branch December 13, 2021 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants