Golang bindings for tree-sitter
Create a parser with a grammar:
import ( "context" "fmt" sitter "github.com/api-extraction-examples/go-tree-sitter" "github.com/api-extraction-examples/go-tree-sitter/javascript" ) parser := sitter.NewParser() parser.SetLanguage(javascript.GetLanguage())Parse some code:
sourceCode := []byte("let a = 1") tree, _ := parser.ParseCtx(context.Background(), nil, sourceCode)Inspect the syntax tree:
n := tree.RootNode() fmt.Println(n) // (program (lexical_declaration (variable_declarator (identifier) (number)))) child := n.NamedChild(0) fmt.Println(child.Type()) // lexical_declaration fmt.Println(child.StartByte()) // 0 fmt.Println(child.EndByte()) // 9This repository provides grammars for many common languages out of the box.
But if you need support for any other language you can keep it inside your own project or publish it as a separate repository to share with the community.
See explanation on how to create a grammar for go-tree-sitter here.
Known external grammars:
- Salesforce grammars - including Apex, SOQL, and SOSL languages.
- Ruby - Deprecated, grammar is provided by main repo instead
- Go Template - Used for helm
If your source code changes, you can update the syntax tree. This will take less time than the first parse.
// change 1 -> true newText := []byte("let a = true") tree.Edit(sitter.EditInput{ StartIndex: 8, OldEndIndex: 9, NewEndIndex: 12, StartPoint: sitter.Point{ Row: 0, Column: 8, }, OldEndPoint: sitter.Point{ Row: 0, Column: 9, }, NewEndPoint: sitter.Point{ Row: 0, Column: 12, }, }) // check that it changed tree assert.True(n.HasChanges()) assert.True(n.Child(0).HasChanges()) assert.False(n.Child(0).Child(0).HasChanges()) // left side of the tree didn't change assert.True(n.Child(0).Child(1).HasChanges()) // generate new tree newTree := parser.Parse(tree, newText)You can filter AST by using predicate S-expressions.
Similar to Rust or WebAssembly bindings we support filtering on a few common predicates:
eq?,not-eq?match?,not-match?
Usage example:
func main() { // Javascript code sourceCode := []byte(` const camelCaseConst = 1; const SCREAMING_SNAKE_CASE_CONST = 2; const lower_snake_case_const = 3;`) // Query with predicates screamingSnakeCasePattern := `( (identifier) @constant (#match? @constant "^[A-Z][A-Z_]+") )` // Parse source code lang := javascript.GetLanguage() n, _ := sitter.ParseCtx(context.Background(), sourceCode, lang) // Execute the query q, _ := sitter.NewQuery([]byte(screamingSnakeCasePattern), lang) qc := sitter.NewQueryCursor() qc.Exec(q, n) // Iterate over query results for { m, ok := qc.NextMatch() if !ok { break } // Apply predicates filtering m = qc.FilterPredicates(m, sourceCode) for _, c := range m.Captures { fmt.Println(c.Node.Content(sourceCode)) } } } // Output of this program: // SCREAMING_SNAKE_CASE_CONSTCheck if any updates for vendored files are available:
go run _automation/main.go check-updates Update vendor files:
- open
_automation/grammars.json - modify
reference(for tagged grammars) orrevision(for grammars from a branch) - run
go run _automation/main.go update <grammar-name>
It is also possible to update all grammars in one go using
go run _automation/main.go update-all