Skip to content

Conversation

@MichaReiser
Copy link
Contributor

@MichaReiser MichaReiser commented May 18, 2023

This removes the need for copying statements and other nodes during parsing.

Performance

This gives me a 4% performance improvement in ruff's parser benchmark (parses a couple of files and traverses the AST)

group main stmt ----- ---- ---- parser/large/dataset.py 1.04 2.6±0.01ms 15.4 MB/sec 1.00 2.5±0.02ms 16.0 MB/sec parser/numpy/ctypeslib.py 1.04 499.8±3.36µs 33.3 MB/sec 1.00 478.8±4.59µs 34.8 MB/sec parser/numpy/globals.py 1.04 51.2±0.76µs 57.6 MB/sec 1.00 49.2±0.45µs 60.0 MB/sec parser/pydantic/types.py 1.04 1100.1±11.99µs 23.2 MB/sec 1.00 1061.7±9.17µs 24.0 MB/sec 
},

// The first compound statement
<s:CompoundStatement> => vec![s],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old grammar created a vec for every statement. The new grammar simply appends statements at the end of the vec.

Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 awesome

MichaReiser and others added 2 commits May 18, 2023 22:32
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants