Skip to content

Conversation

@RobinMalfait
Copy link
Member

@RobinMalfait RobinMalfait commented Mar 11, 2025

This PR adds a small JSON pre processor to improve parsing JSON files. This is because the extractor creates "sub machines" whenever it encounters a [ or a { in the input. We do this because of things like %w[…] strings in Ruby or className={clsx({flex: true})} in JSX.

Due to the sheer amount of potential [ and ] brackets, it could be that parsing JSON files are way slower than they need to be.

To tackle this, after this PR, when given an input like this:

[1,[2,[3,4,["flex flex-1 content-['hello_world']"]]], {"flex": true}] 

We'll preprocess all the important brackets and braces by replacing them with spaces so the extractor doesn't need special casing:

1, 2, 3,4, "flex flex-1 content-['hello_world']" , "flex": true 

We saw this while debugging this issue: #17092

Test plan

  1. Added test to verify the pre processing works
  2. Existing tests still pass
@RobinMalfait RobinMalfait requested a review from a team as a code owner March 11, 2025 12:06
@RobinMalfait RobinMalfait force-pushed the feat/add-json-pre-processor branch from c0ed038 to 08b0203 Compare March 11, 2025 12:06
@RobinMalfait RobinMalfait force-pushed the feat/add-json-pre-processor branch from 3fe4791 to a45a6c4 Compare March 11, 2025 12:12
@RobinMalfait RobinMalfait enabled auto-merge (squash) March 11, 2025 12:15
@RobinMalfait RobinMalfait merged commit 9ddeb09 into main Mar 11, 2025
5 checks passed
@RobinMalfait RobinMalfait deleted the feat/add-json-pre-processor branch March 11, 2025 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants