Fuzzer: Adjust feature fuzzing frequency #6305
Conversation
scripts/fuzz_opt.py Outdated
| # Note that POSSIBLE_FEATURE_OPTS contains --disable-X for each feature, so | ||
| # we enable all features when we add nothing from it, etc. (assert on that | ||
| # form here to guard against future refactorings making a mess here). | ||
| assert '--disable' in POSSIBLE_FEATURE_OPTS[0] | ||
| assert '--enable' not in POSSIBLE_FEATURE_OPTS[0] |
There was a problem hiding this comment.
How about renaming it to POSSIBLE_DISABLE_FEATURE_OPTS or something? Code like
FEATURES_OPTS += POSSIBLE_FEATURE_OPTSlooks like we are enabling all the possible options out there when we are actually doing the very opposite.
There was a problem hiding this comment.
Good idea, thanks, renamed and refactored.
| Actually I may need to change the probabilities here, since I forgot that enabling all features disables parts of the fuzzer... for example enabling simd disables running d8 (since v128 on the boundary traps - we can't compare those values). I'll think about some solution to that first maybe. |
| After recent PRs landing, enabling all features no longer limits V8 fuzzing. It does still limit some things like wasm2js fuzzing, but that is low priority, and so this can land. |
We used to fuzz MVP 1/3, all 1/3, and a mixture 1/3, but that gives far too much priority to the MVP which is increasingly less important. It is also a good idea to give "all" more priority as that enables more initial content to run (the fuzzer will discard initial content if it doesn't validate with the features chosen in the current iteration). Also (NFC) rename POSSIBLE_FEATURE_OPTS to make the code easier to follow.
We used to fuzz MVP 1/3, all 1/3, and a mixture 1/3, but that gives far too much priority to the MVP which is increasingly less important. It is also a good idea to give "all" more priority as that enables more initial content to run (the fuzzer will discard initial content if it doesn't validate with the features chosen in the current iteration). Also (NFC) rename POSSIBLE_FEATURE_OPTS to make the code easier to follow.
3 participants
We used to fuzz MVP 1/3, all 1/3, and a mixture 1/3, but that gives far too much
priority to the MVP which is increasingly less important. It is also a good idea to
give "all" more priority as that enables more initial content to run (the fuzzer will
discard initial content if it doesn't validate with the features chosen in the current
iteration).