Add a suite of spec tests #24
Merged
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
The WebAssembly organization on GitHub has a
spec-testsrepositorywhich contains a collection of tests for WebAssembly implementations.
These tests are quite comprehensive and a great way to ensure good
coverage within our own engine of WebAssembly features, even as they're
added to the spec over time.
This commit adds a submodule to the spec-tests repository and then hooks
up a suite of tests to run them. To "run" a spec test means:
First, the
wabttoolkit'swast2jsontool is executed, performingtwo operations. First, as the name implies, it parses the
*.wastfile and emits a
*.jsonfile describing the test. This*.jsonfileis much easier to parse for us thatn
*.wast. Second the tool willextract a number of
*.wasmfiles from the test, emitting them to thefilesystem.
Next we look at all the test directives in the JSON file. For anything
that's supposed to be an invalid/malformed module we ensure that our
parsing fails (as our parsing currently bundles validation).
For all directives which are a valid wasm test, we parse the
referenced file and then serialize it back out, round-tripping the
wasm file through
walrus.Finally we execute
wabt's toolspectest-interp. This takes in theJSON file as input and then executes all of the tests, presumably in
its built-in interpreter. This hopefully ensures that we actually
emitted the correct wasm, as the tool is ingesting wasm that walrus
produced.
Turns out we were already passing all the spec tests basically (yay!)
except for a few more validations added here:
Exports are now an
Arenainstead of anArenaSetto generate anerror when two exports have the same name (and the same type).
Previously they silently overrode each other.
Alignment on loads/stores are validated to be less than the width of
the load/store operation.
The signature of the start function is checked.
Added support for the sign-extension-ops proposal