Skip to content

Conversation

@alexcrichton
Copy link
Contributor

The WebAssembly organization on GitHub has a spec-tests repository
which contains a collection of tests for WebAssembly implementations.
These tests are quite comprehensive and a great way to ensure good
coverage within our own engine of WebAssembly features, even as they're
added to the spec over time.

This commit adds a submodule to the spec-tests repository and then hooks
up a suite of tests to run them. To "run" a spec test means:

  • First, the wabt toolkit's wast2json tool is executed, performing
    two operations. First, as the name implies, it parses the *.wast
    file and emits a *.json file describing the test. This *.json file
    is much easier to parse for us thatn *.wast. Second the tool will
    extract a number of *.wasm files from the test, emitting them to the
    filesystem.

  • Next we look at all the test directives in the JSON file. For anything
    that's supposed to be an invalid/malformed module we ensure that our
    parsing fails (as our parsing currently bundles validation).

  • For all directives which are a valid wasm test, we parse the
    referenced file and then serialize it back out, round-tripping the
    wasm file through walrus.

  • Finally we execute wabt's tool spectest-interp. This takes in the
    JSON file as input and then executes all of the tests, presumably in
    its built-in interpreter. This hopefully ensures that we actually
    emitted the correct wasm, as the tool is ingesting wasm that walrus
    produced.

Turns out we were already passing all the spec tests basically (yay!)
except for a few more validations added here:

  • Exports are now an Arena instead of an ArenaSet to generate an
    error when two exports have the same name (and the same type).
    Previously they silently overrode each other.

  • Alignment on loads/stores are validated to be less than the width of
    the load/store operation.

  • The signature of the start function is checked.

  • Added support for the sign-extension-ops proposal

@alexcrichton
Copy link
Contributor Author

Note that this builds on #23, so the first commit can be ignored

Copy link
Contributor

@fitzgen fitzgen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

The WebAssembly organization on GitHub has a `spec-tests` repository which contains a collection of tests for WebAssembly implementations. These tests are quite comprehensive and a great way to ensure good coverage within our own engine of WebAssembly features, even as they're added to the spec over time. This commit adds a submodule to the spec-tests repository and then hooks up a suite of tests to run them. To "run" a spec test means: * First, the `wabt` toolkit's `wast2json` tool is executed, performing two operations. First, as the name implies, it parses the `*.wast` file and emits a `*.json` file describing the test. This `*.json` file is much easier to parse for us thatn `*.wast`. Second the tool will extract a number of `*.wasm` files from the test, emitting them to the filesystem. * Next we look at all the test directives in the JSON file. For anything that's supposed to be an invalid/malformed module we ensure that our parsing fails (as our parsing currently bundles validation). * For all directives which are a valid wasm test, we parse the referenced file and then serialize it back out, round-tripping the wasm file through `walrus`. * Finally we execute `wabt`'s tool `spectest-interp`. This takes in the JSON file as input and then executes all of the tests, presumably in its built-in interpreter. This hopefully ensures that we actually emitted the correct wasm, as the tool is ingesting wasm that walrus produced. Turns out we were already passing all the spec tests basically (yay!) except for a few more validations added here: * Exports are now an `Arena` instead of an `ArenaSet` to generate an error when two exports have the same name (and the same type). Previously they silently overrode each other. * Alignment on loads/stores are validated to be less than the width of the load/store operation. * The signature of the start function is checked. * Added support for the sign-extension-ops proposal
@alexcrichton alexcrichton merged commit e8f257f into wasm-bindgen:master Jan 24, 2019
@alexcrichton alexcrichton deleted the spec-tests branch January 24, 2019 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants