new RFC: 2019 Roadmap!

Author: fitzgen

text/000-2019-roadmap.md

@@ -0,0 +1,336 @@
+- Start Date: 2019-01-23
+- RFC PR: (leave this empty)
+
+# Summary
+
+<!-- One paragraph explanation of the proposal. -->
+
+**2019 is the year WebAssembly with Rust goes from "usable" to "stable,
+batteries-available, and production-ready."**
+
+To realize this goal, the Rust and WebAssembly domain working group will:
+
+* Cultivate a library ecosystem by collaborating on a modular toolkit
+
+* Bring multithreading to Rust-generated Wasm
+
+* Integrate best-in-class debugging support into our toolchain
+
+* Polish our toolchain and developer workflow, culminating in a 1.0 version of
+ `wasm-pack`
+
+# Motivation
+
+<!-- Why are we doing this? What use cases does it support? What problems does it -->
+<!-- solve? What is the expected outcome? -->
+
+This proposed roadmap draws upon
+
+* [the community's blog posts][rustwasm2019-issue] in response to the working
+ group's [call for roadmap suggestions,][rustwasm2019-call]
+
+* and the working group's core team's intuition and experience.
+
+[rustwasm2019-issue]: https://github.com/rustwasm/team/issues/241
+[rustwasm2019-call]: https://rustwasm.github.io/2018/12/06/reflecting-on-rust-and-wasm-in-2018.html#rustwasm2019
+
+# Detailed Explanation
+
+## Collaborating on a Modular Toolkit
+
+> The idea of building [high-level libraries] in a modular way that will allow
+> others in the community to put the components together in a different way is
+> very exciting to me. This hopefully will make the ecosystem as a whole much
+> stronger.
+>
+> In particular I’d love to see a modular effort towards implementing a virtual
+> DOM library with JSX like syntax. There have been several efforts on this
+> front but all have seemed relatively monolithic and “batteries included”. I
+> hope this will change in 2019.
+
+<cite>&mdash; Ryan Levick in [Rust WebAssembly
+2019](https://blog.ryanlevick.com/posts/rust-wasm-2019/)</cite>
+
+> Don't create branded silos. Branding might perhaps be useful to achieve
+> fame. But if we truly want Rust's Wasm story to succeed we should think of
+> ways to collaborate instead of carving out territory.
+
+<cite>&mdash; Yoshua Wuyts in [Wasm
+2019](https://blog.yoshuawuyts.com/wasm-2019/)</cite>
+
+In 2018, we created foundational libraries like [`js-sys` and
+`web-sys`][announcing-web-sys]. In 2019, we should build modular, high-level
+libraries on top of them, and collect the libraries under an umbrella toolkit
+crate for a holistic experience. This toolkit and its libraries will make
+available all the batteries you want when targeting Wasm.
+
+Building a greenfield Web application? Use the whole toolkit to hit the ground
+running. Carefully crafting a tiny Wasm module and integrating it back into an
+existing JavaScript project? Grab that one targeted library you need out from
+the toolkit and use it by itself.
+
+* **Modular:** Take or leave any individual component. Prefer interfaces over
+ implementations.
+
+* **Cultivate collaboration:** We've already seen an ecosystem sprouting up in
+ the Rust and WebAssembly domain, and lots of great experiments, but we haven't
+ seen a lot of collaboration between projects. By deliberately creating a space
+ for collaboration, we can reduce effort duplication, multiply impact, and help
+ the ecosystem stay healthy.
+
+[announcing-web-sys]: https://rustwasm.github.io/2018/09/26/announcing-web-sys.html
+
+## Multithreading for Wasm
+
+> We must bring Rust’s [fearless
+> concurrency](https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.html)
+> to the Web!
+
+<cite>&mdash; Nick Fitzgerald in [Rust and WebAssembly in
+2019](http://fitzgeraldnick.com/2018/12/14/rust-and-webassembly-in-2019.html)</cite>
+
+> Be the absolute cutting edge when it comes to WebAssembly, we should be
+> thinking about being the first to satisfy [threads and atomics].
+
+<cite>&mdash; richardanaya in [My Rust 2019
+Dream](https://www.reddit.com/r/rust/comments/aac8zk/my_rust_2019_dream_dominate_the_web/)</cite>
+
+Out toolchain already has [experimental support for multithreading in
+Wasm][multithreading]. Browsers are currently shipping `SharedArrayBuffer` and
+atomics (the primitives of multithreading for Wasm) behind feature flags, and
+they expect to start shipping them enabled by default in 2019.
+
+One of WebAssembly's selling points is the ability to effectively utilize
+available hardware. Multithreading extends that story from a single core to
+many. While multithreading will be *literally possible* for both JavaScript and
+any compiled-to-Wasm language, Rust's unique ownership system makes it
+*economically realistic*.
+
+There are some technical snags (see the link above for details) that mean we
+can't get Rust's standard library's `std::thread::*` working on Wasm. But it is
+still crucial that we have shared implementations of core multithreading
+building blocks like thread pools and locks across the ecosystem. In 2019, we
+should transform our experimental multithreading support into a production-ready
+foundation for multithreading on Wasm, get popular crates like `rayon` working
+on the Web, and cash in on Rust's fearless concurrency.
+
+[multithreading]: https://rustwasm.github.io/2018/10/24/multithreading-rust-and-wasm.html
+
+## Debugging
+
+> Before [debugging] is working properly (including variable inspection, which
+> doesn't work with wasm at all right now), everything else is just toying
+> around.
+
+<cite>&mdash; anlumo in [a comment on
+r/rust](https://www.reddit.com/r/rust/comments/aac8zk/my_rust_2019_dream_dominate_the_web/ecqu3wu/)</cite>
+
+> Having [source maps] would be excellent for debugging.
+
+<cite>&mdash; Yoshua Wuyts in [Wasm 2019](https://blog.yoshuawuyts.com/wasm-2019/)
+
+Debugging is tricky because much of the story is out of this working group's
+hands, and depends on both the WebAssembly standardization bodies and the folks
+implementing browser developer tools instead. However, there are some concrete
+steps we can take to improve debugging:
+
+1. Get `println!`, `dbg!`, and friends working out of the box with Wasm. To
+ achieve this, we will build support for the [WebAssembly reference
+ sysroot][sysroot] and standard system calls for Wasm that are in the
+ standardization pipeline.
+
+2. Create the ability to compile our Rust-generated Wasm to JavaScript with
+ source maps when debugging. Source maps are a limited debug info format for
+ JavaScript that enable stepping through source locations in a debugger,
+ instead of stepping through compiler-generated JavaScript code.
+
+3. Add debugging-focused tracing and instrumentation features to our
+ toolchain. For example, it is currently difficult to debug a JavaScript array
+ buffer view of Wasm memory getting detached because Wasm memory was
+ resized. We can make debugging easier by optionally instrumenting `mem.grow`
+ instructions with logging.
+
+In addition to that, we should work with the WebAssembly standardization bodies
+and browser developer tools makers, and actively participate in the WebAssembly
+debugging subcharter to create some movement in the debugging space. By keeping
+up the environmental and social pressure and lending a hand where we can, we
+will eventually have rich, source-level debugging for Wasm.
+
+[sysroot]: https://github.com/WebAssembly/reference-sysroot
+
+## Toolchain and Workflow Polish
+
+> Setting up a Wasm project requires quite some boilerplate. It'd be nice if we
+> could find ways to reduce this.
+
+<cite>&mdash; Yoshua Wuyts in [Wasm
+2019](https://blog.yoshuawuyts.com/wasm-2019/)</cite>
+
+> There are a few things that we intended to include in `wasm-pack` in 2018 that
+> didn’t quite make the cut. [...] We should finish these tasks and polish
+> `wasm-pack` into a 1.0 tool.
+
+<cite>&mdash; Nick Fitzgerald in [Rust and WebAssembly in
+2019](http://fitzgeraldnick.com/2018/12/14/rust-and-webassembly-in-2019.html)</cite>
+
+In 2019, our toolchain and workflow should feature complete and
+polished. `wasm-pack`, being the entry point to our toolchain, will bear the
+brunt of this work, but much of it will also be in tools that are invoked by
+`wasm-pack` rather than work in `wasm-pack` itself.
+
+* Generate JavaScript API documentation from the Rust doc comments on
+ `#[wasm_bindgen]` exports.
+
+* Finish and implement the RFC for library crates depending on external NPM
+ packages. TODO link
+
+* Finish and implement the RFC for local JavaScript snippets. TODO link
+
+* Support running Binaryen's `wasm-opt` on Rust-generated Wasm. TODO link
+
+* Integrate `cargo generate` for new project scaffolding. TODO: one less tool
+
+* RFC and implementation for generating portable, universal NPM packages that
+ work on the Web, with Node.js, and in any minimal JavaScript environment.
+
+* Define a philosophy for `wasm-pack`'s user experience, interaction, and
+ display. Once defined and agreed upon, we should triage each `wasm-pack`
+ subcommand and ensure that it is consistent with our philosophy.
+
+Given that this work is largely about plugging missing holes and improving user
+experience, it is a bit of a laundry list. But that is also good sign: it means
+that `wasm-pack` is actually fairly close to being feature complete.
+
+After we've finished all these tasks, we should publish a 1.0 release of
+`wasm-pack`.
+
+# Rationale, Drawbacks, and Alternatives
+
+<!-- This is your chance to discuss your proposal in the context of the whole design -->
+<!-- space. This is probably the most important section! -->
+<!-- - Why is this design the best in the space of possible designs? -->
+<!-- - What other designs have been considered and what is the rationale for not -->
+<!-- choosing them? -->
+<!-- - What is the impact of not doing this? -->
+
+We choose to focus our efforts in 2019 where:
+
+1. *We* &mdash; the Rust and WebAssembly working group &mdash; can build and
+ ship features. Areas where we aren't potentially blocked by external factors,
+ such as still-in-progress standards.
+
+2. We can leverage advantages that are *unique to Rust* in the WebAssembly
+ domain.
+
+## Things *We* can Build and Ship
+
+We don't want our fate in anyone's hands but our own.
+
+The toolkit and toolchain polish work don't involve any external entities that
+could slow our progress to a halt. For debugging, where the larger story
+involves significant consensus with external groups and standards work, we
+explicitly choose to focus on what we can do ourselves to improve our own
+debugging story. We do not set ourselves up to block on anything produced by the
+WebAssembly community group's debugging subcharter, and we won't wait on browser
+vendors to implement new Wasm debugging support in developer tools.
+
+Of the roadmap items, the multithreading story has the most risk: our success in
+this domain relies on browsers enabling Wasm's multithreading primitives by
+default. However, this seems like a relatively safe bet, since the
+multithreading primitives have moved past their most experimental phase, [Chrome
+is already shipping them enabled by default, and all other major browsers have
+implementations that just aren't enabled by default yet.][sab-caniuse]
+
+[sab-caniuse]: https://caniuse.com/#feat=sharedarraybuffer
+
+## Leveraging Unique Advantages
+
+We want to focus our efforts where we get the biggest effort to impact
+efficiency, and establish ourselves as leaders in WebAssembly in ways that no
+one else even has a route towards catching up.
+
+The multithreading story is perhaps the biggest example of unique advantage:
+multithreading is *infamously* bug prone (to say the least!) and Rust's
+ownership system eliminates data races at compile time.
+
+By building a modular toolkit of libraries, we bolster our ability to target the
+full spectrum from tiny module surgically inserted into an existing JavaScript
+application, to building a complete Web application in Rust. Any language that
+relies on a garbage collector, fat runtime, or is overly opinionated about FFI
+and interaction with the outside world can't reach the tiny module end of that
+spectrum.
+
+The toolchain polish and debugging work have less clearly *unique*
+advantages. But both are table stakes for a good development experience, and the
+par for the course for these things in the Wasm domain is currently so low that
+we can and should stand out from the crowd.
+
+## Considered Alternative Roadmap Items
+
+Here are a few alternative items that were considered for the roadmap, perhaps
+because they were called out in `#RustWasm2019` posts, but ultimately were not
+included.
+
+### Pushing `anyref` Integration into the Rust Language
+
+We've already been well positioned to take advantage of host bindings and GC
+reference types once they ship in Wasm via `wasm-bindgen`. We could take it even
+further and imagine a future where the Rust language was able to pass around
+opaque references to objects in alternative memory spaces (some of which might
+be GC'd) in a first class way: structs that are split across memory spaces, fat
+pointers into multiple memory spaces, etc.
+
+However, it isn't clear that pushing this all the way into the language will
+bring that much utility over the existing ["`anyref` at the edges"
+implementation that `wasm-bindgen` already has.][anyref-wasm-bindgen]
+Additionally, cashing in on this work could easily be blocked in a couple ways:
+`anyref` isn't shipping in any mainstream wasm engine yet, and getting this
+language-level integration through the larger Rust RFC process with all of its
+stakeholders would happen at a glacial pace (if it even happened!)
+
+[anyref-wasm-bindgen]: https://github.com/rustwasm/wasm-bindgen/pull/1002
+
+### A Focus Only on Pure-Rust Web Applications
+
+We prefer to answer "yes and" to pure-Rust Web applications via the modular
+toolkit that can service the full spectrum of tiny module to whole Web app, than
+to focus only on the whole Web app end of the spectrum. Our hope with the
+toolkit is that a rising tide will lift all boats, regardless where your project
+lands on that spectrum.
+
+Additionally, full Web apps are not a *unique* advantage for Rust. JavaScript
+has been doing it for a while, and as far as Wasm goes, there are better-funded
+"competitors" in the space that will be able to provide a more compelling
+monolithic Web app development experience more quickly (via integration with
+tooling, existing ecosystems, or throwing money and developers at the
+problem). Microsoft and Blazor, Google and Go, bringing existing native
+applications to the Web with Emscripten, etc. We should compete where we are
+best positioned to do so, and monolithic Web applications is not that.
+
+All that said, if you want to build a whole Web application with Rust-generated
+Wasm and don't want to write any JavaScript at all, you should be able to do
+so. In fact, [you already can with `#[wasm_bindgen(start)]` and the `no-modules`
+target][no-modules]. We will never remove this ability, and the new toolkit will
+only make developing a whole Web app easier.
+
+[no-modules]: https://github.com/rustwasm/wasm-bindgen/tree/master/examples/no_modules
+
+### Non-JavaScript and Non-Web Embeddings
+
+While the whole of every non-Web and non-JavaScript WebAssembly embeddings looks
+very exciting, each embedding is a unique environment, and there is not yet a
+standard set of capabilities available. We don't want to block on waiting for a
+full set of standard capabilities to emerge, nor do we want to choose one
+particular embedding environment.
+
+We do intend to support the reference sysroot work, and any follow up work that
+comes after it, but we will take advantage of these things on a opportunistic
+basis rather than making it a roadmap item.
+
+We encourage anyone interested in non-JavaScript and non-Web embeddings to
+collaborate with the WebAssembly community group to push this story forward by
+defining standard Wasm capabilities!
+
+# Unresolved Questions
+
+To be determined.