Rust 2019: Think Bigger
Rust shines when we find ways to have our cake and eat it too: memory safety without runtime garbage collection, abstraction without overhead, threading without data races. We must find new ways to continue this tradition for Rust 2019 and beyond.
On a day-to-day basis, I am dedicated to small, incremental progress. If a pull request is an improvement over the status quo, merge it now! Don’t wait for the pull request to be perfectly pristine or the feature to be 100% complete. Each day we drag reality inch by inch towards the ideal.
However, when planning on the scale of years, our vision must not be weighed down by discussion of incremental improvements: we must rise and collectively define the lofty ideals we aim for. It requires avoiding local maxima. Nick Cameron’s Rust in 2022 post, where he starts with what we might want in a Rust 2022 edition and then works backwards from there, is a great example.
With that out of the way, I will make a couple suggestions for the Rust 2019 roadmap. I will leave my thoughts for the Rust and WebAssembly domain working group’s 2019 roadmap for a future post.
Speed Up Compilation
Tired: make
rustc
faster.Wired: integrate distributed compilation and artifact caching into
cargo
and crates.io.
Of course we should continue identifying and implementing performance wins in
rustc
itself. We should even invest in larger scale rearchitecting like adding
finer-grained parallelism in with rayon
(I won’t go into too many specifics
here because I’m largely ignorant of them!)
But we should also be thinking bigger.
The fastest compilation is the one that you didn’t have to do. If we integrate
something like sccache
into cargo
and crates.io, then individuals
can download pre-built artifacts for common dependencies from a shared cache and
save big on local CPU time. In comparison, a 5% speedup to trait resolution is
peanuts. This is an opportunity that is not available to most language
ecosystems! Most languages don’t have a compiler toolchain, build system, and
package manager that are widely used together and well integrated.
First-Class, Domain-Specific Workflows
Tired: make
wasm-pack
really good.Wired: make
wasm-pack
unnecessary by building generic task hooks intocargo
itself.
Different domains have different workflows that extend past cargo build
. With
WebAssembly, we must also generate JavaScript bindings, run tools like
wasm-opt
, create a package.json
to integrate with NPM and JavaScript
tooling, etc… For embedded development, you need to at minimum flash your
built program into your microcontroller’s persistent memory.
To perform these tasks today, we typically write whole new tools that wrap
cargo
(like wasm-pack
), invoke external tools manually (like using openocd
by hand), or write a cargo-mytask
package to add the cargo mytask
subcommand. These solutions suffer from either repetition and a lack of
automation, or they wrap cargo
but fail to expose all the wonderful features
that cargo
supports (for example, you can’t use the --feature
flag yet with
wasm-pack
). We should not write these tools that wrap cargo
, we should write
generic build tasks, which are invoked automatically by cargo
itself.
cargo
should not just grow a post_build.rs
hook, its build tasks and
dependencies between tasks and artifacts should become fully extensible. I
should be able to depend on wasm build tasks in my Cargo.toml
, and then after
that cargo build
should just Do The Right Thing. I shouldn’t have to compile
these wasm build tasks for every project I use them with. cargo
and crates.io
should handle transparently distributing the wasm task binaries to me.
Growing Working Groups
Tired: the Rust project should start a working group for
$PROJECT_OR_DOMAIN
.Wired: the Rust project should have a working group template, and system of mentorship for new (and old!) working group leads.
The more we collaborate and work together, the better we can tackle problems that are larger than any one of us. The primary way we’ve been organizing technical efforts in the Rust project has been working groups. But starting a new working group is hard, and leading a working group is hard.
We should have a template for new working groups that comes with cookie-cutter steps to follow to help build an open community, articulate working group vision, and collectively organize. Of course these steps will need to evolve for each particular working group’s needs, but having something to help new working groups hit the ground running is incredibly valuable. It would have been so useful for me when we were kicking off the WebAssembly domain working group last year. A lot of things that are obvious in retrospect were not at the time: hold weekly meetings, adopt an RFC process, communicate(!!), create room for contributors to own sub-goals, etc…
Additionally, every working group lead should have a mentor who is in a leadership position within the Rust project: someone who is a member of the Rust core team or a lead of another working group or team. Someone to rubber duck with and seek guidance from in the context of leadership and organization.
Instead of enabling Rust users to ask Rust leadership for a working group for
X
, we should empower them to start the working group for X
themselves, and
we should continuously follow up to ensure that they succeed. To have our cake
and eat it too, Rust development must be a positive-sum game.
#Rust2019
Whatever we end up with in the 2019 roadmap, I have faith that what we choose will be worthy. We don’t suffer from a lack of good options.
I hope we never stop dreaming big and being ambitious.