Schema Evolution, Career Edition

🔔 I've Joined AWS ... with one foot still at Berkeley.

For years I’ve been asserting that it’s time to change the way we write software, and stop pretending we're programming one machine at a time. With coding agents emerging as a disruptive force, I’m convinced the moment has come—it’s time to get real!

In pursuit of that vision, I have joined AWS to help shape the next generation of distributed software development. I’m not coming alone—three other core members of the Hydro project have joined with me (newly-minted Dr. Shadaj Laddad, Mingwei Samuel and Lucky Katahanas). Together, we’ll continue advancing Hydro as an open-source initiative, collaborating with colleagues at institutions like Berkeley and Princeton.

Distributed Coding Challenges, Post-LLMs

The timing of this move is no coincidence. LLM-driven coding agents are changing the landscape by radically lowering the burden of authoring code—even in new languages and frameworks. But LLMs are undisciplined by definition, and nobody likes an undisciplined software engineer. LLMs need to be coupled with core computer science techniques for ensuring safety of code. This is the “neuro-symbolic” vision of AI, which pairs up “guessers” (LLMs in this case) and “checkers” (deterministic deduction, as in type systems and declarative query languages.)

Modern languages like Rust are helpful checkers in this environment, by addressing longstanding challenges like memory safety without introducing performance overheads. Hydro takes this further, allowing developers to write Rust functions that can span multiple machines, with type system guarantees of distributed safety. Hydro code is guaranteed to be safe in the face of non-determinism inherent in modern distributed systems​—from communication delays to network interleavings—before it will even compile. And developers still get all the goodies they like about Rust, from IDE support to LLM assistance.

AWS

Choosing AWS was strategic. I’m excited to be part of an organization that leads in spec-driven development (hello, Kiro!) and the use of formal methods for proving correctness of critical infrastructure, while contributing in a major way to the Rust open source community. The folks who run the world's biggest cloud have a vested interest in extremely reliable distributed systems, making them an ideal champion for bringing years of our research into practice.

UC Berkeley

I’m still maintaining a role as Professor of the Graduate School at Berkeley, enabling me to continue steering the open Hydro research agenda and co-advising students alongside brilliant colleagues like Natacha Crooks and Max Willsey.

While I generally keep CALM, I’m fired up to pursue this moment at AWS.