Great product engineering in the AI era

· 14 min read · Ani Ravi · Engineering, Technology

An earlier version of this essay was submitted to The Pragmatic Engineer’s essay challenge.

This builds on my earlier post, Doing great product engineering, looking at how those principles apply in the AI era.

The most important aspect of product engineering

The role of a product engineer has radically changed, but the craft of great engineering has not changed all that much. While I used to obsess over every little detail of how the code was structured and written, I now get to spend most of my time thinking about the system that produces high-quality output. AI changes some technical decision-making, but the core principles for doing great work and picking the right tools are still largely the same.

I’ve been lucky to be part of small teams that punch way above their weight, and the latest developer tooling has amplified my thoughts on how to do great product engineering. My worldview on how to do great engineering has been shaped by continually iterating on the question of how to set up a codebase such that a small team can be as productive as possible. It’s been the central question of my efforts as a technical leader beyond building a great product that actually solves the problem for customers.

One of the most important engineering decisions in the early days is choosing your tools. It’s fundamental to how engineers express themselves. The teams that (often quietly) ship great products with small teams are principled first and foremost about the tools they choose and play to their strengths. This comes in many forms. For some teams, it’s using an obscure programming language no one has heard of (which tends to self-select for more hardcore/passionate programmers, aside from the unfair advantage the tooling itself may provide when driven by the right person). For others, this means pushing parts of a “conventional” stack much farther than people think you can (relying on a database like PostgreSQL to also hold a lot of business logic). For some, it means using a stack that’s fully open-source and self-hosted with complete control and deep understanding of every tool that’s being used. Using high-leverage tools allows you to build a leaner team while shipping higher-quality product faster.

Choosing great (high-leverage) tools

So how do you choose the right tools? I like to describe a codebase as a living, dynamic system. A human being is an example of such a system - our bodies are constantly going through different processes to keep us alive, and we have to consciously do work to perform at an optimal level (go to the gym, sleep properly, eat well, etc.). Similar to how your body evolves when you go to the gym, the only thing that’s constant about a great codebase is a need to evolve. This is probably a heretical statement, because I’ve seen many engineers praise great codebases for how little work needs to be done to them after things work. However, the reality of a small team or organization is that your work is just getting started, and more likely than not, as your organization grows, you will need to evolve the codebase. The thing is, surprisingly often, engineers don’t seem to have the foresight to think about what the consequences are of choosing tools that are not suitable for growing teams, evolving problems, and evolving products. The tools you choose should allow you to confidently change any part of the system knowing that things are not going to break, and to iterate quickly. The truth about an early product is that you usually can’t (and shouldn’t) be solving scale problems at the beginning. But what you do want is a setup that’s capable of being evolved/improved. You’re not going to get every decision right, but it should be fixable over time instead of feeling like it’s going to fall apart at any second. High-leverage tools that let you punch well above your weight are probably the biggest unlock an engineering team can have - the most incredible thing about choosing high-leverage tools is that they also have a much greater likelihood of attracting great engineers (ask your favorite designer if they would choose Microsoft Paint over Figma/Photoshop/Claude Design). My favorite high-leverage tool is a language with a powerful type system like TypeScript (which also happens to come with a ton of additional tooling that I also use, such as linters, formatters, monorepos, unique libraries, etc.).

A common adage you’ll see people repeating online nowadays is that the tools you choose don’t matter. All that matters is choosing tools that AI agents are comfortable with so you can go as fast as possible. I disagree with this notion. Choosing the right tools is more important than ever, and they’re not always tools that agents are trained on, but the models have gotten so good nowadays that if you choose tools with the right principles in mind, the agents can be incredibly effective and produce output that you sometimes don’t even have to review (although you still should most of the time). Not all tools are created equal (and certainly not created equal for all types of problems). One of the simplest tells that an engineer hasn’t internalized the importance of tools is when they claim that all tools are somehow equal, and that it all depends on the player’s skill to be able to turn something good or bad (i.e., it’s all relative). I’ve seen this incorrect train of thought expressed by both individual contributors and engineering leaders alike. The best engineers tend to learn (typically the hard and painful way) that tools are not equivalent commodities (which is the common and incorrect wisdom) and success is not entirely dependent on the player’s skill alone.

To accomplish the goal of choosing great tools, we need to reason through what it takes to produce good output that is reliable and correct for the sake of building a product. Engineers often do not logically handle this - choosing tools based on the vibe of their favorite language - but leading to many management headaches when in production. The core detail to get right is that you want as many feedback loops as possible to know what is happening in your code prior to it ending up in production. Vibe coding has gotten a bad rap nowadays, but the reality is that before AI agents, most engineers, especially in the web era, have just been vibe coding, using languages like JavaScript and Python with no static types or tooling that would help them check if the code is correct (and before AI, do you really think people were spending time writing tests at small companies?). The problem is that if you don’t choose good tooling that guarantees a level of baseline correctness out of the box, you will end up producing a system that is incredibly hard to fix over time. This happened to me at company after company, and I found myself repeating myself over and over. It’s been a problem at nearly every startup I’ve worked at, and I got so fed up that I had to start my own company, just to have a chance at making the decisions that I thought were most optimal (even if questionable to others).

How AI has changed tool selection

AI has changed a couple of components of tool selection:

  1. It has changed the caliber of tools that you can use
  2. It has changed the specific tools you may choose (same principles for tool selection, but different tools that accomplish that goal)
  3. It has reduced the effort of keeping up with evolving tools, as a single agent can often handle much of the migration
  4. It has made large-scale refactors/improvements/“tending to the garden” a regular practice of hygiene

Being able to pick tools that are of a different caliber than you otherwise might have been able to is extremely freeing, because LLMs can help fill in the gaps of your knowledge. There are often questions of build versus buy, or self-host versus cloud offering, and that calculus has changed with the advent of these tools. This often means you can have not just a smaller team, but significantly reduce costs because you might be able to maintain and run something yourself versus needing to use a cloud offering that is more expensive.

Call me old-fashioned, but I think that businesses are meant to be efficient, and I understand that many companies today want to grow quickly because they want to make a lot of progress in a short amount of time and tend to burn a ton of capital to do it. But on the flip side, AI tools allow you to monitor and decrease your spend in several areas. You can self-host more services that you otherwise might not have spent the time writing all the infrastructure code to do so. You have more choice in terms of hosting vendors because the LLM can help set you up (and write all the infrastructure code for anything automatable). This also allows you to deploy tools that give your entire team the ability to answer their own questions and free up your time. Less time spent context switching and answering your teammates’ questions about how some feature works or about some data in your database!

Some of our highest leverage tool choices

Concretely, I think it’s worth talking through some of the technical choices we’ve made that would probably be somewhat unorthodox in case people are looking for tactical tools they should be using. Our tools:

  1. Increase the odds that AI can produce reliable output on the first try
  2. More importantly, when it doesn’t produce correct output, it has the right feedback loops in place to fix the issues before ending up in production

This is called “shifting left” in engineer parlance, where you want to have as many layers of defense as possible prior to something getting into production and affecting real customers. You don’t want to find out all of your problems only when a user tells you in production. Of course, all software is going to have issues, and it’s very hard to build a system with perfect reliability, but we can aspire to find the right balance between moving fast and using tools that allow us to build for correctness, even if it means we have to go a tiny bit slower in the short term to set up the tooling (positive compounding is often a better choice than short-term junk food, even if the progress looks slower initially).

One of our highest-leverage tools has been Effect. It’s been gaining traction recently, but I started using it back in 2023 when it had no documentation and a small group of people who were interested in functional programming ideas brought to TypeScript in a practical way. It turns out that if you see that vision through, you get a library that is extremely powerful and can add value in almost every part of the product stack and gives us useful feedback as we build and change the product.

The reason Effect is so powerful is because it contains a lot of information that is useful feedback for the LLM (and you, the human). The interesting thing about using a tool like Effect is that it gets the LLM to produce more reliable output out of the box. This is an extremely underrated part of using Effect. You’ve probably noticed when writing typical TypeScript/JavaScript/Python that the LLM will suddenly decide to add some poorly written hack, like adding a timeout instead of doing some specific business logic. In my experience, a consistent codebase written in Effect makes it easier to steer the LLM away from those patterns. There seems to be some merit here in using good tooling across our codebase, which the LLM will continue to propagate.

We stopped using Tailwind and started using Vanilla Extract combined with Storybook driving our components (from primitives all the way up to entire forms) in React because it allows us to use the full power of CSS typed with TypeScript. LLMs can also test the components themselves. One of the things LLMs suck at is building consistent UI that works across your app. With this setup, you now have a UI-based feedback loop that is extremely powerful. Someone who is a product designer or frontend engineer will quickly notice the inconsistencies that an LLM can produce if not taught/conditioned to follow certain guardrails.

Along with this, we use linters, tests, etc. to give LLMs as many feedback loops as possible to know that the work they do is closer to being correct. Make sure to do code review in CI! That catches a lot of issues. We built our own review setup in-house combined with some popular code-review tools. A lot of this tooling is evolving rapidly in the TypeScript ecosystem due to AI making it easier to build developer tooling, so this post might already be outdated by the time you’re reading this.

With that, the LLM can:

  1. Build things across the stack
  2. Diagnose issues anywhere in the stack
  3. Test as much as possible locally, and if not, in our isolated preview environments

Managing technical debt

There are primarily three types of technical debt in the early days:

  1. The “debt” that’s incurred because you’ve learned or found a new tool/library/language that’s better for solving your problem than what you currently do.
  2. The debt that’s incurred from knowing you wrote code in an ugly/hacky way the first time to move faster, but at the cost of requiring cleanup or needing to be completely redone long term.
  3. The debt that’s incurred from the system not being designed in the optimal way to continue producing based on your users’ and company’s needs, which you may only find out later.

Pre-AI, one of the fundamental problems of engineering at most companies is that a lot of engineers develop this strange sense of learned helplessness where they see that the system (they often built) sucks and produces lots of errors or issues, and they become numb to it. In the age of LLMs, we can decide we want to make things better, and it should be a personal mandate to do so. The world improves as the quality of our software improves, and so much software in the world is in dire need of improvement. Hopefully, with AI tools, we can at least increase the baseline quality of how software is written. Even now, you may have noticed that many of your regularly used apps have gotten UI refreshes or other improvements. I hope that AI tooling is used to keep building better software, although there is still plenty of slop to wade through in the meantime.

Looking forward

There’s been a lot of talk around building a great software factory where functionality gets built with minimal to no review. I’m not sure we have gotten there yet. There’s still a lot of slop and unnecessary output, and so some amount of manual code review is necessary, especially when building mission-critical features. I would caution against merging things as a small team without any sort of review. LLMs ultimately give you more freedom to make choices.

I used to get into the flow state when coding, and it was a big part of how I thought about my own identity. I used to think of myself as at least a half-decent software engineer who knew how to make good technical decisions (even if I didn’t write the fanciest code). That decision-making has significantly amplified my ability to use LLMs. The downside is that I no longer feel emotion around code the way that I used to (whether it was the rush of something working or the sheer pain of debugging something difficult), and that has definitely changed my sense of how to do work. It is kind of boring sometimes to feel like I’m a glorified LLM prompter. I’ve learned that my job is more like an investor or manager now, where I need to absorb information, learn things, think about ideas, and make bets that change the future. We now can have a smaller team and have the freedom to make progress without sitting in front of a computer all the time (although herding these agents can feel like it at times). I wonder if herding agents is a better life than entering keystrokes on a keyboard day in and day out, hoping that your company will someday work. Maybe it is a better life than trying to figure out what obscure errors mean instead of solving problems for real people.