“$10 million?!” We felt like shit.

We had just learned our competitors raised $10 million. Worst of all, they were just taking money off the table and not investing it in the business. This was a signal that they were profitable, healthy, and had more than enough capital without VC money to be competitive.

Meanwhile, we were trying to scrounge together a $5 million round and having a hard time getting through. We were burning cash.

Our competitors were outmaneuvering us both in terms of winning customers and securing financing. It sucked.

How did we get here?

In short, we weren’t being particularly thoughtful about how we were building the product. We coupled product and technology decisions together, causing us to always fall behind.

It’s taken me some time to reflect on the mistakes of that time and correlate them with other successes and missteps throughout my career.

A successful product must be highly precise, impervious to failure, delightful, and profitable. Creating and extending successful products requires effort.

As we build technical products, we can change our behavior as creators to increase the chances of success.

The following is a blog post that’s taken me a few years to write down. It captures some of the better and worse decisions in my career. This framework is my distillation of “Oh, that’s why that worked.” I hope you find is as useful as I have.

Product and Technology

When it comes to creating software products, we have two axes: product and technology.

Can we create a viable business with interesting product choices X and Y? Does this new technology unlock previously impossible or uneconomical opportunities? Will introducing a new technology into this existing product drastically improve maintenance costs?

These are just some of the high-level questions we ask when we look at these two axes. They are interrelated but ultimately distinct. Technology is an enabler for the product. We can reason that most consumers don’t care about the technology choices of a product: they just want it to work.

How can we reason about reliably delivering the most value to our customers consistently?

The Framework

The framework’s goals are as follows:

  • Optimize for delivering value reliably to the customer.
  • Minimize execution risk of new product offerings.
  • Ensure that the company does not end up with a tech stack that is difficult to maintain or hire into.

We have four quadrants:

  • Replacing. Unknown Tech, Known Product. Assumes that the product is a financial success, well understood, relatively stable. We need to swap something out.
  • Out on a Limb. Unknown Tech, Unknown Product. Uhoh. We’re taking on a lot of risk. If the product is a hit, we have some execution risk. As we respond to customer demand, we will also be learning how our new technology behaves under stress. If the product fails, all accumulated learnings might be worthless.
  • Stasis. Known Tech, Known Product. Stagnation risk, but the business is stable. It’s a predictable revenue stream.
  • Exploring. Known Tech, Unknown Product. We are venturing off into new territory and we don’t know if the market wants our offering at all. We need to minimize the unknowns and want to chose proven technology that the company has a strong familiarity with

Let’s take a look at each quadrant in more depth.

Replacing: Unknown Tech, Known Product

If the code didn’t serve a purpose, we would just delete it. —A wise programmer

Primary Risk: Integrating new tech into the old product is difficult.

Every organization has “legacy code.” Interpretations of exactly what “legacy” means vary. It could be code that is so old that folks are afraid to touch it, or it could just be code that’s deprecated that will be removed shortly.

Whatever the interpretation, legacy code has one thing in common: It cannot be safely deleted.

If the code could be safely deleted, we would! But we don’t quite know what would happen if we did.

Here lies fertile ground for experimentation with technology. We have a well-understood product with clear acceptance criteria. The business has one requirement: Please don’t break it.

Personally, I’ve found these parts of a code base a great place to use refactoring techniques to improve my own understanding before making substantial changes. Here too is where we should look to deploy new, unproven technology in a controlled manner.

As we slowly integrate the new technology, we can see how it performs under the stress of customers and the business. Does it make things easier to change? Does it make concepts easier to understand for a new member of the team? Does it introduce new performance characteristics that can might unlock a new product line? Does it solve the problem?

When experimenting with new technology, we look for the most stable parts of the product.

Out on a Limb: Unknown Tech, Unknown Product

Highway to the danger zone. Gonna take you, right into the danger zone. —Kenny Loggins

Primary Risk: Coupling. Success of the technology is coupled to the success of the product. If one fails, so does the other. These risks compound.

In a science experiment, we try to use as few variables as possible. An experiment with fewer variables is cheaper to run. Our control groups can be more precise, and we can be more confident that our changes cause the results we observe.

Given the choice, we should look to take a similar approach to developing products. Playing with multiple variables is akin to playing fire; we will likely get burned.

Why? As outlined above, we have different success criteria for product and technology. For a product, the market decides. For technology, our ability to operate it decides. The market doesn’t care what technology we use, and our technology doesn’t care if your aunt is using the product.

Bundling these two together is something I’ve personally done and I continued to see throughout my career. I’m not quite sure where it comes from. Is the boredom associated with working in the old stack? Is is the fear of being made obsolete? Or is it the thought that “Well, this is new so let’s make everything new”?

There are a few rare success stories out there, where companies threaded the needle to introduce new technology and a new product simultaneously.

But usually, we have high coordination costs. We can’t advance the product functionality until we figure out how to make the new technology’s test framework run. We’ll be overbuilding our the technology side if we don’t wait for feedback on our nascent product. It takes a lot of courage in this situation to say, “How much faster could we go if we used what we were familiar with?” Hopefully the decisions we’ve made to date are reversible, even if we feel we have a substantial sunk cost.

Stasis: Known Tech, Known Product

If it ain’t broke, don’t fix it. —Someone optimizing for quarterly results.

Primary Risk: Stagnation.

Every product becomes old eventually. It’s remarkable how quickly something like the original iPhone feels heavy, underpowered, and low-res. The default state of any product is decay.

The same is true with the technology. Fashionable technologies come and go. Java and C++ may have been in their heyday in the 90’s, Python and Ruby in the 00’s. The only thing we can be sure of is that the same languages and technologies we use today may not be what we use tomorrow or next year.

Are these problems? Depends on who you ask. If you ask few souls that are production COBOL experts, maybe not. They can charge a hefty fee to keep that bank running. If you ask the poor soul tasked with recruiting COBOL programmers, they might have a different story.

An aging product might be just fine, provided there is no competition. Not many, if any, products exist naturally without a single competitor. Products locked in stasis will eventually be yesterday’s product. This will be a business problem if there is not a revenue stream to replace the one of the aging product.

Products in stasis are a long-term risk, but do not pose a risk short-term.

Exploring: Known Tech, Unknown Product

When all you’ve got is a hammer, everything looks like a nail. Good thing we have a lot of nailing to do. —Me, justifying my use of Ruby on Rails

Primary Risk: Inability to find product-market fit.

Willing a new product into existence, whether inside a company or as a startup, is a monumental task. The landscape is littered with failure. The default is death.

Within a large company, it can be difficult to tell if a new product is succeeding or failing. A medium success with a middling number of customers is the worst. Do we kill it? Do we invest in it? Pivot? Or persevere?

Creating a new product or feature is fraught with market risk: Will people pay for this? Experimentation and ability to make substantial tweaks quickly is key requirement for the product’s success.

For engineers, experience in a technology is one of the sources of confidence. “I know that Redis will be good enough for what we’re trying to do here. I’ve used it for the past 7 years at larger scales with more throughput. Plus, I can get it stood up in an hour.”

When creating new products, we therefore look to lean on our known technology.

Case Studies

I’m calling these case studies mostly because they aren’t scientific. Take them with a grain of salt. You may also refer to them as anecdata.

Executing — Known Tech, Unknown Product

Interestingly six of the top ten built on Ruby! —AlchemistCamp on Y Combinator Top Companies List 2019

I try to not read the Hacker News comments too much, because I have other hobbies. But this particular comment stood out to me as interesting.

As a long-time Ruby apologist, I was happy to see a few home team wins. I’m also currently employed by one of these top 10 companies at the time of writing, so my paycheck comes from Ruby being successful. Caveat emptor.

But even more interesting is that Ruby popularity peaked in 2008 and has been waning since then. All of the Ruby-based YC companies were created after that. I interpret this as “We’re going to create a novel product, let’s choose technology we are familiar with.”

Now, YC doesn’t have a monopoly on successful companies. Other successful companies have been created in the same time frame and not gone through YC. Many of the companies on the list no longer exclusively write Ruby.

Having started a company myself back in the day, I realize that worrying about unproven technology is the last thing you want early on. Keeping the lights on and finding customers take a much higher precedence. Early products require a customer obsession.

When optimizing for customer obsession, some successful companies seem to choose known technology.

Replacing — Unknown Tech, Known Product

The company I founded, LayerVault, aimed to be a GitHub for designers. We never quite found product-market fit, and eventually went belly up.

So it goes.

The company served as a great education for me and hopefully for others on the team as well. One of the lessons learned was how and when to introduce a new technology.

We stored a lot of metadata to make the product work. For every Photoshop and Sketch document, we would retain all sorts of layer information. This allowed us to do some slick stuff like telling you which layers changed when.

We chose boring technology, so we had all of this stored in MySQL. At some point that table—which was really just a single TEXT column of JSON—grew to be 95% of the whole database. We had vertically scaled to the largest instance Rackspace would give us. The table was 100’s of gigabytes, if memory serves me correctly. It was clear that the solution was not tenable even for our small customer base. Because of this, using the product during business hours was not a great experience.

So we decided to find something new. We knew how much data was generated per day, the average size of each document, and the expected customer experience. We evaluated a few document storage options and landed on Cassandra. A week or two later, it was running parallel and dark in production. Things looked good, so we cut over. And then…

Nothing happened. Well, nothing bad happened. Lots of good things happened. Our primary MySQL database breathed a sigh of relief, response times dropped across the board. We dropped the old table and bought years of headroom. We could add instances to our Cassandra ring as necessary to scale up total capacity without downtime.

Most successful uses of new technology in my career look like this. Few endorphins, little excitement. Just a sober read of the situation, a search, and then a rollout. Not much fanfare. We solve technology problems while keeping the product stable.

Out on a Limb — Unknown Tech, Unknown Product

Netscape 6 is canon here: “Let’s rewrite the app (unknown tech) while staying ahead of competitors (unknown product).”

More recent examples and sentiment come from folks like Dan McKinley with his talk “Choose Boring Technology.”

Drawing from personal example, I once worked at a company with more programming languages in production than engineers. We would constantly couple product improvements to new technologies. We couldn’t improve the product without upgrading the technology, or so we thought.

We were disproven by a competitor time and time again. They had a Ruby stack and were constantly winning customers. We thought more sophisticated technology would prevail, but we were proven wrong time and again.

I could go on and find more examples, but I’ll stop here. Choosing to iterate on the product and technology simultaneously is putting your team Out on a Limb. It’s possible to succeed, but much less likely. It could be a wild success, but it most likely will not.

Are you willing to answer that risk?

Further Reading

These ideas draw a lot on others in the industry. Here are some links to people and things that directly influenced these ideas:

Special thanks to Mark Wheeler, Kent Beck, Emily Field, Matt Blake, and Ian Smith for reading early drafts of this post and providing feedback on the early idea.