Selecting a startup stack for scale

“Do things that don’t scale.”

That YCombinator mantra may be the single most common piece of advice given to early-stage tech startups.

And with good reason – it’s great advice! But if you follow it, with a little luck you’ll reach a point where it no longer applies (at least on a technical level).

Of course, “Do things that don’t scale” is advice about how to grow an early-stage startup, not advice about how to architect your SaaS application. But the reality is that many SaaS applications are initially built with a tech stack that doesn’t scale, or doesn’t scale evenly. In the early days, founders are understandably concerned with getting the application to a working MVP, not trying to ensure it can survive 100,000 concurrent users before it even has 100.

Once you’ve passed that point and are building a growing user base, though, it’s time to reevaluate your tech stack with an eye on how things scale.

Often, that evaluation will reveal that you’ve got some technical challenges to wrestle with.

Technical challenges at a mid-stage startup

The crux of the problem is that at a certain point, how well your tech stack can scale starts to matter, even if it didn’t at first.

Precisely when that point comes in a startup’s growth trajectory can be difficult to predict, particularly in the context of SaaS startups. A business with slow-but-steady month-over-month growth can see its subscriber count explode overnight because the right person raved about it to their audience. But if there’s a bottleneck somewhere in the application’s tech stack that prevents it from being able to handle the sudden increased demand, performance will suffer, driving away new and old users alike.

Replacing any part of your tech stack can be painful. But if it needs to happen eventually, sooner is better than later. You do not want to find yourself in the position of having to change a critical element of your stack on the fly because it can’t keep up with your growth.

At the same time, though, the upgrades and additions you can make to your stack are likely to be limited by a number of factors, including:

• Financial considerations. Paying for too much scale too early can burn up your runway quickly.
• Developer ability. If adding new tech to your stack requires new skills, that’s something you’ll have to build in learning time for (or hire for).
• Talent acquisition. The best engineering talent tends to be attracted to companies that are building with interesting tech.
• The product roadmap. The desire to upgrade elements of your application’s tech stack may have to be balanced against your product roadmap, since any time developers spend adding features is time they cannot spend implementing new things into the stack.
• Geographic scale. Depending on your application and your audience, you may need to think about moving elements of your application architecture geographically closer to users to reduce latency. The extent to which this is an important consideration at this stage will depend on the specifics of your business and your target markets.

When confronted with the complexity of assessing and prioritizing what to replace, it can be tempting to simply do nothing, and worry about replacing things only once they’ve genuinely become a bottleneck. But unless your budget prevents you from acting sooner, it is almost always going to be less painful in the long term to refactor your stack for scale before you’re actually hitting those bottlenecks.

An important question to answer is thus: what are the highest priority upgrades? What elements of the stack are most likely to create bottlenecks in the short- or long-term?

Those are questions you’ll have to answer based on the specifics of your application. But once you’ve decided to switch out a part of your tech stack, you face another challenge: what to replace it with?

Here, there are some general principles that’ll apply to most startups at this stage. Ideally, you’ll want to find something that:

• Can scale but doesn’t cost much before you need to scale.
• Has a shallow learning curve and won’t eat tons of engineering time.
• Gives you as much flexibility as possible to make future changes.

The ideal solution

Whether you’re looking at upgrading a few elements of your tech stack or completely refactoring an early MVP, there’s a good chance that your search for scale will lead you in the direction of a microservices architecture.

This approach can make it easier to achieve the elastic scale and flexibility that many startups need, but it does also come with some challenges to be aware of. For example, breaking large services down into smaller microservices introduces more individual points of failure. It also increases the complexity of communications between services, and the operational complexity of monitoring each of these services for performance, downtime, etc.

In most cases, though, these challenges are offset by the benefits of microservices architecture, especially when moving to microservices allows you to switch to pay-for-what-you-use billing and/or managed services that can take challenging and time-consuming ops work off of your team’s plate.

Every company is different, and there is no one-size-fits all stack that’ll work well for every startup’s application and budget. But there are some types of services that are particularly worth considering when evaluating your tech stack and this stage – services that can offer elastic scale while minimizing costs and ops requirements.

Specifically, it makes sense to look for things that are:

• Managed. Managed services can look more expensive, but when you factor in the costs of hiring or training up in-house experts and the time required to manage those services, going with managed can end up being the more affordable option.
• Serverless. As many offer usage-based billing, serverless services can be a best-of-both-worlds solution for many startups: available scale when you need it without the high cost of dedicated machines when you don’t.
• Cloud-native. Modern applications need to run on the cloud, and trying to retrofit cloud support onto products that weren’t designed for it is rarely the best use of your engineers’ time.
• Flexible. It’s good to look for services that facilitate rapid iteration and make deploying changes easy. If it’s possible to avoid cloud vendor lock-in, that’s also worth considering, as the best cloud for you right now may not still be the best choice when you reach enterprise scale.

A specific example: the transactional database

Let’s take a deeper look at a specific example to see how some of these principles can actually be applied in the real world. We’ll focus on the transactional database.

Most startups build their MVPs using an RDBMS that’s free and familiar – something like MySQL or PostgreSQL. These databases are perfectly serviceable, but they weren’t built for scale, so when you start to need scale, you’re confronted with a few potential options.

Option one: You can try to modernize your existing RDBMS. This is possible, but doing something like sharding a Postgres database places a heavy load on your engineering team, and at the end of the day, it still might not work.

That’s a lesson that SaaS education company Kami learned during the pandemic – they tried to scale up their Postgres database using sharding, but it simply couldn’t keep up with the traffic increases they were seeing. Ultimately, they had to hot-swap in a different solution (we’ll get to that shortly).

Option two: You can switch to a NoSQL database such as Cassandra. This solves your scalability problem, but it also introduces some new ones. For example, you’ll have to move some of the schema that your old Postgres database enforced into the application. Consistency problems can also arise in NoSQL databases, which generally don’t provide ACID transactional guarantees, and working around those problems can be a heavy engineering lift that results in restrictive solutions like the use of CRDTs.

Option three: You can switch to a Distributed SQL database, which offers the best of both worlds: the easy elastic scale and cloud-native nature of NoSQL solutions combined with the familiarity and consistency of a traditional SQL RDBMS.

This is the option that Kami ultimately chose, moving from sharded PostgreSQL to CockroachDB, a distributed SQL database that’s Postgres wire compatible, which enabled them to swap it in without having to do a ton of refactoring or endure significant downtime.

Another advantage of CockroachDB from a startup’s perspective is that it offers flexibility, both in terms of clouds and in terms of its own consumption models. While its pay-for-what-you-use Serverless model is ideal for many startups and SMEs, it’s good to have the flexibility of being able to switch to a Dedicated setup, which is often the preferred option once you reach enterprise scale.

The transactional database is just one example, of course. Building your entire application architecture with tools that balance your requirements for scale, budget, and time can be a delicate balancing act. But the proliferation of serverless tools are making it easier than it used to be to build an application with a stack that scales on a startup budget. If you’re not taking advantage already, maybe it’s time to give serverless a try!