Bazel Is Not For You

I know the title sounds like clickbait, but hear me out.

I make a living helping people with their Bazel builds. My ability to put food on the table is directly correlated with more people using Bazel. I’ve staked my career in the fact that people need great builds, and that Bazel is a great way to get there.

So please believe that I don’t say this lightly: Most organizations shouldn’t be using Bazel.

Here’s what I’ve observed, after almost a decade knee-deep in the hermeticity trenches.

Point 1: Never Start With Bazel

Please, please don’t start a new project with Bazel. Unless you’re absolutely sure that your organization is going to grow to hundreds of engineers, just stick to whatever Hello World your language of choice gives you.

Let’s really think about it. When you’re starting out in a project, you have Big Questions to think about: What am I building? Is this even worth building? Should I be building this? Does anyone want this?

Compared to those, “should I strive for hermetic builds to gain 50% faster CI?” shouldn’t even qualify as a nagging thought. Any extra effort you spend on finding the right ruleset, or wiring toolchains together, or anything that is not absolutely required to get the thing running is time you’re taking away from the things that will make your project succeed. Just pick whatever build tool is in vogue for the language of your choosing, and move on. Until you get to ~100k lines of code, you’re not going to notice the difference in build performance anyway.

The only exception I’ve seen, the only case where I would recommend starting with Bazel, is any time you really care about which compiler you’re using. Mostly, this applies to embedded systems, and systems that need to cross-compile to target a specific, esoteric, and custom system. In these cases, you’re already opting into build pain, so it makes sense to pick a tool that helps you surface, reason about, and manage that pain, like Bazel.

But if you’re building a web service, even in Rust, please just use Cargo.

If the project gets big enough, we can talk about migrating to Bazel. But even then…

You Need At Least 6 People In Your Bazel Team

After working with dozens of teams and organizations, and gathering hundreds of second-hand data points, I observed the following: The teams that struggled with Bazel the most were the smallest ones. This appears pretty obvious, until you couple it with the fact that this held true, regardless of the overall size of the org they were serving.

So, a team of 2-4 people struggled to serve 20 engineers just as much as 100, whereas I’ve seen build teams of 8-10 serve companies of 1000 engineers comfortably. To put it in other words: Bazel’s returns on investment are non-linear. Or even better, to put it visually:

This seems to indicate that, unless you’re ready to dedicate a minimum of 5-6 people exclusively to becoming Bazel experts, you’re going to have a bad time, and should probably stick to native tooling.

Which is my main point: Most organizations aren’t ready to have 6 full-time Bazel engineers, so they just shouldn’t do it.

Wait, Why Does This Happen?

To be honest, I don’t know. I have enough data to say that it is true, but I only have guesses as to why.

Anecdotally, it seems to me that:

• There is always a minimum of work involved in keeping the Bazel lights on: You need to make sure the rulesets you depend on are up to date (and contribute fixes if not), you need to keep up with Bazel versions, figure out what the heck to do with protobuf, that sort of thing. This is a constant of work that (1) will probably be unique to your organization, and (2) doesn’t really depend on the size of the org.
• You may find yourself maintaining infrastructure. Bazel’s promise is that it’s so hermetic you can have an artifact cache in the cloud to avoid 80% of your build actions. Well, since you did all the work to migrate, it stands to reason that you should stand up a cache to reap the benefits, right? Well, keeping a live service up is a non-trivial task. Suddenly, you have oncall rotations, and outages, and you need to procure capacity from whoever manages your clusters, etcetera. Build cache services are relatively easy to maintain, but even in the best of days they take a non-trivial amount of overhead¹.
• You get more customer support requests: Most engineers out there are very comfortable with native tools, and have no idea Bazel exists. When something breaks in cargo, their first instinct is usually to Google it, and it’s likely they’ll find a solution. When something breaks in Bazel, even if they try to Google it, chances are they’ll end up just reaching out to your team.

So, yeah. Unfortunately, I’d recommend your team be at least 6 tall to ride on the Bazel rollercoaster.

Your Project Is Mostly In One (Modern) Language, And Your Compile Times Are Fine For Now

I hope I don’t have to convince you that Bazel is much harder to maintain than any other mainstream, modern build system.

Even further: Despite the recent leaps and bounds made by the Bazel community in usability, I think it will always be harder. After all, Bazel needs to do exactly the same work (calling the same compiler with the same args as any other build system), but it needs to know about more stuff (e.g., Bazel really cares about which CC toolchain you have installed. Most build systems don’t).

So, if your codebase is:

• Mostly in one language,
• compiles from scratch in less than 30 minutes²,
• you can set up a brand new development laptop in less than a day, and
• you can be confident that a green test is truly green and not flaky,

I’d say just don’t bother – Bazel will probably not transform your build enough to be worth the effort.

Put another way, Bazel is the best build system to migrate to when you have no other choice.

And anyway…

Your IDE Support Will Likely Get Worse

Here’s a fact that has become obvious after 10 years in DevEx: Developers care far more about their local environment than they care about CI (and most other things)³. To prove the point, here are two situations I have found in the past:

1. The CI server is so unreliable that 10% of the jobs just failed for no reason, and engineers need to re-start a 2-hour job.
2. Every hour, the IDE just crashes randomly and developers need to take 10 seconds to re-start it.

Most often, (1) will be taken as “par for the course”, and may just be joked about in private Slack channels, whereas (2) will have angry developers sending “friendly pings” to your DMs like you cut off the electricity and are holding their PRs hostage.

Unfortunately, IDE support for Bazel has been a huge pain point for as long as I’ve been in the community. When I was a maintainer of the IntelliJ plugin, I saw first-hand the great work and passion excellent engineers are putting in to bridge this gap. However, the gap remains large, and I’d argue that it will continue to grow.

So far, there may be 10 or 15 engineers in the world working full time to make IDE support better in Bazel. No matter how good they are, that is just not comparable to the army of engineers working on more mainstream extensions. If Bazel could integrate with existing language servers, this issue would be ameliorated. For now, Bazel will continue to run into the N+M problem, and IDE support will continue to lag behind the mainstream.

So, if your developers really value their local workflows, please consider that before migrating to Bazel. There are very few ways to lose developer trust faster than telling them they can’t use their favourite editor.

Is It All Ash, Then?

“So Borja,” you may very reasonably ask. “When should I use Bazel? Surely, you wouldn’t stake your career on a build system you know has no use, right?”

Good question! Of course, I think Bazel is a wonderful tool… to solve a certain class of problem. There has been a lot written about why Bazel is cool, so I won’t elaborate. But TL;DR if your organization:

• Is ready to dedicate around 6 people to a full-time build team,
• Suffers from multi-hour CI builds routinely,
• Needs a lot of language interoperability, or works on old languages without modern tooling support,

Bazel might be a good choice for you.

If you’re working on something that requires cross-compilation, or otherwise need to think hard about your CC toolchain, Bazel might be for you.

It’s still unclear without knowing your specific problems.

In all other cases, please just use {cargo,uv,pnpm,go,...}.

If you’d like a second opinion, get in touch! I’d be very happy to chat about your organization, your problems, and do my best to help you decide. And if you’re in the middle of a Bazel migration and find yourself underwater and understaffed, there is hope! This post can be a good tool to secure more headcount.

The important thing is to emphasize that Bazel’s problems are the same size, no matter the size of your org. One or two more people can allow you to serve your organization now, and scale the organization to thousands of engineers. You may want to create a slightly more professional-looking graph though.

We’ll return to our regularly scheduled tips and tricks soon.

– Borja