.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
Polyrepo vs. Monorepo - How Does it Impact Dependency Management?
In this article, we explore the impact of using a monorepo vs a polyrepo architecture on dependency management.
In this article, we explore the impact of using a monorepo vs a polyrepo architecture on dependency management.
In this article, we explore the impact of using a monorepo vs a polyrepo architecture on dependency management.
In this article, we explore the impact of using a monorepo vs a polyrepo architecture on dependency management.
In this article, we explore the impact of using a monorepo vs a polyrepo architecture on dependency management.
Introduction
When starting Endor Labs, one of the first questions we asked ourselves was “Should we build with a monorepo or a polyrepo architecture?” We ended up choosing a monorepo for multiple reasons, dependency management being a major one. In this article, I will explain the impact of using a monorepo vs polyrepo architecture on dependency management.
The purpose of this write-up is not to cover every detail of the polyrepo vs monorepo debate, but to specifically look at how each architecture impacts dependency management.At Endor Labs we believe that better dependency management makes development faster, while reducing risks.
First, let us introduce some basic terms:
Glossary
Monorepo: a version-controlled code repository that stores all of the code, projects, and libraries of the organization.
Polyrepo architecture: several version-controlled repositories for code, projects and libraries of the organization.
Dependency: a third party library your code uses. For instance, in golang it could be an open source library imported from github, in java it could be a jar you are downloading from Maven etc.
SBOM (“software bill of materials”): a list of all of the open source and third party components present in the codebase.
Poly vs. Mono - How are dependencies handled?
Dependencies are generally defined in a dependency file, such as a go.mod/go.sum file or a pom.xml file in java. These files are used to describe which dependencies and versions that a repository depends on.
Both polyrepo and monorepo environments generally handle these files differently.
In a polyrepo architecture, each repository will usually contain its own dependency file.
In a monorepo, usually the repository will have one dependency file per language to support every project within the repository. This concept is illustrated in figure 1 below:
So what are the effects of having one dependency inventory rather than several dependency inventories?
With a monorepo, there is one unique SBOM
Having a monorepo makes it straightforward for a company to understand which dependencies and versions are being used because information is centralized and available at a unique location. There is only one version of a dependency used at any given time, and that’s it!
For instance, in Figure 1, it’s easy to know which dependencies and versions are being used (here github.com/protocolbuffers/protobuf-go@v1.28.0, github.com/google/go-cmp@v0.5.5 and github.com/golang/protobuf@v1.5.0).
Since this information is centralized and unique, only one SBOM will be generated from a monorepo.
With a polyrepo architecture, a company would need to go through all of its repositories to know which dependencies and versions are being used. This can become a tedious operation if a company owns hundreds of repositories distributed in different source-control systems.
In comparison to monorepo, multiple SBOMs will be generated from the polyrepos.
With a monorepo, dependency ownership is different
In a polyrepo architecture, a group of developers will often own a repository. Multiple teams may own separate repositories as illustrated by Figure 2 below.
Each repository’s owner will be responsible for measuring the impact, both positive and negative, of a dependency update. As a result we may have three different groups of developers owning the same dependency (for instance github.com/protocolbuffers/protobuf-go).
With a monorepo, since a dependency is only defined once at a unique location, it is possible to define one unique group of developers (Team A) that owns the dependency. For instance, we could define a group of developers that would be responsible for maintaining github.com/protocolbuffers/protobuf-go. However, there is no tool used in practice today to achieve this. As a result, Endor Labs has taken on the responsibility to build this tool into reality.
Monorepos make actions scale more effectively, while polyrepos isolate impact
Let’s say a new critical vulnerability has been found in one dependency. Company leadership is now asking you to update repositories that use this dependency as soon as possible.
Having a monorepo makes this operation scalable and impactful, you will only need to update the version of this dependency in one location and all of the code will start to use the fixed dependency version.
In a polyrepo architecture, this can be tedious. The bigger the company, the more tedious it becomes, or even impossible. Figure 3 displayed the following workflow:
- Find which repositories depend on the affected dependency.
- Ask owners to update the dependency file of all repositories affected.
If this is an emergency, and the update of the dependency must be done as soon as possible, monorepo will allow you to mitigate the issue very quickly in comparison to the polyrepo architecture.
However this comes with a catch, what would happen if the impact of the action is negative and not positive?
Let’s say you are updating this dependency and there is a regression (for instance one of the APIs does not return the same output anymore).
Thanks to the decentralized architecture of a polyrepos, updating of the dependency will be less impactful for the company. Since each update for each repository is controlled by different groups of developers, the dependency will be incrementally updated. Instead of having a negative impact on all of the code of the company, only a subset of the company’s code will be affected by the regression.
On the other hand, a company utilizing monorepo architecture for their code will suffer greater negative impacts by the update. However with a monorepo, all of the company’s code will be affected by the update, and this can have a huge negative impact. Dependencies in a monorepo are not lightweight operations; nonetheless, appropriate tooling (linting, performance testing, security checking, etc.) can overcome these constraints.
But is a monorepo always more scalable?
As mentioned before, updating a dependency in a monorepo is not something that can be done carelessly. It becomes even more complicated if the updated dependency results in a regression.
For instance, let’s say we have two projects dependending on Lib A (Figure 4):
Now when updating the dependency, you realized that Project 1 can not be built anymore! Lib A did not respect semver (https://semver.org) and broke one of its APIs.
With a polyrepo, Project 2’s owners will not have any trouble updating Lib A. However, in a monorepo, Project 2’s owners will be blocked until Project 1’s owner fixes the issue (as shown in Figure 5). This is an unfortunate situation since it decreases the development velocity of Project 2 and adds friction between teams to move forward.
Conclusion
Choosing between monorepo vs a polyrepo architecture is not a binary choice, both have their own benefits and drawbacks. Some people hate monorepos while other people love them!
There are a lot of factors involved (size of the repository, number of teams involved etc.), but with the right tooling we believe that Monorepos will increase the development velocity of your teams and make dependency’s updates more impactful. With a polyrepo, the decentralized aspect of a polyrepo will alternatively decrease development velocity and will make dependency’s updates less impactful.
At Endor Labs, we believe that choosing a monorepo better fits our engineering culture and will allow us to deliver our product faster. We are aware of the limits of monorepo and we are closely monitoring which dependencies are going into our base code. Monorepo struggles once a “bad” dependency goes in. And this is one of the challenges we are working on here!
TL;DR?
- Dependency inventory is centralized with monorepo while it is decentralized for polyrepo. Without tooling, dependency inventory is hard to achieve with polyrepo.
- A monorepo will generate one SBOM while a polyrepo architecture will generate several of them.
- Ownership of dependencies is easier to achieve through a monorepo than a polyrepo.
- Monorepo will provide a more scalable way of maintaining dependencies than polyrepo.
- Impact (negative or positive) of a dependency update will be larger with monorepo. Updating a library is not a lightweight operation in monorepo and appropriate tooling should be used with monorepo.
- Dependencies not respecting semver (or with y regressions) will take more time to be updated in a monorepo.
Alexandre Wilhelm is a Founding Engineer at Endor Labs. As the first engineer in the company, Alexandre built the foundation of the backend platform and set up monorepo at Endor Labs. He is now focusing on developing solutions to resolve callgraph and bom for golang. During his free time, Alexandre loves hiking in the mountains, especially exploring the Sierra Nevada in California.
Links
- What is monorepo? (and should you use it?) - https://semaphoreci.com/blog/what-is-monorepo
- Monorepo vs Polyrepo - https://earthly.dev/blog/monorepo-vs-polyrepo/
- Dependency Management from Software Engineering at Google - https://abseil.io/resources/swe-book/html/ch21.html
- The Issue with Monorepos - https://www.squash.io/the-issue-with-monorepos/
- What is an SBOM? Linux Foundation - https://www.linuxfoundation.org/blog/what-is-an-sbom/