“Make vs buy” is a common decision that software engineers make when designing a new solution. Most software programs need to store some data for the user, and in my earliest days we used to use “files” for this with some format. Maybe comma, tab delimited or even some custom format that was most efficient for the operation. Back in the day, there was no internet and no “package managers” from where you can download libraries, so one had to entirely code the solution from the set of libraries your programming language / platform shipped with.
This was the instinct I grew up with. Need a web framework, make one. Want to run a multi-tenant cloud, make it. Want to manage SAAS billing, build it. Same with online learning, team collaboration, analytics etc. At Frappe we have built a lot of stuff in-house, that most software architects would never do in this day and time. This is because it is much easier to either download libraries or just use 3rd part SAAS tools to do your work. Once “make” was the default way of building software, these days it's “buy”.
This post is not about which one is better (there is no clear answer, and it's both), but the fact is that today buy is the dominant model of software development and there is very little opposition to this. This makes perfect sense in one way. Why reinvent the wheel? Why build something that people have already perfected? The availability of open source software and the explosion of projects that cover every niche in engineering makes it easy for engineers to go and pick things “off the shelf”. And it is free. If free food is available, why bother cooking your own, right?. Good engineers pride themselves not in writing lines of code but getting the job done in “as few lines as possible”. The whole point of inventing computers was to help us from solving the same problem again and again. Every engineer is brought up to do the same.
Couple of days ago I was chatting with one of our engineers who explaining their team's project about the new CI (continuous integration) system they were building for self hosted customers using Jenkins + Ansible (both established technologies for managing deployments). My first question to the engineer was, why would you use Jenkins + Ansible? I was pretty sure this could be easily done with a script.
In simple words, “continuous integration” means a system for automatically executing application updates on remote servers. This system monitors for new releases, checks if all is okay, and then runs updates on all servers that need to be updated. Jenkins is a tool to manage the applications to be monitored and the servers to be updated, and Ansible is a technology that executes commands remotely (via secure shell). It makes sense if there were a variety of applications and a variety of servers that we were managing, but in this case, we just manage servers running Frappe.
And we already have a framework in place that we had built in-house, Press and Agent, which do a similar job as Jenkins and Ansible. Press is a Frappe based app that maintains the state of the applications and servers and Agent is a simple Python app that contains scripts for completing simple actions on remote servers to manage Frappe. Press and Agent were designed to run our hosting platform Frappe Cloud, and not self-hosted customers. The ideal way according to me was to make Press and Agent more flexible so that they can handle our self-hosted customers as well. This way we don’t have to deploy and maintain multiple platforms and learn new ways of writing and managing scripts. Unfortunately the engineer's first instinct was to "buy" (for free) a pre-existing platform stack.
The problem with “buy” is that people don’t look at problems from first principles. Some of the “tooling” that is available is so sophisticated, that learning and maintaining it is another project in itself. An entire domain is created of engineers whose job is only to maintain software (DevOps). As people use more and more components to build out their systems, the only way to manage it becomes to run them in “containers” rather than install them directly. And then building and maintaining containers becomes a new activity by itself. Each component also comes with its own programming interface (API), so now you also need an API management platform that keeps track of all these components. We just end up creating a complexity hell to maintain things that should be really easy to maintain.
While I understand that what I am saying sounds archaic to some, I want to make a strong argument for “make”. "Make" is harder to get started and is slow and does not cover all use cases, but you get so much depth by making things rather than just integrating them. "Make" is the slow meandering path in the woods and "buy" is the express highway with the shiny lorries. The most common example I use is that if you want to cross the river, you should make a simple canoe, not a ship. In most of the cases, you don’t want tools that are over engineered to do simple tasks. Also using overengineered tools is intellectually challenging but also exhausting because you don’t really understand how these tools work and I reckon that makes software engineering so frustrating. There is infinitely more “joy” in hand-crafting solutions and adapting them to handle new and new scenarios. To extend the free food analogy, eating free food all the time will leave your stomach full but unsatisfied.
Since I grew up in another era, I am very comfortable with the idea of making things from as little as possible. At some point, if the tools you make become too sophisticated, you have to trash and redesign them. But this is the fun of life. The biggest satisfaction of my life is not founding Frappe or running a large open source company, but the sheer joy I get from making things and understanding deeply how they work. “Make” is Frappe’s superpower and I really want the engineering team to embrace it (and other engineering teams too!). It definitely makes for more fulfilling journeys and meaningful work. This is what makes us special. Life is not about destinations but about journeys.
All of us need to waste more time and make more things!
