The Muddling Road to Success
Starting from first principles makes progress seem slow, uneven and complicated. Is it worth it?
“Why don’t we accept best practices that are out there, why do we have to always reinvent the wheel?”, people in Frappe often ask me. Starting from first principles is one of our core values and this often goes against conventional wisdom. To many people this seems naive and extravagant. Starting from first principles is slow and chaotic and leads to unnecessary confusion. I often wonder if I am slowing the team down? Is there a faster way to success that we were missing out on?
All through human history, we have muddled through ideas rather than work on some definitive grand plan. Around 3 million years ago, we figured that breaking things with a stone was easier than trying to break them by hand. Then around 30,000 years ago, we figured putting the stone at the end of a stick makes it even easier, thus inventing the humble hammer. The first hammers were held together by strips of leather. Later humans, some 20,000 years later, figured out how to separate metal from their ores, by using the process of smelting. Slowly metal was found to be a much more durable head for a hammer, rather than a rock. It took us a really long time to figure this out.
Pic: Early Stone Hammer. Source Wikipedia
If you look closely, the pattern repeats everywhere. Take something as complex as human flight. We have built contraptions (contraption: a strange or complicated piece of equipment) through the ages, before the Wright brothers finally made the first human flight. This has been on the backs of the discoveries that we have muddled through, like the hammer and metalworking. Even these contraptions took a really long time and countless iterations till we discovered the safe passenger flight that we all take for granted today. Each iteration resulted in us discovering an underlying principle and then simplifying the “mechanism” until it became more and more stable.
Most of the things we use in our everyday life, have been polished and fine-tuned via countless hacks and and iterations. There is rarely an aha moment, when the grand scheme of the universe falls into place to produce a product that can’t be improved any further. Innovation is rarely smooth, and often happens in spurts sandwiched between long periods where nothing seems to happen.
Through the ages though we have accumulated a number of these contraptions that have helped us to run these experiments faster. But even then, most of these experiments are ugly and end up in failure. It took us 40 years to make a commercial light bulb, since the time it was demonstrated that a steady source of light can be created from an electric arc. Infact Edison was only one of the 22 inventors who were able to perfect this process, and it took Edison 10,000 attempts to find the right carbon filament. 10,000 boring, mundane, unglamourous attempts where nothing seemed to happen.
In my 15 years of writing code, I have realized much of original work is like this. It feels boring and wrong. Surely there must be a better way to do this, you keep asking yourself. With the explosion driven by the internet, we are exposed to brilliantly packaged sexy “solutions” to every problem imaginable that it feels very wrong to do boring iterative work. Every new shiny solution promises a paradigm shift in the way of working, creating a new “layer” of contraption to hide the problems of the layer underneath.
Software programming has been often described as the writing of algorithms to solve problems. Infact, it is a lot more like a weaving complex tale, than a short mathematical proof. This tale often has various sub-plots and complex storylines to cater to various situations where it is applied. There are also many ways and languages, with their own styles and idioms, to tell this tale. Often these are just retellings of the popular tales rather than composing new ones.
Each style comes with a fresh perspective and is not necessarily superior. Non relational databases are not necessarily better than relationals ones, declarative style of code is no better than imperative style, neither is functional better than object oriented. These are all just different styles of storytelling. What really matters is how well the plot is held together, how consistent and lucid the prose is, and how aligned it is to the overall goal. A new retelling may be good in some parts, and not so good in others, and this is very difficult to predict at first. A retelling in the same language sometimes will yield better results than writing in a new one.
“Let’s use SalesForce”, our new sales leader tells me enthusiastically. “It will boost our sales productivity”. “Tell me what you need and we will build it in ERPNext” was my answer. Our whole existence is based on the premise that we can build anything in ERPNext. Realising that I was not going to give ground, he requested an integration that allowed us to take and receive calls for the sales people directly from the CRM. It took the dev team 8-10 weeks to build it. The new feature is doing the job quite well and we hardly required SalesForce for this. What I found later was that the calling feature was not even a part of SalesForce, but an external plugin. This urge to take a “ready-to-deploy utopia” and use it goes clearly beyond software development.
T2D3 (tripling of revenue for the first two years and doubling for the next three) is the maxim of growth in our age. Articles popular on social media tell stories of startups reaching astonishing revenue numbers in a very short time. “How we reached $50,000 monthly recurring revenue in 12 months”, screams post after post. Anything less feels like being a loser.
When we want to move at such a fast velocity, there is rarely an opportunity to explore. Take the most widely used solution and apply it. Everything happens in a super emergency. “Yesterday” is the common answer to the question, “by when do you want it?”. Companies end up using dozens of different software tools each performing just one minor function. Developers end up mixing dozens of existing libraries each performing a very specific function. The modern software ecosystems are full of highly specialized single purpose libraries that sometimes end up abandoned.
In a society determined by pace, muddling is no longer an acceptable principle. Muddling takes time, it is messy, confused and appears unnecessarily complicated. There are too many conversations, too much exploration and uncertainty involved. It is, and has always been, the non traditional path to success. It is also boring and unglamourous. There are bound to be periods of long failures, mismatched expectations and just general confusion and uncertainty and there are bound to be bruises along the way. For highly talented, ambitious and motivated people, it feels odd writing contraptions when you could be doing Kubernetes.
Original thought is undervalued till it becomes successful, and then it probably is overvalued. And success itself isn’t guaranteed. Pulling a team along this message is harder than it seems. Our society puts disproportional weight on outcomes rather than the process. Ensuring excellence for every customer, every product, every message can happen only if every nuance of the process is understood by everyone. Being extremely thorough with each aspect of the process is the only way to build great companies.
Slowing down, pondering, muddling, while persisting and relentlessly experimenting and failing is how great original work is done. It may be the alternative path, since being original is not really a requirement to commercial success. This is what has worked so for us in Frappe, and it has been incredibly satisfying, rewarding and magical. Our challenge is to keep this in our culture as we continue to grow and expand rapidly.
Rushabh is a software developer and founder of ERPNext. He usually writes about the startup experience, open source and the technologies he is working on.