The Agile Cult

The modern startup scene is dominated by small scrum teams that work on weekly, sometimes even hourly, customer-development feedback loops, strictly sticking to a "lean" diet of "MVPs". These teams follow what is known as "Agile" development, a development framework meant to minimize the risk of making products that customers dislike and maximize your chances of success in product development. 

Sounds great, where do I start?

You start by accepting first and foremost that you, the unknown developer, are definitely not Steve Jobs. You, a mere mortal, dare not dream up an iPhone when your early 2000s customers explicitly ask for a better Blackberry. You must let go of all wild ambition, accept your limits, and only through the teachings of Agile, shall you be saved from certain product damnation.

Why is it that you can't aspire to be Steve Jobs while going Agile? This is because Agile believers disregard all strategic business planning that stretches beyond a few weeks, and will outright anger when listening to five year plans. To Agile followers, the market is hopelessly complex and unpredictable. Anybody who thinks he can reliably envision, plan, develop, market, and distribute a product that takes more than a few months from idea to launch is to them, a false prophet. A brand new product category that takes years of product development, global supply chains, and massive marketing effort to deliver? Think again. If consumers can't keep their attention for more than a few minutes, why should anybody be able to predict what the market wants in a few months or a few years? In this worldview, Steve Jobs is the "lucky" false prophet who was right but only randomly so. The only surefire way to product success in the Agile worldview is to submit to indefiniteness, forego all plans or visions of the future, and instead build exactly what the market wants, at this exact very moment.

To capture such fleeting market demands, you must minimize product R&D investment to near-zero, launch "Minimal-Viable-Products" after days (not years) of development, and listen to customers with a cult-like conviction as they half-heartedly preach how to fix your infant product. To agile believers, the words of the user reflect the will of a higher (and random) power (= the market). If developers move fast enough before the higher power changes its mind, agile believers may one day lay eyes on the holy grail of product development; "PMF". 

So, whats wrong with Agile? 

First of all, it doesn't work nearly as well or reliably as its followers believe. Look to the hundreds, if not thousands, of startups that fail every year despite their deep faith and adherence to Agile development. They do weekly scrums, they listen to customers, they iterate fast, and yet they fail. Sure, the startups that succeed also use agile development principles such as iterative development and customer feedback loops, but this is no proof of assured agile success. The truth is, the vast majority of agile believers will fail, only a handful of lucky ones will achieve PMF, and maybe one or two will build great companies. Based on hard data, agile is a far cry from a working framework to build a great product; in fact it may even be toxic. When 99.9% of agile developers are failing, can you argue otherwise? If given data that an experimental drug kills 99.9% of patients, but 0.1% comes out healthy, a doctor would definitely conclude "poison" instead of "revolutionary treatment". But even given this damning data, true believers argue still that failure of agile teams is evidence to support the agile world view; that the market is random. The ones that failed simply succumbed to indefiniteness, and the only way to survive a random market is to divest in the future and follow the present. Therefore Agile.

Human beliefs are self-reinforcing; If you believe you are the son of god and decide to take a spiritual trip to India, you will indeed find evidence that you are the descendant of Vischnu. If you believe that the market is random and give up on plans, so too will you reinforce this belief. This is why agile is such a dangerous framework. By denying its believers foresight and planning, agile has become a strong self-reinforcing belief for a random world/market.

Is the market truly random, or are there unchanging truths about markets that can be gleamed and monetized? I will not attempt to argue in depth such high level concepts in this short piece (if tempted by the latter, look no further than the writings of Peter Theil, Warren Buffett, or any notable economist), but hear this; All of our modern technology is built on "science"; a collection of unchanging truths about a seemingly random natural world. Science is the conviction that random events have patterns, and patterns have truths. Our entire world is built on the conviction that the randomness of the natural world can be tamed by discovering truths through hypothesis, data, analysis, and a dash of genius. Given that science has yet to fail us, why should we build tech products on the notion that markets are truly random? Can't there be truths in markets upon which we can build reliable business plans?

Herein lies the second problem with agile; like all good cults it is a partial truth. Agile development does in fact tell its followers to conduct pseudo-scientific processes in product development. It preaches to formulate hypotheses, run experiments with real customers, and test the validity of hypotheses based on data. Agile does believe that real results can be attained by uncovering truths about products and customers using scientific method. But heres the catch; Agile culture vehemently refuses to believe that truths exist about the larger market, or that these truths can be gleamed in foresight without conducting incremental "lean/agile" experiments/rituals of customer feedback. 

Agile believes that such foresight/larger truths about the market are unprovable, and therefore don't exist. They are right about the first point; large market truths cannot be proved, for if proven true, you're probably entering the market too late. For example, If you've waited to enter the enter the search business until after Google captured the majority of search and established adwords, you're probably too late. If you're not too late, you're probably going to encounter competition, and a lot of it. "Air travel is going to explode" was a common market truth in the 1950's and continued to be so well into the 2000s, and hundreds of companies joined in on this truth. Air travel is still very important today, but none of the hundreds of companies reaped sustained profits, and none never will.

Market truths exist, but capitalizing them will always entail the risk you're wrong because the only way to prove a large market truth is to actually build a business. And even if you were lucky enough to prove a large market truth, that in no way guarantees you profits; you have merely welcomed others to join in. The Wright brothers entrepreneurship proved powered flight was possible; they have yet to reap the profits of this discovery. Only a very few businesses discover new market truths and end up with lasting profitability. This is where the analogy between scientific truths and market truths diverge vastly. There is no end to how many theories and technologies can be built on Newton's third law, Einsteins special relativity or understanding of biological processes. With market truths, it's either winner take all or everyone takes nothing. 

The third and final problem with Agile is its uncompromising cultish nature. Agile compels its followers, mostly comprised of broke desperate entrepreneurs with months of runway left, to fully believe in its teachings. Agile is not just a general teaching about product development, but it also includes rituals such as Agile "ceremonies" to be performed with fellow cultists on a weekly/daily basis. Agile even has an actual "manifesto" of values. Agile is literally a cult, and like all cults, there is no middle ground. If you don't go full agile, you deserved failure. If you fail while agile, you were just unlucky and therefore deserve another try within the agile/startup cult. Should you spend your last day of solvency on a sprint retrospective, you become a startup martyr. A little bit of peer pressure goes a long way to keeping cult members in place. 

Contrary to what all cultists believe, the full truth always lies in the middle ground. Yes, scientific method should be applied liberally in business/product development as it is applied in scientific research. Yes, listening to customers is important. Yes, investing the least amount of R&D to build and test a product is good idea. But, if you don't believe in larger market truths you are as good as wandering a dark mansion with nothing but a microscope. Market truths are different in nature from scientific truths, therefore merely scientific method will not lead you them. For example, scientific method could have informed anybody that the internet was the fastest growing industry and books was the product category with the greatest number of products within said category. However, in no way would these two random data points lead any Agile team to build an online bookstore. The "scientific method" of talking to 1996 customers on how to build an online bookstore would have been futile as well. The market for online books, let alone online retail, did not exist back then. Even if through extensive customer research, one could gleam that book purchases were highly inefficient and online sales were the way to go, that in no way assures success. No amount of customer research would have inspired in a group of people believing in randomness, the grand plan of building an online empire to encompass a whole "amazon" of product categories. But Bezos is not an "Agile team". It only took one man with strong commonsense and ambition to bet on an unproven market truth; that there was massive efficiency to be gained from selling books online, and that dominance in books was the first step in building an online retail empire. These large market truths simply cannot be uncovered by pure science or taught by customers. It takes even greater courage and commonsense to act on these truths. No cultist can ever dream to achieve such grandeur.

In hindsight, market truths are glaringly obvious, almost scientific. Building a search engine is obvious. People will obviously want to ride-share, and home-share. Who doesn't want next-day shipping with online sales? But in foresight, uncovering these truths takes more than just "Agile" and talking to customers. It requires strong commonsense, deep market knowledge, and vision. Furthermore, simply uncovering market truths is not enough for the entrepreneur who must build a business. In capitalizing on market truths, entrepreneurs need massive amounts of leadership, longterm vision, and most importantly, planning. The Agile allergy to these sectors all work against the entrepreneur.

After fully immersing in the Agile cult for several years, and witnessing first hand its shortcomings, I have come to the conclusion that building a durable business is no pure scientific process. It is an art/science. Analogies can be drawn between science and business, but unlike science where same methods produce the same results, copying business methods cannot be relied upon to produce profits. Agile attempts to condense business building to a pure science and method. In doing so, it not only detaches entrepreneurs from the reality of building, it also diminishes their greatest strength; art. Larger, well financed institutions will always have the upper hand in science. The playing field is much more level in art where a single creator can infuse beauty into a creation. Everyone knew that books were numerous and the internet was growing; only Bezos the artist concocted these commodities of knowledge into deep market truths and a truly historical business plan of online retail dominance. 

Steve Jobs, Jeff Bezos, Bill Gates, Elon Musk, and many more are more than just "lucky" people. Just as great artists produce multiple great works in their lifetimes, great entrepreneurs produce multiple great products and businesses. Agile believers think these great businessmen are the random winners in a random market. They might as well believe that a novice with a paint brush could paint the next Picaso, if only he meticulously consults potential buyers. The fact that there are multiple people who have built multiple independently successful businesses is the best proof there is that business success is not random. If the Agile thesis of randomness is correct, that even under best practice (which in their view is Agile) only 0.1% of companies succeed, serially successful entrepreneurs should be vastly rarer. Less than one in one hundred thousand for two and one in a 100 million for three successful businesses. Estimates put total number of US entrepreneurs in all industries at about 90 thousand, so Agile believers are off by roughly three magnitudes. The probable truth is not that market success is random, rather, it is simply the case that success in creative businesses such as software is not as abundant as we would like it to be. But its scarcity shouldn't shy us away from learning from these great artists. Just because copying Monet will not garner a young artist success, it does not mean one shouldn't study his art. Very few aspiring artists become great, but all of them learn and improve from studying great artists. This will not be the case for an aspiring artist who despises past artworks and attempts a purely scientific approach to art.

Excluding the few truly talented and lucky, we probably won't become world-renowned businessmen. But understanding that business is no pure science (nor pure art), learning from the greats, and attempting what has been proven to work, will definitely increase our odds of both small and large scale success. 

As I have argued, Agile is a partial truth. It is extraordinarily useful in generating incremental gains and in optimizing product features. However it teaches nothing of building businesses, and might even be outright toxic towards that end. Its philosophy has great utility, but just as reciting Nietzche everyday will lead you to misery, cult-like application of philosophy is never a good idea.

I write this article as a word of warning for aspiring product teams and entrepreneurs; Agile is a tool meant for specific situations to gain specific results. Do not be bewildered by it's cult like characteristics and apply its practices and teachings blindly to everything you do. If your task is to optimize a product feature, go Agile. But if your task is to build a business, look elsewhere.