Does Optimal Mean "Best"? Not Really.

In the second half of the twentieth century it has become very popular to seek optimal solutions to a broad spectrum of problems: portfolios,  engineering systems, strategies, traffic systems, distribution channels, networks, policies, etc. But have you ever wondered if optimal really means best? Well, it does not. Optimality is not the most convenient state in which to function. The reason? Optimal solutions are inherently fragile Anything that is optimal is, "by definition", fragile, hence vulnerable. This is the price one pays for excessive specialization or extreme organization.  Let us see why. There are very few things that are stationary. In fact, we live in a quickly evolving environment in which there is little time for equilibrium and in which irreversible and dissipative mechanisms, together with chaos, randomness, not to mention extreme events (the so-called Black Swans) produce a sequence of unique events in which only fundamental patterns can be distinguished but in which the search for repeatable details is futile. This simple fact clashes frontally with the concept of optimality which hinges on precision and details. Sure, one can identify sweet spots in a multi-dimensional design space. From a mathematical perspective many things are possible. However, the dynamic non-equilibrium character of Nature guarantees that the conditions for which a given system has been optimized soon cease to exist. The pursuit of perfection is, therefore, an attampt to ignore the ways of Nature and Nature taxes similar efforts in proportion to the magnitude of the intended crime. The above is true not only in the global economy. In the biosphere it is also risky to be optimal, precisely because ecosystems are dynamic, and there is little time to enjoy optimality. As Edward O. Wilson stated in one of his wonderful books: "excessive specialization is a tender trap of evolutionary opportunism." Nature very rarely tolerates optimal designs. In fact, natural systems are, in the majority of the cases, fit for the function, not optimal. But there is more. High complexity compounds the dangers of optimality. As a system becomes more complex, approaching its own critical complexity, it possesses an increasingly large number of the so-called modes of behaviour (or attractors). Because these modes of behaviour are often very close to eachother, tiny perturbations are sufficient for a given system to suddenly transition from one mode of behaviour to another. These sudden mode transitions are more frequent as complexity approaches its upper limit. This is why humans intuitively try to avoid highly complex situations - they are unmanageamble precisely because of the mentioned unexpected mode transitions. In layman's terms, high complexity reflects a system's capacity to deliver surprises. This is why when speaking of a highly complex system a good design is not an optimal one but one that is fit and resilient. In other words: Attmpting to construct optimal solutions in the face of high complexity increases the cost of failure The following question arises at this point. Knowing that an optimal system is fragile, why then not design systems to be sub-optimal in the first place? Why not settle for a little less performance, gaining in robustness and resilience? Why this obsession to be perfect? Why push a system into a very tight corner of its design space, out of which it pops out at the snap of the fingers? Why do people pursue optimal solutions knowing that an optimal system, precisely because it is optimal, can only get worse, never better? As the ancient Romans claimed even the Gods are powerless against stupidity. But how do you get a solution that is fit and not optimal (=fragile)?. More than a decade age we have come up with a very simple algorithm called SDI (Stochastic Design Improvement) which is described here and which establishes the following new paradigm in system design: Specify an initial (nominal) design (or solution) to a given problem. Specify acceptable target behaviour of the system, i.e. an improved design with tolerable (but not optimal) performance. Run SDI - it is an iterative procedure which produces multiple solutions in the vicinity of the target behaviour. All have very similar performance. Measure the complexity of each solution. Select the least complex one as the final solution. This is because, with all things being equal, the least complex solution is the least fragile. The above philosophy is superior to conventional approaches to design, strategy and decision-making because it is tailored to highly uncertain, interconnected and turbulent environments, in which fitness counts much more than ephemeral perfection. Our economy (but not only) is fragile because everything we do is focused on maximizing something (profits, performance, success) while minimizing something else (risk, time, investment, R&D) at the same time. This leads to strains within the system. Everything is stretched to the limit (or as much as physics will allow). This is exactly what one should not do when facing turbulence. The focus should, instead, be on: Solutions that are fit, not optimal. Simplifiying business models and strategies. Accepting compromises not seeking perfection. Improve, don't optimize. Corruptio optimi pessima! www.ontonix.com