Techno-utopians got it wrong: their tireless search for new technologies must start in the past. Most new technologies are just a rehash of past ones and most resources are devoted to maintaining existing technological infrastructure or towards incremental advances in old technologies: the new and innovative is extraordinarily rare. A fact so ignored but so intuitive, since most human needs have always been the same.
Mandelbrot’s “[amazon_link id=“0716711869” target=“_blank” ]The fractal geometry of nature[/amazon_link]” summarizes this line of thinking as the Lindy effect: the future survival of any Broadway show is best predicted by how long it has been running already. Itself based on a much older assertion that the “the future career expectation of a television comedian is proportional to his past exposure” (The New Republic, June 13th 1964). Thus, a statistical distribution that extends beyond the arts to other phenomena like the survival of technologies: the longer a technology has been in use, the longer we shall expect it to last, or more empirically, we shall conclude that every year that a technology survives may even double its additional life expectancy, contrary to the life expectancy of any living being. An insight that warns us against miracles when introducing new technologies without any precedent.
This next paper is the only one I could find that combines this effect with other power laws to try to ascertain the economic returns of basic research, and so the optimal level of investment:
Functional programming has been in the top of the hype-cycle for decades: no side effects, methods written in a purely mathematical style, strong type systems… What’s not to like about functional programming? It’s even a loaded term! Just ask someone: “But is not your code functional , yet?”
Although the Turing machine (imperative) model of computation was shown to be equivalent to the λ‑calculus (functional), there’s actually a long gap in its asymptotics.
Start with the most simple of the data structures, the array: in a purely functional style of programming, to update an array you would need O(log N) time, but in an imperative language it’s just O(1). And this is not a specific problem with arrays, it’s the standard behaviour within pure functional programming languages due to the their overrated immutability of data: in “Pure versus Impure LISP”, it’s shown that O(N) functional programs written with lazy, non-strict evaluation semantics can be translated to a purely functional style in O(N log N). That is, at worst an O(log N) slowdown per operation may occur by simulating mutable memory access: immutability carries a very high cost, indeed.
In practice, extensions to the pure functional paradigm have been considered to address the need to directly reference and change the state of variables (monads in Haskell), and most algorithms can be implemented efficiently by using purely functional programming, but it also involves extensive and difficult knowledge (“[amazon_link id=“0521663504” target=“_blank” ]Purely Functional Data Structures[/amazon_link]”) for even the most basic of data structures, and the recognition that they require a significant slowdown by constant factors and CPU stalling due to many cache misses.
And, as if that is not enough, debugging any functional program is a quest in itself: I would say that it’s an order of magnitude more difficult than an imperative language, vastly increasing software maintenance costs.
Why did software component reuse never take root? Why is outsourcing within the software industry such a bad practice, even if other industries outsource all of their developments? Why do big software companies perform so bad and have such a poor innovation record?
From a purely financial viewpoint, focused on profitability ratios, these and other questions are far beyond all comprehension. But paradoxically, they emerge as the result of the blind application of financial theory to the software industry:
And there are many more examples: striving for a better ROCE (Return on Capital Employed) on software projects is not a very good idea, since capital is not a scarce resource, qualified labour is; so economizing on an abundant asset is absolutely wrong. And that for any other return-like ratio: they will strip off any software company and deprive it of its most precious assets.
Fifty years ago, two researchers started writing this seminal paper, kicking off the research field of computational complexity, the core of computer science, with now tens of thousands of publications.
Although the concepts and ideas underlying the paper were not new, as a letter by Kurt Gödel show us, it was the foundational moment for a field that still produces deep, beautiful and practical results: in the last decade, Williams’ lower bound on non-uniform circuits; Agrawal-Kayal-Saxena primality test that included primality testing in P (although Miller-Rabin and Solovay-Strassen primality tests still live strong since they are much faster than AKS) or Vassilevska’s lower bound on matrix multiplication.
I see tons of start-ups and projects fail because they ignore the most basic algorithmic prudence: that is, that only sub-logarithmic algorithms should be accessible to the mass-public is one the ignored maxims of the computer industry that can only be learned by the proper interpretation of the absence of market offerings refuting it (vg. regular expressions within search engines’ queries, which could run in exponential time; or all the AI promises about solving optimization problems that never delivered).
Speaking with some researchers lately, they expressed the hope that the coming end of Moore’s law would vindicate this field, making its results much more relevant: and although they were absolutely right in that proportionally more resources will be directed towards these ends, they also failed to consider that the likely crash following this event may also reduce the total, aggregated opportunities.