The End of Theory: Financial Crises, the Failure of Economics, and the Sweep of Human Interaction

by Richard Bookstaber

An in-depth look at how to account for the human complexities at the heart of today's financial system

Our economy may have recovered from the Great Recession—but not our economics. In The End of Theory , Richard Bookstaber discusses why the human condition and the radical uncertainty of our world renders the standard economic model—and the theory behind it—useless for dealing with financial crises. What model should replace it? None. At least not any version we've been using for the past two hundred years. Instead, Bookstaber argues for a new approach called agent-based economics, one that takes as a starting point the fact that we are humans, not the optimizing automatons that standard economics assumes we are.

Bookstaber's groundbreaking paradigm promises to do a far better job at preventing crises and managing those that break out. As he explains, our varied memories and imaginations color our economic behavior in unexpected hues. Agent-based modeling embraces these nuances by avoiding the mechanistic, unrealistic structure of our current economic approach. Bookstaber tackles issues such as radical uncertainty, when circumstances take place beyond our anticipation, and emergence, when innocent, everyday interactions combine to create sudden chaos. Starting with the realization that future crises cannot be predicted by the past, he proposes an approach that recognizes the human narrative while addressing market realities.

Sweeping aside the historic failure of twentieth-century economics, The End of Theory offers a novel and innovative perspective, along with a more realistic and human framework, to help prevent today's financial system from blowing up again.

Genres: Economics, Finance, Nonfiction, Business, History, Research, Mathematics

224 pages, Kindle Edition

First published April 17, 2017

About the author

Richard Bookstaber is the author of A Demon Of Our Own Design, a book highlighting the fragility of the financial system that occurs from tight coupling and complexity. The book is noted for its foreshadowing of the financial crisis of 2007–08.

Reviews

[Athan Tolis · Rating 3 out of 5]

The last time I was so enthusiastic about a book half-way in and so despondent about having wasted the time to finish it was when I read “The End of Power” by Moises Naim.

Spot a trend?

I read and enjoyed author Richard Bookstaber’s “A Demon of our own Design” when it came out, and totally took to heart his findings at the time, regarding how a complex system with close couplings has the potential to go all wrong, so I was elated to hear he was prepared to tell the world where ten more years of deep thinking had taken a man who had in essence written a sober and well-reasoned book to predict the crisis of 2007.

On the other hand, I am extremely suspicious of books that declare they will provide the necessary content to shed new light on (let alone upend) the work done by generations of scientists and academics. If Thomas Kuhn is mentioned anywhere in the introduction it is solid advice to stop reading a book with immediate effect. I’ll read a thousand books in my life, but none of them will have been written by Malcom Gladwell.

Thing is, I enjoyed the previous book, I share background with this author, he is bloody knowledgeable, he co-wrote important pieces of post-crisis law, whatever… Long story short, I ordered “the End of Theory” and, 100 pages in, I was totally hooked: the author explains (eloquently, through wonderful examples like Conway’s “game of life”) that if you are looking to model the evolution of something as complex as the economy, you are better-served trying to understand the motivations and actions of the participants in the economy, the “agents.” He further argues that this is the only acceptable approach, because the alternative (which is to try to model the whole economy from forty thousand feet) presents a number of problems, which he calls the “four horsemen:”

1. You can’t hope to do the math: mankind has yet to find the answer to how three planets will follow Newton’s laws of motion. Do we really think we have the tools to figure out what three traders will do? This “horseman” is called “computational irreducibility.”

2. Suppose (heroically) that you can kind of model what individuals do. You still have no idea what they will behave like when they become a crowd: modeling crowd behavior is so intractable, that, for example, hundreds of people suffer in stampedes every year when they go to Mecca for the Hadj. It’s not like the Saudis do not understand this is a massive problem. It’s that the problem is hard. And we have not solved it. This “horseman” comes under the moniker of “emergent phenomena.”

3. Behavior changes. If you are studying Physics, your time is well-spent looking for iron laws. If you are studying people, not only do individuals learn and change, entire societies do. The same input will not have the same output every time. By the time you’ve gathered the necessary information to populate your matrix of covariances, you will have modeled a past that’s only tenuously related to the future. This “horseman” of the Economics apocalypse is called “non-ergodicity.”

4. In the words of a famous butcher, there are “unknown unknowns.” Expressing this observation in terms of math, you can say that it is impossible to assign a probability to a future event if it has never occurred to you to include it in the possible set of outcomes. Mervyn King explains this amazingly well in his magnum opus, “the End of Alchemy,” (yes, yet another “End of”) and Richard Bookstaber is happy to borrow the name for the fourth “horseman” from him, calling it “radical uncertainty.”

Bookstaber actually has five ideas, though, so there’s a d’Artagnan waiting in the wings (yes, I know, poor analogy, he was the fourth musketeer, sorry) in the shape of another major reason we must use an “agent-driven” model rather than seek iron laws:

Sometimes you run into situations where your map is of no use unless it is as detailed as the territory itself. For example, when you are looking for a needle in a haystalk. Under circumstances where to be of use a map needs to be as detailed as the territory, the map is useless. The author explains this through the wonderful story of the Library of Babel. This is obviously a situation where the only way to arrive at conclusions about the future is to “run” your model a number of times and see what happens.

So I was AMAZINGLY excited to hear what comes next, namely how the author goes about implementing the “agent-based” model.

Excited enough to ignore that, while the Fed’s, the ECB’s and the BoE’s “dynamic stochastic general equilibrium” models (DSGE models, for short) did not predict the crisis and always assume there is one “representative agent” and do not allow for the existence of a monetary system, and always tend toward equilibrium, this is all stuff that is not taught in Economics classes until your fifth or sixth year of study, deep into graduate school, and in no way refutes the much simpler laws of supply and demand taught in Economics 101, their elaborations in your second year of study, or even the sundry mystical musings of Andreu Mas Colell.

I was so excited about reading on, that I ignored major recent results from my biggest hobby, namely that people can and do model emergent phenomena: Formula 1 engineers create vortices at the front wing that “seal” the back-end of the car from cross-winds and allow the diffuser to create negative lift, even though “sliding skirts” were outlawed in 1982. This did not become possible until 2011, more or less, so it took 29 years of Moore’s Law to achieve, but we got there, and all in the interest of going a couple seconds a lap faster, not to prevent world famine. If you ask me, sometime in the near future it will be possible to argue that the mullahs in Mecca have blood on their hands, basically: they owe it to the faithful to get some scientists to design their space better and prevent the stampedes.

I was so excited to follow how an “agent-based model” could be set up, that I actually ignored that I run a business which models past behavior of human systems and concentrates in cutting down the information that is necessary to come up with a spec, with the aim of doing so before non-stationarity kicks in and ruins my work. That’s what I do for a living! But I suspended that thought, because I wanted to read the model.

There’s another book out there that takes the Vladimir and Estragon thing to an extreme. It’s Alvin Roth’s “Who Gets What and Why?” That book leaves it till page 131 out of 230 to explain the Nobel Prize winner’s algorithm for matching kidney donors to recipients.

Bookstaber does one better: there’s no algorithm in this book!

No, seriously.

Dunno, maybe his former employer, Ray Dalio, would sue him if he published it (though the author does take a pot-shot at “risk-parity strategies” on p. 153, so maybe I need to look elsewhere for my explanation), but I feel robbed. A hundred pages on topics along the lines of why a baseball player catches the ball using heuristics rather than by solving for the ballistics of the ball offers me no consolation. Neither do a bunch of charts from papers the author has written. Neither does his enumeration of who the agents are.

The description of what happened in October of 1987 is alright, the explanation of the flash-crash of 2010, on the other hand, does not even contain an indictment of the Einsteins (or are they merely CYA-agents) who are suing some poor idiot who traded the mini S&P from his parents’ basement in Hounslow. It would not really amount to “standing up” for him: it would simply have been a finding in keeping with the theory expounded here, which seems to imply we simply could not know enough to pin it on him, and that if we ran an “agent-driven” model a billion times in at least a few of thousands of them there would be some flash crashes. That’s the work I was expecting him to do, really. But he does not. It’s all rather anti-climactic, basically.

So I really don’t know what to tell you. There’s genius in this book. But we are spared its inner workings.

Perhaps what we have here is an elaborate advertisement for the author’s services. And then again, probably not: he dedicates the book to his son, after all.

So I’m stumped. But I’m also left feeling robbed.

[Mbogo J · Rating 4 out of 5]

This book is a clever and polite way of saying we are unable to predict financial crises using the current models. A system that mostly comes under assault is neoclassical economics with its notion of rational man. That man is a ghost during a financial crisis.
Bookstaber's explanations for complete model breakdown are valid even though he gave them serious sounding names such ergodicity, computational irreducability,emergent phenomena and what not. He proposes the use of agent based modelling instead to better come to grips with the uncertainty present during a crisis.
Even though Bookstaber joins the cadre of new agey economists who make their names calling bs on the rational man idea, I personally believe the rational man idea should be allowed to stick as a prodding tool into academic insights. The physics of motion was better understood when people started thinking about motion through a vacuum and unless economics utilizes the rational man, it will be stuck describing things the way they are and will be bereft of knowledge. As much as agent based modelling is a good idea,it is likely to result into complex and unwieldy results that have little utility.
This book is about epistemic humility and requires an open mind to read, if you are set in your positions you might not like it.

[InvestingByTheBooks.com · Rating 4 out of 5]

This is a text on financial theory and the author advocates a switch from the use of a rigid neoclassical theory based on a number of unrealistic assumptions to a fluent, messy but flexible use of so-called agent based modeling (ABM). Epistemology is the type of philosophy that concerns itself with the theory of knowledge, the nature and rationality of belief. Bookstaber wants to challenge how we understand and think about economics and uses the occurrences of financial crisis as the test environment for his endeavor. The author is the Chief Risk Officer at the pension fund University of California Board of Regents. Earlier he has been both a PM and a risk manager at numerous hedge funds and investment banks. Few have longer experience of financial risk than Bookstaber.

In his 2007 bestselling book A Demon of Our Own Design the author reviews his dramatic experiences from the investment bank and hedge fund world and how liquidity, leverage, crowding and tight coupling – the speedy interconnectedness of events – are key parameters in causing cascading that leads to a full blown financial crisis. He also begins to discuss the topic of complexity. The End of Theory could be seen as a freestanding appendix to the first book. By now the author has had the time to better develop a theory around what he had experienced first hand and he also offers a practical tool to use. Since the theory is so vastly different from conventional economics the book becomes a crusade against how economic theory address crises currently (if it does at all).

The financial system is described using 4 building blocks: 1) computational irreducibility – a system without mathematical shortcuts to describe it, 2) emergent phenomena – that the overall effect is different from the sum of the individuals actions, 3) non-ergodicity – the concept that actions of one agent depend on and are shaped by history, context and the actions of other agents and 4) radical uncertainty – the fact that the system cannot be modeled by using historical events. The really important future developments will be unprecedented. All this creates a financial system that I have come to call a complex adaptive system. It is full of self-enforcing loops; developments are non-linear and unpredictable. Then the author goes on and offers the computer modeling technique ABM as a tool to understand and handle the complexity. ABM tries to simulate system effects by the actions and interactions of autonomous agents with separate decision heuristics. Chapters 11 through 13 model the financial system using the method. The exercise is thought provoking and I especially liked the description of the multi-layering within banks.

Nothing of all this is new and Bookstaber never claims that it is. The notion of complex adaptive systems amongst others builds on George Soros’ concept of reflexivity as described in his 1987 book The Alchemy of Finance, on complexity theory popularized by the Santa Fe Institute and on Andrew Lo’s concept of adaptive markets. The merit of this book is rather the compilation of the many parts into a whole and especially the application on special situations – financial crises. The author doesn’t really take the knowledge about complex adaptive markets further, but he improves our crises-knowledge.

The writing and language is relatively accessible for a text on financial theory, the boundaries of human knowledge and the intricacies of the plumbing in the financial system. The author takes the time to explain and exemplify. At first this is a positive but during the course of reading the book the notion is reversed. What starts out as illuminating turns into being repetitive. In an attempt to win the reader over to the author’s point of view too much is said too many times. The book would benefit greatly from being slimmed down some 40-50 pages.

The End of Theory will advance your thinking on financial calamities but it isn’t always fun to read.

[Novice · Rating 4 out of 5]

Bookstaber's new book is not as entertaining a read as his first, A Demon of Our Own Design: Markets, Hedge Funds, and the Perils of Financial Innovation. The End of Theory... is informative and a must read for anyone keen to understand our evolving financial system. The book comes across as geared to an academic economic community. It often reads as a text book and sometimes requires careful study. The author's thesis and suggestions represent valid arguments and the philosophical disposition of Bookstaber's case is finely presented. But to me at least, it all remains unconvincing. Not because the arguments ring false, but because there is no real case for the economic field to change or adopt new thinking - recent financial crises and economic failure not withstanding.

The field of economic theory is ripe for serious introspection, new thinking, and frankly reform. I doubt Bookstaber will convince anyone within the economic academic institution to seriously consider promoting or even incorporating ABM analysis (have to read the book - no spoilers) into macro theory and modeling. Unfortunately, his own narrative history of the 2007/8 financial crisis and his personal engagement with the regulatory reform infrastructure established post crisis detailed in the book undermines any reason why, or confidence in, the powers that be to adopt any serious correction or improvement. ABM analysis related to capital markets requires a moral/political consensus and foundation - not likely when money and power are at stake. After all, political and economic actors around the world still seek to implement Marxist economic theory and policy, a curiously unexamined case relative to Bookstaber's promotion of ABM.

[Val · Rating 3 out of 5]

In this book, Bookstaber argues for the use of agent based models and the use of heuristics to simulate crises. He proposes this methodology in juxtaposition to the current use of mathematical econometric models.

The ideas are not particularly new. The literature of artificial intelligence has proposed similar schemes to model various domains, for several decades, with limited success.

Bookstaber enumerates a number of qualities of human-based systems, including financial crises, (ergodicity, computational irreducibility, emergent phenomena, radical uncertainty), that make them exceedingly difficult to model. It is precisely this difficulty that leads Bookstaber to propose this new methodology for modelling crises, though the book does not go much beyond proposing this new methodology.

Overall, the book is light on the technical details on what would constitute an agent based model. Although it is clear current models have their failings, the book does not make clear to me that an agent-based approach would be much more successful.

[Roger B · Rating 3 out of 5]

A bit of slog getting through this one. Bookstaber lays out four considerations that challenge the premises on which neoclassical economic theory is based particularly during a crisis: non-ergodicity, radical uncertainty, emergent phenomena and computational irreducibility. He then goes on to lay out a framework for an agent-based simulation which he contends has more narrative ability than the neoclassical model. The trick seems to be in capturing the ever changing dynamics between an agent and the environment (or other agents). It's not clear at all if the agent-based simulations provide better prescriptive advice than neoclassical models - something that bookstaber does not address despite citing experience in implementing regulations such as volker rule and dodd-frank. still, the shortcomings of the neoclassical model give credence to the agent-based approach.

[Gints Dreimanis · Rating 5 out of 5]

I would suggest everybody with an interest in economics to read it, and for everybody with an interest in the complexity to read the first half.

It's a very clear and concise introduction into complexity, then later it is a complex (no, really complex, if you are not into economics, you might not enjoy this) description into workings of financial crisis, but luckily these crisis can be kinda modeled with agent-based models which do not press unrealistic axioms on you but reproduce human behaviour, incomplete as it is, and full with heuristics.

The agent-based models, unfortunately, you will have to seek elsewhere.

[Chris Liberty · Rating 5 out of 5]

Understanding financial crises is a hobby of mine, and Bookstaber's The End of Theory is elucidating. The premise of the book is that he recommends using agent-based modeling (simulating actions and interactions between and among the myriad of financial agents) as a better way of figuring them out. He spends a lot of time explaining why traditional economic theory is inadequate in times of crisis. And he argues that although human behavior is extraordinarily complex and impossible to model, agent-based modeling can add valuable insight. Worthwhile reading!

[Achal Augustine · Rating 4 out of 5]

New approach to financial modeling

Agent based models seem to offer a compelling perspective for understanding financial crisis as they unfold. Like weather forecasting it may not be useful in predicting long term climate but could provide guidance on potential paths for financial storms.

[Hong Cheng · Rating 3 out of 5]

The agent-based model makes sense in terms of explaining the past crises, traffic and stampedes etc. However I cannot envision how one can implement such model in real world decision making. The book also spends way too much time bitching about the traditional neoclassical economics, which is quite repetitive.

[Nahum Timerman · Rating 5 out of 5]

Thought provoking

Admirable criticism of orthodox economics, interesting review of agent based modelling in analyzing real financial situations, and a thought provoking rejoinder that when we only go through a process once, we must use inductive narrative like tools, rather than deduction and axiomatic general solutions.

[Piotr Pabis · Rating 3 out of 5]

These 200 pages can be just summarized in: we cannot predict crises because of heuristics and computational non-reducibility of this world but we can theoretically build an agent based model which will in essence run in real-time so it’s useless.
I agree with this statement but the book is very difficult to read.

[Mike · Rating 4 out of 5]

A heady book but well written. I'm 100 percent with his conclusions that traditional economic models fail; history speaks the truth. Perhaps he's a bit too harsh in his argument that they don't work--perhaps we should not put so much stock into them in the first place. Thought provoking all around.

[Pamela · Rating 4 out of 5]

Good book, focused on why theoretical models fail to explain the responses people make to financial crises in the stock market:because the situations are basically psychological in nature. Well written.,engaging. I like his suggestions for improving.

[An Do · Rating 3 out of 5]

It's encouraging to see that a lot of the ideas mentioned here have caught on. Perhaps the bit on the GFC can be shortened and the rest of the discussion deserves more attention: model risk and how life doesn't let you flip coins a million times.

[João · Rating 2 out of 5]

TIL I can largely agree with a book and still think it's lousy.

[Robin Michael · Rating 4 out of 5]

great intro to agent based modelling (or story telling)

[A Ter · Rating 4 out of 5]

original

[Ahmed Al-Majid · Rating 4 out of 5]

Interesting approach to microeconomics. I wonder if the same treatment can be extended easily to macroeconomics.

[Richard Marney · Rating 5 out of 5]

Having just read "Radical Uncertainty" (King and Kay), I was prompted to reread this outstanding book, whose message is similar.

In the End of Theory, the author warns us that complex, real-life systems defy our capacity to understand fully "what's going on.". As a result, deduction-based, predictive modelling fails in practice. Four factors (the four "Horsemen of the Econopalypse") help see this: (i) Emergent Phenomena (the unexplained traffic jam that comes and goes); (ii) Non-Ergodicity (there is no "same-old, same-old"); (iii) Radical Uncertainty (K&K's unknown, unknowns); and (iv) Computational Irreducibility (math, science, etc. can't figure it all out). The financial system is complex. Given the interplay of many heterogeneous (human) agents operating in an unpredictable, dynamic environment, traditional new classical theory, micro-foundations, rational expectations-based models have failed to provide a realistic view of financial markets and their crises. As an alternative, a heuristics-based, context-focused is a modest attempt to cope with a world where so much challenges our processing capacity.

This is an important book.