In the mid-1990s, California—like many other economically booming states of the time that wanted to tap the excess generation capacity of neighboring states—restructured its electricity market, shifting from heavily regulated vertical local monopolies to what might be called a conglomeration of competitive markets. This “form of organizational planning [had] to strike a balance between imperatives to simplify, bound, and control the market,” explains Rilinger. The tradeoffs chosen by lawmakers opened the way for the electricity crisis.

According to one story, told by “marketers” and “producers,” California’s market for electricity seized up because of a lack of supply. The wholesale price of electricity skyrocketed and fluctuated wildly, while the retail price was capped, meaning that price could not moderate consumption. According to another story, told by “utilities and the California government,” the market malfunctioned because of “outrageous trader behavior” in pursuit of illicit profits: Electricity intermediaries deliberately misaligned supply and demand to produce large profits on the spot market. Neither story is obviously true or false. There were scheming electricity traders, but it’s unclear whether they broke any laws. Regulators and politicians were influenced by marketers such as Enron. Economists disagreed over econometrics and market power. Neither of the two stories will ever prevail, writes Rilinger, because “the criteria guiding the evaluation of evidence are not clear or set in stone.”

He begins to explain the crisis by asking, “What structural features of the system created opportunities and incentives for behavior that was incompatible with the reliable operation of the electricity system?” He suspects that the origin of the crisis lies in the design of the market.

Market design is an intellectual endeavor that aims to create or improve a market. Contrast that with the evolution of a market as a phenomenon of spontaneous order. Economists are not the only market designers; their ranks include engineers, computer scientists, and operations researchers. Consider this description:

At its core, market design views markets as search algorithms that solve constrained optimization problems. This is not just a matter of the formalizations designers use to conceptualize markets. It reflects how designer markets are supposed to operate. A combination of human activity and software should realize the search algorithm and produce custom-tailored results. If market actors follow the desired calculated logic, they realize the subroutines of the larger search algorithm. In that way, market design is similar to organizational planning.

Rilinger points out that “we are now quite far away from the commonsense understanding of a market as a place where people exchange money for goods and services.” Indeed, we are. Notice the prominence of algorithms in the above description. Exchange is a feature so long as it is consistent with the plans of the designers. In fact, the author equates market design to social engineering. To encourage (nudge or yoke) market participants into doing what they want them to do, designers “simplify, bound, and control.”

“Simplifying,” the author tells us, “means reducing the behavioral options inside the market as well as the information participants need to consider.” Understanding that in the context of the market for electricity is, ironically, not so simple. Rilinger tries to help:

To give an example: the auctions required a standardized input—a step function that established how much the buyer or seller was willing to buy or sell at a given price for a given hour. This information had to be submitted at a particular time. After the auction closed, the software calculated the optimal combination of trades and informed buyers and sellers about sales and purchases. The algorithm incentivized all generators to bid their true marginal cost and buyers to bid their true preferences. The setup made it easy and intuitive to do what the market design required.

Simplifying the market that way worked well enough to achieve the objective of delivering electricity at a low price for a limited period of time. However, simplification was “differential” or inconsistent across the many different markets that made up the whole. Consequently, the capital expenditures that would be necessary to increase the supply of electricity in the future never materialized.

Perils of compromise / “Bounding,” the author continues, “refers to an effective insulation of the market from other spheres of action.” His easy-to-understand example is bullfighting: A ring bounds the matador, the picadores, and the bull. As with simplification, bounding in the context of an electricity market is hard to comprehend, but it has to do with generation companies providing electricity to California utilities either under contract or on the spot market.

The system, in fact, consisted of many markets. The author’s diagram of markets and participants shows a California Independent System Operator (CAISO) market, an imbalance market, five ancillary service markets, and a retail market. There is a distinction between “physical markets” for electricity and “financial markets.” If this reviewer’s understanding is correct, “access rules” allowed traders to trade in either a physical market or a financial market, but not both. “The access rules,” Rilinger explains, “effectively protected an effort to simplify different market settings to different degrees.” He adds that the rules “ensured that each sphere could operate on the basis of its own social and technical logic—a prime example of bounding.” Traders violated the rules to reap “bandit profits,” which contributed to the crisis.

Traders were able to violate rules because market designers neglected to incorporate effective enforcement. “Control,” in general, “refers to ongoing efforts to identify novel behavior and either constrain it or adjust the market mechanism to accommodate it.” An example of novel, objectionable behavior is “market manipulation.” Rilinger and others often use that term, though they do not define it. Specific cases included turning off a generator to reduce supply and increase the price of electricity. Other cases were “games” that Enron and others played, such as being “paid to relieve artificial congestion.” But the Federal Energy Regulatory Commission (FERC) regulated wholesale prices. Designers created “monitoring units,” which identified misbehavior but “had no authority to sanction or penalize rule violations directly.” The author puts it bluntly: “The control structure was weak, fragmented, and underprepared to react to any manipulative behavior.”

Political compromise is one reason market designers created a flawed system. Several groups—including utilities, marketers, and industrial users—agreed to transform the market structure from regulated monopolies to the conglomeration of competitive markets. Disagreement emerged, however, over the nature of the transformation. One group preferred a “pool” that “would buy electricity from generators, sell it to distributors, and organize the transmission.” Another group preferred a “direct-access model that allowed retail and wholesale customers to choose their supplier on the basis of bilateral contracts.” The groups compromised. The first group would get the pool it wanted; to satisfy the second group, “the pool would be separated from grid management.” Rilinger reports that “market designers were uniformly aghast.” They warned that organizing the system based on the compromise would lead to “inefficiencies.”

Market designers tried to persuade the interested parties that their compromise was unworkable. But the parties refused to listen. “In a world where everybody seemed to know how to use economic reasoning,” says Rilinger, “the market designers appeared to be just one group with strange opinions.” He adds, though, that “What the ideal electricity market should be and how it should work were objectively ambiguous.” That seems to contradict his point that failing to heed the warnings of market designers hobbled the system from the beginning.

The second reason designers created a flawed system relates to their decision-making procedure. To complete the work on time, they chose “modularization,” which Rilinger characterizes as “a radical division of labor to build the market.” Now, modularization and the way it hinders the design process are among the most difficult concepts to understand in this book. Here is a rough description of the problem: Designers developing a given module, say a market, cannot imagine the effects on other markets. No one individual can oversee all modules and foresee these interaction problems. Although designers make rules to prevent bad behavior in the system, traders continue to find new ways to misbehave. Designers will not see all the modules (markets) that will be required in the system during the design phase. Even if they recognize the need for a module, they might neglect to develop it. “Modularization,” the author sums up, “prevented designers from deploying strategies of simplification and bounding effectively.”

Social engineering / Three kinds of designers developed the system. Each viewed markets or market behavior in unique ways. This “intellectual fragmentation” is a third reason why the system failed. Economists who designed the auction markets for electricity supply reckoned that by giving traders the right incentives in the first place, traders would do what the designers intended them to do. Markets would “self-regulate.”

Economists specializing in industrial organization expected sellers to exercise market power, though not as frequently as they did. These economists recommended a “monitoring function” that would leave regulatory action to FERC. Engineers concentrated on “grid management”; they did not fathom that traders “would try to make money in any way possible.” They recommended light regulation, such as requiring utilities to disclose their costs of production. The upshot of these various views of markets and market behavior is that effective measures to control behavior were not designed.

To Rilinger, market design is “a form of social engineering.” Whereas some of us recoil at the thought of social engineering, the author is hopeful because “digitalization has breathed new life into projects of social engineering.” He wants to give market design (or “government design”) a chance so long as we understand when it will not work.

Market design will not work if decisions made in the political arena determine that something impossible must be designed. Recall that the political compromise “to separate the market from grid management” dealt a blow to the successful design of California’s electricity system. Market design will not work if a large number of designers develop the system, which is what happens when the problem to be solved is a difficult one. The groups that designed California’s market were apparently numerous enough that they did not communicate effectively; they did not foresee behavior that would disrupt the market. Finally, market design will not work if designers with different training or experience do not share a common view of how to develop the market. Economists and engineers did not work from the same “blueprint” to build California’s market.

All this means designers should be wary of using their tools to tackle “complex allocation problems.” Anyone currently involved in drafting a market solution for allocating water from the Colorado River, for instance, should be mindful that Rilinger considers water, along with electricity, clean air, and healthcare, to be such a complex allocation problem.

Market design is a technical subject involving more than markets. Thus, much of Failure by Design is challenging to read and understand. The layman may learn something about the basics of market design. However, the best audience is probably economists who specialize in market design and engineers who specialize in electricity transmission. According to the author, businesses including Walmart, Amazon, and Uber use the techniques of market design apparently unbeknownst to their consumers. Also, the author reveals that socialists are looking forward to “nationalizing the logistic systems” of those businesses. It will be necessary to defend market designers from government officials with socialist tendencies to preserve private property and consumer sovereignty.