Perhaps you went to your bank to get a mortgage. Or maybe a fancy Super Bowl ad enticed you to get your mortgage with a nonbank lender. In the former case, the bank or credit union might keep your mortgage on its books. In the latter case, the nonbank lender almost always sells your mortgage to someone else, often Fannie Mae or Freddie Mac, the giant government‐​sponsored enterprises at the heart of American housing finance. But the lender does not always sell all of your mortgage; many lenders retain the right to service your mortgage.

The company from which you received your mortgage is the originator. The originator can be a traditional lender, such as a bank or credit union. The originator can be a mortgage banker, who relies on short‐​term (“warehouse”) money to fund your loan until it sells the loan to another entity, which may or may not be the ultimate investor. Your originator can also be a mortgage broker, who arranges the terms of your loan, but the short‐​term funding as well as the ultimate funding of your mortgage will be performed by someone else.

The servicer is the entity that receives your monthly mortgage payments. In the old days, it was to whom you mailed your check, but now that is mostly done electronically. Once the servicer receives your payment, it forwards most of the money to the ultimate investor while retaining some portion to pay your property taxes and make insurance payments.

The servicer also plays a critical role when something goes wrong with the loan, such as when a borrower can no longer pay. The servicer interfaces with the borrower, offering mitigation options such as forbearance or an arrangement of a short sale if the mortgage is no longer sustainable. In the unfortunate instance of a foreclosure, the servicer is also responsible for maintaining the property in good condition. Servicers perform a variety of recordkeeping and reporting obligations related to the mortgage as well. In a general way, the servicer is the entity that administers or manages your mortgage.

The Conference of State Bank Supervisors, a trade association representing state bank regulators, estimates that within the United States there were 19,655 active nonbank mortgage companies as of April 1, 2021. About 80 percent of them were mortgage brokers, which do not make or fund the loans themselves. Most of the 4,978 federally insured banks, of which the Federal Deposit Insurance Corporation (FDIC) considered only 270 as specializing in mortgage lending, also originated mortgages. The 5,068 federally insured credit unions also made mortgages, although only to their eligible members.

To make the topic even more complex, originators can be servicers and servicers can be originators—but they do not have to be.

Rise of Nonbank Mortgage Lenders

Before the Savings and Loan Crisis of the 1980s, it was most common for the originator, servicer, and investor in mortgages to all be the same institution. While there has long been an active mortgage banking industry—going back to at least the 1870s—in which loans were originated and then sold to investors such as life insurance companies, even those mortgage bankers were both originator and servicer. They also did not dominate the overall market, except in then‐​frontier areas lacking established deposit‐​taking banks.

The 1980s collapse of the savings and loan industry is still being felt in the mortgage market today. The disappearance of thousands of thrifts opened the door for mortgage bankers to gain considerable market share. This expansion of mortgage companies would not have been possible had it not been for the willingness of Fannie Mae to massively increase its buying of mortgages, taking on the credit risk once borne by the savings and loans. At the time, Fannie Mae operated mainly as a giant thrift, taking mortgage risk directly onto its balance sheet. This tendency resulted in the effective failure of Fannie Mae in 1981, leading to a government rescue. Some things never change.

The growth of nonbank mortgage lenders was also facilitated by the development of automated underwriting, which was made easier by the maturation of the three dominant credit bureaus, Equifax, Experian, and TransUnion, as well as the introduction of the Fair, Isaac and Co. (FICO) credit‐​rating score. Before these technological developments, the norm in mortgage lending was a by‐​hand underwriting, often requiring considerable time and expertise. To some extent, the massive growth in the market share of nonbank lenders was a result of the 1980s and 1990s revolution in personal desktop computing. The technological and process limitations of manual underwriting often meant that only the loans we would today call “prime” credit were being made before the 1980s. Today, prime is generally defined as a FICO or equivalent score of 680 or more (on a scale of 300 to 850). Before the 1980s, borrowers who would now be labeled “subprime” often did not get mortgages at all. Even government programs, such as the Federal Housing Administration (FHA) mortgage insurance under the Department of Housing and Urban Development (HUD), were limited to prime borrowers until the 1970s.

Some of the soon‐​to‐​be‐​removed constraints were legal. Without the ability to price mortgages according to borrower risk, to maintain profitability and hence a sustainable mortgage business, lenders limited credit to the middle to upper band of borrowers. While this resulted in some cross‐​subsidies, they were believed to be minor and difficult to evaluate before the growth of automated underwriting.

Lender Failures

A big change came in 1982 with the passage of the Alternative Mortgage Transaction Parity Act. This legislation helped introduce risk‐​based mortgage pricing, contributing to the later development of the subprime mortgage market.

Whereas the savings and loan crisis opened the door for nonbank mortgage lenders, the proliferation of subprime and Alt‑A (typically, prime borrowers with loans with high‐​risk characteristics) lending opened the floodgates. The model during the early days of subprime, the late 1980s and 1990s, was one in which nonbank lenders, such as Household Financial, Beneficial, The Money Store, and Long Beach Mortgage, would originate and service loans that were packaged into private‐​label mortgage‐​backed securities (MBS), generally assembled by investment banks.

Whether it was concerns about reputation, or regulatory scrutiny, or just plain old risk management, commercial banks largely avoided subprime mortgages except for FHA lending. The Russian default crisis in August 1998, along with the Federal Reserve–assisted rescue of hedge fund Long‐​Term Capital Management, resulted in temporary disruptions in the U.S. capital markets that caused several nonbank mortgage lenders to fail, as those lenders were highly dependent on the short‐​term money markets.

Failure among nonbank mortgage companies can mean little more than declaring bankruptcy, walking away from creditors, and then starting business all over again under a new name. Our economy barely registered the failure of numerous nonbank mortgage lenders that resulted from Russia’s default. The nonbank lenders that survived grew and expanded as the mortgage markets reached new heights with the housing boom of the early 2000s.

2000s boom and bust / While mortgage servicing continued to be dominated by depositories in the 2000s, nonbanks made almost a third of mortgage originations, by dollar volume, in the boom year of 2006. According to Inside Mortgage Finance, 15 of the 25 largest subprime mortgage lenders in 2006 were nonbanks, a significant shift from previous years.

The dramatic increase in nonbank mortgage origination in the 2000s was facilitated by both investment banks, which would often pool the loans into securities, and Fannie Mae and Freddie Mac, which would end up as the largest purchasers of subprime MBS as well as significant buyers of individual subprime mortgages. Without the support of Wall Street, along with Fannie and Freddie, nonbanks would have remained a marginal player in the years leading up to the 2008 crisis.

Although nonbank lenders were a sizable minority of the market going into 2007, they constituted a large share of institutional failures. Their heavy dependence on short‐​term funding, particularly lines of credit from banks, left them vulnerable. The mismatch between the maturity of their assets and their overnight funding sources also left nonbanks highly exposed to interest rate movements. Starting in the summer of 2006, the yield curve inverted, leaving short‐​term rates often higher than long‐​term rates. The yield curve remained inverted for almost a year, resulting in significant losses to nonbanks.

According to FDIC estimates, the number of nonbank mortgage lenders declined by almost a third between 2005 and 2009, whereas the number of depository mortgage lenders decreased by only about 4 percent. The market share of nonbanks in mortgage origination plunged. By 2008, nonbanks had almost completely left the business of servicing residential mortgages. Despite the dramatic shakeout of nonbanks during the 2008 financial crisis, their demise was short‐​lived. By 2012, the mortgage market began shifting away from banks and again toward their nonbank competitors.

Ironically, the widely touted National Mortgage Settlement in 2012 had the long‐​term effect of pushing banks away from mortgage servicing, leaving borrowers to be more likely served by nonbank servicers, who are subject to less oversight. In February 2012, the U.S. Department of Justice (DOJ) and HUD, along with 49 state attorneys general, announced a global agreement on mortgage servicing issues with Bank of America, CitiBank, Wells Fargo, JPMorgan Chase, and Ally Bank.

The agreement included over 300 individual changes to servicing standards, intended to both improve the quality of mortgage servicing and bring greater uniformity. The settlement also incorporated consumer relief and payments to state governments.

President Barack Obama called the settlement “landmark” and discussed the role that “robo‐​signing” played in the run‐​up to the housing collapse. He was correct to point out these fraudulent practices, but both banks and nonbanks engaged in them. By working with only the largest banks and excluding nonbanks from coverage, the settlement created a regulatory playing field now tilted further toward nonbanks.

In addition to increased compliance costs from the arrangement, the largest banks also took a major blow to their corporate reputations, and rightly so. It might not always seem this way, but the biggest U.S. banks spend a lot of time and resources worried about how they are perceived by the public. Since having a checking or savings account at, say, Bank of America provides almost the same benefits as having an account at Wells Fargo, a bank can take a sizable hit to its market share because of reputational concerns. For instance, in its 2015 letter to shareholders, JPMorgan Chase cited reputational concerns for its plans to reduce its participation in FHA lending and to exit its servicing of delinquent Fannie and Freddie mortgages.

Because nonbank mortgage lenders are not household names and have few relationships with consumers other than mortgages, they are less vulnerable to concerns about their image. There are also few reputational penalties attached to individual bad actors in the mortgage industry. Of course, not everyone at Countrywide, one of 2008’s largest subprime lenders, was responsible for the company’s misdeeds and failure, but the degree to which the nonbank mortgage industry is still staffed with Countrywide alumni is simply shocking. As the Center for Public Integrity has noted, top executives from numerous failed subprime mortgage lenders were soon back at work within the field after the crisis. The point is that one individual is not at fault; it is a systemic problem facing the nonbank mortgage sector that a reputation for misconduct counts for virtually nothing as long as one can bring in business.

One cannot attribute all the changes in the market to the 2012 National Mortgage Settlement. However, it is stunning that the overall nonbank share of servicing in the year before the settlement was 7 percent and then rose to 24 percent the following year. In 2013, banks sold the servicing rights to more than $500 billion in mortgages to nonbanks.

I do not doubt that Obama, along with the 49 state attorneys general, believed that the settlement would be a long‐​run positive for borrowers and perhaps even for financial stability. But whatever its intentions, the agreement accelerated a move of servicing from banks to nonbanks, with a resulting decline in both financial stability and consumer protection.

False Claims Act / The Obama administration also drove both mortgage origination and servicing from banks to nonbanks through its use of the federal False Claims Act. This legislation was passed during the Civil War out of fear that suppliers to the Union army were engaging in fraudulent behavior, such as providing defective materials or spoiled foods. The False Claims Act has always included penalties in addition to actual damages. In 1986, those penalties were increased to treble—that is, three times—the amount of harm. Having penalties that are multiples of actual damages is not a novel concept and is standard in circumstances in which the government has been defrauded, especially where such fraud is hard to detect. And of course, there must be actual damages for there to be penalties.

The DOJ under Obama took the view that the False Claims Act covered lenders doing business with the FHA, located within HUD. There’s nothing particularly novel in this interpretation. The FHA is part of the federal government, and some lenders had defrauded the FHA. What made this a significant issue was the large wave of delinquencies within the FHA and the degree to which the DOJ pursued minor defects in the mortgage process. If there isn’t a delinquency resulting in a loss to the FHA, then there’s no damage to treble. But the FHA, which the Obama administration inherited, was witnessing record delinquencies, with some crisis‐​era loan cohorts showing past‐​due rates of over 30 percent. Clearly, the FHA was bleeding. Obama’s DOJ was looking for lenders to cover some of those losses. I have considerable sympathy for that view.

I once heard Obama’s first FHA head, Dave Stevens, say that “all mortgages have some defects.” Stevens’s point was not that all mortgages were faulty, but rather that the DOJ was going after mistakes that he and the industry viewed as immaterial. The lenders that complained about this did have a point. The FHA’s standards of very low down payments, poor borrower credit, and extremely high debt burdens will, in a stressed environment, result in a lot of delinquencies. The high level of FHA delinquencies witnessed in the Great Recession was mostly the result of the FHA’s own weak underwriting standards, not widespread fraud. Yet lenders were being held responsible for insignificant errors that would have been ignored in a strong housing market. In a very real sense, lenders were making shoddy loans for the FHA because that’s what the FHA wanted or at least allowed.

There were cases of lenders setting out to cheat the FHA. The most infamous example was that of mortgage lender Taylor, Bean & Whitaker. In that instance, Fannie was aware of some of the bad behavior but did not feel any obligation to tell the FHA or Freddie, the latter of which was cheated by the same lender.

To some degree, the Obama DOJ was making up for a weak enforcement culture at the FHA. Punishment of lenders by the FHA has historically been quite rare. And there has long been a revolving door between the FHA and the mortgage business. For instance, Stevens went directly from being head of the FHA to being head of the Mortgage Bankers Association, the primary lobbying arm of the industry.

Although the mortgage industry considered the DOJ’s use of the False Claims Act problematic, it was actually a poorly crafted substitute for enforcement that the FHA should have been doing for itself. The real problem was that, like the 2012 National Mortgage Settlement, it was aimed mostly at banks. This created incentives for nonbanks to enter FHA lending. Nonbanks do not suffer the same sort of reputation losses that a False Claims Act carries. They also can more easily “go out of business” to avoid penalties in a manner that is not really an option for banks. The ease of entry and ability to move seamlessly across companies may make the nonbank mortgage business more competitive, but it also reduces the penalties for misconduct.

There were undoubtedly bad actors in the mortgage industry before the 2008 financial crisis. Some of them are still in the business. That, of course, does not minimize the role that destructive government policies played in the crisis. The biggest driver of the financial crisis was normal, relatively decent people rationally responding to the perverse incentives they faced.

Warehouse lenders / Ironically and hopefully unintentionally, many of the commercial banks that pulled back from directly originating and servicing mortgages now serve as warehouse lenders to the very nonbank lenders that have taken over the mortgage business. A warehouse lender provides direct short‐​term facilities or lines of credit to nonbank mortgage lenders. As this lending is not tied directly to any specific mortgage, the warehouse lender avoids the reputational and regulatory risk associated with the mortgage but still generates a profit from the mortgage business. Because warehouse lending is usually short-term—for the time between closing of the loan and delivery to the next player in the chain—the warehouse lender also avoids the interest rate risk inherent in mortgage lending. The time from closing to delivery normally runs from 30 to 90 days.

Because of these policy changes and shifts in the marketplace, nonbank mortgage lenders came back in force. By 2016, nonbank mortgage origination for the first time surpassed that of banks. The phoenix‐​like rise of the nonbank mortgage lenders could have been possible only with the assistance of federal subsidies. In the decade from 2010 to 2020, nonbanks effectively doubled their market share of Fannie, Freddie, and FHA lending. In market segments not dominated by government lenders, such as the jumbo mortgage market, banks continued their dominance.

Conclusion

Unbundling in consumer markets is usually driven by economic efficiencies. In the mortgage industry, however, the decades‐​long unbundling of originating, servicing, funding, and investing of mortgages has been driven largely by regulatory arbitrage.

The 2012 National Mortgage Settlement and the DOJ’s aggressive use of the False Claims Act in relation to FHA lending were imperfect and flawed responses to very real abuses and problems. Both set in motion forces in the mortgage market that would leave the system, including borrowers, more vulnerable than ever.

Readings

• A History of Mortgage Banking in the West: Financing America’s Dreams, by E. Michael Rosser and Diane M. Sanders. University Press of Colorado, 2017.
• “Bank and Nonbank Lending over the Last 70 Years,” by Kathryn Fritzdixon. FDIC Quarterly Banking Profile 13(4): 31–39 (2019).
• “What’s Behind the Non‐​Bank Mortgage Boom?” by Marshall Lux and Robert Greene. Mossavar–Rahmani Center for Business and Government Associate Working Paper no. 42, June 2015.
• “Why Have Banks Stepped Back from Mortgage Servicing?” by Michael Fratantoni. International Banker, September 2, 2020.