|Cato Policy Analysis No. 25||June 30, 1983|
by Randolph H. Boehm
Randolph H. Boehm is an editor at University Publications of America.
The growing significance of federal budget politics in the American political dialogue cannot be dismissed. The sums involved are fairly described as staggering and will likely have a major impact on the performance of the American economy. Consequently, budget policies have become a topic of considerable concern. Supply-side economists commanded national attention when they questioned the revenue (tax-raising) policies of the federal government, which they say pose a serious threat to economic prosperity. Many of their critics, in turn, point to potentially disastrous effects of federal deficits running into the hundreds of billions of dollars. Special interest groups with a major stake in some aspect of government outlays voice vigorous opinions on the alleged shift in federal priorities from social services to defense with some of the partisans warning of dire consequences should the shift continue and others of dire consequences should it be checked.
The Reagan administration has helped force budget politics to the top of the American political agenda with its stated determination to check a number of fiscal trends which have been gaining momentum for at least a decade. Less appreciated, perhaps, but no less significant than the new administration in focusing attention on the federal budget is the cumbersome congressional budget process itself. It ensures that budget politics consume an inordinately large portion of each congressional session to the inevitable exclusion of many other issues.
One important aspect of the federal budget that is frequently overlooked, however, is the reliability of forecasts of major budget components such as the deficit, outlays and receipts, Gross National Product (GNP), unemployment, and inflation. In examining previous forecasts on these economic variables in federal budgets between 1971 and 1982, it is evident that the forecasters' records have been very poor, that many indicators such as GNP and unemployment are seriously deficient as mirrors of economic reality, and that the uncritical acceptance of aggregate budget figures by the mass media, policymakers, and citizens alike is wholly unwarranted.
Aside, perhaps, from compiling the historical tables in order to monitor the results of previous economic forecasts used in the budget, much of the following analysis is not original. Academic economists as well as economists in the federal agencies who compile the data making up each of the aggregates under discussion regularly point to their deficiencies. Most opine that these deficiencies can be overcome in time with more complete statistical information and greater conceptual refinement. Others are not sure that it will ever be possible to make the aggregates accurately reflect economic reality.
As an academic pursuit, the efforts to quantify data and make predictions may be unexceptionable. Serious errors in academic pursuits often prove beneficial so far as they serve to educate. But such errors obtruding upon public policymaking or codified in law have pernicious effects far beyond their educational value. More attention needs to be focused on the limitations of these various budget components and of efforts to forecast changes in them.
Policymakers' attitudes toward these concepts, however, seem to reflect a trend that is moving in just the opposite direction. President Reagan recently proposed "triggering" stand-by income tax increases on a ratio between two of the aggregates under discussion here -- the deficit and the GNP. The Humphrey-Hawkins full employment bill of the last decade proposed the unemployment rate as a "trigger" for wideranging public works projects. Other instances of this sort could be noted. In addressing this trend of relying upon aggregate economic statistics, a recent study by the Government Accounting Office discusses GNP figures:
The accounts help Federal policymakers pursue the goals of the Employment Act of 1946 -- full employment, price stability, and economic growth. Federal economists use the accounts for short term fiscal, monetary, and wage-price policy analysis, for managing the nation's employment and anti-inflation goals, and analyzing long-term demands for skilled labor and financing for capital formation. Major users include the Council of Economic Advisers, the Federal Reserve Board, the Office of Management and Budget, and the Departments of Treasury and Commerce....
Although the analytical uses of the accounts are primary, additional uses are being made. The Trade Act of 1974 (P.L. 93-618) specifies the use of annual GNP estimates in determining limitations on preferential treatment extended to countries exporting goods to the United States. The GNP price deflator is used as a component in the inflation adjustment factor in the Natural Gas Policy Act of 1978 (P.L. 95-621) and the Crude Oil Windfall Profits Tax Act of 1980 (P.L. 96-223) for determining the ceiling price on certain types of natural gas and the windfall profits on crude oil, respectively....
Proposed legislation in the 96th Congress would have further extended the use of GNP beyond that of an analytic tool and could have generated concern about what it represents and how well it does so. Federal spending would have been affected by the definition and accuracy of GNP.
One proposed amendment to the Employment Act of 1946, H.R. 2314, would have limited Federal outlays in the President's Budget to equal the Council of Economic Advisers' estimated Federal receipts. The receipts would have been based on real economic growth using real GNP estimates as part of the formula for the calculation.
Another proposed bill, H.R. 4610, would have limited Federal outlays to a specific percentage of GNP for the last complete calendar year occurring before the beginning of the fiscal year. Fiscal year 1982 spending, for instance, would have been limited to 23% of calendar year 1980 GNP.
Lastly, H.R. 7112 proposed an antirecession assistance program for aid to State and local governments to be triggered by two consecutive quarterly declines in real GNP and real wages and salaries. Allocation of funds to States and local governments were to be based in part on the aggregate real wages and salaries component.
An equally long recitation might also be made for policy uses of other aggregates like the unemployment rate or the Consumer Price Index (CPI). Clearly, the tendency to accept official economic projections at face value cuts across party and ideological lines. It is common in the executive bureaucracy and is becoming more so within Congress. The honest disclaimers from government agencies that compile the data and from economists who have studied the concepts are lost in the rush, or perhaps it comes closer to the truth to suggest they are ignored under pressure to seize upon "official" statistical data on the economic forces policymakers hope to affect.
A common belief, which may blunt the concern over the limitations of making forecasts from economic aggregates, is that forecasting is a scientific procedure which is slowly but surely undergoing refinements which make its measurements and projections more exact. Such scientific exactitude, however, must exhibit two criteria: 1) a high probability of accurate projections and 2) concepts that are themselves clear and objectively meaningful. By way of assessing the "scientific" status of budget forecasts, therefore, we will assess each of the selected components on the basis of the accuracy of past projections and the clarity and meaning of the concepts themselves.
There are several ways to evaluate the federal forecasting record. All have strengths and weaknesses. The first approach would be to compare the size of the forecasting error with the size of the total budget. This tends to yield only a small percentage error. However, these small percentages should not detract from the dollar magnitude of the error, which despite being a small percentage of the total budgets, can be extremely significant as gross figures in themselves. That is to say, a small percentage error produced by this formula can disguise a massive error in forecasting as a glance at the dollar amount of the errors in forecasting deficits makes clear.
A second approach is to evaluate the error in forecasting the percentage change. This is a stiffer criterion and would seem to be the more appropriate in assessing the claim of forecasters to scientific rigor in their calculations. It may be unrealistic to expect that forecasters can come close to the precise figure when the size of the aggregate in question, such as the deficit or the unemployment rate, is so large. It should, however, be possible to assess the magnitude of the impending change -- that is, whether the economic indicators are pointing toward a large change or a small one.
But here again the formula has its drawbacks. With the first formula meager percentage errors tend to detract from significant total dollar errors; the percentage change formula suffers from an almost opposite sort of quirk. It can be overly sensitive to insignificant errors when both the predicted and actual changes are of small magnitudes. For example, if the forecast predicts a small change, say, a $1 billion increase in the federal deficit, and there is a small increase of, say, $2 billion, then the error in forecasting the percentage change would be an impressive 100% despite a relatively minor error in the dollar magnitude.
On the other hand, small changes in some of the concepts we will analyze -- the GNP, the unemployment rate, and the CPI -- actually translate into huge changes in the magnitude of economic activity. A change of a couple of points in these calculations is significant, and a formula which is sensitive to small changes would seem to be required.
A final criterion is somewhat like the assessment in the error of percentage change, and it enjoys the same attractions and drawbacks. It is to assess the error as a percentage of the forecasted change. Put colloquially, it means that "so much" change was forecast, and the actual change was "twice as much" or "half as much." As a rule of thumb in assessing forecast accuracy, this formula is probably the most easily comprehensible to the nonspecialist layman or policymaker.
As with the previous criteria, it does not suffer from the problem of being overly sensitive to comparisons of small magnitude. But, in some cases, as we pointed out, this can be a virtue. Each of the proposed criteria is useful for asking different questions about forecasting reliability, but the results of each still need to be considered in the light of common sense.
There is yet another criterion of evaluation -- a proposed margin of possible error -- to evaluate the forecasters' accuracy. For present purposes, however, the three formulas outlined above are adequate.
Apparent from an inspection of previous forecasts on the federal deficit is that not only is there a poor record of accuracy, but there is not even a consistent direction of error. Between 1971 and 1982, the errors swing by wide margins both in overestimation and underestimation of the deficit. As for the margin of error, in only two of the 12 years under scrutiny were projections even close to accurate (1977 and 1978). Also, beginning with the 1978 budget published in early 1977, forecasters ventured long-range projections beyond the budget year in question. The results are equally, if not more, egregious.
Federal Budget Deficit Projections, 1971-1982
|' 71||' 72||' 73||' 74||' 75||' 76||' 77||' 78||' 79||' 80||' 81||' 82|
|1981 Budget||15.8||+ 5|
|Error in billions of $||24.3||11.8||10.6||8||35.8||14.5||1.9||1.8||32.9||30.6||42.1||80.9|
|Error as % of total budget||11.2%||5%||4.3%||2.9%||11%||3.9%||.4%||.3%||6.6%||5.2%||8.6%||9%|
|Error as % of year-to-year change||99%||2950%||125%||78%||88%||68%||8%||46%||156%||96%||2476%||154%|
|Error as % of projection||1869%||102%||42%||63%||381%||28%||4%||4%||54%||106%||266%||272%|
Viewed as a percentage of the total budget, the error will almost always seem smaller, since gross errors in each of the 12 years are a relatively small fraction of the entire federal budget. However, the gross dollar figures can hardly be viewed as insignificant, particularly for FY 1971, 1975, and every year since 1979. Except for a brief interlude between 1977 and 1978, budget projections did not improve. Large errors are evident throughout the 12-year period. If any trend is apparent, it is that the projections have been chronically unreliable since 1978 (the year in which projections were made for FY 1979).
Assessing the forecasters' record on the magnitude of percentage changes, the results are even more disappointing. In seven of the past 12 years forecasters deviated from the actual percentage change in the deficit by at least 96%. Twice they deviated by more than 2,000%. Only in 1977 was the forecast respectably close to predicting the magnitude of change in the deficit (although for 1978, the flaw inherent to the formula biases the result). They erred widely both in underestimating the magnitude of change (FY 1971, 1975) and in overestimating it (FY 1972, 1973, 1979, 1981). The average error in forecasting the magnitude of change in the deficit over the 12 years is almost 530%.
Turning to the third criterion, error as a percentage of forecasted change, we see that in six of the past 12 years the percentage of error exceeded 100%. In four of those years it exceeded 200%. In 1981 and 1982, it was more than three times higher than projected. In 1975, it was close to five times higher than projected. In 1974 and 1979 it was only about one third and one half, respectively, of the original projection. The average error between projected and actual deficits as a percentage of the projection is 266% over the 12-year period. Excluding 1971, the average error is 120%. This is to say, on average, deficits have been either more than twice as much or less than half as much as forecast over the past 11 years. Again, it must be emphasized that the errors run in both directions over the whole course of study, without the slightest trace of improvement in accuracy or even consistency in direction of error in the later years. Certainly this should give policymakers pause in considering budget projections as an element of law or policy.
Unlike the other concepts, there can be little dispute over the meaning of "deficit." It is simply an accounting sum whose definition is arithmetically clear. The major problem with it as an instrument of fiscal or economic policy is that it is highly unpredictable.
Outlays And Receipts
Outlays and receipts would seem to be, like the deficit, simple accounting concepts. But this is not quite so because they are aggregates of many lesser accounts. The aggregate totals tend to obscure what happens in the component accounts, as we shall see. And the component forecasts definitely detract from apparent, forecasting successes of the aggregate totals.
Studying Tables 2 and 3, it is apparent that, in general, the forecasting errors are not so far off as those for the deficit. The highest error in estimating percentage change of outlays occurred in FY 1971 -- 72%. There were two very close forecasts in FY 1973 and 1974, but that trend was quickly reversed. The average error in forecasted percentage change in outlays for the 12 years was 29%. When the 12-year average is taken as the ratio between the forecasted change in outlays and the actual error, the forecasts were off by an average of 59.5%; that is, the change in outlays was on average 59% higher than that forecast from the previous year.
Turning to receipts, the record is far worse. This should not be surprising since receipts are dependent to a much greater extent on general economic conditions that affect the tax base. As we will see, federal efforts to forecast economic conditions have had poor results. The worst forecasts from the perspective of percentage change are the first and the last years of the 12-year survey (1971, 1982). Undoubtedly, the case of 1982 was in part the result of the Reagan administration-sponsored tax rate cut passed in 1981 after the FY 1982 forecast was published. But the possibility of significant legislative changes is but another reminder that forecasts are likely to prove unreliable outside an academic context where other factors can remain "constant." Both in terms of the dollar magnitude of the errors and in terms of the error in percentage change, there is no discernible pattern to the size of the mistakes. The average error in forecasting percentage change in receipts is 91%. Excluding the 1982 forecast, the average error remains at 53%.
There are factors behind the aggregate totals for outlays and receipts that make even these figures too generous an assessment of forecasting accuracy. In the case of outlays, for example, a considerable portion of the total is, in theory, uncontrollable. Generally, there are two types of uncontrollable outlays. One is the entitlement programs, with expenditures triggered to economic conditions -- unemployment insurance in time of recession, for example, or indexed increases in Social Security, Medicaid, and other
FEDERAL BUDGET OUTLAYS
|Error(billions of $)||-10. 6||-2.7||-.2||+.3||20.2||-17.1||-7.8||-11||+6||-47.6||-41.2||+10.6|
|Error(estimating % change)||72%||13%||1%||1%||36%||41%||22%||22%||14%||56%||53%||15%|
|Error as % of
|Table 3 FEDERAL BUDGET RECEIPTS|
|Error in billions of $||+13.7||+9||-11.4||-8.9||+14||-2.5||-6.5||-9||-26||-17||+.8||+94|
|Error in % change||258%*||45||48%||27%||87%||13%
|Error as % of projected change||163%||31%||93%||37%||47%||15%
*Based on actual change of -5.3 billion from $193.7 FY 1970 receipts.
entitlements during inflationary periods. This sort of uncontrollable makes forecasting outlays more difficult because it hinges on general economic conditions. On the other hand, the second type involves accounts whose budget authority has already been fixed by previous Congresses. Since the spending scheme for many accounts runs over several years, federal forecasters begin their task with a major portion of outlays already known. The task should be much more easy, but still the errors are significant.
Part of this paradox is the fact that many of these uncontrollable accounts have also proven uncontrollable in a pejorative sense. Many are subject to serious cost overruns. In recent years the Government Accounting Office has tried to monitor the cost overrun phenomenon. Even though the GAO had to rely on the agencies themselves to provide the mass of data necessary to audit the multibillion-dollar cash flows, it assembled some startling statistics. (See Table 4, originally published in 1982, and valid through September 30, 1981.)
Even though this is only a partial list of federal accounts and the GAO was not able to validate the data for accuracy, the cost overrun on projects depicted by this partial list is almost $317.7 billion. Not all of this is charged as an outlay for FY 1981. It represents cumulative overruns on accounts which the federal government was servicing as of September 30, 1981. Still, given the magnitude of the over-runs and the unavailability of validated data, the difficulty of drawing reliable forecasts for outlays should become apparent.
Another point to keep in mind in looking at the aggregate outlays figure -- which does inadvertently enhance the forecasters' record -- is that many of the erroneous estimates for the individual accounts that add up to total outlays cancel each other out. Errors of underestimation for many accounts are mitigated or canceled out by errors of overestimation for other accounts.
Studying 50 large accounts for fiscal years 1977 and 1978, for example, the GAO notes that there were actually $27 billion in combined over- and underestimates of FY 1977 outlays. This contrasts sharply with the outlay error depicted by Table 2, where the gross outlay error for all government accounts shows up as merely $7.8 billion. Of the original mistakes in the 50 accounts, however, $10.4 billion were overestimates and $16.4 were underestimates -- canceling out to a $6 billion aggregate error for the 50 large accounts. The same 50 accounts contained $35 billion in wrong estimates for FY 1978, but this was disguised by the cancelling effect of $12.3 billion in overestimates and $22.3 billion in under-estimates.
Federal Cost Growth from Initial Congressional Budget Estimates
|Department or Agency||No. of Projects||Current
|Current over initial|
|$ Variance||% Variance|
|Architect of the Capitol||2||268.4||122.9||145.5||118|
|Appalachian Regional Commission||9,533.0||805.0||8,728.0||1,084|
|Corps of Engineers||130||39,743.3||10,583.6||29,159.7||276|
|Corp. for Public Broadcasting||1||60.7||57.7||3.0||5|
|District of Columbia||2||186.4||176.0||10.4||6|
|Dept. of Health & Human Serv.||3||289.6||236.0||53.6||23|
|Dept. of Justice||1||57.2||55.4||1.8||3|
|Dept. of Commerce||8||1,832.3||1,041.6||790.7||76|
|Dept. of Defense||137||427,241.0||184,117.3||243,123.7||132|
|Dept. of Energy||9||2,218.5||1,270.2||948.3||75|
|Dept. of Interior||22||10,830.9||3,973.5||6,857.4||173|
|Dept. of Transportation||6||3,134.1||2,638.9||495.2||19|
|General Services Administration||4||332.1||329.1||3.0||1|
|Nat'l. Railroad Passenger Corp.||6||887.5||761.8||125.7||17|
|Nat'l. Science Foundation||1||78.2||76.0||2.2||3|
|Pennsylvania Ave. Develop. Corp.||1||223.0||223.0||-||-|
|Dept. of State||3||304.8||257.6||47.2||18|
|Dept. of Treasury||1||174.0||210.0||(36.0)||(17)|
|Tennessee Valley Authority||10||20,043.5||3,596.9||16,446.6||457|
|Washington Metro Transit Authority||2||5687.0||2,559.6||3,127.4||122|
If the gross dollar error of forecasted outlays for 1977 were changed from $7.8 billion to $27 billion, then the error in forecasting percentage change in outlays would jump from 22% to 76%. Doing the same for FY 1978, the error jumps from 22% to 99%. Thus, the apparent improvement in forecasting outlays, compared with either receipts or the deficit, is in large part the result of random cancelling-out rather than demonstrative forecasting skill.
With respect to receipts, the cancelling-out phenomenon proves to be much less of a factor. There are far fewer accounts, and the major accounts -- individual income, corporate income, and Social Security taxes -- all generally react the same to upturns or downturns in the performance of the national economy. Forecasting receipts from the national tax base, then, inevitably points toward forecasting GNP, unemployment, and inflation.
Gross National Product
No aggregate computation is larger or more problematic than Gross National Product. It is one of the best reasons to distrust economic aggregates and forecasts of aggregate changes. The media, however, report official announcements of GNP calculations without a trace of reservation, and policymakers increasingly embrace the figures as guides to policy and legislation.
Efforts to assess national product or national wealth have been a lure to academic economists at least as far back as Adam Smith's Wealth of Nations. Since the 1930s governments such as the United States have drawn on academic efforts to assess national product as a means, at the very least, of assessing their tax base, but also to collect other data that might help in formulating economic policies. Despite the fact that GNP has been carried afield from academia, however, academicians are still not agreed on the correct definition of the concept, or on the way to compute it, or in some cases, whether it can be meaningfully computed at all. As a result, GNP figures are subject to repeated "revisions," not only because of the need to have the most accurate data, but also because of chronic disputes over the concepts under which the data is grouped. To those who will take heed, the government's own agencies issue disclaimers on this point:
Federal Budget Gross National Product Forecasts
(year-over-year percentage change)
|Margin of error
|Error in forecasting
|Error as % of projection||14%||14%||33%||108%||10%|
Prior efforts in addressing the issues surrounding GNP's accuracy and reliability, methods and concepts have brought changes to the estimates.... Nevertheless, issues and problems concerning the estimates still exist.... The lack of precise error measures for the estimates -- even though there are valid reasons why they cannot be provided -- limits users' and researchers' capability to judge how accurate the estimates are and should be.... Given the size and complexity of the accounts, we are not certain if the questions surrounding the issues can ever be fully answered.
Because the gross dollar amount of GNP figures has been subject to revision every few years between 1971 and 1982, and because it normally takes up to two years before the relevant data can be gathered and factored into the equation for any given fiscal year, it would be misleading to compare actual figures computed under one conceptual framework with estimates made under another. The focus instead will be on the forecasts of percentage change from year to year which, according to Commerce Department economists, are less subject to fluctuation as a result of conceptual revisions than the gross dollar figures.[7,8]
While none of the forecasts depicted by the table is quite accurate, the margin of error does not appear to be very wide. But as economist Alex Rubner points out, the margin of change in GNP accounts for Western industrialized nations since the close of World War II has never been more than a few points in either direction. More violent fluctuations are simply not to be expected. This narrows the acceptable margin of error considerably. Of the five years covered by the table, the percentage change has never been greater than five percentage points. Suppose we assume, generously and along lines suggested by Rubner, that the most a forecaster could possibly be amiss is by five percentage points -- that is, five points would constitute the largest possible error, or a 100% error. Each percentage point of error, then, corresponds to an actual forecasting error of 20%.
Assuming that the "actual" GNP figures are reliable, we see an array of errors from 4% to 54%, with an average error over the five-year period of 24%. Also worth noting is the fact that the percentage of error is much greater for the long-range forecasts aimed beyond the upcoming fiscal year. Taking the forecasts for only the second year ahead, where data are available (1978-1981), the errors range from 30% to 100%, with an average error of 57.5%. Except for 1981, all of the forecasts are shown to have been overly optimistic, and wildly optimistic projections for that year were only brought down in the immediately preceding year. Given the standard we have imposed, the forecasting errors are quite large. But whether an average error of one percentage point (20%) of change in GNP is tolerable depends in large part on the purpose of the forecast. As an academic exercise, it may be considered acceptable. As a factor in shaping public policy or determining federal expenditures, a one-percentage-point error can certainly have unfortunate ramifications.
As to what it tells about the growth of the national economy, the concept itself is open to objections that it factors more than it can explain and also needs to factor in much more than it does in order to fulfill its ambitious goal. It does both too much and too little. In factoring vast collections of statistics and data, it loses sight of the meaning of many particulars making up the GNP. Department of Commerce economists recognize, for example, that measurements of (and hence errors in) many types of data are simply not commensurate with measurements of many other types of data. What brings coherence to the disparate collection of economic data is not scientific measuring procedures but the fortuitous mutual cancellation of different errors within the aggregate, much like the federal outlays discussed above. Thus, even if it may prove useful, the GNP distorts and obscures component parts of the economy that make up the aggregate.
Yet granting the formidable level of aggregation, GNP also fails to take significant aspects of the economy into account. Or where it does so, the interpretation is controversial. The eminent economist, Oskar Morgenstern, focused critical attention on conceptual and statistical difficulties in computing vast economic aggregates such as the GNP in his book, On the Accuracy of Economic Observation, published 20 years ago. The problems raised by Morgenstern continue to diminish the reliability of GNP calculations and thus bear repeating.
A major omission of long dispute is that the GNP limits itself to computing cash flows (either income or expenditures) and thus either ignores economic growth where there is no money transaction or imputes its own value to that growth. Cash flows are the focus of GNP accounts because they are more easily countable. This does hold the compilers to a more rigorous standard, but it excludes major contributions to economic wealth such as the work of housewives, child home labor, and volunteer work, as well as unrecorded cash flows in the underground economy. In other cases of accrual of value without cash flow, the Commerce Department economists venture to impute their own assessment of value, as in the case of assigning rent values to all residential housing in America. This latter is, as Morgenstern observes, "a tricky affair."
Then there is the problem of how to factor in government enterprise. Some argue that many government expenditures actually diminish GNP. Others feel that government expenditures are neutral or "intermediate" in that they do not create wealth themselves but only lay the groundwork. Others, including those who currently define official GNP components, simply calculate all government expenditures as an addition to GNP.
Objectively assessing capital depreciation is yet another nettlesome issue. Standard concepts of depreciation, such as "double declining balance," "sum of the years digits," LIFO, FIFO, etc. are merely accounting fictions and frequently unrealistic guides to true capital depreciation. Moreover, the Commerce Department has to rely on private businesses' own estimates of capital depreciation. Beyond the fact that even the most meticulous calculation using one of the standard accounting procedures can prove deceptive, many businesses are likely to have unrealistic assumptions about the impact of inflation or taxes on their capital stock.
In this vein, we turn to the seemingly inevitable sampling errors in the collection of original data. Much of the data comes from government agencies such as the Bureau of Labor Statistics, the Commerce Department, and the Census Bureau. It may be prudent not to accept the data on face value. For example, by the Census Bureau's own recent estimate it missed over 4 million people in the census of 1970. This was but a small percentage of the vast aggregate total U.S. population, but it was still more people than counted for the entire state of Maryland or the city of Chicago. Certainly the four million discrepancy has a major impact on the national product. The influx of millions of illegal aliens in the 1970s -- many of whom surely contribute to the GNP --further damages the credibility of the census figures used in computing GNP. Other sources of federal data are also suspect, as we shall see as we turn to the unemployment rate and the CPI.
An important issue that Morgenstern did not consider is the "underground economy" in the United States. It is made up partly of illegal economic activity such as trade in drugs and stolen property, but it also encompasses legal economic activity conducted on a cash basis such as self-employment, moonlighting, and employment of illegal aliens. The Commerce Department does not factor all of this potentially huge cash flow into its GNP calculation. Estimates as to the size of the underground economy vary widely, but even conservative estimates placed the figure as high as $100 billion as long ago as 1976. Other estimates are as high as $369 billion for 1976, which is 22% of the entire Gross National Product calculated for that year.
It is difficult to determine exactly how much of this underground economic activity is reflected in the cash flows used to measure the GNP. No illegal transactions are factored into GNP, but legal transactions made with earnings from underground activity will show up in the cash flows which the Commerce Department studies. By any account, however, the large amount of underground economic activity alone is enough to question official GNP calculations (to say nothing of other official economic aggregates such as the unemployment rate). A Congressional Research Service economist underscores this point:
An active, expanding underground economy can present serious problems for economic policymakers. Data purporting to show the state of the economy will actually provide a misleading picture of existing conditions. An example of this may have occurred in 1978 and 1979. At that time many analysts were predicting that an economic downturn was imminent. They also expected such a recession to last about one year. Concern over imbalances in the consumer sector, particularly a sizable increase in consumer debt, was largely responsible for these forecasts. The recession occurred much later and was much shorter than predicted. Because of underground activity consumer incomes may have been considerably higher than reported, thus consumer ability to take on new debt obligations may have been greater than believed, accounting for the actual course of events and misleading forecasters and policymakers....
However, it is abundantly evident that knowledge of the magnitude of underground activity is not very advanced. Precise statistical indicators which can be explicitly factored into the decisionmaking process do not exist. Perhaps the best course of action is to follow the advice of economist James Henry. In testimony before the Joint Economic Committee Henry counseled, "Overall, we need to think much more rigorously about conducting economic policy in an environment where most economic variables -- not just target variables like unemployment, employment, and inflation, but also planning variables like money supply, effective tax rates, and the Federal deficit -- are measured with error."
This should make it clear that along with the disappointing efforts to forecast supposed changes in the Gross National Product, the reliability of the concept itself as an accurate portrait of economic reality is also in question. Similar questions arise with respect to other official economic aggregates.
Upon cursory inspection, unemployment projections appear to have an accurate track record. A closer look, however, indicates an inability to forecast dramatic shifts in the unemployment rate. As in the case of GNP changes, a small percentage change actually counts for a large number of unemployed. Following Rubner's suggestion of imposing a standard of error on GNP, we will do the same for the unemployment rate. Two or three percentage points over the course of a year is about the largest swing one way or the other that we are likely to experience. Taking 3% as our widest possible margin of error, the rule of thumb would be that forecasts are off by 33.3% for every percentage point they fall off from the actual count.
Thus, the 1978 budget forecast erred by 17%, underestimating the cyclical turn toward economic recovery. As the brief recovery aborted in 1980, budget forecasters again missed the cue; the 1980 budget projection was amiss by approximately 30%. As the recession deepened in 1982, projections this time were off approximately 73%.
Because federal budgets didn't publish unemployment forecasts prior to 1976, it may be unfair to comment on unemployment assumptions between 1971 and 1975. Nevertheless, this period was the era of President Nixon's much-vaunted "full employment budget" concept. This was an ambitious attempt to manage the performance of the national economy -- particularly to smooth out cyclical turns -- through the impact of federal fiscal policy.
According to President Nixon, the federal government would simply spend the same amount of revenue that was assumed to be collectible with the economy at "full employment" -- then defined as 4% unemployment. If the economy was "fully employed," the fiscally responsible federal government would spend no more than it took in. If the economy was tending toward a higher level of unemployment, however, the resultant deficit spending would stimulate the economy, nudge it toward 4% unemployment, and ultimately close the deficit with full employment revenues. The president called this scheme a "self-fulfilling prophecy." The statistics appear to contradict these optimistic claims. The administration managed to achieve the deficit but not the resulting reduction in unemployment.
Federal Budget Unemployment Projections
|Error in % change||25%||33%||50%||33%||69%||40%||105%|
|Error as % of
|Margin of error
From 1969, when the full-employment scheme was officially embraced, to 1970, unemployment rose by 1.4% to 4.9% (from 3.5% in 1969). The rate rose another full point to 5.9% in 1971 and remained between .9% and 1.6% above the full-employment assumption for three more years before leaping to 8.5% in 1975. Treasury forecasters relying on the 4% full-employment assumption for 1975 would have been amiss by some 150%, given our margin of error (3 points = 100%). Indeed, the full-employment episode offers a profitable case study in the pitfalls of relying on economic aggregates in setting fiscal policy.
Returning to the post-1975 statistics, Table 5 indicates that forecasters of unemployment were fairly accurate as long as the economy remained on a plateau. When significant turns occur for better or worse in the economy, forecasters begin to err by wide margins.
With regard to the accuracy of the concept itself as a measure of economic reality, we find that as with GNP, the unemployment rate is too easily taken at face value. Changes in the rate in one direction or the other are frequently the subject of alarm or self-congratulation among economic policymakers. A rising unemployment rate is often portrayed by the mass media as a stagnant pool whose membership is being added to with each month's figures. In fact, the pool is substantially changing, with many leaving to take up new work even if there is a net increase. The rise in the rate could simply reflect economic change as certain occupations are abandoned for others. The change may be necessary and economically healthy -- perhaps not something that government should attempt to reverse.
New jobs may not be opening up as fast as unprofitable older jobs are closing, but even this can be relative to the length of unemployment. It is certainly conceivable that a rise in the total unemployment rate could simply be a function of a temporary time lag between the opening of new jobs and the abandonment of older ones. Whatever the cause, there is a large and continuing segment of the unemployed who are finding work each month but only after a longer than average period of unemployment. A Census Bureau economist cautions against viewing the unemployed as a long-term homogeneous group. Upon closer analysis of the duration of unemployment among those making up the aggregate statistic, he notes that even at the height of the 1982 recession in November, the number of people who had either been looking for work for more than 49 weeks or who had become totally discouraged and ceased looking was 1.5% of the labor force, contrasted with an aggregate unemployment rate of 10.7%.
How the ranks of the unemployed change, and how during periods of a lower aggregate unemployment rate massive numbers of workers might be temporarily unemployed but find work relatively quickly, can be illustrated by the following example. For 1981, the aggregate unemployment rate for the year was 7.6%, yet the total percentage of the workforce experiencing unemployment during the year amounted to 18.4%. The average duration of unemployment during 1981 was 13.3 weeks. It is not difficult to imagine that an extension in the duration of average unemployment by even a few weeks could create enough of a bottleneck in the transition from job to job to have a very significant impact on the unemployment rate. But even an average layoff of, say, 20 weeks need not necessarily evoke the concern that is incited by a 10% aggregate unemployment rate.
The aggregate unemployment rate also disguises the fact that even during what are relatively low unemployment years, there are pockets of hardcore unemployment. Many people in this category do not even enter the Labor Department's calculations, as they have given up the search for work and also may not have a permanent address where phone canvassers can reach them. As a result, the Labor Department may be unable to ascertain the true level of unemployment, let alone depict the nation's employment levels via aggregate rates.
Probably the most challenging obstacle to assessing the true level of joblessness is, again, the existence of a significant underground economy. Estimates of the size of the underground labor force are inconclusive, but figures run as high as 2.35 million workers. This figure includes workers who moonlight, or take for-cash pick-up work while on strike or unemployment. It includes those engaged in criminal activity, as well as self-employed contractors and professionals who report no taxes to the IRS. Illegal aliens are probably its largest component.
Until such issues as the time span of unemployment, the underground labor force, and the pool of permanently discouraged workers are dealt with comprehensively, the media and policymakers both would do well to look at this concept -- and predicted changes in its magnitude -- more critically.
The record of the budget forecasts for changes in the Consumer Price Index is obviously very poor. For 1979 and 1980 the rate of inflation as measured by the CPI was virtually twice that forecast. Again following Rubner's suggestion, if we assign a possible margin of error as a 5% change in year-over-year figures, then each percentage point the forecasts err counts for a 20% deviation. Using such a measure, the forecasting errors range from 10% in the 1977 budget to 136% in the 1980 budget. The average over the seven-year span for which forecasts were made was 63.4%. Forecasts beyond the upcoming budget year are, with few exceptions, much worse. The average error in assessing the percentage change in the CPI was 130%. And as a percentage of what was originally forecast, the average error was 759%, or 186% if 1982 is excluded.
This unreliability is overshadowed, though, by the widespread disenchantment with the CPI itself as an accurate measure of price changes and inflation. To a far greater extent than either the GNP or the unemployment rate, the CPI has been the object of well-merited scorn by professional economists. Many of the arguments against the concept were discussed in a 1981 article by Prof. Robert J. Gordon. In reviewing these objections, it becomes apparent that the main problem, like that of the GNP and the unemployment rate, is that the aggregate concept is insensitive to the thousands of possible components that it is supposed to represent.
The CPI is based on a hypothetical household expenses list -- a shopping basket, as it's colloquially known. Precisely what items are included and how they are weighted are, of course, matters of contention. But the issue is made more dramatic by the fact that major revisions in the shopping basket are made only on the average of once a decade. By the time the new survey is taken and the arguments settled over what to include, and how much to weight each item, several years can pass. Prof. Gordon notes, for example, that despite the fact that America became an automobile-oriented society in the 1920s and 1930s, autos were not added to the CPI until 1940. Computers are still not a component.
Next, an issue arises regarding the weighting of specific items known as the "substitution effect." Not all prices for the items chosen for the shopping basket are likely to rise at the same rate. Indeed, some may actually fall over time. But as selected items do rise substantially in price, it seems reasonable to assume that the typical consumer substitutes other items for the more costly ones. Thus, if the price of beef rises, consumers may buy chicken as a substitute and then move back to beef when relative prices change again. If fuel prices rise dramatically, people may substitute smaller autos. But the CPI's rigid system of weights is insensitive to such changes. If when the survey was taken it was established that the average household needed enough beef for six meals a week and enough gasoline to fuel a V-8, that is how the weights will remain. Of course, it is probably impossible to develop an index that is truly sensitive to the thousands of patterns of substitutions involving millions of families and goods.
Federal Budget C.P.I. Projections
(in percentage change year-over-year, all urban consumers)
|Error in % change||40%||10%||46%||104%||136%||24%||84%|
|Error as % of
|Margin of error
Another aspect of the CPI's gross insensitivity to real economic change is that it fails to account for important qualitative changes in goods whose prices have demonstrably risen. The improvements can be so vast, that, in essence, it is not even the same good that originally entered the CPI hypothetical list. Hospital costs have risen sharply, for example, but by most accounts the quality of care has improved dramatically. Airline tickets rise in price, but computerized reservations make the entire process of booking a flight more convenient. Gordon cites a study of auto tires, where it is claimed that improvements between 1935 and 1978 have actually decreased the tire-mile cost of tires by 9%; the CPI shows a 140% price increase. A similar study on the oil-mile price of motor oil shows a 52% decrease versus a 234% increase registered over the same period by the CPI. Like the substitution effect, the ignoring of product quality appears to be an inherent defect in any scheme to index the thousands of varieties of consumer goods.
Until just recently, the CPI showed a large weighted bias toward housing costs, with average housing costs assumed to be 30% of household income. This may be the case for some purchasers, but it certainly does not hold for households generally. The CPI assumed that everyone was a recent homebuyer. To make matters worse, it failed to depreciate even this artificially high computation with federal income tax savings and real estate appreciation. On the contrary, it actually double-counted housing expenses by calculating the cost as the list purchase price and then adding all the mortgage payments in later years.
So obviously flawed was the treatment of housing costs that a chorus of economists protested that it be changed. Indeed, a commission of experts recommended changes in the assigned weight of housing in the CPI as long ago as 1977. It wasn't until January 1983, however, that the CPI housing component was finally revised along lines suggested by the commission. It was apparently for political reasons that the change was refused for so long by the Bureau of Labor Statistics. Certain special interest groups -- particularly organized labor, Social Security recipients, and federal and military retirees -- had a definite stake in maintaining the upward bias of the CPI.
The situation is a good case study in the danger of attaching government policies to vague economic aggregates. Since many federal entitlement programs are indexed to the CPI, federal outlays in selected areas have increased dramatically. In one year alone, according to Professor Gordon, cost of living increases based on rises in the CPI were awarded to 31 million Social Security beneficiaries, 2.5 million retired federal and military employees, 20 million food stamp recipients, and 25 million recipients of subsidized school lunches. As with their uncritical acceptance of other official economic aggregate figures, private businesses have been no wiser than the government in many cases: Collective bargaining agreements governing 8 million unionized workers are also tied to changes in the CPI.
In analyzing the accuracy of federal budget projections, there is ample evidence that official figures for the deficit, outlays, and receipts, the Gross National product, the unemployment rate, and the Consumer Price Index should be examined more critically. The accuracy of the predictions has generally been poor, and the meaning of several of the concepts is subject to serious misinterpretation.
The forecasts of the impending federal deficit show an average error in assessing percentage change of 529%. Only three times between 1971 and 1982 has the estimate been within 75% of the actual deficit.
Forecasts of federal outlays are shown to err in the assessment of percentage change by an average of 28.8%, but the aggregate disguises even more egregious forecasting errors that cancel each other out. Forecasts of receipts err by an average of 91% in assessing the percentage of change.
GNP forecasts display an average error in predicting percentage change of 36% and an average gross margin of error of 24%, according to the margin of error we have used. In addition, the concept of Gross National Product is beset with seemingly inherent conceptual and data gathering problems. It relies very often upon imputed values and incomplete data and excludes large segments of the economy such as domestic work and underground economic activity.
By comparison, unemployment forecasts have been fairly accurate, but they show serious errors in assessing cyclical turns in the economy. As with GNP, more serious problems arise with the concept of the unemployment rate itself. It fails to show the changing composition of the unemployed and provides an inaccurate reflection of hardcore unemployment areas and of the underground workforce.
Budget forecasts of changes in the Consumer Price Index erred by an average of 63% over the 12 years studied. Moreover, the CPI as a concept has been widely discredited as a measure of economic reality. Its survey changes are sluggish, making the index chronically obsolete. It fails to account for substitution effects and quality improvements in the goods it purports to reflect, and it has been thrown out of balance by its idiosyncratic weighting of housing costs.
The problems of both predictive accuracy and conceptual coherence tend to diminish the claim that economic forecasting is truly a scientific endeavor. In fact, it is little more than an effort to ascertain present trends in the economy and project them into the future. Concluding just what the trends are in the first place is in itself an extremely controversial effort, as the debate over problems in counting the GNP, the CPI, or the unemployment rate makes clear. Yet academic efforts to gauge the economic past and present are valuable since they add to our store of knowledge. As the precondition to forecasting the future, however, uses of such aggregates are bound to be misleading. Even if the aggregates were correctly assessed for the past and present, we would still need to assume that no substantial unforeseen changes would enter into the countless relationships (including potential relationships) of consumers, products, competitors, etc. that exist in our modern global economy. This would certainly be an impossible, even foolish assumption.
These problems are not peculiar to the government. Private forecasts have been shown to err widely and regularly. None of the private models are consistently right, and most are consistently wrong. It may seem that since private firms make use of economic forecasts in planning their future, government is equally obliged to make use of this tool in its own planning. But there are good reasons why the tendency to rely on forecasts can have far more mischievous effects in government than in the private sector. First, private firms relying heavily on erroneous forecasts accept the responsibility when they fail. The government, on the other hand, takes its losses out on the tax base. Also, private industries tend to adapt quickly and change their plans as forecasts are gradually proven wrong. Governments, by contrast act sluggishly in adapting to changing economic conditions, whether it be a matter of trimming expenditures, adjusting taxes, reformulating policies, or revising laws. The forecasting mistakes of private firms are likely to be changed by managers sensitive to potential losses, while government forecasts can become rallying points for vested interests that often lobby against changing policies predicated on mistaken assumptions. Thus, any legislation or policy that is linked to questionable changes in economic aggregates is more resistant to revision than private-sector planning.
There may be an even more important reason to be wary of predicted changes in economic aggregates as a part of policy formulation. The reason has to do with government candor. In the interest of government accountability, we rightly expect that the federal government will explain in detail what its budget plan is and describe how it calculates its plan. Unfortunately, the use of complicated forecasting techniques, together with unwieldy concepts of economic aggregates, runs the risk of lulling people out of critical thinking on budget assumptions. Voters and taxpayers are thereby encouraged to leave such matters to "the experts," which can create an illusion of official omniscience.
This is not to suggest that the federal government should cease trying to establish long-range budget plans or cease to reveal its assumptions about those plans. It is rather to suggest that the government show more candor in what it cannot do and then take its own admonition seriously. It is to suggest making explicit that economic forecasts have usually been wrong in the past and that they will probably prove erroneous in the future; to make clear that macroeconomic concepts are too easily taken as simple "buzzwords" when, in fact, they are very imprecise and potentially misleading. The government, like everyone else, must operate in a future fraught with uncertainties that no one, least of all its experts, can foresee clearly. This would mean shunning legislation that links economic policy to forecasts or even to the historical computation of macroeconomic aggregates. Above all, it would mean rejecting those policies -- whether industrial policies, foreign trade plans, or budget formulas -- which assume an ability of the government to successfully forsee and plan economic conditions. Economic policymakers should be more honest with themselves as well as with us.
 U.S. General Accounting Office, A Primer on Gross National Product Concepts and Issues, April 8, 1981, pp. 6-7.
 Even this simple accounting term has been subject to recent controversy over its definition in the federal budget, but this controversy seems easily resolved. Many federal accounts are "off budget" and if included in the budget, would add substantially to the size of the deficit and to the overall size of the budget. The off-budget accounts are so grouped because, in theory, most are self-financing, such as the Post Office, the Federal Financing Bank and the Synfuels Corp. Many have been running seriously in the red, however. The U.S. Comptroller General notes that inclusion of these figures in the 1981 Budget would have increased the total for federal outlays by $120 billion, for receipts by $102 billion, and the size of the deficit by $18 billion. Although this situation does not pose an insurmountable problem in defining the deficit -- the off-budget accounts can simply be added to the overall budget -- it does tend to make the figures reported by the President's budget misleading to citizens, the mass media, and policymakers alike. The Comptroller General warns that "the excluded amounts are now so large that they limit the unified budget's usefulness as a controlling device." See U.S. Comptroller General, Federal Budget Totals Are Understated Because of Current Budget Practices, December 31, 1980, p.2.
 U.S. Comptroller General, Status of Major Acquisitions as of September 30, 1981: Better Reporting Essential to Controlling Cost Growth, April 22, 1982, p. 8.
 U.S. Comptroller General, Federal Budget Outlay Estimates: A Growing Problem, February 9, 1979, p. 3.
 A study by the Congressional Budget Office notes that of these three major components of aggregate federal receipts, corporate income taxes were the most difficult to estimate in budgets between 1963 and 1978. CBO, "A Review of the Accuracy of Treasury Revenue Forecasts, 1963-1978," Staff Working Paper, February 1981, p. 13.
 GAO, A Primer, April 8, 1981, pp. 56-57.
 Author's personal communication with Thomas J. Cuny Historical Data Division, U.S. Office of Management and Budget.
 Since the 1977 Budget is the first in which forecasted percentage changes in GNP are published, using constant 1972 dollar figures, only fiscal years 1977-81 are covered in this analysis.
 Alex Rubner, Three Sacred Cows of Economics (London: MacGibbon & Kee, 1970), pp. 117-118.
 Economists specializing in the measurement of GNP have been reluctant to suggest a tolerable margin of error with which to assess their computations. They point out that such a gauge may be theoretically impossible because so many of the individual components making up the GNP are subject to different kinds of errors that can't be measured in the same way. See GAO, A Primer, pp. 21-28. However professionally unsound it may seem for academic economists, the fact that GNP figures are increasingly used as instruments of government policy requires, at least in the interest of government accountability, that some standard of measure be used to assess the government's record. Thus, following Rubner, we impose a margin of error drawn from the record of recent historical changes in GNP. See Rubner, Three Sacred Cows of Economics, pp. 117-118.
 GAO, A Primer, pp. 21ff.
 The major passages concerning GNP computation can be found in Oskar Morgenstern, National Income Statistics (Washington, D.C.: Cato Institute, 1979).
 U.S. Bureau of the Census, Coverage of the National Population in the 1980 Census by Age, Sex, and Race, February 2, 1982, p. 6.
 GAO, A Primer, p. 53.
 Barry Molefsky, America's Underground Economy, Congressional Research Service, July 28, 1981, pp. 35-36.
 "Budget Message of the President," Federal Budget, FY 1972, p. 7.
 Dave M. O'Neill, "Short, Medium and Long Unemployment: The Numbers Don't Tell the Whole Story," Washington Post, February 17, 1983, p. A19.
 Molefsky, America's Underground Economy, p. 27.
 Robert J. Gordon, "The Consumer Price Index: Measuring Inflation and Causing It," The Public Interest, Spring 1981.
 A review of the factors that militate against our ability to forecast future economic activity can be found in Ralph Harris, "A Skeptical View of Forecasting in Britain," in James B. Ramsey, Economic Forecasting -- Models or Markets? (Washington D.C.: Cato Institute, 1980).
 See Brian McAndrew, "The Failure of Econometric Forecasting," Policy Report, (Cato Institute), November 1981.
© 1983 The Cato Institute
Please send comments to webmaster