Many governments have explicitly or implicitly engaged in policy experimentation in various forms to resolve policy uncertainty and to facilitate policy learning. Sophisticated policy experimentation has included sequences of trial and error and rigorous randomized control trials in subregions of a country. Few policy experiments, however, can compare with the systematic policy experimentation in China in terms of its breadth, depth, and duration. Since the 1980s, the Chinese government has been systematically trying out different policies across regions and often over multiple iterations of one or more policies before deciding whether to roll out the policies to the entire nation.
This project aims to describe and understand China’s policy experimentation since the 1980s. Many scholars have argued that the pursuit of extensive, continuous, and institutionalized policy experimentation was a critical mechanism that led to China’s economic rise over the past four decades. Nonetheless, surprisingly little is understood about the characteristics of policy experimentation or how the structure of experimentation may affect policy learning and policy outcomes.
We focus on two characteristics of policy experimentation that may determine whether it provides informative and accurate signals on general policy effectiveness. First, to the extent that policy effects vary across localities, representative selection of experimentation sites is critical to ensure unbiased learning of the policy’s average effects. Second, to the extent that the efforts of key actors (such as local politicians) can play important roles in shaping policy outcomes, experiments that induce excessive efforts through local political incentives can result in exaggerated signals of policy effectiveness.
We ask three questions. First, has the sample selection in China’s policy experiments been representative? Second, do policy experiments create additional incentives and induce extra efforts that are not replicable outside the experimentation? Third, how do the nonrepresentative sample selection and nonrepresentative experimental situation affect government’s policy learning and shape national policy outcomes?
To answer these questions, we collect comprehensive data on policy experimentation in China between 1980 and 2020. Based on 19,812 government documents, we construct a database of 633 policy experiments initiated by 98 central ministries and commissions. For each policy experiment, we link the central government document that outlines the overall experimentation guidelines with all corresponding local government documents to record its local implementation, and we trace its rollout across the country. We measure various characteristics of policy experiments based on the associated government documents and other linked data sets, including uncertainty about policy effectiveness, career trajectories of central and local politicians involved in the experiment, the bureaucratic structure of the ministries initiating these policies, the degree of differentiation in policy implementation across local governments, and local socioeconomic conditions.
We begin by investigating the selection of experimentation sites. A primary goal for the central government is to learn from a balanced, representative sample, as prescribed by the National Development and Reform Commission, which oversees many key experiments. Nonetheless, when we compare the preexperimentation characteristics of the locations that are selected as test sites with those that are not, we observe that more than 80 percent of the experiments were conducted in sites that are favorably selected in terms of local economic conditions. Such deviation from representativeness cannot be fully justified by optimal experimentation considerations. Rather, we document that nearly half the observed positive selection can be accounted for by misaligned incentives across political hierarchies. Specifically, the level of promotion incentives faced by local politicians (which is greater for politicians who are sufficiently far away from retirement and for those who have ample room for upward mobility) shapes their participation in the experiments, and political patronage affects how ministers choose experimentation sites.
Next, we examine whether policy experimentation induces politicians’ strategic efforts during experiments, thus generating nonrepresentative experimental situations. We find that during experimentation, local governments spend almost 5 percent more of their funds in the domains relevant to the policy on trial; this is particularly the case for politicians facing stronger promotion incentives. Such an increase in fiscal support is absent when the policy rolls out to the entire country. Moreover, we find that, among local politicians participating in a specific policy experiment, those facing greater career incentives act significantly differently in terms of policy implementation than those politicians who are not facing such strong career incentives. Such differentiation and potential recognition by the central government could earn local politicians substantial political credits.
Finally, we investigate whether the presence of positive selection in experimentation sites and local politicians’ strategic efforts during experimentation affect the central government’s policy learning and national policy outcomes. We present evidence that the central government does not fully account for positive sample selection or strategic local effort when evaluating policy experimentation. Experiments conducted in favorably selected sites are substantially more likely to be promoted to national policies. When experimentation sites undergo positive shocks in fiscal resources (due to externally induced, unexpected land revenue windfalls during the experimentation) or political incentives (due to local politician turnover occurring during the experimentation), the policies on trial are significantly more likely to be rolled out as national policies despite the fact that the innate effectiveness of these policies is unrelated to those shocks. Furthermore, we find that evaluations of experimentation outcomes in the presence of biased sample selection and nonrepresentative experimental situations can influence national policy outcomes. When the trial policies are rolled out to the entire country, localities benefit substantially more from the policies if they share similar socioeconomic conditions with the corresponding experimentation sites or have comparable career incentives for local politicians. This could systematically bias the effectiveness of reforms in China and generate distributional consequences across regions.
Taken together, these results highlight that China’s remarkable policy experiments, as with any other undertaking in policy learning at this scale, take place in complex political and institutional contexts. On the one hand, certain institutional and bureaucratic conditions may serve as the engine to coordinate experimentation, to motivate politicians’ participation, and to stimulate local policy innovations. Thus experimentation can help circumvent political and bureaucratic frictions that may prevent reform and policy adoption. On the other hand, as our results suggest, the very same institutional and bureaucratic contexts also imply the presence of factors that could result in deviation from representativeness in both sample selections and experimental situations. If these characteristics of the policy experiments are not sufficiently accounted for, policy learning can be biased and national policy outcomes may be affected.
NOTE
This research brief is based on Shaoda Wang and David Y. Yang, “Policy Experimentation in China: The Political Economy of Policy Learning,” NBER Working Paper no. 29402, October 2021.
