Impact of pilot public hospital reform on efficiencies: a DEA analysis of county hospitals in East China, 2009–2015

Abstract

Background

China started a pilot public hospital reform in 2012 to improve governance and efficiency in healthcare services delivery among county-level hospitals. This study aims to investigate the impact of the pilot reform on hospital efficiency and productivity by using a unique dataset of county hospitals in East China during 2009–2015.

Methods

A three-stage approach is used. First, this study uses the output-oriented data envelopment analysis (DEA) to estimate hospital efficiency with variable returns to scale. Second, propensity score matching is used to address potential biases associated with the selection of counties for the pilot program. In the third stage, we assess the impact of the pilot reform on efficiency by using a Tobit Difference-in-Differences approach.

Results

The average level of hospital efficiency for the whole sample experienced a rapid drop in 2013, then returned to a peak in 2014. Except in the reform year (2012), the overall hospital efficiency for the post-reform period is higher than that for the pre-reform period. The baseline model results show that the pilot reform is associated with a 3% decline in pure technical efficiency and a 2.3% increase in hospital scale efficiency, respectively. Our findings are robust when we apply bootstrapped DEA efficiency scores and use different specifications.

Conclusion

The findings of this study suggest no improvements in overall hospital efficiency associated with the pilot reform, possibly due to the combined effects of inefficient governance and hospital scale expansion. This study suggests that further efforts are needed to increase county hospital performance by strengthening management and optimizing resource utiliziation.

Introduction

For developing and developed countries alike, assessing health system performance is critical to implementing appropriate strategies to control costs and efficiently use scarce resources [1, 2]. China’s health system has undergone tremendous transformation over the last couple of decades [3, 4]. The 2009 national health reform aimed to increase access to care, expand insurance coverage, and strengthen primary care capacity. However, the early accomplishments of the reform were accompanied by soaring health expenditures [4, 5]. The public hospital reform in 2012 attempted to rein in the increases in healthcare expenditure and improve the performance of China’s health system through the zero-markup drug policy and hospital governance. Previous studies show that the health reform has promoted health technology adoption and reduced inpatient spending but led to overtreatment in some settings [6,7,8]. However, few empirical studies investigate the policy effect on hospital performance and operational management, i.e., how public hospital utilizes limited healthcare resources, an important policy objective of China’s health reform.

This paper focuses on the change in hospital performance from the perspective of efficiency measures. Early research used efficiency as a comprehensive and encompassing indicator for evaluating the performance of health systems. Efficiency was defined as the attainable level of a health system to what it could achieve [9]. The farther from the ideal health attainment, the lower the efficiency ratio. Under this context, the World Health Organization (WHO) analyzed the comparative efficiency of health system performance in 191 member states [10]. As for methods of efficiency measures, parametric and non-parametric methods are the main approaches to efficiency measurement in health care [11, 12]. Giuffrida and Gravelle used two parametric estimators, corrected ordinary least squares (COLS) and stochastic frontier approach (SFA), to evaluate the performance of 90 family health service authorities in the UK [13]. Hollingsworth and Wildman used Data Envelopment Analysis (DEA) for efficiency measurement with data from the WHO study [10, 14]. Hunt and Link conducted a bootstrapped DEA approach to measure the technical efficiency of local hospitals in the US [15]. In this study, we use DEA to measure the efficiency of county-level hospitals in East China.

There is an extensive research body using DEA to measure efficiency and analyze determinants of efficiency in China’s health system. Jiang et al. analyzed the efficiency of a large sample of county hospitals in China from 2008 to 2012 [16]. They found decreasing hospital efficiency after the 2009 health reform and geographic disparities in technical efficiency between the eastern and western regions. Current studies adopted the Malmquist DEA efficiency index to analyze the efficiency change in provincial and county-level hospitals [17, 18]. Through the decomposition of productivity, the researchers attributed the improvement of hospital efficiency to technological progress after the 2009 health reform. Social health insurance and personal health expenditures may have unintended effects on local healthcare systems. A recent study reported an inverse relationship between personal health expenditure and the performance of provincial health systems [19].

Few studies explore the change in hospital efficiency when specific policies are implemented in the health system. Taking hospital privatization in Germany as an example, hospitals in Germany experienced a set of reforms over the last decades. Many public hospitals in Germany converted to private hospitals and started to implement the Diagnosis-related Group (DRG) financing scheme between 1997 and 2007. Tiemann and Schreyogg found that hospitals converted from public to private performed better in hospital operations and obtained higher quality of care [20]. Specifically, privatized hospitals were more likely to improve quality under the DRGs scheme [21]. Another study in Japan showed that hospital efficiency was not significantly affected by the change in regulation and improvement in financial conditions [22]. During hospital reform, local public hospitals in Japan recruited more healthcare staff while the number of beds was constrained, which caused a misallocation of medical resources on health services provision and quality of care. Besides, the regulatory costs imposed on the pharmaceutical and medical device industries slowed down the technological progress of healthcare system [22]. However, limited research detected the association between policies’ impacts and hospital efficiency changes in China, especially concerning the public hospital reform in county hospitals.

Using county-level hospital data across six provinces and one metropolitan area in East China, this study investigates the impacts of pilot reform on hospital efficiency. Hospital efficiency scores are measured by using DEA. The result shows that all county hospitals experienced a short-term decline in hospital efficiency in 2013. We use propensity score matching to address the selection biases of counties associated with the inclusion in the pilot reform. Then, a difference-in-differences approach is applied to estimate the impact of the pilot reform on hospital efficiency. The result indicates that public hospital reform is associated with a reduction of 3% in pure technical efficiency scores. However, hospitals in the pilot counties are more efficient than those that are not. For robustness analysis, we use semi-parametric bootstrapping to address the potential bias in efficiency estimation. The estimates from different methods are robust in terms of magnitude.

Our research has three key contributions to existing literature. First, this study is the first to explore the impact of pilot public hospital reform on hospital efficiency. Few studies conducted policy evaluation studies on a large sample of county hospitals and assessed the reform’s impacts on healthcare services delivery [16, 17]. Our findings reveal that the difference in efficiency between the pilot and non-pilot hospitals has unexpectedly narrowed after the pilot public hospital reform. Second, this research extends current studies on the potential mechanism behind the efficiency changes by decomposing efficiency. The insignificance of the pilot reform is driven by the mixed impact of the decline in pure technical efficiency and hospital scale expansion. Third, this study fills the gap in policy heterogeneity in terms of hospital types. Our findings provide evidence that the pilot reform has a limited effect on the efficiency decline of traditional Chinese medicine hospitals.

Literature review

Healthcare system and county hospitals in China

In expanding healthcare accessibility to the population, the main concerns are the heavy burden of healthcare expenditures and inefficient utilization of medical resources [23,24,25]. China’s health reform has achieved universal health insurance coverage by the end of 2012. More than 90% of Chinese citizens are covered by one of the three major health insurance schemes¹, but with expanding healthcare costs over the past decades [26]. Government health expenditure reached 1.74 trillion Chinese Yuan in 2019, around three times that of 2009 (National Health Commission, 2020). The annual growth rate of total health expenditure was 14.98% from 2009 to 2019, which was 3.77% points higher than the annual growth rate of GDP. However, the health system in rural area still faces challenges from limited resource allocation, the lack of healthcare workforce, and diagnostic inaccuracies [26,27,28]. The dramatic growth of public spending and severe urban-rural disparity in healthcare utilization alarmed policymakers, who prioritized controlling the growth of healthcare costs and improving the capacity building of healthcare systems.

In China, public hospitals follow a three-tier healthcare services delivery system: tertiary hospitals, secondary hospitals, and primary healthcare facilities. Most tertiary and secondary hospitals are located in metropolitan areas and prefecture cities with adequate or better resources. As part of the public healthcare system, county-level hospitals delivered services to 900 million people, covering 70% of county residents’ medical service needs [29]. However, most health resources were concentrated among tertiary hospitals, while resources available to the secondary and primary care facilities were around 40% [4]. The imbalanced resource distribution intensified the inequality of healthcare services and quality of care provided by county-level hospitals. Outpatient visits in tertiary hospitals experienced an 8% annual increase from 2009 to 2015 [27]. In contrast, a research based on the China Family Panel Studies survey pointed out that less than 60% of rural residents selected county hospitals or primary care facilities for health services [30]. To achieve a better quality of care and reduce inequality, public county hospitals need to improve operational efficiency through resource optimization and professional management. Hence, our study focuses on the change in hospital performance, and try to investigate the impact of pilot reform on county-level hospital efficiency.

The pilot public hospitals reform

China launched a systematic health reform in 2009 with the overarching goal of expanding insurance coverage and ensuring accessible and affordable services for the population. The reform has three planned phases [25]. The first phase was from 2009 to 2012. The goals of the health reform in this period were to meet the priorities of infrastructure building and strengthen the capacity of primary healthcare facilities. The second phase of the health reform was from 2012 to 2015. Recognizing the ballooning healthcare costs and inequalities in the delivery system, China has focused on improving the governance of public hospitals. The last phase of health reform is from 2017 to 2020, aiming to establish an integrated health system for both medical and public health services.

As an important component of the health reform, a multi-stage pilot reform in county-level hospitals was implemented from 2012 to 2015. The main elements of the reform included strategic planning, financing policy reform, department reorganization, and performance assessment. In June 2012, China’s central government kickstarted the pilot reform in 311 counties. The pilot reform is planned to cover the entire counties in mainland China at the end of 2015. To keep most medical services provision within the county, the county governments integrated resources to operate at least one general or traditional Chinese medicine hospital for residents. The pilot design of county hospital reform consists of three components: first, the zero-markup drug policy (ZMDP) to remove the 15% markup on drug prices sold at public hospitals; second, restructuring hospital governance and managerial compensation; third, promote cooperation and partnership between major hospitals and community health centers, i.e., tertiary hospitals support county hospitals through medical training and paired assistance in labor resources. According to the National Health Commission guidelines², hospitals in pilot counties will receive government subsidies and have more autonomy on hospital expansion, human resources, and payment incentives.

Methods

Data sources

Our study combines three separate datasets. The first dataset is the National Health Statistical Information Report System, which provides the financial statements and information on facilities and staff of hospitals from 2009 to 2015. We extract hospital data from this dataset to measure the efficiency of county hospitals. This study only includes general and traditional Chinese medicine hospitals to ensure the comparability of healthcare services delivery. Mental health hospitals, long-term care facilities, and dental clinics are excluded. We also reviewed the data with specific hospital grades and fiscal budgets to ensure the hospital received financial support and administration from the corresponding county government.

The second dataset is the official documents issued by the National Health Commission³, which contains information on public hospital reform and the list of pilot counties. Due to the data availability, we restrict our sample to 205 hospitals in the treated group from 93 pilot counties, which represents 29% of all pilot counties in China during the first batch county-level public hospital reform. As for the regional distribution, our study focuses on the healthcare system in Eastern China. County public hospitals from Shanghai, Anhui, Fujian, Jiangsu, Jiangxi, Shandong, and Zhejiang are included in our sample.

This study specifies two inputs and two outputs in the DEA process for measuring hospital efficiency scores. The inputs considered serve as proxies for delivering daily medical services to patients. Following previous studies, we use the number of full-time equivalency health care professionals (physicians, nurses, medical technicians, and other health professionals) and the number of hospital beds [20, 22, 31, 32]. In China, providing fundamental medical services to the residents is one of the key objectives of county hospitals in the new healthcare reform phase. Following previous studies, we use emergency visits and inpatient days as the output indicators [16, 21]. Outpatient and emergency visits represent the number of patients from outpatient and emergency visits per year. Inpatient days is the total number of occupied bed days for patients.

Aside from efficiency measurement, our study also uses the average length of stay and bed use rate as the indicator of healthcare utilization. Table 1 presents summary statistics for the hospital variables. Most county hospitals in this study are secondary hospitals (88%). As for hospital type, approximately 60% of hospitals are general hospitals, while the rest of them are traditional Chinese Medicine hospitals.

Table 1 Summary of Hospital and County characteristics (before Propensity score matching specification)

The main covariates used in PSM specification and DID analysis are demographical and socioeconomic information collected from the third dataset, the China Statistical Yearbook (County-level) and the National Economic and Social Development Statistical Bulletin (County-level). Our study matches individual hospitals with the socioeconomic information of the counties where the hospitals are located. Those variables include socioeconomic status indexes, demographic indicators, governmental expenditure, and local healthcare infrastructure. We also provide a comparative summary of the counties’ characteristics of pilot and non-pilot hospitals in Table 1. The average population of pilot and non-pilot counties is about 643,700 persons. Gross Domestic Product per capita (GDPPC) is calculated as the value of county-level GDP divided by the population of this county. The number is close to 40,000 RMB in our study.

The original dataset contains 749 county-level hospitals from 366 counties. Observations with missing values and outliers are excluded from the requirement of efficiency measures. We construct a 7-year balanced panel data to assess the policy impact of the first-batch public hospital reform before and after 2012, the year of the pilot reform. Hospitals are divided into pilot and non-pilot groups based on the list of the first-batch pilot counties in public hospital reform. After PSM matching, 190 pilot and 175 non-pilot hospitals from 172 counties (85 pilot and 87 non-pilot counties) are selected as the final sample.

We use a three-stage strategy to estimate the impact of the pilot reform on hospital efficiency. First, hospital yearly efficiency scores are analyzed using the output-oriented DEA approach. Second, we use propensity score matching to pair pilot counties with those counties with similar group characteristics. Third, the impact of the pilot reform is assessed by using a difference-in-differences (DID) specification of the Tobit regression model. Efficiency scores calculated in the first stage were used as the dependent variable in the DID analysis.

Data envelopment analysis

Data envelopment analysis is a classical non-parametric method for measuring the efficiency and productivity of a decision-making unit (DMU) [33]. Over the past decades, DEA has been widely applied to assess efficiency and performance in healthcare, education, operational research, and transport industries [34]. We use the DEA approach to measure county hospitals’ efficiency for the following reasons. First, DEA approach has its advantages in handling multiple outputs compared with other parametric efficiency measures. Second, DEA is more flexible in model setting than other parametric methods (such as SFA or COLS) because no assumptions are needed on the functional form of the production frontier [35].

Through linear programming, DEA generates a ratio that reflects the distance of each DMU to the efficient frontier. The efficient frontier is based on the best estimated input-output relationship under a certain assumption and framework [34]. The DEA approach could be fitted with different frameworks in its production frontier. In the output-oriented DEA model, the efficient frontier is derived from those units that achieved maximum outputs under given inputs. The efficient frontier in the input-oriented model is the best practice for accomplishing the minimum inputs in production for given outputs [36]. In this study, we concentrate on the efficiency of county-level hospitals in China and assess the impact of public hospital reform on efficiency. We use the output-oriented DEA model for analysis because it fits the context that county hospitals exploited given inputs to get maximum outputs. On the one hand, limited medical resources are allocated to county hospitals in China. County Hospital leaders had no control over inputs (like human resources) in the short run [37,38,39]. On the other hand, the main target for county hospitals over the observation period was enhancing the capacity building of primary healthcare facilities and providing most medical services to rural residents [4].

Farrell is among the first to apply the basic DEA approach with one input and one output for the DMU [40]. Charnes et al. extended Farrell’s work and proposed the DEA approach with a constant return to scale assumption, CRS Model [41]. Banker et al. extended it into a variable return to scale (VRS) model [42]. The main difference between CRS and VRS model is whether a unit increase (decrease) in the input will contribute to a proportional increase (decrease) in the output. The constant return to scale model is more appropriate when the DMUs operate at the optimal scale, reflecting the perfect competition in the market. When DMUs faces incomplete competition, the VRS Model is used to alter the efficiency scores from CRS model by considering the scale of each DMU. In this study, public hospitals in China operate under budget constraints, regulations, and imperfect competition [43,44,45]. Thus, we adopt a variable return-to-scale model to calculate the efficiency scores of county-level hospitals [14]. The model is denoted as follows:

$$\:max\:{\theta\:}_{0}\left(w,\:\mu\:\right)=\frac{\sum_{i=1}^{r}\mu\:*{y}_{r}}{\sum_{i=1}^{k}w*{X}_{k}}$$

(1)

$$\:s.t.\:\sum\limits_{i=1}^{r}\mu\:*{Y}_{r}-\sum\limits_{i=1}^{k}w*{X}_{k}\le\:0,\:because\:\theta\:\le\:1$$

$$\:\sum\limits_{i=1}^{N}{w}_{N}=1$$

$$\:{w}_{1},\:{w}_{2},\:\dots\:,\:{w}_{k}\ge\:0$$

$$\:{\mu\:}_{1},\:{\mu\:}_{2},\:\dots\:,\:{\mu\:}_{r}\ge\:0$$

Efficiency score $\:{\theta\:}_{0}$ is the estimated efficiency of $\:{DMU}_{0}$ for $\:N$ decision DMU with $\:K$ inputs and $\:R$ outputs. Where $\:Y(r\times\:n)$ and $\:X(k\times\:n)$ are the matrices for hospitals with $\:k$ inputs and $\:r$ outputs, respectively. Hospitals maximized the efficiency ratio under the constraints of given inputs. $\:w$ and $\:\mu\:$ are vectors for the weight of inputs and outputs. The weights of the input-output combination are obtained by dual programming. The efficiency scores calculated by DEA model denote the overall efficiency (hospital efficiency in this study), which measures how health resources are transformed into the production process of each county hospital in healthcare services delivery. Overall technical efficiency could be decomposed into pure technical efficiency and scale efficiency. Pure technical efficiency denotes the efficiency under a variable return of scale related to management and technology innovations. Scale efficiency is the gap between the actual and optimal scales, measuring whether a hospital operates at the optimal scale.

Propensity score matching

We aim to investigate the policy impact on hospital efficiency among county-level hospitals. However, the counties selected for the pilot reform are more likely to be distinct in terms of access to healthcare and economic resources. We use county-level covariates in a propensity score matching to address the potential selection bias to construct a counterfactual. We include county-level covariates because the pilot reform was implemented at the county level rather than for specific hospitals. Counties were enrolled in the pilot program, and hospitals in pilot counties were included in the pilot reform. This study used propensity score matching to balance the treatment and control groups and to match pilot counties with other counties that are similar to the pilot counties but not enrolled in the pilot program.

Propensity score matching results in a balanced comparison to create a “quasi-random experiment.” This approach assigns treatment and control groups in terms of the conditional probability given by the observed covariates [46]. To adjust multivariate sampling to the control group, we use a logit regression model to estimate the propensity scores [47]. We set 2011 (one year before the pilot reform) as the base year. To find appropriate covariates that determine the pilot grouping, we select socioeconomics and health system covariates based on previous studies [20, 43]. The county-level covariates in the logit model include GDP per capita, government revenue, government expenditure, the number of beds in healthcare institutions, the output in the services industry as a share of GDP, and the share of education enrollment in the population. We minimize the distance of propensity scores of the nearest neighbor to reduce the disparity between pilot and non-pilot counties. We used one-to-one matching without replacement to match each county in the treatment group to at least one non-pilot county in the control group. After propensity score matching, counties under common support remain in the final sample. We examine standardized differences of the covariates to evaluate the balance of the two sub-samples. The result of the balanced test of covariates is shown in Table 8 in Appendix 1.

Difference-in-differences

This study uses a standard difference-in-differences specification to assess the impact of pilot reform on hospital efficiency. The dependent variables are the estimated scores of hospital efficiency, pure technical efficiency, and scale efficiency of hospitals by DEA measures. Our study applies the Tobit regression model because of two-side truncated efficiency scores computed by DEA as dependent variables. Specifically, we conduct a 7-year panel Tobit model with hospital individual fixed and year fixed effects as our main DID specification. The reasons for using two-way fixed effects DID model are as follows. First, adding individual fixed effects could deal with the unobserved factors and time-invariant characteristics of hospitals, affecting hospital efficiency. Second, we use year effects to control the common policy impacts on all hospitals after the nationwide healthcare reform launched in 2009. Aside from the pilot hospital reform, a set of healthcare policies and guidelines were released from 2009 to 2015. The year-fixed effects address the time-specific effects on all county hospitals. The DID model can be specified as follows:

$$\:{{\uptheta\:}}_{it}=\lambda\:{(Post}_{it}*{Pilot}_{i})+\beta\:{Z}_{ct}+{a}_{i}+{{\upgamma\:}}_{t}+{{\upepsilon\:}}_{ict}$$

(2)

Where $\:{\theta\:}_{it}$ denotes the efficiency scores of hospital $\:i$ in year $\:t$. $\:\lambda\:$ is the coefficient of the interaction term, capturing the impacts of the pilot reform on county hospitals in different groups. $\:{Post}_{it}$ is the time dummy, which equals 1 in the years after pilot reform was implemented for hospital $\:i$ in 2012, and 0 otherwise. As for the treated dummy, $\:{Pilot}_{i}$, equals 1 if the hospital $\:i$ was in the pilot reform counties from 2009 to 2015 and zero otherwise. $\:{a}_{i}$ and $\:{{\upgamma\:}}_{t}$ represent hospital individual fixed effects and year fixed effects, respectively. $\:{{\upepsilon\:}}_{ict}$ denotes the error term.

We control the county-level characteristics that have potential confounding effects on hospital efficiency. $\:{Z}_{ct}$ represents a set of control variables of socioeconomic status, health system, and demographic information on county $\:c$ where hospital $\:i$ located. Specifically, we included GDP per capita, local government revenue, local government expenditure, the proportion of services industry to GDP, the number of beds in healthcare institutions, and the share of education enrollment⁴ in the regression model [17, 21, 24]. GDP per capita, government revenue, government expenditure, and the number of beds in healthcare institutions were log-transformed following prior studies and for easy interpretation.

This study does not use hospital-level characteristics as the control variables. The reasons are as follows. As mentioned in the data sector, our study only includes general and traditional Chinese medicine hospitals associated with local governmental financial support. Those inclusion criteria control the variation of hospital type and subsidy. Besides, the efficiency indicator in our study represents how county hospitals utilize medical resources to deliver healthcare services. Compared with the hospital characteristics, this indicator is more relevant to local demographic information and health infrastructure.

The key assumption in DID identification strategy is the change in dependent variables for the treatment and control group should share identical trends before the policy implementation. In this study, we use a more flexible form of DID specification to test the parallel trend assumption (PTA) and capture the dynamic effects of the pilot reform. The event study method is as follows:

$$\:{{\uptheta}}_{it}=\sum\limits_{k=-3}^{3}{\lambda}_{k}*{D}_{ki}+\beta\:{Z}_{ct}+{a}_{i}+{{\upgamma\:}}_{t}+{{\upepsilon\:}}_{ict}\:$$

(3)

Where $\:{D}_{ki}$ denotes the interactive term of the treatment dummy and period dummy. Our study contains 7-year observation periods, which could be divided into 3-year pre-intervention period, policy implementation time, and 3-year post-intervention period. $\:K$ is the footprint of periods. We drop one year before the pilot reform ($\:k$ = −1) to avoid multicollinearity. $\:{\lambda\:}_{k}$ is the coefficient indicating dynamic policy effects.

Robustness check

This study transforms hospital efficiency scores from conventional DEA estimates to Bias-corrected estimates. Conventional DEA estimates would be biased because of the unknown efficient frontier and exogenous affecting efficiency measures. First, the efficiency score generated by DEA is the deviation from the estimated efficient frontier (a set of observed units) to each DMUs, rather than the distance of true full-efficiency units to other DMUs. However, the true possible full-efficiency frontier could not be illustrated due to the finite sample in DEA framework [44]. Second, the inefficiency of hospitals could not be entirely explained by resource misallocation or scale inefficiency. Hospital efficiency is also affected by external factors and organizational characteristics [20, 48]. Conventional DEA may overestimate the inefficiency of hospital governance. Simar and Wilson proposed a bias-corrected efficiency estimator based on the two-stage bootstrapped process to address the potential bias in efficiency measures [49]. The details of bootstrapped DEA procedure are in Appendix 2.

This study uses bias-corrected estimates in hospital technical efficiency measurement based on Simar and Wilson’s approach. Specifically, we implemented 1000 times bootstraps to produce bias-corrected estimates and 500 bias-corrected bootstraps to construct confidence intervals of bootstrapped estimates [50]. The bias-corrected estimates are obtained using simarwilson, and implemented using Stata 18.0 (Stata Corp, College Station, TX).

Aside from the robustness check in replacing efficiency scores, we conduct a set of estimations of the policy impact. The original estimation of the panel data model with two-way fixed effects is Least Squares Dummy Variable (LSDV) or Within-Group estimation. However, those estimators would be consistent and unbiased under the assumption that the error terms are independent and asymptotically normally distributed. Due to the trimmed distribution of the error term in censored regression models (Tobit model), Least Squares Dummy Variable and other estimation methods will be inconsistent in this sense. Honore applied a trimmed least absolute deviations estimator to construct the moment conditions for one-sided censoring data with fixed effects [51]. By using this method, Honore maintained assumptions on consistency and asymptotically normal distribution. Alan et al. extended this method in two-sided censoring panel data regression [52]. In our study, we follow Alan et al. method and use two_side command to conduct the main estimation. In contrast, we also applied Tobit panel data model with LSDV, and Tobit model with random effects as the robustness check.

Results

Descriptive statistics

Table 8 in Appendix 1 reports the result of propensity score matching specification. Our study uses data from 2011 for the matching. Column (3) of Table 8 in Appendix 1 shows the difference in most county-level characteristics between pilot and non-pilot counties before matching. Only the proportion of the services industry in GDP demonstrates insignificant differences before matching. Column (6) of Table 8 in Appendix 1 shows the matching result, which suggests no significant difference in county characteristics between the treatment and control groups after a one-to-one matching. Therefore, the matching helped to reduce the bias associated with the selection into the pilot program.

Table 2 presents the descriptive statistics of control variables after matching. Those variables included socioeconomic status indexes, demographic indicators, and information on healthcare conditions. Hospitals located in pilot counties are usually with better economic infrastructure. The pilot counties’ GDP per capita, government revenue, and government expenditure were higher than those of the non-pilot counties over the research period. There is no significant difference between pilot and non-pilot counties regarding average education.

Table 2 Descriptive Statistics of Socioeconomics for Pilot and Non-pilot hospitals during 2009–2015 (after propensity score matching specification)

The summary statistics of the inputs and outputs used in hospital efficiency measures are shown in Table 3. The average number of health care professionals and hospital beds in pilot hospitals is larger than in non-pilot hospitals. The input variables in DEA estimates experienced steady growth from 2009 to 2015 in both pilot and non-pilot hospitals. The number of health care professionals in pilot hospitals increased annually by 6.4%, from 390 in 2009 to 567 in 2015. At the same time, the number of healthcare professionals in non-pilot hospitals shows a 6.7% annual increase. As for the number of hospital beds, the difference between pilot and non-pilot hospitals has been narrowed from 38 in 2009 to 29 in 2015.

Table 3 Summary statistics of the inputs and outputs in Efficiency Measurement (after propensity score matching specification)

As for the outputs, the number of outpatient visits in pilot hospitals increased from 273.7 thousand visits per year in 2009 to 430.6 thousand visits per year in 2015, representing a 7.8% annual growth. Non-pilot hospitals demonstrate a similar trend, with a 7.9% annual growth rate. The average number of inpatient days in pilot hospitals increased slightly lower than that of non-pilot hospitals from 2009 to 2015. In conclusion, pilot hospitals and non-pilot hospitals experience similar increases in trends in outputs and inputs for efficiency measures. The disparity in healthcare services delivery between pilot and non-pilot hospitals widened after the pilot reform.

Hospital efficiency measurement

Table 4 illustrates the efficiency scores of both pilot and non-pilot hospitals during 2009–2015. We estimate hospital efficiency, pure technical efficiency, and scale efficiency by using DEA in an output-oriented setting. The average level of hospital efficiency for the whole sample experienced a rapid drop in 2013, then returned to the peak in 2014. Except in the reform year (2012), the overall hospital efficiency for the post-reform period is higher than that for the pre-reform period. In addition, the efficiency gap between pilot and non-pilot hospitals has narrowed from 0.083 in 2010 to 0.061 in 2015.

Table 4 Efficiency of pilot and non-pilot hospitals in 2009–2015

The change in pure technical efficiency shows the same trend as hospital efficiency. Both pilot and non-pilot hospitals have increased efficiency scores after the pilot reform. Pure technical efficiency reached to peak in 2014. The average pure technical efficiency of pilot hospitals and non-pilot are 0.758 and 0.710, respectively. The difference in the pure technical efficiency is slightly smaller than the hospital efficiency. Given these trends, the results suggest county hospitals have improved efficiency after pilot reform. The gap between pilot and non-pilot hospitals has been gradually narrowing since 2013.

Although most county hospitals are not fully scale efficient (scale efficiency = 1), the average scale efficiency achieves a high interval over the observation period. Hospitals have experienced a steady improvement in scale efficiency since 2009. The average scale efficiencies of pilot and non-pilot hospitals were 0.968 and 0.963 in 2015, respectively. Scale efficiency peaked in 2012 and slightly declined from 2013 to 2015. The inverse U-shaped curve of scale efficiency indicates that county hospitals fail to allocate resources efficiently when resources are beyond the optimal scale. Besides, non-pilot hospitals’ scale efficiency is better than pilot hospitals, with a relatively small gap for the most time.

Baseline results

Table 5 shows the results of DID specification with two-way fixed effects after the propensity score matching process. We use hospital efficiency, pure technical efficiency, and scale efficiency as the dependent variables in the regression model. The interaction term between the pilot reform dummy and post-reform indicator measures the differential policy effects on hospitals. As shown in Column (1) of Table 5, the pilot reform has a negative effect on hospital efficiency when we control the year-fixed effects and hospital-fixed effects. In Column (2), the results indicate that pilot hospitals have a decrease of 3% ((p-value < 0.05) in pure technical efficiency from the pilot reform. Compared with overall hospital efficiency, the result of pure technical efficiency shares a similar magnitude but is highly significant. In contrast, the result in Column (3) suggests that the pilot reform leads to a significant 2.3% improvement in scale efficiency.

Table 5 Impact of pilot reform on hospital efficiency

Regarding socioeconomic factors, the GDP per capita of the county is negatively related to hospital efficiency and pure technical efficiency of the local county hospitals. In contrast, hospitals in better-off counties are more likely to be scale-efficient. We fail to find statistical evidence that government revenue affects any kind of efficiency. As for the industrial structure and health system infrastructure, we also find that hospitals in counties with a high proportion of services industry obtained higher hospital efficiency and pure technical efficiency. In contrast, the services industry proportion has a negative coefficient to scale efficiency. It is worth noting that the number of beds in local healthcare institutions is negatively related to scale efficiency. This negative impact on scale efficiency might be attributed to the substitution effect between local healthcare facilities and county hospitals.

Results of the robustness check

We apply a bias-corrected efficiency estimate as a robustness check for hospital efficiency. The new efficiency scores are generated using 1000 replications in Simarwilson’s bootstrapping procedure. The result of bias-corrected efficiency estimates is shown in Column (4) of Table 9 in Appendix 1. The pilot reform is negatively associated with bias-corrected efficiency. The coefficient of the bias-corrected efficiency score is identical to the original hospital efficiency score. The difference in significance level might be attributed to the different settings in standard error calculation.

The results of the event study are in Fig. 1 in Appendix 1. This study estimates the dynamic policy effects on hospital efficiency, pure technical efficiency, scale efficiency, and bias-corrected efficiency. The omitted period is one year before the implementation of pilot reform. We report both coefficients and 95% confidence intervals for policy impact by year. From the trends of the pre-stage, we fail to reject that the treatment group and the control group had the same trend of efficiency changes before the intervention of the pilot reform. Notably, pure technical efficiency significantly increased two years before the policy implementation. Hospital efficiency, pure technical efficiency, and bias-corrected efficiency have identical time trends over the whole observation period, which presents a negative impact after the pilot reform. As for scale efficiency, we find a perfect parallel time trend before the treatment. The impact of pilot reform materializes quickly in the change of scale efficiency. The magnitude of the coefficient demonstrates a positive effect immediately when the pilot reform was launched.

This study conducts different estimation methods as the robustness check to the baseline results. Tobit model with Least Squares Dummy Variable (LSDV) and random effects settings are introduced to estimate policy impact. Table 10 in Appendix 1 reports the results of robustness in hospital efficiency. As for the policy impact, the magnitude of the LSDV method estimate is identical to the baseline estimates. The results of control variables are also robust compared with the baseline model. However, the estimates of policy impact in the random effects model demonstrate a slightly lower magnitude, comparing the estimates of the baseline model and LSDV model. The difference in estimation could be attributed to the misspecification without addressing year and individual fixed effects. Table 11 in Appendix 1 and Table 12 in Appendix 1 clearly demonstrate similar patterns as Table 10 in Appendix 1 in robustness checking. Overall, the magnitude and significance of estimates in pure technical efficiency and scale efficiency are robust between the baseline model and Tobit model with LSDV setting.

Heterogeneous treatment effect

We also investigate the impact of the pilot reform on healthcare utilization and the heterogenous treatment effects in terms of hospital types. As mentioned in the background section, the zero-markup drug policy shares the overlapping pilot list with county hospital pilot reform. Previous studies on zero-markup drug policy found that the average length of stay in primary care facilities expanded due to the compensation of profit loss [53]. Fu et al. found a negative and insignificant impact on average stay length and inpatient admissions [24]. In our study, pilot hospital reform is negatively associated with county hospitals’ average length of stay and bed use rate (Table 6). The average length of stay in the pilot hospitals reduces to around 0.31 days after the policy implementation. This result could be explained by the constraints from the demand side. County hospitals experienced scale expansion in labor and capital resources (supply side). However, the demand side of health seeking may not consistently increase as the supply side.

Table 6 Impact of pilot reform on health care utilization

Furthermore, the impact of the pilot reform on efficiency is heterogeneous in terms of hospital type. We conducted a subgroup analysis by separating the sample into general hospitals and traditional Chinese medicine hospitals (TCM hospitals). Overall, general hospitals experience a larger decline in efficiency after the policy implementation. Table 7 shows that general hospitals have a 2.8% decrease in overall hospital efficiency and a 3.7% decrease in pure technical efficiency. In contrast, the coefficients for TCM hospitals suggest that the pilot reform has a lower and insignificant effect on efficiency for TCM hospitals.

Table 7 Heterogenous treatment effects: impact of pilot reform on hospital efficiency (by hospital types)

As shown in Table 4, we observed a drop in average hospital efficiency in 2013. One of the potential mechanisms is that the shift of an efficient frontier will widen the gap between the most efficient hospitals and other hospitals. The change of the efficient frontier may result in relatively lower efficiency scores for hospitals that experienced the adjustment from the pilot reform. Hence, we provide further analysis of the distribution of hospitals in terms of different efficiency categories. As Table 13 in Appendix 1 presented, we observe a rapid decline in the number of second-tier efficient hospitals in 2013. Additionally, the trend of efficiency decline reversed in 2014, consistent with the hospital average efficiency scores findings.

Discussion

This study aims to measure the efficiency changes of county-level hospitals and investigate the impact of pilot reform on hospital efficiency. Using data envelopment analysis, we conducted the efficiency measurement of county-level hospitals in East China from 2009 to 2015. Our results suggest that hospitals in pilot counties are 6.1% (percentage point) more efficient than those in non-pilot counties. County hospitals’ overall efficiency and pure technical efficiency declined one year after the policy was implemented, then experienced a dramatic increase in 2014. Results of DID specification point to a mixed effect on hospital efficiency. On the one hand, the pilot reform is negatively associated with the pure technical efficiency of county hospitals. On the other hand, the scale efficiency of hospitals in pilot counties increases by 2.3% after policy implementation.

Although surprising at first glance, our findings on efficiency estimates are consistent with recent studies on China’s health system and county hospital efficiency measures. While some researchers reported that most county hospitals performed better in hospital efficiency after public hospital reform [16, 17]. Technological diffusion theory explain the decline in hospital efficiency and pure technical efficiency [54]. After the national healthcare reform, hospitals were encouraged to introduce modern technologies to increase revenue. However, technological diffusion is driven by hospitals and healthcare staff. It takes time for county hospitals to achieve technical progress after systematic reform. We also need to be cautious about the disparity of technology diffusion in different regions. Chai et al. observed heterogeneities in health system productivity in different provinces [23]. They found some less well-off counties experienced a shock of declining productivity after 2009. Policy-makers need to pay more attention to the regional heterogeneity of healthcare infrastructure, which may lead to the negative impact of the pilot reform and exacerbate the inequality of healthcare services.

Another possible reason for the inconclusive association between the pilot reform and hospital efficiency is the outward movement of the efficiency frontier. We examined this hypothesis by illustrating the distribution of hospitals in different efficiency categories. It is noted that many hospitals experienced downshifts in efficiency in 2013. In contrast, the number of most efficient hospitals remains stable over the observation period. During the pilot reform, few highly efficient hospitals quickly adjusted to the change and pushed the frontier up, eventually resulting in relatively lower efficiency measures for other hospitals.

Our study complements existing research on hospital efficiency measurement from a policy evaluation perspective. The results of DID approach showed that most pilot county hospitals experience a decline in hospital efficiency and pure technical efficiency, indicating inefficient resource utilization during the first phase of pilot reform. This phenomenon might be attributed to inadequate hospital management [35]. County hospitals in China are under the auspices of the local government and health bureau. The principal managers are often appointed by government bodies. Most county hospital managers are medical specialists but lack managerial experience running a hospital. Once the pilot hospitals receive government support, hospital managers invest the resources in scale expansion (such as adding more beds or hiring staff). The findings of a positive association in scale efficiency partially verified the hypothesis of scale expansion in pilot hospitals. Previous studies on government subsidies also revealed that hospitals with more local government support are more likely to face difficulties in resource utilization [16, 17]. Challenges in hospital leadership and management practice call for further reforms to improve hospital governance, enhance regulation, and promote a comprehensive assessments of hospital performance [55].

Aside from hospital governance, balancing the multiple objectives of a hospital may also be relevant to the reduction in hospital efficiency associated with the pilot reform. Findings in Japan’s health system show that hospitals would experience a short-term efficiency decline when faced with the trade-off between medical service provision and quality of care [22]. The public hospital reform in China has multiple mandates for county hospitals, including reducing the reliance on medical sales, controlling the cost of medical treatment, establishing the role of gatekeeper of family physicians for healthcare utilization, etc. The main objectives of the pilot reform at that period were to maintain social welfare, enhance healthcare capacity, and provide accessible healthcare services for the population [56]. Devoting resources to those objectives may lead to better accessibility and equity but with short-term reductions in technical efficiency among county hospitals.

The sensitivity analysis explores the policy impact heterogeneity in terms of hospital types. The results of heterogenous treatment effects suggest that the pilot reform has limited effects on Traditional Chinese Medicine hospitals to improve efficiency and achieve scale efficiency. The role in healthcare service delivery could be one possible explanation. Compared with general hospitals, TCM hospitals mainly provide herbal medicine and minority general medical services to county residents [57]. Besides, the difference in resource allocation strategies could result in inconsequential impacts among TCM hospitals. The Zero-markup Drug Policy was implemented in the pilot county hospitals after 2012, which strongly reduced the cost of medicine in medical treatment. However, the cost of Traditional Chinese Medicine is not controlled by the ZMDP. Even if they received governmental subsidies and medical resources, TCM hospitals might be less motivated to expand scale or purchase high-value devices to compensate for reduced drug sales than general hospitals.

Conclusion

This study examines the association between the pilot public hospital reform and county hospital efficiency. We find an unintended consequence of the reform: no improvement in hospital efficiency in healthcare services delivery was found after the pilot reform in county hospitals. The findings imply inefficient utilization of medical resources and shed light on scientific management, which helps to address the potential conflict on a multitude of policy objectives for public hospitals.

Our research also extends current studies in hospital efficiency by investigating the impact of the pilot reform and provides valuable policy implications to county hospitals and the health system. County-level hospital leaders must improve their internal management and hospital operations competency. Then, county hospitals would be more likely to achieve optimal scale efficiency. Policymakers could regard efficiency scores as one of the monitoring indices in assessing the performance of hospitals in certain periods and guiding decision-making.

Our study has at least three limitations. First, the sample of county hospitals is selected from six provinces in East China. The impacts of the pilot reform may differ in the central and western areas; thus, we need to use caution in extrapolating our results to other regions of China. Second, we analyze the policy effects shortly after the reform. It may take longer for the pilot hospitals to materialize the efficiency gains. Last, we could not assess the change in the quality of care due to the data limitations. Further studies are needed to investigate the quality performance by adding health outcomes indicators.

Appendices

Appendix 1

Table 8 Covariates before and after propensity score matching specification (Match year: 2011)

Table 9 Impact of Pilot Reform on Hospital Efficiency (with Bias-corrected efficiency)

Table 10 Robustness: impact of Pilot Reform on Hospital Efficiency (different estimation methods)

Table 11 Robustness: impact of pilot reform on pure technical efficiency (different estimation methods)

Table 12 Robustness: impact of Pilot Reform on Scale Efficiency (different estimation methods)

Table 13 The distribution of hospitals by different efficiency categories

Fig. 1 The event study results in Hospital efficiency, Pure technical efficiency, Scale efficiency, and Bias-corrected efficiency

Appendix 2

 The Procedure of Bias-corrected Efficiency Scores

Simar and Wilson first focused on the data-generating process of estimated efficiency. A truncated regression⁵ model illustrated the relationship between efficiency and exogenous factors

$$\:{{\uptheta\:}}_{i}={Z}_{i}{\upbeta\:}+{{\upepsilon\:}}_{i}$$

In this data-generating process, $\:{{\uptheta\:}}_{i}$ is the technical efficiency and $\:{\text{Z}}_{i}$ denotes the exogenous factors influencing hospital efficiency. $\:{{\upepsilon\:}}_{i}$ is the error item following a truncated distribution with zero mean and constant variance.

The main steps of bias-corrected efficiency estimates are as follows. Step 1–4 generate the bias-corrected efficiency scores. Step 5–7 expand the confidence intervals of bias-corrected efficiency to all DMUs.

• 1. Estimate efficiency scores by using conventional DEA.

• 2. For $\:{{\uptheta\:}}_{i}$ > 1, use maximum likelihood to obtain $\:\widehat{{\upbeta\:}}$ and $\:\widehat{{\upepsilon\:}}$ in the truncated regression of $\:{{\uptheta\:}}_{i}$.

• 3. Loop step 3.1 to 3.4 $\:{B}_{1}$ times and generate estimates of $\:\widehat{{{\uptheta\:}}_{i}}$ for $\:DM{U}_{i}$, i = 1-N.

  • 3.1 For each DMU, draw an artificial $\:\stackrel{\sim}{{{\upepsilon\:}}_{i}}$ from N (0,$\:{\widehat{{{\upepsilon\:}}_{i}}}^{2}$) at $\:1-{Z}_{i}\widehat{{\upbeta\:}}$.

  • 3.2 Compute artificial efficiency scores based on the $\:\widehat{{\upbeta\:}}$ and $\:\stackrel{\sim}{{{\upepsilon\:}}_{i}}$.

  • 3.3 Generate artificial x and y of each DMU with $\:\stackrel{\sim}{{y}_{i}}=\left(\widehat{{{\uptheta\:}}_{i}}/\stackrel{\sim}{{{\uptheta\:}}_{i}}\right){y}_{i}$.

  • 3.4 Replace DMU with artificial one generated in step 3.3, let them be the sample to draw the bias-corrected efficiency score for each DMU.

• 4. For $\:DM{U}_{i}$, i = 1-N, calculated the bias-corrected efficiency score $\:\widehat{{{\uptheta\:}}_{i}^{bc}}$.

• 5. Use maximum likelihood to estimate new $\:\widehat{\widehat{{\upbeta\:}}}$ and new $\:\widehat{\widehat{{\upepsilon\:}}}$.

• 6. Loop step 6.1 to 6.3$\:\:{B}_{2}$ times to obtain $\:\widehat{\widehat{{{\uptheta\:}}_{i}}}$.

  • 6.1 For each DMU, draw an artificial $\:\stackrel{\sim}{\stackrel{\sim}{{{\upepsilon\:}}_{i}}}$ from N (0,$\:{\widehat{{{\upepsilon\:}}_{i}}}^{2}$) at $\:1-{Z}_{i}\widehat{\widehat{{\upbeta\:}}}$.

  • 6.2 Compute artificial efficiency scores based on the $\:\widehat{\widehat{{\upbeta\:}}}$ and $\:\stackrel{\sim}{\stackrel{\sim}{{{\upepsilon\:}}_{i}}}$ for each DMU.

  • 6.3 Use maximum likelihood to estimate $\:\widehat{\widehat{{{\uptheta\:}}_{i}}}$ on $\:{Z}_{i}$ to obtain bootstrapped $\:\widehat{\widehat{{\upbeta\:}}}$ and $\:\stackrel{\sim}{\stackrel{\sim}{{{\upepsilon\:}}_{i}}}$.

• 7. Use bootstrapped $\:\widehat{\widehat{{\upbeta\:}}}$ and $\:\stackrel{\sim}{\stackrel{\sim}{{{\upepsilon\:}}_{i}}}$ to construct the confidence interval for $\:{\upbeta\:}$ and $\:{{\upepsilon\:}}_{i}$.