case study 1 government intervention

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 16 November 2020

Ranking the effectiveness of worldwide COVID-19 government interventions

• Nina Haug   ORCID: orcid.org/0000-0002-5130-9193 1 , 2   na1 ,
• Lukas Geyrhofer   ORCID: orcid.org/0000-0002-8043-2975 2   na1 ,
• Alessandro Londei   ORCID: orcid.org/0000-0002-5748-9578 3   na1 ,
• Elma Dervic   ORCID: orcid.org/0000-0001-7168-3310 1 , 2 ,
• Amélie Desvars-Larrive   ORCID: orcid.org/0000-0001-7671-696X 2 , 4 ,
• Vittorio Loreto   ORCID: orcid.org/0000-0002-2506-2289 2 , 3 , 5 ,
• Beate Pinior   ORCID: orcid.org/0000-0001-8554-5963 2 , 4 ,
• Stefan Thurner 1 , 2 , 6 &
• Peter Klimek   ORCID: orcid.org/0000-0003-1187-6713 1 , 2  

Nature Human Behaviour volume  4 ,  pages 1303–1312 ( 2020 ) Cite this article

528k Accesses

847 Citations

8819 Altmetric

Metrics details

• Epidemiology
• Viral infection

Assessing the effectiveness of non-pharmaceutical interventions (NPIs) to mitigate the spread of SARS-CoV-2 is critical to inform future preparedness response plans. Here we quantify the impact of 6,068 hierarchically coded NPIs implemented in 79 territories on the effective reproduction number, R t , of COVID-19. We propose a modelling approach that combines four computational techniques merging statistical, inference and artificial intelligence tools. We validate our findings with two external datasets recording 42,151 additional NPIs from 226 countries. Our results indicate that a suitable combination of NPIs is necessary to curb the spread of the virus. Less disruptive and costly NPIs can be as effective as more intrusive, drastic, ones (for example, a national lockdown). Using country-specific ‘what-if’ scenarios, we assess how the effectiveness of NPIs depends on the local context such as timing of their adoption, opening the way for forecasting the effectiveness of future interventions.

Similar content being viewed by others

Effects of government policies on the spread of COVID-19 worldwide

Impact of government policies on the COVID-19 pandemic unraveled by mathematical modelling

A comparative analysis of the effects of containment policies on the epidemiological manifestation of the COVID-19 pandemic across nine European countries

In the absence of vaccines and antiviral medication, non-pharmaceutical interventions (NPIs) implemented in response to (emerging) epidemic respiratory viruses are the only option available to delay and moderate the spread of the virus in a population 1 .

Confronted with the worldwide COVID-19 epidemic, most governments have implemented bundles of highly restrictive, sometimes intrusive, NPIs. Decisions had to be taken under rapidly changing epidemiological situations, despite (at least at the very beginning of the epidemic) a lack of scientific evidence on the individual and combined effectiveness of these measures 2 , 3 , 4 , degree of compliance of the population and societal impact.

Government interventions may cause substantial economic and social costs 5 while affecting individuals’ behaviour, mental health and social security 6 . Therefore, knowledge of the most effective NPIs would allow stakeholders to judiciously and timely implement a specific sequence of key interventions to combat a resurgence of COVID-19 or any other future respiratory outbreak. Because many countries rolled out several NPIs simultaneously, the challenge arises of disentangling the impact of each individual intervention.

To date, studies of the country-specific progression of the COVID-19 pandemic 7 have mostly explored the independent effects of a single category of interventions. These categories include travel restrictions 2 , 8 , social distancing 9 , 10 , 11 , 12 and personal protective measures 13 . Additionally, modelling studies typically focus on NPIs that directly influence contact probabilities (for example, social distancing measures 18 , social distancing behaviours 12 , self-isolation, school closures, bans on public events 20 and so on). Some studies focused on a single country or even a town 14 , 15 , 16 , 17 , 18 while other research combined data from multiple countries but pooled NPIs into rather broad categories 15 , 19 , 20 , 21 , which eventually limits the assessment of specific, potentially critical, NPIs that may be less costly and more effective than others. Despite their widespread use, relative ease of implementation, broad choice of available tools and their importance in developing countries where other measures (for example, increases in healthcare capacity, social distancing or enhanced testing) are difficult to implement 22 , little is currently known about the effectiveness of different risk-communication strategies. An accurate assessment of communication activities requires information on the targeted public, means of communication and content of the message.

Using a comprehensive, hierarchically coded dataset of 6,068 NPIs implemented in March–April 2020 (when most European countries and US states experienced their first infection waves) in 79 territories 23 , here we analyse the impact of government interventions on R t using harmonized results from a multi-method approach consisting of (1) a case-control analysis (CC), (2) a step function approach to LASSO time-series regression (LASSO), (3) random forests (RF) and (4) transformers (TF). We contend that the combination of four different methods, combining statistical, inference and artificial intelligence classes of tools, also allows assessment of the structural uncertainty of individual methods 24 . We also investigate country-specific control strategies as well as the impact of selected country-specific metrics.

All the above approaches (1–4) yield comparable rankings of the effectiveness of different categories of NPIs across their hierarchical levels. This remarkable agreement allows us to identify a consensus set of NPIs that lead to a significant reduction in R t . We validate this consensus set using two external datasets covering 42,151 measures in 226 countries. Furthermore, we evaluate the heterogeneity of the effectiveness of individual NPIs in different territories. We find that the time of implementation, previously implemented measures, different governance indicators 25 , as well as human and social development affect the effectiveness of NPIs in countries to varying degrees.

Global approach

Our main results are based on the Complexity Science Hub COVID-19 Control Strategies List (CCCSL) 23 . This dataset provides a hierarchical taxonomy of 6,068 NPIs, coded on four levels, including eight broad themes (level 1, L1) divided into 63 categories of individual NPIs (level 2, L2) that include >500 subcategories (level 3, L3) and >2,000 codes (level 4, L4). We first compare the results for NPI effectiveness rankings for the four methods of our approach (1–4) on L1 (themes) (Supplementary Fig. 1 ). A clear picture emerges where the themes of social distancing and travel restrictions are top ranked in all methods, whereas environmental measures (for example, cleaning and disinfection of shared surfaces) are ranked least effective.

We next compare results obtained on L2 of the NPI dataset—that is, using the 46 NPI categories implemented more than five times. The methods largely agree on the list of interventions that have a significant effect on R t (Fig. 1 and Table 1 ). The individual rankings are highly correlated with each other ( P  = 0.0008; Methods ). Six NPI categories show significant impacts on R t in all four methods. In Supplementary Table 1 we list the subcategories (L3) belonging to these consensus categories.

The left-hand panel shows the combined 95% confidence intervals of Δ R t for the most effective interventions across all included territories. The heatmap in the right-hand panel shows the corresponding Z -scores of measure effectiveness as determined by the four different methods. Grey indicates no significantly positive effect. NPIs are ranked according to the number of methods agreeing on their impacts, from top (significant in all methods) to bottom (ineffective in all analyses). L1 themes are colour-coded as in Supplementary Fig. 1 .

A normalized score for each NPI category is obtained by rescaling the result within each method to range between zero (least effective) and one (most effective) and then averaging this score. The maximal (minimal) NPI score is therefore 100% (0%), meaning that the measure is the most (least) effective measure in each method. We show the normalized scores for all measures in the CCCSL dataset in Extended Data Fig. 1 , for the CoronaNet dataset in Extended Data Fig. 2 and for the WHO Global Dataset of Public Health and Social Measures (WHO-PHSM) in Extended Data Fig. 3 . Among the six full-consensus NPI categories in the CCCSL, the largest impacts on R t are shown by small gathering cancellations (83%, Δ R t between −0.22 and –0.35), the closure of educational institutions (73%, and estimates for Δ R t ranging from −0.15 to −0.21) and border restrictions (56%, Δ R t between −0.057 and –0.23). The consensus measures also include NPIs aiming to increase healthcare and public health capacities (increased availability of personal protective equipment (PPE): 51%, Δ R t −0.062 to −0.13), individual movement restrictions (42%, Δ R t −0.08 to −0.13) and national lockdown (including stay-at-home order in US states) (25%, Δ R t −0.008 to −0.14).

We find 14 additional NPI categories consensually in three of our methods. These include mass gathering cancellations (53%, Δ R t between −0.13 and –0.33), risk-communication activities to inform and educate the public (48%, Δ R t between –0.18 and –0.28) and government assistance to vulnerable populations (41%, Δ R t between −0.17 and –0.18).

Among the least effective interventions we find: government actions to provide or receive international help, measures to enhance testing capacity or improve case detection strategy (which can be expected to lead to a short-term rise in cases), tracing and tracking measures as well as land border and airport health checks and environmental cleaning.

In Fig. 2 we show the findings on NPI effectiveness in a co-implementation network. Nodes correspond to categories (L2) with size being proportional to their normalized score. Directed links from i to j indicate a tendency that countries implement NPI j after they have implemented i . The network therefore illustrates the typical NPI implementation sequence in the 56 countries and the steps within this sequence that contribute most to a reduction in R t . For instance, there is a pattern where countries first cancel mass gatherings before moving on to cancellations of specific types of small gatherings, where the latter associates on average with more substantial reductions in R t . Education and active communication with the public is one of the most effective ‘early measures’ (implemented around 15 days before 30 cases were reported and well before the majority of other measures comes). Most social distancing (that is, closure of educational institutions), travel restriction measures (that is, individual movement restrictions like curfew and national lockdown) and measures to increase the availability of PPE are typically implemented within the first 2 weeks after reaching 30 cases, with varying impacts on R t ; see also Fig. 1 .

Nodes are categories (L2), with colours indicating the theme (L1) and size being proportional to the average effectiveness of the intervention. Arrows from nodes i to j denote that those countries which have already implemented intervention i tend to implement intervention j later in time. Nodes are positioned vertically according to their average time of implementation (measured relative to the day where that country reached 30 confirmed cases), and horizontally according to their L1 theme. The stacked histogram on the right shows the number of implemented NPIs per time period (epidemic age) and theme (colour). v.p., vulnerable populations; c.e., certain establishments; quarantine f., quarantine facilities.

Within the CC approach, we can further explore these results on a finer hierarchical level. We show results for 18 NPIs (L3) of the risk-communication theme in Supplementary Information and Supplementary Table 2 . The most effective communication strategies include warnings against travel to, and return from, high-risk areas (Δ R CC t  = −0.14 (1); the number in parenthesis denotes the standard error) and several measures to actively communicate with the public. These include to encourage, for example, staying at home (Δ R CC t  = −0.14 (1)), social distancing (Δ R CC t  = −0.20 (1)), workplace safety measures (Δ R CC t  = −0.18 (2)), self-initiated isolation of people with mild respiratory symptoms (Δ R CC t  = −0.19 (2)) and information campaigns (Δ R CC t  = −0.13 (1)) (through various channels including the press, flyers, social media or phone messages).

Validation with external datasets

We validate our findings with results from two external datasets ( Methods ). In the WHO-PHSM dataset 26 we find seven full-consensus measures (agreement on significance by all methods) and 17 further measures with three agreements (Extended Data Fig. 4 ). These consensus measures show a large overlap with those (three or four matches in our methods) identified using the CCCSL, and include top-ranked NPI measures aiming at strengthening the healthcare system and testing capacity (labelled as ‘scaling up’)—for example, increasing the healthcare workforce, purchase of medical equipment, testing, masks, financial support to hospitals, increasing patient capacity, increasing domestic production of PPE. Other consensus measures consist of social distancing measures (‘cancelling, restricting or adapting private gatherings outside the home’, adapting or closing ‘offices, businesses, institutions and operations’, ‘cancelling, restricting or adapting mass gatherings’), measures for special populations (‘protecting population in closed settings’, encompassing long-term care facilities and prisons), school closures, travel restrictions (restricting entry and exit, travel advice and warning, ‘closing international land borders’, ‘entry screening and isolation or quarantine’) and individual movement restriction (‘stay-at-home order’, which is equivalent to confinement in the WHO-PHSM coding). ‘Wearing a mask’ exhibits a significant impact on R t in three methods (Δ R t between −0.018 and –0.12). The consensus measures also include financial packages and general public awareness campaigns (as part of ‘communications and engagement’ actions). The least effective measures include active case detection, contact tracing and environmental cleaning and disinfection.

The CCCSL results are also compatible with findings from the CoronaNet dataset 27 (Extended Data Figs. 5 and 6 ). Analyses show four full-consensus measures and 13 further NPIs with an agreement of three methods. These consensus measures include heterogeneous social distancing measures (for example, restriction and regulation of non-essential businesses, restrictions of mass gatherings), closure and regulation of schools, travel restrictions (for example, internal and external border restrictions), individual movement restriction (curfew), measures aiming to increase the healthcare workforce (for example, ‘nurses’, ‘unspecified health staff’) and medical equipment (for example, PPE, ‘ventilators’, ‘unspecified health materials’), quarantine (that is, voluntary or mandatory self-quarantine and quarantine at a government hotel or facility) and measures to increase public awareness (‘disseminating information related to COVID-19 to the public that is reliable and factually accurate’).

Twenty-three NPIs in the CoronaNet dataset do not show statistical significance in any method, including several restrictions and regulations of government services (for example, for tourist sites, parks, public museums, telecommunications), hygiene measures for public areas and other measures that target very specific populations (for example, certain age groups, visa extensions).

Country-level approach

A sensitivity check of our results with respect to the removal of individual continents from the analysis also indicates substantial variations between world geographical regions in terms of NPI effectiveness ( Supplementary Information ). To further quantify how much the effectiveness of an NPI depends on the particular territory (country or US state) where it has been introduced, we measure the heterogeneity of NPI rankings in different territories through an entropic approach in the TF method ( Methods ). Figure 3 shows the normalized entropy of each NPI category versus its rank. A value of entropy close to zero implies that the corresponding NPI has a similar rank relative to all other NPIs in all territories: in other words, the effectiveness of the NPI does not depend on the specific country or state. On the other hand, a high value of the normalized entropy signals that the performance of each NPI depends largely on the geographical region.

Each NPI is colour coded according to its theme of belonging (L1), as indicated in the legend. The blue curve represents the same information obtained from a reshuffled dataset of NPIs.

The values of the normalized entropies for many NPIs are far from one, and are also below the corresponding values obtained through temporal reshuffling of NPIs in each country. The effectiveness of many NPIs therefore is, first, significant and, second, depends on the local context (combination of socio-economic features and NPIs already adopted) to varying degrees. In general, social distancing measures and travel restrictions show a high entropy (effectiveness varies considerably across countries) whereas case identification, contact tracing and healthcare measures show substantially less country dependence.

We further explore this interplay of NPIs with socio-economic factors by analysing the effects of demographic and socio-economic covariates, as well as indicators for governance and human and economic development in the CC method ( Supplementary Information ). While the effects of most indicators vary across different NPIs at rather moderate levels, we find a robust tendency that NPI effectiveness correlates negatively with indicator values for governance-related accountability and political stability (as quantified by World Governance Indicators provided by the World Bank).

Because the heterogeneity of the effectiveness of individual NPIs across countries points to a non-independence among different NPIs, the impact of a specific NPI cannot be evaluated in isolation. Since it is not possible in the real world to change the sequence of NPIs adopted, we resort to ‘what-if’ experiments to identify the most likely outcome of an artificial sequence of NPIs in each country. Within the TF approach, we selectively delete one NPI at a time from all sequences of interventions in all countries and compute the ensuing evolution of R t compared to the actual case.

To quantify whether the effectiveness of a specific NPI depends on its epidemic age of implementation, we study artificial sequences of NPIs constructed by shifting the selected NPI to other days, keeping the other NPIs fixed. In this way, for each country and each NPI, we obtain a curve of the most likely change in R t versus the adoption time of the specific NPI.

Figure 4 shows an example of the results for a selection of NPIs (see Supplementary Information for a more extensive report on other NPIs). Each curve shows the average change in R t versus the adoption time of the NPI, averaged over the countries where that NPI has been adopted. Figure 4a refers to the national lockdown (including stay-at-home order implemented in US states). Our results show a moderate effect of this NPI (low change in R t ) as compared to other, less drastic, measures. Figure 4b shows NPIs with the pattern ‘the earlier, the better’. For those measures (‘closure of educational institutions’, ‘small gatherings cancellation’, ‘airport restrictions’ and many more shown in Supplementary Information ), early adoption is always more beneficial. In Fig. 4c , ‘enhancing testing capacity’ and ‘surveillance’ exhibit a negative impact (that is, an increase) on R t , presumably related to the fact that more testing allows for more cases to be identified. Finally, Fig. 4d , showing ‘tracing and tracking’ and ‘activate case notification’, demonstrates an initially negative effect that turns positive (that is, toward a reduction in R t ). Refer to Supplementary Information for a more comprehensive analysis of all NPIs.

a , National lockdown (including stay-at-home order in US states). b , A selection of three NPIs displaying ‘the earlier the better’ behaviour—that is, their impact is enhanced if implemented at earlier epidemic ages. c , Enhance laboratory testing capacity and Surveillance. d , Tracing and tracking and Activate case notification. Negative (positive) values indicate that the adoption of the NPI has reduced (increased) the value of R t . Shaded areas denote s.d.

Our study dissects the entangled packages of NPIs 23 and quantifies their effectiveness. We validate our findings using three different datasets and four independent methods. Our findings suggest that no NPI acts as a silver bullet on the spread of COVID-19. Instead, we identify several decisive interventions that significantly contribute to reducing R t below one and that should therefore be considered as efficiently flattening the curve facing a potential second COVID-19 wave, or any similar future viral respiratory epidemics.

The most effective NPIs include curfews, lockdowns and closing and restricting places where people gather in smaller or large numbers for an extended period of time. This includes small gathering cancellations (closures of shops, restaurants, gatherings of 50 persons or fewer, mandatory home working and so on) and closure of educational institutions. While in previous studies, based on smaller numbers of countries, school closures had been attributed as having little effect on the spread of COVID-19 (refs. 19 , 20 ), more recent evidence has been in favour of the importance of this NPI 28 , 29 ; school closures in the United States have been found to reduce COVID-19 incidence and mortality by about 60% (ref. 28 ). This result is also in line with a contact-tracing study from South Korea, which identified adolescents aged 10–19 years as more likely to spread the virus than adults and children in household settings 30 . Individual movement restrictions (including curfew, the prohibition of gatherings and movements for non-essential activities or measures segmenting the population) were also amongst the top-ranked measures.

However, such radical measures have adverse consequences. School closure interrupts learning and can lead to poor nutrition, stress and social isolation in children 31 , 32 , 33 . Home confinement has strongly increased the rate of domestic violence in many countries, with a huge impact on women and children 34 , 35 , while it has also limited the access to long-term care such as chemotherapy, with substantial impacts on patients’ health and survival chance 36 , 37 . Governments may have to look towards less stringent measures, encompassing maximum effective prevention but enabling an acceptable balance between benefits and drawbacks 38 .

Previous statistical studies on the effectiveness of lockdowns came to mixed conclusions. Whereas a relative reduction in R t of 5% was estimated using a Bayesian hierarchical model 19 , a Bayesian mechanistic model estimated a reduction of 80% (ref. 20 ), although some questions have been raised regarding the latter work because of biases that overemphasize the importance of the most recent measure that had been implemented 24 . The susceptibility of other modelling approaches to biases resulting from the temporal sequence of NPI implementations remains to be explored. Our work tries to avoid such biases by combining multiple modelling approaches and points to a mild impact of lockdowns due to an overlap with effects of other measures adopted earlier and included in what is referred to as ‘national (or full) lockdown’. Indeed, the national lockdown encompasses multiple NPIs (for example, closure of land, sea and air borders, closure of schools, non-essential shops and prohibition of gatherings and visiting nursing homes) that countries may have already adopted in parts. From this perspective, the relatively attenuated impact of the national lockdown is explained as the little delta after other concurrent NPIs have been adopted. This conclusion does not rule out the effectiveness of an early national lockdown, but suggests that a suitable combination (sequence and time of implementation) of a smaller package of such measures can substitute for a full lockdown in terms of effectiveness, while reducing adverse impacts on society, the economy, the humanitarian response system and the environment 6 , 39 , 40 , 41 .

Taken together, the social distancing and movement-restriction measures discussed above can therefore be seen as the ‘nuclear option’ of NPIs: highly effective but causing substantial collateral damages to society, the economy, trade and human rights 4 , 39 .

We find strong support for the effectiveness of border restrictions. The role of travelling in the global spread of respiratory diseases proved central during the first SARS epidemic (2002–2003) 42 , but travelling restrictions show a large impact on trade, economy and the humanitarian response system globally 41 , 43 . The effectiveness of social distancing and travel restrictions is also in line with results from other studies that used different statistical approaches, epidemiological metrics, geographic coverage and NPI classification 2 , 8 , 9 , 10 , 11 , 13 , 19 , 20 .

We also find a number of highly effective NPIs that can be considered less costly. For instance, we find that risk-communication strategies feature prominently amongst consensus NPIs. This includes government actions intended to educate and actively communicate with the public. The effective messages include encouraging people to stay at home, promoting social distancing and workplace safety measures, encouraging the self-initiated isolation of people with symptoms, travel warnings and information campaigns (mostly via social media). All these measures are non-binding government advice, contrasting with the mandatory border restriction and social distancing measures that are often enforced by police or army interventions and sanctions. Surprisingly, communicating on the importance of social distancing has been only marginally less effective than imposing distancing measures by law. The publication of guidelines and work safety protocols to managers and healthcare professionals was also associated with a reduction in R t , suggesting that communication efforts also need to be tailored toward key stakeholders. Communication strategies aim at empowering communities with correct information about COVID-19. Such measures can be of crucial importance in targeting specific demographic strata found to play a dominant role in driving the spread of COVID-19 (for example, communication strategies to target individuals aged <40 years 44 ).

Government food assistance programmes and other financial supports for vulnerable populations have also turned out to be highly effective. Such measures are, therefore, not only impacting the socio-economic sphere 45 but also have a positive effect on public health. For instance, facilitating people’s access to tests or allowing them to self-isolate without fear of losing their job or part of their salary may help in reducing R t .

Some measures are ineffective in (almost) all methods and datasets—for example, environmental measures to disinfect and clean surfaces and objects in public and semi-public places. This finding is at odds with current recommendations of the WHO (World Health Organization) for environmental cleaning in non-healthcare settings 46 , and calls for a closer examination of the effectiveness of such measures. However, environmental measures (for example, cleaning of shared surfaces, waste management, approval of a new disinfectant, increased ventilation) are seldom reported by governments or the media and are therefore not collected by NPI trackers, which could lead to an underestimation of their impact. These results call for a closer examination of the effectiveness of such measures. We also find no evidence for the effectiveness of social distancing measures in regard to public transport. While infections on buses and trains have been reported 47 , our results may suggest a limited contribution of such cases to the overall virus spread, as previously reported 48 . A heightened public risk awareness associated with commuting (for example, people being more likely to wear face masks) might contribute to this finding 49 . However, we should note that measures aiming at limiting engorgement or increasing distancing on public transport have been highly diverse (from complete cancellation of all public transport to increase in the frequency of traffic to reduce traveller density) and could therefore lead to widely varying effectiveness, also depending on the local context.

The effectiveness of individual NPIs is heavily influenced by governance ( Supplementary Information ) and local context, as evidenced by the results of the entropic approach. This local context includes the stage of the epidemic, socio-economic, cultural and political characteristics and other NPIs previously implemented. The fact that gross domestic product is overall positively correlated with NPI effectiveness whereas the governance indicator ‘voice and accountability’ is negatively correlated might be related to the successful mitigation of the initial phase of the epidemic of certain south-east Asian and Middle East countries showing authoritarian tendencies. Indeed, some south-east Asian government strategies heavily relied on the use of personal data and police sanctions whereas the Middle East countries included in our analysis reported low numbers of cases in March–April 2020.

By focusing on individual countries, the what-if experiments using artificial country-specific sequences of NPIs offer a way to quantify the importance of this local context with respect to measurement of effectiveness. Our main takeaway here is that the same NPI can have a drastically different impact if taken early or later, or in a different country.

It is interesting to comment on the impact that ‘enhancing testing capacity’ and ‘tracing and tracking’ would have had if adopted at different points in time. Enhancing testing capacity should display a short-term increase in R t . Counter-intuitively, in countries testing close contacts, tracing and tracking, if they are effective, would have a similar effect on R t because more cases will be found (although tracing and tracking would reduce R t in countries that do not test contacts but rely on quarantine measures). For countries implementing these measures early, indeed, we find a short-term increase in R t (when the number of cases was sufficiently small to enable tracing and testing of all contacts). However, countries implementing these NPIs later did not necessarily find more cases, as shown by the corresponding decrease in R t . We focus on March and April 2020, a period in which many countries had a sudden surge in cases that overwhelmed their tracing and testing capacities, which rendered the corresponding NPIs ineffective.

Assessment of the effectiveness of NPIs is statistically challenging, because measures were typically implemented simultaneously and their impact might well depend on the particular implementation sequence. Some NPIs appear in almost all countries whereas in others only a few, meaning that we could miss some rare but effective measures due to a lack of statistical power. While some methods might be prone to overestimation of the effects from an NPI due to insufficient adjustments for confounding effects from other measures, other methods might underestimate the contribution of an NPI by assigning its impact to a highly correlated NPI. As a consequence, estimates of Δ R t might vary substantially across different methods whereas agreement on the significance of individual NPIs is much more pronounced. The strength of our study, therefore, lies in the harmonization of these four independent methodological approaches combined with the usage of an extensive dataset on NPIs. This allows us to estimate the structural uncertainty of NPI effectiveness—that is, the uncertainty introduced by choosing a certain model structure likely to affect other modelling works that rely on a single method only. Moreover, whereas previous studies often subsumed a wide range of social distancing and travel restriction measures under a single entity, our analysis contributes to a more fine-grained understanding of each NPI.

The CCCSL dataset features non-homogeneous data completeness across the different territories, and data collection could be biased by the data collector (native versus non-native) as well as by the information communicated by governments (see also ref. 23 ). The WHO-PHSM and CoronaNet databases contain a broad geographic coverage whereas CCCSL focuses mostly on developed countries. Moreover, the coding system presents certain drawbacks, notably because some interventions could belong to more than one category but are recorded only once. Compliance with NPIs is crucial for their effectiveness, yet we assumed a comparable degree of compliance by each population. We tried to mitigate this issue by validating our findings on two external databases, even if these are subject to similar limitations. We did not perform a formal harmonization of all categories in the three NPI trackers, which limits our ability to perform full comparisons among the three datasets. Additionally, we neither took into account the stringency of NPI implementation nor the fact that not all methods were able to describe potential variations in NPI effectiveness over time, besides the dependency on the epidemic age of its adoption. The time window is limited to March–April 2020, where the structure of NPIs is highly correlated due to simultaneous implementation. Future research should consider expanding this window to include the period when many countries were easing policies, or maybe even strenghening them again after easing, as this would allow clearer differentiation of the correlated structure of NPIs because they tended to be released, and implemented again, one (or a few) at a time.

To compute R t , we used time series of the number of confirmed COVID-19 cases 50 . This approach is likely to over-represent patients with severe symptoms and may be biased by variations in testing and reporting policies among countries. Although we assume a constant serial interval (average timespan between primary and secondary infection), this number shows considerable variation in the literature 51 and depends on measures such as social distancing and self-isolation.

In conclusion, here we present the outcome of an extensive analysis on the impact of 6,068 individual NPIs on the R t of COVID-19 in 79 territories worldwide. Our analysis relies on the combination of three large and fine-grained datasets on NPIs and the use of four independent statistical modelling approaches.

The emerging picture reveals that no one-size-fits-all solution exists, and no single NPI can decrease R t below one. Instead, in the absence of a vaccine or efficient antiviral medication, a resurgence of COVID-19 cases can be stopped only by a suitable combination of NPIs, each tailored to the specific country and its epidemic age. These measures must be enacted in the optimal combination and sequence to be maximally effective against the spread of SARS-CoV-2 and thereby enable more rapid reopening.

We showed that the most effective measures include closing and restricting most places where people gather in smaller or larger numbers for extended periods of time (businesses, bars, schools and so on). However, we also find several highly effective measures that are less intrusive. These include land border restrictions, governmental support to vulnerable populations and risk-communication strategies. We strongly recommend that governments and other stakeholders first consider the adoption of such NPIs, tailored to the local context, should infection numbers surge (or surge a second time), before choosing the most intrusive options. Less drastic measures may also foster better compliance from the population.

Notably, the simultaneous consideration of many distinct NPI categories allows us to move beyond the simple evaluation of individual classes of NPIs to assess, instead, the collective impact of specific sequences of interventions. The ensemble of these results calls for a strong effort to simulate what-if scenarios at the country level for planning the most probable effectiveness of future NPIs, and, thanks to the possibility of going down to the level of individual countries and country-specific circumstances, our approach is the first contribution toward this end.

We use the publicly available CCCSL dataset on NPIs 23 , in which NPIs are categorized using a four-level hierarchical coding scheme. L1 defines the theme of the NPI: ‘case identification, contact tracing and related measures’, ‘environmental measures’, ‘healthcare and public health capacity’, ‘resource allocation’, ‘returning to normal life’, ‘risk communication’, ‘social distancing’ and ‘travel restriction’. Each L1 (theme) is composed of several categories (L2 of the coding scheme) that contain subcategories (L3), which are further subdivided into group codes (L4). The dataset covers 56 countries; data for the United States are available at the state level (24 states), making a total of 79 territories. In this analysis, we use a static version of the CCCSL, retrieved on 17 August 2020, presenting 6,068 NPIs. A glossary of the codes, with a detailed description of each category and its subcategories, is provided on GitHub . For each country, we use the data until the day for which the measures have been reliably updated. NPIs that have been implemented in fewer than five territories are not considered, leading to a final total of 4,780 NPIs of 46 different L2 categories for use in the analyses.

Second, we use the CoronaNet COVID-19 Government Response Event Dataset (v.1.0) 27 that contains 31,532 interventions and covers 247 territories (countries and US states) (data extracted on 17 August 2020). For our analysis, we map their columns ‘type’ and ‘type_sub_cat’ onto L1 and L2, respectively. Definitions for the entire 116 L2 categories can be found on the GitHub page of the project.

Using the same criterion as for the CCCSL, we obtain a final total of 18,919 NPIs of 107 different categories.

Third, we use the WHO-PHSM dataset 26 , which merges and harmonizes the following datasets: ACAPS 41 , Oxford COVID-19 Government Response Tracker 52 , the Global Public Health Intelligence Network (GPHIN) of Public Health Agency of Canada (Ottawa, Canada), the CCCSL 23 , the United States Centers for Disease Control and Prevention and HIT-COVID 53 . The WHO-PHSM dataset contains 24,077 interventions and covers 264 territories (countries and US states; data extracted on 17 August 2020). Their encoding scheme has a heterogeneous coding depth and, for our analysis, we map ‘who_category’ onto L1 and either take ‘who_subcategory’ or a combination of ‘who_subcategory’ and ‘who_measure’ as L2. This results in 40 measure categories. A glossary is available at: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/phsm .

The CoronaNet and WHO-PHSM datasets also provide information on the stringency of the implementation of a given NPI, which we did not use in the current study.

COVID-19 case data

To estimate R t and growth rates of the number of COVID-19 cases, we use time series of the number of confirmed COVID-19 cases in the 79 territories considered 50 . To control for weekly fluctuations, we smooth the time series by computing the rolling average using a Gaussian window with a standard deviation of 2 days, truncated at a maximum window size of 15 days.

Regression techniques

We apply four different statistical approaches to quantify the impact of a NPI, M , on the reduction in R t ( Supplementary Information ).

Case-control analysis considers each single category (L2) or subcategory (L3) M separately and evaluates in a matched comparison the difference, Δ R t , in R t between all countries that implemented M (cases) and those that did not (controls) during the observation window. The matching is done on epidemic age and the time of implementation of any response. The comparison is made via a linear regression model adjusting for (1) epidemic age (days after the country has reached 30 confirmed cases), (2) the value of R t before M takes effect, (3) total population, (4) population density, (5) the total number of NPIs implemented and (6) the number of NPIs implemented in the same category as M . With this design, we investigate the time delay of τ days between implemention of M and observation of Δ R t , as well as additional country-based covariates that quantify other dimensions of governance and human and economic development. Estimates for R t are averaged over delays between 1 and 28 days.

Step function Lasso regression

In this approach we assume that, without any intervention, the reproduction factor is constant and deviations from this constant result from a delayed onset by τ  days of each NPI on L2 (categories) of the hierarchical dataset. We use a Lasso regularization approach combined with a meta parameter search to select a reduced set of NPIs that best describe the observed Δ R t . Estimates for the changes in Δ R t attributable to NPI M are obtained from country-wise cross-validation.

RF regression

We perform a RF regression, where the NPIs implemented in a country are used as predictors for R t , time-shifted τ  days into the future. Here, τ accounts for the time delay between implementation and onset of the effect of a given NPI. Similar to the Lasso regression, the assumption underlying the RF approach is that, without changes in interventions, the value of R t in a territory remains constant. However, contrary to the two methods described above, RF represents a nonlinear model, meaning that the effects of individual NPIs on R t do not need to add up linearly. The importance of a NPI is defined as the decline in predictive performance of the RF on unseen data if the data concerning that NPI are replaced by noise, also called permutation importance.

Transformer modelling

Transformers 54 have been demonstrated as models suitable for dynamic discrete element processes such as textual sequences, due to their ability to recall past events. Here we extended the transformer architecture to approach the continuous case of epidemic data by removing the probabilistic output layer with a linear combination of transformer output, whose input is identical to that for RF regression, along with the values of R t . The best-performing network (least mean-squared error in country-wise cross-validation) is identified as a transformer encoder with four hidden layers of 128 neurons, an embedding size of 128, eight heads, one output described by a linear output layer and 47 inputs (corresponding to each category and R t ). To quantify the impact of measure M on R t , we use the trained transformer as a predictive model and compare simulations without any measure (reference) to those where one measure is presented at a time to assess Δ R t . To reduce the effects of overfitting and multiplicity of local minima, we report results from an ensemble of transformers trained to similar precision levels.

Estimation of R t

We use the R package EpiEstim 55 with a sliding time window of 7 days to estimate the time series of R t for every country. We choose an uncertain serial interval following a probability distribution with a mean of 4.46 days and a standard deviation of 2.63 days 56 .

Ranking of NPIs

For each of the methods (CC, Lasso regression and TF), we rank the NPI categories in descending order according to their impact—that is, the estimated degree to which they lower R t or their feature importance (RF). To compare rankings, we count how many of the 46 NPIs considered are classified as belonging to the top x ranked measures in all methods, and test the null hypothesis that this overlap has been obtained from completely independent rankings. The P  value is then given by the complementary cumulative distribution function for a binomial experiment with 46 trials and success probability ( x /46) 4 . We report the median P  value obtained over all x  ≤ 10 to ensure that the results are not dependent on where we impose the cut-off for the classes.

Co-implementation network

If there is a statistical tendency that a country implementing NPI i also implements NPI j later in time, we draw a direct link from i to j . Nodes are placed on the y  axis according to the average epidemic age at which the corresponding NPI is implemented; they are grouped on the x  axis by their L1 theme. Node colours correspond to themes. The effectiveness scores for all NPIs are re-scaled between zero and one for each method; node size is proportional to the re-scaled scores, averaged over all methods.

Entropic country-level approach

Each territory can be characterized by its socio-economic conditions and the unique temporal sequence of NPIs adopted. To quantify the NPI effect, we measure the heterogeneity of the overall rank of a NPI amongst the countries that have taken that NPI. To compare countries that have implemented different numbers of NPIs, we consider the normalized rankings where the ranking position is divided by the number of elements in the ranking list (that is, the number of NPIs taken in a specific country). We then bin the interval [0, 1] of the normalized rankings into ten sub-intervals and compute for each NPI the entropy of the distribution of occurrences of that NPI in the different normalized rankings per country:

where P i is the probability that the NPI considered appeared in the i th bin in the normalized rankings of all countries. To assess the confidence of these entropic values, results are compared with expectations from a temporal reshuffling of the data. For each country, we keep the same NPIs adopted but reshuffle the time stamps of their adoption.

Reporting Summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

The CCCSL dataset can be downloaded from http://covid19-interventions.com/ . The CoronaNet data can be found at https://www.coronanet-project.org/ . The WHO-PHSM dataset is available at https://www.who.int/emergencies/diseases/novel-coronavirus-2019/phsm . Snapshots of the datasets used in our study are available in the following github repository: https://github.com/complexity-science-hub/ranking_npis .

Code availability

Custom code for the analysis is available in the following github repository: https://github.com/complexity-science-hub/ranking_npis .

Qualls, N. L. et al. Community mitigation guidelines to prevent pandemic influenza – United States, 2017. MMWR Recomm. Rep. 66 , 1–34 (2017).

Tian, H. et al. An investigation of transmission control measures during the first 50 days of the COVID-19 epidemic in China. Science 368 , 638–642 (2020).

Article   CAS   Google Scholar  

Chen, S. et al. COVID-19 control in China during mass population movements at New Year. Lancet 395 , 764–766 (2020).

Lee, K., Worsnop, C. Z., Grépin, K. A. & Kamradt-Scott, A. Global coordination on cross-border travel and trade measures crucial to COVID-19 response. Lancet 395 , 1593–1595 (2020).

Chakraborty, I. & Maity, P. Covid-19 outbreak: migration, effects on society, global environment and prevention. Sci. Total Environ. 728 , 138882 (2020).

Pfefferbaum, B. & North, C. S. Mental health and the COVID-19 pandemic. N. Eng. J. Med. 383 , 510–512.

COVID-19 dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University of Medicine (Johns Hopkins University of Medicine, accessed 4 June 2020); https://coronavirus.jhu.edu/map.html .

Chinazzi, M. et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science 368 , 395–400 (2020).

Arenas, A., Cota, W., Granell, C. & Steinegger, B. Derivation of the effective reproduction number R for COVID-19 in relation to mobility restrictions and confinement. Preprint at medRxiv https://doi.org/10.1101/2020.04.06.20054320 (2020).

Wang, J., Tang, K., Feng, K. & Lv, W. When is the COVID-19 pandemic over? Evidence from the stay-at-home policy execution in 106 Chinese cities. Preprint at SSRN https://doi.org/10.2139/ssrn.3561491 (2020).

Soucy, J.-P. R. et al. Estimating effects of physical distancing on the COVID-19 pandemic using an urban mobility index. Preprint at medRxiv https://doi.org/10.1101/2020.04.05.20054288 (2020).

Anderson, S. C. et al. Estimating the impact of Covid-19 control measures using a Bayesian model of physical distancing. Preprint at medRxiv https://doi.org/10.1101/2020.04.17.20070086 (2020).

Teslya, A. et al. Impact of self-imposed prevention measures and short-term government intervention on mitigating and delaying a COVID-19 epidemic. PLoS Med. https://doi.org/10.1371/journal.pmed.1003166 (2020).

Kraemer, M. U. et al. The effect of human mobility and control measures on the COVID-19 epidemic in China. Science 497 , 493–497 (2020).

Article   Google Scholar  

Prem, K. & Liu, Y. et al. The effect of control strategies to reduce social mixing on outcomes of the COVID-19 epidemic in Wuhan, China: a modelling study. Lancet Public Health 5 , e261–e270 (2020).

Gatto, M. et al. Spread and dynamics of the COVID-19 epidemic in Italy: effects of emergency containment measures. Proc. Natl Acad. Sci. USA 117 , 10484–10491 (2020).

Lorch, L. et al. A spatiotemporal epidemic model to quantify the effects of contact tracing, testing, and containment. Preprint at arXiv https://arxiv.org/abs/2004.07641 (2020).

Dehning, J. & Zierenberg, J. et al. Inferring change points in the spread of COVID-19 reveals the effectiveness of interventions. Science 369 , eabb9789 (2020).

Banholzer, N. et al. Impact of non-pharmaceutical interventions on documented cases of COVID-19. Preprint at medRxiv https://doi.org/10.1101/2020.04.16.20062141 (2020).

Flaxman, S. et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature 584 , 257–261 (2020).

Hsiang, S. et al. The effect of large-scale anti-contagion policies on the COVID-19 pandemic. Nature 584 , 262–267 (2020).

Nachega, J., Seydi, M. & Zumla, A. The late arrival of coronavirus disease 2019 (Covid-19) in Africa: mitigating pan-continental spread. Clin. Infect. Dis. 71 , 875–878 (2020).

Desvars-Larrive, A. et al. A structured open dataset of government interventions in response to COVID-19. Sci. Data 7 , 285 (2020).

Bryant, P. & Elofsson, A. The limits of estimating COVID-19 intervention effects using Bayesian models. Preprint at medRxiv https://doi.org/10.1101/2020.08.14.20175240 (2020).

Protecting People and Economies: Integrated Policy Responses to COVID-19 (World Bank, 2020); https://openknowledge.worldbank.org/handle/10986/33770

Tracking Public Health and Social Measures: A Global Dataset (World Health Organization, 2020); https://www.who.int/emergencies/diseases/novel-coronavirus-2019/phsm

Cheng, C., Barceló, J., Hartnett, A. S., Kubinec, R. & Messerschmidt, L. COVID-19 government response event dataset (CoronaNet v.1.0). Nat. Hum. Behav. 4 , 756–768 (2020).

Auger, K. A. et al. Association between statewide school closure and COVID-19 incidence and mortality in the US. JAMA 324 , 859–870 (2020).

Liu, Y. et al. The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories. Preprint at medRxiv https://doi.org/10.1101/2020.08.11.20172643 (2020).

Park, Y., Choe, Y. et al. Contact tracing during coronavirus disease outbreak. Emerg. Infect. Dis . 26 , 2465–2468(2020).

Adverse Consequences of School Closures (UNESCO, 2020); https://en.unesco.org/covid19/educationresponse/consequences

Education and COVID-19: Focusing on the Long-term Impact of School Closures (OECD, 2020); https://www.oecd.org/coronavirus/policy-responses/education-and-covid-19-focusing-on-the-long-term-impact-of-school-closures-2cea926e/

Orben, A., Tomova, L. & Blakemore, S.-J. The effects of social deprivation on adolescent development and mental health. Lancet Child Adolesc. Health 4 , 634–640 (2020).

Taub, A. A new covid-19 crisis: domestic abuse rises worldwide. The New York Times https://www.nytimes.com/2020/04/06/world/coronavirus-domestic-violence.html (6 April 2020).

Abramian, J. The Covid-19 pandemic has escalated domestic violence worldwide. Forbes https://www.forbes.com/sites/jackieabramian/2020/07/22/the-covid-19-pandemic-has-escalated-global-domestic-violence/#57366498173e (22 July 2020).

Tsamakis, K. et al. Oncology during the COVID-19 pandemic: challenges, dilemmas and the psychosocial impact on cancer patients (review). Oncol. Lett. 20 , 441–447 (2020).

Raymond, E., Thieblemont, C., Alran, S. & Faivre, S. Impact of the COVID-19 outbreak on the management of patients with cancer. Target. Oncol. 15 , 249–259 (2020).

Couzin-Frankel, J., Vogel, G. & Weiland, M. School openings across globe suggest ways to keep coronavirus at bay, despite outbreaks. Science https://www.sciencemag.org/news/2020/07/school-openings-across-globe-suggest-ways-keep-coronavirus-bay-despite-outbreaks# (2020).

Vardoulakis, S., Sheel, M., Lal, A. & Gray, D. Covid-19 environmental transmission and preventive public health measures. Aust. N. Z. J. Public Health 44 , 333–335 (2020).

Saadat, S., Rawtani, D. & Hussain, C. M. Environmental perspective of Covid-19. Sci. Total Environ. 728 , 138870 (2020).

Covid-19 Government Measures Dataset (ACAPS, 2020); https://www.acaps.org/covid19-government-measures-dataset

Brockmann, D. & Helbing, D. The hidden geometry of complex, network-driven contagion phenomena. Science 342 , 1337–1342 (2013).

Guan, D. et al. Global supply-chain effects of Covid-19 control measures. Nat. Hum. Behav. 4 , 577–587 (2020).

Malmgren, J., Guo, B. & Kaplan, H. G. Covid-19 confirmed case incidence age shift to young persons aged 0–19 and 20–39 years over time: Washington State March–April 2020. Preprint at medRxiv https://doi.org/10.1101/2020.05.21.20109389 (2020).

Gentilini, U., Almenfi, M., Orton, I. & Dale, P. Social Protection and Jobs Responses to COVID-19 (World Bank, 2020); https://openknowledge.worldbank.org/handle/10986/33635

Cleaning and Disinfection of Environmental Surfaces in the Context of COVID-19 (World Health Organization, 2020); https://www.who.int/publications/i/item/cleaning-and-disinfection-of-environmental-surfaces-inthe-context-of-covid-19

Shen, J. et al. Prevention and control of COVID-19 in public transportation: experience from China. Environ. Pollut. 266 , 115291 (2020).

Islam, N. et al. Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries. BMJ 370 , m2743 (2020).

Liu, X. & Zhang, S. Covid-19: face masks and human-to-human transmission. Influenza Other Respir. Viruses 14 , 472–473 (2020).

2019 Novel Coronavirus COVID-19 (2019-nCoV) Data Repository by Johns Hopkins CSSE (Johns Hopkins University of Medicine, 2020); https://github.com/CSSEGISandData/COVID-19

Griffin, J. et al. A rapid review of available evidence on the serial interval and generation time of COVID-19. Preprint at medRxiv https://doi.org/10.1101/2020.05.08.20095075 (2020).

Hale, T., Webster, S., Petherick, A., Phillips, T. & Kira, B. Oxford COVID-19 Government Response Tracker (Blavatnik School of Government & University of Oxford, 2020); https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-government-response-tracker

Zheng, Q. et al. HIT-COVID, a global database tracking public health interventions to COVID-19. Sci. Data 7 , 286 (2020).

Vaswani, A. et al. in Advances in Neural Information Processing Systems 30 (eds Guyon, I. et al.) 5998–6008 (Curran Associates, 2017).

Cori, A., Ferguson, N. M., Fraser, C. & Cauchemez, S. A new framework and software to estimate time-varying reproduction numbers during epidemics. Am. J. Epidemiol. 178 , 1505–1512 (2013).

Valka, F. & Schuler, C. Estimation and interactive visualization of the time-varying reproduction number R t and the time-delay from infection to estimation. Preprint at medRxiv https://doi.org/10.1101/2020.09.19.20197970 (2020).

Download references

Acknowledgements

We thank A. Roux for her contribution to the coding of the interventions recorded in the dataset used in this study. We thank D. Garcia, V. D. P. Servedio and D. Hofmann for their contribution in the early stage of this work. N.H. thanks L. Haug for helpful discussions. This work was funded by the Austrian Science Promotion Agency, the FFG project (no. 857136), the WWTF (nos. COV 20-001, COV 20-017 and MA16-045), Medizinisch-Wissenschaftlichen Fonds des Bürgermeisters der Bundeshauptstadt Wien (no. CoVid004) and the project VET-Austria, a cooperation between the Austrian Federal Ministry of Social Affairs, Health, Care and Consumer Protection, the Austrian Agency for Health and Food Safety and the University of Veterinary Medicine, Vienna. The funders had no role in the conceptualization, design, data collection, analysis, decision to publish or preparation of the manuscript.

Author information

These authors contributed equally: Nina Haug, Lukas Geyrhofer, Alessandro Londei.

Authors and Affiliations

Medical University of Vienna, Section for Science of Complex Systems, CeMSIIS, Vienna, Austria

Nina Haug, Elma Dervic, Stefan Thurner & Peter Klimek

Complexity Science Hub Vienna, Vienna, Austria

Nina Haug, Lukas Geyrhofer, Elma Dervic, Amélie Desvars-Larrive, Vittorio Loreto, Beate Pinior, Stefan Thurner & Peter Klimek

Sony Computer Science Laboratories, Paris, France

Alessandro Londei & Vittorio Loreto

Unit of Veterinary Public Health and Epidemiology, Institute of Food Safety, Food Technology and Veterinary Public Health, University of Veterinary Medicine, Vienna, Austria

Amélie Desvars-Larrive & Beate Pinior

Physics Department, Sapienza University of Rome, Rome, Italy

Vittorio Loreto

Santa Fe Institute, Santa Fe, NM, USA

Stefan Thurner

You can also search for this author in PubMed   Google Scholar

Contributions

N.H., L.G., A.L., V.L. and P.K. conceived and performed the analyses. V.L., S.T. and P.K. supervised the study. E.D. contributed additional tools. N.H., L.G., A.L., A.D.-L., B.P. and P.K. wrote the first draft of the paper. A.D.-L. supervised data collection on NPIs. All authors discussed the results and contributed to revision of the final manuscript.

Corresponding author

Correspondence to Peter Klimek .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Peer review information Peer review reports are available. Primary handling editor: Stavroula Kousta.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended data fig. 1 main results for the cccsl dataset..

Normalised scores (relative effect within a method) of the NPI categories in CCCSL, averaged over the four different approaches.

Extended Data Fig. 2 Main results for the CoronaNet dataset.

Normalised scores (relative effect within a method) of the NPI categories in CoronaNet, averaged over the four different approaches. Full names of the abbreviated L2 categories can be looked up in SI; Supplementary Table 3.

Extended Data Fig. 3 Main results for the WHO-PHSM dataset.

Normalised scores (relative effect within a method) of the NPI categories in WHO-PHSM, averaged over the four different approaches. Full names of the abbreviated L2 categories can be looked up in SI; Supplementary Table 4.

Extended Data Fig. 4 Measure effectiveness in the WHO-PHSM dataset.

Analogue to Fig. 1 of the main text if the analysis is done on the WHO-PHSM dataset. Full names of the abbreviated L2 categories can be looked up in SI; Supplementary Table 4.

Extended Data Fig. 5 Measure effectiveness in the CoronaNet dataset(part 1).

Analogue to Fig. 1 of the main text if the analysis is done on the CoronaNat dataset (continued in Extended Data Fig. 6). Full names of the abbreviated L2 categories can be looked up in SI; Supplementary Table 3.

Extended Data Fig. 6 Measure effectiveness in the WHO-PHSM dataset (part 2).

Analogue to Fig. 1 of the main text if the analysis is done on the CoronaNat dataset (continued from Extended Data Fig. 5). Full names of the abbreviated L2 categories can be looked up in SI; Supplementary Table 3.

Supplementary information

Supplementary information.

Supplementary Methods, Supplementary Results, Supplementary Discussion, Supplementary Figs. 1–26 and Supplementary Tables 1–6.

Peer Review Information

Rights and permissions.

Reprints and permissions

About this article

Cite this article.

Haug, N., Geyrhofer, L., Londei, A. et al. Ranking the effectiveness of worldwide COVID-19 government interventions. Nat Hum Behav 4 , 1303–1312 (2020). https://doi.org/10.1038/s41562-020-01009-0

Download citation

Received : 15 October 2020

Accepted : 28 October 2020

Published : 16 November 2020

Issue Date : December 2020

DOI : https://doi.org/10.1038/s41562-020-01009-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Evaluation of angiotensin converting enzyme 2 (ace2), angiotensin ii (ang ii), mir-141-3p, and mir-421 levels in sars-cov-2 patients: a case-control study.

• Ehsan Kakavandi
• Kaveh Sadeghi
• Jila Yavarian

BMC Infectious Diseases (2024)

Systematic review of empiric studies on lockdowns, workplace closures, and other non-pharmaceutical interventions in non-healthcare workplaces during the initial year of the COVID-19 pandemic: benefits and selected unintended consequences

• Faruque Ahmed
• Livvy Shafer
• Amra Uzicanin

BMC Public Health (2024)

Multi-sectoral collaborations in selected countries of the Eastern Mediterranean region: assessment, enablers and missed opportunities from the COVID-19 pandemic response

• Fadi El-Jardali
• Racha Fadlallah
• Najla Daher

Health Research Policy and Systems (2024)

Spatial spread of COVID-19 during the early pandemic phase in Italy

• Valeria d’Andrea
• Filippo Trentini
• Stefano Merler

The impact of spatial connectivity on NPIs effectiveness

• Chiara E. Sabbatini
• Giulia Pullano
• Vittoria Colizza

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Lead in the Americas: A Call for Action (1996)

Chapter: session d: case studies of interventions, case studies of interventions, introduction: case studies of interventions.

D AVID R ALL *

The task of reducing lead exposures and preventing lead poisoning across the Americas seems a daunting task. This session highlights four case studies of interventions in different sectors. The first describes a successful voluntary industry initiative in Mexico to remove lead solder from the canning process. The second presents an example of government regulation—in this instance, the U.S. Clean Air Act—that dramatically reduced population lead levels in the United States. The third case study explores the role of international organized labor—in this case the U.S. Carpenter's Union—in educating and training workers about lead poisoning and describes the union's efforts to work with government agencies to ensure stricter protective policies for workers and their families. The final case study describes the role of community activism and education in empowering local communities in Mexico to design and implement focused public health programs to reduce lead exposure in their populations.

What these four case studies illustrate is that successful control and prevention strategies require the involvement of people and organizations at all levels of society, from the federal sector down to the grassroots or community. Lead poisoning is a problem that directly affects people at all levels of society. Its solutions must, therefore, also be shared.

A LFONSO DE L EÓN *

In 1992, the metallic containers industry in Mexico stopped producing tin cans with lead soldering as food containers, substituting instead a process that closes tin cans with electrical solder. Mexican public health authorities are now interested in determining whether a quantifiable reduction in population blood lead levels, especially in children, has occurred as a result of this voluntary industry change.

The National Chamber for Metallic Containers of Mexico represents more than 85 percent of steel tin can and 100 percent of aluminum tin can production, making it the leading manufacturer of metallic containers in Mexico. The process leading to the total elimination of lead soldering in food cans was begun as a voluntary initiative by industry, although the increased pace of change in the latter phases of conversion was dictated by external events. This process is described briefly below.

Metallic containers have been used to hold and conserve food for more than 180 years. In the beginning, tin was used in soldering, but it resulted in little flexibility and a fragile seam. Lead rapidly replaced tin in solder because it is a ductile material that easily adheres to the tin plate and can be mixed with the tin to produce a more flexible and less fragile soldering. At one point the solder commonly used contained 90 percent lead and 10 percent tin. Such soldering was universally adopted and, with it, many billions of cans were produced globally, without an understanding that lead in external soldering posed a public health hazard.

When medical and public health authorities began to acknowledge concerns about the effects of lead exposure on human health, attempts were made to identify the different sources of the metal. Leaded gasoline, paints, and ceramicware glazes and food cans containing leaded solder were rapidly identified as important sources.

Manufacturers of metallic containers had already began, by the late 1970s, to substitute lead soldering with electric soldering. This change was instituted for reasons apart from public health concerns about lead, which remained largely unknown at the time. The main motives for converting to electrical soldering were to have a cleaner production process, greater productivity, and a larger surface area on the cans for the purposes of advertising (leaded seams are broader than electrically soldered seams). Mexico adopted the electrical soldering technique, despite the many lead soldering production lines that still had many years of usable life and the $1.2-2.0 million investment that was required to refit production lines. This change was an unusual one for Mexico, which does not usually afford itself the luxury of getting rid of production equipment that is still considered useful.

At the beginning of 1991, a strong popular movement against lead in the environment was organized in Mexico City. The metallic container industries became involved in this movement, along with other lead industries. As an outcome, ecologists, united with health authorities, insisted that the electrical soldering substitution process that had been under way for over 10 years be abruptly accelerated. We cannot deny that moments in the ensuing deliberations were amusing.

The industry's basic argument in 1991 against an abrupt acceleration in the substitution process was based on the following: (1) in 1979 the industry voluntarily initiated a gradual substitution of production lines that utilized lead solder; (2) the cost for substituting each line was too high to warrant wholesale, rapid substitution; (3) until research proved that inward migration of lead from the external soldering of tin cans was contributing to the amount of lead in canned foods, accelerated action was premature; and (4) the recommended threshold for lead in foods was still subject to debate.

To support industry's argument, we noted U.S. FDA reports that indicated similar voluntary changes occurring at the same pace in the U.S. metallic container industry. According to these reports, in the early 1980s, 90 percent of tin cans produced in the United States had leaded solder; by the 1990s, the proportion had dropped to approximately 4 percent and was expected to decrease even more in the coming years. Clearly, industry in two major markets was voluntarily making changes, and the demand that the Mexican sector suddenly accelerate its rate of conversion—at considerable additional cost—was unfair.

The FDA , to my knowledge, has not yet been able to establish legislation totally prohibiting the use of lead soldering in tin cans for food. In 1994, for

example, 139,000 million tin cans were produced in the United States; of these, 20,000 million were three-piece tin cans used for food. Assuming that only 2 percent of these had lead soldering means that 400 million tin cans would have still contained lead soldering. It is our industry's belief that, in the United States, voluntary action by industry will eliminate lead soldering in the near future without the need for legislation.

Returning to the story in Mexico, none of the arguments listed above swayed the necessary Mexican decisionmakers. Health authorities, pressured by ecologists, continued to demand immediate conversion to electrical soldering. This activity culminated in the 5 July 1991 signing of the “Actions for the Integral Solution of the Problems Related to Lead Content in Products that Could Constitute a Risk for Health and Ecosystems” in Mexico City. The National Chamber for Metallic Containers agreed in this document to accelerate the transformation of all of its production lines in order to eliminate lead from the soldering process. We were given a very short time to complete the necessary changes. This posed special problems—for example, the machinery for producing electrical soldering was manufactured in Europe and required 10 to 12 months for delivery. Our organization was also required to close 15 new production lines. Recognizing the difficulties of implementing wholescale change in a short time frame, the Mexican government generously granted our industry 18 months to fulfill our task.

At that time, as an active member of the “Normalization of the Metallic Food and Drink Containers Subcommittee, ” I began work with colleagues on the elaboration of a preliminary program to regulate the types of solder that would be permitted on tin cans for food. On 2 June 1993, a text of the program was made available and accepted by both the private and public sectors. The health secretary in the Office of the General Management of Environmental Health visited our plants to verify that lead soldering had effectively been eliminated from the soldering process.

Finally, on 11 November 1993, the report of the subcommittee was published in the Diario Oficial (Mexico's largest newspaper). According to the law, 90 days were given to interested parties to express comments and concerns about the proposed project. On 8 February 1994, the secretary of health held a meeting to analyze the commentaries given. Revisions were made, and the final text revised. On 14 November 1994, the Norma Oficial Mexicana (Official Mexican Norm) NOM-002-SSAT-1993 “Metallic Containers for Food and Beverages: Seal Specifications and Sanitary Requirements,” was published in the Diario Oficial . The norm prohibits the use of

lead soldering in metallic containers for food. The norm is also applicable to imported products, serving as a way to prohibit lead-soldered cans from entering the country. Helped by a loan of $35 million, our industry had fulfilled its commitment to change to lead-free soldering in the 18 months accorded us.

One aspect of Mexico's experience that was not given the attention it deserved was the unfilled need for assistance to workers who had been poisoned through occupational exposure to the lead soldering process. On many of our old production lines, the containers of melted solder sat open, generating lead fumes. Systems of absorption and filtration existed for these emissions, but were rarely adequate. In addition, masks usually given to workers were frequently not used because the workers complained of hindered breathing. As a result, cases of lead poisoning were common, and many of these workers still require medical attention.

Our organization also believes that food containing lead before processing and canning should be given careful attention. Tuna, for example, because of its metabolism, has a tendency to absorb heavy metals such as lead, cadmium, mercury, and arsenic. The National Institute of the Consumer ( INCO ) of Mexico on one occasion reported large amounts of canned tuna containing lead above the threshold limit established at that time. The cans in which the tuna was stored were of the two-piece variety that did not contain solder of any kind.

R OBERT A. S CALA *

This paper will trace a successful story of government intervention to reduce the risk of airborne lead intake and possible lead poisoning. The story is told from the perspective of the U.S. Environmental Protection Agency ( EPA ), the main government agency involved.

Lead had been used as an additive in gasoline from the 1920s to boost octane and to provide lubrication for certain engine parts. Tetraethyl lead and tetramethyl lead, both high in octane value, lubricate intake and exhaust valves and help to reduce engine knock (EIA, 1992). Over time the use of additives became increasingly widespread, and the amount in fuel increased as octane demand increased. Although the limit for lead in gasoline was approximately 4 grams a gallon, a usage level of 2.5 g/gal was more typical. Because of increasing health and environmental concerns over atmospheric lead, lead reduction began with the 1970 Clean Air Act, which authorized restrictions on the use of lead in gasoline. In 1970 there were almost 90 million passenger cars registered in the United States, and motor gasoline consumption was 5.78 million barrels daily (approximately 915 million liters daily). Under the Clean Air Act, the U.S. EPA was authorized to regulate fuels and fuel additives. Unleaded fuels also appeared in the early 1970s. Auto manufacturers were required to design and build vehicles that could operate on unleaded fuels or low-lead fuels, and a schedule was set for the reduction of lead levels in leaded fuels.

Before the Clean Air Act, the EPA had strong concerns about the potentially harmful effects of lead, but was unable to persuade the scientific community or industry that airborne lead represented a health hazard outside the workplace. Under the Clean Air Act, the EPA was given the authority to control airborne lead attributable to motor vehicle emissions for reasons beyond potential health hazards. A 1980 National Research Council publication outlined a model for regulatory decisionmaking re-

garding potential health hazards of environmental agents (NRC, 1980). There were nine steps:

Identify sources of lead and pathways of environmental transfer;

Identify specific human populations with exposure to lead;

Estimate the level of exposure to lead by each environmental pathway for each specific population;

Establish the association between exposure to lead and the level of lead in the body for each specific population;

Establish the association between the level of lead in the body and biological change caused by lead for each specific population;

Estimate the upper limit of nondetrimental biological change for each specific population and the level of lead in the body associated with that degree of biological change;

Identify and describe alternative control strategies;

Apply risk-benefit, cost-benefit, and other considerations, compare alternatives for control, and decide what is an acceptable level of lead in the environment for each specific population; and

Evaluate the process and the decision.

Of greatest interest with respect to motor gasoline is its contribution to airborne lead. The amount of lead in the air appears to be related in large measure to the amount of lead in fuel. Leaded fuels generated 24,000 µg/m3 of lead at the tail pipe in the era before lead phasedown (NRC, 1993). Typical lead levels in urban environments in the 1970s were in the range of 0.5 to 10 µg/m3, and perhaps 90 percent of this is attributable to lead from gasoline. Most lead is emitted as halides and oxides, but virtually all of it is eventually converted to the sulfate. Most of the lead is deposited near the vehicular source. Particles with diameters in the range of 10 µm are deposited over a broad distance, and there is long-range transport of particles with a diameter of less than 0.1 µm for over a month (NRC, 1993). Lead is widely distributed in the body, with a preferential uptake by bone.

From the outset, EPA held that leaded gasoline was a source of air and dust lead that could be reduced readily and significantly in comparison with other sources. It also held that young children in the age range of 1 to 5

years should be regarded as a group sensitive to lead exposure (EPA, 1978). The third link in the EPA chain of logic is that contaminated dust and dirt from motor vehicle exhausts are the most important exposure routes for children (EPA, 1973). The health status of children was the principal driving force for the regulation of the lead content of fuels. As late as 1984, about 6 million children and 400,000 fetuses in the United States were exposed to lead at concentrations that placed them at risk of adverse health effects, defined as blood lead levels of at least 10 µg/dl (NRC, 1993).

The EPA put great emphasis on the work of Azar et al. (1975), which showed a ratio of lead in the air to lead in the blood of 1:1.8 at airborne levels of 1.5 µg lead/m3, where lead in blood was expressed in the usual terms of µg/dl (micrograms lead per deciliter of blood). The work was in adults, not children. the EPA holds that children have a greater net absorption and retention of lead than adults. The agency assumes that the air-lead to blood-lead relationship for this sensitive population exposed to lead in ambient air equals or exceeds the relationship for adults. The literature also suggests (ACGIH, 1991) that this relationship is nonlinear with concentration in air. Yankel et al. (1977), using children living near a smelter, found an average ratio of 1:1.9. Some of the nonlinearity in the relationship is explained by particle size changes with concentration, 24-hour vs. 8-hour exposures, and certain kinds of avoidance behavior by workers.

Preceding presentations have discussed the issues centering on clinical and subclinical effects of various body burdens of lead, expressed either in blood lead levels or tissue concentrations. the EPA emphasized minimizing lead burden. The agency position was that air lead contributed to general population lead exposure and that airborne lead levels below 2 µg/m3 affect blood lead levels. With the promulgation of a National Ambient Air Quality Standard for Lead in 1978 (EPA, 1978), the EPA stated that for the sensitive population previously defined (children ages 1–5 years), a blood lead level above 30 µg/dl was associated with an impairment in heme synthesis in cells, as indicated by an elevation in erythrocyte protoporphyrin. This finding was judged by the EPA to be adverse to the health of chronically exposed children. the EPA also declared that there were a number of other adverse health effects associated with blood lead

levels above 30 µg/dl in children, as well as in the general population. Three systems appear to be most sensitive to the effects of lead: the hematopoietic system, the nervous system, and the renal system. Inhibition of enzymes systems has a threshold as low as 10 µg/dl; at the other end of the scale, permanent, serious neurological damage or death have thresholds approaching or exceeding 80 to 100 µg/dl in children.

The EPA acknowledged that the lead exposure problem arose from a combination of sources, including food, water, air, leaded paint, and dust. The contribution of each source varies depending on the environment, bioavailability, and individual exposure and uptake. Reducing lead from all sources would improve health; the EPA , however, viewed lead in gasoline as a source of air and dust lead that could be readily and significantly reduced.

The first steps taken to reduce lead in gasoline in the United States resulted from the introduction of the platinum-based catalytic converter. This device, which reduced emissions of polycyclic aromatic hydrocarbons and other pollutants from gasoline, was damaged by lead. Starting in 1973, it was necessary for EPA to ensure a supply of lead-free fuels for new cars equipped with catalytic converters. In 1978, EPA moved to evaluate the public health benefits of removing lead from gasoline in order to reduce lead in the ambient environment. The initial regulations set quarterly limits on allowable amounts of lead used by refiners and permitted averaging this amount across all grades of gasoline produced. It was assumed that the natural replacement of older vehicles with cars requiring unleaded fuels would result in the programmed reduction of lead in gasoline, without further intervention by government. By 1982, however, two trends were recognized that prevented this natural reduction: first, a substantial number of motorists continued to use leaded gasoline because of its lower pump price (misfueling); and second, since the allowable content of gasoline was defined as the average of all fuels produced, the per gallon content of leaded gasoline could actually increase as the fraction of leaded/unleaded decreased. Because of these two phenomena, the EPA took further regulatory action to ensure the phasedown of lead in gasoline in 1983–1985. Simply by changing the basis of calculating allowable levels of lead in gasoline from the total fuel base to leaded gasoline only, EPA caused a substantial decrease in the amount of lead used in gasoline. In 1990, the U.S. Congress mandated the eventual phaseout of lead in gasoline in the United States.

There were some environmental consequences of the motor gasoline lead phasedown. One was misfueling, or using leaded fuel in a vehicle designed to use unleaded fuel. Misfueling was found in about 6 percent of vehicles in the inspection and maintenance programs.

Of interest is how much lead use was reduced by the phasedown of leaded fuels. With the introduction of the catalytic converter in 1973, the next 10 years showed an increase in unleaded gasoline use. By 1983, unleaded fuel was just over 50 percent of the total gasoline market in the United States, and by 1995 it represented almost the entire market. the EPA estimated a 34 percent reduction in lead use over the interval 1983 –1990, with a savings of almost 130,000 tons of lead (EPA, 1982c). Natural Resources Defense Council figures estimate more comprehensible figures. Lead used in motor gasoline in the United States totaled 243 thousand metric tons in 1971; 309 thousand metric tons in 1976; and 138 thousand metric tons in 1981. When the phasedown really took effect, lead use was reduced to 7.1 thousand metric tons in 1986 and 4.4 thousand in 1992 (NRDC, n.d.).

Companies that sold lead additives for gasoline were faced with the prospect of seeing their business virtually vanish over a period of just a few years. They were in the position of buggy whip makers at the beginning of the automotive era. The prudent companies diversified. There was also some reasonable resistance on the part of auto manufacturers. Lead in gasoline not only provided octane value but was also a lubricant for engine parts. The presence of lead prevented valve seats from being “beaten in” because the relatively soft lead deposits provided a form of cushion. Work on metallurgy had to be done as part of the lead phaseout. In the end, unleaded and low-lead gasolines were widely available on schedule.

The EPA examined the costs of lead phasedown in terms of reduced lead use (EPA, 1984). The agency estimated that the marginal cost of removing lead from gasoline was about U.S. 1¢ for each gram of lead. EPA also found that lead removal presented a positive cost-benefit. The benefit fell into three

categories: savings on vehicle maintenance, reduced misfueling, and reduced health care costs. For the period 1986–1988, the projected 0.1 gram per gallon for leaded gasoline would produce annual benefits over costs of greater than $1 billion ($1,000 million).

The findings of the National Health and Nutrition Examination Surveys II ( NHANES II ), which covered the period of 1976–1980 (NRC, 1993), used a stratified multistage probability cluster sample of U.S. households. Blood lead levels were measured for persons aged 6 months to 74 years, and almost 10,000 samples were used. As Figure 3-8 depicts, there was a striking

Figure 3-8. Lead used in gasoline production and average NHANES II blood lead (February 1976-February 1980). Source: Reprinted, with permission, from J.L Annest, “Trends in the bood lead levels of the U.S. population: The Second National Health and Nutrition Examination Survey ( NHANES II ) 1976-1980,” in Lead Versus Health, M. Rutter and R.R. Jones, eds., New York: John Wiley & Sons. ©1983, John Wiley & Sons, Ltd.

association between declining lead levels in gasoline in the period of 1976-1980 and the subsequent decline in blood lead levels in the U.S. population. Although the choice of scale for each of the vertical axes in Figure 3-8 permitted the dramatic overlay of the declines in amount of lead in gasoline and blood lead levels, one is hardpressed to deny that the association is not a correlation. For NHANES III (Pirkle et al., 1994), the survey covered the period 1988-1991, with over 12,000 blood samples from a population whose ages ranged upward from one year. The mean blood lead level of persons aged 1-74 years decreased 78 percent, from 12.8 to 2.8 µg/dl, a drop of 10 µg/dl. Comparable percentage decreases were seen for non-Hispanic white and non-Hispanic black children aged 1 to 5 years. There was a decrease in the prevalence of blood lead levels equal to or greater than 10 µg/dl, from 85 percent to 5.5 percent for non-Hispanic white children and from 97.7 percent to 20.6 percent for non-Hispanic black children. The authors attributed the majority of this decrease in blood lead levels to the virtual removal of lead from gasoline and the reduction of lead in soldered cans (Pirkle et al., 1994).

Government intervention in the form of control regulations can be successful in reducing the body burden of lead in the population as measured by blood lead levels. This reduction in burden surely represents a comparable reduction in the risk of lead-related diseases and dysfunctions.

J OHN D. R EPKO *

The United Brotherhood of Carpenters, with about 550,000 members in North America, has members who work in all aspects of the trade, including general construction, renovation, and repair, as well as working on bridges and other steel structures. Our members work in buildings and on steel structures known to contain lead-based paint. Piledrivers are frequently exposed to lead-based paint when applying acetylene torches to bridges undergoing renovation.

Lead intoxication is a continuum, in which the adverse effects are expressed at the cellular, organ, or whole organism level, depending on the dose (Silbergeld et al., 1991). There is no magic level; the toxic effects of lead are evident when increased absorption begins.

In addition to working in an occupation that exposes a worker to lead, the worker may come to the job with exposures to lead from the home or living environment. Whatever standards are ultimately established for workers, they must provide for this margin of prior exposure.

“Take-home” lead is also a concern. Recently the National Institute for Occupational Safety and Health ( NIOSH ) (Sussell, in press) reported 62 incidents worldwide of paraoccupational, or “take-home” lead exposure. Industries with reported paraoccupational lead exposures included lead smelting, battery manufacturing and recycling, radiator repair, electrical components manufacturing, pottery and ceramic production, and stained glass making.

The Carpenters' general approach to intervention is very much proactive. Generally, our approaches fall into four categories: (1) education and training, (2) collaboration with scientists involved in occupational health research, (3) support for occupational and environmental health standards and regulations, and (4) what we call, “dealing an honest hand.”

Health and safety education for workers about the occupational dangers of lead exposure is an important part of both apprenticeship education and the ongoing training of the journeyman. The U.S. Society for Occupational and Environmental Health, for example, developed a document that was used extensively by the EPA in the development of regulations governing required curriculums for lead-based paint worker and supervisor training (SOEH, 1993). This document and the training programs it has engendered have been effective among our membership in preventing work-related diseases and injuries, keeping workers up-to-date, and helping workers develop an attitude and willingness to practice prevention.

The Carpenters have collaborated on many research activities designed to identify health problems arising from workplace exposures to lead. In 1991, Dr. Irwin Selikoff, who headed Mount Sinai Environmental Sciences Laboratory, in collaboration with NIOSH and Harvard Medical School, conducted the first membershipwide screening of Carpenter workers. A subgroup underwent an even more detailed study, including questionnaires on lead exposure history, blood lead testing, and in vivo measurement of bone lead levels. The study found that age was the dominant predictor of both tibia and patella bone lead. Demolition, carpet laying, and alcohol ingestion were also significant predictors of bone lead (Watanabe et al., 1994). The authors concluded that the data reflect a subclinical effect of bone stores of lead on hematopoiesis and that this effect is the first epidemiologic evidence that bone lead may be an important biologic marker of ongoing chronic toxicity. The researchers also found that the differences in concentrations of bone lead between the tibia and patella are suggestive of ALAD -2-associated pharmacokientic effects. Further, they suggest that subclinical lead-associated kidney dysfunction is found with relatively low

current blood lead concentrations, and that the ALAD -2 genotype may be an additional modifier of this effect (Smith et al., 1995). The Carpenters have also worked with Dr. Selikoff's successors, a network of devoted scientists, by promoting continued study of the membership.

These studies demonstrate that information useful to workers' health can be obtained when workers and unions collaborate with their scientific and academic colleagues. By providing additional evidence of lead's toxicity at very low blood and bone levels, such studies can promote changes in occupational standards that better protect the health of workers.

Numerous standards have been set for lead in both the occupational and nonoccupational setting. Some standards set limits on allowable concentrations of lead in the ambient air or workplace air, as well as other environmental compartments, and some standards are based upon biological monitoring. In the workplace, both airborne lead and biomarkers are components of preventing lead poisoning in most national systems.

The U.S. occupational lead standard was promulgated in 1978, one of the first de novo standards developed by the Occupational Safety and Health Administration ( OSHA ), without reliance on earlier guidelines proposed by the American Council of Government and Industrial Hygienists ( ACGIH ). the OSHA lead standard was noteworthy in that it proposed both a limit on airborne lead in the workplace (50 microgram/m 3 ) and a mandatory program of biological monitoring in most work settings. In addition, the OSHA standard protected worker health and employment rights by establishing a medical removal program: workers whose blood lead levels exceeded the standards were temporarily shifted to jobs without lead exposure with no loss of pay, benefits, or seniority. This approach was intended to change the incentives in the labor management relationship, to encourage employers to reduce lead exposures, and to protect workers from job termination or loss of income.

In the United States, occupational standards cover most, but not all, workers exposed to lead. In 1993 the lead standard was finally extended to workers in the construction industry, who are often highly exposed to lead during repair and maintenance of steel structures (which may still be painted with lead-based paints); demolition workers; and workers involved in abating lead hazards in housing. Small workplaces are still imperfectly covered, and some of these, such as battery repair shops, may be sources of intense exposure that not only pose problems for workers, but

can also be sources of contamination to workers' homes and releases to the environment (Matte et al, 1989b; Nunez et al, 1993). In several countries of the Americas, informal “cottage industries,” such as recycling facilities, are almost wholly outside any regulatory surveillance, as demonstrated in a report from Tijuana, Mexico (Leung, 1988).

Historically, organized labor and the lead industries have not had an easy relationship. A number of incidents illustrate their lack of cooperation. The events surrounding the American Smelting and Refining Company ( ASARCO ) lead smelter emissions in El Paso, Texas, in the late 1960s (Repko et al., 1978) and the Bunker Hill smelter in Kellog, Idaho, in the 1970s (EDF, 1992) are two such examples. The lead industry also consistently attacks studies that demonstrate a causal relationship between lead exposure and adverse health effects in order to counter this increasing body of evidence. Another industry approach is to claim simply that lead is not a problem and that nothing, therefore, needs to be done about limiting exposures to the metal.

As a result, labor has had to consistently challenge the positions taken by the lead industry. NIOSH 's legal right of entry, for example, has been used to conduct health hazard evaluations in the workplace. Because of the active involvement and support of labor, however, the lead industries have never won a major victory over the EPA or OSHA , the government organization most responsible for worker health.

In response, there is a current effort by industry to shift its worksites into international markets where regulation is less restrictive or does not exist. Moreover, environmental advocacy groups or scientific communities may also be absent or small in a number of these areas. Workers all over the world are increasingly under attack by corporate and financial interests who use the globalization of products such as lead as a lever to restrict worker rights and to lower workplace standards. It is the responsibility of labor to fight against this trend.

Workers continue to be poisoned by lead on the job. Members of labor groups or health professionals can help reduce worker exposure in several ways:

Worker education is the first step to prevention. Workers have the right to know. An informed workforce increases its control over its exposure. Inspectors and government officials responsible for workplace health and safety must also be well-trained and informed about lead hazards.

International and national protective standards for workers should be widely adopted across the Americas, but these should not necessarily copy those of the United States. Labor and workers should have an opportunity to establish stringent standards for lead exposure consistent with new scientific evidence that was not available or not used when the United States first set its standards.

Existing environmental or workplace standards that limit workers' exposure to lead must be enforced. To adequately enforce regulations will require a coordinated effort by an educated workforce, competent inspectors, an effective judicial system, and policymakers who can support the courts and the inspectors.

Productive collaborations among unions, scientists, occupational health professionals, and academics should be fostered.

Union officials should be informed of the importance of occupational health and encouraged to raise the level of workers' health to the same level of concern in contract negotiations that is routinely given to economic issues.

Finally, persist, and don't give up. Whenever new ways of making a dollar, peso, real, other currency involve new hazards, those who work to protect worker health are playing “catch up.” Remember that it is always the workers who pay first and worst.

C HRISTINA VON G LASCOE , M.D., P H .D. *

If people in the developing world are to have radically improved lives, it is first of all necessary to teach them to be dissatisfied with the present situation and at the same time make them appreciate how they can work towards a better future.

Bidwell, 1988

Environmental health issues are of recent concern at the northern border of Mexico, but to date have not been addressed in a concerted fashion by either community interest groups or public health authorities. For example, public health authorities and two major nongovernmental organizations ( NGO s) working in primary care and family planning in Tijuana, Mexico, have yet to incorporate environmentally driven concerns such as lead poisoning into their regular health activities. Nevertheless, there are indications that lead poisoning and other environmental health concerns are gaining visibility at the local level in Mexico, as they are elsewhere in the Americas.

A major requirement in creating the necessary infraculture for community-based action is the need to increase popular knowledge and public awareness of environmental health hazards and the steps that can and should be taken to reduce these hazards. An example of such work is provided in this paper. Ways to incorporate such knowledge into the cultural horizon of a community under environmental health stress are highlighted.

Anthropologists interested in health recognize that community members practice a wide range of activities related to public health, with or without the direction of health authorities. In any community there coexist interpretations of an official and a traditional system of dealing with public health threats, and these do not necessarily influence each other. In order to deal

effectively on a grassroots level, it is necessary as a first step to define the “cultural horizon” of the community. Broadly defined, the cultural horizon is the wide range of public health activities practiced by community members, with or without the direction of the health authorities. Thus, the development of effective community strategies to reduce or prevent lead poisoning will depend not only on information on the sources of contamination and number and distribution of cases identified, but also on being able to convey that information in a manner that is reflective of the community's cultural horizon. That horizon can be defined by a number of factors, including:

the level of personal behavior engaged in health promotion (individual, family, and community/public);

the level of interaction and cooperation with local health authorities in matters of personal and public health.

Responsibility for health is transferred to individual households through the empowerment of individuals to act in ways consonant with that responsibility. Assumptions underlying this transition are:

People will act responsibly on their own behalf given sufficient information, resources, and interest, and in the context of a particular cultural horizon that sanctions this behavior;

Information can consist of information and resources, but interest must be stimulated from within the community. Information presentation in and of itself does not guarantee the “success” of community-based programs;

Health professionals need to determine what the focus of prevention activities should be;

The potential threat of a hazardous substance must be made visible to the community so that it may be communicated in ways that can be understood by community members. In transmitting this information, it is imperative to impart knowledge on specific steps that can and should be taken by community members to reduce their exposure, and that of their families, to the hazardous substance in question. Allowances should be made for community-driven modifications in these steps deriving from the culture or other values of the inhabitants.

Underlying these four assumptions is the recognition that in the past, the community has often been the missing link in addressing environmental and occupational health problems at the local level. The community has often been excluded because members were not considered knowledgeable enough to participate in problem solving. While this paternalistic belief continues to be held by many traditional health professionals, particularly in less-developed countries or in inner-city or rural regions of the more developed countries of the Americas, there is a growing movement toward including community members and other “stakeholders” in the development and implementation of prevention and control programs. Growing experience indicates that such individuals can be vital to the development of innovative and sustainable solutions. Thus, community empowerment should remain a cornerstone of prevention and control efforts.

The best community interventions expand the cultural health horizon of a community by adding culturally relevant alternatives to standard or “textbook” approaches to prevention. In order to do this, the goals and methods of agencies must be aligned with the goals and methods of the community; in other words, they must be made relevant to the community members and vice-versa. How can this be done?

The first step is to appraise the “health horizon" of the community—for example, by determining, in the case of lead, the level of personal behavior directed toward reducing industrial and family exposures. The current degree of information exchanged about the sources and nature of lead poisoning in the community and the interaction between the public health authorities and community members on health issues important to the community also need to be assessed.

As a general rule, we have observed that where the health authorities have a strong bureaucracy, community leadership is suppressed, and health professionals manage the public's health; conversely, where the public health bureaucracy is weak, alternative forms of representation naturally arise. For example, in underserved areas, health authorities are often helped by local grassroots organizations and other NGO s. As a consequence, some of the most innovative experimentation in community education and empowerment is occurring in the most economically disadvantaged populations or in rural areas. This experimentation should be carefully studied to determine the circumstances that foster increased

community autonomy and decisionmaking power that translates into improved community health and well-being.

To what extent can interested volunteers support and sustain the health of their communities in the face of an increasingly toxic environment? As a first step, there have to be sufficient health personnel, laboratories, and budget to respond to the threat of exposure to hazardous substances. Mexico provides an example where the direct participation of citizens is a real resource. In the small rural towns of Sonora, for example, there are voluntary groups called health committees. These health committees, which are organized by local health authorities but include community membership, have general oversight in matters pertaining to the running of the local health centers and their prevention programs. The committees, in general, have significant political clout with the regional health administrators; thus they can often exert sufficient pressure to obtain needed personnel and services.

In many of the northern border cities of Mexico, NGO s also depend heavily on community volunteers. Although few are currently addressing the issue of lead poisoning in the region, NGO s provide a potentially useful mechanism for dealing with environmental hazards in a manner that can be useful to communities. the NGO s are often staffed by volunteers who act on behalf of their own and their neighbors' best interests. The question then becomes one of how to motivate community members to participate.

Lead toxicity is a major problem in Mexico, as it is in every other country of the Americas. Whole neighborhoods in Tijuana, for example, are exposed to lead poisoning, and this has resulted in significantly lower IQ levels in grade school children living near the lead emission point sources (Guzmán, 1994). Of the children surveyed by Guzmán, 80 percent had blood lead levels that have been associated in cognitive studies with a seven-point performance IQ deficit, and the average blood lead of residents is higher than that associated with severe symptoms of lead poisoning.

To date there has been no consistent public health response to the crisis of lead poisoning in Tijuana, nor has any effective community action been undertaken, in a large part because of a lack of community awareness about the hazards. For many in Tijuana, lead is invisible; for others, who may be aware of the hazards of lead, there is little knowledge about common routes

of exposure, for example, lead-glazed ceramicware or local remedies for such maladies as stomach disorders that contain lead (Baer et al., 1989).

Our study of community perceptions of lead as a health risk in Tijuana produced seven distinct and contrasting perspectives (see Box 3-1 ). These seven views demonstrate increasing knowledge of the dangers, and the nature and sources of community lead exposure.

The preferred approach to developing strategies to reduce lead exposures is one that utilizes and is based on the capacities, skills, and assets of community members. Much of past public health experience demonstrates that significant advances in community health take place only when local community mentors are committed to investing themselves and their resources in the effort. Depending on its size and sophistication, a given community may have local institutions (such as businesses, schools, parks, libraries, hospitals, and colleges), citizens' associations (including churches, block clubs, and cultural groups), and gifted individuals (for example, artists, retired people, and young people) whose skills and knowledge can be used in the development of effective education and intervention.

Specific steps for accomplishing these ends include:

Discovering the existence of active and trusted organizations or other resources within a community. These may include formal organizations (churches, government offices, and the like); informal organizations and networks (such as family and friends); or capacities and assets of individuals, citizens' associations, and local institutions;

Undertaking a community-led needs assessment that will allow residents to identify problems and priorities that are most important to them; this step should involve the building of productive relationships among local and state health networks who share the common goal of preventing lead poisoning; and

Identifying resource needs and the mechanisms to meet them.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Downloadable Content

Government Intervention and Free Markets: Case Studies in Voter Opinion

• Student Research
• Rhodes, Robert T.
• Guerrero, Mario
• Letters, Arts, & Social Sciences
• Political Science
• California State Polytechnic University, Pomona
• political science
• free market
• 2015-04-21T18:18:21Z
• http://hdl.handle.net/10211.3/138063
• All rights reserved.

Relationships

• 2015 SRC Oral Presentation Sessions
• Dr. Mario Guerrero

Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.

Government Intervention in Markets

Governments intervene in markets to try and overcome market failure. The government may also seek to improve the distribution of resources (greater equality). The aims of government intervention in markets include

• Stabilise prices
• Provide producers/farmers with a minimum income
• To avoid excessive prices for goods with important social welfare
• Discourage demerit goods/encourage merit good

Forms of government intervention in markets

• Minimum prices

Maximum prices

• Minimum wages
• Nudges/Behavioural unit

Minimum Prices

This involves the government setting a lower limit for prices, e.g. the price of potatoes could not fall below 13p.

The minimum price could be set for a few reasons:

• Increase farmers incomes
• Increase wages
• Make demerit goods more expensive. For example, a minimum price for alcohol has been proposed.

Diagram Minimum Price

A minimum price will lead to a surplus (Q3 – Q1). Therefore the government will need to buy the surplus and store it. Alternatively, it may impose quotas on farmers to decrease the quantity of the good put onto the market.

Problems of minimum prices

• It could be costly for the government to buy the surplus
• A minimum price guarantee acts as an incentive for farmers to try and increase supply. As an unintended consequence, the minimum price encourages more supply than expected and the cost for the government rises. This happened with the EEC Common Agricultural Policy.
• To ensure minimum prices, the government may have to put tariffs on cheap imports – which damages the welfare of farmers in other countries.

Maximum Price

This involves putting a limit on any increase in price e.g. the price of housing rents cannot be higher than £300 per month.

Maximum prices may be appropriate in markets where

• Suppliers have monopoly power and are able to generate substantial economic rent by charging high prices
• The good is socially important – e.g. good quality housing is important to labour productivity and a nations’ health.
• Demand is price inelastic because the good is necessary for maintaining minimum standards of living.

Diagram Maximum Prices

The Maximum price will be set below the equilibrium. This makes sure the price is less than the market clearing price.

• However, the problem of a maximum price is that there will be a shortage. At Max Price, Demand is greater than supply. (Qe-Q1) This leads to queues and consumers unable to buy.
• This will encourage the operation of black markets.
• Therefore the government will have to ration the goods or increase supply

If supply and demand are very inelastic, then a maximum price may have little adverse impact on creating shortages. For example, if supply housing for rent is very profitable, then a maximum price will not stop landlords putting the house on the market.

• Buffer Stocks

Agriculture suffers from various problems. These include:

• Fluctuating Prices
• Uncertainty leads to lack of income
• Low-Income elasticity of demand
• Positive Externalities of Farming

Therefore the government may feel there is a case to intervene and stabilise prices. A buffer stock involve a combination of minimum and maximum prices. The idea is to keep prices within a target price band.

This is a different kind of government intervention. It is a government policy to influence demand indirectly. For example, putting cigarettes behind closed covers – makes it harder or less enticing for people to buy.

The government may also place flashing speed limit signs to give a smiley face to drivers under the speed limit, but an unhappy face to drivers exceeding the speed limit.

See: nudges

Tax is a method to discourage consumption of certain goods. For example, taxes on demerit goods – goods with negative externalities. Taxes both discourage consumption and raise revenue for the government.

In the above example, the tax moves output to Q2

Problems of tax

• Demand may be inelastic
• Hard for the government to know external cost and how much to tax
• May encourage tax evasion – e.g. rubbish tax can encourage fly-tipping

The government may subsidise goods with positive externalities (for example, public transport or education).

In the above example, a subsidy shifts output to 120 (where SMB = SMC) so it is more socially efficient.

Problems of subsidies

• Cost to government
• Subsidies may encourage firms to be inefficient because they can rely on government aid.
• Government intervention in the labour market

Cookies on GOV.UK

We use some essential cookies to make this website work.

We’d like to set additional cookies to understand how you use GOV.UK, remember your settings and improve government services.

We also use cookies set by other sites to help us deliver content from their services.

You have accepted additional cookies. You can change your cookie settings at any time.

You have rejected additional cookies. You can change your cookie settings at any time.

• Education, training and skills
• Further and higher education, skills and vocational training
• Sector-based Work Academy Programme: qualitative case study research
• Department for Work & Pensions

SWAP Qualitative Case Study Research: Annexes

Updated 16 May 2024

© Crown copyright 2024

This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: [email protected] .

Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.

This publication is available at https://www.gov.uk/government/publications/sector-based-work-academy-programme-qualitative-case-study-research/swap-qualitative-case-study-research-annexes

Annex 1: SWAP Theory of Change Logic Model

The flow diagram presents the following information:.

Inputs mainly under the heading ‘Government (Continued benefits, training costs, barriers to participation, analytical resources)’:

‘Employer -Time for set up/management/ feedback’

‘Training Provider - Time for set up/management’

‘Employer Advisor’, ‘Work Coach’

These inputs flow into the following activities:

• ‘Local Labour market analysis’
• ‘Arrange SWAP ’
• ‘Disseminate available SWAP with JCP ’
• ‘Awareness of available SWAP ’
• ‘Sell SWAP to claimant’
• ‘Refer claimant’

These activities flow to the output: ‘Claimant agrees to participate in SWAP ’. The claimant agreement flows on to the claimant input: ‘Investment of time and initial travel costs (reimbursable)’ and the following work coach activities:

• ‘Barrier Assessment’
• ‘Record referral’

Then the following employer activities:

• ‘Background checks (if applicable)’
• ‘Workplace adjustment (if applicable)’ 

These all flow into the output ‘Claimant starts SWAP ’.

The main SWAP portion of the theory of change starts here with the activities:

• ‘Pre-employment Training’ (with side flow to ‘Claimant gains certification (if applicable)’)
• ‘Work placement’
• ‘Guaranteed Interview’

The main SWAP activities flow to the following three possible outputs:

• ‘Claimant completes SWAP : Interview successful’
• ‘Claimant completes SWAP : Interview unsuccessful’
• ‘Claimant does not complete SWAP ’)

Regardless of which output, the diagram shows a flow to the activity ‘WC completes SWAP Tracker’. 

If Claimant completes SWAP with an unsuccessful interview, or the Claimant does not complete the SWAP , these outputs flow into the short term outcome of ‘additional needs identified’. The unsuccessful interview also flows into the short term outcome ‘interview feedback from employer’.

Both of these short term outcomes flow into the activity ‘Reflection with WC’ and this can flow into the activity ‘Claimant applies for other jobs in sector’. Claimant applies for other jobs in sector can flow into a successful short term outcome ‘claimant gains employment in new sector’ or an unsuccessful short term outcome, in which case ‘claimant reengages with WC’. 

Claimants unsuccessful at SWAP or subsequent interviews (through engagement with WC activities) flow into either:

• the short-term outcome ‘Increased employability’
• the medium term outcome ‘improve claimants employability’.

Collectively the short and medium term outcomes flow into the impact ‘Value for unsuccessful claimants’. 

If the claimant is successful at either the guaranteed or subsequent interview, these flow into the output ‘ Claimant enters work in new sector’ which flows onto the following four short-term outcomes:

• ‘Change in attitude towards working in new sector’
• ‘Claimant has skills to succeed at new job’
• ‘Increased earning’
• ‘Reduced UC /benefit’

This may flow into the following medium term outcomes:

• ‘ UC ends (or is maintained at reduced level)’
• ‘Sustained employment (18 months)’
• ‘Career progression’

These medium term outcomes flow into the impacts:

• ‘Increased employment’
• ‘Reduced UC costs’

If the claimant is successful at the guaranteed interview, and subject to the assumption ‘Employer outputs and outcomes are dependent on the SWAP meeting employer expectations’, the following employer outputs are recorded:

• ‘Reduced vacancies’
• ‘Employer social responsibility goals met’
• ‘Development of local workforce’ 

‘Reduced vacancies’ and ‘Employer social responsibility goals met’ flow to the short-term outcome ‘Employer satisfied with SWAP experience’. This then flows to the activity ‘ DWP gathers feedback from claimants and employers’ and the short-term outcome ‘Collated employer success stories’.

These short-term outcomes flow to the medium term outcomes:

• ‘Businesses return for additional SWAP ’
• ‘Increased employer uptake in SWAP or other provisions’
• ‘Improved Attitudes towards hiring DWP claimants’
• ‘Employers approaching DWP with vacancies more readily’ 

‘Development of local workforce’ output flows to the following short-term outcomes:

• ‘ SWAP aligns with local market need including sector shortages’
• ‘Sector pathways identified’
• ‘Change in attitude towards working in new sector’, which flow to the medium term outcome ‘Improved fit between employers and claimants’. 

Medium term outcomes in this employer focussed part of the theory of change flow to the impacts ‘Improved DWP relationship with business sector’ and ‘increased employment’.

Annex 2: Participant characteristics

Table 3: employer, training provider and claimant participants by swap sector, table 4: claimant participant characteristics, annex 3: additional methodology details.

This annex includes additional information about how the case study research was conducted.

Contacting claimants

A random sample of 150 claimants was drawn for each case study area (600 claimants in total across the four areas) in order to achieve 10 claimant interviews in each district. This sample size was in line with previous, similar research (in terms of mode, length and recruitment approach), which achieved a response rate of approximately 1 in 15 claimants. The sample was sourced from the SWAP manual trackers completed by each district which detail which claimants are referred each week to the programme. Claimant identification numbers were then linked to centrally held contact information (for example, postal address and telephone number).

The stratification of the sample was limited by the quality of data DWP holds on certain claimant characteristics (for example, ethnicity and disability information was not available) as well as claimants’ SWAP journey (only claimant start dates on the pre-employment training ( PET ) were consistently recorded by all areas). As a result, it was impossible to identify in advance claimants who had dropped out of a SWAP part-way through, or claimants who were successful at the guaranteed interview stage, which limited the study’s ability to explore these aspects in detail. The sample drawn was, therefore, broadly reflective (rather than representative) of the claimant population who started on a SWAP in terms of gender and sector of SWAP , and consisted of individuals who had started the SWAP PET within the previous 12 weeks of the sample being drawn. This time period was agreed in order to ensure the feasibility of obtaining a sample of 150 claimants from each area, while minimising as much as possible the risk of recall bias within claimant accounts of their experience.

It is important to note the claimant sample was delivered in two separate stages, to reflect the gap in fieldwork between Area 2 and Area 3. The samples for Areas 3 and 4, was additionally stratified by age (18 to 24 years vs. 25+ years) to account for the small number of potential participants aged 18 to 24 years provided in the sample for Areas 1 and 2.

All claimants in the sample were sent an advance letter to the address held on DWP ’s central records. This letter provided further information about the research, what their participation would involve, data processing information and an email address to which they could write if they wanted to opt-out. Claimants were called using the telephone numbers provided in each sample, and while formal quotas for recruiting participants were not used, calls were targeted to achieve a spread in terms of claimant gender, age and sector of SWAP (the latter was obtained from the SWAP manual trackers and was therefore dependent on DWP staff interpretations of this at the local level). Claimants were called up to three times without a response before they were not contacted any further. During the calls, researchers emphasised their independence from benefits processing and that decisions regarding participation would not affect claimants’ benefits in any way. Each interview lasted approximately 30 to 45 minutes and claimants received a £20 voucher for their time.

In Area 4, fieldwork was terminated early due to an underlying issue with this sample in which few claimants could be contacted (many claimants did not pick up the phone) and of those who did, few recalled the programme or had actually started the SWAP to which they had been referred. Only two interviews were completed from 207 recruitment calls, compared to 10 interviews completed from 88 recruitment calls in Area 3. The study team attempted to unpick the reasoning for the issues with the underlying sample in subsequent meetings and interviews with the local operational contacts, however, it was difficult to pinpoint this exactly. The information gathered suggested that the issue was likely a result of error(s) completing the local manual SWAP trackers. As a result, fieldwork was terminated early so that the findings could be reported to the timetable agreed.

Contacting employers and training providers

As described in the main report, the study was reliant on the case study areas to supply the contact details of employers and training providers who had taken part in a SWAP in their districts, as there was no alternative way of identifying these organisations. Within each area, the study aimed to interview a total of 7 employers, and 3 training providers, and so local contacts were asked to provide approximately 15 to 20 employer contacts and 5 to 10 training provider contacts to account for uncertainty in likely response rates. Obtaining contacts was more difficult in some of the case study areas and was affected by factors such as local record keeping of this information (for example, some training providers were listed as employers, and other contact information was out of date), and busyness of the staff involved. In all areas, subsequent samples of employers were requested due to poor response rates for this participant group.

To counter the risk of staff supplying only contacts for similar organisations, and therefore similar experiences of the programme, contact information for a range of organisations in terms of key characteristics (size, sector of SWAP , length of SWAP , number of SWAPs involved in, and how the SWAP was initiated) was requested. Organisations were then approached by researchers to ensure a spread across these characteristics, although achieving this was limited by response rates, particularly among employers.

Organisations were initially emailed using a template which explained the purpose of the research and asked if they were able to participate. Where organisations agreed to take part, they were then sent an additional information sheet and booked in for an interview at a convenient date and time. Where no response was received, a follow-up email was sent a few days later prompting them about the study. Finally, where the target number of interviews had not yet been reached, organisations were contacted by telephone for up to a maximum of two attempts. Where this was the case, the researcher verbally communicated the key information about the study contained in the initial emails.

For most areas, the first time employers and training providers heard about the research was when they were contacted via email about the study. In Area 4, however, DWP staff approached employers in advance before handing contact details over to the study team. This approach was taken as DWP staff in this area felt it would be beneficial in securing employer participation and minimised any risk to their relationships with these contacts if the study team were to contact them without warning. It should be noted that this may have increased the risk that some employers may have felt obligated to take part in the research and/or restricted their feedback due to a perceived lack of separation between DWP researchers and operational staff leading on SWAPs . As with all areas, researchers in Area 4 emphasised their independence from jobcentres ( JCPs ) and SWAP policy decision-making during each contact with participants, and the questions asked during data collection were framed in a way to encourage and enable participants to be honest about their experience. Despite this, it’s likely that a certain level of bias related to this aspect remains in the dataset obtained.

Each interview lasted approximately 30 minutes to an hour, depending on how much each organisation wanted to share. To ensure the most appropriate person was spoken to, the information shared during the recruitment stages requested that the participant was an individual who had knowledge of, or was responsible for, the SWAP that their organisation had been involved in.

Contacting staff

Once JCP Service Leaders had agreed for fieldwork to take place in their district areas, the study team were signposted to operational staff who would be able to facilitate the research. These individuals became key contacts for the study team during fieldwork. In initial meetings, these local contacts provided a broad overview of the SWAP set-up in their district, and the types of staff involved in delivery, from which a list of different staff roles to speak to as part of the fieldwork was agreed. Due to the varying nature of the local staffing models, it was easier to understand how SWAP delivery was organised in some areas more than others.

The project manager and case study leads maintained regular contact with these local contacts while fieldwork took place in each area. In Areas 1 and 2 this mostly consisted of contact via email, whereas for Areas 3 and 4 this took the form of a weekly scheduled meeting. In Areas 2, 3 and 4, a follow-up meeting took place with the local contacts to check the study team’s understanding of local SWAP delivery obtained through data collection, and to clarify any aspects of delivery that remained unclear.

The local contacts provided a list of suggested staff who could be approached for the fieldwork based on their role and involvement in local SWAP delivery. The study team then arranged the interviews and focus groups for these staff around their availability. In setting up the interviews and focus groups, an information sheet was provided about the research, and it was emphasised that their participation was voluntary. Despite this, some staff may have only participated in the study because they felt obligated to. As with other participant groups, the independence of the study team was emphasised, and participants were offered the opportunity to withdraw from the study if they wanted to.

Piloting [footnote 1] interviews were conducted with two members of DWP staff, one employer and one training provider who agreed to this. These interviews were conducted to test the length and appropriateness of the topic guides for these participant groups, and the quality of the data obtained. These individuals were recruited from a separate JCP district to the case study areas and so the information collected was not used in the analysis and reporting of this study. The topic guides were amended following these pilots.

For claimants, the first week of fieldwork was considered a pilot. Minimal changes were made to the topic guide following these interviews, and so, unlike the other fieldwork strands, the data collected during these interviews was analysed and reported on. It should, however, be noted that the topic guides and fieldwork processes were continually reviewed and modified throughout the data collection periods to ensure they were as efficient and effective as possible. The study team met multiple times a week to reflect on interviews, and formal debrief sessions were held within the study team, and separately with wider supporting researchers, following the end of data collection in each study area. This process allowed learning from each area to be implemented in subsequent fieldwork.

Analysis and Reporting

Once all interviews had been conducted, the interview notes formed the final dataset. The dataset was explored using a thematic analysis approach. As there were multiple researchers involved in the coding of the data, a coding framework (Annex 4) was developed to ensure consistency in coding across the study team. The research questions were used as a guide to ensure the framework aligned with the objectives of the research, and the framework was tested with an initial sample of interviews before a final version was agreed for coding the rest of the data (although this was still subject to ongoing tweaks as coding progressed).

Members of the study team were paired up to code a specific strand of data (for example, employers) and each pair coded the same initial set of notes to check alignment in their coding approach, before separately coding the remaining data. A separate member of the study team then examined a selection of coded notes from each pair to quality assure the completed coding. Feedback on the coding approach, particularly inconsistencies within each pair, was provided to the coders so that this could be incorporated into the analysis of future notes.

The project team met multiple times to discuss and agree the themes identified within the coded data. The themes identified via this process of analysis structured the findings within this report. When analysing the data, findings were explored by participant group (for example, claimants vs. employers) as well as by case study area (for example, Area 1 vs. Area 2), and these were included in the reporting where relevant.

A Quality Assurance ( QA ) panel was established to review the work of the research team during the analysis stage. The panel included researchers external to the project, senior researchers, and a fieldworker external to the study team, who was involved in conducting the research. This panel was engaged to review the initial coding framework that had been developed, and again to review how the codes had been applied to a sample of the data collected. This ensured that the approach taken to analysis had been peer reviewed, and that the data analysis conducted was of good quality. The final report was separately quality assured by an academic on secondment to the In-House Research Unit ( IHRU ), as well as senior researchers in the unit.

Annex 4: Initial coding framework

A pilot is a small-scale, preliminary study that is used as a test run for a particular research instrument to ensure its efficacy.  ↩

Is this page useful?

• Yes this page is useful
• No this page is not useful

Help us improve GOV.UK

Don’t include personal or financial information like your National Insurance number or credit card details.

To help us improve GOV.UK, we’d like to know more about your visit today. Please fill in this survey .

Advertisement

Impact of government interventions on the stock market during COVID-19: a case study in Indonesia

• Original Article
• Open access
• Published: 17 August 2022
• Volume 2 , article number  136 , ( 2022 )

Cite this article

You have full access to this open access article

• Josua Sinaga 1 ,
• Ting Wu 1 &
• Yu-wang Chen   ORCID: orcid.org/0000-0002-2007-1821 1  

7208 Accesses

3 Citations

Explore all metrics

This paper aims to examine the short-term impact of government interventions on 11 industrial sectors in the Indonesian Stock Exchange (IDX) during the COVID-19 pandemic. Whereas earlier studies have widely investigated the impact of government interventions on the financial markets during the pandemic, there is lack of research on analysing the financial impacts of various interventions in different industrial sectors, particularly in Indonesia. In this research, five key types of government interventions are selected amid the pandemic from March 2020 to July 2021, including economic stimulus packages, jobs creation law, Jakarta lockdowns, Ramadan travel restrictions, and free vaccination campaign. Based on an event study methodology, the research reveals that the first economic stimulus package was critical in reviving most sectors following the announcement of the first COVID-19 case in Indonesia. Jakarta lockdowns impacted stock returns negatively in most sectors, but the impacts were relatively insignificant in comparison to other countries in the region. The recurrence of lockdowns in Jakarta had a minor detrimental impact, showing that the market had acclimated to the new normal caused by the COVID-19 pandemic. Additionally, Ramadan travel restrictions caused minor negative impacts on the stock market. Furthermore, the second Ramadan travel restrictions generated a significant reaction from the technology sector. Finally, while free vaccination campaign and job creation law did not significantly boost the stock market, both are believed to result in a positive long-term effect on the country’s economy if appropriately executed. The findings are critical for investors, private companies, and governments to build on recovery action plans for major industrial sectors, allowing the stock market to bounce back quickly and efficiently. As this study limits its analysis to the short-term impact of individual interventions, future studies can examine long-term and combined effects of interventions which could also help policy makers to form effective portfolios of interventions in the event of a pandemic.

Similar content being viewed by others

Does Digital Financial Inclusion Have an Impact on High-Quality Development of Trade? Evidence from China

Estimating the scale of profit shifting and tax revenue losses related to foreign direct investment

Effect of exchange rate misalignments on foreign direct investment in sub-saharan africa.

Avoid common mistakes on your manuscript.

Introduction

A contagious disease that started in December 2019, so-called COVID-19, had infected more than 197 million people globally, with 4.2 million deaths as of 31 July 2021 (WHO 2021 ). The virus attacks the human respiratory system, which leads to a more severe effect on people with underlying medical problems. There have been a few disease outbreaks since the beginning of the twenty-first century when a deadly Severe Acute Respiratory System (SARS) virus tore East Asian countries in 2002 with over 8000 cases. In addition, in early 2009, the United States discovered a new influenza outbreak called Swine Flu (H1N1), which lasted around 19 months and was estimated to cause between 105,000 and 395,000 deaths in over 214 countries. The COVID-19 pandemic is still incomparable to these outbreaks as it utterly changed the way of living, which appeared to cause a domino effect to the world’s economy. The pandemic led to an economic downturn indicated by most countries declaring a recession in the third quarter of 2020. In addition, the stock markets worldwide had significantly lost investor’s confidence due to the uncertainty during the COVID-19 pandemic, which was supported by past experiences (Fan 2003 ; Bloom et al. 2005 ). However, although an increase in new cases has shown a negative return in the stock market, each country has faced different impacts from the COVID-19 pandemic. Thus, despite the relatively low mortality rates, this pandemic has created a more significant financial crisis than any other extraordinary event in this century.

This study centres its discussion on the COVID-19 situation in Indonesia, which is selected to reach wider audience from relevant countries in terms of geographic and economic conditions. Geographically, Indonesia is the largest tropical archipelago in the world and it is located in the most populous continent (Cribb and Ford 2009 ; United Nations 2021 ). Furthermore, from an economic perspective, Indonesia is also the largest economy in Southeast Asia and the world's seventh largest economy by purchasing power (World Bank 2018 ). The fourth most populated country declared its first case on 2 March 2020 when the president announced that foreign citizens had transmitted the virus to a local citizen in Jakarta. The COVID-19 cases in Indonesia have proliferated for the past 16 months since March 2020, with 3.4 million cases that caused over 90,000 (WHO 2021 ). By the end of July 2021, Indonesia has the most active cases in the Southeast Asia region, with over 40,000 daily new cases as shown in Fig.  1 .

The daily COVID-19 cases in Indonesia from March 2020 to July 2021 (WHO 2021 )

This study uses the local stock market as the instrument to quantify the impact of the COVID-19 pandemic to the Indonesia’s economy. A stock market is used as it could be utilised as another vital economic indicator beside the country’s GDP. Masoud ( 2013 ) supported that the stock market has played a significant role in emerging countries like Indonesia. Hall ( 2020 ) also found that the market volatility trend was correlated negatively and significantly with real per capita GDP growth. Therefore, it is believed that any significant impact from government interventions could be detected from the stock market’s short-term movements.

Indonesia is selected also due to its relatively stable financial performance over any major indices in the Southeast Asia region as illustrated in Fig.  2 . Despite its rapid growth in the number of cases, there is an indication that the government interventions might have played a significant role to recover the stock market’s fall. Although it experienced a significant loss of around 20% in March 2020, the Indonesian Stock Exchange (IDX) has rebounded and performed relatively well at least until July 2021. Hence, the Indonesian Government’s approach during the COVID-19 pandemic could provide an exciting insight into the literature.

Monthly returns of Southeast Asia’s major indices from July 2019 to July 2021 (Reuters 2021 )

The challenging situation has led the Indonesian Government to implement a series of non-pharmaceutical interventions, such as social distancing, travel restriction, and lockdown. In addition, several economic policies, including debt relief, electricity subsidy, tax exemption, and microbusiness support, have also been implemented as it is critical to keep the investors’ positive sentiment, as the country targets an optimistic 5.6–6.2% average GDP growth by 2024 (Reuters 2020 ). Furthermore, the president had also used political interventions by restructuring his cabinet to strengthen the health and economy ministers to gain more trust from the public.

To understand the effectiveness of the Indonesian government interventions, this study uses an event study methodology to quantify the possibility of short-term indirect impact on each individual sector in the Indonesian Stock Exchange (IDX). Additionally, an event study can analyse numerous similar events to forecast how stock values in a certain sector normally react to a particular event. The methodology has also been applied extensively on both firm specific and industry level in analysing the short-term impact of corporate-related news, financial crisis, marketing strategy, natural disasters and disease outbreaks.

The study contributes to the literature by providing insights into the short-term impact of government interventions during a pandemic. The findings are expected to help investors, regulators, and government understand the short-term impact of government interventions on each industry sector. The insights could also help them create better policies and decisions to avoid a severe economic impact on any future pandemics. The remainder of the study is organised as follows: Sect.  2 reviews the literature related to the research topic, while Sect.  3 discusses the methodology and data collection process for event study analysis. Section  4 presents the results and findings from the analysis. Last, Sect.  5 concludes the study.

Literature review

Impact of pandemics on the economy.

Many studies have investigated the economic impact of pandemics on regional and worldwide levels. Most of the literature indicated that a pandemic would have a short-term impact on the global economy, with a longer duration for countries identified as the pandemic's epicentres. SARS, the first pandemic of the twenty-first century, began in Hong Kong and infected over 8000 individuals worldwide in 18 months, with a 10% fatality rate. Mackellar ( 2007 ) calculated that a relatively small outbreak triggered a 1% drop in China’s GDP and a 0.5% drop in Southeast Asia’s GDP, resulting in a substantial loss of US$ 30 billion. Although Sánchez and Liborio ( 2012 ) determined that a relatively insignificant decrease in GDP would not immediately cause a dramatic increase in unemployment rates, they argued that a recession could be incurred with a likelihood of 33%. H1N1 was initially reported in the United States in the spring of 2009, with over 400, 000 confirmed cases by the end of the year (CDC 2019a ). The economic impact was difficult to quantify as the economy was still dealing with the effects of the 2008 Global Financial Crisis. However, Barua ( 2020 ) suggested that a clear enlightenment might be gained from the tourism business, and they found that Asia’s tourism sector was the quickest to recover from the pandemic by reviving tourist arrivals in the second half of 2009. In addition, Rassy and Smith ( 2013 ) analysed the effect on Mexico, which was one of the pandemic’s epicentres, estimated US$ 2.8 billion losses from the tourism industry given the intensity of the H1N1 Kim et al. ( 2012 ) estimated that the indirect cost from a pandemic could be 5–10 times more than the direct cost. The 2014 Ebola pandemic was 11 times larger than the previous Ebola outbreaks combined, with over 11,000 deaths by the end of 2016 (CDC 2019b ). The outbreak was centred in the West African region, affecting the following three nations in particular: Guinea, Liberia, and Sierra Leone. According to the World Bank report ( 2016 ), the three countries lost a total of US$ 2.8 billion. Furthermore, the report also stated that Sierra Leone’s private sector had lost half of its workers, while the unemployment rate in Liberia had climbed by 40%. Interestingly, Ighobor ( 2014 ) found that neighbouring countries with no confirmed cases also faced a substantial decline of their GDP by at least 1%, suggesting that tourist and transportation sectors had spillover effects. According to Omoleke et al. ( 2016 ), high death rates have resulted in an unprecedented number of persons being pulled from the labour market, resulting in a drastic fall in public consumption.

In comparison to the above-mentioned pandemics, the COVID-19 pandemic has a greater impact. The IMF (International Monetary Fund 2020 ) reported that the cumulative GDP loss could be around US$ 9 trillion in 2020, with US$ 4 trillion solely contributed from the tourism industry. Additionally, emerging economies were significantly damaged as many declared recessions. Moreover, Jawaid and Garrido ( 2020 ) discovered that in the third quarter of 2020, 31 developed economies with a GDP over US$ 200 billion were in a recession. Rose ( 2021 ) also predicted that the economic impact of the COVID-19 pandemic would likely to be 30–50 times greater than the 9/11 tragedy.

Stock market reaction to the COVID-19 pandemic

The adverse reaction in the global stock markets started when WHO declared COVID-19 as a global pandemic, and many governments started to declare its first cases (more details in Appendix A). Most agreed that stock markets underreacted as many showed positive returns within the first few days but suddenly plummeted to their lowest level in the decade. Additionally, firms were showing different reactions to the COVID-19 as SMEs showed significant negative returns while large firms still survived with slight positive returns in the United States (Harjoto Rossi and Paglia 2020). The stock markets’ delayed response might have two possible explanations as follows: different market efficiencies in different countries and the investors’ confidence in the government to deal with the pandemic (Khatatbeh et al. 2020 ).

The countries with more complex COVID-19 situations had proven to have higher volatility in the stock market. Harjoto et al. ( 2020a , b ) found that emerging markets tend to be more volatile than the developed market with 1% increase in daily cases and deaths caused a 2.37% and 14.94% volatility in the stock market. Fu et al. ( 2021 ) examined the global stock markets and concluded that South America was highly exposed to the contagion economic risk, while Asia experienced the most negligible severe risks in the same period. Their study also indicated that the panic caused by the uncertainties led to a severe drop in investor confidence, resulted in a nose-dive response in the global stock market. Gupta et al. ( 2021 ) strengthened the claim that the COVID-19 pandemic has negatively impacted all major stock markets in the short term. According to Reuters ( 2021 ), the monthly returns for major indices, including Dow Jones, FTSE 100, Euro Stoxx 50, Shanghai and ASX 200 in Fig.  3 showed a relatively stable return before the COVID-19 outbreak but declined sharply from February 2020 to April 2020. Australia’s ASX-200 was one of the most affected indices at the beginning of the pandemic, with a negative 20% return in March 2020. On the other hand, China’s Shanghai index showed a relatively stable return as the country successfully controlled the disease’s transmission rate.

The monthly returns of global indices from July 2019 to July 2021 (Reuters 2021 )

It was discussed that the increase in investor herding behaviour during the COVID-19 epidemic might have contributed to stock market volatility. In general, herding behaviour aggravates fluctuations, resulting in inefficiencies in the stock markets (Blasco et al. 2012 ). In research of 49 countries around the world, Bouri et al. ( 2021 ) discovered a strong connection between herding behaviour and stock market uncertainty, and they discussed that emerging markets exhibit more herding behaviour as a result of more volatility, which is consistent with prior research. Chong et al. ( 2016 ) concluded that companies with a high turnover ratio and systematic risk would be swiftly exposed to herding behaviour in their study of China's stock market. In addition, analyst recommendation is one of the factors that played a vital role in causing herding behaviour.

Government interventions during the COVID-19 pandemic

Disease outbreaks, including the rapid spread of COVID-19, have caused severe human and economic costs to any country globally. Governments were forced to implement strict measures to limit these costs in the short and longer-term. Several studies showed the importance of enforcing the interventions that could change people’s behaviour, which eventually would help reduce the COVID-19 transmission rate, as mentioned in Cowling et al. ( 2020 ) study in Hong Kong. Non-pharmaceutical measures, according to many studies, effectively reduced the number of cases reproduced; however, pharmaceutical interventions, such as immunisation, were critical in controlling the COVID-19 pandemic.

We reviewed the impact of the four widely implemented non-pharmaceutical interventions, including contact tracing, social distancing, lockdown, and border restriction in Appendix B. Koh, Naing and Wong (2020) stated that these interventions were implemented by over 142 countries before the 100th COVID-19 cases identified in each country due to their effectiveness in limiting people’s movement and hence reducing the transmission rate of the infectious disease. Governments implemented lockdown as one of the earliest interventions. Lockdown is widely regarded as an effective tool to restrict the transmission rate of infectious diseases, though any lockdown over 120 days is considered ineffective (Koh et al. 2020 ). Goldstein et al. ( 2021 ) suggested that countries should not impose a blind national lockdown which could place the underprivileged people at high risk of unemployment. Instead, a short and strict lockdown would be ideal for emerging countries with a data-driven decision on hospital system situations. When combined with quarantine policies, contact tracking proved to be effective. However, the implementation could be costly for emerging countries, while data protection could be a complicated challenge to overcome by Western countries.

Furthermore, Adekunle et al. ( 2020 ) argued that the impact of border restrictions was largely reliant on the underlying situation of the country. Steyn et al. ( 2021 ) suggested that Australia and New Zealand had the strongest border restrictions, which were seen to be crucial during the early epidemic, delaying the pandemic by four weeks and giving the government more time to prepare. On the other hand, African countries that enforced border restrictions witnessed an increase in new cases due to a lack of official support to ensure the lives of the people (Emeto et al. 2021 ). In fact, social distancing was found to be the most effective non-pharmaceutical intervention by significantly reducing the transmission rate by 25% (Li et al. 2020 ).

Impact of government interventions on the stock markets

Non-pharmaceutical measures were also applied to reduce COVID-19 transmission. Governments implemented a variety of economic recovery interventions. In doing so, governments previously had enforced direct and indirect stock market interventions. During a crisis, direct interventions had varying results. During the Asian Financial Crisis of 1998, Hong Kong allocated US$15 billion to acquire the Hang Seng Index’s 33 stocks. A positive abnormal return for at least 30 days was found to restore investor confidence successfully, with a short-term spillover impact to other stocks (Su et al. 2001 ). During the 2008 Financial Crisis, Russia injected the banking system with US$ 150 billion. As a result, the market overreacted on the intervention day, resulting in a significant negative return. Therefore, many governments selected the safer approach with indirect interventions, which had succeeded in many developed countries (Swaine 2008 ; Murphy 2008 ). Many studies found that social distancing and lockdown had a short-term negative influence on markets.

We reviewed the impact of three non-pharmaceutical interventions, including lockdown, gathering restrictions, and economic support on the stock markets in Appendix C. Stock markets across the world had plummeted as a result of the lockdown’s implementation. When local governments announced the lockdown, both developed and emerging markets overreacted. Lockdown also had a spillover impact on interconnected countries, as several economies experienced a brief downturn when their neighbours went into lockdown (Eleftheriou and Patsoulis 2020 ). The cancellation of public events was proven to be the most impactful restriction in causing excessive volatility in the global stock markets (Zaremba et al. 2020 ). Gathering restrictions were also found to cause high volatility in the global stock markets, with the cancellation of public events believed to be the most impactful restriction (Zaremba et al. 2020 ). The restriction also significantly reduced the illiquidity situation in America, Europe, and the Middle East emerging economies. However, Asian emerging economies showed no impact on the market’s liquidity (Haroon and Rizvi 2020 ). Last, economic supports were insignificant in helping the recovery of stock markets. However, several studies concluded that these interventions directly targeted households and not corporations, which generated a relatively small indirect impact. Besides, monetary policies and fiscal policies were impactful in helping stock markets to rebound in all continents. Asian emerging markets were found to have a spillover impact from developed countries’ quantitative easing policies, contributing to an 8% surge on average (Beirne et al. 2021 ).

Data and methodology

Data collection, stock market.

The Indonesia Stock Exchange (IDX) was established in 2007 and has shown significant development during the past decade. As of 31 July 2021, the stock market has 746 listed companies, an approximately 50% increase since 2014. However, during the COVID-19 pandemic, the Jakarta Composite Index (JKSE) has reached its 7-year low in March 2020. Nevertheless, JKSE recovered towards the end of 2020 with only a 4.8% loss and outperformed other major indexes in the Southeast Asia region (Maulia 2021 ).

As a sample for the analysis, we selected the industry sector leaders based on the market capitalisation before the COVID-19 pandemic began. Indonesia, as an emerging market, has a different characteristic from the developed market. As illustrated in Table 1 , the financial sector still dominates and has become most of the market in Indonesia, where the technology sector has been on the top list for decades in developed countries like the United States. The stock market data used in this study is considered as a secondary dataset obtained from the Yahoo Finance database from 1 January 2020 to 31 July 2021 to analyse the impact of government interventions selected for this study.

• Government interventions

This study selected nine events between March 2020 and July 2021, including two events of economic stimulus packages, one event of jobs creation law, three events of Jakarta lockdowns, two events of Ramadan travel restrictions, and one instance of a free vaccination campaign. The timing for each intervention's announcement is highlighted in Fig.  4 . The selection of the nine events is justified in the following sub-sections.

The timeline of five Indonesian government interventions for the analysis

Economic stimulus packages

The Indonesian president unveiled the economic stimulus package on 24 March 2020, and it was the first economic policy responded to the COVID-19 pandemic. According to his statement, the economic stimulus package was designed to assist corporate firms in surviving and maintaining people’s purchasing power. The package includes tax breaks for all industries, credit relief for small businesses, and an increase in the amount accessible to Staple Food Card recipients. A total of nine incentives were implemented, resulting in a rise of IDR 405 trillion in the state budget, which is equivalent to US$ 27 billion (Gorbiano and Akhlas, 2020 ).

This intervention was chosen for various reasons, including the fact that it was the Indonesian government’s first and largest economic intervention during the COVID-19 pandemic. The economic stimulus package also resulted in a substantial increase to the state budget for 2020, which was in place until December 2020. It was believed that the nine incentives provided possibly had an indirect impact on various business sectors, which might be the turning point for stock market downturns. The government had also decided to extend several incentives from the 2020 economic stimulus package into 2021, which was announced by the president on 4 January 2021. The 2021 package focused more on small businesses, impoverished families, and unemployed citizens with a total of IDR 110 trillion allocated, which is reduced than the previous year.

Jobs creation law

The global unemployment rates were soaring at the beginning of the COVID-19 pandemic caused by cost-optimisation by most businesses. Indonesia has also experienced a sudden increase within a few months after the pandemic. Figure  5 shows the sharp movement between the first and second quarters of 2020, indicating that at least three million people lost their jobs throughout the period.

The unemployment rates in Indonesia from February 2018 to February 2021 (BPS Statistics Indonesia 2020 )

During the challenging times, the Indonesian parliament decided to authorise a new law to simplify investment regulations called Omnibus Law (Jennings 2020 ). The complexity of foreign investments in Indonesia has been a long-standing issue, as The World Bank ( 2020 ) highlighted the country’s rigid investment regulations as a factor limiting growth. It is also reflected from the 2020 World Bank’s Ease of Business Index that put Indonesia in 73rd place among 190 countries, far behind other Southeast Asia countries. Oxford Business Group ( 2020 ) also suggested that the law could provide jobs for six million people who have been left unemployed during the COVID-19 pandemic.

This intervention was selected as one of the most vital and controversial interventions during the COVID-19 pandemic. This is because the international world looked at the law as a brighter future for the Indonesian industries, while the domestic workforce strongly opposed the law due to a weakening in job security. Nevertheless, the Omnibus Law was a concrete step for Indonesia’s investments regulations, which could positively impact the country’s economy during the COVID-19 pandemic.

Jakarta lockdowns

Jakarta is the Indonesian capital, categorised as a unique province among 33 other provinces in the country. The capital is the most populated city in the Southeast Asia region, with over 10.57 million population at the beginning of 2020 (BPS Statistics Indonesia 2020 ). In addition, Jakarta is also the headquarters of big companies and where the Indonesia Stock Exchange sits.

Jakarta’s population density is extremely high, over one hundred times the country’s average, with 16,704 people per Km 2 in 2020 (BPS Statistics Jakarta 2020 ). Due to the high population density, the COVID-19 cases’ growth in Jakarta was the fastest among other provinces. Figure  6 illustrates the significant difference in the number of COVID-19 cases in Jakarta compared to the five most populated provinces. The six provinces had around 63% of the total cases in Indonesia at the end of 2020, implying that high-population provinces were the pandemic’s epicentres in Indonesia.

The number of COVID-19 cases in six provinces in Indonesia (Indonesia COVID- 19 Response Acceleration Task Force 2021 )

This has forced the Indonesian government to impose a regional-level lockdown instead of a national lockdown like other high-population countries like China and Brazil. Jakarta was the first province to impose the lockdown when the governor announced 14 days of large-scale social restrictions on 7 April 2020 when the number of daily new cases surpassed one hundred for two consecutive days (BPS Statistics Jakarta 2020 ). This was followed by two other lockdowns in September 2020 and July 2021 mainly to avoid public health collapse due to the increase in the new cases.

This intervention was selected for several reasons, considering that Jakarta is the most populated city in Indonesia and headquarter of most businesses in the country. Although Jakarta is the smallest province geographically, it represents the biggest contributor to the country’s GDP with 17.67% in the second quarter of 2020. This indicates that any extreme measurements in Jakarta could impact the country’s economic condition.

Ramadan travel restrictions

Indonesia has the largest Muslim population, accounting for approximately 12% of the global Muslim population. The country’s biggest annual event occurs during Ramadan month when tens of millions of people travel to their hometown; a tradition locally called Mudik, like Christmas in western countries or the Chinese New Year. This event generally impacts the country’s economy, as people tend to spend more due to the compulsory Ramadan bonus granted to all employees. Moreover, Muslims must pay alms during Ramadan, which creates a massive surge in the money circulation during the season. Hence, the consumer sector typically significantly impacts Ramadan, with an approximately 30% increase in sales (Halimatussadiah 2015 ).

However, Ramadan travel during the COVID-19 pandemic could result in a massive surge in the new cases as millions of people would travel in any transportation mode. For example, 23 million people travelled domestically during Ramadan in 2019 (Wight 2020 ). To prevent the disaster, the Indonesian government temporarily banned domestic flights, busses, and ferries for at least 14 days before and after Ramadan Day. These restrictions were imposed in both 2020 and 2021, which have proven to reduce the transmission rate. However, the economic impact was unclear to the country’s economic condition. Hence, this intervention was selected as it would be insightful to explore the economic impact created by the restrictions.

Free vaccination campaign

The Indonesian journey with vaccination finally started when the first batch of Sinovac vaccines arrived in Jakarta on 7 December 2020. A week later, President Widodo announced to provide free Covid-19 vaccines, which plays a vital role in other countries to recover their economic condition quickly. For example, the vaccination policy and people’s willingness to get vaccinated gave strong sentiments from the US Stock Market, which is a good indicator of economic healing (US Bank Asset Management Group 2021 ). In other words, vaccines could help the government to have significant economic growth as people’s movement would be less limited.

Several studies have also found a reasonable hope that vaccines should help the world stop the pandemic. For example, Powell ( 2021 ) concluded that vaccines are created to establish herd immunity, which can be achieved when 50–60% population is vaccinated regardless of the virus mutations, as it would still have the same structure. Therefore, this intervention was selected as it might be the booster for the stock market to gain more trust from the investors and started to grow strongly towards 2021.

Methodology

The study used the event study methodology, which has been widely utilised to assess the valuation impacts of extraordinary corporate actions (more details in Appendix D). In addition, event study has revealed vital information about how an industrial sector is likely to react in a short-term to a given extraordinary event, such as natural disaster, disease outbreaks, and geopolitical issues. A short-term analysis is critical for the stock market since an extraordinary event might alter investor behaviour and the entire market environment.

The approach examines the stock price’s response around the announcement by looking at the stock returns first introduced by Dolley ( 1933 ). Moreover, event study with known event dates has a relatively statistical solid power to support the result, which is required to understand the short-term impact during unprecedented events like the COVID-19 pandemic. Although Dyckman et al. ( 1984 ) pointed out several problems of daily return analysis, such as nonsynchronous trading and biased estimation, Brown and Warner ( 1985 ) concluded that the potential problems with daily returns are unimportant easily corrected in the standard event study. Figure  7 illustrates the process taken by this study to perform the event study analysis, and the key steps are described in the following subsections.

The process diagram for event study analysis

Three event study models were commonly used in previous studies, namely constant mean return, market model, and capital asset pricing model (CAPM). The main difference among these models lies in the way of calculating the expected returns and abnormal returns. The constant mean return model has the simplest from by subtracting the stock return with the simple mean return, but Brown and Warner ( 1985 ) criticised that the method does not consider the abnormal return in reflection to the stock market condition. The market model and CAPM are the most popular yet similar methods in practice. The difference is that the CAPM imposes an additional restriction (e.g., intercept equals the risk-free rate). Due to the added restriction, the variance of error terms in CAPM is generally more significant than the market model (MacKinlay 1997). Consequently, a significant variance of error leads to a less powerful test for the result than the market model. This study uses the seminal market model introduced by Scholes and Williams ( 1977 ).

There are two crucial parameters that determine the analysis’s outcome in the event study, namely the estimation window and the event window. This study used 42 days of the estimation window, equivalent to 2 months of trading days, as the events analysed happen within a short period of 16 months. Furthermore, we used an event window of 7 days, which centres symmetrically around the event day, as shown in Fig.  8 , and a similar approach has been taken by previous literature (Bash and Alsaifi 2019 ; Buigut and Kapar 2020 ). A short event window is also applied to prevent overlapping event window periods, as some events in the analysis are only a few weeks away. Further, the small period within the event window, the anticipation window and the adjustment window are used to capture the short-term abnormal returns before and after the event day. In contrast, a long event window could reduce the statistical power of analysis but suffer from essential limitations (Brown and Warner 1985 ).

Illustration of the event window timeline used in the paper

Estimation of expected return

The market model assumes that the asset returns are given by the following:

where \({R}_{i,t}\) represents the return for each company i on day t, which belongs to the estimation window, while the expected return is established as follows:

The \({R}_{m,t}\) represents the market portfolio’s return, and the linear specification of the model arises from the assumed joint normality of returns. The market portfolio used is the Jakarta Composite Index (JKSE), the composite index for IDX. The market model also assumes that \({\varepsilon }_{i,t}\) changes related to the return on the market portfolio iare removed as follows:

Estimation of abnormal return

This study used Buy and Hold Abnormal Returns (BHAR), which employs a geometric method to calculate abnormal returns. The BHAR approach was used as several economists such as Ritter ( 1991 ) and Lyons ( 1999 ) have argued that CAR is not appealing from the economic perspective. The CAR approach could lead to biases due to the continuous compound rate of appreciation. The calculation of BHAR is established as follows:

where i represents each company in the analysis, t represents the event window start date, and k represents the duration of the event window. Furthermore, the calculation of Abnormal Return (AR) is as follows:

Test procedure

The parametric test is the only approach to test the null hypothesis for the event study that analyses multiple individual events’ impact (Boehmer 1991 ). We used a statistical test to determine whether enough evidence exists to reject a hypothesis about the process. The following hypothesis testing is adapted to the parametric test:

The statistical test is formulated as follows:

The \({t}_{\mathrm{BHAR}}\) represents the t -score of the BHAR for each company at different event windows and \({\sigma }_{BHAR}\) is the standard deviation of the BHAR for the estimation window.

Implementation

This paper used an analytical approach to answer the research questions, mainly for the event study analysis. Open-source packages in Python are used to automate the data extraction process, calculate the daily returns, estimate the abnormal returns, and validate the results.

Results and discussion

Preliminary analysis.

To demonstrate the unique circumstances surrounding the COVID-19 pandemic, descriptive statistics for the years 2019–2021 are presented for comparison. A majority of industries experienced a fall in the average stock price, as demonstrated by the Jakarta Composite Index (JKSE), which experienced a roughly 16% decline in the average index price in 2020 (Tables 2 and 3 ). However, two of the eleven sectors, basic material and property, experienced a significant increase in the average stock price, nearly doubling the previous year.

Table 4 summarises the statistics data for 2021, which is considerably different from the previous year. The market has generally recovered, with an average index price increase of 16% until July 2021. Ten of the eleven industries also had strong growth, except for the property sector, which experienced a substantial decline. Although there are still a few months remaining before the end of 2021, these figures indicate that the market has begun to recover, despite Indonesia continuing to have the most cases in Southeast Asia as of 31 July 2021.

The range is a crude measure of the spread in stock prices, and it indicates that the range for the majority of sector leaders increased significantly in 2020 compared to 2019. The range, however, is subject to outliers that occur under an extreme COVID-19 situation. In comparison, the standard deviation is a more accurate tool for detecting outliers in a normal distribution. This method showed a substantial increase from 2019, indicating that the stock market saw much higher volatility returns as the dispersion of company prices compared to their average increased dramatically. Additionally, it implies that the stock market became a riskier investment during the COVID-19 pandemic in 2020.

On the other hand, both measures have been significantly reduced for the first half of 2021. The composite index’s range and standard deviation have decreased by 72% compared to the previous year. Additionally, nine of eleven industry sectors followed the movement of the composite index. This could indicate that the market has grown less risky and has rebounded from its early 2020 collapse. Furthermore, low volatility attracts additional investors, signalling the prospect of significant growth during the pandemic.

Hale et al. ( 2021 ) developed a government intervention tracer that covers 23 different types of government interventions, including containment and closure policies, economic policies, health system policies, and vaccination policies. In this research, we focussed primarily on the following two indices: overall government response and stringency. We compared and analysed the daily global average government response index to the Indonesian daily index in Fig.  9 . At the start of the pandemic, the Indonesian government responded more adequately than the global average, implementing at least three critical interventions. However, this condition did not last long, as it continued below the global average for the remainder of 2020. In contrast, the Indonesian government has responded more positively from the beginning of 2021, whereas the global average has declined since May 2021.

The comparison of Government Overall Index between Indonesia and the Global average

The stringency index provided a similar trend to the government response index in Fig.  10 . Global average stringency increased rapidly from March to May 2020 but then stabilised until early 2021. Henceforth, the worldwide average has been close to or below the index level of 60. In comparison, Indonesia showed its most stringent condition in May 2020, when the government planned to suspend all modes of public transportation for more than 14 days, with an index was above 80. Although the trend had been downward for several months, it started to rise again in September due to Jakarta's second lockdown. Thus, throughout the COVID-19 pandemic, it is safe to say that Indonesia had implemented tighter restrictions than the global average.

The comparison of Stringency Index between Indonesia and the Global average

Event study analysis

The results for each intervention type are presented and discussed in the following sub-sections, with the individual parametric test results presented in Appendix E.

The impact made by the first economic stimulus package was believed to be highly significant for several sectors (Table 5 ). It is evident that, on the announcement day, the stock prices for financial, consumer non-cyclical, and consumer cyclical sectors were significantly improved. At the same time, infrastructure and healthcare were also increased to a certain extent. In contrast, significant adverse reactions were shown by the stock prices of basic material and property sectors.

It is believed that the financial sector’s reaction was due to the inclusion of financial system stability policies in the package. The policies grant authority to five vital government bodies to establish steps on handling financial stability matters by formulating government support such as short-term liquidity loans and financing on the sharia principle to all financial institutions (Molina and Ramadhan 2020 ). The stability policies were implemented to avoid a serious banking crisis in 1998 when half of the private banks collapsed due to the government’s unpreparedness in handling a crisis (Fane and McLeod 2002 ).

Moreover, additional funds for staple food card beneficiaries and pre-employment card holders significantly impacted both consumer sectors’ stock prices. According to Fang (2021), additional funds could stimulate the subsidy by the government to generate more consumption. Liu et al. ( 2020 ) strengthened the theory by finding that a consumption coupon of RMB 1 can drive excess spending of RMB 3.4 to RMB 5.8 at the beginning of the pandemic in China. Thus, these findings could be an essential variable that increased the market’s confidence in both consumers sectors on the announcement day.

Additionally, the property sector signalled an under-reaction but rebounded with a positive 16.50% abnormal return during the adjustment window. The under-reaction to stock-related news could be due to an anchoring bias or slow information diffusion (Lansdorp and Jellema 2013 ). On the other hanfd, basic materials, industrials, and technology significantly suffered. We believed that the corporate tax reduction for 2020 and 2021 is the only policy that directly impacts these sectors.

The impact made by the second economic stimulus package on stock markets was not significant compared to the first one (Table 6 ). The announcement day has resulted in diverse reactions, where six sectors reacted negatively and five sectors reacted positively. The second economy package was different as it was more focused on income support and debt relief. Ashraf ( 2020 ) found that the insignificant impact caused by the package was only directed to households and did not directly impact corporations. Additionally, the results showed that consumer non-cyclical overreacted on the announcement day, followed by a significant negative abnormal return of 3.10% in the subsequent days, while basic material showed an under-reaction with a significant increase of 4.95% to the stock prices in the same period. Besides, basic material showed an underreaction with a significant increase of 4.95% to the stock prices in the same period. Although the impact was not significant, we believed that the announcement was still made at the right time. The market could lead to high volatility if the government did not announce the continuation of several economic stimulus at the beginning of 2021.

In addition, we also believed that the Indonesian government had also successfully maximised its capabilities. Table 7 shows the percentage of total COVID-19 economic support to the respective country’s GDP in 2020. Indonesia only spent 2.6% of its GDP to support the economy during the COVID-19 pandemic, which was only higher than China and Taiwan. From the relatively low budget, the Indonesian government successfully targeted the support’s recipients, which reflected by the stock market reaction. Moreover, a low cost of living could also contribute heavily to the economic stimulus package. As an illustration, the average meal price in Jakarta is US$2.51 compared to Singapore and Hong Kong, with US$9.84 and US$7.70, respectively (Numbeo 2021 ). Therefore, Singapore and Hong Kong must have provided direct household support over US$1000 a month for vulnerable groups. In comparison, the Indonesian government only provided additional support of less than US$100 a month.

The Authorisation of the Omnibus Law elicited negative sentiments from most Indonesia’s sectors, with eight out of eleven sectors reacting negatively on the day of the announcement (Table 8 ). By comparison, the infrastructure and financial sectors generated strong positive abnormal returns of 4.17% and 1.92%, respectively. Moreover, under-reaction was detected as the stock price for the transportation sector rose significantly by 19.31% in the subsequent days after the announcement. Overall, most sectors suffered a short-term negative impact during the event window, where the healthcare sector plunged significantly by 9.07% and transportation rose strongly by 17.15%.

The early reaction of the financial and infrastructure sectors was expected, given our belief that the Omnibus Law would directly influence those sectors. For instance, the financial sector might save high operational costs because of worker protections. On the other hand, enterprises involved in telecommunications infrastructure may benefit from the law as it enables them to share their infrastructure, generating enormous synergy and accelerating the sector’s growth.

Despite successfully lowering unemployment rates in the hope of accelerating economic growth during COVID-19, the stock market reacted unexpectedly. Before the event, various Non-Governmental Organisations (NGOs) publicly criticised the law on social media platforms and on national televisions. Valle-Cruz et al. ( 2022 ) proved that social media transmission via Twitter directly affected the indices' behaviour, particularly in Indonesia, which has the fourth-largest Twitter user base in July 2021 with 15.7 million active users. Additionally, they determined that the drop in market values during the COVID-19 pandemic was more severe than during the H1N1 pandemic, owing to the abundance of speculation and rumours about the virus. Furthermore, thousands of people protested in October 2020 to express their dissatisfaction with the new law, which could temper market enthusiasm, as happened in the United States when the government’s proposal was encountered with widespread scepticism on its implementation (Randall 2021 ).

While the Omnibus Law was insignificant in the short term, we believed that the new laws would positively affect several sectors over time. The property sector should be a clear winner, as the Omnibus Law simplified the land permission process, which had been a major obstacle for decades. The simplified foreign property ownership restrictions would also entice many international investors to invest in the Indonesian market. Additionally, the energy and basic material sectors would greatly benefit from a 0% royalty on value-added to raw materials. The industrial sector may also benefit from simplified foreign direct investment (FDI), which could boost the sector’s long-term growth. To summarise, the Omnibus Law represents a promising start for Indonesia’s future investment, while implementation would be critical.

Indonesia was the last major country in Southeast Asia to impose a lockdown. The announcement of Jakarta’s first lockdown appeared to cause adverse reactions from most sectors in Indonesia (Table 9 ). The stock market showed relatively normal returns on the announcement day of the first lockdown, with only the financial sector, reacting significantly positive. Financials and consumer non-cyclical sectors were underreacted on the announcement day as both plummeted significantly during the adjustment window. At the same time, properties continued to plunge significantly with a negative 14.45% of abnormal return. Overall, eight sectors suffered negative abnormal returns during the event window, with properties significantly suffered while basic materials and transportations were positively affected.

The first lockdown had such a minor detrimental impact that could be due to the Indonesian government’s delay in enforcing one. According to Ozili and Arun ( 2020 ), the first regional restriction had a more significant impact than the first national restriction. In other words, when the Philippines announced its first lockdown, the stock market in Indonesia may have already experienced a spillover effect. Furthermore, an introduction of lockdown measures has also been linked to undesirable reactions known as overreaction. Within a few days after the release, most industries showed overreaction, suggesting a delay in absorbing unusual news. Liew and Puah ( 2020 ) also verified that various industries experienced varied levels of lockdown based on their market conditions and business nature.

The property sector was also predicted to suffer substantial losses during the initial lockdown. This suggests that the debt-relief scheme included in the economic stimulus package was insufficient to convince the market. Furthermore, transportation sectors reacted negatively to the statement, which was expected given that logistic companies were permitted to operate regularly during the lockdown. Instead, the sector should have received more demands due to the drastic increase in transactions from online marketplaces, as offline stores were forced to halt its operations during lockdown.

The second lockdown in Jakarta elicited a range of responses from most sectors, with less severe adverse effects (Table 10 ). Though six sectors saw abnormal returns on the announcement, the impact was substantially negative for consumer non-cyclical and basic materials. Additionally, the stock market experienced under-reaction, as basic materials and technology stocks rose 5.75% and 21.89%, respectively, in the days following the announcement. The technology sector’s response was expected, as most enterprises, organisations, and schools must rely extensively on technology companies because of the lockdown restrictions. Additionally, the basic materials sector has recovered globally and was expected to be immune to COVID-19 by the second half of 2020 due to a resurgence in demand from China as the largest global importer (Barman 2020 ). Overall, the announcement resulted in five of eleven sectors suffering negative abnormal returns, while the remaining nine experienced positive abnormal returns.

The repetition of the same interventions appeared to have had a stabilising effect on the stock market, particularly proven by the third lockdown in Jakarta (Table 11 ). Our results aligned with Scherf et al. ( 2022 ), who concluded that multiple lockdown restrictions generally caused smaller negative returns than the first one. On the announcement day, consumer non-cyclical and property substantially impacted stock prices, gaining 5.78% and 23.79%, respectively. Additionally, a substantial under-reaction was detected in the consumer cyclical sector, which increased by 18.85% during the adjustment window. Six sectors, in aggregate, responded negatively to the announcement, with the infrastructure sector suffering the most, with an abnormally negative return of 8.43%. In comparison, five sectors experienced positive reactions, with consumer cyclical and property experienced considerable increases in stock prices of 12.42% and 35.87%, respectively.

China enforced its first nationwide travel restriction during the 2020 Lunar New Year. Huo and Qiu ( 2020 ) discovered a significant negative impact on the Chinese stock market during the period, with 22 out of 28 sectors experiencing negative abnormal results. Similarly, Indonesia enforced four-week nationwide travel restrictions during Ramadan in 2020 and 2021, but it was found to have a minor impact than China.

On the announcement day, only four sectors suffered a negative abnormal return, while the remainder experienced a positive abnormal return (Table 12 ). Additionally, throughout the anticipation window, a similar trend was seen with no substantial abnormal returns. However, the consumer cyclical sector underreacted on the announcement day, as indicated by a significant increase of 8.64% within the adjustment window, while properties underreacted by 11.43%. In general, the announcement had no discernible effect on any sector in Indonesia. Most sectors demonstrated insignificantly positive reaction, apart from the financial, energy, property, and transportation sectors, which suffer severe abnormal returns throughout the event window. Additionally, the overall positive returns demonstrated by both consumer sectors indicated that Ramadan spending was resilient to the COVID-19 pandemic.

We believed that information leakage was critical in preventing a short-term stock market collapse. For instance, a few days before the announcement, government officials talked about the possibility of travel restrictions in an open public forum. Journalists spread the rumours over multiple media resulting in a higher anticipated reaction from the stock market. Brunnermeier ( 2005 ) supported the notion that information leakage makes price processes more informative in the short run, indicating that information leakage frequently helped investors in managing expectations, hence stabilising the stock price in the short term. However, the study also discovered that information leakage might eventually diminish information efficiency. Additionally, the relatively positive responses from most sectors may reflect the market's confidence in the government's commitment to reduce the COVID-19 transmission rate.

A less severe impact because of subsequent measures is also demonstrated by the second nationwide travel ban in 2021 (Table 13 ). Only the technology sector reacted positively on the announcement, with an 11.64% gain in its stock price. Additionally, there were no substantial reactions before the announcement day since all sectors experienced divergent returns, which is believed to have been exacerbated by information leaks a few days before the announcement. However, the properties sector overreacted, with the stock price plunging by 11.01% following a slight positive abnormal return on the day of the announcement. Most sectors responded positively, with the technology sector yielding a remarkable 24.54% throughout the event window. On the other hand, the financial, energy, and transportation sectors all suffered declines, with the property sector suffering the most, with a decline of 22.01%.

Most sectors expressed support for the free vaccination campaign (Table 14 ). The reaction on the announcement day was moderate, with an expected significant increase in the healthcare sector’s stock price. Additionally, only the basic material and consumer cyclical sectors showed significant abnormal returns during the anticipation window, at − 6.23% and 7.03%, respectively. Moreover, a signal of underreaction was observed in the technology sector, with the stock price increasing significantly by 29.66% inside the adjustment window. In comparison, we concluded that the considerable reduction in basic material was unrelated to the intervention since the intervention was irrelevant to the sector. Overall, eight out of eleven sectors responded positively to the announcement, with the healthcare and technology sectors bearing the brunt of the impact.

The Indonesian stock market remained unaffected by the free vaccination campaign. The reaction was entirely contradictory for that in the United Kingdom and the United States, where the FTSE100 and Dow Jones both increased by around 5% on the day of the announcement. Additionally, in some cases, both countries have seen a substantial improvement in the stock prices of airlines, hotels, and energy companies, in some cases by more than 40% (Jack 2020 ).

Furthermore, Rouatbi et al. ( 2021 ) indicated that free vaccination campaigns had decreased global stock market volatility. According to the study, a 10% rise in vaccination might result in a 0.245% positive reaction in the stock market. However, adoption in emerging economies was regarded to be a hurdle to gaining investor confidence. A significant divide between emerging and developed countries could hamper economic growth, as The Economist ( 2021 ) estimated that emerging countries would not have widespread access to vaccines until 2023. Emerging economies such as Indonesia and India only have fully vaccinated rates of 11.2% and 9.2%, respectively. In contrast, developed economies such as the United Kingdom and the United States have already surpassed the 50% mark. As a result, the vaccination campaign is a critical factor in a country's economic recovery from the uncertainty during the COVID-19 pandemic.

Summary and conclusions

This paper examined the short-term impact of government interventions on the industrial sectors of the Indonesian stock market during the COVID-19 pandemic, and the analysis focused on the following five types of interventions: economic stimulus packages, job creation law, Jakarta lockdowns, Ramadan travel restrictions, and free vaccination campaign. The results from the proposed event study analysis indicated that the initial economic stimulus package was critical in reviving the stock market following its collapse to a 7-years low in March 2020. It was also observed that the combination of household and corporate support was the most powerful economic stimulus package. In contrast, the enactment of ****the jobs creation law ushered in a new era of hope for the Indonesian bureaucracy. Although the authorisation had a minor influence on several sectors, it was anticipated that the law would benefit the country’s economy in the future. Furthermore, the Jakarta lockdowns had no noticeable impact on any industrial sector in Indonesia. Indonesia was the last major country in the Southeast Asia region to impose lockdown, which indicated that the Indonesian stock market had experienced spillover impact from other countries’ announcements. Additionally, the September and July lockdowns were less severe than the initial one, showing that the COVID-19 situation affected investors’ behaviour, likely resulting in market resistance during the COVID-19 pandemic. Whereas Ramadan travel restrictions had historically resulted in significant negative sentiment in global stock markets, it was not the case in Indonesia and the announcement had no discernible impact on any industrial sector amid the pandemic. Additionally, the stock market was unaffected negatively by the recurrence of Ramadan travel restrictions in 2021. It was also anticipated that a free vaccination campaign would benefit the Indonesian healthcare sector in the short run. Additionally, the announcement was well received by most sectors. However, it was believed that emerging countries’ lack of access to vaccines could impede gaining market trust. As a result, the intervention’s long-term impact is likely to rely on the government’s commitment to distribute vaccines.

In conclusion, the Indonesian government took a relatively conservative strategy by enforcing a series of government interventions to reduce COVID-19 transmission rates while also stabilising stock market volatility. This was reflected in the stock market’s rapid recovery with a monthly return of 6.53% in December 2020, which was a higher return than other major countries in Southeast Asia, including Singapore (1.13%), Malaysia (4,13%), Thailand (2.91%), and Philippines (5.13%) (Yahoo Finance 2021 ). Presumably the government had gained knowledge from prior disease outbreaks such as SARS and H1N1, in which Indonesia was involved in the fight against the virus. In addition, Indonesia had also experienced two financial crises in the past 25 years, including the 1997 Asian Financial Crisis and the 2008 Global Financial Crisis, which helped the current government take steps to prevent another crisis.

The event study analysis used in the paper limited the focus to immediate and short-term analysis. However, it would be interesting to examine the longer-term impact of the interventions implemented by the government and further extend the market model to include investor behaviour and political factors in quantifying the short-term impact of recurrence events. Furthermore, due to many interventions implemented in a tight timeframe, an analysis of the impact of combined interventions could also help policy makers to gain better understanding on the formation of effective portfolios of interventions in the event of a pandemic.

Availability of data and material (data transparency)

Publicly accessible data used in the research.

Code availability

Available on request.

Adekunle A, Meehan M, Rojas-Alvarez D, Trauer J, McBryde E (2020) Delaying the COVID-19 epidemic in Australia: evaluating the effectiveness of international travel bans. Aust N Z J Public Health 44(4):257–259

Article   Google Scholar  

Ashraf BN (2020) Stock markets’ reaction to COVID-19: Cases or fatalities? Res Int Bus Financ 54(1):101249

Barman A (2020) 5 Basic materials stocks to play the sector’s strong rebound. Nasdaq. https://www.nasdaq.com/articles/5-basic-materials-stocks-to-play-the-sectors-strong-rebound-2020-09-24

Barua S (2020) Understanding Coronanomics: the economic implications of the coronavirus (COVID-19) Pandemic. Available at SSRN 3566477

Bash A, Alsaifi K (2019) Fear from uncertainty: an event study of Khashoggi and stock market returns. J Behav Exp Financ 23(2019):54–58

Beirne J, Renzhi N, Sugandi E, Volz U (2021) COVID -19, asset markets and capital flows. Pac Econ Rev 2021:1–41

Google Scholar  

Blasco N, Corredor P, Ferreruela S (2012) Market sentiment: a key factor of investors’ imitative behaviour. Account Finance 52(3):663–689

Bloom E, de Wit V and Carangal - San Jose MJ (2005) Potential economic impact of an avian flu pandemic on Asia. Asian Development Bank. https://www.adb.org/sites/default/files/publication/28082/pb042.pdf

Boehmer E (1991) Event-study methodology under conditions of event-induced variance. J Financ Econ 30(2):253–272

Bouri E, Demirer R, Gupta R, Nel J (2021) COVID-19 pandemic and investor herding in international stock markets. Risks 9(9):168

BPS Statistics Indonesia (2020) Statistical Yearbook of Indonesia 2020. BPS Statistics Indonesia. Jakarta: BPS Statistics Indonesia. https://www.bps.go.id/publication/2020/04/29/e9011b3155d45d70823c141f/statistik-indonesia-2020.html

BPS Statistics Jakarta (2020) DKI Jakarta province in figures. BPS Statistics Jakarta. Jakarta: BPS Statistics Jakarta. https://jakarta.bps.go.id/publication/2020/04/27/20f5a58abcb80a0ad2a88725/provinsi-dki-jakarta-dalam-angka-2020.html .

Brown SJ, Warner JB (1985) Using daily stock returns. J Financ Econ 14(1):3–31

Brunnermeier MK (2005) Information leakage and market efficiency. Rev Financ Stud 18(2):417–457

Buigut S, Kapar B (2020) Effect of Qatar diplomatic and economic isolation on GCC stock markets: an event study approach. Financ Res Lett 37(2020):101352

CDC (2019a). 2009 H1N1 Pandemic (H1N1pdm09 virus). Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/flu/pandemic-resources/2009-h1n1-pandemic.html

CDC (2019b) Cost of the ebola epidemic. [online] CDC. https://www.cdc.gov/vhf/ebola/history/2014-2016-outbreak/cost-of-ebola.html

Chong TT-L, Liu X, Zhu C (2016) What explains herd behavior in the chinese stock market? SSRN Electron J 18(4):448–456

Cowling BJ, Ali ST, Ng TWY, Tsang TK, Li JCM, Fong MW, Liao Q, Kwan MY, Lee SL, Chiu SS, Wu JT, Wu P, Leung GM (2020) Impact assessment of non-pharmaceutical interventions against coronavirus disease 2019 and influenza in Hong Kong: an observational study. Lancet Public Health 5(5):e279–e288

Cribb RB, Ford M (eds) (2009) Indonesia beyond the water’s edge: managing an archipelagic state. Institute of Southeast Asian Studies, Singapore

Dolley JC (1933) Open market buying as a stimulant for the bond market. J Polit Econ 41(4):513–529

Dyckman T, Philbrick D, Stephan J (1984) A comparison of event study methodologies using daily stock returns: a simulation approach. J Account Res 22:1–30

Eleftheriou K and Patsoulis P (2020) COVID-19 lockdown intensity and stock market returns: a spatial econometrics approach. MPRA Paper, 100662. https://mpra.ub.uni-muenchen.de/100662/

Emeto TI, Alele FO, Ilesanmi OS (2021) Evaluation of the effect of border closure on COVID-19 incidence rates across nine African countries: an interrupted time series study. Trans R Soc Tropic Med Hygiene 115(10):1174–1183

Fan E (2003) SARS: Economic Impacts and Implications. ERD Policy Brief by Asian Development Bank. https://www.adb.org/publications/sars-economic-impacts-and-implications

Fane G, McLeod RH (2002) Banking Collapse and Restructuring in Indonesia, 1997–2001. 1997–2001. Cato J 22:277

Fu S, Liu C, Wei X (2021) Contagion in Global Stock Markets during the COVID-19 Crisis. Global Chall 2021(2000130):1–10

Goldstein P, Yeyati EL and Sartorio L (2021) Lockdown fatigue: the declining effectiveness of lockdowns. VoxEU.org. https://voxeu.org/article/declining-effectiveness-lockdowns

Gorbiano M and Akhlas A (2020) Indonesia announces Rp 405 trillion COVID-19 budget, anticipates 5% deficit in historic move. The Jakarta Post. https://www.thejakartapost.com/news/2020/03/31/indonesia-announces-rp-405-trillion-covid-19-budget-anticipates-5-deficit-in-historic-move.html . ( Accessed 2 Jul. 2021 )

Gupta H, Chaudhary R, Gupta S (2021) COVID-19 impact on major stock markets. FIIB Bus Rev 1(11):231971452199451

Hale T, Angrist N, Goldszmidt R, Kira B, Petherick A, Phillips T, Webster S, Cameron-Blake E, Hallas L, Majumdar S, Tatlow H (2021) A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat Hum Behav 5(4):529–538

Halimatussadiah A (2015) Why Ramadan is a special economic season in Indonesia. The Conversation. https://theconversation.com/why-ramadan-is-a-special-economic-season-in-indonesia-43399 . ( Accessed 2 Jul 2021 )

Hall M (2020) How the stock market affects GDP. Investopedia. https://www.investopedia.com/ask/answers/033015/how-does-stock-market-affect-gross-domestic-product-gdp.asp

Harjoto MA, Rossi F, Lee R, Sergi BS (2020a) How do equity markets react to COVID-19? Evidence from emerging and developed countries. J Econ Bus 1(1):105966

Harjoto MA, Rossi F, Paglia J (2020b) COVID-19: stock market reactions to the shock and the stimulus. SSRN Electron J 28(10):795–801

Haroon O, Rizvi SAR (2020) Flatten the curve and stock market liquidity—an inquiry into emerging economies. Emerg Mark Financ Trade 56(10):2151–2161

Huo X and Qiu Z (2020) How does China’s stock market react to the announcement of the COVID-19 pandemic lockdown? Economic and Political Studies, pp 1–26

Ighobor K (2014) Ebola threatens economic gains in affected countries | Africa Renewal. Un.org. https://www.un.org/africarenewal/magazine/december-2014/ebola-threatens-economic-gains-affected-countries

International Monetary Fund (IMF) (2020) World Economic Outlook: the great lockdown. International Monetary Fund, Washington

Indonesia COVID-19 Response Acceleration Task Force (2021) Daily cases in Indonesia. https://covid19.go.id/peta-sebaran

Indonesia Stock Exchange (2021) IDX Index fact sheet. PT Bursa Efek Indonesia. https://www.idx.co.id/en-us/market-data/statistical-reports/index-fact-sheet/

Jack S (2020) Covid-19: Global stock markets rocket on vaccine hopes. BBC News. 9 Nov. https://www.bbc.co.uk/news/business-54874108 . ( Accessed 10 Dec. 2020 )

James A, Plank MJ, Hendy S, Binny R, Lustig A, Steyn N, Nesdale A, Verrall A (2021) Successful contact tracing systems for COVID-19 rely on effective quarantine and isolation. PLoS One 16(6):e0252499

Jawaid A and Garrido F (2020) COVID-19 recession touches all 5 continents as recovery looks distant. www.spglobal.com . https://www.spglobal.com/marketintelligence/en/news-insights/latest-news-headlines/covid-19-recession-touches-all-5-continents-as-recovery-looks-distant-60168984 . ( Accessed 24 Jun. 2021 )

Jennings R (2020) Indonesia’s pandemic response: a law to create millions of jobs | Voice of America - English. VOA News. https://www.voanews.com/economy-business/indonesias-pandemic-response-law-create-millions-jobs . ( Accessed 2 Jul. 2021 )

Khatatbeh H, Abu-Alfoul (2020) The impact of COVID-19 pandemic on global stock markets: an event study. Int J Econ Bus Adm 8(4):505–514

Kim Y-W, Yoon S-J, Oh I-H (2012) The economic burden of the 2009 pandemic H1N1 influenza in Korea. Scand J Infect Dis 45(5):390–396

Koh WC, Naing L, Wong J (2020) Estimating the impact of physical distancing measures in containing COVID-19: An empirical analysis. Int J Infect Dis 100:42–49. https://doi.org/10.1016/j.ijid.2020.08.026

Lansdorp S and Jellema W (2013) Momentum from Underreaction. https://www.robeco.com/media/6/5/5/65562514d3c17a4f69cf50f5963b2a0b_robeco-whitepaper-momentum-from-underreaction-2013-10_tcm20-11257.pdf

Li ML, Bouardi HT, Lami OS, Trikalinos TA, Trichakis NK and Bertsimas D (2020) Forecasting COVID-19 and analyzing the effect of government interventions. MedRxiv, pp 2020–2006

Liew V and Puah C-H (2020) Chinese stock market performance in the time of novel coronavirus pandemic. MPRA (100414). https://mpra.ub.uni-muenchen.de/100414/

Liu H, Manzoor A, Wang C, Zhang L, Manzoor Z (2020) The COVID-19 outbreak and affected countries stock markets response. Int J Environ Res Public Health 17(8):2800

Lyons C (1999) Definiteness. Cambridge University Press

Book   Google Scholar  

MacKellar L (2007) Pandemic influenza: a review. Popul Dev Rev 33(3):429–451

Masoud N (2013) The impact of stock market performance upon economic growth. Int J Econ Financ Issues 3(4):788–798

Maulia E (2021) Indonesia bourse launches tech classification to lure investors. Nikkei Asia. https://asia.nikkei.com/Business/Markets/Indonesia-bourse-launches-tech-classification-to-lure-investors . ( Accessed 8 Jul. 2021 )

Molina K and Ramadhan IF (2020) COVID-19 response of the indonesian government on funding and financial support measures. White & Case LLP. https://www.whitecase.com/publications/alert/covid-19-indonesian-government-financial-assistance-measures

Murphy K (2008) Swan’s $4bn injection for small lenders. The Sydney Morning Herald. https://www.smh.com.au/national/swans-4bn-injection-for-small-lenders-20080926-4ovd.html . ( Accessed 15 Aug. 2021 )

Numbeo (2021) Cost of living. Numbeo.com. https://www.numbeo.com/cost-of-living/in/Singapore

Omoleke SA, Mohammed I, Saidu Y (2016) Ebola viral disease in West Africa: a threat to global health, economy and political stability. J Public Health Afr. https://doi.org/10.4081/jphia.2016.534

Oxford Business Group (2020) Can Indonesia’s Omnibus Law drive its Covid-19 economic recovery? Oxford Business Group. https://oxfordbusinessgroup.com/news/can-indonesia-s-omnibus-law-drive-its-covid-19-economic-recovery . ( Accessed 5 Jul 2021 )

Oxford Economics (2020) The economic impact of Covid19 on Asia Pacific. Oxford Economics, p 14. https://www.oxfordeconomics.com/resource/the-economic-impact-of-covid-19-on-asia-pacific/

Ozili PK and Arun T (2020) Spillover of COVID-19: impact on the Global Economy. Available at SSRN 3562570

Powell A (2021) Vaccines should end the pandemic, despite the variants, say experts. Harvard Gazette. https://news.harvard.edu/gazette/story/2021/02/vaccines-should-end-the-pandemic-despite-the-variants-say-experts/

Randall LK, David (2021) Infrastructure bill could fuel stocks, but investors temper enthusiasm. Reuters. 25 Mar. https://www.reuters.com/article/us-usa-stocks-infrastructure-analysis-idUSKBN2BH0G1 . ( Accessed 20 Aug 2021 )

Rassy D, Smith RD (2013) The economic impact of H1N1 on Mexico’s tourist and pork sectors. Health Econ 22(7):824–834

Reuters (2020) Indonesia targets 5.6–6.2% average GDP growth to 2024. Reuters. https://www.reuters.com/article/us-indonesia-economy-idUSKBN1Z516O . ( Accessed 6 Jul 2021 )

Reuters (2021) Stock market headlines | breaking stock market news. Reuters. https://www.reuters.com/markets/stocks/asia-pacific/

Ritter JR (1991) The long-run performance of initial public offerings. J Financ 46(1):3–27

Rose A (2021) COVID-19 economic impacts in perspective: a comparison to recent U.S. disasters. Int J Disaster Risk Reduct 60:102317

Rouatbi W, Demir E, Kizys R, Zaremba A (2021) Immunizing markets against the pandemic: COVID–19 vaccinations and stock volatility around the world. SSRN Electron J. https://doi.org/10.2139/ssrn.3815000

Sánchez JM and Liborio CS (2012) The relationships among changes in GDP, employment, and unemployment: this time, it’s different. research.stlouisfed.org. https://research.stlouisfed.org/publications/economic-synopses/2012/05/18/the-relationships-among-changes-in-gdp-employment-and-unemployment-this-time-its-different

Scherf M, Matschke X, Rieger MO (2022) Stock market reactions to COVID-19 lockdown: a global analysis. Fin Res Lett 45:102245

Scholes M, Williams J (1977) Estimating betas from nonsynchronous data. J Financ Econ 5(3):309–327

Steyn N, Plank MJ, James A, Binny RN, Hendy SC, Lustig A (2021) Managing the risk of a COVID-19 outbreak from border arrivals. J Royal Soc Interface 18(177):20210063

Su Y, Yip Y, Wong RW (2001) The impact of government intervention on stock returns evidence from Hong Kong. Int Rev Econ Financ 11(2002):277–297

Swaine J (2008) Bank bailout: Alistair Darling unveils ?500billion rescue package. Telegraph.co.uk. https://www.telegraph.co.uk/finance/financialcrisis/3156711/Bank-bailout-Alistair-Darling-unveils-500billion-rescue-package.html

The Economist (2021) More than 85 poor countries will not have widespread access to coronavirus vaccines before 2023. Economist Intelligence Unit. https://www.eiu.com/n/85-poor-countries-will-not-have-access-to-coronavirus-vaccines/

The World Bank (2016) The economic impact of the 2014 ebola epidemic. https://openknowledge.worldbank.org/bitstream/handle/10986/20592/9781464804380.pdf . ( Accessed 12 Aug 2021 )

The World Bank (2020) Reforms in Indonesia’s financial sector open new economic opportunities. World Bank. https://www.worldbank.org/en/news/press-release/2020b/03/23/reforms-in-indonesias-financial-sector-open-new-economic-opportunities

United Nations (2021) World population prospects—population division—United Nations. un.org. https://population.un.org/wpp/

US Bank Asset Management Group (2021) How the COVID vaccine affects the economy | U.S. Bank. www.usbank.com . https://www.usbank.com/investing/financial-perspectives/market-news/how-does-the-covid-vaccine-affect-the-economy.html . Accessed 2 Jul 2021

Valle-Cruz D, Fernandez-Cortez V, López-Chau A, Sandoval-Almazán R (2022) Does twitter affect stock market decisions? financial sentiment analysis during pandemics: a comparative study of the h1n1 and the covid-19 periods. Cogn Comput 14(1):372–387

WHO (2021) WHO coronavirus disease (COVID-19) dashboard. Who.int. https://covid19.who.int/table

Wight A (2020) Will the coronavirus follow when 20 million indonesians go home for Mudik? Forbes. https://www.forbes.com/sites/andrewwight/2020/04/08/will-the-coronavirus-follow-when-20-million-indonesians-go-home-for-mudik/?sh=d1eb562fc13c . ( Accessed 2 Jul. 2021 )

World Bank (2018) Indonesian overview. World Bank. https://www.worldbank.org/en/country/indonesia/overview

Yahoo Finance (2021) Historical stock market data. Accessed through Yahoo Finance API

Zaremba A, Kizys R, Aharon DY, Demir E (2020) Infected markets: novel coronavirus, government interventions, and stock return volatility around the globe. Financ Res Lett 1(1):101597

Zaremba A, Aharon DY, Demir E, Kizys R, Zawadka D (2021) COVID-19, government policy responses, and stock market liquidity around the world: a note. Res Int Bus Financ 56(2021):101359

Download references

Not applicable.

Author information

Authors and affiliations.

Alliance Manchester Business School, The University of Manchester, Manchester, M15 6PB, UK

Josua Sinaga, Ting Wu & Yu-wang Chen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Yu-wang Chen .

Ethics declarations

Conflict of interest.

The authors declare that they have no conflict of interest.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 79 kb)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Sinaga, J., Wu, T. & Chen, Yw. Impact of government interventions on the stock market during COVID-19: a case study in Indonesia. SN Bus Econ 2 , 136 (2022). https://doi.org/10.1007/s43546-022-00312-4

Download citation

Received : 24 December 2021

Accepted : 26 July 2022

Published : 17 August 2022

DOI : https://doi.org/10.1007/s43546-022-00312-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Financial markets
• Emerging country
• Stock return
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 13 May 2024

The impact of the world’s first regulatory, multi-setting intervention on sedentary behaviour among children and adolescents (ENERGISE): a natural experiment evaluation

• Bai Li   ORCID: orcid.org/0000-0003-2706-9799 1 ,
• Selene Valerino-Perea 2 ,
• Weiwen Zhou 3 ,
• Yihong Xie 4 ,
• Keith Syrett 5 ,
• Remco Peters 1 ,
• Zouyan He 4 ,
• Yunfeng Zou 4 ,
• Frank de Vocht 6 , 7 &
• Charlie Foster 1  

International Journal of Behavioral Nutrition and Physical Activity volume  21 , Article number:  53 ( 2024 ) Cite this article

507 Accesses

122 Altmetric

Metrics details

Regulatory actions are increasingly used to tackle issues such as excessive alcohol or sugar intake, but such actions to reduce sedentary behaviour remain scarce. World Health Organization (WHO) guidelines on sedentary behaviour call for system-wide policies. The Chinese government introduced the world’s first nation-wide multi-setting regulation on multiple types of sedentary behaviour in children and adolescents in July 2021. This regulation restricts when (and for how long) online gaming businesses can provide access to pupils; the amount of homework teachers can assign to pupils according to their year groups; and when tutoring businesses can provide lessons to pupils. We evaluated the effect of this regulation on sedentary behaviour safeguarding pupils.

With a natural experiment evaluation design, we used representative surveillance data from 9- to 18-year-old pupils before and after the introduction of the regulation, for longitudinal ( n  = 7,054, matched individuals, primary analysis) and repeated cross-sectional ( n  = 99,947, exploratory analysis) analyses. We analysed pre-post differences for self-reported sedentary behaviour outcomes (total sedentary behaviour time, screen viewing time, electronic device use time, homework time, and out-of-campus learning time) using multilevel models, and explored differences by sex, education stage, residency, and baseline weight status.

Longitudinal analyses indicated that pupils had reduced their mean total daily sedentary behaviour time by 13.8% (95% confidence interval [CI]: -15.9 to -11.7%, approximately 46 min) and were 1.20 times as likely to meet international daily screen time recommendations (95% CI: 1.01 to 1.32) one month after the introduction of the regulation compared to the reference group (before its introduction). They were on average 2.79 times as likely to meet the regulatory requirement on homework time (95% CI: 2.47 to 3.14) than the reference group and reduced their daily total screen-viewing time by 6.4% (95% CI: -9.6 to -3.3%, approximately 10 min). The positive effects were more pronounced among high-risk groups (secondary school and urban pupils who generally spend more time in sedentary behaviour) than in low-risk groups (primary school and rural pupils who generally spend less time in sedentary behaviour). The exploratory analyses showed comparable findings.

Conclusions

This regulatory intervention has been effective in reducing total and specific types of sedentary behaviour among Chinese children and adolescents, with the potential to reduce health inequalities. International researchers and policy makers may explore the feasibility and acceptability of implementing regulatory interventions on sedentary behaviour elsewhere.

The growing prevalence of sedentary behaviour in school-aged children and adolescents bears significant social, economic and health burdens in China and globally [ 1 ]–[ 3 ]. Sedentary behaviour refers to any waking behaviour characterised by an energy expenditure equal or lower than 1.5 metabolic equivalents (METs) while sitting, reclining, or lying [ 3 ]. Evidence from systematic reviews, meta-analyses and longitudinal studies have shown that excessive sedentary behaviour, in particular recreational screen-based sedentary behaviour, affect multiple dimensions of children and adolescents’ wellbeing, spanning across mental health [ 4 ], cognitive functions/developmental health/academic performance [ 5 ], [ 6 ], quality of life [ 7 ], and physical health [ 8 ]. In China, over 60% of school pupils use part of their sleep time to play mobile phones/digital games and watch TV programmes, and 27% use their sleep time to do homework or other learning activities [ 9 ]. Screen-based, sedentary entertainment has become the leading cause for going to bed late, which is linked to detrimental consequences for children’s physical and mental health [ 10 ]. Notably, academic-related activities such as post-school homework and off campus tutoring also contribute to the increasing amounts of sedentary behaviour. According to the Organisation for Economic Co-operation and Development (OECD) report, China is the leading country in time spent on homework by adolescents (14 h/week on average) [ 11 ].

The COVID-19 pandemic exacerbated this global challenge, with children and adolescents reported to have been the most affected group [ 12 ]. Schools are a frequently targeted setting for interventions to reduce sedentary behaviour [ 13 ]. However, school-based interventions have had limited success when delivered under real-world conditions or at scale [ 14 ]. School-based interventions alone have also been unsuccessful in mitigating the trend of increasing sedentary behaviour that is driven by a complex system of interdependent factors across multiple sectors [ 13 ]. Even for parents and carers who intend to restrict screen-based sedentary behaviour and for children who wish to reduce screen-based sedentary behaviour, social factors including peer pressure often form barriers to changing behaviour [ 15 ]. In multiple public health fields such as tobacco control and healthy eating promotion, there has been a notable shift away from downstream (e.g., health education) towards an upstream intervention approach (e.g., sugar taxation). However, regulatory actions for sedentary behaviour are scarce [ 16 ]. World Health Organization (WHO) 2020 guidelines on sedentary behaviour encourage sustainable and scalable approaches for limiting sedentary behaviour and call for more system-wide policies to improve this global challenge [ 8 ]. Up-stream interventions can act on sedentary behaviour more holistically and have the potential to maximise reach and health impact [ 13 ]. In response to this pressing issue, and to widespread demands from many parents/carers, the Chinese government introduced nationwide regulations in 2021 to restrict (i) the amount of homework that teachers can assign, (ii) when (and for how long) online gaming businesses can provide access to young people, and (iii) when tutoring businesses can provide lessons [ 17 ], [ 18 ]. Consultations with WHO officials and reviewers of international health policy interventions confirmed that this is currently the only government-led, multi-setting regulatory intervention on multiple types of sedentary behaviour among school-aged children and adolescents. A detailed description of this programme is available in the Additional File 1 .

We evaluated the impact of this regulatory intervention on sedentary behaviour in Chinese school-aged children and adolescents. We also investigated whether and how intervention effects differed by sex, education stage, geographical area, and baseline weight status.

Study design

The introduction of the nationwide regulation provided a unique opportunity for a natural experiment evaluation where the pre-regulation comparator group data (Wave 1) was compared to the post-regulation group data (Wave 2). Multiple components of the intervention (see Additional File 1 ) were introduced in phases from July 2021 with all components being fully in place by September 2021 [ 17 ], [ 18 ]. This paper follows the STROBE reporting guidance [ 19 ], [ 20 ].

Data source, study population and sampling

We obtained regionally representative data on 99,947 pupils who are resident in the Chinese province of Guangxi as part of Guangxi Centre for Disease Control and Prevention’s (CDC) routine surveillance. The data, available from participants in grade 4 (aged between 9 and 10 years) and higher, were collected using a multi-stage random sampling design (Fig.  1 ) through school visits by trained health professionals following standardised protocols (see Supplementary Fig.  1 , Additional File 1 ). In Wave 1 (data collected from September to November 2020), pupils were randomly selected from schools in 31 urban/rural counties from 14 cities in Guangxi. At least eight schools, including primary, secondary, high schools, and ‘vocational high schools’, were selected from urban counties. Five schools were selected from rural counties. Approximately 80 students were randomly selected from each grade at the schools selected. The same schools were invited to participate in Wave 2 (data collected from September to November 2021), and new schools were invited to replace Wave 1 schools that no longer participated. Children with available data at both Wave 1 and Wave 2 represented approximately 10% of the sample ( n  = 7,587). Paper-based questionnaires were administrated to students by trained personnel or teachers. The questionnaires were designed and validated by China National Health Commission, and have been utilised in routine surveillance throughout the country.

Flow diagram of participants included in the ENERGISE study

We used data from the age groups 7–18 years for most analyses. For specific analyses of homework and out-of-campus tutoring, we excluded high school pupils (16–18 years) because the homework and out-of-campus tutoring regulations apply to primary (7–12 years) and middle (13–15 years) school pupils only. Furthermore, participants without socio-demographic data or those who reported medical history of disease, or a physical disability were excluded. This gave us a total sample of 7,054 eligible school-aged children and adolescents with matching data (longitudinal sample).

Outcomes and subgroups

Guangxi CDC used purposively designed questions for surveillance purposes to assess sedentary behaviour outcomes (Table  1 ).

The primary outcomes of interest included: (1) total sedentary behaviour time, (2) homework time, (3) out-of-campus learning (private tutoring) time, and (4) electronic device use time (Table  1 ). We considered electronic device use time, including mobile phones, handheld game consoles, and tablets, the most suitable estimator of online game time (estimand) in the surveillance programme since these are the main devices used for online gaming in China [ 23 ]. Secondary outcomes were: (1) total screen-viewing time, (2) internet-use time, (3) likelihood of meeting international screen-viewing time recommendations, and (4) likelihood of meeting the regulation on homework time (Table  1 ).

We calculated total sedentary behaviour time as the sum of total screen-viewing time (secondary outcome), homework time, and out-of-campus learning time (Table  1 ). Total screen-viewing time represents the sum of electronic device use time per day, TV/video game use time per day, and computer use time per day (Table  1 ). Total screen-viewing time was considered as an alternative estimator of online game time (estimand) since TV/videogame console use time and computer time could also capture the small proportion of children who use these devices for online gaming (Table  1 ). The international screen-viewing time recommendations were based on the American Academy of Paediatrics guidelines [ 21 ]. We did not include internet use time (secondary outcome) in total screen-viewing time, and total sedentary behaviour time, because this measure likely overlaps with other variables.

We defined subgroups by demographic characteristics, including the child’s sex (at birth: girls or boys), date of birth, education stage [primary school or secondary school [including middle school, high school, and ‘occupational schools’]), children’s residency (urban versus rural) and children’s baseline weight status (non-overweight versus overweight/obesity). Each sampling site selected for the survey was classified by the surveillance personnel as urban/rural and as lower-, medium-, or higher-economic level based on the area’s gross domestic product (GDP) per capita. The area’s GDP per capita was measured by the Chinese Centre for Disease Control and Prevention (CDC). Trained personnel also measured height, and weight using calibrated stadiometers and scales. Children’s weight/height were measured with light clothing and no shoes. Measurements during both waves were undertaken when students lived a normal life (no lockdowns, school were opened normally). We classified weight status (normal weight vs. overweight/obesity) according to the Chinese national reference charts [ 24 ].

Statistical analyses

We treated sedentary behaviour values that exceeded 24-hours per day as missing. We did not exclude extreme values for body mass index from the analyses 25 . Additional information, justifications, and results of implausible and missing values can be found in the Supplementary Table 1 , Additional File 1 .

The assumptions for normality and heteroscedasticity were assessed visually by inspecting residuals. We assessed multicollinearity via variance inflation factors. The outcome variables for linear regression outcomes were transformed using square roots to meet assumptions. We reported descriptive demographic characteristics (age, sex, area of residence, socioeconomic status), weight status, and outcome variables using means (or medians for non-normally distributed data) and proportions [ 26 ]

We ran multilevel models with random effects nested at the school and child levels to compare the outcomes in Wave 1 against Wave 2. We developed separate models for each sedentary behaviour outcome variable. We treated the introduction of the nationwide regulation as the independent binary variable (0 for Wave 1 and 1 for Wave 2). We ran linear models for continuous outcomes, logistic models for binary outcomes, and ordered logistic models for ordinal outcomes in a complete case analysis estimating population average treatment effects [ 27 ]. For the main analysis, in which participants had measurements in both Waves (longitudinal sample), only those with non-missing data at both time points were included.

We estimated marginal effects for each sedentary behaviour outcome. With a self-developed directed acyclic graph (DAG) we identified age (continuous), sex (male/female), area of residence (urban/rural), and socioeconomic status (high/medium/low) as confounders (see Supplementary Figs. 2–4, Additional File 1 ).

We evaluated subgroup effects defined by child’s sex at birth (boys versus girls), child’s stage of education (primary school versus secondary school [including middle school, high school, and ‘occupational schools’]), children’s residency (rural versus urban), and children’s baseline weight status (non-overweight versus overweight/obesity). We also repeated the covariate-adjusted model with interaction terms (between Wave and sex; Wave and child stage of education; Wave and residency; and Wave and weight status). We adjusted for multiple testing using Bonferroni correction ( p 0.05 divided by the number of performed tests for an outcome). The resulting cut-off point of p  < 0.005 was used to determine the presence of any interaction effects.

We also conducted exploratory analyses (including subgroup analyses) by evaluating the same models with a representative, cross-sectional sample of 99,947 pupils. This cross-sectional sample included different schools and children at Wave 1 and Wave 2. We therefore used propensity score (PS) weighting to account for sample imbalances in the socio-demographic characteristics. Propensity scores were calculated by conducting a logistic regression, which calculated the likelihood of each individual to be in Wave 2 (dependent variable). Individual’s age, sex, area of residence and the GDP per area were treated as independent variables. Subsequently, inverse probability of treatment weighting was applied to balance the demographic characteristics in the sample in Wave 1 (unexposed to the regulatory intervention) and Wave 2 (exposed to the regulatory intervention). The sample weight for individuals in Wave 1 were calculated using the Eq. 1/ (1-propensity score). The sample weight for individuals in Wave 2 were calculated using the Eq. 1/propensity score [ 28 ].

We only ran linear models for continuous outcomes since it was not possible to run PS-weighted multilevel models with this sample size in Stata. We conducted all statistical analyses in Stata version 16.0.

Participant sample

In our primary, longitudinal analyses, we analysed data from 7,054 children and adolescents. The mean age was 12.3 years (SD, 2.4) and 3,477 (49.3%) were girls (Table  2 ). More detailed information on characteristics of subgroups in the longitudinal sample are presented in the Supplementary Tables 2–5, Additional File 2 .

Primary outcomes

Children and adolescents reported a reduction in their daily mean total sedentary behaviour time by 13.8% (95% CI: -15.9 to -11.7), or 46 min, on average between Waves 1 and 2. Participants were also less likely to report having increased their time spent on homework (adjusted odd ratio/AOR: 0.39; 95% CI: 0.35–0.43) and in out-of-campus learning (AOR: 0.53; 95% CI: 0.47 to 0.59) in Wave 2 in comparison to Wave 1, respectively (Tables  3 and 4 ). We did not find any changes in electronic device use time.

Secondary outcomes

Participants reported reducing their mean daily screen-viewing time by 6.4% (95% CI: -9.6 to -3.3%), or 10 min, on average (Tables  3 and 4 ). Participants were also 20% as likely to meet international screen time recommendations (AOR: 1.20; 95% CI: 1.09 to 1.32) and were 2.79 times as likely to meet the regulatory requirement on homework time (95% CI: 2.47 to 3.14) compared to the reference group (before the introduction of the regulation).

Subgroup analyses

Most screen- and study-related sedentary behaviour outcomes differed by education stage ( p  < 0.005) (see Supplementary Tables 6–13, Additional File 2 ), with the reductions being larger in secondary school pupils than in primary school pupils (Tables  3 and 4 , and Table  5 ). Only secondary school pupils reduced their total screen-viewing time (-8.4%; 95% CI: -12.4 to -4.3) and were also 1.41 times as likely to meet screen-viewing recommendations (AOR: 1.41; 95% CI: 1.23 to 1.61) at Wave 2 compared to Wave 1.

Conversely, at Wave 2, primary school pupils reported a lower likelihood of spending more time doing homework (AOR: 0.30; 95%: 0.26 to 0.34) than secondary school pupils (AOR: 0.58; 95% CI: 0.50 to 0.67) compared to their counterparts at Wave 1. At Wave 2, primary school pupils also had a higher likelihood of reporting meeting homework time recommendations (AOR: 3.61; 95% CI: 3.09 to 4.22) than secondary school pupils (middle- and high school) (AOR: 2.11; 95% CI: 1.74 to 2.56) compared to their counterparts at Wave 1 (Table  5 ). There was also a residence interaction effect ( p  < 0.001) in total sedentary behaviour time, with participants in urban areas reporting larger reductions (-15.3%; 95% CI: -17.8 to -12.7) than those in rural areas (-11.2%; 95% CI: -15.0 to -7.4). There was no evidence of modifying effects by children’s sex or baseline weight status (Tables  4 and 5 ).

Findings from the exploratory repeated cross-sectional analyses were similar to the findings of the main longitudinal analyses including total sedentary behaviour time, electronic device use time, total screen-viewing time and internet use time (see Supplementary Tables 14–23, Additional File 2 ).

Principal findings

Our study evaluated the impact of the world’s first regulatory, multi-setting intervention on multiple types of sedentary behaviour among school-aged children and adolescents in China. We found that children and adolescents reduced their total sedentary behaviour time, screen-viewing time, homework time and out-of-campus learning time following its implementation. The positive intervention effects on total screen-viewing time (-8.4 vs. -2.3%), and the likelihood of meeting recommendations on screen-viewing time (1.41 vs. 1.02 AOR) were more pronounced in secondary school pupils compared with primary school pupils. Intervention effects on total sedentary behaviour time (-15.3 vs. -11.2%) were more pronounced among pupils living in the urban area (compared to pupils living in the rural area). These subgroup differences imply that the regulatory intervention benefit more the groups known to have a higher rate of sedentary behaviour [ 29 ].

Interestingly, the observed reduction in electronic device use itself did not reach statistical significance following implementation of regulation. This could be viewed as a positive outcome if this is correctly inferred and not the result of reporting bias or measurement error. International data indicated that average sedentary and total screen time have increased among children due to the COVID-19 pandemic [ 12 ]. However, such interesting finding might be explained by the absence of lockdowns in Guangxi during both surveillance waves when most school-aged students outside China were affected by pandemic mitigation measures such as online learning.

Strengths and weaknesses

Our study has several notable strengths. This is the first study to evaluate the impact of multi-setting nationwide regulations on multiple types of sedentary behaviour in a large and regionally representative sample of children and adolescents. Still, to gain a more comprehensive view of the regulatory intervention on sedentary behaviour across China, similar evaluation research should be conducted in other regions of China. Furthermore, access to a rich longitudinal dataset allowed for more robust claims of causality. The available data also allowed us to measure the effect of the intervention on multiple sedentary behaviours including recreational screen-time and academic-related behaviours. Lastly, the large data set allowed us to explore whether the effect of the regulatory intervention varied across important subgroups, suggesting areas for further research and development.

Some limitations need to be taken into consideration when interpreting our findings. First, a common limitation in non-controlled/non-randomised intervention studies is residual confounding. We aimed to limit this by adjusting our analysis for confounders known to impact the variables of interest, but it is impossible to know whether important confounding may still have been present. With maturation bias, it is possible that secular trends are the cause for any observed effects. However, this seems unlikely in our study as older children may spend more time doing homework [ 23 ] and engage more in screen-viewing activities [ 30 ]. In this study, we observed reductions in these outcomes. The use of self-reported outcomes (social desirability bias) was a limitation and might have led to the intervention effects being over-estimated [ 13 ]. However, since our data were collected as part of a routine surveillance programme, pupils were unaware of the evaluation. This might mitigate reporting bias. In addition, the data were collected in Guangxi which might not representative of the whole population in China. Another limitation is using electronic device use time as a proxy measure of online gaming time. It is possible that electronic devices can be used for other purposes. However, mobile phones, handheld game consoles and tablets are the main devices used for online gaming. In this study, electronic device use time provided a practical means of assessing the broad effects of regulatory measures on screen time behaviours, including online gaming, in a large (province level) surveillance programme. In the future, instruments specifically designed to capture online gaming behaviour should be used in surveillance and research work.

Comparisons with other studies

Neither China nor other countries globally have previously implemented and evaluated multi-setting regulatory interventions on multiple types of sedentary behaviour, which makes comparative discussions challenging. In general, results of health behaviour research over the past decades have shown that interventions that address structural and environmental determinants of multiple behaviours to be more effective in comparison with individual-focussed interventions [ 31 ]. Furthermore, the continuous and universal elements of regulatory interventions may be particularly important explanations for the observed reductions in sedentary behaviour. Standalone school and other institution-led interventions may struggle with financial and logistic costs which threaten long-term implementation [ 13 ]. In contrast, the universality element of regulatory intervention can reduce or remove peer pressures and potential stigmatisation among children and teachers that are often associated with more selective/targeted interventions [ 24 ]. Our findings support WHO guidelines for physical activity and sedentary behaviour that encourage sustainable and scalable approaches for limiting sedentary behaviour and call for more system-wide policies to improve this global challenge[ 8 ].

Implications for future policy and research

Our study has important implications for future research and practice both nationally and internationally. Within China, future research should focus on optimising the implementation of the regulatory intervention through implementation research and assess long-term effects of the regulation on both behavioral and health outcomes. Internationally, our findings also provide a promising policy avenue for other countries and communities outside of China to explore the opportunities and barriers to implement such programmes on sedentary behaviour. This exploratory process could start with assessing how key stakeholders (including school-aged children, parents/carers, schoolteachers, health professionals, and policy makers) within different country contexts perceive regulatory actions as an intervention approach for improving health and wellbeing in young people, and how they can be tailored to fit their own contexts. Within public health domains, including healthy eating promotion, tobacco and alcohol control, regulatory intervention approaches (e.g., smoking bans and sugar taxation) have been adopted. However, regulatory actions for sedentary behaviour are scarce [ 19 ]. Within the education sector, some countries recently banned mobile phone use in schools for academic purpose [ 25 ]. While this implies potential feasibility and desirability of such interventions internationally, there is little research on the demand for, and acceptability of, multi-faceted sedentary behaviour regulatory interventions for the purpose of improving health and wellbeing. It will be particularly important to identify and understand any differences in perceptions and feasibility both within (e.g., public versus policy makers) and across countries of differing socio-cultural-political environments.

This natural experiment evaluation indicates that a multi-setting, regulatory intervention on sedentary behaviour has been effective in reducing total sedentary behaviour, and multiple types of sedentary behaviour among Chinese school-aged children and adolescents. Contextually appropriate, regulatory interventions on sedentary behaviour could be explored and considered by researchers and policy makers in other countries.

Data availability

Access to anonymised data used in this study can be requested through the corresponding author BL, subject to approval by the Guangxi CDC. WZ and SVP have full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Abbreviations

Centre for disease control and prevention

Directed acyclic graph

Gross domestic product

Metabolic equivalents

Organisation for Economic Co-operation and Development

Bao R, Chen S-T, Wang Y, Xu J, Wang L, Zou L, Cai Y. Sedentary Behavior Research in the Chinese Population: a systematic scoping review. Int J Environ Res Public Health 2020, 17(10).

Nguyen P, Le LK-D, Ananthapavan J, Gao L, Dunstan DW, Moodie M. Economics of sedentary behaviour: a systematic review of cost of illness, cost-effectiveness, and return on investment studies. Prev Med. 2022;156:106964.

Article   PubMed   Google Scholar  

World Health Organization. WHO guidelines on physical activity and sedentary behaviour. In. Geneva: World Health Organization; 2020.

Zhang J, Yang SX, Wang L, Han LH, Wu XY. The influence of sedentary behaviour on mental health among children and adolescents: a systematic review and meta-analysis of longitudinal studies. J Affect Disord. 2022;306:90–114.

Madigan S, Browne D, Racine N, Mori C, Tough S. Association between Screen Time and children’s performance on a Developmental Screening Test. JAMA Pediatr. 2019;173(3):244–50.

Article   PubMed   PubMed Central   Google Scholar  

Pagani LS, Fitzpatrick C, Barnett TA, Dubow E. Prospective Associations between Early Childhood Television Exposure and academic, psychosocial, and Physical Well-being by Middle Childhood. Arch Pediatr Adolesc Med. 2010;164(5):425–31.

Boberska M, Szczuka Z, Kruk M, Knoll N, Keller J, Hohl DH, Luszczynska A. Sedentary behaviours and health-related quality of life. A systematic review and meta-analysis. Health Psychol Rev. 2018;12(2):195–210.

Fiona CB, Salih SA-A, Stuart B, Katja B, Matthew PB, Greet C, Catherine C, Jean-Philippe C, Sebastien C, Roger C et al. World Health Organization 2020 guidelines on physical activity and sedentary behaviour. British Journal of Sports Medicine 2020, 54(24):1451.

China Sleep Research Association. Sleep White Paper of Chine People’s Health. In. Beijing, China; 2022.

Chaput J-P, Gray CE, Poitras VJ, Carson V, Gruber R, Olds T, Weiss SK, Gorber SC, Kho ME, Sampson M, et al. Systematic review of the relationships between sleep duration and health indicators in school-aged children and youth. Appl Physiol Nutr Metab. 2016;41(6):S266–82. (Suppl. 3)).

OECD. Does Homework Perpetuate inequities in Education? OECD Publishing 2014(46):4.

Trott M, Driscoll R, Iraldo E, Pardhan S. Changes and correlates of screen time in adults and children during the COVID-19 pandemic: a systematic review and meta-analysis. eClinicalMedicine 2022, 48.

van Sluijs EMF, Ekelund U, Crochemore-Silva I, Guthold R, Ha A, Lubans D, Oyeyemi AL, Ding D, Katzmarzyk PT. Physical activity behaviours in adolescence: current evidence and opportunities for intervention. Lancet. 2021;398(10298):429–42.

Cassar S, Salmon J, Timperio A, Naylor P-J, van Nassau F, Contardo Ayala AM, Koorts H. Adoption, implementation and sustainability of school-based physical activity and sedentary behaviour interventions in real-world settings: a systematic review. Int J Behav Nutr Phys Activity. 2019;16(1):120.

Article   Google Scholar  

Martins J, Costa J, Sarmento H, Marques A, Farias C, Onofre M, Valeiro MG. Adolescents’ perspectives on the barriers and facilitators of physical activity: an updated systematic review of qualitative studies. Int J Environ Res Public Health 2021, 18(9).

Gelius P, Messing S, Tcymbal A, Whiting S, Breda J, Abu-Omar K. Policy Instruments for Health Promotion: a comparison of WHO Policy Guidance for Tobacco, Alcohol, Nutrition and Physical Activity. Int J Health Policy Manage. 2022;11(9):1863–73.

Google Scholar  

The General Office of the CPC Central Committee and the General Office of the State. Council issued the opinions on further reducing the Burden of Homework and off-campus training for students in the stage of Compulsory Education. https://www.gov.cn/zhengce/2021-07/24/content_5627132.htm .

Notice of the State Press and Publication Administration on Further Strict. Management to Effectively Prevent Minors from Being Addicted to Online Games. https://www.gov.cn/zhengce/zhengceku/2021-09/01/content_5634661.htm .

Craig P, Cooper C, Gunnell D, Haw S, Lawson K, Macintyre S, Ogilvie D, Petticrew M, Reeves B, Sutton M, et al. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Commun Health. 2012;66(12):1182–6.

Craig P, Campbell M, Bauman A, Deidda M, Dundas R, Fitzgerald N, Green J, Katikireddi SV, Lewsey J, Ogilvie D, et al. Making better use of natural experimental evaluation in population health. BMJ. 2022;379:e070872.

American Academy of Pediatrics. Children, adolescents, and television. Pediatrics. 2001;107(2):423–6.

Bauer CP. Applied Causal Analysis (with R). In. Bookdown; 2020.

Matthay EC, Hagan E, Gottlieb LM, Tan ML, Vlahov D, Adler NE, Glymour MM. Alternative causal inference methods in population health research: evaluating tradeoffs and triangulating evidence. SSM - Popul Health. 2020;10:100526.

Greenberg MT, Abenavoli R. Universal interventions: fully exploring their impacts and potential to produce Population-Level impacts. J Res Educational Eff. 2017;10(1):40–67.

Selwyn N, Aagaard J. Banning mobile phones from classrooms—An opportunity to advance understandings of technology addiction, distraction and cyberbullying. Br J Edu Technol. 2021;52(1):8–19.

Boushey CJ, Harris J, Bruemmer B, Archer SL. Publishing nutrition research: A review of sampling, sample size, statistical analysis, and other key elements of manuscript preparation, Part 2. J Acad Nutr Dietet. 2008;108(4):679–688.

Matthay EC, Hagan E, Gottlieb LM, Tan ML, Vlahov D, Adler NE, Glymour MM. Alternative causal inference methds in population health research:Evaluating tradeoffs nd triangulating evidence. SSM - Population Health. 2020;10:100526.

Chesnaye NC, Stel VS, Tripepi G, Dekker FW, Fu EL, Zoccali C, Jager KJ. An introduction to inverse probability of treatment weighting in observation research. Clin Kid J. 2021;15(1):14–20.

Song C, Gong W, Ding C, Yuan F, Zhang Y, Feng G, Chen Z, Liu A. Physical activity and sedentary behaviour among Chinese children agd 6-17 years: a cross-sectional analysis of 2010-2012 China National Nutrition and Health Survey. BMC Public Health. 2019;19(1):936.

Zhu X, Haegele JA, Tang Y, Wu X. Physical activity and sedentary behaviors of urban chinese children: grade level prevalence and academic burden associations. BioMed Res Int. 2017;2017:7540147.

Rutter H, Bes-Rastrollo M, de Henauw S, Lathi-Koski M, Lehtinen-Jacks S, Mullerova D, Rasmussen F, Rissanen A, Visscher TLS, Lissner L. Balancing upstream and downstream measures to tackle the obesity epidemic: a position statement from the european association for the study of obesity. Obesity Facts. 2017;10(11):61–63.

Download references

Acknowledgements

We would like to acknowledge Dr Peter Green and Dr Ruth Salway for providing feedback on the initial data analysis plan, and Dr Hugo Pedder and Lauren Scott who provided feedback on the statistical analyses.

This work was funded by the Wellcome Trust through the Global Public Health Research Strand, Elizabeth Blackwell Institute for Health Research. The funder of our study had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Author information

Authors and affiliations.

Centre for Exercise, Nutrition and Health Sciences, School for Policy Studies, University of Bristol, Bristol, UK

Bai Li, Remco Peters & Charlie Foster

Public Health Wales, Cardiff, UK

Selene Valerino-Perea

Department of Nutrition and School Health, Guangxi Center for Disease Control and Prevention, Nanning, Guangxi, China

Weiwen Zhou

School of Public Health, Guangxi Medical University, Nanning, Guangxi, China

Yihong Xie, Zouyan He & Yunfeng Zou

Centre for Health, Law, and Society, School of Law, University of Bristol, Bristol, UK

Keith Syrett

Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK

Frank de Vocht

NIHR Applied Research Collaboration West (ARC West), Bristol, UK

You can also search for this author in PubMed   Google Scholar

Contributions

BL conceived the study idea and obtained the funding with support from WZ, CF, KS, YX, YZ, ZH and RP. BL, CF, FdV and KS designed the study. WZ led data collection and provided access to the data. YX, SVP and ZH cleaned the data. SVP analysed the data with guidance from BL, FdV and CF. BL, SVP and RP drafted the paper which was revised by other authors. All authors read and approved the final manuscript for submission.

Corresponding author

Correspondence to Bai Li .

Ethics declarations

Ethical approval and consent to participate.

Ethics approvals were granted by the School for Policy Studies Research Ethics Committee at the University of Bristol (reference number SPSREC/20–21/168) and the Research Ethics Committee at Guangxi Medical University (reference number 0136). Written informed consent was obtained from each participant, and a parent or guardian for participants aged < 20 years.

Consent for publication

The co-authors gave consent for publication.

Competing interests

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential competing interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Li, B., Valerino-Perea, S., Zhou, W. et al. The impact of the world’s first regulatory, multi-setting intervention on sedentary behaviour among children and adolescents (ENERGISE): a natural experiment evaluation. Int J Behav Nutr Phys Act 21 , 53 (2024). https://doi.org/10.1186/s12966-024-01591-w

Download citation

Received : 20 December 2023

Accepted : 04 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s12966-024-01591-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sedentary behaviour
• Physical activity
• Regulatory intervention
• Health policy
• Screen time
• Natural experiment
• Mental health
• Health promotion
• Child health

International Journal of Behavioral Nutrition and Physical Activity

ISSN: 1479-5868

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

A .gov website belongs to an official government organization in the United States.

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• What is Advanced Molecular Detection (AMD)?
• AMD Scientific Superhero Posters
• About CDC's Advanced Molecular Detection Program
• Broad Agency Announcement Awards
• Success Stories
• COVID-19 Genomic Epidemiology Toolkit
• AMD Investments

Module 2.1 – SARS-CoV-2 sequencing in Arizona

What to know.

This case study from Arizona provides insight into how SARS-CoV-2 sequencing is used to describe the genomic epidemiology of a state and as an investigative tool in COVID-19 outbreak settings.

Presenter: Hayley Yaglom, MS, MPH | Genomic Epidemiologist, Translational Genomics Research Institute View Presentation [Full Version] | [Short Version]

You can read more about this work: An Early Pandemic Analysis of SARS-CoV-2 Population Structure and Dynamics in Arizona

Further Reading

• An early pandemic analysis of SARS-CoV-2 population structure and dynamics in Arizona. Ladner, et al. mBio , 2020.
• AZ-Strain: Genomic epidemiology of SARS-CoV-2 in Arizona . Arizona COVID Genomics Union (ACGU).

Additional Resources

• Communicating results using narratives.external icon Nextstrain.org (documentation).
• Coast-to-coast spread of SARS-CoV-2 during the early epidemic in the United States.external icon Fauver, et al. Cell , 2020.
• Large-scale sequencing of SARS-CoV-2 genomes from one region allows detailed epidemiology and enables local outbreak management.external icon Page, et al. Microb Genom , 2021.
• Viral genomes reveal patterns of the SARS-CoV-2 outbreak in Washington State.external icon Mueller, et al. Sci Transl Med , 2021.
• Revealing fine-scale spatiotemporal differences in SARS-CoV-2 introduction and spread.external icon Moreno, et al. Nat Commun , 2020.

AMD integrates next-generation genomic sequencing technologies with bioinformatics and epidemiology expertise to help us find, track, and stop pathogens.

For Everyone

Public health.