Did It Work? Using Interrupted Time Series to Evaluate Health System Change

On a Tuesday in March 2025, Maternal and Child Health Aide, Nancy Koroma, opened the family planning register at Bunumbu Community Health Centre and blinked in surprise: adolescent family‑planning visits had doubled since the facility’s upgrade in infrastructure and medical supplies and a friendlier approach in service delivery in 2024. Family planning services are a cornerstone of women’s health, helping individuals and families make informed decisions about if and when to have children, decisions that can shape educational, economic, and health outcomes for years. But was this surge in visits just a lucky spike or the effect of a well-calculated intervention.

Moments like this are common in low-resource health systems, where change happens incrementally and often without the luxury of controlled trials. Yet, decision-makers still need reliable evidence to understand what is working. Interrupted Time Series (ITS) analysis helps fill this gap by using routine monthly data to estimate the real-world impact of interventions, making it a valuable tool when randomised trials are not possible.

What is Interrupted Time Series Analysis?

ITS is a type of quasi-experimental design that compares an outcome’s trajectory before and after a dated “interruption” such as a facility-level intervention. It is used to evaluate the impact of an intervention when a randomised controlled trial (RCT) is not feasible.

Here is how it works:

1. Track a key metric over time: for example, how many adolescents access family planning services each month.
2. Introduce an interruption:  this could be an intervention like a staff training program.
3. Compare trends before and after: does the trend shift? Accelerate? Flatten? Drop?
4. Contrast with the counterfactual: that is, what would have happened if the intervention had not occurred?

Why use ITS in Health Systems Research?

ITS is especially valuable in health system contexts where:

• Changes are rolled out over time, not all at once
• Where you already have routinely collected data (like facility registers or monthly service reports)

“We could walk into a district review meeting and point to the counterfactual trend and say: without this initiative, we would expect 25 fewer family‑planning visits for adolescents in January. This programme has closed that gap.” Saidu Wurie Jalloh, IfD Researcher.

A Real-World Example: Using ITS to Understand Sexual Reproductive Health Services in Sierra Leone

ITS has been central to the evaluation of the Foundations project in Sierra Leone, a multi-country initiative focused on improving adolescents’ access to sexual and reproductive health and rights (SRHR) services. In Sierra Leone, the project supports several facility-level interventions, so how could the project team know if these efforts improved access for adolescents?

That is where Interrupted Time Series comes in.

Together with the SickKids Centre for Global Child Health, the Institute for Development (IfD) tracked 39 primary health units from 2019 to 2025. Each PHU’s monthly counts of adolescent antenatal care (ANC) and family‑planning (FP) visits were modelled with ITS, pinpointing the start month of every facility-level intervention.

At Bunumbu Community Health Centre, for example, the data confirmed what Maternal and Child Health Aide Nancy Koroma had seen with her own eyes: a notable increase in adolescent family planning visits following the facility’s upgrade. But the numbers alone only tell part of the story. That is why researchers also conducted interviews with facility in-charges and SRHR providers. Their insights helped interpret what the data couldn’t show, such as changes in staff attitudes, confidence, or workload. This combination of routine data and lived experience helps build a more complete picture of what is working, what needs adjusting, and why?

Thinking about using ITS in your own work?

1. Start with the data you have, or know where to get it. ITS works best when you have access to regular, time-based data (monthly or quarterly) stretching across several years. This could be: Existing sources like facility registers
2. Map interventions. Knowing exactly when an intervention started (and what it involved) is key to identifying changes in trend or level.
3. Consider complementary methods. Interviews or qualitative insights can help explain why the trend changed or why it did not?

Interrupted Time Series gives us a powerful lens to evaluate change in dynamic health systems, even without experimental controls. In a country like Sierra Leone, where routine data is growing but still uneven, ITS can help turn that data into insight, but only if it is built into the intervention from the start. Planning for ITS early ensures that the necessary data is consistently collected and that the analysis can generate meaningful conclusions.

Used well, it can answer some of the most important questions in global health: Did it work? How did it work? And where do we go next?

Anaïs Bash-Taqi and Saidu Wurie Jalloh