Poor geographic access can persist even when affordable and well-functioning health systems are in place, limiting efforts for universal health coverage (UHC). It is unclear how health facilities and community health workers contribute to achieving UHC. Using geographic information from thousands of patients in a rural district of Madagascar we evaluate how a health system strengthening (HSS) intervention aimed towards UHC affects the geography of primary care access. We find that facility-based interventions (user-fee exemptions, improved readiness) achieved high utilization rates only among populations who lived in close proximity to supported facilities. Scaling only facility-based HSS programs across the district would result in large gaps in health care access for the majority of the population. Community health provided major improvements in service utilization for children regardless of their distance from facilities. Our results have implications for UHC policies and suggest that greater emphasis on professionalized community health programs is essential.