The Problem of Many Hands
First defined in Outpaced by AI by Waydell D. Carvalho.
- Each party handles its piece correctly and assumes someone else holds the whole.
- No single actor has the integrated picture across all the sites or stages.
- Accountability is divided so finely that responsibility for the outcome lands nowhere.
Outcome: Every part is owned, and the whole is owned by no one.
A system that sits inside an org chart where every actor owns a piece of it can end up belonging to no one in particular. Dividing the work divides the accountability. The Problem of Many Hands is what is left when responsibility has been split so many ways that no single party owns the integrated question of whether the system is doing what it was built to do.
A health-risk algorithm is the case. Optum, the health-services arm of UnitedHealth Group, built a risk-prediction tool that hospitals used to decide which patients were enrolled in high-risk care management. Tools of that general type were applied to roughly 200 million people a year in the United States. The tool predicted need by predicting cost, and because the health system spends less on Black patients than on equally sick white patients, it scored equally sick Black patients as less in need of care.
Every party acted within its scope. Optum built and sold the model. The hospitals ran it. The doctors trusted the scores. The insurers used the output. The piece each one held was handled. What no one held was the whole: the pattern across hospital after hospital that would have shown the bias was systemic, not local. The harm was invisible at every individual point and visible only in the aggregate, and that aggregate did not exist anywhere until Ziad Obermeyer and colleagues assembled it from the outside and published it in Science in October 2019, years into the tool’s use.
That is the structure of the failure. When operation, oversight, and harm are distributed, each party can be diligent and the system can still fail, because diligence inside a slice does not add up to ownership of the whole. The gaps between the hands are where the failure lives, and the gaps are precisely the places no one was assigned to watch.
AI systems multiply the hands. A model is built by one team, trained on data from another, integrated by a third, deployed by a fourth, and overseen by a fifth, often across separate companies. Each can do its part well. The question of whether the whole system is producing the outcomes the institution needs belongs to no one by default. Someone has to be made to own the whole, or the many hands will hand the failure to each other.