AI Safety Is More Than a Technical Problem

When engineers talk about AI safety, they tend to picture model robustness, stress tests, red teams, benchmark scores, and edge cases that break a pipeline. But what about the cases where the pipeline works exactly as intended, and harm happens anyway? That question sat at the heart of a provocative session at the Global AI Summit in New Delhi, organized by the Association for Computing Machinery.

The session, titled "From Technical Safety to Societal Impact: Rethinking AI Governance," convened a multidisciplinary panel of researchers, policymakers, and civil society leaders to push on a growing tension in the field: the gap between what technical fixes can solve and what institutional, cultural, and political conditions actually shape.

When Safety Works and Still Fails

One of the panel's sharpest provocations was deceptively simple: a system can perform exactly as designed and still cause harm. Is that a technical failure? The panellists argued no, it's a failure of imagination and inclusion. The problem isn't necessarily the model; it's the framing that determines what the model is asked to do, whose experiences are considered in its design, and who bears the consequences of outcomes.

The session chairs were direct: the dominant discourse around AI safety, centred on red-teaming or benchmark performance, is too narrow. AI systems don't operate in a vacuum. They are embedded in complex institutional and political environments, and the risks they create are often invisible to the engineers who build them.

"AI fails not just due to model flaws, but because it is embedded in complex institutional and political systems."

Safety Is a Human Rights Issue

The panel was unequivocal that AI's impacts on human rights and democratic values must be part of any serious governance conversation. Technical safety, preventing hallucinations, adversarial attacks, or model collapse, is necessary but not sufficient. A system that accurately amplifies existing social biases, undermines freedoms, or is deployed in a context for which it was never designed can cause real damage while passing every benchmark on the board.

Panellists raised pointed examples: data centres being built in India that consume scarce local resources, including groundwater, while the communities living near them are excluded from any of the benefits. The political and extractive dimensions of AI infrastructure are, they argued, squarely within the scope of AI governance, even if they rarely appear in safety papers.

Introducing AI Metrology

One of the session's most substantive contributions was the proposed concept of AI Metrology, the science of measuring AI's actual societal impact. The session coincided with the announcement of a new ACM journal dedicated to the field.

The framing draws on the idea that AI systems function as "social machines," where technology and society are inextricably linked. Understanding the consequences of deploying AI at scale requires longitudinal studies, the kind of patient, empirical research that is structurally undervalued in a sector that celebrates speed.

Precision Over Platitudes

One panellist called for considerably more precision in the conversation about safety. The critique: model providers make sweeping claims about what their systems can safely do, without disclosing which languages, geographies, and contexts they don't cover. Real accountability would require explicit reporting of known gaps, not broad assurances that obscure them.

That call for precision extended to governance summits themselves. Several speakers warned against high-level convenings that produce only vague statements of good intent, without establishing mechanisms for accountability, recovery, or remuneration when things go wrong. "Moving from should to must" was a refrain, the recognition that voluntary norms, however well-meaning, have limits when no one faces consequences for ignoring them.

Who Is at the Table?

A recurring theme was that inclusion is who shapes AI governance and whose experience is centred when risks are assessed. The panel put it plainly: if the people designing AI systems and writing the rules around them are not diverse, the resulting ethics cannot be trusted. The persistent lack of women in senior AI leadership positions was raised as a direct contributor to embedded bias, not a side issue but a causal one.

Equally, the panel stressed the importance of Global South voices in shaping international governance frameworks. History, as one panellist noted bluntly, has shown that those with power and resources shape the narrative. The session was itself a modest corrective, held in New Delhi, drawing speakers from Africa, Asia, Europe, and North America, and explicitly centering civil society and social science perspectives alongside technical ones.

“Beyond its technical dimension, AI safety is in the use of AI in the hands of people, the data they use, and the ways they use it."

What Comes Next

The session's outcomes were partly conceptual and partly practical. On the conceptual side: a broader, more honest definition of AI safety that includes societal, institutional, ethical, and democratic dimensions. On the practical side: a call for governments to evaluate and govern AI systems based on real-world impact, not on speculative scenarios; mandating transparency and accountability for impacts of AI; socio-technical, multidisciplinary frameworks that bring social scientists and affected communities into the room; and stronger participation by Global South actors in the institutions that write global AI rules.

The launch of the ACM AI Metrology journal is a step in that direction, an institutional signal that measuring AI's societal effects deserves the same rigour that has long been applied to measuring its technical performance. Whether that signal translates into changed practice remains to be seen. The panel made clear it intends to keep asking.

On the Summit

The India AI Impact Summit 2026 took place in New Delhi from February 16-21, 2026, and marked a historic moment as the first global AI summit hosted by a Global South nation. The summit brought together an impressive array of participants, including 15-20 Heads of Government, over 50 international ministers, and more than 40 global and Indian CEOs, including tech leaders like OpenAI's Sam Altman and Alphabet's Sundar Pichai.

The summit was structured around three foundational pillars - People, Planet, and Progress - emphasizing that AI must serve humanity in all its diversity, align with environmental stewardship, and ensure equitable sharing of benefits. Over 100 countries engaged through seven working groups covering human capital, inclusion, trust, resilience, science, resources, and social good.

What distinguished this summit from previous AI gatherings was its deliberate shift away from abstract discussions of AI safety and governance toward practical impact, implementation, and measurable outcomes. A key focus was "small AI," i.e., practical, affordable AI solutions designed to run on everyday devices in settings with limited connectivity and infrastructure, reflecting the realities of many Global South contexts.

The summit drew over 300,000 attendees, ranging from schoolchildren and families to everyday Delhi residents, creating unprecedented public engagement with AI technology. However, the event faced criticism for granting multinational corporations parity with sovereign governments while providing no equivalent platform for civil society or human rights defenders.

• Rasmus Fonnesbæk Andersen, Senior Manager - AI and Data, Tony Blair Institute for Global Change
• Lourino Chemane, Board Chairman, National Institute of Information and Communication Technology (INTIC),
• Virginia Dignum, ACM Technology Policy Council Chair, Umeå University
• Jibu Elias, Country Lead for India / Fellow, Mozilla
• Dame Wendy Hall, Regius Professor of Computer Science, University of Southampton.
• Merve Hickok, President & Policy Director, Center for AI and Digital Policy
• Sara Hooker, Co-Founder, adaption
• Yannis Ioannidis, President, ACM and Professor, University of Athens
• Neha Kumar, Associate Professor, Georgia Institute of Technology
• Jeanna Matthews, ACM Technology Policy Council Chair, Clarkson University
• Tom Romanoff, Policy Director, ACM

A recording of the session is available at youtube.com/watch?v=y2fpcrkesGw.

PDF available here.