XAI 2.0 and the Road Ahead: Open Challenges and Future Directions

Introduction

The XAI 2.0 manifesto, authored by 27 leading XAI researchers and published in Information Fusion in 2024, identifies 22 open challenges across five themes [1]. This final article in the series synthesises those challenges, connects them to the findings of the previous four articles, and maps the research directions that will define the next generation of explainability.

The central thesis of the XAI 2.0 manifesto is that the field must transition from a method-centric discipline focused on producing explanations to a human-centred discipline focused on supporting understanding. This framing recasts every technical challenge as a human-centred challenge, and every methodological question as an interdisciplinary question.

This article is not legal advice.

Vocabulary for XAI 2.0

Method-centric XAI

The dominant paradigm in XAI 1.0: develop an explanation method, demonstrate it on benchmark datasets, evaluate with proxy metrics. The focus is on the method, not the user.

Human-centred XAI

The paradigm proposed for XAI 2.0: start with what humans need to understand, design explanations to meet those needs, evaluate by measuring whether understanding improves.

Explanation personalisation

The design of explanations tailored to individual users' cognitive characteristics, domain expertise, and task context. A single explanation format cannot serve all stakeholders.

Cognitive load

The mental effort required to process and evaluate an explanation. Explanations that impose high cognitive load may be counterproductive, particularly in high-stakes, time-constrained decisions.

The 22 Open Challenges

Technical Challenges

1. Formal definitions. The field still lacks a universally accepted formal definition of explainability. The definition of Barredo Arrieta et al. is the closest to consensus, but it is descriptive rather than formal. Without formal definitions, mathematical guarantees about explanation quality are impossible.

2. Evaluation metrics. The metrics crisis documented by Pawlicki et al. (Critical Perspectives and Limits of Current Explainability Methods) remains unresolved. Over 50 metrics exist, many are redundant, and no standardised evaluation framework has emerged. The XAI 2.0 manifesto calls for community-wide agreement on core evaluation constructs.

3. Robustness of explanations. Explanations should be stable under small input perturbations that do not change the prediction. Current methods vary in their robustness, and there is no standard for acceptable stability.

4. Completeness vs. selectivity trade-off. Comprehensive explanations (showing all contributing features) may be too complex for human understanding. Selective explanations (showing only the most important features) risk omitting essential information. There is no principled framework for navigating this trade-off.

5. Ground truth for explanations. Unlike classification, there is no ground truth for explanations. Even for simple models, what counts as the “correct” explanation depends on the explanatory target (model, data, or phenomenon. Watson’s first challenge).

6. Reproducibility and generalisation. Explanation methods often produce different results across random seeds, data splits, or software implementations. Reproducibility standards for XAI research are underdeveloped.

Human-Centred Challenges

7. Cognitive load. Explanations impose cognitive demands on users that may exceed their processing capacity, particularly in high-stakes environments like clinical decision-making (the inscrutability challenge of Ghassemi et al.[3]).

8. Trust calibration. Explanations can produce both under-trust (users ignore reliable models) and over-trust (users accept unreliable explanations). The goal is calibrated trust: matching user confidence to actual model reliability.

9. Explanation personalisation. Different stakeholders (regulators, clinicians, patients, engineers) need different explanations. Current methods produce one-size-fits-all outputs.

10. Mental models. Explanations should help users build accurate mental models of system behaviour. Current methods provide attribution scores that support shallow understanding.

11. Longitudinal effects. Most XAI research studies one-shot explanation consumption. Real-world use involves repeated, cumulative engagement with explanations over time.

Interdisciplinary Challenges

12. Bridging XAI and HCI. Explanation method design is currently driven by ML researchers, not HCI experts. The result is methods optimised for technical properties (axioms, completeness) rather than usability.

13. Cognitive science integration. Understanding how humans process explanations requires integrating cognitive science findings on causal reasoning, counterfactual thinking, and explanation evaluation.

14. Philosophy of explanation. Watson’s conceptual critique draws on philosophy of science. The field must engage more deeply with philosophical work on what constitutes a good explanation.

15. Social science perspectives. Organisational adoption of XAI (the deployment gap of Bhatt et al.[4]) is a social science question as much as a technical one.

16. Legal and regulatory alignment. The EU AI Act and emerging regulations create requirements that current XAI methods may not satisfy. Legal scholars must collaborate with XAI researchers to operationalise regulatory concepts.

Regulatory Challenges

17. Operationalising the right to explanation. GDPR Article 22 and the EU AI Act establish rights to meaningful information about automated decisions. What counts as “meaningful” is undefined, creating legal uncertainty.

18. Auditing frameworks. The access-level framework of Casper et al. provides a starting point, but auditing standards for explanation quality do not exist.

19. Standardisation. Unlike software testing or security evaluation, there are no ISO standards for XAI evaluation.

Methodological Challenges

20. Longitudinal studies. Most XAI research evaluates explanations in single-session lab studies. Longitudinal studies tracking how understanding develops over time are almost absent.

21. Field studies. The deployment study of Bhatt et al. remains the only empirical characterisation of XAI in practice. More field studies are needed.

22. Human-grounded evaluation validation. Functionally-grounded metrics require validation against human judgement. This validation is almost never performed, meaning the field uses proxy metrics whose relationship to actual interpretability is unknown.

Connecting the Challenges to the Series Findings

The 22 challenges are not independent. They form a coherent structure that connects directly to the findings of the previous four articles.

Research Directions

From Method-Centric to Human-Centred

The paradigm shift that Longo et al. advocate requires reorienting XAI research. Instead of asking “what explanation method should we develop?” the field should ask “what do humans need to understand, and how can we support that understanding?” This shifts the evaluation criterion from technical properties (axioms satisfied) to human outcomes (understanding improved).

Interdisciplinary Infrastructure

Several challenges require infrastructure that no single discipline can build. The integration of cognitive science findings into XAI design requires sustained collaboration between ML researchers and cognitive scientists. The development of auditing standards requires collaboration between technical researchers, legal scholars, and regulators. The longitudinal and field studies required for challenges 20-21 need organisational access that is currently rare.

Evaluation as a First-Class Problem

The metrics crisis (challenge 2) connects multiple other challenges. Without reliable evaluation, progress on robustness (3), reproducibility (6), and trust calibration (8) cannot be measured. The call of the XAI 2.0 manifesto for evaluation standardisation is the single most actionable recommendation for the field.

Regulatory Engagement

The EU AI Act and emerging regulations in other jurisdictions create both requirements and opportunities. Regulatory engagement can motivate evaluation standards (requiring validated explanations), mandate adequate access levels (the outside-the-box access of Casper et al.), and create demand for field studies (as organisations document their XAI compliance). The risk of regulatory engagement is that it may codify premature standards, locking in methods that the field has not yet properly evaluated.

Conclusion: The Road Ahead

The XAI 2.0 manifesto identifies where the field needs to go. The previous four articles in this series establish where it currently stands. The gap between them defines the research agenda.

The central challenge is the transition from method-centric to human-centred XAI. This transition requires not only new methods but new evaluation frameworks, new interdisciplinary collaborations, new organisational structures, and new regulatory engagement. It is a generational research programme, not a quick fix.

The five articles in this series have examined what explainability means (What Does It Mean for AI to Be Explainable? Foundations of Interpretable ML), how current methods work (Methods and Techniques for Explaining Machine Learning Models), where they fall short (Critical Perspectives and Limits of Current Explainability Methods), how they are applied in practice (Explainability in Practice: Domains, Evaluation, and Governance), and where the field needs to go (this article). The unifying message is that explainability is neither solved nor hopeless. It is a young field whose most important questions remain open, and whose progress depends on the willingness of the field to confront its limitations honestly.

Frequently Asked Questions

What are the main categories of open challenges in the XAI 2.0 manifesto?

Longo et al. (2024)[1] organise 22 challenges into five categories: technical challenges including formal definitions and evaluation metrics, human-centred challenges including cognitive load and trust calibration, interdisciplinary challenges spanning HCI and philosophy, regulatory challenges concerning auditing frameworks and standardisation, and methodological challenges involving longitudinal and field studies. Technical challenges are most mature while interdisciplinary challenges require structural changes.

What distinguishes human-centred challenges from technical challenges in XAI?

Technical challenges focus on formal properties of explanations such as completeness, robustness, and reproducibility of attribution methods. Human-centred challenges address how explanations interact with human cognition including how much detail users can process, how explanations affect trust, and whether explanations personalise to different user expertise levels. Solving technical challenges does not automatically resolve human-centred ones.

Why does the XAI 2.0 manifesto call for interdisciplinary infrastructure?

The manifesto argues that no single discipline can address all 22 challenges because explanation is simultaneously a mathematical, psychological, and sociological phenomenon. Interdisciplinary infrastructure means shared experimental platforms, standardised evaluation protocols that work across disciplines, funding mechanisms that reward collaboration, and publication venues that accept hybrid contributions spanning computer science, cognitive science, and law.

How can evaluation become a first-class problem in XAI research?

The manifesto argues that evaluation should shift from a post-hoc activity to a design-stage requirement. This means developing community-standard benchmarks, establishing ground-truth datasets for explanation correctness, creating reproducible evaluation protocols that span research groups, and making evaluation quality a publication criterion equivalent to methodological novelty. Currently, evaluation quality varies widely and is rarely treated as a primary contribution.

What regulatory challenges does the XAI 2.0 manifesto identify?

Three regulatory challenges are identified: operationalising the right to explanation under GDPR and the EU AI Act into technically specific requirements, developing auditing frameworks that can verify explanation quality at scale, and creating standardisation mechanisms that allow different regulatory regimes to accept common evidence of explainability. These challenges are urgent because regulatory timelines are shortening while technical standards remain fluid.