by Daniela Muhaj & Jayeeta Putatunda
Chapter Originally Published on April 29, 2026, at ASFAI.org
As artificial intelligence, especially generative AI systems that utilize large language models (LLMs), becomes increasingly embedded in high-stakes domains, the need for rigorous, continuous socio-technical evaluation is critical. Existing approaches often focus narrowly on technical metrics and post-deployment audits, overlooking broader questions of fairness, accountability, and real-world societal impact.
Recent research confirms the limitations of current AI reasoning capabilities. Research shows that Large Reasoning Models collapse under high problem complexity, often failing to generalize even when given exact algorithms. Meanwhile, other analysis finds that more deliberate reasoning prompts decreased performance in financial sentiment tasks, with simple intuitive prompting outperforming complex chains of thought. These findings reveal hard ceilings in current reasoning models, emphasizing why we shouldn't outsource all decision-making to AI, especially in high-stakes domains like finance, healthcare, and public policy.
This chapter introduces Responsible Evaluation by Design (REvD), a practical and scalable framework for socio-technical evaluation of AI systems. It emphasizes early integration and continuous monitoring of both performance and societal outcomes, embedding evaluation throughout the AI lifecycle—from design and development to deployment and ongoing use. To demonstrate the framework's practical value, we apply it to generative AI in credit risk evaluation, illustrating how REvD can address potential regulatory risks, mitigate bias, and enhance transparency.
The rise of generative AI systems has sparked significant ethical and societal concerns. While these models show strength at content generation, they risk amplifying social biases, spreading misinformation, and violating privacy. Early deployment failures illustrate these challenges: Google's Search "AI Overviews" instructed users to add glue to pizza sauce and eat rocks daily for minerals. These misfires demonstrate how AI systems, in pursuing perfection, can lose common sense and accuracy. Moreover, research reveals that even as models become less overtly discriminatory, they continue to embed covert prejudices that surface through subtle linguistic cues, suggesting that surface-level bias mitigation may merely mask deeper systemic issues rather than resolve them.
Current evaluation frameworks fall short of capturing these impacts. Standard practices focus mainly on technical performance metrics—accuracy, latency, and robustness—that fail to surface critical societal risks such as biased decision outcomes or degradation in user trust that emerge when AI systems interact with real-world institutions and communities. Moreover, evaluations are typically isolated in time and discipline, lacking continuity and interdisciplinary insight. As models are fine-tuned and integrated at scale, additional risks emerge that are not easily captured through static or task-specific benchmarks. These include distributional shifts, unintended behaviors, and difficulties in tracing interpretability or responsibility when outputs influence consequential decisions.
This need is particularly acute in public services, where AI systems increasingly support decisions about healthcare, education, employment, and social protection. In such domains, evaluation failures can produce disproportionate harms, erode public trust, and create accountability gaps. As AI systems become more central to public administration, proactive, rigorous, and context-aware evaluation becomes not only a technical concern but also a question of responsible governance.
To address these challenges, this chapter introduces Responsible Evaluation by Design (REvD)—a framework that embeds evaluation throughout the AI system lifecycle. Drawing on insights from safety engineering, social systems analysis, and measurement science, this approach emphasizes continuous, context-calibrated evaluation and stakeholder engagement. In contrast to audit-centric models that treat evaluation as a final step, REvD treats it as an integral part of system design and implementation.
To illustrate the framework's application, this chapter examines the deployment of generative AI in credit risk evaluation. While finance is one of many domains where evaluation challenges are salient, it offers a useful lens for examining how technical and institutional factors interact in high-stakes AI deployment.
Existing toolkits like IBM's AI Fairness 360 and Google's What-If Tool provide important functionality for identifying statistical disparities, but focus primarily on technical metrics at the model level. They are not designed to capture broader systemic effects such as institutional misalignment, long-term social dislocation, or shifts in public trust that arise during deployment.
The REvD framework draws inspiration from privacy by design principles, embedding evaluation practices across the full AI system lifecycle. It aligns with emerging paradigms like Google DeepMind's three-layered model, distinguishing capability evaluation, human interaction evaluation, and systemic impact assessment. The framework is consistent with emerging governance standards such as the NIST AI Risk Management Framework and the EU Artificial Intelligence Act, both emphasizing risk-based, lifecycle-oriented approaches.
The REvD framework, as shown in Figure 1, offers a structured, lifecycle-oriented approach to assessing the full range of impacts associated with artificial intelligence systems, particularly in high-stakes, dynamic, or regulated environments.

Figure 1: Overview of the Responsible Evaluation by Design Framework
REvD is anchored in three foundational principles:
Continuity: Evaluation is embedded across the full system lifecycle—from problem definition and data selection to model deployment and post-market monitoring.
Inclusivity: The evaluation integrates perspectives from diverse stakeholders, including developers, domain experts, regulators, civil society, and impacted users.
Transparency: Evaluation criteria, metrics, and decision thresholds are clearly defined and documented to enhance trust and provide a basis for regulatory alignment.
Unlike traditional evaluation frameworks that focus narrowly on technical performance (e.g., accuracy, latency), REvD emphasizes a broader set of impact categories. This initial set proposes a baseline of impact categories to be tested, refined, and contextualized to specific use cases, with dynamic updates to remain effective and keep pace with the rapid evolution of AI capabilities.
These include:
Each category is operationalized through both quantitative and qualitative measures. This dual approach allows for the detection of risks that may not be apparent from technical metrics alone and supports the development of richer, context-aware evaluation practices.
To enable systematic tracking of system behavior and impact, REvD introduces a set of composite metrics:
Before implementing generative AI in credit risk, executives must address fundamental strategic questions that align with regulatory oversight expectations:
Primary Strategic Questions:
Business case validation is critical, as regional financial institutions report that unclear business objectives represent the primary barrier to successful AI implementation. Organizations with well-defined strategic frameworks achieve three times higher success rates in AI deployments compared to those without structured approaches.
Critical Compliance Questions:
See Appendix A, Table 1, for a detailed breakdown of key regulatory body frameworks to review.
The EU AI Act classifies credit scoring as "high-risk," requiring comprehensive risk assessment and human oversight. US regulators increasingly scrutinize AI in lending, with the CFPB issuing guidance on algorithmic fairness.
Risk Management Questions:
Financial services firms experience AI-related incidents at 2.3 times the rate of other industries, making robust incident response critical.
Research Design and Methodology
To develop an evidence-based understanding of REvD implementation challenges in credit risk applications, we designed a comprehensive stakeholder research approach. While many current evaluation methods focus on post-deployment auditing or isolated technical testing, REvD advances a process-oriented model that treats expert validation as an institutional function rather than an ad hoc intervention.
We conducted a subject-matter expert survey within the domain to gather critical feedback on current bottlenecks and failure points in the financial credit risk domain.
See Appendix A, for Recommended Research Approach, Research Survey Questions, Institutional Diversity Requirements and Formalizing Stakeholder Roles for a detailed breakdown of how to set up the survey to initialize this research.
To sum it up, Figure 2 below outlines the lifecycle steps for integrating stakeholder feedback into the development process.

Figure 2: Stakeholder Roles and Integration
Analysis of industry reports and survey feedback reveals five critical implementation challenges for financial services organizations implementing AI in credit risk applications:
Challenge 1: Executive Technical Literacy Gap
The technical gap between leadership and engineering has become a significant risk surface. Executives need to understand model classes, red teaming results, and be able to challenge vendor claims. Research shows that 43% of AI implementation failures stem from executive teams lacking sufficient technical understanding to make informed governance decisions. This creates blind spots in risk assessment and strategic decision-making.
REvD Solution: Executive AI literacy programs, regular technical briefings, and technical advisory roles within governance structures.
Challenge 2: Regulatory Uncertainty and Compliance Gaps
The regulatory landscape remains fragmented, with 73% of financial institutions reporting challenges in interpreting AI-specific requirements. Federal guidance emphasizes "appropriate risk management" but provides limited technical specificity, while regulators acknowledge that traditional model validation may be insufficient for complex AI systems.
REvD Solution: Proactive regulatory engagement through structured documentation and comprehensive audit trails that exceed current requirements.
Challenge 3: Model Interpretability vs. Performance Trade-offs
Traditional credit models achieve 78% accuracy with full interpretability, while AI models reach 89% accuracy but struggle with explanation requirements. Different stakeholders require different explanation types—technical for data scientists, operational for loan officers, and plain-language for customers and regulators.
REvD Solution: Multi-layered explanation architectures providing stakeholder-specific interpretability without compromising performance.
Challenge 4: Bias Detection and Continuous Monitoring Complexity
AI systems require continuous monitoring across multiple demographic dimensions with exponentially increasing computational complexity. The 2023 economic environment revealed previously undetected bias patterns in major banks' AI systems under novel stress scenarios.
REvD Solution: Automated bias monitoring infrastructure with real-time alerts and synthetic data generation for stress testing.
Challenge 5: Operational Integration and Legacy System Compatibility
Legacy banking systems often lack the necessary API infrastructure for real-time AI integration. Operational integration costs typically exceed initial development costs by 2-3 times.
REvD Solution: Phased implementation with geographic pilots, comprehensive staff training, and gradual workflow integration, preserving human oversight.
This section presents the comprehensive four-phase implementation framework for REvD in credit risk applications. Each phase includes executive actions, technical implementation details, and visual process maps that demonstrate how the framework translates into measurable business outcomes.
See Appendix Table 2 for a detailed explanation of the Key Metrics
As shown in Figure 3, Executive Actions would be:
1. Establish AI Governance Committee
2. Define Success Metrics and KPIs
3. Regulatory Engagement Strategy

Figure 3: Executive Governance Workflow
Key Deliverables:
Technical implementation integrates continuous evaluation checkpoints throughout development, including bias testing, explainability validation, and stakeholder feedback loops, as shown in Figure 4.
See Appendix Table 3 for a detailed explanation of the Technical Implementation Framework

Figure 4: Technical Implementation with embedded evaluation
As shown in Figure 5, below is the Pilot Implementation Strategy workflow
1. Limited Geographic Rollout
2. Gradual Feature Activation
3. Stakeholder Feedback Integration


Figure 5: Pilot Deployment and Continuous Monitoring
See Appendix Table 4 for a detailed explanation on Deployment & Continuous Monitoring.
Continuous Monitoring Framework:
Real-time Monitoring Architecture: The monitoring system, as shown in Figure 5, processes multiple data streams to provide immediate alerts when performance thresholds are exceeded:*
* These benchmarks were derived from limited stakeholder interviews, and we will be continuing to get more feedback to validate the thresholds. This will include calibration of the use case for sector adaptation.
1. Geographic Expansion
2. Feature Enhancement
3. Operational Excellence
Overall Temporal Trend Index (T(t)) Tracking System
This is an illustrative example of how a longitudinal performance monitoring system tracks changes in key metrics over time, providing early warning indicators of system degradation or stakeholder dissatisfaction. The automated alert system ensures that declining trends trigger appropriate organizational responses before they become critical issues.

Figure 6: Temporal Trend Tracking Workflow
Alert Escalation Framework:
Before REvD Implementation:
After REvD Implementation:
Measurable Process Enhancements:
Key Workflow Improvements:
1. Evaluation Timing: From quarterly post-deployment audits to continuous real-time monitoring
2. Stakeholder Integration: From annual feedback surveys to structured monthly engagement processes
3. Monitoring Capability: From manual compliance reviews to automated dashboard monitoring with alert escalation
4. Response Mechanisms: From manual incident investigation to automated alert generation and predefined response protocols
5. Documentation Standards: From compliance-driven annual reports to comprehensive real-time audit trail generation
The REvD framework repositions evaluation from downstream compliance to an embedded design principle. By emphasizing continuity, inclusivity, and transparency, REvD enables holistic socio-technical governance that is adaptable to the complexity of AI deployment. REvD is sector-agnostic and applicable across healthcare, hiring, public services, and education—domains where algorithmic decisions intersect with rights and systemic equity. The framework supports organizations navigating evolving regulatory regimes like the EU AI Act, U.S. Blueprint for an AI Bill of Rights, and NIST AI Risk Management Framework.
Implementing REvD offers more than compliance. Organizations that proactively embed evaluation throughout the AI lifecycle build mechanisms and capacity to mitigate reputational and legal risk, reduce downstream costs, differentiate through responsible innovation, and earn stakeholder trust while future-proofing systems.
REvD faces practical challenges, including cultural inertia, fragmented data governance, limited interdisciplinary capacity, and inadequate tooling. Smaller organizations may lack resources for comprehensive monitoring or diverse stakeholder engagement. Expert input introduces subjectivity, requiring careful procedural design.
Clearer standards are needed to quantify emergent harms and calibrate risk thresholds. While REvD provides a robust blueprint, widespread adoption requires additional technical, institutional, and legal scaffolding. Advancing REvD requires research into dynamic metric calibration, cross-modal evaluation strategies for generative AI, empirical studies of stakeholder-informed evaluation effectiveness, and development of interoperable open-source tools for lifecycle monitoring.
REvD represents a fundamental shift in AI governance from reactive compliance to proactive design integration. As AI systems increasingly shape critical societal decisions, this shift—from audit to architecture—is essential for responsible innovation at scale.
Appendix A
Table 1: Regulatory and Legal Compliance Framework
Recommended Research Approach:
Sample Research Survey Questionnaire: REvD Implementation in Financial Services:
Section I: Executive Background (5 minutes)
Organization Profile:
Your Role:
Section II: Strategic Alignment (10 minutes)
Key Questions:
1. AI Use Cases and Business Objectives
2. Governance and Risk Appetite
3. Regulatory Readiness
Section III: Implementation Challenges (10 minutes)
Core Challenge Areas:
Rate each challenge's impact on your organization (1=Low, 5=High):
Follow-up Questions:
Section IV: Risk Management and Monitoring (10 minutes)
Current Practices:
1. AI Risk Assessment
2. Incident Preparedness
3. Stakeholder Engagement
Section V: Success Factors and Lessons Learned (5 minutes)
Key Insights:
1. Critical Success Factors
2. Implementation Advice
3. Future Outlook
Quantitative Assessment (Post-Interview Email)
Rate your agreement (1=Strongly Disagree, 5=Strongly Agree):
Priority Ranking Exercise
Rank the following AI evaluation activities by importance (1=Most Important, 8=Least Important):
___ Bias testing and monitoring
___ Model performance validation
___ Regulatory compliance documentation
___ Stakeholder engagement and feedback
___ Incident response and remediation
___ Staff training and change management
___ Community impact assessment
___ Competitive advantage measurement
Open-Ended Reflection
Final Question: What additional support, guidance, or resources would be most valuable for implementing responsible AI evaluation in your organization?
Institutional Diversity Requirements:
● Large Regional Banks ($10B+ assets): 30% of sample
● Community Banks ($1B-$10B assets): 35% of sample
● Credit Unions: 20% of sample
● Fintech/Digital Lenders: 15% of sample
Formalizing Stakeholder Roles
To ensure accountability and mitigate blind spots, REvD calls for structured integration of domain experts, impacted communities, and oversight bodies (see Appendix A, Figure 3):
These roles are supported by predefined responsibilities and escalation protocols that determine when systems must be re-evaluated or redesigned based on stakeholder concerns.
Key Metrics Implementation:
Table 2: Technical Implementation Framework
Technical Implementation Framework:
Table 3: Key Metrics Implementation (*Based on a limited number of stakeholder interviews)
Continuous evaluation lifecycle:
Table 4: Step-by-step explanation of the continuous evaluation lifecycle
© 2026 Daniela Muhaj & Jayeeta Putatunda. All rights reserved.
AUTHORS:
Daniela Muhaj is the Director for Research and Technology Initiatives at Georgetown University, leading innovative research and technology efforts to shape global business practices and private sector development. She serves as Head of Research and Development and Co-Chair of the Global Fellows Program at AI 2030, an organization focused on responsible artificial intelligence development. Additionally, she is a Research Affiliate with MIT's FutureTech group at the Computer Science and Artificial Intelligence Laboratory (CSAIL), researching emerging technology trends and their impact on productivity and economic growth.
With a decade of experience in applied economic research and development policy, Daniela has worked across diverse institutions including the Central Bank of Slovenia, International Monetary Fund, Harvard's Center for International Development, MIT's J-PAL, and Amazon Web Services. As AWS's Global Economic Research Lead, she pioneered research on cloud computing's economic and environmental impacts. Her achievements include selection as a G20 Young Global Changer in 2019 and presenting inequality research at G20 events.
Daniela holds a Master's in International Economics from Johns Hopkins SAIS and dual Bachelor's degrees in Mathematical Economics and International Studies from Baldwin Wallace University, graduating summa cum laude.
Jayeeta Putatunda is an award-winning Lead Data Scientist and Director at Fitch Ratings, specializing in NLP and Generative AI. Her diverse career path from Economics to Data Science has equipped her with technical expertise and strong business acumen. At Fitch, she develops scalable Generative AI solutions while exploring open-source models. Her achievements include the AI100 award in Generative AI (2024), recognition among Top 25 Visionary Women in FinTech AI, and selection as one of 33 global advisors for NVIDIA's Enterprise Platform Advisor Program.
Her research on AI for financial inclusion has been presented at prestigious conferences like ACM's ICAIF and AAAI-25, and she serves as a technical paper reviewer for NeurIPS and COLING. Beyond technical accomplishments, Jayeeta champions diversity in tech as NYC Chapter Lead for Women in AI and Stanford's Women in Data Science ambassador. She has conducted over 40 free tech talks, mentored 100+ individuals, and was featured on a Times Square Billboard as among North America's Top 1% mentors. She regularly speaks at major conferences and guest lectures at Columbia, NYU, and PACE University.
Thank you for exploring AI for Humanity, a project built by humans, powered by AI, and guided by values. Join us in shaping a more human‑centered future.
Adobe Firefly, Canva Magic Studio, ChatGPT, Gamma.app, Gemini Notebook, Otter.ai, Perplexity, and Suno.
Copyright © 2026 American Society for AI (ASFAI) and The International Social Impact Institute® (The ISII®). All rights reserved.
The American Society for AI is a non-profit and the preeminent organization for Artificial Intelligence (AI).
Our mission is to create a better world with AI.
Learn how information is handled and review the terms governing use of this platform:
Part 1: Ethical & Responsible AI