Identifying theoretical architectural vulnerabilities through STRIDE and estimating subjective risks through Delphi elicitation represent the crucial foundational phases of risk management. However, these models must be rigorously validated through empirical, adversarial testing. As organisations shift their security posture from reactive compliance auditing to proactive discovery, advanced testing techniques, specifically AI Red Teaming and Failure Mode and Effects Analysis (FMEA), become absolutely essential. The ultimate, defining challenge of this module is teaching students how to systematically translate the highly obscure findings from these specialised disciplines into formal, actionable enterprise risk registers that drive executive decision-making.

Moving from Reactive to Proactive Discovery: AI Red Teaming

AI Red Teaming is a highly structured, adversarial testing methodology designed to empirically evaluate how AI systems fail under hostile, real-world conditions before they are deployed to production environments. Standard penetration testing, which relies heavily on automated vulnerability scanners to detect traditional code flaws such as SQL injection and cross-site scripting, is entirely blind to the natural language manipulation, model evasion, and data poisoning techniques utilised by modern threat actors against generative AI.

A rigorous AI red team must be demographically and interdisciplinary in its diversity, combining traditional offensive security skills with deep domain expertise in machine learning and sociocultural awareness to uncover context-specific flaws. These complex exercises involve crafting highly specific adversarial prompts, testing the boundaries of indirect prompt-injection vulnerabilities, evaluating the model's overall susceptibility to complex jailbreaks, and verifying that built-in safety guardrails and LLM firewalls function as intended under sustained attack.

Crucially, red teaming must evaluate the entire agentic workflow. By executing realistic simulated attacks against the model, its orchestrators, and all connected third-party tools, red teaming uncovers emergent systemic risks, such as unintended tool misuse, data exfiltration via covert channels, and cascading action chains, that remain completely invisible when foundational models are tested in sterile isolation. The findings generated from these exercises provide invaluable empirical evidence regarding the true effectiveness of the system's mitigating controls.

Adversarial AI Testing and Failure Mode Analysis

While AI Red Teaming focuses aggressively on adversarial exploitation, Failure Mode and Effects Analysis (FMEA) provides a highly structured, engineering-driven methodology for analysing both adversarial and non-adversarial system failures. FMEA is a classical reliability engineering technique that systematically evaluates three core criteria for each potential failure mode identified in a system: the severity of the business impact, the statistical likelihood of occurrence, and the probability that existing monitoring tools will detect the failure before it impacts the business.

Adapting FMEA for advanced, generative AI requires a shift in thinking: practitioners must move away from simple component-level analysis and embrace a systemic, causal-pathway approach. In an AI context, a "failure mode" is rarely a broken server; it is a hallucinated legal precedent that leads to a lost lawsuit, or an algorithmic bias that results in discriminatory lending practices. The AI-adapted FMEA maps the entire causal chain from the initial triggering hazard (e.g., poisoned training data ingested from a public repository) to the ultimate organisational harm (e.g., massive regulatory fines, GDPR violations, and reputational destruction).

Furthermore, because AI failures are frequently correlated (for example, the concept of an "algorithmic monoculture," in which multiple disparate enterprise systems fail simultaneously due to their shared reliance on a single compromised foundational API), the FMEA process must integrate advanced probabilistic modelling to account for these systemic dependencies.

Translating Specialised Findings into Actionable Risk Registers

The most critical gap in contemporary organisational security programs is that engineering teams often don't translate the highly technical, sometimes complex outputs of AI Red Teaming and FMEA worksheets into the standardised language of enterprise risk management. Per the NIST AI Risk Management Framework (AI RMF), raw red-teaming outputs must be analysed further to inform organisational governance, policy updates, and executive decision-making. This crucial translation occurs within the Risk Register.

A formal AI Risk Register is not a static document; it is a dynamic, continuously updated tracking system that synthesises all identified risks, assigns explicit executive ownership, and comprehensively documents the ongoing risk-mitigation lifecycle. It forces the necessary translation of abstract technical vulnerabilities into concrete business impacts.

The inherent risk (the calculated exposure before any controls are applied) is derived by multiplying the impact rating by the likelihood probability, which are directly informed by the preceding Delphi elicitations and FMEA evaluations. The register then maps these inherent risks to specific mitigating controls (e.g., rigorous input validation, differential privacy implementations, LLM firewalls, or mandatory human-in-the-loop oversight workflows) to calculate the remaining residual risk.

The following table exemplifies the structured translation of a highly technical AI Red Teaming finding (an Indirect Prompt Injection vulnerability) into a formal risk register entry. This demonstrates the exact level of detail, accountability, and regulatory alignment required for enterprise governance.