Failure Mode and Effects Analysis (FMEA) provides a systematic methodology for analysing both adversarial and non-adversarial system failures. This procedure adapts classical reliability engineering techniques for modern cybersecurity applications.

Step 1: Define System Scope and Failure Context (15 minutes)

Establish the specific system or subsystem under analysis, such as “Document ingestion and RAG query pipeline.” Clearly define what constitutes a “failure” within the assessment scope, distinguishing between technical faults (service crashes), security failures (data exfiltration), and AI-specific failures (hallucinated outputs). This definitional clarity ensures consistent evaluation across the analysis team.

Step 2: Identify Failure Modes Through Causal Mapping (25 minutes)

For each critical system component or process, systematically identify potential failure modes by asking: “How can this component fail in a way that materially impacts the business?”

Document each failure mode using the causal chain format:

[Triggering Cause] → [Failure Mode] → [Business Effect].

 For example: “Malicious metadata in document → AI executes adversarial instructions → Sensitive data exfiltration.” This approach moves beyond simple component analysis to embrace systemic, pathway-based thinking essential for complex modern systems.

Step 3: Quantitative Risk Assessment (30 minutes)

For each identified failure mode, assign numerical ratings across three critical dimensions:

Severity (S): Rate the magnitude of business impact if the failure occurs on a 1-5 scale, where 1 represents negligible impact, and 5 represents catastrophic consequences, such as regulatory breaches or existential organisational harm.

Occurrence (O): Evaluate the likelihood of the failure mode materialising, considering current controls and threat landscape dynamics, where 1 represents a rare occurrence, and 5 indicates a frequent or highly probable event.

Detectability (D): Assess the probability that existing monitoring and detection capabilities will identify the failure before it causes business harm, where 1 indicates almost certain detection and 5 indicates scenarios very unlikely to be detected.

Step 4: Calculate Risk Priority Numbers (10 minutes)

Compute the Risk Priority Number for each failure mode using the formula:

RPN=Severity×Occurrence×Detectability

Higher RPN values indicate higher priority for mitigation investment. This mathematical approach provides objective prioritisation criteria and supports resource allocation decisions.

Step 5: Develop Mitigation Strategies (20 minutes)

For failure modes with the highest RPNs, develop comprehensive mitigation strategies encompassing technical controls (firewalls, encryption), architectural improvements (segmentation, least privilege), and process enhancements (human oversight, approval workflows).

Document the responsible owner, implementation timeline, and expected impact on the S/O/D ratings. Recalculate projected RPNs after implementing the mitigation to validate control effectiveness.

Step 6: Integration with Risk Management Processes (10 minutes)

Transform high-priority failure modes into formal entries in the risk register to ensure alignment with enterprise governance frameworks. Establish continuous monitoring requirements based on detectability insights, feeding detection-related findings into SIEM rules, metrics, alerting systems, and periodic validation testing such as red team exercises.

Scenario 2: AI/RAG - Legal Contract Analysis with Indirect Prompt Injection Risk

Context of the Problem

A global enterprise legal department has deployed an advanced Retrieval-Augmented Generation (RAG) system utilising Large Language Models to analyse confidential vendor contracts, intellectual property agreements, and attorney-client privileged communications.

 This system operates fundamentally as a probabilistic “black box” in which outputs are non-deterministic, vulnerabilities arise from training processes rather than explicit code flaws, and traditional penetration testing methodologies prove entirely ineffective.

The system architecture includes automated ingestion of external vendor contracts (often as PDFs with complex metadata), indexing into vector databases, and AI agent capabilities, including exporting contract summaries to vendor portals and creating internal tickets for unusual clauses. The organisation faces unprecedented risks, including indirect prompt injections, hallucinated legal precedents, and systematic algorithmic bias in contract risk assessment.