What is an AI threat model and why does it matter?

An AI threat model is a structured analysis of the attack surface, potential adversaries, attack paths, and security controls for an AI system. It maps every point at which an adversary could interact with the system, from training data ingestion through inference endpoints and tool integrations, and identifies what controls are in place to detect or prevent exploitation at each point. AI threat modeling matters because AI systems have attack surfaces that differ from conventional software: training pipelines, model weights, embedding stores, retrieval systems, and tool interfaces all require analysis that conventional threat modeling frameworks do not address. GDF's AI security architecture engagements begin with a structured AI threat model that drives all subsequent architecture recommendations.

What is the difference between AI governance and AI security?

AI governance addresses the policies, processes, and oversight structures that determine how AI systems are developed, deployed, monitored, and retired. AI security addresses the technical controls that protect AI systems from adversarial attack, unauthorized access, and data exposure. The two are closely related: governance structures create the accountability and review processes that make security controls effective, while security findings provide the empirical evidence that governance decisions should be based on. GDF's AI security architecture work addresses both dimensions, helping organizations build governance frameworks that incorporate security requirements and security architectures that are supported by governance processes.

How does RAG security differ from standard application security?

Retrieval-augmented generation introduces security requirements specific to how LLMs interact with external data. The vector database must enforce authorization at retrieval time, ensuring that users can only receive documents their permissions allow. The ingestion pipeline must sanitize documents before indexing to prevent embedding poisoning and indirect prompt injection. The similarity search mechanism must not be manipulable to return records from unauthorized namespaces. The context construction logic that assembles retrieved documents into prompts must avoid leaking cross-user data. These requirements are distinct from conventional application security and require architecture and testing approaches specifically designed for RAG systems.

What does EU AI Act compliance require for AI security?

The EU AI Act's requirements for high-risk AI systems include cybersecurity measures adequate to protect against attacks that could compromise system integrity and availability, logging and auditability requirements sufficient to reconstruct system behavior, robustness testing requirements to verify the system performs consistently under adversarial conditions, and conformity assessment procedures that include technical security documentation. Organizations must maintain technical documentation, implement quality management systems, and for certain system types, register the system in an EU database before deployment. GDF's AI security architecture assessments are structured to produce the technical documentation and security analysis needed to support EU AI Act conformity assessments.

AI Security

Secure-by-Design Architecture Consulting for LLM and ML Deployments

The cheapest place to fix an AI security defect is at the architecture level, before the design has committed to a pattern that later has to be undone. Threat modeling anchored on MITRE ATLAS and the OWASP LLM Top 10, architecture review of the specific decisions that determine whether a deployment can be attacked at the boundary between the model and the systems it reaches, and governance-ready documentation that supports the operator’s AI risk-management posture from launch through examination.

What Architecture Consulting Actually Delivers

Threat model anchored on the specific deployment, not on a generic LLM threat catalog. Which attack classes actually apply given the deployment’s architecture, which reach consequential outcomes, and which are theoretical for the specific context.
Architecture-level defects surfaced before development commits. Authorization boundary design, trust boundaries between user input and instruction context, retrieval-corpus integrity model, and the specific patterns that decide whether prompt injection can reach the systems the model touches.
Remediation guidance at the architectural level. Design patterns that close the surfaced defects at their root rather than layering compensating controls on top of a flawed foundation. Patterns anchored on what has actually held up in production rather than on speculative best practice.
Governance-ready documentation. The threat model, the design decisions, the residual-risk register and the rationale for each significant choice preserved in a form that supports the operator’s AI risk-management file from launch through subsequent examination.
Coordination with downstream security testing. Architecture engagements produce the reference model that anchors subsequent adversarial testing, so the testing team knows what the design intended and can measure the deployment against it.

When Architecture Consulting Actually Matters

Pre-development or early development, when the fundamental patterns are still open. The most consequential defects live at this stage, and remediation is cheap.
Before a major deployment change, when the operator is about to extend an existing AI system to touch new systems, new data, or new user classes. Extensions frequently break assumptions the original design was safe against.
Before a governance or regulatory examination, when the operator needs to demonstrate that the architecture reflects a defensible risk posture rather than convenient defaults.
After an incident or a testing engagement that surfaced architectural rather than implementation defects, when the useful corrective action is a design rather than a patch.
During vendor selection where the operator is deciding between AI platforms, model providers, or retrieval frameworks and wants an independent view of the security-architecture implications of each choice.

Architecture Scope

Trust boundaries and instruction-context integrity

The most consequential architectural decision in an LLM deployment is where the trust boundary sits between user-supplied content and the model’s instruction context. Designs that let attacker-controllable content reach the instruction context (directly through user input, or indirectly through pasted content, retrieved documents, or tool outputs) inherit the entire prompt-injection attack surface. Designs that isolate untrusted content from the instruction context cut most of that surface at the root. The architecture engagement identifies where each boundary currently sits and where it should sit.

Authorization boundary between model and downstream systems

Where the model has access to functions, tools or systems that touch real data or operations, the architectural question is whether authorization is enforced on the server side against the authenticated session, or whether it is enforced by the model itself acting on identity information the model has been told about. Server-side authorization survives prompt injection; model-side authorization does not. The engagement surfaces this pattern in the current design and proposes the corrections.

Retrieval-augmented generation integrity

Where the deployment uses RAG, the retrieval corpus itself becomes attack surface. Whether attacker-controlled content can be introduced into the corpus (through public-facing document ingestion, through user-submitted content, through vendor integrations), and whether retrieved content reaches the instruction context unfiltered, decide whether the RAG pipeline can be turned into an indirect-injection channel. The engagement reviews corpus governance, ingestion controls and the specific handling of retrieved content in the prompt assembly.

Data flow and privacy-boundary design

Where the AI system processes sensitive content (PHI, PCI, PII, attorney-client, trade secret), the architectural question is which content actually reaches the model provider, which is redacted or tokenized before it leaves the operator’s control, and how that decision is enforced in the pipeline. The engagement surfaces the current data-flow reality and proposes the pipeline changes required to align it with the operator’s privacy posture.

Model-provider trust and vendor-integration design

The choice of model provider, hosted vs self-hosted deployment, and the specific integration pattern with the provider all carry architectural consequences the operator often has not thought through. The engagement covers what the specific choice implies for data residency, provider-side retention, incident-response coverage and the operator’s ability to audit the deployment.

Cost and denial-of-wallet architecture

Where the deployment can be induced to consume disproportionate model resources on attacker-supplied inputs, the architecture engagement surfaces the specific patterns (unbounded generation, unlimited retrieval, tool-call amplification) that turn the AI feature into a cost attack against the operator and proposes the boundary controls that close them.

Case in point

Fintech function-calling architecture review that prevented a costly rebuild

A South Florida fintech was six weeks into building a customer-facing LLM assistant with function-calling access to account operations. The engineering team had a working prototype, an initial system-prompt and function-allow-list design, and a target launch date twelve weeks out. Before development committed to the pattern for the remaining twelve weeks, the founders retained us for an architecture-level security review of the proposed design.

The review focused on the authorization boundary between the model and the account-operation functions rather than on any specific implementation defect. The design as proposed had the model receive the caller’s identity in the system prompt and pass it forward as an argument to each function call, with the function-side authorization checking the passed identity against the allow-list. The pattern is common and it does not survive prompt injection: any pathway that lets attacker-controlled content reach the model’s instruction context (indirect injection through summarization of pasted content, through retrieved documents, or through tool outputs) can induce the model to call functions with arguments the model itself has been told to use, escaping the intended caller-scoping.

We proposed an alternative pattern: authorization performed on the server side against the authenticated session, with the model receiving only opaque scoping tokens that the server resolves back to the caller identity when the function actually executes. The model cannot forge a token it does not have, and the authorization decision is made in code the model cannot influence. The pattern also produces a cleaner audit trail for the AI governance file.

Remediation cost twenty person-days at the architecture stage. The equivalent rewrite after ship (with production customer data, active support obligations and a documented incident record) would have been an order of magnitude larger. The launch went ahead on schedule, and the deployment’s subsequent pentest (with our AI security testing team, running the same adversarial scope described on the AI security testing page) found no path from prompt injection to function-calling abuse.

How We Actually Work

Scoping and architecture review kickoff with the engineering, security and product teams responsible for the deployment.
Design-artifact review of the current architecture diagrams, threat model (if one exists), data-flow documentation and any prior security review output.
Design-team interviews covering the reasoning behind the current design decisions, the constraints the design is working within, and the changes the team knows are already coming.
Threat modeling against the specific deployment, anchored on MITRE ATLAS and OWASP LLM Top 10, extended with the attack classes that have surfaced in comparable production deployments.
Findings and remediation proposal presented to the engineering and security teams, structured so the team can act on the specific patterns rather than parse an abstract report.
Governance-file documentation preserving the threat model, the design decisions, the residual-risk register and the reasoning for each significant choice.
Coordination handoff to the AI security testing team where the operator wants to confirm the remediated design against adversarial testing before launch.

Governance and Documentation Posture

Architecture engagements produce the documentation the operator will need for the AI governance file: the threat model, the design decisions, the residual-risk register, and the reasoning for each significant choice. The documentation is structured against the frameworks the operator is accountable to (NIST AI Risk Management Framework, ISO/IEC 42001, EU AI Act where applicable, sector-specific AI guidance) so the record slots into the compliance file rather than requiring translation.

For operators subject to ongoing AI governance obligations (regulated financial services, healthcare, high-risk AI deployments under the EU AI Act), we structure the engagement so the resulting architecture record supports periodic re-review as the deployment evolves rather than becoming a one-time snapshot.

Where Our Miami AI Architecture Practice Runs Deepest

Fintech, crypto and digital-asset AI features

Customer-facing assistants with function-calling access to account operations, automated trading and market-surveillance systems, fraud-detection and compliance AI. The Miami digital-asset cluster is one of the densest concentrations of consequential AI deployment work in the country, and architecture engagements anchor the deployment’s posture before either an incident or an examination forces the same analysis under worse conditions.

Regional hospital and clinical AI

Clinical decision-support systems, patient-facing conversational AI and administrative AI in healthcare environments. Architecture engagements scoped against HIPAA obligations and against the specific patient-safety and evidentiary standards clinical AI deployments face.

Hospitality and cruise-line AI

Guest-facing chatbots, booking assistants and revenue-management AI. Architecture engagements focused on the specific abuse patterns these deployments face and on the boundary between guest-facing AI features and the operator’s reservation and payment systems.

Cross-border and multilingual deployments

Deployments serving Latin American markets, where architecture engagements cover the specific data-residency and privacy-framework implications of multi-jurisdiction AI deployment.

Aviation and connected-infrastructure AI

AI and machine-learning components in aviation software and connected-infrastructure systems, structured against the specific evidence expectations of the applicable airworthiness cybersecurity framework.

Standards and Standing

Methodology anchored on NIST AI Risk Management Framework, ISO/IEC 42001 AI management systems standard, MITRE ATLAS (Adversarial Threat Landscape for AI Systems), OWASP Top 10 for LLM Applications, and the emerging body of adversarial-robustness architectural literature. Architects hold OSCP, OSWE, CISSP and specific practical experience with the current generation of LLM and ML deployment patterns. Documentation is structured to support the specific AI governance frameworks the operator is accountable to.

Last updated: September 4, 2026

The Cheapest Time to Fix an AI Architecture Defect Is Now

Architecture engagement before development commits to the pattern produces a cleaner deployment at a fraction of the cost of remediation after ship. Whether the deployment is pre-launch, mid-development or under review before a major change, the engagement pays for itself in the rewrites it prevents.

Request Consultation Call (786) 219-1210