The Question Federal Agencies Should Ask Before Deploying AI
This post was originally published on OrangeSlices AI.
Federal agencies are under real pressure to deploy AI. Leadership wants results, mission demands are growing, and every week, there's a new capability that seems to promise a faster, smarter way to get work done.
But there's a question that many agencies skip, and skipping it leads to misallocated security controls, compliance gaps, and deployments that create more risk than they resolve.
The question is: Are you buying AI, building with AI, or owning AI?
On the surface, it sounds simple, but it isn't. And getting it wrong has real consequences.
Why This Question Changes Everything
When most agencies say "we're integrating AI," they're actually describing one of three very different scenarios, each with a distinct risk profile, threat surface, and set of required controls.
If you skip the scoping step and jump straight into security architecture, you're essentially designing a physical security plan without knowing whether you're protecting a storefront, a warehouse, or a data center. The assets and threats are different. The controls need to be different too.
When working with one large federal health agency, our team went through this exact scoping process on a machine learning (ML) and AI engagement before writing a single line of security architecture. The agency knew what outcomes to expect with their AI chatbot deployment, and we worked with them to assess whether buying, building, or fully owning their AI infrastructure would produce the desired results, assessing their needs against what is currently available. We evaluated each scenario against cost, usability, and mission fit before landing on the right approach. That conversation shaped every security decision that followed.
Here's what the three scenarios actually look like in practice.
Scenario 1: Buying AI
This is where most agencies start. You're procuring a commercial SaaS AI product such as Microsoft Copilot, a cloud-hosted large language model (LLM) application programming interface (API), or a government-specific AI interface. The vendor handles the model and the infrastructure. You're a consumer.
The key question to ask yourself in this scenario is, “Are we sending agency data to a commercial provider we don't fully control?”
If the answer is yes, your biggest risks are data leakage and terms-of-service compliance, not model architecture. Employees may paste controlled unclassified information (CUI) into a chat interface without realizing it leaves your perimeter. The model may retain or use submitted data in ways that violate policies.
Having spent significant time building SaaS governance programs, I can tell you that third-party risk management is where this scenario lives or dies. Assessing the risks in a SaaS procurement isn't a checkbox exercise; it's the work. Your controls should focus on data loss prevention (DLP), acceptable use policies, and a thorough legal review of vendor terms before anyone types a single prompt. The same rigor you'd apply to any sensitive SaaS adoption applies here, just with higher stakes.
If this is your scenario, prioritize a vendor risk assessment, DLP tuned for AI interfaces, and a formal acceptable use policy that defines trust zones for what data can go to which tool. If your vendor can't give you clear answers on data retention, training use, or where prompts are processed, treat that as a procurement blocker. Agencies without in-house capacity to vet these terms should bring in a partner who can pressure-test vendor claims before contract signature.
Scenario 2: Building with AI
Your agency wants more control. You're using a cloud AI service such as Amazon Bedrock, Azure OpenAI, or Google Vertex AI, and layering your own data on top through retrieval-augmented generation (RAG). Maybe it's a document Q&A system, a policy knowledge base, or a case summarization tool.
The key question here is, “Are we connecting a pre-trained model to our own data without fully controlling what comes back?”
This is where the threat model shifts significantly. Prompt injection attacks become a serious concern. An adversary who can manipulate what gets retrieved from your knowledge base can corrupt the model's reasoning. Your vector database needs proper access controls. Your RAG pipeline needs input sanitization and output filtering.
The risks here aren't about whether your data leaves the perimeter. They're about what happens when external inputs interact with internal data.
If this is your scenario, your priorities are access controls on the vector database, input sanitization and output filtering in the RAG pipeline, and authorization logic that enforces document-level permissions on what the model can retrieve, especially when your corpus spans multiple classification levels. If your cloud provider or integrator can't walk you through how retrieval is secured end-to-end, or how they'd detect a poisoned document in the knowledge base, you need a partner with demonstrated RAG security experience, not just deployment experience.
Scenario 3: Owning AI
Some agencies, particularly those with classified data or unique mission requirements, are training or fine-tuning their own models. This gives you the most control and the most complexity.
The key question to ask is, “Are we building and managing the model itself, including the training pipeline and underlying data?”
Here, the model is a security asset. The training corpus needs to be protected. Your entire ML pipeline, including third-party libraries, compute infrastructure, and data ingestion processes, becomes an attack surface. Supply chain risk is real. A compromised dependency or poisoned dataset doesn't just create a bug; it corrupts the intelligence of the system itself.
If this is your scenario, treat the model, training corpus, and ML pipeline as mission-critical assets. That means data version control linking each model to its exact training data, an AI bill of materials (AIBOM) for every dependency, and supply chain controls across libraries, compute, and ingestion. This is the scenario where the wrong partner creates the most damage. Vet for demonstrated experience securing training pipelines and defending against supply chain and model extraction threats, not general ML delivery.
Why Getting This Wrong Is Costly
The mistake agencies make isn't deploying AI. It's applying the same generic controls across all three scenarios regardless of what they're actually doing.
An agency in scenario 1 (buying AI) that's worried about training data security is solving the wrong problem. An agency in scenario 3 (owning AI) that's focused on vendor terms of service review is missing the real threat. And an agency that doesn't know which scenario it's in is flying blind.
The scoping decision shapes everything downstream: your security architecture, your compliance posture, and what threats your team should be pressure-testing. Cost and usability factor into the decision, too, but you can't make a sound tradeoff if you haven't first mapped the risk landscape for each path.
Ask the Question First
Before your agency stands up its next AI initiative, before you design a single control or draft a single policy, ask the question: Are we buying, building, or owning?
The answer won't just help you pick the right guardrails. It'll help you avoid wasting resources on the wrong ones and give you a clear, defensible rationale for why your AI deployment is secure.
The AI scoping matrix in Aquia’s Federal AI Guardrail Framework includes a side-by-side view of risks, controls, and OWASP threat priorities for each scenario, which is the natural next step once you've answered the buy/build/own question. You can access that resource for free here.
