FedRAMP for Government LLM Use

Understanding Federal Risk and Authorization Management Program requirements for using AI and LLMs in government agencies

The Federal Risk and Authorization Management Program (FedRAMP) is a government-wide program that provides a standardized approach to security assessment, authorization, and continuous monitoring for cloud products and services used by federal agencies. U.S. federal agencies can only use LLM services that have achieved FedRAMP authorization at the appropriate impact level for their data classification.

📋 Key Requirement

Per OMB memoranda and agency policies, federal agencies must use FedRAMP authorized cloud services. Using non-authorized LLM services for government work—even for unclassified data—violates federal policy and can result in security incidents, contract violations, and potential data breaches.

1

FedRAMP Impact Levels Explained

FedRAMP categorizes cloud services into three impact levels based on FIPS 199 standards. The impact level determines the security controls required:

Low Impact (LI-SaaS)

125 Controls

Use Case: Public-facing applications with data already available to the public or intended for public disclosure.

Confidentiality Impact: Loss would have limited adverse effect on operations, assets, or individuals.

Examples:

  • Public-facing chatbots for general inquiries
  • Analysis of publicly available datasets
  • Content generation for public websites
  • Processing Freedom of Information Act (FOIA) responses

Note: Even for Low impact, FedRAMP authorization is still required for federal use.

Moderate Impact

325 Controls

Use Case: Most federal data not intended for public release, including internal communications and operational data.

Confidentiality Impact: Loss would have serious adverse effect on operations, assets, or individuals.

Examples:

  • Internal policy drafting and review
  • Grant application analysis (non-PII)
  • Summarizing internal meeting notes
  • Budget and financial planning documents
  • Contract analysis and recommendations

Most Common: Over 80% of federal agencies require Moderate or higher impact authorization.

High Impact

421 Controls

Use Case: Critical systems and data where loss could have severe or catastrophic adverse effects.

Confidentiality Impact: Loss would have severe or catastrophic adverse effect on operations, assets, or individuals.

Examples:

  • Law enforcement case files and investigations
  • National security information (up to SECRET level with additional controls)
  • Critical infrastructure protection data
  • Emergency response systems
  • Sensitive intelligence analysis

⚠️ High Impact services are rare. Only a handful of LLM providers offer this level.

📊 How Impact Levels Are Determined

Agencies assess information and information systems using FIPS 199 security categories based on three security objectives:

Confidentiality

Preserving authorized restrictions on access and disclosure

Integrity

Guarding against improper modification or destruction

Availability

Ensuring timely and reliable access to information

The highest rating across all three objectives determines the overall impact level.

2

LLM Vendors with FedRAMP Authorization

As of 2025, the following major cloud providers offer LLM services with FedRAMP authorization. Always verify current status on the FedRAMP Marketplace:

Google Cloud (Vertex AI)

Including Gemini models

Moderate

Google Cloud Platform has FedRAMP High authorization. Vertex AI (including Gemini Pro and other models) inherits this authorization when deployed within Google Cloud's FedRAMP boundary.

  • Must use us-gov regions for High impact workloads
  • Data residency controls available
  • Zero data retention policies configurable
  • Customer-managed encryption keys (CMEK) supported

Amazon Web Services (AWS)

Bedrock and SageMaker

High

AWS has FedRAMP High authorization. Amazon Bedrock (Claude, Llama, etc.) and SageMaker are authorized services when used in AWS GovCloud.

  • AWS GovCloud (US) regions required for Moderate/High
  • Bedrock offers Claude, Llama, Mistral, and Amazon Titan models
  • SageMaker for custom model hosting
  • No data used for model training when using Bedrock

Microsoft Azure

Azure OpenAI Service

High

Azure Government has FedRAMP High authorization. Azure OpenAI Service (GPT-4, GPT-3.5, embeddings) is available in Azure Government regions.

  • Must use Azure Government regions (US Gov Virginia, US Gov Arizona)
  • GPT-4, GPT-4 Turbo, GPT-3.5 Turbo available
  • Data is NOT used for model training or improvement
  • Customer Lockbox available for High impact

Anthropic Claude (via AWS Bedrock)

Available through AWS GovCloud

High

Claude models (Sonnet, Haiku) are available via AWS Bedrock in GovCloud regions, inheriting FedRAMP High authorization.

  • Claude 3 Sonnet and Haiku available in GovCloud
  • Must access through AWS Bedrock (not direct Anthropic API)
  • No training on federal data
  • Longer context windows than OpenAI models

Oracle Cloud Infrastructure (OCI)

OCI Generative AI Service

Moderate

OCI has FedRAMP Moderate authorization and offers generative AI services including Cohere models.

  • Cohere Command and Embed models
  • Available in US government regions
  • Custom model fine-tuning options

❌ NOT FedRAMP Authorized

The following popular LLM services do NOT have FedRAMP authorization and cannot be used for federal government work:

  • OpenAI API (direct access to openai.com) - use Azure OpenAI instead
  • Anthropic API (direct access to anthropic.com) - use AWS Bedrock instead
  • ChatGPT consumer interface
  • Claude.ai consumer interface
  • Gemini consumer interface (gemini.google.com) - use Vertex AI instead
  • Open-source models on non-FedRAMP infrastructure
3

FedRAMP Authorization Process

Cloud service providers must undergo rigorous assessment to achieve FedRAMP authorization. Understanding this process helps explain why only established providers offer FedRAMP-authorized LLMs:

1

Package Development

Cloud service provider (CSP) creates a System Security Plan (SSP) documenting how they meet security controls.

  • 125 controls for Low, 325 for Moderate, 421 for High
  • Includes architecture diagrams, data flow, encryption methods
  • Typically 500-1000+ pages for Moderate/High
2

Independent Assessment

Third-Party Assessment Organization (3PAO) validates the CSP's security controls.

  • Penetration testing and vulnerability scanning
  • Interviews with security and engineering teams
  • Review of policies, procedures, and technical implementations
  • Results documented in Security Assessment Report (SAR)
3

Authorization

Two pathways to authorization:

JAB P-ATO (Provisional Authority to Operate)

Joint Authorization Board (JAB) reviews and grants provisional authorization. Considered "gold standard" - all agencies can use it.

Agency ATO

Individual agency reviews and grants authorization. Other agencies can leverage this, but may conduct additional review.

4

Continuous Monitoring

Authorization is not a one-time event. CSPs must continuously monitor and report:

  • Monthly: Automated vulnerability scans
  • Quarterly: Security posture updates, ConMon deliverables
  • Annually: Full assessment and re-authorization
  • As needed: Significant change requests (SCRs) for major updates

⏱️ Timeline and Cost

Initial FedRAMP authorization typically takes 12-18 months and costs $250,000-$500,000+ for Moderate/High impact. Continuous monitoring adds $100,000-$300,000 annually. This is why smaller LLM startups don't have FedRAMP authorization - it requires significant enterprise investment.

4

Using FedRAMP-Authorized LLMs Correctly

Simply having FedRAMP authorization doesn't automatically make LLM usage compliant. Agencies must follow these requirements:

✅ 1. Use the Correct Region

FedRAMP authorization is region-specific. For Moderate/High impact:

  • AWS: Must use AWS GovCloud (US-East or US-West), NOT commercial us-east-1
  • Azure: Must use Azure Government (US Gov Virginia, US Gov Arizona), NOT Azure Commercial
  • Google Cloud: Use us-gov regions for High impact; some commercial regions authorized for Moderate

❌ Common Mistake: Using Azure OpenAI in commercial Azure regions. This is NOT FedRAMP compliant even though the service is authorized in government regions.

✅ 2. Configure for Zero Data Retention

Ensure your LLM configuration prevents data from being used for training:

  • AWS Bedrock: Data is NOT used for training by default (confirmed in AWS Customer Agreement)
  • Azure OpenAI: Government deployments do NOT use data for training; no opt-out needed
  • Google Vertex AI: Configure data residency and opt out of data logging if required by agency policy

✅ 3. Obtain Agency ATO

Even with FedRAMP P-ATO, your agency must grant its own Authority to Operate (ATO):

  • Submit request to your agency's Authorizing Official (AO)
  • Document your specific use case and data classification
  • Agency reviews FedRAMP package and may add additional controls
  • ATO granted for specific time period (usually 1-3 years)

This can take 30-90 days depending on agency processes.

✅ 4. Implement Interconnection Security Agreements (ISAs)

If integrating LLM services with other systems:

  • Document data flows between your systems and the LLM service
  • Identify boundary protections (firewalls, encryption, access controls)
  • Establish monitoring and incident response procedures
  • Review and update ISA annually or when significant changes occur

✅ 5. Maintain Audit Logs

NIST 800-53 requires comprehensive audit logging:

  • Log all API calls to LLM services (use CloudTrail, Azure Monitor, etc.)
  • Record user identity, timestamp, prompt summary, response metadata
  • Retain logs per agency retention policy (typically 90 days to 7 years)
  • DO NOT log full prompts/responses if they contain sensitive data

✅ 6. Train Users on Acceptable Use

Establish and enforce acceptable use policies:

  • What data classifications can be processed (e.g., CUI allowed, classified forbidden)
  • Prohibited uses (personal tasks, non-official business)
  • How to report security incidents or unexpected outputs
  • Consequences of policy violations
5

Common Government LLM Use Cases

Federal agencies are using FedRAMP-authorized LLMs for these approved use cases:

✅ Policy Analysis & Drafting

Analyze existing regulations, identify conflicts, suggest language improvements.

Impact Level: Moderate (internal drafts) to Low (public-facing policies)

✅ FOIA Request Processing

Categorize requests, search document repositories, draft initial responses.

Impact Level: Moderate (requests may reference CUI)

✅ Grant Application Analysis

Score applications, identify missing requirements, summarize proposals.

Impact Level: Moderate (pre-award applications contain business info)

✅ Citizen Support Chatbots

Answer FAQs, guide users through forms, provide program information.

Impact Level: Low (public-facing information)

✅ Contract Review & Analysis

Extract key terms, identify non-standard clauses, check FAR compliance.

Impact Level: Moderate (pre-award source selection sensitive)

✅ Congressional Correspondence

Draft responses to constituent inquiries, research relevant programs and statistics.

Impact Level: Moderate (may reference internal deliberations)

✅ Data Quality Improvement

Identify inconsistencies in databases, standardize formats, suggest corrections.

Impact Level: Depends on data sensitivity (often Moderate)

✅ Code Documentation

Generate inline comments, API documentation, and architecture diagrams from code.

Impact Level: Moderate (source code may reveal vulnerabilities)

❌ Prohibited Use Cases

Even with FedRAMP High authorization, these use cases are typically prohibited:

  • Processing classified information (requires separate accreditation beyond FedRAMP)
  • Law enforcement investigative case files (CJIS compliance may be required)
  • Tax return information (IRS-specific authorization required per IRC 6103)
  • Export-controlled technical data without proper controls (ITAR, EAR compliance)
  • Healthcare records without HIPAA Business Associate Agreement (BAA)
6

Key Differences: Government vs. Commercial Cloud

FedRAMP-authorized services in government regions have important differences from commercial offerings:

Aspect Commercial Cloud Government Cloud (FedRAMP)
Available Models Latest models (GPT-4 Turbo, Claude 3 Opus) available first Slightly delayed rollout (3-6 months lag for new models)
Regions 30+ global regions US-only regions (GovCloud, Azure Gov)
Pricing Standard commercial rates 10-30% premium for government regions
Support Personnel Global support teams US persons only (screened personnel)
Data Residency May replicate across regions Guaranteed US residency, no cross-border transfers
Account Setup Self-service, instant Requires vetting, 1-4 weeks approval process

Best Practices for FedRAMP LLM Usage

DO

  • Verify FedRAMP authorization status on marketplace.fedramp.gov before using any service
  • Use government-specific regions (GovCloud, Azure Gov) for Moderate/High impact data
  • Obtain agency ATO before deploying, even with FedRAMP P-ATO
  • Maintain comprehensive audit logs of all LLM API calls and access
  • Train users on data classification and acceptable use policies
  • Review FedRAMP continuous monitoring reports quarterly

DON'T

  • Use consumer LLM interfaces (ChatGPT, Claude.ai) for official government work
  • Deploy in commercial cloud regions for Moderate/High impact workloads
  • Assume FedRAMP P-ATO is sufficient without agency-specific ATO
  • Process classified information with FedRAMP services (requires separate accreditation)
  • Mix government and commercial cloud resources in the same architecture
  • Skip continuous monitoring requirements after initial authorization

Need Help Navigating FedRAMP for Your Agency?

We can help you select the right FedRAMP-authorized LLM service, obtain agency ATO, and implement compliant AI solutions for government use.

Schedule a Consultation