AI-Powered Applications on Akamai
This document describes a reference architecture for AI-powered applications running on Akamai Connected Cloud. The architecture uses Zuplo AI Gateway with MCP server capabilities and Akamai AI Firewall to enable secure, enterprise-grade AI applications that can access internal data systems while maintaining strict security and compliance controls.
Overview
Enterprise AI applications face several challenges:
- Data access - AI models need access to live customer data from internal systems to provide accurate, personalized responses
- Security threats - AI-specific attacks like prompt injection can manipulate models into revealing sensitive information or behaving unexpectedly
- Data leakage - Models may inadvertently expose PII, credentials, or other sensitive data in their responses
- Cost management - Uncontrolled AI usage can lead to unexpected costs from LLM API calls
- Compliance - Organizations need audit trails and governance controls for AI interactions
This architecture addresses these challenges by combining Akamai's edge security platform, Zuplo's AI Gateway with MCP server capabilities, and Akamai AI Firewall.
Architecture
The following diagram shows the complete architecture:
Components
| Component | Description |
|---|---|
| AI Chat Application | Customer-facing chat interface that sends AI requests through the Akamai platform. |
| WAF + DDoS | Akamai's web application firewall and DDoS protection at the edge. |
| Akamai CDN | Global content delivery network that routes requests to the appropriate backend services. |
| Zuplo AI Gateway | Routes AI requests to configured LLM providers. Applies cost controls, rate limiting, and security policies. |
| Akamai AI Firewall | Analyzes AI interactions in real-time to detect and block prompt injection, PII leakage, and toxic content. |
| MCP Server | Exposes internal APIs as tools that the AI model can discover and invoke to retrieve live customer data. |
| AI Model (LLM) | Language model that generates responses, optionally trained on domain-specific knowledge. |
| Internal Data API | Backend services that provide access to customer records, account information, and other business data. |
Request Flow
A typical interaction flows through the system as follows:
-
Application sends request - The AI chat application sends a request to the Akamai edge platform.
-
Edge security - Akamai WAF and DDoS protection inspect the request for malicious patterns and attacks before routing through the CDN.
-
CDN routes to AI Gateway - The Akamai CDN forwards the request to the Zuplo AI Gateway, which applies authentication, rate limiting, and cost controls.
-
AI Firewall inspects request - The AI Gateway sends requests to the Akamai AI Firewall, which analyzes prompts for injection attempts, sensitive data, and policy violations.
-
Model invokes MCP tools - When the AI model needs customer data to answer a question, it invokes MCP server tools to query internal APIs.
-
MCP server retrieves data - The MCP server executes the tool call against the internal data API, returning structured information to the model.
-
AI Firewall inspects response - The model's response passes through the AI Firewall, which checks for PII leakage and inappropriate content.
-
Response delivered to application - The validated response streams back through the Akamai platform to the chat application.
MCP Server for Data Access
The MCP Server Handler transforms internal APIs into tools that AI models can discover and invoke. This pattern allows AI applications to access live data while maintaining security through the gateway's authentication and authorization policies.
Rather than embedding static data in the model or relying on retrieval-augmented generation (RAG) alone, the MCP server enables the model to make real-time API calls to fetch current information. The gateway enforces access controls on every tool invocation, ensuring the model can only access data the requesting user is authorized to see.
For more information, see the MCP Server documentation.
AI Firewall Protection
The Akamai AI Firewall provides enterprise-grade security for AI interactions:
- Prompt injection defense - Detects and blocks attempts to manipulate the AI model through deceptive inputs
- Data loss prevention - Identifies sensitive data (personal identifiers, credit cards, credentials) in both requests and responses
- Toxic content filtering - Prevents inappropriate or harmful content from being generated
- Adversarial attack protection - Guards against model exploitation attempts
When the firewall detects a threat, it can take one of three actions:
- Monitor - Log the threat for analysis without blocking
- Modify - Remove or redact sensitive content while allowing the request
- Deny - Block the request entirely and return an error
Cost and Usage Controls
The Zuplo AI Gateway provides hierarchical budget controls to manage AI spending:
- Organization limits - Maximum daily and monthly spending across all AI usage
- Team budgets - Allocated budgets for departments or customer segments
- Application limits - Per-application or per-use-case cost controls
- Rate limiting - Request throttling to prevent abuse
Security Model
This architecture enforces security at multiple layers:
Edge Security - Akamai WAF and DDoS protection filter malicious traffic before it reaches the AI infrastructure.
API Authentication - The AI Gateway authenticates all requests using API keys, JWT tokens, or other credentials before processing.
AI-Specific Security - The Akamai AI Firewall analyzes AI interactions for prompt injection, data leakage, and policy violations.
Data Access Controls - The MCP server mediates all data access through controlled API endpoints, preventing direct database access and enforcing field-level permissions.
Audit Trail - All AI interactions flow through the gateway, providing complete audit logs for compliance and security analysis.
Deployment
Zuplo provides a fully managed deployment experience on Akamai Connected Cloud. The Zuplo account team handles infrastructure provisioning, configuration, and ongoing maintenance.
Deployment options include:
- Any Akamai region - Deploy to Akamai Cloud regions that best serve users and meet data residency requirements
- Multi-region availability - Distribute the AI Gateway across multiple regions with automatic failover through Akamai GTM
- Custom networking - Private connectivity to backend services hosted on Akamai, other cloud providers, or on-premises
- Flexible scaling - Capacity scaling based on traffic patterns and performance requirements
Related Resources
- Akamai Dedicated Architecture - Overview of Zuplo on Akamai Connected Cloud
- MCP Server Handler - Technical documentation for MCP server configuration
- Akamai AI Firewall - AI security policy configuration
- Zuplo AI Gateway - Introduction to AI Gateway capabilities