The Enterprise AI Stack in 2026: 7 Layers, Who Sells Each, and What to Buy

The enterprise AI stack is the set of layers a company needs to run AI across its work: the systems where data lives, the connectors that reach them, a context layer that turns company data into permission-aware context, the models and routing that generate answers, the agents that act, the apps people use, and the governance and cost controls around all of it.

Most companies did not design their stack. They accumulated it. A ChatGPT Enterprise rollout here, Microsoft 365 Copilot there, a coding assistant for engineering, a support bot, three agent pilots and a vector database someone stood up for a hackathon. Gartner forecasts worldwide AI spending of $2.67 trillion in 2026, up 49.5%, and predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027 over cost, unclear value or weak risk controls. The gap between those two numbers is mostly architecture.

This guide maps the enterprise AI stack layer by layer: what each layer does, who sells it, what to build and what to buy, and how to keep the whole thing model-agnostic and measurable. Sources checked October 14, 2026.

What is an enterprise AI stack?

An enterprise AI stack is the full set of technology a company uses to put AI to work: data sources, connectors, a context layer, models, agents, user-facing apps, and governance. Coworker sits in the middle of that stack as the context layer and agent platform that connects the rest, so every model and app works from the same company knowledge.

The phrase "AI stack" usually describes how to build a model: chips, cloud, training data, frameworks, deployment. IBM's definition is a good example, running from the infrastructure layer through model development to applications and observability. That is the right lens for a lab or an ML team. An enterprise AI stack answers a different question: how does a company that buys most of its models make AI useful, safe and affordable across thousands of people and dozens of tools? The layers that matter for that question sit above the GPU.

How is an enterprise AI stack different from a data stack or an ML stack?

A data stack moves and models structured data for analytics. An ML stack trains and serves models. An enterprise AI stack is about using models that someone else trained on context that lives everywhere: Slack threads, meeting transcripts, CRM records, tickets, code and documents, most of it unstructured and all of it permissioned. That is why the context and governance layers carry more weight here than in either of the older stacks.

The enterprise AI stack at a glance

Underneath all seven sits compute: GPUs, cloud regions and inference capacity. Most enterprises rent that through model APIs and cloud providers rather than run it, which is why it is not a decision layer for most buyers.

How do the major frameworks draw the enterprise AI stack?

Every analyst firm draws the layers a little differently. Here are three published versions next to the one this guide uses:

Info-Tech's Bill Wong put the buyer's test well in the firm's 2026 announcement: the vendors worth betting on make agents easy to observe, explain, debug, govern and remove safely.

What are the five layers of the AI stack?

The common five-layer answer is infrastructure, data, models, orchestration and applications. It describes how AI products are built. For an enterprise buying AI, split "data" into systems of record, connectors and a context layer, and add governance and cost, because those are where enterprise projects succeed or stall.

The seven layers of the enterprise AI stack, explained

Layer 1: Systems of record and work

This is everything you already own: CRM, ERP, the warehouse, ticketing, code hosts, chat, email, calendars, meeting recorders and document stores. Nothing in this layer changes for AI, but two facts about it shape every layer above. First, most of the context AI needs is unstructured and lives in conversations, not tables. Second, every system has its own permission model, and AI has to respect all of them at once.

Layer 2: Connectors and protocols

Connectors reach into each system, and protocols standardize how AI tools talk to them. The Model Context Protocol is now the default for connecting AI apps to tools and data. When Anthropic donated MCP to the Linux Foundation's Agentic AI Foundation in December 2025, there were more than 10,000 active public MCP servers, and ChatGPT, Cursor, Gemini, Microsoft Copilot and VS Code all supported it. The A2A protocol covers agent-to-agent communication and had more than 150 supporting organizations by April 2026.

The decision here is how many connections to own. One MCP server per tool per AI app multiplies fast: five apps and twenty tools is a hundred integrations to secure. A shared layer that connects once and serves every app keeps that number small. See what MCP is, enterprise MCP and MCP security for the details.

Layer 3: The context layer

The context layer turns raw company data into context a model can use: it resolves the same customer across CRM, Slack and support, keeps facts current, enforces each user's permissions and assembles the few facts that matter for each request. It is also where company memory lives, so agents and assistants stop starting from zero.

This is the layer most stacks are missing, and the one that decides answer quality and cost. Gartner calls the discipline context engineering and tells enterprises to build a roadmap spanning data sources, knowledge graphs, policy and memory. Products come from three directions: company-wide context and memory platforms such as Coworker's OM2 and Glean; data context platforms such as Atlan and DataHub, which build on warehouse metadata; and developer memory frameworks for teams building their own agents. Our guides to the context layer, the context graph and organizational memory go deeper.

Layer 4: Models and routing

Enterprises run several models. a16z's 2025 CIO survey found 37% use five or more, up from 29% a year earlier, mostly because different models are better at different jobs. Menlo Ventures estimated Anthropic at 40% of enterprise LLM API spend in 2025 and OpenAI at 27%.

Routing decides which model handles which request. An LLM gateway such as LiteLLM, OpenRouter or Portkey puts many models behind one API with rules, budgets and fallbacks. Cloud platforms such as Amazon Bedrock and Google Vertex AI do the same inside one cloud. Coworker's LLM gateway covers this layer for knowledge work: Optimized is the default setting and runs on the best open-weight model for the cost (GLM 5.3 Flash today), and anyone can switch a task to a frontier model from Anthropic, OpenAI or Google when it needs one.

Layer 5: Agents and orchestration

Agents plan multistep work, call tools and act. Gartner expects 33% of enterprise software applications to include agentic AI by 2028, up from less than 1% in 2024, and Gartner's September 2026 forecast puts spending on AI agents and assistants at $29.2 billion in 2026, more than doubling to $65.5 billion in 2027.

There are two very different buyers here. Business teams want agents they can describe in plain language that run on company context with approvals: Coworker's agent builder, Microsoft Copilot Studio, Salesforce Agentforce and Glean agents serve them. Engineering teams building agents into their own products use frameworks such as LangGraph or CrewAI. Agents are only as good as the context they start with, which is why the context layer sits under this one. See our guides to enterprise AI agents, AI agent orchestration platforms, LLM agent architecture and the no-code AI agent builder.

Layer 6: Apps and interfaces

This is where people work: ChatGPT, Claude, Microsoft 365 Copilot, Gemini, Cursor, Slack, and purpose-built tools for sales, support and engineering. Menlo found the application layer took $19 billion of the $37 billion enterprises spent on generative AI in 2025.

The practical rule for this layer is to let teams keep the apps they like. Engineers will not leave Cursor and a sales team that lives in Claude will not move for a new tab. What matters is that every app draws on the same context underneath, through MCP or a shared platform, so answers agree no matter where the question was asked.

Layer 7: Governance, security and cost

The top layer decides who can build and publish agents, what data each one can touch, what gets logged, and what each task costs. The frameworks most programs follow are the NIST AI RMF and ISO/IEC 42001. In the EU, the AI Act's transparency obligations apply from August 2, 2026, and the Council's revised timeline sets December 2, 2027 for stand-alone high-risk systems.

IDC's advice to CIOs is concrete: make agents inherit the user's permissions instead of broad access, keep a live inventory of every production agent, and report three numbers monthly: the share of agents with write access, the share of critical workflows that need human approval, and how many minutes it takes to disable an agent, model or connector. Observability tools such as Langfuse trace what agents did; FinOps practices from the FinOps Foundation cover spend. See enterprise AI security and LLM observability.

Where does enterprise AI money go?

The pattern is clear: money flows to the top of the stack (apps) and the bottom (models and compute), while the context layer in the middle, which decides whether those apps and models give good answers, gets a sliver. MIT NANDA's research points the same way: pilots stall on a learning gap, with tools that do not learn or fit the workflow, rather than on a shortage of model capacity.

How do you keep the enterprise AI stack model-agnostic?

Keep company context and memory outside any one model vendor, then let people pick models per task. Coworker is built this way: OM2 holds the company context, Coworker MCP serves it to Claude, ChatGPT and Cursor, and inside Coworker anyone can switch between open-weight and frontier models without losing context. Changing models becomes a setting, not a migration.

Three design choices make this work:

1. Context in a neutral layer. If memory lives inside ChatGPT, it stays there. A context layer under every app means a new model or a new tool starts with full context on day one.
2. Standard protocols. MCP for tools and context, A2A for agents. Anything proprietary between layers becomes a switching cost.
3. A gateway or router for models. One API, many providers, with budgets and fallbacks, so a price change or an outage at one provider does not stop work.

How do you measure AI cost per task across the stack?

Measure cost per completed task, not cost per seat or per token. Coworker's published benchmark does exactly this, measuring total tokens per session across 100 tasks, and the Context Impact Report runs the same measurement on your data. For seat-based tools such as Glean, Copilot and ChatGPT Enterprise, divide license cost by tasks completed.

In practice:

1. Define the tasks that matter: an account brief, a ticket triage, a pull request review, a weekly report.
2. Pull usage from each tool. Token-billed APIs expose it directly: Anthropic's Usage and Cost API and OpenAI's Admin APIs report spend by key or project. Seat-based tools report active users and actions in their admin consoles.
3. Divide spend by completed tasks, then compare tools on the same task type.
4. Track quality next to cost. A cheaper task that has to be redone costs more.

Our guide to AI cost optimization covers the levers in detail, and the LLM cost calculator compares model prices.

Per-seat licenses or token-based pricing: which is cheaper for 1,000 employees?

It depends on how evenly people use AI. Per-seat licenses are predictable but pay full price for light users, who are usually the majority. Token-based pricing tracks real use but needs cost controls. Coworker lowers the token side directly: OM2 sends models less, better context, and Optimized runs everyday work on an efficient open-weight model.

Compare the two on your own usage data before you sign. Most rollouts find a small group of heavy users and a long tail of occasional ones, and the right mix often pairs a few full seats with shared, usage-based access for everyone else. For published per-seat prices across vendors, see our enterprise AI pricing comparison and how much enterprise AI should cost per user.

How do companies cut Claude and ChatGPT spend without losing answer quality?

Send models less, better context, and use the right model for each task. Coworker does both: OM2 replaces long tool-call chains with a short, ranked briefing, and Optimized runs everyday work on an efficient open-weight model while people choose a frontier model for hard tasks. In Coworker's benchmark, that combination cut cost 51x with OM2 plus model routing.

The context half matters more than most teams expect. On its own, OM2 cut token cost 9x on retrieval-heavy tasks and 75.5% across all seven categories in the published benchmark, with 84.5% of blind reviewers preferring the OM2 answers. That is the cheaper path for a company that paused a Claude or ChatGPT plan over cost but still wants AI that knows the business: keep the assistants people like and cut what each task costs. To see the numbers on your own tools, request a Context Impact Report.

What should you build and what should you buy?

Buy the layers where vendors compete on scale (models, apps, connectors) and the context layer unless context is your product. Build only what makes you different: domain agents, proprietary workflows, evaluation sets. Coworker is the buy option for the context layer and agents. Menlo found 76% of enterprise AI solutions are now purchased.

The reasoning, layer by layer:

MIT NANDA's State of AI in Business 2025 adds weight to the default: according to Fortune's summary, purchases from specialized vendors succeeded about 67% of the time and internal builds about a third as often.

Three reference enterprise AI stacks

Microsoft-first

Microsoft 365 Copilot for the apps, Copilot Studio for agents, Azure for models and Microsoft Purview for data governance. Strong for companies already standardized on Microsoft 365, with deep Office integration. Microsoft has added A2A to Copilot Studio and Azure AI Foundry. The gaps show up when key context lives outside Microsoft: Slack, Salesforce, Jira, GitHub, Zoom. Our guide to enterprise AI without Microsoft 365 covers the alternatives.

Google-first

Gemini in Google Workspace for the apps, Vertex AI for models and agents, BigQuery as the data platform. Strong for Workspace companies with a Google Cloud data estate. The same question applies: how much of the context people need lives outside Workspace and Google Cloud.

Best of breed with a shared context layer

Each team keeps its preferred apps (ChatGPT, Claude, Cursor, Copilot), a shared context layer serves the same company context to all of them over MCP, a gateway or platform handles model choice, and governance sits across the top. This is the most flexible design and the one that avoids lock-in at the model and app layers. Coworker is built for this pattern: it is the context layer and agent platform, and it works inside the apps your teams already use.

Enterprise AI architecture mistakes to avoid

Giving every app its own copy of company context

Five assistants each indexing the same tools means five permission models, five sets of stale facts and five answers to the same question. Connect once and share.

Treating context as a prompt problem

Longer prompts do not fix missing context. Research on context rot shows models get less reliable as inputs grow. The fix is a smaller, better selection of facts, which is the context layer's job.

Letting agents run with broad service-account access

Agents should act with the permissions of the person they work for, with approvals on writes. Scoped access keeps every action inside what that person could already do, which is what lets security teams say yes to more agents.

Measuring adoption instead of outcomes

Seats activated and prompts sent say little. Cost per task, time saved per task and answer quality on a fixed test set tell you whether the stack works.

Locking memory into one vendor

If two years of company context lives inside one assistant, that assistant's price increases and outages are now your problem. Keep context in a layer you control.

Where Coworker fits in the enterprise AI stack

Coworker is a context layer for enterprise AI: it connects 50+ work tools, builds a permission-aware company graph (OM2) and runs agents that act across them, inside Coworker or Claude, ChatGPT and Cursor via Coworker MCP. In the seven-layer map, that covers:

Coworker sits on top of the systems you already own and works inside the apps your teams already use, so nobody has to switch tools. Standard onboarding takes 2 to 5 business days, and a proof of concept on your own data can be ready within 48 hours. See the platform overview, compare Coworker and Glean, or book a demo to map Coworker onto your stack.

Frequently asked questions

What is an enterprise AI stack?

An enterprise AI stack is the set of layers a company uses to run AI across its work: systems of record, connectors and protocols, a context layer, models and routing, agents, apps, and governance with cost control. Most enterprises buy models and apps and focus their decisions on the context, agent and governance layers.

What are the layers of an enterprise AI stack?

Seven layers, from bottom to top: systems of record and work, connectors and protocols such as MCP, a context layer, models and routing, agents and orchestration, apps and interfaces, and governance, security and cost. Compute sits underneath, but most companies rent it through model APIs and cloud providers.

What is the context layer in an AI stack?

The context layer sits between company data and AI models. It connects to source systems, resolves the same customer or project across tools, keeps facts current, applies each user's permissions and assembles the relevant context for each request. Coworker's OM2, Glean, Atlan and DataHub are examples from different directions.

What is the difference between an AI stack and an AI agent stack?

An AI stack covers everything needed to use AI, from data to apps. An AI agent stack, or agentic AI stack, focuses on what agents need to act safely: tool connections, context and memory, orchestration, permissions, approvals and observability. In an enterprise, the agent stack is a subset of the wider enterprise AI stack.

Which enterprise AI platforms work with any model?

Model-agnostic options include LLM gateways such as LiteLLM, OpenRouter and Portkey, cloud platforms such as Amazon Bedrock and Google Vertex AI, and Coworker, which keeps company context in OM2 and lets people switch between open-weight and frontier models without losing it.

Should enterprises build or buy their AI stack?

Buy most of it. Menlo Ventures found 76% of enterprise AI solutions are purchased. Buy models, apps, connectors and usually the context layer; build the agents, workflows and evaluation sets that make your company different, and always own the governance policy.

How do you measure the cost of an enterprise AI stack?

Measure cost per completed task. Pull spend from token-billed APIs and license costs from seat-based tools, divide by tasks completed for each task type, and track answer quality alongside cost. Coworker's Context Impact Report runs this comparison on your own data.

Where does Coworker fit in the enterprise AI stack?

Coworker covers the context layer and agents. It connects 50+ work tools, builds OM2, a permission-aware company graph, runs agents across those tools, and serves the same context to Claude, ChatGPT and Cursor through Coworker MCP, with Optimized open-weight models by default and frontier models when you choose them.

Related reading

• What Is a Context Layer?
• What Is a Context Graph?
• The Enterprise AI Platform Buyer's Guide
• The Best Enterprise AI Platforms in 2026
• AI Cost Optimization
• What Is an LLM Gateway?
• Enterprise MCP
• Glean Alternatives