Copilot & Foundry: The Microsoft AI Stack for Small Business · Lesson 4 of 10

Tier 3: Microsoft Foundry (Customize)

Time: ~30 min reading + ~15 min worksheet Goal: Understand what Foundry actually is, when it's the right tier, how it differs from Copilot Studio, and how to talk about it credibly with clients who think they want it (but usually don't yet).

---

Microsoft Foundry is Microsoft's pro-code platform for building AI applications and agents. Formerly Azure AI Studio, then Azure AI Foundry, now just "Foundry." The name will keep drifting — the capability is what matters.

Foundry is where developers, data scientists, and software engineers build production-grade AI. It exposes the full stack: model selection, fine-tuning, prompt engineering, grounding/RAG, evaluation, safety, observability, deployment, scaling. Python SDK, REST APIs, VS Code integration, CI/CD support. Everything a real engineering team needs to ship AI software.

In the consume/compose/customize model, Foundry is customize. You're not snapping building blocks together — you're architecting a system.

Foundry has many moving parts. The ones you need to know for client conversations:

The Model Catalog

Over 11,000 models available. Not just OpenAI. That catalog includes:

• OpenAI's frontier models (GPT-5, GPT-5 nano, etc.)
• Microsoft's own models (Phi family)
• Open-source: Meta's Llama, Mistral, Alibaba's Qwen, Google's Gemma
• Specialized: domain-specific models (medical, legal, financial), vision models, audio models, code models
• Partner models from NVIDIA, Cohere, Hugging Face, and others

The significance: you're not locked into OpenAI. If your client has data sovereignty concerns, wants a specific open-source model, or needs a cheaper model for high-volume low-complexity work, Foundry lets you pick.

Foundry IQ

The RAG/grounding layer. Works with Azure AI Search to build sophisticated retrieval over diverse data sources (structured, unstructured, multi-modal). This is how you build a serious knowledge base — one that can handle tens of thousands of documents with proper semantic search.

Foundry IQ is a step up from the RAG built into Copilot Studio. More control over chunking, embedding strategy, ranking, and hybrid search. Necessary for scale.

Foundry Agent Service

The managed runtime for production agents. Handles:

• Conversation state
• Tool/function calling orchestration
• Safety and content filtering
• Identity and auth
• Observability and tracing
• Dev-to-prod promotion

Think of it as the "hosting and scaffolding" for a real agent you built in Foundry. Without it, you're wiring up all this infrastructure yourself.

Foundry Local

The big one for regulated-industry sales. Foundry Local lets you run models on your own hardware — on-premises servers, edge devices, even laptops. All the governance, tooling, and management of Foundry, but with no data leaving your physical environment.

This is a real answer to the "we can't use cloud AI" objection. Healthcare, legal, defense contractors, regional banks, anyone with strict data residency requirements — Foundry Local is specifically designed for them.

Azure AI Search

The underlying search and indexing service. Vector search, semantic search, hybrid search. Not strictly part of Foundry, but essential for any real RAG application built on Azure.

Prompt Flow

A visual and code-first tool for orchestrating prompts, calls, and logic flows. Think of it as the Foundry equivalent of wiring up an agent's internal workflow.

Evaluation and Observability

Built-in tools for evaluating model quality against test sets, monitoring live production, catching regressions, tracking drift. Absent from Copilot Studio at anything near this depth — this is one of Foundry's biggest advantages.

Content Safety

Programmable filters for input and output — detect prompt injection attempts, block harmful content, enforce topic boundaries. Essential for customer-facing applications.

Foundry earns its keep when at least one of these is true:

1. Scale

Your agent needs to handle thousands or millions of interactions. High volume, low latency, multi-region. Copilot Studio will choke; Foundry is built for it.

2. Knowledge base size

Knowledge bases above roughly 500 documents degrade in Copilot Studio. Foundry IQ + Azure AI Search handles hundreds of thousands.

3. Non-OpenAI models

You need a specific model that isn't in Copilot Studio's built-in lineup. Could be Llama for cost, Phi for edge, a medical/legal specialized model, a Chinese-only model for a specific market, whatever.

4. Fine-tuning or custom model training

Copilot Studio doesn't expose this. Foundry does.

5. Custom logic that breaks low-code

Your agent needs to do something that doesn't fit into the topics/actions/flows model — complex state management, sophisticated orchestration, real machine learning in the middle.

6. Customer-facing product

You're not building an internal tool; you're building something that sits in front of your paying customers. You need the SLAs, the monitoring, the custom UX, the A/B testing — all the things a real product needs.

7. Regulatory / data residency constraints

Your client needs on-prem or sovereign-cloud deployment. Foundry Local is the answer.

8. Multi-agent orchestration

Sophisticated agent-to-agent patterns, hierarchical orchestration, decentralized agent networks. Copilot Studio is adding these features; Foundry has them in deeper form today.

If none of these apply — and for most SMB projects, none do — you don't need Foundry yet. Copilot Studio + maybe some Power Automate integration will do the job at a fraction of the cost and complexity.

Equally important, know when NOT to recommend Foundry:

• Simple internal Q&A bots — overkill
• Sub-500-doc knowledge bases — Copilot Studio is fine
• Limited budget — Foundry builds are real software projects, not $10K weekend gigs
• No in-house engineering capacity — Foundry requires real developers; if the client has none, they're betting on you indefinitely
• "We just want to try AI" — they're not ready for Foundry. Start with Tier 1 and 2.

The temptation for consultants is to always recommend the biggest project. Resist. Foundry where Copilot Studio would have worked = overbuild, unhappy client, bad review. Foundry where Foundry is genuinely warranted = career-making engagement.

Foundry is fundamentally different from Copilot Studio pricing. Not per-seat, not per-message. Consumption-based across multiple Azure services.

The typical line items on an Azure bill for a Foundry-powered app:

Azure OpenAI / model hosting

Per-token costs for input and output. Varies by model — GPT-5 is expensive, Phi is cheap, open-source models are cheapest. This is often the biggest line item.

Azure AI Search

Indexing and query costs. Tiered by scale. Typically $100–$2,000/month for SMB-scale deployments.

Foundry Agent Service

Per-agent runtime costs, plus per-session or per-message charges.

Azure Storage

Where your data sits. Usually trivial for SMB scale.

Compute

If you're running workers, webhooks, APIs — whatever surrounds the AI. VM or App Service or Container Apps costs.

Networking

Egress, VNet, private endpoints. Often underestimated.

Foundry Local (if applicable)

No consumption costs per se — you buy or lease the hardware, pay for the software licensing.

Typical SMB Foundry app: $500–$5,000/month in Azure spend depending on traffic. Lower for internal tools with modest usage; higher for customer-facing apps with real volume.

Foundry projects are real software projects. Budget accordingly:

• Small Foundry integration (Copilot Studio front-end, simple Foundry backend for RAG at scale): $25K–$60K
• Standalone Foundry agent (custom UX, meaningful business logic, real integrations): $60K–$150K
• Production Foundry product (customer-facing, SLAs, full engineering lifecycle): $150K–$500K+

Plus ongoing. Maintenance, API cost changes, model updates, tuning, feature work — budget 20–30% of build cost annually.

For IT partners and consultants working with healthcare, legal, financial services, defense — Foundry is a premium differentiator. Here's the pitch:

> "Microsoft 365 Copilot and Copilot Studio are great for most of the workflow stuff, and we'll deploy those. But for your [sensitive workflow], we're going to use Microsoft Foundry, possibly with Foundry Local for on-prem deployment. That means: > > - You choose the model — open-source, licensed, your own fine-tuned version. > - Data residency is under your control — we can keep everything on-prem or in a sovereign cloud region. > - You get full audit trails through Purview. > - You own the IP — every prompt, every configuration, every integration. > - Microsoft's BAA extends to the entire stack. > > This isn't 'ChatGPT with more words.' This is your own AI platform, inside your own governance perimeter, built on Microsoft-grade infrastructure."

This pitch converts with careful buyers who've been burned by AI hype or are paranoid about data.

When a client actually needs Foundry, here's how a well-scoped engagement lands:

• Discovery and architecture (2–4 weeks): Scope the problem, design the architecture, get buy-in. Produce a real architecture doc.
• Azure foundation setup (2–3 weeks): Subscription structure, resource groups, identity, networking, Purview integration, Foundry workspace.
• Proof of concept (4–6 weeks): Smallest working version. Prove the core technical approach works before committing to full build.
• Pilot build (6–10 weeks): Production-capable version for a pilot user group. Iterate.
• Full build and rollout (4–8 weeks): Scale to all users. Monitor. Tune.
• Steady-state (ongoing): Observability, tuning, new features.

That's a 5–8 month engagement for a real Foundry project. Price it honestly up front — these don't fit in a PO approval window.

---

• Foundry is the pro-code, enterprise-grade AI platform. Formerly Azure AI Studio / Azure AI Foundry.
• 11,000+ models in the catalog, not just OpenAI.
• Foundry Local runs models on your own hardware — critical for regulated industries.
• Foundry IQ + Azure AI Search is the serious RAG stack. Handles knowledge bases far beyond Copilot Studio's limits.
• Use it when: scale, large knowledge base, non-OpenAI models, fine-tuning, custom logic, customer-facing product, data sovereignty, multi-agent orchestration.
• Do NOT use it when: Copilot Studio would have worked.
• Consumption-based pricing. Typical SMB spend: $500–$5K/month in Azure.
• Build costs: $25K–$500K+ depending on scope. Real software project.
• The regulated-industry pitch is Foundry's most differentiated use case for SMB consultants.

---

Next up: Module 5 — How They Fit Together. The architecture patterns that show all three tiers in action.

---

Time: ~15 min Goal: Build the muscle for correctly recommending (or declining) Foundry.

---

For each client situation, decide: Foundry / Not Foundry / Foundry Plus Copilot Studio. One-sentence why.

• A 35-person accounting firm wants an internal Q&A bot on their tax research docs (~150 documents). They have M365 Business Premium.

→ ______________ / ______________

• A regional hospital system wants an AI tool for clinical documentation that processes PHI. Compliance team is very strict; CIO says everything must run on equipment they physically control.

→ ______________ / ______________

• A logistics startup wants a customer-facing chatbot on their website that answers shipping questions, creates quotes, and integrates with their dispatch system. Expected traffic: 50K conversations/month.

→ ______________ / ______________

• A law firm wants an internal research agent that searches 40,000 past case briefs with semantic understanding, cites specific sources, and answers complex multi-part legal questions.

→ ______________ / ______________

• A 12-person real estate brokerage wants a listing description generator trained on their voice.

→ ______________ / ______________

• A manufacturing company wants an agent that takes field technician voice memos, converts them to structured service reports, posts to their FSM system, and flags safety issues for manager review. Daily volume: ~200 reports.

→ ______________ / ______________

• A financial advisory firm wants a client-onboarding assistant that collects KYC/AML info via chat, validates documents, and creates the client record in their advisor platform. Must meet SEC record-keeping requirements.

→ ______________ / ______________

Client from Exercise 1, #3 (logistics startup, customer-facing chatbot, 50K conversations/month).

Estimate:

Build price range: $________ – $________ Timeline: ______ months Typical monthly Azure spend: $________ – $________ Ongoing maintenance (% of build/year): ________%

One-sentence justification for your build price: ______________

One-sentence risk you'd flag to the client: ______________

A 20-person real estate firm says: *"We want to build something custom in Foundry. The generic AI stuff is too basic for us — we need our own AI."*

Write a 45-second response that declines respectfully and proposes a better path.

Your response: ```

```

---

Exercise 1:

• Not Foundry — 150 documents is well within Copilot Studio's sweet spot. Save the money.
• Foundry (specifically Foundry Local) — PHI + on-prem requirement = textbook Foundry Local pitch.
• Foundry Plus Copilot Studio — volume and customer-facing both say Foundry on the back end. Copilot Studio (or a custom web UI) could front it, but many teams just build their own UX. Either way, Foundry is central.
• Foundry — 40K documents blows past Copilot Studio's practical limits. Complex reasoning + citations = Foundry IQ + a good frontier model + Azure AI Search.
• Not Foundry — this is either a Custom GPT, a Claude Project, or a tiny Copilot Studio agent. Foundry is massive overkill.
• Foundry Plus Copilot Studio — Copilot Studio could collect the voice memo via Teams; Foundry does the transcription + structured extraction at scale and integration. Volume (200/day) is borderline — push for Foundry if complexity of the FSM integration warrants it.
• Foundry — regulated, SEC record-keeping, KYC/AML validation, integration with advisor platform, sensitive data. Foundry with proper Purview governance.

Exercise 2 — sample sizing:

Build price range: $80K – $150K

Timeline: 5–7 months

Typical monthly Azure spend: $1,500 – $4,000 (depending on model choice and mix of frontier vs. cheap models)

Maintenance: 25%/year

Justification: "This is a production customer-facing product with real volume. Full engineering lifecycle — architecture, POC, pilot, build, rollout, monitoring. The cost reflects the SLA expectations of a customer-facing system."

Risk to flag: "Azure costs scale with conversation volume. If traffic is 2x what you expect, monthly Azure goes from $3K to $6K. Budget for that upside. Alternatively, we can architect with tiered model usage — cheaper models for routine questions, premium models for complex ones — which caps the cost curve."

Exercise 3 — sample response:

> "I appreciate that you want something that feels custom, and you're right that generic AI doesn't feel like yours. Here's why I'd push back on starting in Foundry: for a 20-person firm, you'd be spending $50K-$100K+ on a custom build for something Copilot Studio and a good configuration could deliver for $15K in about 6 weeks. The 'custom' feeling comes from configuring the agent to your voice, your listings, your process — and that's a configuration job, not an engineering project. Let me show you what a well-tuned Copilot Studio agent can do with your content. If, after 90 days in production, we hit a ceiling that genuinely requires Foundry, we graduate then — the Copilot Studio investment isn't wasted. Sound reasonable?"

Why this works: Validates their aspiration (wanting something custom), exposes the real cost difference, proposes a better starting point, commits to revisiting Foundry if actually warranted. Doesn't dismiss them — respects them.

---

Check yourself: If a client says "we heard Foundry is the future, we want that" — can you talk them into (or out of) it credibly, with pricing and timelines? If yes, ready for Module 5.

Previous: Tier 2: Copilot Studio (Compose) · Next: How They Fit Together: Architecture Patterns