Greenfield setup sequence
Greenfield setup sequence: AWS Bedrock AgentCore
Eight phases, 28 steps, each with a doneness check. Build bottom-up: identity before data, tools before agents, evals before go-live, the master reference project last because it proves the whole stack. The sequence follows the platform overlay selection above: the neutral and AWS views show the AWS reference implementation; selecting Azure or GCP swaps each step to that platform's services. Source of truth: environment spec, section 5.
The full sequence (click to collapse)
Phase 0, Foundations P
- AWS Organization + account structure; tenant-isolation strategy decided (account-per-client at enterprise tier)
- IAM Identity Center (SSO) + least-privilege role baseline + permission boundaries
- Networking: VPC, private subnets, VPC endpoints, security groups
- KMS per-tenant keys + Secrets Manager with rotation schedules
- Git organization + repo scaffolding from the standard skeleton; ontology mirrored read-only, version-pinned; branch protection with a single gatekeeper on universal files
- CI/CD + infrastructure-as-code bootstrap; validators (referential integrity, schemas) block merge
Phase 1, Identity & tenancy P
- AgentCore Identity wired to the IdP; one service identity per agent; OAuth flows to source systems
- Tenant model implemented; automated cross-tenant read-path test running in CI, red blocks deploys
Phase 2, Model access H
- Bedrock model enablement, model router, version pins, fallback chains, the portability boundary (a model swap is a one-file change)
- Embeddings + reranker endpoints behind the same boundary, nothing above this layer names a vendor
Phase 3, Data plane P
- Per-tenant document stores + ingestion: managed path via Bedrock Knowledge Bases (chunking, embedding, indexing, sync over per-tenant S3) as the default; custom Glue pipeline (OCR → conversion → chunking → embedding → indexing) where taxonomy enforcement or special processing requires it; dead-letter handling on both paths. Manual or periodic sync is POC-only; production sync is change-detection driven
- Vector store (per-tenant): OpenSearch Serverless for low-latency, high-throughput retrieval or S3 Vectors for massive, cost-optimized collections (Bedrock Knowledge Bases can back onto either); plus the graph store; pinned ontology version loaded read-only, client data can never reach the ontology store
Phase 4, Tool layer H
- AgentCore Gateway: one logical MCP server per data source; permissions defined once per server; tool names and descriptions to ACI standard; schemas validated in CI
- Domain rule library packaged as thin compute tools, compute, validate, fetch, file; never decide or draft
- Stand up the North-facing Platform MCP Gateway on AgentCore Runtime: one endpoint template, stamped per tenant. Done when a sanctioned host can list exactly its own tenant's tools and no other.
- Tool-layer eval harness wired: accuracy, call count, error rate, passing baseline before first agent use
Phase 5, Agent runtime H
- Orchestrator on AgentCore Runtime: deterministic state machine, capability-to-agent routing, input validation, session isolation
- AgentCore Memory: short-term session + long-term store; compaction policy (what must survive summarization)
- Code Interpreter sandbox enabled (agents verify computations rather than asserting them)
- AgentCore Policy (Cedar) + hooks engine: the enforcement half of the four guardrail locks, a gate cannot be bypassed by any code path
- Run-contract enforcement: declared end state + token budget on every dispatch; over-budget loops killed and raised as incidents
- Everything registered in AgentCore Registry, generated from the solution repo, never hand-edited
Phase 6, Observability & evaluation H
- AgentCore Observability: OTEL traces to CloudWatch; content-blind configuration verified; dashboards for cost-vs-budget, latency, gate metrics per agent per tenant
- AgentCore Evaluations + golden-exemplar suites and LLM-as-judge pipelines; pre-deploy eval gates on agents and tools; clean-instance A/B tooling
- Per-tenant budget metering and alerts; monthly cost attribution, spend per use case per client is a report, not an estimate
Phase 7, Hardening & go-live P
- Backup / disaster recovery, incident runbooks, on-call rotation
- Security review: pen test, tenant-isolation audit, compliance evidence pack
- Master reference project instantiated; one full use case run through every layer, done when a new builder ships a use case writing only a skill file
Greenfield setup sequence: Azure AI Foundry
Eight phases, 28 steps, each with a doneness check. Build bottom-up: identity before data, tools before agents, evals before go-live, the master reference project last because it proves the whole stack. The sequence follows the platform overlay selection above: the neutral and AWS views show the AWS reference implementation; this Azure view swaps each step to that platform's services. Source of truth: environment spec, section 5.
The full sequence (click to collapse)
Phase 0, Foundations P
- Azure management groups + subscription structure; tenant-isolation strategy decided (subscription-per-client at enterprise tier)
- Microsoft Entra ID (SSO) + least-privilege role baseline + permission boundaries
- Networking: VNet, private subnets, Private Link endpoints, network security groups
- Key Vault per-tenant keys and secrets with rotation schedules
- Git organization + repo scaffolding from the standard skeleton; ontology mirrored read-only, version-pinned; branch protection with a single gatekeeper on universal files
- CI/CD + infrastructure-as-code bootstrap; validators (referential integrity, schemas) block merge
Phase 1, Identity & tenancy P
- Entra workload identities wired to the IdP; one service identity per agent; OAuth flows to source systems
- Tenant model implemented; automated cross-tenant read-path test running in CI, red blocks deploys
Phase 2, Model access H
- AI Foundry model catalog enablement, model router, version pins, fallback chains, the portability boundary (a model swap is a one-file change)
- Embeddings + reranker endpoints behind the same boundary, nothing above this layer names a vendor
Phase 3, Data plane P
- Per-tenant document stores + ingestion: managed path via Azure AI Search integrated vectorization (chunking, embedding, indexing over per-tenant Blob storage) as the default; custom Data Factory pipeline (OCR → conversion → chunking → embedding → indexing) where taxonomy enforcement or special processing requires it; dead-letter handling on both paths. Manual or periodic sync is POC-only; production sync is change-detection driven
- Vector store (per-tenant): Azure AI Search for low-latency, high-throughput retrieval, with blob-backed vector options where scale outweighs latency; plus the graph store; pinned ontology version loaded read-only, client data can never reach the ontology store
Phase 4, Tool layer H
- API Management: one logical MCP tool server per data source; permissions defined once per server; tool names and descriptions to ACI standard; schemas validated in CI
- Domain rule library packaged as thin compute tools, compute, validate, fetch, file; never decide or draft
- Stand up the North-facing Platform MCP Gateway on the Foundry Agent Service runtime: one endpoint template, stamped per tenant. Done when a sanctioned host can list exactly its own tenant's tools and no other.
- Tool-layer eval harness wired: accuracy, call count, error rate, passing baseline before first agent use
Phase 5, Agent runtime H
- Orchestrator on the Foundry Agent Service runtime: deterministic state machine, capability-to-agent routing, input validation, session isolation
- Foundry Agent Service memory: short-term threads + long-term store; compaction policy (what must survive summarization)
- Dynamic-sessions code sandbox enabled (agents verify computations rather than asserting them)
- Policy engine (Azure Policy + custom checks) + hooks engine: the enforcement half of the four guardrail locks, a gate cannot be bypassed by any code path
- Run-contract enforcement: declared end state + token budget on every dispatch; over-budget loops killed and raised as incidents
- Everything registered in the Foundry agent registry, generated from the solution repo, never hand-edited
Phase 6, Observability & evaluation H
- Observability: OTEL traces to Azure Monitor; content-blind configuration verified; dashboards for cost-vs-budget, latency, gate metrics per agent per tenant
- Foundry evaluations + golden-exemplar suites and LLM-as-judge pipelines; pre-deploy eval gates on agents and tools; clean-instance A/B tooling
- Per-tenant budget metering and alerts; monthly cost attribution, spend per use case per client is a report, not an estimate
Phase 7, Hardening & go-live P
- Backup / disaster recovery, incident runbooks, on-call rotation
- Security review: pen test, tenant-isolation audit, compliance evidence pack
- Master reference project instantiated; one full use case run through every layer, done when a new builder ships a use case writing only a skill file
Greenfield setup sequence: Google Vertex AI Agent Engine
Eight phases, 28 steps, each with a doneness check. Build bottom-up: identity before data, tools before agents, evals before go-live, the master reference project last because it proves the whole stack. The sequence follows the platform overlay selection above: the neutral and AWS views show the AWS reference implementation; this GCP view swaps each step to that platform's services. Source of truth: environment spec, section 5.
The full sequence (click to collapse)
Phase 0, Foundations P
- GCP organization + project structure; tenant-isolation strategy decided (project-per-client at enterprise tier)
- Cloud Identity (SSO) + least-privilege role baseline + permission boundaries
- Networking: VPC, private subnets, Private Service Connect, firewall rules
- Cloud KMS per-tenant keys + Secret Manager with rotation schedules
- Git organization + repo scaffolding from the standard skeleton; ontology mirrored read-only, version-pinned; branch protection with a single gatekeeper on universal files
- CI/CD + infrastructure-as-code bootstrap; validators (referential integrity, schemas) block merge
Phase 1, Identity & tenancy P
- Workload Identity Federation wired to the IdP; one service identity per agent; OAuth flows to source systems
- Tenant model implemented; automated cross-tenant read-path test running in CI, red blocks deploys
Phase 2, Model access H
- Vertex AI Model Garden enablement, model router, version pins, fallback chains, the portability boundary (a model swap is a one-file change)
- Embeddings + reranker endpoints behind the same boundary, nothing above this layer names a vendor
Phase 3, Data plane P
- Per-tenant document stores + ingestion: managed path via Vertex AI Search (chunking, embedding, indexing over per-tenant Cloud Storage) as the default; custom Dataflow pipeline (OCR → conversion → chunking → embedding → indexing) where taxonomy enforcement or special processing requires it; dead-letter handling on both paths. Manual or periodic sync is POC-only; production sync is change-detection driven
- Vector store (per-tenant): Vertex AI Vector Search for low-latency, high-throughput retrieval, with BigQuery vector options where scale outweighs latency; plus the graph store; pinned ontology version loaded read-only, client data can never reach the ontology store
Phase 4, Tool layer H
- Apigee: one logical MCP tool server per data source; permissions defined once per server; tool names and descriptions to ACI standard; schemas validated in CI
- Domain rule library packaged as thin compute tools, compute, validate, fetch, file; never decide or draft
- Stand up the North-facing Platform MCP Gateway on the Vertex AI Agent Engine runtime: one endpoint template, stamped per tenant. Done when a sanctioned host can list exactly its own tenant's tools and no other.
- Tool-layer eval harness wired: accuracy, call count, error rate, passing baseline before first agent use
Phase 5, Agent runtime H
- Orchestrator on the Vertex AI Agent Engine runtime: deterministic state machine, capability-to-agent routing, input validation, session isolation
- Agent Engine memory: short-term sessions + long-term memory bank; compaction policy (what must survive summarization)
- Code-execution sandbox enabled (agents verify computations rather than asserting them)
- Policy engine (Organization Policy + custom checks) + hooks engine: the enforcement half of the four guardrail locks, a gate cannot be bypassed by any code path
- Run-contract enforcement: declared end state + token budget on every dispatch; over-budget loops killed and raised as incidents
- Everything registered in the Agent Engine registry, generated from the solution repo, never hand-edited
Phase 6, Observability & evaluation H
- Observability: OTEL traces to Cloud Trace and Cloud Monitoring; content-blind configuration verified; dashboards for cost-vs-budget, latency, gate metrics per agent per tenant
- Vertex AI evaluation service + golden-exemplar suites and LLM-as-judge pipelines; pre-deploy eval gates on agents and tools; clean-instance A/B tooling
- Per-tenant budget metering and alerts; monthly cost attribution, spend per use case per client is a report, not an estimate
Phase 7, Hardening & go-live P
- Backup / disaster recovery, incident runbooks, on-call rotation
- Security review: pen test, tenant-isolation audit, compliance evidence pack
- Master reference project instantiated; one full use case run through every layer, done when a new builder ships a use case writing only a skill file