TL;DR
VCF 9.1 Private AI security is not one firewall rule, one dashboard, or one product. It is an architecture in which VCF Private AI Services supplies the AI service layer, VCF Networking and NSX control connectivity and segmentation, VMware vDefend provides lateral security and threat prevention, and operations tooling correlates model activity with identity, data access, infrastructure performance, and network behavior.
The supplied image is valuable because it presents these controls as one security fabric. The practical lesson is that model zones, RAG data paths, inference endpoints, Kubernetes workloads, GPU infrastructure, and tenant boundaries must be designed before private AI expands beyond its first proof of concept.
Introduction
The most important question raised by this image is not whether VMware Cloud Foundation can run an LLM.
The better question is whether the platform can control every path that makes the LLM useful.
A production AI request may touch an inference endpoint, model runtime, embedding service, vector database, document repository, GPU worker, external tool, identity provider, logging platform, and policy engine. Keeping the model inside a private data center does not automatically secure those interactions. It only changes who owns the problem.
The phrase VCF NSX 9.1 Private AI should therefore be read as an architectural shorthand. In a current VCF design, the capabilities span VMware Cloud Foundation 9.1, VCF Private AI Services, VCF Networking and NSX, VMware vDefend, vSphere Supervisor, and vSphere Kubernetes Service. These are related components of a private AI operating model, not a single product named NSX Private AI.
The image also includes a 99 percent compliance indicator. That is effective visual storytelling, but it should not be interpreted as evidence of compliance. Compliance must be supported by implemented controls, ownership, testing, retained evidence, exception handling, and continuous review.
The Image Represents a Security Fabric, Not a Feature Checklist
The center of the image is labeled as a Private AI fabric. That is the right mental model.
Private AI security should not be built as a collection of disconnected controls. A firewall team should not design segmentation without understanding the RAG workflow. A data team should not build a vector index without preserving user entitlements. A platform team should not expose GPU-backed inference endpoints before defining identity, tenancy, observability, and incident ownership.
A useful private AI security fabric connects five operational planes:
| Plane | Primary responsibility | Security question |
|---|---|---|
| Experience and inference | User requests, APIs, applications, and agents | Who may invoke which AI capability? |
| Model and runtime | LLMs, SLMs, embedding models, classifiers, and runtime services | Which models are approved, isolated, and reachable? |
| Data and retrieval | Documents, indexes, embeddings, vector databases, and metadata | Which data may be ingested and retrieved for this identity? |
| Infrastructure and acceleration | VMs, VKS clusters, GPU hosts, storage, and networks | Where does the workload run, and how is it contained? |
| Control and operations | Policy, automation, telemetry, governance, and incident response | Can the organization prove and continuously operate the controls? |
Each plane has a different owner and a different failure mode. The architecture becomes defensible only when the planes are joined through shared identity, policy, telemetry, and lifecycle processes.
Private AI Changes the East-West Security Problem
Traditional security designs often concentrate on north-south traffic entering or leaving the data center. Private AI makes east-west communication equally important.
An inference request rarely travels directly from a user to one isolated model. It moves through several internal services, many of which contain sensitive information or privileged access.
The diagram shows why an external perimeter firewall cannot provide the full security boundary. Once the request enters the platform, the architecture must still control which runtime it reaches, whether the runtime can contact the retriever, which retriever can reach the vector database, and which tools or enterprise systems the workflow may call.
That is where distributed enforcement becomes valuable. Controls can be placed closer to the VM, pod, VPC, gateway, model endpoint, and data service instead of forcing every internal flow through a centralized inspection point.
What Each Security Zone Must Accomplish
The image identifies several private AI security domains. Each one represents a design question that must be answered with enforceable controls and evidence.
| Image zone | Design objective | Evidence to retain |
|---|---|---|
| AI workload networking | Permit only the flows required by the AI workflow | Approved flow matrix, realized rules, denied-flow tests |
| Model isolation | Separate models by sensitivity, tenant, environment, and lifecycle state | Model inventory, group membership, cross-zone access tests |
| Private RAG | Protect document ingestion, indexing, retrieval, and vector storage | Source approvals, retrieval logs, access-filter tests |
| Secure inference | Authenticate callers and govern access to model endpoints | Identity logs, authorization policy, rate-limit evidence |
| GPU workload security | Protect accelerator hosts and high-value model workloads | Host placement, management isolation, workload policy |
| Zero Trust AI | Require explicit identity and least-privilege access | Default-deny policy, exception register, periodic review |
| Policy automation | Deploy controls consistently with the workload | Version-controlled policy, pipeline logs, drift reports |
| Threat protection | Detect and contain lateral threats | IDS/IPS events, response workflow, validation results |
| Telemetry and analytics | Correlate model, infrastructure, security, and data behavior | Dashboards, traces, retention policy, correlation IDs |
The important word in the final column is evidence. A diagram may describe the desired architecture, but evidence demonstrates that the architecture exists and continues to operate as intended.
Model Isolation Must Follow Workload Identity
The model isolation section of the image separates LLMs, SLMs, embedding models, and classifiers. That separation should not be treated as a cosmetic grouping exercise.
Different model services often have different users, data access, risk profiles, performance characteristics, and lifecycle requirements. A general-purpose internal model should not automatically share the same trust zone as a finance model that has been tuned on restricted information. An experimental model should not have the same network access as an approved production endpoint.
Static IP-based policies are especially weak for dynamic Kubernetes workloads. Pods may be rescheduled, recreated, or scaled without preserving an address. Security policy should follow workload identity wherever possible, using attributes such as:
- Namespace
- Application label
- Environment
- Model role
- Tenant
- Data classification
- Service account
- Deployment stage
A practical model-zone design might distinguish between approved production models, experimental models, embedding services, regulated workloads, shared utility models, and external model connectors. Each zone should have an explicit owner, allowed communication paths, data classification, and promotion process.
Model isolation also needs an administrative boundary. Preventing one model from reaching another does little good if the same broad administrative role can change every runtime, policy, secret, and data connection without independent review.
Private RAG Is a Data Security System
The private RAG section of the image shows documents being converted into embeddings, stored in a vector database, retrieved, and used to generate an answer.
That workflow is not simply a model feature. It is a data pipeline with multiple security boundaries.
The ingestion identity should be controlled separately from the runtime retrieval identity. The process allowed to read a document repository, create embeddings, and modify an index generally needs more privilege than the inference service that performs read-only retrieval.
The vector database should not become a flattened copy of every document the organization owns. Metadata, tenant boundaries, document permissions, retention, deletion requirements, and source lineage must survive the indexing process.
The critical failure condition is straightforward: the model retrieves information that the requesting user was never entitled to see.
Network segmentation can restrict which services reach the vector database. It cannot independently determine whether a particular employee should retrieve a particular document chunk. That decision must also be enforced in the retrieval and data layers.
A production RAG design should be able to answer:
- Which repositories are approved for ingestion?
- Which identity performed the ingestion?
- Which model or application may use the index?
- Are source access controls represented in retrieval filters?
- Can one tenant retrieve another tenant’s content?
- Can indexed data be deleted when the source is deleted?
- Can an operator trace an answer back to the retrieved source material?
- Are failed and denied retrieval attempts recorded?
Private RAG becomes trustworthy when the data path is as carefully designed as the model path.
Secure Inference Requires a Layered Boundary
The image presents secure inference as a controlled request and response flow. This is another area where network security is necessary but not sufficient.
NSX and vDefend can help control which systems can reach an inference endpoint, which internal services the runtime may contact, and which east-west traffic should be inspected. The application and AI platform layers must still decide whether the caller is authorized to use a particular model.
A mature inference boundary commonly includes:
- User or workload authentication
- Model-level authorization
- Tenant and environment separation
- API quotas and rate limits
- Request-size controls
- Approved model routing
- Prompt and workflow versioning
- Tool-access policy
- Input and output validation
- Sensitive-data handling
- Audit and correlation identifiers
A distributed firewall understands connections, protocols, ports, and workload identity. It does not determine whether an answer is accurate, whether a prompt is attempting semantic manipulation, or whether a model response violates a business policy.
That distinction prevents the networking platform from being positioned as the entire AI security solution. It is a foundational enforcement layer inside a broader AI control model.
GPU Workload Security Must Respect Accelerated Data Paths
GPU infrastructure is expensive, capacity-constrained, and often connected to an organization’s most valuable models and data. That makes GPU-backed workloads a high-value target.
The security architecture should separate at least four concerns:
- GPU host management
- AI workload traffic
- Model and artifact distribution
- Training or inference data movement
Administrative access to accelerator hosts should not share the same unrestricted path as application inference traffic. Model repositories should have controlled publishing and consumption workflows. High-throughput storage or RDMA paths should be reviewed independently rather than assumed to inherit every control applied to ordinary VM traffic.
VCF 9.1 supports multiple accelerator and networking patterns, including direct and enhanced direct-access designs. The operational lesson is not that these paths are inherently insecure. It is that security teams must validate where each path is visible, where policy is enforced, and which telemetry remains available.
Do not assume every packet crosses the same enforcement point simply because every workload runs inside the same VCF environment.
Performance testing must also include the security controls. An architecture that meets latency targets with inspection disabled has not validated the production design. Conversely, inspecting every high-volume flow without classifying its risk can create unnecessary overhead.
The goal is targeted, measurable enforcement, not maximum inspection everywhere.
The Supervisor Network Choice Sets the Security Ceiling
VCF Private AI Services rely on the vSphere Supervisor and associated Kubernetes services. The selected Supervisor networking model influences how far the environment can progress toward self-service, multi-tenancy, and software-defined security.
| Decision area | VDS-based Supervisor networking | VCF Networking with VPC |
|---|---|---|
| Best fit | Initial deployment, constrained environment, or focused proof of concept | Multi-tenant private AI platform with self-service requirements |
| Operational entry point | Familiar VLAN-backed networking | Software-defined VPC and policy model |
| Security model | External and platform controls assembled around the VDS design | VPC, distributed policy, and connectivity controls integrated into the fabric |
| Self-service potential | More limited | Stronger alignment with VCF Automation |
| Design complexity | Lower initial platform change | Requires VPC, NSX, policy, and operating-model readiness |
| Long-term consideration | May require architectural transition as requirements grow | Better foundation for standardized tenant and application patterns |
This should not be treated as a temporary implementation detail.
Broadcom’s VCF 9.1 guidance notes that VDS and VPC networking use fundamentally different fabrics. Moving between them should be planned as a Supervisor redeployment rather than assumed to be a simple configuration switch.
A VDS-first proof of concept may still be the right choice. The architecture team should make that choice with a clear understanding of the future transition, not because the network decision was deferred until installation day.
Policy Automation Turns Design into Repeatable Control
The policy automation area of the image is one of its strongest ideas.
Private AI environments change too quickly for security to depend entirely on manually created rules. Models are promoted, pods are rescheduled, endpoints scale, data sources are added, and new tools are attached to agent workflows. Security policy must be able to follow those changes without becoming a ticket backlog.
The loop matters as much as the initial deployment. Policy must be observed, tested, and refined after workloads begin producing real traffic.
The following example is a conceptual security-intent contract. It is not a native NSX schema. Its purpose is to make the required communication visible before the intent is translated into NSX Policy API resources, Terraform, VCF Automation security profiles, or Antrea policies.
private_ai_security_intent:
workload: finance-rag-production
owner: enterprise-ai-platform
default_action: deny
identities:
inference:
namespace: finance-ai
label: app=inference-runtime
retriever:
namespace: finance-ai
label: app=rag-retriever
vector_database:
security_group: finance-vector-db
allowed_flows:
- name: inference-to-retriever
source: inference
destination: retriever
protocol: tcp
port: 8080
- name: retriever-to-vector-database
source: retriever
destination: vector_database
protocol: tcp
port: 9200
telemetry:
log_denied_flows: true
record_policy_version: true
require_request_correlation_id: true
retention_days: 90
exceptions:
approval_required: true
maximum_duration_days: 14
The values that must change are the workload identities, service ports, owners, retention requirements, and exception process. Successful implementation means the approved paths work, every unapproved path is denied, denied attempts are visible, and the realized policy can be traced back to an approved version.
This pattern also creates a natural handoff between the application, platform, network, and security teams. They can review the same intent instead of maintaining separate interpretations of the architecture.
Telemetry Must Correlate Models, Networks, Data, and Identity
The telemetry panels in the image include model requests, inference events, data access, model performance, GPU utilization, policy insights, and compliance status.
No single telemetry source provides all of that information.
A useful operating model combines at least four telemetry domains:
Infrastructure telemetry
This includes GPU utilization, temperature, memory, CPU, storage, network throughput, Kubernetes health, VM health, and endpoint availability.
Model telemetry
This includes request latency, time to first token, token throughput, cache utilization, error rates, model version, and runtime health.
Security telemetry
This includes permitted and denied flows, firewall-rule matches, IDS/IPS events, malware detections, policy changes, anomalous communication, and administrative activity.
Data and governance telemetry
This includes the user or agent identity, retrieved documents, index version, data source, tool calls, policy version, model approval state, and retention classification.
The most valuable technical decision may be the adoption of an end-to-end request correlation identifier.
Without a shared identifier, an operator may know that the model became slow, the firewall denied a connection, and the vector database recorded an error, but still be unable to prove that those events belonged to the same request.
With correlation, the operational path becomes reconstructable:
Broadcom’s VCF 9.1 materials describe AI metrics for model and GPU activity, with Grafana required for the associated dashboard deployment. That is useful platform visibility, but organizations will still need to correlate those metrics with security, identity, application, and data events.
A polished dashboard is not the same as an actionable telemetry model.
What NSX and vDefend Can and Cannot Do
A credible architecture must preserve the boundary between infrastructure security and AI-specific governance.
What the platform can enforce
NSX and vDefend can provide substantial value by:
- Segmenting AI workloads and data services
- Enforcing least-privilege east-west connectivity
- Applying distributed policy close to VMs and supported Kubernetes workloads
- Controlling VPC and gateway communication
- Producing flow and security telemetry
- Inspecting selected traffic for network threats
- Limiting lateral movement after a workload compromise
- Automating baseline security through reusable profiles and policy as code
- Supporting consistent controls across traditional and cloud-native workloads
What still belongs elsewhere
The surrounding AI platform and governance model must still address:
- Model approval and lineage
- Prompt and workflow governance
- Retrieval authorization at document level
- Model-output quality and factuality
- Prompt-injection and tool-manipulation defenses
- Content safety
- Data-loss prevention
- Model and agent identity
- Human approval for high-risk actions
- Legal and regulatory interpretation
- Evidence required for formal compliance
The most effective design connects these layers. It does not pretend that one layer replaces the others.
A Practical Implementation Sequence
A secure private AI fabric should be built incrementally, with evidence collected at every stage.
Define trust zones and ownership
Begin by identifying the model runtimes, RAG services, vector databases, data sources, inference endpoints, tools, GPU hosts, tenants, environments, and administrative roles.
Assign an owner to every zone. An unowned control eventually becomes an unreviewed control.
Build one approved request path
Select one representative production use case and document its complete flow from user identity to response. Include authentication, inference, retrieval, data access, tool calls, network paths, and telemetry.
This becomes the first golden path for the platform.
Apply minimum required communication
Create workload identities and security groups, then allow only the flows required by the documented path. Use default deny at meaningful boundaries and record every exception.
Validate denied paths as carefully as allowed paths.
Automate policy deployment
Store security intent with the workload blueprint. Add review, testing, change approval, and rollback to the automation pipeline.
The deployment should fail when required identities, owners, or policy references are missing.
Add targeted threat prevention
Inspect the paths where exploit detection and lateral threat prevention provide meaningful risk reduction. Validate throughput and latency with the controls enabled.
Avoid enabling broad inspection without understanding the traffic profile and enforcement location.
Correlate operational telemetry
Connect model, GPU, network, identity, security, and data events through a shared request or session identifier. Define alert ownership and retention before the platform scales.
Scale through approved patterns
Create reusable patterns for common use cases such as internal RAG, shared inference, regulated document search, developer assistants, and tool-enabled agents.
Self-service should allow teams to select approved patterns. It should not allow each team to invent a new security architecture.
Validation Gates Before Production
The environment should not be considered production-ready until each control can be demonstrated.
| Validation gate | Required evidence | Failure response |
|---|---|---|
| Identity | Authorized and unauthorized invocation tests | Correct role, token, or identity mapping |
| Network isolation | Explicit allow and deny flow testing | Remediate group membership or policy scope |
| Model isolation | Cross-model and cross-environment access tests | Separate runtime, identity, or security group |
| RAG authorization | Authorized and unauthorized document retrieval tests | Correct metadata filtering and access propagation |
| Threat prevention | Controlled detection test and event verification | Correct inspection scope, signatures, and response |
| Telemetry | One request traced across all major systems | Add correlation, logging, or integration |
| Policy automation | Rebuild policy from source-controlled intent | Remove manual-only configuration and drift |
| Recovery | Restore or redeploy policy and service configuration | Correct backup, export, or automation process |
| Performance | Latency and throughput tests with security enabled | Tune enforcement scope or redesign the path |
| Governance | Owner, review date, exception, and evidence recorded | Block production approval until complete |
These gates turn the architecture into a repeatable acceptance process. They also give security, infrastructure, AI, and application teams a common definition of ready.
Operational Caveats
The image presents a clean and unified end state. Real implementations will have several areas of friction.
The Supervisor networking choice must be made early enough to support the intended tenancy and automation model. Treating VDS and VPC networking as interchangeable can create avoidable redesign later.
Dynamic grouping depends on accurate labels, namespaces, tags, and inventory. A technically correct policy may still fail if workload identity metadata is incomplete or inconsistently maintained.
Telemetry can become a data-protection problem of its own. Prompts, retrieved content, model responses, document identifiers, and tool parameters may contain sensitive information. Logging everything without classification and retention controls can create a new exposure.
High-throughput AI traffic requires deliberate inspection design. Security controls should be tested with realistic model, storage, and network load instead of validated only with small functional tests.
Finally, private deployment does not equal compliant deployment. Private infrastructure provides control over placement and operation. Compliance depends on how that control is implemented, tested, reviewed, and documented.
Conclusion
The supplied image captures the correct architectural direction: private AI security should operate as a fabric around models, data, inference services, GPU workloads, identity, policy, and telemetry.
VCF 9.1, VCF Networking, NSX, and VMware vDefend can provide a strong infrastructure and lateral-security foundation for that fabric. They can segment AI services, enforce least-privilege communication, protect VM and Kubernetes workloads, automate baseline policies, and generate important security evidence.
They do not remove the need for model governance, document-level authorization, AI gateways, data controls, application security, human approval, and operational ownership.
The practical takeaway is that an AI workload is not private merely because the GPU is on-premises. It becomes meaningfully private when every model, data source, tool, identity, and network path exists inside an explicit and enforceable trust boundary.
That is the real value of the architecture shown in the image. It moves the conversation beyond hosting AI and toward operating AI as a governed enterprise platform.
