Certificate changes in vSphere environments rarely fail in only one place.
The obvious place to look is the browser warning, the expired certificate alarm, or the service that recently had its Machine SSL certificate replaced. But in production VMware Cloud Foundation and vSphere environments, certificate changes can also break something less visible: the extension and agent framework that lifecycle tooling, NSX service deployments, vSAN File Service components, and partner appliances rely on.
That is where ESX Agent Manager, usually shortened to EAM, becomes important.
Broadcom KB 313402 covers the specific failure pattern where an EAM API call fails with CertificateNotTrustedFault or an EAM agent reports a CertificateNotTrusted issue. The log evidence usually points to an OVF or VIB URL that EAM no longer trusts. In the official example, eam_api.log shows eam.fault.CertificateNotTrustedFault, Suitable trust, not found, certificate_unknown(46), and an inability to construct a valid certificate chain for an OVF URL.
This is not just a certificate hygiene problem.
It is an extension health problem.
When EAM cannot trust the source that hosts an OVF, VIB, or related deployment artifact, the dependent solution cannot reliably deploy, remediate, upgrade, or remove its agent. That can surface through vCenter upgrade pre-check failures, NSX Endpoint Protection or Malware Prevention deployment failures, vSAN File Service health alerts, partner security appliance deployment issues, or stale lifecycle remediation tasks. Broadcom’s EAM API documentation describes EAM as an intermediary between vCenter Server and solutions, responsible for provisioning agent virtual machines and VIB modules and monitoring their state.
The practical takeaway is simple:
Any certificate change that affects vCenter, an artifact depot, SDDC Manager, NSX, or a partner file server should be followed by extension health validation.
Scenario
You replace or renew certificates in one of these areas:
- vCenter Server Machine SSL certificate
- vCenter trusted root chain
- SDDC Manager certificate or trust store
- NSX Manager certificate
- internal HTTPS file server certificate
- partner appliance certificate
- offline depot or repository certificate
- PKI root or intermediate certificate used by infrastructure services
The visible certificate task completes, but afterward one or more extension-driven operations fail.
You may see failures during:
| Area | Example failure |
|---|---|
| EAM API | CertificateNotTrustedFault while creating or updating an agency |
| EAM UI | EAM agent or cluster agent shows CertificateNotTrusted |
| vCenter patching or upgrade | Pre-check fails because EAM URLs are not trusted |
| NSX service deployment | Endpoint Protection or Malware Prevention service deployment fails |
| vSAN File Service | vSAN health or remediation fails because the dependent EAM deployment cannot proceed |
| Partner agents | Security, storage, backup, replication, or IO filter components fail to install, upgrade, or uninstall |
| VCF operations | SDDC Manager cannot trust a vCenter or component certificate after out-of-band certificate changes |
In Broadcom’s documented EAM failure case, the issue can be detected in the vSphere UI under Administration → vCenter Server Extensions → ESX Agent Manager → Monitor, or by reviewing /var/log/vmware/eam/eam.log.
Why This Matters Operationally
EAM sits behind tasks that many teams do not think of as “EAM tasks.”
An administrator may believe they are deploying an NSX service, remediating vSAN File Service, patching vCenter, installing an IO filter, or upgrading a partner appliance. Under the covers, the solution may be asking EAM to deploy an agent VM, install a VIB, validate an OVF URL, or enforce a target state.
That distinction matters because EAM is stateful. It tracks agencies and agents, reports health as red/yellow/green, and raises issues when an agency or agent cannot reach the desired goal state. In the vSphere API model, an agency becomes red when an issue prevents it from reaching its desired state, yellow while actively working, and green when the desired state has been reached.
If certificate trust is broken, the issue is not solved by retrying the lifecycle operation over and over.
You need to repair the trust path.
That is especially important in lifecycle workflows. Broadcom documents vCenter upgrade and patch pre-check failures where the source ESX Agent Manager configuration contains URLs that are not trusted by the system. The pre-check is explicitly validating VIB and OVF URLs configured with EAM early in the vCenter upgrade process as a security hardening measure.
NSX can surface the same pattern. Broadcom documents Endpoint Protection or Malware Prevention service deployment failures on ESX 8.0 U2 where the deployment framework relies on EAM to download OVF files, and EAM refuses to download from HTTPS depots whose certificates are not trusted.
In VCF environments, the blast radius can expand further because vCenter and SDDC Manager trust stores are separate. Broadcom notes that in VMware Cloud Foundation environments, vCenter and SDDC Manager maintain independent trust stores with no synchronization between them.
That means a certificate change can be valid in one management component and still break trust from another.
EAM Trust Flow at a Glance
The important detail is not only whether the certificate is valid in a browser. It is whether the component performing the trust check has the correct trust chain and matching endpoint information.
What to notice in this flow: the failing certificate may not be the vCenter certificate itself. It may be the certificate on the HTTPS server that hosts the OVF, VIB, ZIP, or partner deployment artifact. That is why the URL in the EAM log is so important.
Common Symptoms
Start with the symptom, but do not stop there. The same trust failure can show up through several operational surfaces.
| Symptom | Where to check | What it usually means |
|---|---|---|
CertificateNotTrustedFault |
/var/log/vmware/eam/eam_api.log |
EAM API call failed while validating the certificate chain for a specific URL |
eam.issue.CertificateNotTrusted |
/var/log/vmware/eam/eam.log |
A host or cluster agent has a trust issue tied to an agency |
| EAM UI shows certificate issue | vSphere Client → Administration → vCenter Server Extensions → ESX Agent Manager → Monitor | EAM sees an agency or agent issue |
| vCenter upgrade pre-check fails | Upgrade or patch logs | EAM URLs configured on the source are not trusted |
| NSX service deployment fails | NSX deployment workflow and EAM logs | EAM cannot download the service OVF from a trusted source |
| vSAN File Service remediation fails | vSAN Skyline Health and EAM logs | A dependent EAM deployment cannot proceed |
| SDDC Manager connection fails | SDDC Manager logs | vCenter root certificate changes or trust chain changes have not been reflected in SDDC Manager trust |
Broadcom’s KB 313402 specifically calls out eam_api.log, eam.log, the EAM UI path, and vSAN Skyline Health as places where this issue may appear.
Root Cause Model
Most EAM certificate trust failures fall into one of four buckets.
| Root cause | What changed | Practical interpretation |
|---|---|---|
| Endpoint certificate is bad | Expired, invalid, wrong SAN, wrong hostname, incomplete chain | Fix the HTTPS source certificate first |
| Root or intermediate is missing | Internal CA not trusted by Photon OS or VECS TRUSTED_ROOTS |
Add the appropriate trusted CA to the correct trust store |
| Explicit EAM trust no longer matches | Leaf certificate changed but EAM still has old explicit trust | Update or remove the per-URL EAM trust entry |
| Trust store boundary mismatch | vCenter trusts the chain but SDDC Manager, NSX, or another appliance does not | Import or refresh trust in the component that initiates the connection |
Broadcom lists several direct causes for KB 313402: hostname mismatch, invalid certificate, self-signed certificate, certificate not signed by Photon OS or VECS TRUSTED_ROOTS, or an explicit EAM trust entry that no longer matches the actual file server certificate.
For VCF specifically, Broadcom documents a common out-of-band change pattern: the vCenter root certificate changes, but SDDC Manager is still referencing the old root certificate.
Prerequisites and Safety Checks
Before changing trust settings, collect enough evidence to avoid masking the real issue.
You should have:
| Requirement | Why it matters |
|---|---|
| vCenter administrative access | You need EAM logs, EAM UI, and possibly VCSA shell access |
| Root access to VCSA | Required for eam-utility.py and VECS inspection commands |
| Change window or approved maintenance slot | EAM remediation can affect extension deployment workflows |
| A current backup or snapshot strategy | Trust store changes are small, but the blast radius can be large |
| The exact failing URL | The full OVF/VIB/ZIP URL is needed for per-URL trust operations |
| Agency owner identified | NSX, vSAN, partner tools, or another solution may own the agency |
| PKI chain available | You need the root/intermediate chain if the source certificate is valid but untrusted |
| VCF component ownership understood | SDDC Manager, vCenter, NSX, and other appliances may need separate validation |
Do not use disable-trust as a first reaction. It bypasses certificate verification for a specific URL. Broadcom documents it as an option, but the safer operational path is to correct the certificate, add the trusted root, or configure explicit leaf trust for the affected URL.
Runbook Stage 1: Confirm This Is an EAM Trust Issue
Start by confirming the service and collecting the log evidence.
# Check EAM service state service-control --status vmware-eam # Search recent EAM logs for trust-related failures grep -Ei "CertificateNotTrusted|NotTrusted|certificate_unknown|Unable to construct|ovf|vib|ssl|trust" \ /var/log/vmware/eam/eam.log \ /var/log/vmware/eam/eam_api.log
You are looking for entries that include:
CertificateNotTrustedFaulteam.issue.CertificateNotTrustedcom.vmware.eam.security.trust.NotTrustedcertificate_unknown(46)Unable to construct a valid chain- an HTTPS URL pointing to an OVF, VIB, or ZIP
The URL matters more than the generic error text. It tells you which artifact source EAM is trying to validate.
Runbook Stage 2: Identify the Failing URL and Owner
From the log entry, extract the full URL.
Do not trim it down to only the hostname. For EAM trust operations, the full artifact URL is often required. Broadcom’s upgrade pre-check KB notes that the full URL path, including the VIB, OVF, or ZIP, is required when using the EAM utility path.
Example:
:// : / / .ovf
Then identify the owner:
| URL pattern | Likely owner |
|---|---|
/vsanHealth/fileService/ovf/... |
vSAN File Service |
| NSX service deployment repository | NSX Endpoint Protection / Malware Prevention |
| Partner appliance hostname | Third-party security, storage, backup, or replication tool |
| Internal depot or repository | Platform engineering or lifecycle operations team |
| Legacy host or IP-based URL | Old deployment source, stale agency, or retired partner integration |
This ownership step is important because the agency owner may also configure SSL trust through the EAM API. Broadcom notes that agency-owner SSL trust configuration through the EAM API takes precedence over configuration made with /usr/lib/vmware-eam/bin/eam-utility.py.
Runbook Stage 3: Validate the Certificate Presented by the Artifact Source
Run the test from the VCSA or from a host with equivalent network visibility. Testing from your laptop may not prove what EAM sees.
Set the URL, host, and port:
URL=":// : / / .ovf" HOST=" " PORT=" "
Inspect the certificate:
openssl s_client \
-connect "${HOST}:${PORT}" \
-servername "${HOST}" \
-showcerts /dev/null | \
openssl x509 -noout -subject -issuer -dates -fingerprint -sha256
Then test HTTPS validation:
curl -Iv "${URL}"
Look for:
| Check | Healthy result |
|---|---|
| Subject Alternative Name | Includes the FQDN EAM is using |
| Issuer | Matches expected internal or public CA |
| Not Before / Not After | Certificate is currently valid |
| Chain | Complete chain is served or trusted |
| Hostname | URL hostname matches certificate SAN |
| Trust | The root CA is trusted by the component performing the check |
A common failure is using an IP address in the EAM URL while the certificate only contains an FQDN. Another is replacing the HTTPS server certificate but leaving EAM with explicit trust for the old leaf certificate.
Runbook Stage 4: Inspect vCenter Trust Stores
If the endpoint certificate is valid but not trusted, inspect VECS.
# List VECS stores /usr/lib/vmware-vmafd/bin/vecs-cli store list # Review trusted roots /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOTS --text | \ egrep "Alias|Subject:|Issuer:|Not After"
Broadcom’s certificate review guidance uses vecs-cli to list VECS stores and review entries in TRUSTED_ROOTS, including certificate validity and key usage details.
Do not assume SDDC Manager has the same trust. In VCF, vCenter and SDDC Manager trust stores are independent.
There are four practical remediation paths. Pick the least risky path that corrects the real trust failure.
Option A: Fix the Artifact Source Certificate
Use this when the source certificate is expired, invalid, missing SANs, using the wrong hostname, or serving an incomplete chain.
This is usually the cleanest fix.
The certificate on the HTTPS source should be valid and signed by a CA trusted by Photon OS or VECS TRUSTED_ROOTS. Broadcom lists replacing the file server certificate with a valid certificate signed by Photon OS CAs or VECS TRUSTED_ROOTS as a resolution path for KB 313402.
Use this path when:
- the URL is correct
- the server certificate is objectively wrong
- multiple consumers use the same artifact repository
- you want the fix to survive lifecycle operations and future agent updates
Option B: Add the Root CA to VECS TRUSTED_ROOTS
Use this when the certificate is valid, the hostname matches, and the only issue is that vCenter/EAM does not trust the issuing CA.
Broadcom lists adding the root CA certificate that signs the file server certificate to VECS TRUSTED_ROOTS as another resolution path.
This is appropriate for internal PKI-backed artifact repositories.
Be careful with root CA hygiene. Adding every possible internal CA to every appliance is not a strategy. Add the minimum required root or intermediate chain, document the dependency, and include it in certificate lifecycle tracking.
Option C: Configure Per-URL Leaf Trust with eam-utility.py
Use this when you need EAM to trust a specific OVF or VIB URL and you understand the agency dependency.
/usr/lib/vmware-eam/bin/eam-utility.py install-cert ""
Example:
/usr/lib/vmware-eam/bin/eam-utility.py install-cert \ ":// : / / .ovf"
Broadcom documents install-cert as the EAM utility method to configure a leaf SSL certificate trusted for a specific VIB or OVF URL. The same KB notes that this can be reverted with uninstall-cert.
Rollback:
/usr/lib/vmware-eam/bin/eam-utility.py uninstall-cert ""
Use this path when:
- you cannot immediately replace the source certificate
- the agency depends on a specific external URL
- you need a scoped remediation rather than global CA trust
- the owning solution does not provide its own trust update workflow
Option D: Disable Trust for a Specific URL
This is the last-resort path.
/usr/lib/vmware-eam/bin/eam-utility.py disable-trust ""
Rollback:
/usr/lib/vmware-eam/bin/eam-utility.py enable-trust ""
Broadcom documents disable-trust as a way to disable SSL certificate verification for a specific OVF or VIB URL, and enable-trust as the revert operation.
Use this only with change approval, a time-bound exception, and a plan to replace it with valid certificate trust. In production, disabling certificate verification should not become the permanent fix for a broken repository certificate.
Runbook Stage 6: Handle VCF and SDDC Manager Trust Separately
In VCF environments, do not stop after vCenter is healthy.
If the vCenter root certificate changed out-of-band, SDDC Manager may still reference the old root certificate. Broadcom documents symptoms such as inability to see utilization details, commission hosts, add hosts to clusters, log in to the SDDC UI, or connect to vCenter because of PKIX or certificate path failures.
For VCF, validate:
vCenter trust: - VECS TRUSTED_ROOTS - EAM logs - EAM UI - vCenter upgrade/patch pre-checks SDDC Manager trust: - Common Services logs - Operations Manager logs - SDDC Manager UI health - Trusted certificate store - API trust refresh
Broadcom documents adding custom CA roots to SDDC Manager trust stores, including trust store import, Java trust store import, verification, and service restart steps for older versions. It also notes that VCF 4.1 and later can use the public API for trusted certificates, and VCF 4.5.1 and later can add trusted certificates through the SDDC Manager UI.
For a changed vCenter root certificate, Broadcom documents a manual process that includes importing the root into SDDC Manager trust stores, validating it with keytool, and refreshing trusted certificates through the SDDC Manager API.
The operational point is this:
VCF certificate lifecycle is not one trust store. It is a chain of trust relationships between management components.
Do Not Confuse This with EAM Extension Authentication Failure
KB 313402 is primarily about EAM trust for OVF/VIB URLs.
There is a related but different failure mode after certificate replacement: EAM may fail to authenticate to vCenter as the com.vmware.vim.eam extension. In that case, the logs may show authentication or invalid login errors rather than a URL-specific CertificateNotTrustedFault.
Broadcom documents a separate issue where EAM fails to log in to vCenter as an extension after certificate replacement. The cause can be a mismatch between the vpxd-extension certificate stored in VECS and the certificate information stored in the vCenter Server database for the EAM extension.
The symptoms can be broader than EAM startup. Broadcom notes that this condition can impact NSX for vSphere or vCloud Networking and Security VIB deployment, vLCM service VM deployment, EAM service startup, DRS/vCLS-related functionality, upgrade staging, and remediation workflows.
Use this distinction:
| Log pattern | Likely branch |
|---|---|
CertificateNotTrustedFault with OVF/VIB URL |
KB 313402-style artifact URL trust issue |
Unable to construct a valid chain for artifact URL |
Fix source certificate, VECS trust, or EAM per-URL trust |
Failed to authenticate extension com.vmware.vim.eam |
EAM extension certificate/authentication issue |
InvalidLogin after replacing certificates |
Check EAM extension registration and vpxd-extension certificate state |
Do not blindly run install-cert if the problem is actually EAM extension authentication.
This issue matters more now because lifecycle tooling performs stricter validation.
Broadcom’s upgrade pre-check KB states that SSL trust pre-checks for EAM-configured VIB and OVF URLs are executed early during vCenter upgrade to harden security. The same KB lists failures when the certificate has a hostname mismatch, is invalid, or is not signed by a root CA trusted by Photon OS or VECS TRUSTED_ROOTS.
That means a stale EAM agency, old partner depot, retired file server, or self-signed repository certificate can block a vCenter patch or upgrade even if the original solution is no longer actively used.
There is also a forward-looking vSphere API consideration. Broadcom’s Virtual Infrastructure JSON API lists EAM Agency and Agent API categories as deprecated as of vSphere 9.0 and points to vLCM APIs. That does not make EAM health irrelevant in existing environments. It means operators should understand both the legacy EAM dependency model and the lifecycle direction of newer vSphere releases.
In VCF 9, certificate lifecycle improves with certificate auto-renewal, broader management component support, and VCF Operations alerts. Broadcom’s VCF blog describes certificate auto-renewal in VCF 9 and notes support for management elements, infrastructure appliances, ESX hosts, Microsoft ADCS-backed certificates, Fleet Management CA, VMCA, and SDDC Manager OpenSSL CA scenarios.
But auto-renewal does not eliminate the need to validate EAM, partner depots, offline repositories, or out-of-band trust changes after certificate work.
Post-Certificate-Change Validation Checklist
After any vCenter, SDDC Manager, NSX, or repository certificate change, add these checks to the closeout process.
| Validation | Command or location | Expected result |
|---|---|---|
| EAM service | service-control --status vmware-eam |
Service is running |
| EAM logs | /var/log/vmware/eam/eam.log and eam_api.log |
No new CertificateNotTrusted entries |
| EAM UI | Administration → vCenter Server Extensions → ESX Agent Manager → Monitor | Agencies and agents are healthy |
| Artifact URL | curl -Iv " from VCSA |
Certificate validates or expected trust path is configured |
| VECS trust | vecs-cli entry list --store TRUSTED_ROOTS --text |
Required root/intermediate exists and is valid |
| vCenter upgrade readiness | Upgrade or patch pre-check | No EAM URL trust mismatch |
| NSX service deployment | NSX deployment workflow | No EAM agency creation failure |
| vSAN File Service | vSAN Skyline Health and remediation | No EAM trust-related failure |
| SDDC Manager trust | SDDC Manager UI/API/logs | No PKIX/certificate path errors |
| Partner tools | Partner console and deployment logs | Agent deployment/remediation works |
If a vCenter upgrade or patch was previously blocked, rerun the pre-check only after the trust issue is corrected. Retrying before trust is repaired usually just reproduces the same failure.
Rollback and Fallback Guidance
Rollback depends on the remediation path.
| Change made | Rollback |
|---|---|
| Replaced source certificate | Restore prior certificate only if it was valid and still trusted, or replace with corrected certificate |
| Added CA to VECS | Remove only if you are certain no active dependency uses it |
| Added EAM per-URL leaf trust | eam-utility.py uninstall-cert " |
| Disabled trust for URL | eam-utility.py enable-trust " |
| Added SDDC Manager trust | Remove stale or duplicate alias only after confirming active trust dependencies |
| Updated agency-owner trust through API | Coordinate with the owning solution or partner because API-provided trust can override utility-based trust |
The most important fallback is not technical. It is ownership.
If the agency belongs to NSX, vSAN, a storage vendor, a security vendor, or a backup/replication vendor, confirm whether that product expects to manage EAM trust itself. Broadcom explicitly notes that agency-owner configuration through the EAM API takes precedence over the EAM utility script.
Operational Pattern to Adopt
Treat certificate lifecycle as a management-plane dependency map, not a certificate expiration calendar.
A strong change plan should answer:
| Question | Why it matters |
|---|---|
| Which component certificate is changing? | vCenter, SDDC Manager, NSX, depot, partner appliance, or internal PKI |
| Which components initiate outbound TLS to it? | Trust must exist where the connection starts |
| Which extensions depend on this endpoint? | EAM agencies may reference OVF/VIB/ZIP URLs |
| Which trust stores must be updated? | VECS, Photon OS roots, SDDC Manager, Java trust store, partner appliance |
| Which lifecycle pre-checks must be rerun? | vCenter patching, VCF workflows, NSX service deployment, vSAN health |
| Which rollback is safe? | Trust changes can be reversed, but only with dependency awareness |
This turns certificate management from a reactive break/fix task into an operational control.
Conclusion
EAM certificate trust failures are easy to misread.
The first instinct is often to restart EAM, retry the failed task, or assume vCenter certificate replacement broke the whole platform. Sometimes the issue is that broad. But in many cases, EAM is doing exactly what it should do: refusing to deploy an extension agent from an HTTPS source it cannot trust.
The fix is to identify the failing URL, validate the certificate chain from the right appliance, repair the trust relationship, and then rerun the lifecycle or extension workflow.
For VCF and vSphere operators, the larger lesson is this:
Certificate lifecycle and extension health are the same operational conversation.
When certificates change, validate EAM, lifecycle pre-checks, NSX service deployments, vSAN File Service dependencies, SDDC Manager trust, and partner agents before closing the change. That is how you avoid the delayed failure that only appears days later during a patch window, upgrade pre-check, or production service deployment.
