Back to All Scenarios
PASSEDcloud / azure_key_vault_unavailable
Azure Key Vault Unavailable — Secrets Rotation Blocked
Azure Key Vault becomes unreachable due to a misconfigured private endpoint and NSG rule change during a network security audit. All applications that fetch secrets, encryption keys, or certificates from Key Vault at startup or rotation time fail. Services that cache secrets continue working but cannot rotate credentials.
Pattern
AZURE_CLOUD
Severity
CRITICAL
Confidence
95%
Remediation
Auto-Heal
Test Results
| Metric | Expected | Actual | Result |
|---|---|---|---|
| Pattern Recognition | AZURE_CLOUD | AZURE_CLOUD | |
| Severity Assessment | CRITICAL | CRITICAL | |
| Incident Correlation | Yes | 42 linked | |
| Cascade Escalation | Yes | Yes | |
| Remediation | — | Auto-Heal — Corax resolves autonomously |
Scenario Conditions
Azure Key Vault 'prod-kv-eastus2'. Private endpoint enabled. NSG rule accidentally blocking port 443 to Key Vault subnet. 12 Azure App Services and 3 AKS clusters dependent on Key Vault. Secret rotation scheduled for tonight.
Injected Error Messages (3)
Azure Key Vault unreachable — prod-kv-eastus2.vault.azure.net connection refused on private endpoint, NSG 'nsg-keyvault-subnet' rule 'AllowKeyVaultAccess' modified during security audit, port 443 blocked to Key Vault private IP, Azure Monitor alert: Key Vault availability 0%, 47 dependent secrets and 12 encryption keys inaccessible
Azure App Service authentication failing — auth-api unable to retrieve signing keys from Key Vault, DefaultAzureCredential throwing SecretClient error: 'Connection refused to prod-kv-eastus2.vault.azure.net', JWT token signing key rotation blocked, new user logins failing, existing sessions valid until token expiry (1 hour)
AKS pods failing startup — CSI Secrets Store Driver unable to mount secrets volume from Azure Key Vault, pod 'payment-service-7d8f9' stuck in ContainerCreating: 'failed to mount secrets store objects', 6 deployments affected, rolling updates halted, Azure DevOps pipeline deployments failing on secret injection
Neural Engine Root Cause Analysis
The Azure Key Vault is unreachable due to a Network Security Group (NSG) rule modification during a security audit. The NSG 'nsg-keyvault-subnet' rule 'AllowKeyVaultAccess' was modified to block port 443 traffic to the Key Vault's private IP address, breaking connectivity through the private endpoint. This has rendered 47 secrets and 12 encryption keys inaccessible, causing a cascading failure affecting 16 related services that depend on these Key Vault resources.
Remediation Plan
1. Immediately restore the NSG rule 'AllowKeyVaultAccess' in 'nsg-keyvault-subnet' to allow HTTPS (port 443) traffic to the Key Vault private endpoint. 2. Verify the rule priority and source/destination configurations match the original working configuration. 3. Test Key Vault connectivity through the private endpoint. 4. Monitor dependent services for recovery and validate access to secrets and encryption keys. 5. Review and update security audit procedures to prevent accidental blocking of critical infrastructure components.
Tested: 2026-03-30Monitors: 3 | Incidents: 3Test ID: cmncjus0h04w9obqe99g48nqw