Back to All Scenarios
PASSEDinfrastructure / pdu_circuit_breaker_trip
PDU Circuit Breaker Trip in Server Rack
A power distribution unit circuit breaker trips in the primary server rack, cutting power to 6 servers including the domain controller, file server, and monitoring system. UPS did not engage because the PDU breaker is downstream.
Pattern
UNKNOWN
Severity
CRITICAL
Confidence
95%
Remediation
Remote Hands
Test Results
| Metric | Expected | Actual | Result |
|---|---|---|---|
| Pattern Recognition | UNKNOWN | UNKNOWN | |
| Severity Assessment | CRITICAL | CRITICAL | |
| Incident Correlation | Yes | 35 linked | |
| Cascade Escalation | Yes | Yes | |
| Remediation | — | Remote Hands — Corax contacts on-site support via call, email, or API |
Scenario Conditions
Server rack with single PDU feed. 20A circuit breaker tripped due to overload from new server installation. 6 servers lost power instantly. UPS upstream of PDU (breaker is downstream). No redundant power feed to affected servers.
Injected Error Messages (3)
PDU circuit breaker trip in Rack-A — 20A breaker tripped due to overload (measured load: 21.3A before trip), PDU outlets 1-12 de-energized, 6 servers lost power immediately, PDU remaining outlets (13-24) still powered, rack ambient power sensor showing partial outage, PDU total load: 0W on circuit A (was 4100W), facility power otherwise normal
domain controller DC01 hard power-off — server lost power due to PDU breaker trip in Rack-A, domain controller offline causing authentication failures across network, active directory and DNS services unavailable, all domain-joined workstations unable to process new logins, kerberos ticket granting service unreachable, LDAP queries failing for all applications
file server FS01 hard power-off — server lost power due to Rack-A PDU circuit breaker trip, all SMB/CIFS file shares unavailable, 200 users disconnected from mapped drives, file server hosting 8TB of shared department data, potential filesystem corruption from unclean shutdown, file server array will require consistency check on restart
Neural Engine Root Cause Analysis
The PDU circuit breaker in Rack-A tripped due to electrical overload, with the measured load reaching 21.3A on a 20A breaker before the protective trip occurred. This caused immediate power loss to 6 servers connected to outlets 1-12, while outlets 13-24 remain powered on a separate circuit. The 14 correlated incidents likely represent the cascading failures of services running on the 6 affected servers that lost power.
Remediation Plan
1. Immediately assess which critical services are affected by the 6 powered-down servers. 2. Do NOT reset the breaker until load is reduced - investigate what caused the 21.3A overload (new equipment, failed hardware drawing excess power, or load redistribution). 3. Redistribute server connections to balance load across both PDU circuits if possible. 4. Once load is properly balanced below 20A, reset the tripped breaker. 5. Systematically power on servers while monitoring amperage to ensure load stays within safe limits. 6. Verify all affected services are restored and monitor for stability.
Tested: 2026-03-30Monitors: 3 | Incidents: 3Test ID: cmncka1iy07w9obqe361w3sdp