Back to All Scenarios
PASSEDnetwork / mtu_mismatch
MTU Mismatch Causing Packet Fragmentation and Drops
After a firewall firmware upgrade, the MTU on the WAN interface drops from 1500 to 1400 without updating the MSS clamp. Jumbo frames from the server VLAN hit the firewall and get silently dropped, causing intermittent failures for large file transfers and database replication.
Pattern
NIC_ERRORS
Severity
CRITICAL
Confidence
85%
Remediation
Remote Hands
Test Results
| Metric | Expected | Actual | Result |
|---|---|---|---|
| Pattern Recognition | NIC_ERRORS | NIC_ERRORS | |
| Severity Assessment | CRITICAL | CRITICAL | |
| Incident Correlation | Yes | 36 linked | |
| Cascade Escalation | Yes | Yes | |
| Remediation | — | Remote Hands — Corax contacts on-site support via call, email, or API |
Scenario Conditions
Firewall WAN MTU: 1400 (post-upgrade). Server VLAN MTU: 9000 (jumbo frames). No MSS clamping configured. DF bit set on most packets. Affects transfers larger than 1400 bytes. Database replication using 8KB blocks.
Injected Error Messages (3)
Frame error rate spiking on WAN interface — MTU mismatch detected, incoming 9000-byte frames being dropped, ICMP 'fragmentation needed' messages being sent but DF bit set, interface error counter incrementing rapidly
Database replication failing intermittently — frame error on large packets crossing WAN link, 8KB replication blocks exceeding MTU 1400, replication lag growing to 45 minutes, partial packet drops causing retransmissions
SFTP transfers failing for files larger than 1KB — frame error due to MTU mismatch at firewall boundary, large file transfers stalling at 0% progress, small files transfer normally, PMTUD not functioning
Neural Engine Root Cause Analysis
This is a Layer 2 network infrastructure issue caused by an MTU (Maximum Transmission Unit) mismatch on the WAN firewall interface. The firewall is receiving 9000-byte jumbo frames but cannot process them due to a smaller configured MTU, resulting in frame drops and rapidly incrementing error counters. The presence of 14 correlated incidents suggests this MTU mismatch is affecting multiple downstream services and connections that depend on this WAN interface. The ICMP 'fragmentation needed' messages with DF (Don't Fragment) bit set indicates that upstream devices are sending oversized packets that cannot be fragmented, creating a communication deadlock.
Remediation Plan
1. Immediately access firewall management interface to verify current MTU settings on WAN interface. 2. Check upstream network device MTU configuration to identify the source of 9000-byte frames. 3. Either increase firewall WAN interface MTU to 9000 bytes or coordinate with network team to reduce upstream MTU to match firewall configuration (typically 1500 bytes). 4. Clear interface error counters after MTU adjustment. 5. Monitor frame error rates and verify ICMP fragmentation messages cease. 6. Test connectivity to affected downstream services once interface stabilizes.
Tested: 2026-03-30Monitors: 3 | Incidents: 3Test ID: cmncjp8t903fqobqebkboklp5