Back to All Scenarios
PASSEDqueue / service_crash
Apache Kafka 3.7 Out of Memory — Killed by OOM Killer on Arch Linux
The Linux OOM killer terminated the Apache Kafka 3.7 process due to memory pressure. The service was the largest memory consumer on the system.
Pattern
MEMORY_EXHAUSTION
Severity
CRITICAL
Confidence
72%
Remediation
Auto-Heal
Test Results
| Metric | Expected | Actual | Result |
|---|---|---|---|
| Pattern Recognition | MEMORY_EXHAUSTION | MEMORY_EXHAUSTION | |
| Severity Assessment | CRITICAL | CRITICAL | |
| Incident Correlation | N/A | None | |
| Cascade Escalation | N/A | No | |
| Remediation | — | Auto-Heal — Corax resolves autonomously |
Scenario Conditions
Arch Linux. Apache Kafka 3.7 consuming 5GB RAM. OOM killer score: 848. System had 21GB total. No swap configured.
Injected Error Messages (1)
Out of memory: oom-killer terminated process 16522 (kafka) total-vm:5242880kB on Arch Linux, Apache Kafka 3.7 port 9092 down, memory critical, no swap available, 125 users/connections connections dropped
Neural Engine Root Cause Analysis
Memory exhaustion detected — the system has run out of available memory, triggering the OOM killer or forcing heavy swap usage. This causes severe performance degradation and can result in random process termination. Memory leaks in long-running applications are a common root cause, especially after deployments or configuration changes.
Remediation Plan
1. Check which process was OOM-killed using 'dmesg | grep -i oom' or journal logs.
2. Review memory usage by process with 'ps aux --sort=-%mem | head -20'.
3. Restart the affected service to reclaim leaked memory.
4. If a memory leak is suspected, enable heap profiling and analyze memory allocation patterns.
5. Consider increasing server memory or setting appropriate memory limits (cgroups/container limits).
Tested: 2026-04-02Monitors: 1 | Incidents: 1Test ID: cmnhnp2d801etlig7d8dne61e