Back to All Scenarios
PASSEDcloud / gcp_cloud_sql
GCP: Cloud SQL Disk Full — Auto-Storage Increase Limit
A GCP Cloud SQL instance hit its auto-storage increase limit. Database writes failing. Disk cannot grow further.
Pattern
DISK_FULL
Expected: GCP_SQL_FULL
Severity
CRITICAL
Confidence
72%
Remediation
Auto-Heal
Test Results
| Metric | Expected | Actual | Result |
|---|---|---|---|
| Pattern Recognition | GCP_SQL_FULL | DISK_FULL | |
| Severity Assessment | CRITICAL | CRITICAL | |
| Incident Correlation | N/A | None | |
| Cascade Escalation | N/A | No | |
| Remediation | — | Auto-Heal — Corax resolves autonomously |
Scenario Conditions
GCP Cloud SQL PostgreSQL 15. Auto-storage increase enabled but max size hit (1TB). Disk 100% full. All writes failing. Read-only mode.
Injected Error Messages (1)
GCP Cloud SQL disk full — instance 'prod-db' at max storage limit (1TB), auto-storage cannot increase further, all write operations failing, database in read-only mode
Neural Engine Root Cause Analysis
Disk space exhaustion detected — the filesystem has reached capacity, preventing applications from writing data, logs, or temporary files. This can cause databases to crash, services to fail, and logging to stop (masking further issues). Inode exhaustion can also present as 'no space left' even when disk space appears available.
Remediation Plan
1. Identify the fullest filesystems with 'df -h' and check inode usage with 'df -i'.
2. Find and remove large unnecessary files: 'find / -type f -size +100M -exec ls -lh {} \;'.
3. Clear old log files, compress rotated logs, and verify logrotate is configured correctly.
4. Check for runaway log growth from verbose application logging.
5. Expand the volume or add additional storage if the data growth is legitimate.
Improvements Applied
- Pattern classified as DISK_FULL (expected GCP_SQL_FULL)
Tested: 2026-04-02Monitors: 1 | Incidents: 1Test ID: cmnhnr8vc09bslig7d95tdw4v