System Design

Monitoring va Observability

Monitoring — tizim holatini kuzatish va muammolarni aniqlash.

3 Pillars of Observability

1. Metrics (Raqamlar)

CPU: 75%
Memory: 8GB / 16GB
Requests/sec: 1,250
Error rate: 0.5%
Latency p95: 150ms

2. Logs (Xabarlar)

[2024-01-15 10:30:45] INFO: User 123 logged in
[2024-01-15 10:30:46] ERROR: Database connection failed
[2024-01-15 10:30:47] WARN: Cache miss for key:user:456

3. Traces (Request flow)

Request → API Gateway (10ms)
        → Auth Service (5ms)
        → Database (45ms)
Total: 60ms

Key Metrics

RED Method (Requests)

• Rate: Requests/sec
• Errors: Error rate %
• Duration: Latency (p50, p95, p99)

USE Method (Resources)

• Utilization: % busy
• Saturation: Queue depth
• Errors: Error count

Prometheus + Grafana

const client = require('prom-client');

// Counter
const httpRequests = new client.Counter({
  name: 'http_requests_total',
  help: 'Total HTTP requests'
});

// Histogram
const httpDuration = new client.Histogram({
  name: 'http_duration_seconds',
  help: 'HTTP request duration'
});

app.use((req, res, next) => {
  httpRequests.inc();
  const end = httpDuration.startTimer();
  res.on('finish', () => end());
  next();
});

Alerting

groups:
  - name: api_alerts
    rules:
      - alert: HighErrorRate
        expr: rate(http_errors[5m]) > 0.05
        annotations:
          summary: "Error rate > 5%"
      
      - alert: HighLatency
        expr: http_duration_p95 > 1
        annotations:
          summary: "p95 latency > 1s"

Xulosa

3 Pillars:

• Metrics - numbers
• Logs - events
• Traces - request flow

Tools:

• Prometheus - metrics
• Grafana - dashboards
• ELK - logs

Keyingi dars: Security asoslari.