Common Errors
Error codes, their causes, and how to fix them.
This page documents the most common error responses from the KubeWatch API and how to resolve them.
HTTP error reference
| Error message | HTTP status | Cause | Fix |
|---|---|---|---|
unauthorized | 401 | Missing Authorization header or X-API-Key, expired JWT, or revoked API key | Re-authenticate to get a new JWT, or check your API key in Settings → API Keys |
invalid token | 401 | JWT is malformed, has an invalid signature, or has expired | Re-authenticate. If the error persists, verify JWT_SECRET is consistent across services (self-hosted) |
missing org_id claim | 401 | JWT was issued without an org_id claim (corrupted or from an old version) | Re-authenticate. Contact support if this persists after re-login |
org_suspended | 402 | Your organization's billing is suspended (failed payment) | Update your payment method in Settings → Billing |
rate_limit_exceeded | 429 | Too many API requests per second for your plan | Reduce request rate, add backoff/retry logic, or upgrade your plan |
upstream unavailable | 502 | A backend microservice is down or unreachable | Check the gateway's /health endpoint, or self-hosted users can check docker compose ps |
{"error":"unauthorized"} (from agent, on push) | 401 | The agent's session token is invalid, most often because the agent was deleted/removed in the dashboard, or a stale agent process is still pushing with a token from before it re-registered | Delete the stale agent entry if one exists, then restart the agent so it registers fresh |
{"error":"invalid api key"} (from agent, on registration) | 401 | KUBEWATCH_API_KEY on the agent doesn't match a real key for this org | Regenerate the key in Settings → API Keys and update KUBEWATCH_API_KEY on the agent |
org not found | 404 | The organization associated with your session no longer exists | The organization may have been deleted, contact support |
agent_limit_reached | 402 | You've hit the maximum agent count for your plan | Delete unused agents or upgrade your plan |
Self-hosted specific errors
| Symptom | Cause | Fix |
|---|---|---|
| All services return 502 | Gateway can't reach backend services | Run docker compose ps and check which services are Up vs Exit |
| Login fails with 500 | Auth service can't reach the database | Check docker compose logs auth for DB connection errors and verify DATABASE_URL in .env |
| Metrics not saving | Ingestion service crashed | Check docker compose logs ingestion, often a DB schema migration issue on first startup |
| Dashboard shows blank page | NEXT_PUBLIC_API_URL points to wrong address | Update .env and restart the dashboard service |
| TLS certificate errors | Let's Encrypt cert not issued or expired | Check docker compose logs caddy and verify DNS points to your server |
gateway and auth restart in a loop, whole app unreachable | JWT_SECRET is missing from .env, so the services refuse to start | Set JWT_SECRET and restart (see Services keep restarting below) |
Workloads, Config, RBAC, CRDs, or Approvals pages return 500 or say no data | The local migrations bundle is stale (it's only downloaded once, at install time) | Click Update Now in Settings → System Update, or refresh it manually, see Upgrading |
Services keep restarting: JWT_SECRET is required
After an upgrade or a redeploy, the dashboard is unreachable, login fails, and the backend never comes up. docker compose ps shows gateway and auth continuously restarting (on Kubernetes the gateway and auth pods show CrashLoopBackOff).
Check the logs:
cd ~/kubewatch-erp
docker compose logs auth gateway | tail -20
If you see this line, the sign-in secret is missing:
JWT_SECRET is required but not set. Refusing to start with a default secret.
KubeWatch signs every login session with JWT_SECRET. When it is absent the services refuse to start on purpose, rather than fall back to a shared, guessable secret that would let anyone forge a valid session. This is expected safety behavior, not a bug. Every normal install already sets JWT_SECRET (the installer generates one), so you only hit this if the value went missing, usually a .env that was not restored after rebuilding a host.
Fix on Docker Compose
- Check whether the value is present:
grep '^JWT_SECRET=' ~/kubewatch-erp/.env
-
If you have a backup of
.env, restore the originalJWT_SECRETfrom it. Reusing the original value keeps everyone's existing sessions valid. -
If the original is truly lost, generate a new one:
cd ~/kubewatch-erp
echo "JWT_SECRET=$(openssl rand -hex 32)" >> .env
A new secret is fine, it only signs out anyone currently logged in, who can log back in normally.
- Restart the stack:
docker compose up -d
Fix on Kubernetes (Helm)
Set global.jwtSecret in your values (or the Secret it references) and apply it:
helm upgrade kubewatch ./kubewatch-erp \
--reuse-values \
--set global.jwtSecret="$(openssl rand -hex 32)"
The gateway and auth pods roll and leave CrashLoopBackOff once the value is set.
Agent errors
| Log message | Cause | Fix |
|---|---|---|
failed to connect: dial tcp: connection refused | Gateway URL is wrong or gateway is down | Verify KUBEWATCH_SERVER_URL and check gateway health |
permission denied: /var/run/docker.sock | Agent doesn't have Docker socket access | Mount the socket and ensure correct permissions (see No Metrics Showing) |
push failed: 429 Too Many Requests | Pushing too frequently | Increase KUBEWATCH_INTERVAL (default is 15s, try 30s) |
certificate signed by unknown authority | Self-hosted with self-signed cert | Install a valid TLS certificate on your gateway (self-signed certificates are not supported) |
agent_limit_reached | Organization at agent limit | Delete unused agents from the dashboard or upgrade the plan |
Getting more help
If you can't find the answer here, email [email protected].
When contacting support, include:
- Your agent version (
docker inspect kubewatch-agent | grep Image) - Relevant log output
- The full error message including HTTP status code