Kubernetes Agent

The Kubernetes agent uses the in-cluster Kubernetes API to collect pod, node, service, and event data, so there's no Docker socket needed. It's deployed as a DaemonSet (one pod per node) with a ClusterRole that grants read-only access to cluster resources, so per-node metrics like CPU, memory, network, and disk IOPS are collected directly on each node rather than only from wherever a single pod happens to land. Only one pod at a time, elected via a Lease much like Kubernetes' own leader election, reports cluster-wide data (pods, nodes, deployments, and so on). That way the data is never duplicated across nodes.

Prerequisites

• Helm 3.10+
• kubectl configured with access to your cluster
• A KubeWatch API key (from Settings → API Keys in the dashboard)

Install the agent

The agent chart is distributed as a packaged tarball, so you install it directly from the release URL:

KubeWatch Cloud (SaaS):

helm upgrade --install kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch --create-namespace \
  --set apiKey=YOUR_API_KEY \
  --set agentName=my-production-cluster

Replace YOUR_API_KEY with your actual API key and my-production-cluster with a descriptive name for this cluster, since it will appear in the KubeWatch dashboard agent selector. The chart's default serverURL already points at the hosted gateway, so there's nothing else to configure.

Self-hosted: also set serverURL to your own gateway:

helm upgrade --install kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch --create-namespace \
  --set apiKey=YOUR_API_KEY \
  --set agentName=my-production-cluster \
  --set serverURL=https://your-kubewatch-domain.com

Use http://<your-server-ip> instead of https://... if your self-hosted instance is on a bare IP with no domain/TLS yet.

Verify the agent is running

kubectl get pods -n kubewatch

Expected output (one pod per node, since the agent runs as a DaemonSet):

NAME                    READY   STATUS    RESTARTS   AGE
kubewatch-agent-x8k2p   1/1     Running   0          45s
kubewatch-agent-z4jn9   1/1     Running   0          45s

Check the agent logs to confirm it registered successfully:

kubectl logs -n kubewatch -l app=kubewatch-agent

You should see:

2026/09/08 15:04:05 registered as agent agent_abc123
2026/09/08 15:04:05 starting initial collection (interval=15s)

A successful push isn't logged, only failures are, so no further output for a while is normal. Confirm data is actually arriving in the dashboard's agent selector.

Pod stuck in Pending

Agent state (its registered ID and token) is persisted to a hostPath directory on each node (/var/lib/kubewatch-agent by default, configurable via hostStatePath), not a PersistentVolumeClaim, so there's no StorageClass/provisioner dependency to get stuck on. A pod stuck Pending is almost always a scheduling constraint instead. Check the event:

kubectl describe pod -n kubewatch -l app=kubewatch-agent

Updating the agent

The chart tracks image.tag: latest by default, but a plain helm upgrade won't necessarily restart already-running pods to pick up a new build: Helm only triggers a rollout when the rendered manifest actually changes, and with an unpinned latest tag the manifest is identical release to release even though a newer image has been published to the registry. Two steps get you a real update:

helm upgrade kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch --reuse-values

kubectl rollout restart daemonset/kubewatch-agent -n kubewatch

The first command refreshes the chart itself (RBAC rules, resource limits, any other values changes upstream). The second forces every pod to actually restart and pull the latest image, since pullPolicy: Always only re-pulls on pod (re)creation, not on a no-op upgrade. Skipping the restart is the most common reason a newly-released feature doesn't show up even though you just ran helm upgrade.

Confirm the restart picked up a new image with kubectl get pods -n kubewatch -l app=kubewatch-agent. The AGE column should reset to just now.

To control upgrades more precisely instead of always tracking latest, pin a specific version with --set image.tag=<version> and bump it deliberately. See the releases feed for available tags.

RBAC

The Helm chart creates a ClusterRole that grants read-only access to the following resources by default:

The chart also grants one always-on write capability, independent of every opt-in flag below: horizontalpodautoscalers and karpenter.sh nodepools (get, list, watch, create, update, patch), so the Auto Scaling feature can apply HPA/NodePool objects. See that page's Required Kubernetes RBAC for exactly what it can and cannot do. Notably, it has no verb on pods or deployments directly, and no delete.

It also grants a namespaced Role/RoleBinding (not cluster-wide) for coordination.k8s.io leases (get, list, watch, create, update, patch), scoped to the kubewatch namespace. That's the leader-election Lease every agent pod uses to agree on which one reports cluster-wide data: internal coordination state, not customer data.

Every other write capability is off by default, and the agent never touches any other cluster resource unless you explicitly opt in below. Each opt-in flag below is independent: enabling one does not enable another, and each also requires the matching capability to be available on your KubeWatch plan (most are Enterprise-only, see pricing).

Every write action above also requires admin approval in the dashboard before it's ever applied, on top of the plan and RBAC checks: three independent backstops, not substitutes for one another. Approval isn't restricted to a different admin than the requester: any admin, including the one who submitted the request, can approve it. rbac.allowAutonomousRollback is the one exception: it acts with no human approval at all once promoted, which is why it's never bundled into the umbrella flag below.

Enabling everything at once (rbac.enableReadWriteClusterIDE)

Most installs end up wanting the same bundle: exec, port-forward, secrets metadata, workload edit, pod delete, workload delete, RBAC viewing, Apply Manifest, and Helm release management. Setting each flag individually during agent install is real friction, so a single umbrella flag turns all of those on together:

helm upgrade kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch --reuse-values \
  --set rbac.enableReadWriteClusterIDE=true

Six flags are deliberately not included, even under the umbrella, and always need their own explicit --set:

• rbac.allowHelmManagesRBAC, because it lets Helm-managed releases create their own Roles/RoleBindings, a privilege-escalation-sensitive grant that shouldn't turn on as a side effect of a bundle.
• rbac.allowEditRBAC, for the same reason: it's the other flag that lets something other than you directly create/modify RBAC objects, so it stays a standalone opt-in.
• rbac.allowAutonomousRollback, because it's the one write path in this table with no admin-approval gate at all.
• rbac.allowDeleteNamespaces, because it's the most destructive single action in the platform and should never turn on as a side effect of a broader bundle.
• rbac.allowClusterBackup, because cluster/volume backup is an entirely separate capability from live cluster editing and shouldn't switch on just because you wanted the IDE bundle.
• rbac.allowCordonDrainNodes, for the same reason: node cordon/drain is independent of every capability above and stays its own opt-in.

This flag doesn't remove the per-action admin approval gate for anything it enables. It only saves you from setting individual rbac.* flags at install time; every request an admin has to approve still works exactly the same way afterward.

Checking what's actually granted

Settings → Capabilities in the dashboard (admin-only) shows, live, which of the flags above are actually granted on your cluster right now, self-checked by the agent via a SelfSubjectAccessReview against its own ServiceAccount, not just whichever values file you think you last applied. Each row that isn't granted has a "copy the exact helm command" button so you don't have to hand-type the flag name from this table. The list collapses to the first few rows by default with a "Show all" toggle, since it only gets longer as more flags ship.

Setting up Secrets, RBAC, and CRD instance viewing

These three are the ones people most often get stuck on, because each needs two separate switches flipped before any data shows up, and the dashboard's empty state looks identical either way ("no data" vs "not entitled" vs "not granted" all render as an empty table). Both switches are required, and neither is a substitute for the other:

1. Plan: your organization must be on Enterprise. On self-hosted, this means an activated Enterprise license. See Licensing for details. Check it any time from Settings → Billing in the dashboard.
2. RBAC opt-in: the cluster operator (whoever runs the helm upgrade for the agent) must explicitly set the relevant flag below. This is a separate, deliberate step, not a byproduct of upgrading your plan, because these flags widen what the agent's ClusterRole can read on your cluster.

If either one is off, the page renders as if there's no data. There's no error you'll see in the dashboard, only a debug log line server-side. See Troubleshooting below if you want to confirm which one is missing without trial and error.

Secrets (rbac.allowViewSecrets)

Unlocks the Secrets tab on the Config page: names, types, and key names only, never values.

helm upgrade kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch --reuse-values \
  --set rbac.allowViewSecrets=true

--reuse-values keeps your existing apiKey/serverURL/etc, so you're only adding this one flag. The agent pod restarts as part of the upgrade and picks up the new ClusterRole immediately. No separate agent version change is needed, since this is purely an RBAC grant, not a code change.

RBAC objects (rbac.allowViewRBAC)

Unlocks the RBAC page: Roles, ClusterRoles, RoleBindings, and ClusterRoleBindings across your cluster.

helm upgrade kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch --reuse-values \
  --set rbac.allowViewRBAC=true

Same mechanism as Secrets above: this is a pure RBAC grant, and the running agent picks it up on restart.

CRD instances (rbac.customResourceReadGroups)

CRD definitions (the schema: group, kind, versions) are always visible on the CRDs page, no plan or RBAC opt-in needed, since a schema alone reveals nothing sensitive. Browsing the actual instances of a CRD (the real objects your cluster is running) is different and needs both switches above, plus a third: an explicit, per-apiGroup allow-list, since a blanket "read every CRD" grant would functionally bypass allowViewSecrets's scoping (Kubernetes RBAC can't express "everything except Secrets" for a wildcard resource grant).

helm upgrade kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch --reuse-values \
  --set rbac.customResourceReadGroups[0]=cert-manager.io \
  --set rbac.customResourceReadGroups[1]=karpenter.sh

List every apiGroup whose instances you want browsable. This is the one place where per-CRD granularity matters: setting rbac.allowGenericApply/Enterprise plan does not make every CRD's instances viewable, only the groups you name here. Click View instances on a CRD whose group isn't in this list and you'll always get "no instances collected". That's by design, not a bug, since nothing was ever granted for it. Add its group to the list above and upgrade again to fix that for a specific CRD.

Using a values file instead of --set for a growing list is easier to read:

# kubewatch-agent-values.yaml
rbac:
  allowViewSecrets: true
  allowViewRBAC: true
  customResourceReadGroups:
    - cert-manager.io
    - karpenter.sh

helm upgrade kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch -f kubewatch-agent-values.yaml

Troubleshooting empty pages

If a page stays empty after you've flipped the RBAC flag and confirmed Enterprise on your plan:

1. Confirm the ClusterRole actually has the new rule. A helm upgrade that silently no-oped (wrong release name, wrong namespace, a typo in the flag name) won't error. It'll just reconcile to the same RBAC as before.

   Copy

   kubectl get clusterrole kubewatch-agent -o yaml | grep -A3 secrets
   kubectl get clusterrole kubewatch-agent -o yaml | grep -A3 rolebindings
   

   You should see get, list, watch under the matching resource block. If it's not there, the flag didn't apply. Re-run the helm upgrade and double-check the flag name and release name (helm list -n kubewatch).

2. Confirm your plan is actually Enterprise. Open Settings → Billing in the dashboard. If you just activated a self-hosted license and the plan still shows something other than Enterprise, see Licensing. Applying a license and having your self-hosted database schema up to date are two different things, and a stale schema silently keeps every gate closed regardless of plan. Run Update Now (Settings → System Update) or see Upgrading to refresh it.

3. Check the agent actually restarted after the RBAC change. helm upgrade should recreate the pod automatically. Confirm with:

   Copy

   kubectl get pods -n kubewatch -l app=kubewatch-agent
   

   The AGE column should be recent (since your helm upgrade). If not, force it: kubectl rollout restart daemonset/kubewatch-agent -n kubewatch (the agent runs as a DaemonSet, not a Deployment).

4. For CRD instances specifically, confirm the CRD's apiGroup is spelled exactly right in rbac.customResourceReadGroups (it's the group shown on the CRDs page, not the plural resource name or the kind).

Kafka monitoring

The agent can connect directly to one or more Kafka clusters' admin API to collect topic partition offsets and consumer group lag for the Kafka Topics dashboard. No separate collector to deploy, and the agent never consumes or produces records, only reads metadata.

helm upgrade kubewatch-agent \
  https://raw.githubusercontent.com/lloyd-theophilus/kubewatch-releases/main/kubewatch-agent.tgz \
  --namespace kubewatch --reuse-values \
  --set kafkaClusters="prod=broker1.kafka:9092,broker2.kafka:9092"

Monitor more than one cluster by separating entries with ;, each with its own name:

--set kafkaClusters="prod=broker1:9092,broker2:9092;staging=broker3:9092"

A single cluster's name can be omitted, in which case it's labeled default: --set kafkaClusters="broker1:9092,broker2:9092". If a cluster's brokers are unreachable, it shows up in the dashboard filters but every chart for it stays empty rather than the whole agent push failing.

Helm values reference

Uninstall

helm uninstall kubewatch-agent -n kubewatch
kubectl delete namespace kubewatch