Problem state
Every team in my company started building its own AI agent and they were getting more out of control. Most use cases were similar: code review or doc Q&A, the kind of thing an general agent handles fine. The bigger issue was not the count, it was the state. What is that agent even doing? Which skills got installed on it, by whom? All of that lived in config files inside pods. So I've decided to build a platform to address it.
Components
This platform consist of two components
- Web: User-facing web server. It manages Hermes agents.
- Custom resource operator: Reconcile a
HermesAgent custom resource to manage Hermes agents.
State outside the pod
The general agent we standardized on is Hermes. It is a personal agent: config.yaml plus skills, plugins, and cron jobs installed via commands, all stored in local files. Powerful for personal use, but hard for a team, because state inside a pod is invisible state.
An agent needs config.yaml, secrets, network, storage and etc. Kubernetes already has a mechanism for each, so I decided to declares agent state in a CRD. The operator
- config.yaml comes from a ConfigMap the operator mounts
- skills and plugins are listed by identifier in the spec, and init containers install them before the main container starts
- pip/npm packages pre-install the same way
Ex) HermesAgent custom resource
apiVersion: agents.hermeum.app/v1alpha1
kind: HermesAgent
metadata:
name: code-review-agent
spec:
hermes:
config:
raw:
model:
default: glm-5.3
provider: ollama-cloud
platforms:
webhook:
extra: ...
workspace:
files:
SOUL.md: You are a pragmatic repository maintainer ...
storage:
persistence:
enabled: true
size: 5Gi
Now, the agent is declared in a manifest and the operator keeps it in sync. The state of Hermes is configurable from outside the pod.
Hide the infra
The CRD carries infra config too: storage, networking, security. But most users do not want any of it. Hermeum defines a thin agent spec holding only the core settings (config, skills, plugins, crons) plus an agent type field. A mutating admission webhook patches the HermesAgent resource at creation time, filling in the infra fields that match the type. The agent spec no longer touches infrastructure.
Ex) Agent spec
name: Hermeum GitHub Maintainer
description: Maintains hermeum/hermeum on ...
type: small # <- This is the agent type. The admission server patches the storage and networking fields
soul: You are a pragmatic repository maintainer ...
config:
model:
provider: ollama-cloud
default: glm-5.3
platforms:
webhook:
extra: ...
And the greatest thing of management via manifesto is we can leverage the chatbot to create and manage any Hermes agent we want. The chatbot in the platform write up the manifest so non-dev members are able to create their own agent.
What it left us with
Every agent and its state are visible in one place. We raised agent visibility to tackle the agent sprawl problem.
Questions and criticism welcome. I am specifically curious how others here are handling per-agent config drift.
