I have published two articles praising Hermes Agent this week. This one is different. Both of those articles are accurate — Hermes is genuinely excellent at self-directed skill acquisition, and the compounding effect of the memory system is real. But the self-improvement loop is too powerful to enable on day one. Three weeks ago I ignored my own rule on a fresh VPS deployment. By day 14, eleven auto-generated skills had silently degraded my agent's output quality across three production workflows. I lost an entire evening reverting to a clean state and hand-auditing every skill file. That evening is the reason this article exists.
This is the contrarian Hermes Agent setup guide. The thesis is simple: disable auto_create and auto_refine for the first 14 days, observe how the agent actually behaves on your real workload, hand-author your first generation of skills with intention, build the rollback infrastructure you will need later, and then enable self-improvement deliberately. The compounding still happens. It just compounds toward your actual workflow rather than toward the agent's confused first-week impressions of what you do.
The Default Is a Footgun
Hermes Agent ships with self-improvement enabled. That is the right default for a demo environment, a low-stakes personal assistant, or a proof-of-concept. It is the wrong default for a production workload that touches customer data, deploys code, or handles anything with financial consequences. The default configuration assumes the agent will quickly learn what good looks like on your workload. It will learn something — but in the first week, what it observes is your onboarding confusion, your exploratory prompts, your experiments with workflow patterns you will ultimately discard. It encodes all of that as skills.
There is no single flag that cleanly disables self-improvement in the default Hermes config. You have to set it explicitly. Here is the configuration I call "training wheels on" — the starting point for every new Hermes deployment I run:
# config.yaml — Hermes Agent "training wheels on" baseline
# Use this for the first 14 days on any new workload
agent:
name: "hermes"
version: "2.4"
skills:
# Disable autonomous skill creation entirely
auto_create: false
# Disable autonomous skill refinement (agent cannot improve its own skills)
auto_refine: false
# Memory stays ON — you want observation data, just not autonomous action on it
memory_enabled: true
# External dirs for read-only hand-written skills
# The agent can USE these skills but cannot modify or create new ones
external_dirs:
- path: "/home/hermes/skills/hand-authored"
mode: "read-only"
- path: "/home/hermes/skills/pinned"
mode: "read-only"
# Auto-created skills dir exists but nothing writes to it yet
managed_dir: "/home/hermes/skills/managed"
managed_dir_writable: false
memory:
observation_log: "/home/hermes/logs/observations.jsonl"
session_summaries: true
cross_session_retention: true
logging:
level: "info"
skill_invocations: true
skill_outcomes: true
The key decisions in this config: memory stays on because you want the observation data accumulating, you just do not want the agent acting autonomously on those observations. The external_dirs section gives the agent access to your hand-authored skills through a read-only mount so it can use them immediately, but it cannot modify them. The managed directory exists but managed_dir_writable: false ensures nothing auto-generates there even if the agent tries.
Why "Delete Bad Skills Later" Doesn't Work
The most common objection I hear to this approach is: "just let it run and prune the bad skills afterward." This sounds reasonable. It does not work, for three distinct reasons.
Reason 1: Self-Congratulation Bias
The agent grades its own homework. When Hermes evaluates whether a skill it created is good, it uses the same reasoning model that created the skill in the first place. A skill that encodes a subtly wrong heuristic will generate outputs that look correct to the model that wrote the heuristic. The skill gets a high self-evaluation score. You have no external signal that something is wrong until the degradation shows up in actual output quality — which is a lagging indicator, often by days. By the time you notice the degradation, you have a week of accumulated sessions that all reinforced the bad skill.
Reason 2: The Skill Overwrite Problem
If you do write hand-authored skills and enable auto-refinement simultaneously, the agent can and will degrade your hand-authored skills. The refinement loop identifies "opportunities for improvement" based on observed usage. If the agent observes that your hand-authored skill for "deploy to production" is invoked frequently but not always successfully (because some failures are user errors, not skill errors), it may "improve" the skill by adding conditionals that change the behavior for the common-failure cases. Your carefully tested skill is now different from what you wrote. The diff is subtle. You will not notice until a deployment fails in a new way.
Reason 3: The Accumulation Rate
On a busy workload, Hermes generates approximately two to four skills per day on default settings. By day 14, you have 28 to 56 auto-generated skills. Auditing 50 skills is not a quick afternoon task — each skill requires you to understand what it encodes, test whether that encoding matches your actual intent, and decide whether to keep, modify, or delete it. In practice, most people do not do this audit. The skills accumulate. The agent's behavior drifts from what you intended, in small increments, over weeks. The drift is invisible until it is significant.
The math on why starting clean is easier: zero skills to audit on day 14, versus 28–56. The 14-day observation period also gives you the data to write skills that reflect your real workflow, not your first-week exploration of the tool.
What I Do During the 14 Days
Day 1–3: Capture the Baseline
The first three days are observation only. I run all the workflows I intend to automate through Hermes, treating the agent as a capable but un-optimized assistant. I take notes on what I am actually asking the agent to do, not what I thought I would ask it to do when I planned the deployment. These are always different.
The observation log lives in a markdown file in my project directory, not in Hermes's memory system. I want a human-readable record I can query without routing through the agent:
# hermes-observation-log.md
## Day 1 — 2026-05-01
### Workflows run:
- Deploy storefront to production (3x) — agent handled 2/3 correctly, 1 required manual intervention on Redis healthcheck
- Generate and publish blog post (2x) — worked correctly both times, agent remembered WordPress API format from session 1 to session 2
- Sync WooCommerce products from staging (1x) — FAILED: agent used wrong API version, needed correction
### Patterns noticed:
- Agent repeatedly asks clarifying question about git branch before deploys — this should be a skill default
- Agent does not know our VPS SSH key path — has to be prompted each session
- Agent remembers API format within a session but not across sessions (memory not long enough?)
### Skills that would have helped:
- skill/deploy-production: branch=main, ssh-key=/root/.ssh/deploy_key, healthcheck=redis+nextjs
- skill/wordpress-publish: api-version=v2, base-url=wordpress-app:80, auth=env:WP_APP_PASSWORD
## Day 2 — 2026-05-02
### Workflows run:
...
Three days of this log gives you something valuable: the real list of skills your deployment actually needs, derived from observation rather than upfront planning. The list is always shorter than you expected and more specific than any default skill set would produce.
Day 4–10: Hand-Author the First Generation
With three days of observation data, I write skills manually. Every skill I write follows a fixed template with YAML frontmatter, a procedure section, a pitfalls section, and a verification block. The verification block is non-negotiable — it is what makes skill quality objectively measurable rather than subjective:
---
skill_id: deploy-production
version: 1.0.0
author: anup
created: 2026-05-04
pinned: false
no_auto_refine: false
tags: [deploy, production, vps, docker]
description: ">"
Deploys the storefront to the production VPS via SSH.
Handles Redis healthcheck, Docker Compose restart, and
liveness verification.
---
# skill: deploy-production
## Purpose
Deploy the storefront Next.js application to the production VPS.
Use this skill whenever the user asks to deploy, push to production,
or ship changes live.
## Prerequisites
- SSH key at /root/.ssh/deploy_key must exist
- Environment variable VPS_HOST must be set
- User must confirm they are on the correct git branch before proceeding
## Procedure
1. Confirm git branch:
git branch --show-current
Expected: "main". If not "main", STOP and ask user to confirm.
2. Confirm working tree is clean:
git status --short
Expected: empty output. If not empty, STOP and ask user whether to stash or commit.
3. Push to remote:
git push origin main
4. Wait for GitHub Actions to complete (poll gh run list until conclusion=success):
gh run list --branch main --limit 1 --json conclusion,status
Poll every 30 seconds. Timeout after 10 minutes.
5. Verify liveness after deploy:
curl -s -o /dev/null -w "%{http_code}" https://wowhow.cloud/
Expected: 200. If not 200, trigger rollback procedure.
## Pitfalls
- Do NOT deploy if Redis healthcheck fails before deploy — the deploy will not fix it
- Do NOT interpret a 301 redirect as a success — check for 200 from the final URL
- GitHub Actions concurrency means only one deploy runs at a time — do not push again while a run is active
- If conclusion=cancelled, the next queued push will deploy — wait, do not push again
## Verification
After running this skill, the following must all be true:
- curl https://wowhow.cloud/ returns HTTP 200
- gh run list --limit 1 shows conclusion=success
- No active deploy lock at /tmp/wowhow-deploy.lock on VPS
## Rollback
If verification fails, run: skill/rollback-production
This template takes longer to write than an auto-generated skill. That is the point. The effort of writing it forces you to think through edge cases, failure modes, and verification criteria that auto-generation skips. The resulting skill is dramatically more reliable in production.
During days 4–10, I typically write between five and twelve skills, covering the core workflows identified in the observation log. Each skill goes into /home/hermes/skills/hand-authored/ where it is accessible to the agent but protected from modification by the read-only mount in the config.
Day 11–14: Build Rollback Infrastructure
Before enabling self-improvement, I need to be able to revert to a known-good state without manual effort. The rollback infrastructure has three components: git tracking of the skills directory, a wrapper script that enforces the git commit on every skill write, and a restore command.
Initialize git in the skills directory:
# Initialize git tracking for the skills directory
cd /home/hermes/skills
git init
git add hand-authored/ pinned/
git commit -m "day-14 baseline: hand-authored skills before enabling auto_create"
# Tag the baseline explicitly
git tag baseline-day14
The wrapper script is the key piece. When I enable auto-creation on day 15, I want every auto-generated skill to be committed to git immediately when Hermes writes it. This gives me a complete audit trail and a one-command rollback path. Here is the full wrapper script I use:
#!/usr/bin/env bash
# skill-write-with-audit.sh
# Wrap every skill write operation with a git commit.
# Usage: skill-write-with-audit.sh
#
# Hermes Agent calls this script instead of writing skill files directly.
# Configure in config.yaml: skills.write_hook: "/home/hermes/bin/skill-write-with-audit.sh"
set -euo pipefail
SKILL_PATH="${1}"
CONTENT_FILE="${2}"
SKILLS_ROOT="/home/hermes/skills"
LOG_FILE="/home/hermes/logs/skill-audit.log"
TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
# Validate inputs
if [[ -z "${SKILL_PATH}" || -z "${CONTENT_FILE}" ]]; then
echo "[${TIMESTAMP}] ERROR: Missing arguments. Usage: $0 " | tee -a "${LOG_FILE}"
exit 1
fi
# Resolve absolute path
ABS_SKILL_PATH="${SKILLS_ROOT}/${SKILL_PATH}"
# Ensure the target directory exists
mkdir -p "$(dirname "${ABS_SKILL_PATH}")"
# Check if this is an update to an existing skill
IS_UPDATE=false
if [[ -f "${ABS_SKILL_PATH}" ]]; then
IS_UPDATE=true
# Capture the pre-modification state for the log
PREV_VERSION=$(grep -m1 "^version:" "${ABS_SKILL_PATH}" || echo "unknown")
fi
# Write the skill file
cp "${CONTENT_FILE}" "${ABS_SKILL_PATH}"
# Extract metadata from the written file
SKILL_ID=$(grep -m1 "^skill_id:" "${ABS_SKILL_PATH}" | awk '{print $2}' || echo "unknown")
NEW_VERSION=$(grep -m1 "^version:" "${ABS_SKILL_PATH}" | awk '{print $2}' || echo "unknown")
# Commit to git
cd "${SKILLS_ROOT}"
if [[ "${IS_UPDATE}" == "true" ]]; then
COMMIT_MSG="auto-refine: ${SKILL_ID} ${PREV_VERSION} to ${NEW_VERSION} [${TIMESTAMP}]"
else
COMMIT_MSG="auto-create: ${SKILL_ID} v${NEW_VERSION} [${TIMESTAMP}]"
fi
git add "${ABS_SKILL_PATH}"
git commit -m "${COMMIT_MSG}"
COMMIT_HASH=$(git rev-parse --short HEAD)
# Log the operation
echo "[${TIMESTAMP}] skill_write | id=${SKILL_ID} | path=${SKILL_PATH} | version=${NEW_VERSION} | update=${IS_UPDATE} | commit=${COMMIT_HASH}" | tee -a "${LOG_FILE}"
echo "OK: ${SKILL_PATH} written and committed as ${COMMIT_HASH}"
Make it executable and configure Hermes to use it:
chmod +x /home/hermes/bin/skill-write-with-audit.sh
# Add to config.yaml under the skills section:
# skills:
# write_hook: "/home/hermes/bin/skill-write-with-audit.sh"
With this hook in place, every auto-generated skill is a git commit. Rolling back to the day-14 baseline is a single command:
# Roll back all auto-generated skills to the day-14 baseline
cd /home/hermes/skills
git checkout baseline-day14 -- managed/
# Verify the rollback
git log --oneline -5
git diff baseline-day14 HEAD -- managed/ | head -50
End of Day 14: The Activation Ritual
Before flipping the switch, I run through a fixed checklist. Skipping any item means waiting another 24 hours:
# Hermes Agent Day-14 Activation Checklist
# Complete ALL items before enabling auto_create / auto_refine
- [ ] Observation log has at least 3 days of workflow data
- [ ] At least 5 hand-authored skills written and tested manually
- [ ] Each hand-authored skill has a verification block with specific, testable criteria
- [ ] Skills directory is a git repo with baseline-day14 tag
- [ ] skill-write-with-audit.sh is executable and tested (run it manually once)
- [ ] Rollback command tested: git checkout baseline-day14 -- managed/ runs clean
- [ ] Audit log path is writable: touch /home/hermes/logs/skill-audit.log succeeds
- [ ] Config backup taken: cp config.yaml config.yaml.day14-backup
- [ ] Team notified that self-improvement is being enabled (if applicable)
When all items are checked, I update the config to enable auto-creation with conservative guardrails:
# config.yaml — Day 15+ configuration
# Self-improvement enabled with production guardrails
skills:
auto_create: true
auto_refine: true
# Conservative creation threshold — agent must observe the same
# pattern at least 5 times before creating a skill for it
auto_create_threshold: 5
# Refinement requires a meaningful quality delta, not marginal improvement
auto_refine_min_improvement: 0.15
# Maximum skills auto-created per day — prevents accumulation explosions
auto_create_daily_limit: 2
# Write hook — every skill write goes through the audit wrapper
write_hook: "/home/hermes/bin/skill-write-with-audit.sh"
# Hand-authored skills remain read-only even now
external_dirs:
- path: "/home/hermes/skills/hand-authored"
mode: "read-only"
- path: "/home/hermes/skills/pinned"
mode: "read-only"
managed_dir: "/home/hermes/skills/managed"
managed_dir_writable: true
The auto_create_threshold: 5 and auto_create_daily_limit: 2 are the two guardrails I consider non-negotiable even after enabling self-improvement. The threshold prevents the agent from creating skills based on one-off requests. The daily limit means that even in the worst case, you have at most two new things to audit per day rather than four or more.
Pinning Critical Skills
Some skills must never be deleted, archived, or modified by the self-improvement loop, regardless of how the agent evaluates them. Production deployments, security-related workflows, and the audit wrapper itself all belong in this category. Hermes has a pin mechanism for exactly this purpose:
# Pin a skill so the self-improvement loop never touches it
hermes curator pin skill/deploy-production
hermes curator pin skill/rollback-production
hermes curator pin skill/security-review-checklist
# Verify pinned skills
hermes curator list --pinned
# Output:
# PINNED SKILLS
# -------------
# skill/deploy-production v1.0.0 pinned: 2026-05-15
# skill/rollback-production v1.0.0 pinned: 2026-05-15
# skill/security-review-checklist v1.2.0 pinned: 2026-05-15
Pinned skills also live in /home/hermes/skills/pinned/ which is mounted read-only even in the day-15+ configuration above. The git tracking still applies — if you manually update a pinned skill, the audit wrapper logs the change — but the self-improvement loop cannot touch them.
The YAML frontmatter for a pinned skill looks like this:
---
skill_id: deploy-production
version: 1.0.0
author: anup
created: 2026-05-04
pinned: true
no_auto_refine: true
tags: [deploy, production, pinned]
---
Setting both pinned: true and no_auto_refine: true in the frontmatter creates a belt-and-suspenders guarantee. The pinned flag is enforced by the curator. The no_auto_refine flag is enforced by the self-improvement loop itself. Either one alone is sufficient; both together ensure the skill survives any config change or curator state reset.
Four Workflows That Should Never Be on Auto
Even after day 14, there are categories of workflow where I permanently keep auto-refinement disabled. These are set with the no_auto_refine frontmatter flag, not by the global config, so they stay protected regardless of what the global auto_refine setting is.
1. Production Deployments
Any skill that touches the production deployment pipeline gets no_auto_refine: true and pinned: true. The reasoning is simple: a deployment skill that has been tested and is working correctly should not be modified by an automated process. The cost of a bad deployment is high enough that human review of every change to the deploy skill is worth the overhead.
2. Security Reviews and Audit Checklists
Security review skills encode decisions made with deliberate human judgment about what constitutes an acceptable risk on this specific workload. An automated refinement process does not have the context to make those judgments. More importantly, a subtly weakened security review skill — one that drops a check that seems redundant based on recent session patterns — is exactly the kind of silent degradation that causes real incidents.
3. Customer-Facing Workflows
Any skill that generates content, sends communications, or takes actions visible to customers gets no_auto_refine: true. The agent's self-evaluation of output quality is based on patterns in past sessions. Customer-facing quality has dimensions the agent cannot observe: tone consistency with your brand, regulatory compliance, and the implicit expectations of your specific customer base. Automated refinement optimizes for what the agent can measure, which is not the same thing as customer quality.
4. The Audit Wrapper Itself
The skill-write-with-audit.sh wrapper must never be modified by any automated process. This is the ouroboros risk: if the self-improvement loop can modify the mechanism by which skill modifications are logged, the audit trail becomes unreliable. The wrapper lives outside the skills directory entirely, owned by root, and is never referenced in any Hermes skill file as a dependency.
The config block that enforces these permanent exceptions at the skill level rather than globally:
# These skills have auto-refinement permanently disabled
# regardless of the global auto_refine setting.
# Set in each skill's YAML frontmatter:
# skill/deploy-production
no_auto_refine: true
pinned: true
# skill/rollback-production
no_auto_refine: true
pinned: true
# skill/security-review-checklist
no_auto_refine: true
pinned: true
# skill/customer-email-template
no_auto_refine: true
# skill/wordpress-publish
no_auto_refine: true
The Honest Other Side
It is fair to push back on this approach. If you are running Hermes on a low-stakes workload — personal productivity, research summarization, drafting non-critical content — the 14-day observation period is probably unnecessary friction. The default self-improvement settings will produce a reasonably good agent within a week, and the cost of a bad auto-generated skill is low enough that you can simply delete it when you notice the problem.
The rule is specifically for production workloads, customer-facing systems, and workflows where a subtle degradation in agent behavior costs real money or damages real relationships before you catch it. If your Hermes deployment does not touch any of those categories, you can safely ignore this article and turn on self-improvement on day one.
I also want to be honest about the overhead. The 14-day observation period, the hand-authored skills, the git infrastructure, and the rollback tooling together represent roughly eight to twelve hours of work that a default deployment skips. That is a real cost. It pays back on production workloads because the alternative — debugging a degraded agent while production workflows are impaired — costs more. On a side project, the math probably does not work out in the same direction.
The clearest decision heuristic I have found: if you would run a staging environment and a rollback procedure for code changes to this workload, run them for your agent skills too. If you would not, default settings are probably fine.
Closing
Self-improvement is too powerful to enable on a workload you do not yet understand. The first 14 days of any new Hermes deployment are a learning period for you, not just for the agent. You are learning what the agent actually does on your real tasks, what the friction points are, where the agent excels without any skills at all, and where the gaps are that hand-authored skills can fill. Blocking self-improvement during that period is not about distrust of the agent. It is about having the data you need to make the self-improvement loop produce good outcomes.
When I enable self-improvement on day 15 with the conservative guardrails in place, the compounding still happens. It just compounds toward my real workflow rather than toward my first-week confusion. The difference in output quality after 30 more days is significant. The skills the agent generates are coherent with the hand-authored baseline, targeted at actual patterns rather than exploratory noise, and limited in accumulation rate so the audit burden stays manageable.
The evening I spent reverting eleven bad auto-generated skills from my last fresh deployment convinced me to never skip the observation period again. It is a better use of that time than debugging a degraded agent on a live workload.
If you found this useful, the two Hermes Agent articles I referenced at the start cover the positive case — what self-improvement looks like when it is working correctly, and the architecture of the skill system that makes it possible. This article is the prerequisite, not the replacement.
Originally published at wowhow.cloud
