Why My Nightly Ingest Stalled for 3 Days — and How Splitting One Task Into Sub-Steps Fixed It

I went from ¥100k/month as a university student, to ¥600k/month juggling multiple gigs, to exactly zero when my employer pulled the plug. Six months after that, I rebuilt everything around an autonomous Claude Code setup and I'm at ¥1.2M/month in revenue. This series opens up that setup — the actual code, the actual numbers.

Why this setup works

The first wall I hit after making Claude Code the core of my side business was context decay.

Every time a session ends, everything you built up disappears: the architecture you settled on in the last conversation, the implementation you tried and abandoned, the reasoning behind "we're not going in that direction." You have to explain all of it again in the next session. Use it eight hours a day and hundreds of thousands of tokens of back-and-forth evaporate in a week. No matter how smart the model is, it can't be the foundation of a solo business if the context resets to zero every time.

The idea that solves this at the root is an external brain. Every conversation log with Claude is saved in real time, and every night those logs are automatically ingested into an Obsidian vault (a Markdown-based personal wiki) and structured. Project decisions go to wiki/projects/, technical lessons to wiki/learning/, this week's open questions to wiki/hot.md — knowledge keeps accumulating across sessions, maintained mechanically.

It's an environment, not a task list. That's the core of it.

A typical solo developer wakes up and decides what to build that day. In this setup, the moment I open my MacBook, last night's automated run has already organized yesterday's decisions into this morning's brief. The API design direction I hammered out with Claude the day before is sitting there as Markdown. Lessons from the implementation I delegated to Codex have been appended to the relevant existing articles. A spec contradiction I noticed at midnight surfaces in the next morning's brief as a "connection you're missing."

That feeling — that the environment is already ahead of me — is the key to scaling. Switching between client work and personal projects costs almost nothing, and since the answer to "where did I leave that off?" lives in the wiki, the cost of restarting each morning is close to zero. Underneath the ¥1.2M/month number is this reduction in context cost.

The other thing I want to emphasize is self-healing. Automation always jams. launchd fires while the machine is offline, the API hangs temporarily, macOS permission protection blocks the script. Having a human babysit those failures every night defeats the purpose. Because "if it jams, the next slot retries" is baked into the design from the start, the environment recovers before I even notice a failure. How well that design actually holds up — with concrete failure cases and numbers — is covered in the second half.

The overall flow

There are two core components: ~/.claude/scripts/vault-auto-ingest.sh (the processing itself) and ~/Library/LaunchAgents/com.shun.vault-auto-ingest.plist (the scheduled trigger definition). Those two files together run the nightly log→vault pipeline.

Full pipeline diagram

layer 1: 源泉ログ
──────────────────────────────────────────────────────────
  Claude会話ログ ──┐
  Codex会話ログ  ──┤→ extract_conversations.py (step1)
                   └→ ~/Documents/my-knowledge-base/raw/
                        ├── conversations/       (Claude)
                        └── codex-conversations/ (Codex)

launchd com.shun.vault-auto-ingest
  発火スロット: 4:55 / 8:20 / 10:45 / 12:15
──────────────────────────────────────────────────────────

layer 4: Obsidian Vault
                         ┌── step2a (Claude, timeout 1500s)
vault-auto-ingest.sh ───┤── step2b (Codex,  timeout 1500s)
                         └── step2.5 (brief,  timeout 900s)
                                │
                         wiki/ ─┤─ projects/ career/ learning/ life/
                                ├─ hot.md  (直近コンテキスト)
                                ├─ index.md
                                └─ today-brief.md → Desktop/Daily Brief/

step1 expands the Claude/Codex conversation logs into per-source folders, step2a and step2b ingest each of them into the vault, and step2.5 generates today's brief. Finally a git commit + push backs the vault up as a safety net.

The four launchd trigger slots

Here's the trigger definition in com.shun.vault-auto-ingest.plist.

<key>StartCalendarInterval</key>
<array>
  <dict>
    <key>Hour</key><integer>4</integer>
    <key>Minute</key><integer>55</integer>
  </dict>
  <dict>
    <key>Hour</key><integer>8</integer>
    <key>Minute</key><integer>20</integer>
  </dict>
  <dict>
    <key>Hour</key><integer>10</integer>
    <key>Minute</key><integer>45</integer>
  </dict>
  <dict>
    <key>Hour</key><integer>12</integer>
    <key>Minute</key><integer>15</integer>
  </dict>
</array>

The reason for four slots is protection against sleep freezes. 4:55 is the main slot, when the Mac wakes itself up in the middle of the night on AC power. But on battery, caffeinate -s (prevent system sleep) is only effective while plugged into AC — on battery it does nothing. The result: the script goes into lid-close sleep partway through, freezes, and gets reaped by timeout --kill-after=30. 8:20, 10:45, and 12:15 are catch-up slots that redo the frozen run.

On a day that already succeeded, every slot finishes in one line. The same-day completion marker at the top of the script handles that.

TODAY=$(date +%Y%m%d)
DONE_MARKER="$HOME/.claude/logs/.vault-ingest-done-${TODAY}"

# 本日分が既に成功していれば即終了（catch-upスロットの空振り。ログも汚さない）
[ -f "$DONE_MARKER" ] && exit 0

Locking and caffeinate

This is the implementation for sleep prevention and double-execution prevention.

# スクリプト自身をcaffeinate配下で再起動（実行中スリープ防止）
if [ -z "${CAFFEINATED:-}" ]; then
  exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@"
fi

# mkdirアトミック操作でロック（staleはpid死活で自動回収）
LOCKDIR="$HOME/.claude/locks/vault-auto-ingest.lock"
if ! /bin/mkdir "$LOCKDIR" 2>/dev/null; then
  oldpid=$(cat "$LOCKDIR/pid" 2>/dev/null || true)
  if [ -n "${oldpid:-}" ] && kill -0 "$oldpid" 2>/dev/null; then
    echo "[$(date '+%F %T')] 別インスタンス実行中(pid=${oldpid}) — skip" >> "$LOG"
    exit 0
  fi
  rm -rf "$LOCKDIR"
  /bin/mkdir "$LOCKDIR" 2>/dev/null || exit 0
fi
echo $$ > "$LOCKDIR/pid"
trap 'rm -rf "$LOCKDIR"' EXIT INT TERM

Carrying the CAFFEINATED flag forward as an environment variable prevents infinite recursion of "caffeinate → itself → caffeinate → itself." A mkdir-based lock is the simplest mutual exclusion in shell — mkdir itself is an atomic operation, so no race condition occurs. A stale lock (the process died but the lock remains) is automatically reclaimed after checking whether the pid is alive with kill -0.

step2: the vault ingestion function

The star of the pipeline is the ingest_src() function. It decides "did we already do this today?" based on the presence or absence of a marker file, and only fires claude -p when it hasn't completed.

ingest_src() {  # $1=マーカー $2=ソースdir $3=ソース名 $4=timeout秒 $5=追加指示
  local marker="$1" src="$2" name="$3" to="$4" extra="$5"
  [ -f "$marker" ] && {
    echo "[$(date '+%F %T')] step2($name) は本日実施済み — skip" >> "$LOG"
    return 0
  }
  cd "$VAULT" && run_to "$to" "$CLAUDE" -p \
"$VAULT/CLAUDE.md と $VAULT/wiki/CLAUDE.md のルールに従え。${src} に直近28時間で
追加・更新されたファイル（${name}由来の会話ログ）だけを対象に、このVaultの wiki/ を
更新せよ。手順: 1)新しい知識・進捗・決定・教訓を抽出 2)ドメイン別構造を厳守し
適切な記事に追記・新規作成 3)wiki/index.md と wiki/hot.md も追記更新。${extra}" \
    --dangerously-skip-permissions >> "$LOG" 2>&1 \
    && { touch "$marker"; echo "[...] step2($name) 完了" >> "$LOG"; return 0; } \
    || { echo "[...] WARN: step2($name) 失敗/timeout（次スロットで再試行）" >> "$LOG"; return 1; }
}

The places this function gets called are the substance of step2.

# 2a: Claude Code ログを先に処理（hot.mdの土台を作る）。timeout 1500秒（25分）
ingest_src "$STEP2A_MARKER" "$KB/raw/conversations/" "claude" 1500 ""

# 2b: Codexログを統合。同じく1500秒。2aの結果を踏まえて重複話題は追記でまとめる
ingest_src "$STEP2B_MARKER" "$KB/raw/codex-conversations/" "codex" 1500 \
  "Codex由来でも既存記事に統合し、Claude側と重複する話題は新記事を作らず追記でまとめろ。"

# 両サブが済んだ時だけ「本日ingest完了」マーカーを立てる
# 片方がtimeoutなら次スロットが残りだけ再試行する
[ -f "$STEP2A_MARKER" ] && [ -f "$STEP2B_MARKER" ] && touch "$STEP2_MARKER"

The fact that three markers exist independently is the crux of this design.

STEP2A_MARKER  = ~/.claude/logs/.vault-ingest-step2a-claude-YYYYMMDD
STEP2B_MARKER  = ~/.claude/logs/.vault-ingest-step2b-codex-YYYYMMDD
STEP2_MARKER   = ~/.claude/logs/.vault-ingest-step2-done-YYYYMMDD

step2a completes → the step2a marker is created. step2b completes → the step2b marker is created. Only when both are present is STEP2_MARKER created, and subsequent slots skip step2 entirely. On a day when only one finished, the next slot behaves as "skip the completed sub-step, retry only the unfinished one."

Proving the markers are idempotent

Checking whether this design actually works is simple.

ls ~/.claude/logs/.vault-ingest-step2*

Result (as of 2026-07-16):

~/.claude/logs/.vault-ingest-step2-done-20260711
~/.claude/logs/.vault-ingest-step2-done-20260712
~/.claude/logs/.vault-ingest-step2-done-20260713
~/.claude/logs/.vault-ingest-step2-done-20260714
~/.claude/logs/.vault-ingest-step2a-claude-20260711
~/.claude/logs/.vault-ingest-step2a-claude-20260712
~/.claude/logs/.vault-ingest-step2a-claude-20260713
~/.claude/logs/.vault-ingest-step2a-claude-20260714
~/.claude/logs/.vault-ingest-step2b-codex-20260709
~/.claude/logs/.vault-ingest-step2b-codex-20260711
~/.claude/logs/.vault-ingest-step2b-codex-20260712
~/.claude/logs/.vault-ingest-step2b-codex-20260713
~/.claude/logs/.vault-ingest-step2b-codex-20260714

Markers older than 7 days are removed by the automatic cleanup at the end of the script, so only the last 7 days remain. Three things are visible here.

1. step2a-claude and step2b-codex exist independently as separate files. If one finishes first, it doesn't have to wait for the other.
2. step2-done is only created once both sub-steps have completed. From 07-11 to 07-14, both sub-steps completed four days in a row, and the done-marker is present for each.
3. There's no marker for 07-10. That means both sub-steps failed to complete that day for some reason (details in the second half).

Also, the cleanup find "$HOME/.claude/logs" -maxdepth 1 -name '.vault-ingest-*' -mtime +7 -delete runs at the end of the script every time, so markers never pile up indefinitely. Evidence of completions older than 7 days is deleted automatically.

Implementation details

Designing launchd's PATH to "check first, then fill in"

The shell launchd starts is a separate process from the GUI environment. Neither ~/.zshrc nor ~/.bash_profile gets loaded, so Homebrew's /opt/homebrew/bin and nvm's Node.js aren't in PATH to begin with.

The top of vault-auto-ingest.sh contains code to solve that.

NODE_BIN=$(ls -d "$HOME"/.nvm/versions/node/*/bin 2>/dev/null | sort -V | tail -1)
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$PATH"
[ -n "$NODE_BIN" ] && export PATH="${NODE_BIN}:$PATH"

What matters is that the Node path is resolved dynamically rather than pinned to a version. Taking the tail of sort -V (version sort) means it won't break when I upgrade Node with nvm later. If you hardcode the path as something like v24.13.0, the script dies the moment you move to the next version. In fact, my environment still had several files with hardcoded paths, and I had to hunt them down and fix them every time Node updated. Since switching to this style, that problem is gone.

This PATH backfill is also for git's commit hooks. The vault is managed as a git repository, and a hook runs on git commit. That hook is set up to call a Node-based tool, so if Node isn't in PATH, the commit itself fails. When git commit fails, the log shows an error saying "suspect the commit-msg hook / anything Node-related," which is how you end up burning 30+ minutes investigating the wrong root cause.

The 90-second network wait loop right after wake

Even when the Mac wakes itself from sleep at 4:55, establishing a Wi-Fi connection takes anywhere from a few seconds to a few dozen seconds. claude -p is an API call, so running it while the network is down fails immediately.

net_ok=""
for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18; do
  if /usr/bin/nc -z -G 3 1.1.1.1 443 2>/dev/null; then net_ok=1; break; fi
  sleep 5
done
[ -z "$net_ok" ] && echo "[$(date '+%F %T')] WARN: 網未接続のまま続行（失敗時は次スロットが再試行）" >> "$LOG"

18 iterations × 5 seconds = up to 90 seconds of waiting. It breaks out as soon as it connects, so it normally passes through in 5–15 seconds. If it still isn't connected after 90 seconds, it leaves a warning in the log and continues anyway. If you made it "exit 1 immediately when offline," the script would pointlessly stop from the start, even though the catch-up slots (8:20 / 10:45 / 12:15) could retry in a connected environment.

The check target is 1.1.1.1:443 (Cloudflare DNS) because its downtime risk is nearly zero and a timeout of -G 3 (3 seconds) is set. It also avoids an unnecessary DNS resolution.

A TCC preflight to prevent silent failures

macOS TCC (privacy protection) kills developers quietly. When a process running under launchd tries to write under ~/Documents, it fails silently without returning an error. Nothing is left in the log. No FAILED_FILE is created. You wake up to a state that is neither success nor failure.

The countermeasure is a TCC preflight that always runs before the main body of the script.

if ! ( cd "$VAULT" 2>/dev/null && git rev-parse --git-dir >/dev/null 2>&1 ); then
  echo "[$(date '+%F %T')] ❌ FDA未付与: launchdから '$VAULT' にアクセス不可(TCC保護)。" >> "$LOG"
  echo "    解決: システム設定 > プライバシーとセキュリティ > フルディスクアクセス で /bin/bash を許可。" >> "$LOG"
  notify_fail "FDA未付与: vault にアクセス不可（設定→フルディスクアクセス→/bin/bash）"
  exit 1
fi

git rev-parse --git-dir checks whether the vault can be recognized as a git repository. When TCC blocks it, cd succeeds but git's writes fail, producing a false negative overall. Inserting this before step 1 makes the behavior "on a day when FDA isn't granted, stop immediately and shout about it in the log." Silent failure and loud failure are fundamentally different. With the latter, you know the cause immediately the next morning.

Making failures visible with a FAILED file

When claude -p fails, you won't notice unless you have a terminal open. That's what the FAILED file notification on the Desktop is for.

FAILED_FILE="$HOME/Desktop/Daily Brief/FAILED-${TODAY}.md"
notify_fail() {
  local reason="$1"
  mkdir -p "$HOME/Desktop/Daily Brief" 2>/dev/null
  { echo "# Daily Brief 生成失敗 — $(date '+%F %T')"; echo ""
    echo "- 理由: ${reason}"
    echo "- 詳細ログ: ~/.claude/logs/vault-auto-ingest.log"
    echo "- 自動再試行: 8:20 / 10:45 / 12:15 とログイン時（成功したらこのファイルは自動で消える）"
  } > "$FAILED_FILE" 2>/dev/null
  /usr/bin/osascript -e "display notification \"${reason}\" with title \"Daily Brief 生成失敗\"" >/dev/null 2>&1
}

The key design point is that the file disappears automatically on success. The script deletes FAILED_FILE itself in the archive step on success (step2.6). In other words, "while the file exists = currently failing." If the file is there when you open your Desktop, you know today is still failing. If it isn't, it already succeeded. The check is visually obvious.

Freshness checking for "today's output" with START_STAMP

After step2.5 generates the brief, there's a step that archives it to the Desktop. A freshness check lives there.

START_STAMP=$(mktemp /tmp/vault-ingest-start.XXXXXX)
# ... step2.5 でブリーフ生成 ...
if [ -s "$BRIEF_SRC" ] && [ "$BRIEF_SRC" -nt "$START_STAMP" ]; then
  # アーカイブ処理
  touch "$DONE_MARKER"
else
  notify_fail "ブリーフ生成が未完（claude 失敗/timeout の可能性）— 次スロットで自動再試行"
fi

It creates an empty START_STAMP file at script start, and only accepts the brief as "today's output" when its mtime is newer than that. Without it, a today-brief.md left over from a successful step2.5 yesterday would be mistaken for "today's brief was generated," and DONE_MARKER would get created. The result is an incident where "yesterday's document, which today's claude never touched, gets copied to the Desktop as today's brief." I actually did this once.

Log rotation and debris cleanup

All output from claude -p is appended to vault-auto-ingest.log, so during busy periods it balloons to tens of MB within days. Right now the file is 4,884 KB.

if [ -f "$LOG" ] && [ "$(stat -f%z "$LOG" 2>/dev/null || echo 0)" -gt 5242880 ]; then
  mv "$LOG" "${LOG}.old"
fi

Once it exceeds 5 MB (5,242,880 bytes), it's renamed to .old. The old .old gets overwritten at the next rotation. Two generations are retained.

The other bit of cleanup is removing old FAILED markers.

find "$HOME/Desktop/Daily Brief" -maxdepth 1 -name "FAILED-*.md" \
  ! -name "FAILED-${TODAY}.md" -delete 2>/dev/null

In the early version without this one line, FAILED files with past dates kept piling up on the Desktop after consecutive failures. Since a run on a healthy day only deleted that day's file, I ended up with FAILED-20260611.md, FAILED-20260612.md, and FAILED-20260613.md lined up — three days' worth. As described below, this actually happened.

Where I got stuck

hot.md froze for 3 days — the root cause

The first "wall" after building this environment was a design mistake: cramming too much into a single step2 task.

Back then, step2 was a single task that processed the Claude logs and the Codex logs together, with a timeout of 2400 seconds (40 minutes). The comment is still in the script.

# step2 は活動多発期に28h分のClaude+Codexログ消化＋全記事リライトが40分枠に収まらず
# 連日timeout（hot.md凍結の真因, 2026-06-11〜13）

Here's what was happening. When several days in a row involve heavy use of both Claude Code and Codex, 28 hours' worth of conversation logs adds up to several hundred files. A single prompt saying "read every log from the last 28 hours and update every article in the vault" effectively becomes "read 184 files and rewrite over 100 pages of Markdown." There's no way that fits into 40 minutes.

The 4:55 slot runs the full 40 minutes and gets timeout-killed. The 8:15 slot runs the full duration at the same place and gets timeout-killed. So does 10:15, and so does 12:15. All four slots failed to finish the same processing. As a result, hot.md was never ingested and stayed stale — frozen for three days.

The fix was to split the one task. Separate the Claude logs and Codex logs into separate processes, each with its own independent marker. If the marker exists, skip it as "this source was already processed today." Even if one can't finish within 25 minutes, the next slot can retry just the remainder.

After the split, each timeout is 1500 seconds (25 minutes). Digesting one source's logs finishes in around 20 minutes even on an active day. The hot.md that had been frozen for three days started updating every morning from the day after the fix.

I couldn't write to ~/Documents from launchd

One thing I hit right after starting to assemble this environment was the macOS TCC (privacy protection) wall. This was on 2026-06-01.

I had confirmed with pmset that the Mac auto-wakes at 4:55. The launchd job was starting too. And yet today-brief.md wasn't updated. No git commit to the vault either. Nothing in the log.

Narrowing it down: cd ~/Documents/claude-obsidian works in bash. ls works. But when the claude binary, git, or python is invoked under launchd, writing to ~/Documents fails silently. That's because macOS blocks writes to ~/Documents, ~/Desktop, and ~/Downloads at the TCC layer from binaries that haven't been granted Full Disk Access (FDA). It doesn't even return an error code. It dies quietly with exit 0.

There were two candidate solutions: "add /bin/bash to Full Disk Access in System Settings," or "move the vault itself outside of ~/Documents." Granting FDA means reconfiguring every time the claude binary is updated (because the check is per-binary). Moving the vault out of the protected area only has to be done once, and it doesn't break when claude updates.

When I did move it, the next trap I tried was a symlink into ~/. But iCloud treats symlinks as conflicts and breaks them, so that approach is unusable — confirmed by measurement on 2026-06-01. It's still recorded verbatim in the script header.

# ※ symlinkで ~/ に逃がす案は iCloud が symlink を競合処理して壊すため不可（2026-06-01 検証済み）

Now that the TCC preflight is in place, on a day when FDA isn't granted, it stops and shouts "FDA isn't granted" into the log. At the very least, silent failures are gone.

caffeinate -s goes quiet on battery power

"Processing continues even with the lid closed" was the plan, but freezes on battery power kept happening. The reason is recorded in a comment in the script.

# caffeinate -s はバッテリー駆動だと無効＝蓋閉じスリープで凍結する。凍結したランは
# wake後に timeout が刈り、次スロットがやり直す（step2 は半マーカーでスキップ）。

caffeinate -s is an option that prevents system sleep, but it's only effective while connected to AC power. On battery, closing the lid puts the Mac to sleep. During sleep, claude -p's network connection drops and processing stops. It then lingers as a zombie until timeout --kill-after=30 reaps it.

I pass the -i option (prevent display sleep) as well, but that doesn't affect system sleep. Even combined, they don't prevent lid-close sleep on battery.

I looked for a technical fix that would "never freeze even on battery," but given macOS power management constraints, avoiding it unattended is difficult. The practical countermeasure is designing for "even if it freezes, the catch-up slot handles it." Thanks to the step2a and step2b sub-markers, even when a freeze interrupts a run, the next slot behaves as "skip the completed sub-steps and retry only the remainder." The loss from an incomplete run is limited to the log-processing time for a single source.

Making a habit of plugging in the charger when I get up increases the odds that the 4:55 main slot runs on AC. But if I was working unplugged late at night, the next morning's catch-up slots fill the gap.

2026-06-13 — the day every slot hung completely

This is the most memorable failure. On June 13, all four slots hung fully at the same place and were timeout-killed.

04:55:00 開始 → 05:40:01 step2 timeout
08:15:xx 開始 → 09:40:xx step2 timeout
10:15:xx 開始 → 11:40:xx step2 timeout
12:23:xx 開始 → 13:40:xx step2 timeout

The log pattern was perfectly consistent. claude -p --dangerously-skip-permissions waits the full 2400 seconds and then gets timeout-killed. Four slots, four in a row. The previous day, June 12, was normal. The next day, June 14, was normal too.

The cause was a transient failure on the claude side. Rate limiting, an auth problem, a momentary network drop — it had resolved by the next day, so I couldn't pin it down. I can't promise that automation runs perfectly forever, but I can make it so that it never goes missing silently. On that day a FAILED file was created on the Desktop and a system notification fired.

There was one more cleanup problem. The cleanup code at the time was designed to "delete only that day's FAILED file when that day succeeds," and it never touched FAILED files from earlier dates. When things recovered on the 14th, three files — FAILED-20260611.md, FAILED-20260612.md, and FAILED-20260613.md — were sitting on the Desktop. The only way to clean them up was to delete them manually.

From that experience, I added one line so that a run on a healthy day sweeps up FAILED files from past dates all at once.

find "$HOME/Desktop/Daily Brief" -maxdepth 1 -name "FAILED-*.md" \
  ! -name "FAILED-${TODAY}.md" -delete 2>/dev/null

Since then, even when failures continue for multiple days, the old debris disappears automatically on the morning of recovery. A clean Desktop means "things are fine right now"; a FAILED file means "today is still failing" — that judgment stays visual.

Every failure makes the script one line tougher. Its current form isn't something I "designed up front" — it's a pile of evidence accumulated in the order I actually got stuck. Automation isn't done when you build it; its shape only settles once you keep it running. Next time, I'll look at how well everything so far actually works — with concrete numbers.

Last time I covered four incidents: "hot.md frozen for 3 days," "TCC's silent death," "caffeinate -s ineffective on battery," and "2026-06-13, every slot fully hung." As the script matures, the varieties of jams multiply too. Here are the additional sticking points I couldn't fit in last time.

Sticking points (continued — 9 more)

① iCloud breaks symlinks

When solving the TCC problem, the first thing I tried was "put the vault itself in a non-protected directory and symlink to it from ~/Documents." Measurement on 2026-06-01 confirmed that iCloud treats symlinks as conflicts and breaks them, so I abandoned it immediately. That record is still in the script header verbatim.

# ※ symlinkで ~/ に逃がす案は iCloud が symlink を競合処理して壊すため不可（2026-06-01 検証済み）

Placing a symlink into an iCloud-synced folder needs to be eliminated as an option at the planning stage.

② Hardcoded absolute Node paths all collapse at once when Node updates

My wiki's B+ audit record notes "5 files currently reference the pinned v24.13.0, with breakage risk on Node update." Scripts with a hardcoded version all die right after a Node upgrade. Taking that lesson, the current vault-auto-ingest.sh resolves the version dynamically rather than pinning it.

NODE_BIN=$(ls -d "$HOME"/.nvm/versions/node/*/bin 2>/dev/null | sort -V | tail -1)

Taking the tail of sort -V (version sort) means it automatically follows future Node upgrades.

③ launchd codesigning spawn failure (a Homebrew binary in argv[0])

A plist that put /opt/homebrew/bin/timeout directly as the first element of ProgramArguments died completely silently due to a codesigning spawn failure. launchd verifies the argv[0] binary with adhoc signing. The Homebrew-installed timeout didn't pass, and not even a trace of the job starting was left in the log. Self-repair was falsely declaring this "healthy."

The fix is to make ProgramArguments a wrapper of the form /bin/bash -c "exec /opt/homebrew/bin/timeout ...". /bin/bash is a standard macOS binary, so it passes signature verification. There were 7 plists of the same shape, and I converted them all to the wrapper form.

④ NFD/NFC mangling of Japanese paths prevents launchd from starting

A plist with a Japanese directory written directly into ProgramArguments failed to start with can't open input file. The macOS filesystem holds characters in NFD (decomposed form), but if the string baked into the plist is NFC (composed form), the byte sequences don't match. Even for the same notation "新アカ", if the normalization forms differ between the plist and the filesystem, it can't be opened.

The countermeasure is to look up the real target via glob from a wrapper script at an ASCII path.

TARGET=($HOME/Desktop/SugarLAB_*/run_daily.sh)
[ -f "${TARGET[0]}" ] && exec /bin/bash "${TARGET[0]}"

The plist's ProgramArguments points only at this wrapper. Even if the Japanese directory name changes, the glob absorbs it.

⑤ Letting jq open a file on iCloud directly fails probabilistically with EINTR

If you pass a file in an iCloud-synced folder as an argument like jq '.slug' "$FILE", you probabilistically hit EINTR (Interrupted system call) at open() time. The slug came back empty, and an incident occurred where that day's entire article generation was skipped. The fix is a one-character change.

# 変更前（EINTR で確率的に失敗）
jq '.slug' "$ICLOUD_FILE"

# 変更後（stdin 経由。jq 自身に open() させない）
jq '.slug' < "$ICLOUD_FILE"

This has become an established principle applied across every command that touches files on iCloud.

⑥ The git commit-msg hook as a red herring that looks like the real culprit

The log from the 2026-06-13 all-slot hang (see the previous article) contained an error saying "suspect the commit-msg hook / Node dependency." The actual commit-msg hook is a pure bash conventional-commits regex with no Node dependency. chore(vault): passes without issue. The real cause was that step2, well before reaching the commit, was dying — the hook was completely unrelated. Before taking an error message at face value, actually checking whether the claude binary is alive with claude -p "say OK" saves 30+ minutes of wasted investigation.

⑦ Using the Desktop as an execution directory → iCloud deletes it

I had put an executable script under ~/Desktop/, and iCloud Drive's sync behavior made the file disappear. The Desktop is under iCloud Drive's management, and conflict resolution with other devices deletes or overwrites files. I changed the setup so that executable code lives in ~/dev/ and the Desktop only holds references. That lesson is codified in ~/FILEMAP.md as the "dev = things that run" principle.

⑧ The "total page count" the LLM updates drifts from the real number

When claude -p updates wiki/index.md, the "total page count" is LLM output. It repeatedly disagreed with the actual file count. Now, immediately after running claude -p and before the git commit, I insert a deterministic recount.

real=$(find "$VAULT/wiki" -name '*.md' -not -path '*/.*' | wc -l | tr -d ' ')
sed -i '' -E "s/総ページ数：[0-9]+/総ページ数：${real}/" "$INDEX_FILE"

Overwrite numeric LLM output with deterministic downstream processing — this is a design principle that goes beyond page counts.

⑨ Self-repair destroying production assets outside its scope

On 2026-06-22, in affiliate-factory, .env vanished, post-to-hatena.sh was corrupted, a launchd plist's XML was corrupted, and the Desktop working directory disappeared — all at the same time. Strong suspicion falls on the self-repair watchdog overwriting and deleting files due to crossed output. The scope restriction that held in the canary test environment wasn't in effect for production jobs. Files that are gitignored and unrecoverable, like .env, must be explicitly excluded from self-repair's reach. The wiki records the resulting policy: "keep secrets and 'delete-it-and-it-breaks' working directories outside self-repair's reach. Non-idempotent repair deletes your most important assets."

Best practices

Implementation guidelines derived from real failures, organized with code and numbers.

1. One task, one marker — cut timeout boundaries finely

This is the biggest lesson. A single task of "digest 28 hours of Claude logs + Codex logs" didn't fit in the 2400-second (40-minute) window, and froze hot.md for three days. The traces are still in the script's comments.

# step2 は活動多発期に28h分のClaude+Codexログ消化＋全記事リライトが40分枠に収まらず
# 連日timeout（hot.md凍結の真因, 2026-06-11〜13）。

After the split, each sub-step gets 1500 seconds (25 minutes). Even on active days it finishes in around 20 minutes. Cutting things to a granularity where a single claude -p can run to completion is the precondition for a self-recovering design.

2. caffeinate self-invoke + a CAFFEINATED env var to prevent infinite recursion

if [ -z "${CAFFEINATED:-}" ]; then
  exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@"
fi

Because exec replaces the process, you don't get infinite recursion of "caffeinate → bash → caffeinate → …". Carrying CAFFEINATED forward to the child process as an environment variable is the key to preventing double startup.

3. mkdir atomic locking + kill -0 for automatic stale reclamation

mkdir itself is an atomic operation, so no race condition occurs. A stale lock (the process died but the lock remains) is automatically reclaimed after checking whether the process is alive with kill -0 $pid. Since a real race between the launchd run and a manual run occurred on 2026-06-10, I apply this to every job.

4. Use a TCC preflight to create "loud failure"

macOS TCC makes writes to ~/Documents from under launchd fail silently. It dies quietly with exit 0, leaving nothing in the log. Inserting git rev-parse --git-dir before processing starts means that on a day when FDA isn't granted, it stops right at the start and shouts about it in the log. Silent failure and loud failure are fundamentally different. With the latter, you can pinpoint the cause immediately the next morning.

5. FAILED file "exists = currently failing" design — auto-delete on success

notify_fail() {
  { echo "# Daily Brief 生成失敗 — $(date '+%F %T')"
    echo "- 自動再試行: 8:20 / 10:45 / 12:15（成功したらこのファイルは自動で消える）"
  } > "$FAILED_FILE"
}

The script deletes the FAILED file itself on success. While the file exists on the Desktop, it's failing; if it isn't there, it already succeeded. On top of that, FAILED files from past dates are also swept automatically on healthy days.

find "$HOME/Desktop/Daily Brief" -maxdepth 1 -name "FAILED-*.md" \
  ! -name "FAILED-${TODAY}.md" -delete 2>/dev/null

6. START_STAMP freshness gate — don't mistake yesterday's output for today's

Create an empty file with mktemp at script start, and only accept the brief as "today's output" when its mtime is newer than that.

START_STAMP=$(mktemp /tmp/vault-ingest-start.XXXXXX)
# ... claude -p 実行 ...
[ -s "$BRIEF_SRC" ] && [ "$BRIEF_SRC" -nt "$START_STAMP" ] && touch "$DONE_MARKER"

In the early version without this gate, I once had an incident where a today-brief.md generated the previous day was misjudged as "today's brief was generated" and DONE_MARKER was created.

7. Resolve the Node path dynamically, version-independently

sort -V | tail -1 automatically selects the latest installed version. A pinned path like v24.13.0 breaks every script the moment you upgrade. I actually had pinned paths left in 5 files, requiring manual fixes on every Node update.

8. The 90-second network wait loop should continue, not "exit immediately"

Even if it continues at 4:55 while offline and fails, the 8:20 catch-up slot retries. If you make it "exit 1 immediately if it can't connect," it stops from the start even though it could retry in a connected environment. The cost of failure is only "this slot's processing is skipped," and the next slot fills the gap.

9. Auto-clean markers after 7 days

find "$HOME/.claude/logs" -maxdepth 1 -name '.vault-ingest-*' -mtime +7 -delete 2>/dev/null

Without this one line, markers accumulate indefinitely. Only the last 7 days remain, and evidence of older completions disappears automatically. ls ~/.claude/logs/.vault-ingest-step2* shows you today's state instantly.

10. Don't write Japanese paths directly into launchd's ProgramArguments

NFD/NFC normalization mismatches cause can't open input file. The workaround is to look up the real target via glob from a wrapper script at an ASCII path.

TARGET=($HOME/Desktop/SugarLAB_*/run_daily.sh)
[ -f "${TARGET[0]}" ] && exec /bin/bash "${TARGET[0]}"

11. Limit self-repair's scope to inside ~/.claude/

If the reason for repair is out of scope, declare UNFIXABLE: <reason> and exit. Rewriting out-of-scope files on a guess produces simultaneous .env loss and launchd plist corruption, like the affiliate-factory incident on 2026-06-22. The principle that self-repair "only fixes what it can fix" matters most.

12. Don't let jq open iCloud files directly

Always pass files in iCloud-synced folders via stdin with < "$file". Specifying them as an argument like jq '.key' "$file" probabilistically hits EINTR.

13. The three-piece set of plutil -lint + .bak + bootout/bootstrap

Every time you edit a plist:

1. Back up the pre-change version as .bak
2. Check syntax with plutil -lint (if NG, restore the .bak immediately)
3. Reload with launchctl bootout gui/$UID/$LABEL → launchctl bootstrap gui/$UID /path/to.plist

Forget the bootout and the old plist stays loaded while the new one also starts — double execution begins. My wiki has a record of "a launchd job that had been double-executing for 6 weeks."

14. Overwrite numeric LLM output with deterministic downstream processing

Page counts and file counts output by claude -p are LLM-generated values. For numbers that need to be accurate, splice in a real measurement downstream.

real=$(find "$VAULT/wiki" -name '*.md' -not -path '*/.*' | wc -l | tr -d ' ')
sed -i '' -E "s/総ページ数：[0-9]+/総ページ数：${real}/" "$INDEX_FILE"

Not mixing "the number the LLM said" with "the number counted by find | wc -l" is the foundation of reliability.

15. You can't promise "perfect, every day." You can design "it never goes missing silently."

What the combination of four trigger slots + marker idempotency + FAILED notifications + a git safety net guarantees isn't "running perfectly forever." It's that when a failure happens you will always notice, and it will recover at the next opportunity. Actually running ls ~/.claude/logs/.vault-ingest-step2* shows that only 07-10 is missing its step2b-codex marker. Even so, by the next day, 07-11, all three are back in place. A design that tolerates one day's gap and recovers on its own is what makes zero-operational-cost automation possible.

Summary

Here's the current state of the markers.

$ ls ~/.claude/logs/.vault-ingest-step2*

~/.claude/logs/.vault-ingest-step2-done-20260711
~/.claude/logs/.vault-ingest-step2-done-20260712
~/.claude/logs/.vault-ingest-step2-done-20260713
~/.claude/logs/.vault-ingest-step2-done-20260714
~/.claude/logs/.vault-ingest-step2a-claude-20260711
~/.claude/logs/.vault-ingest-step2a-claude-20260712
~/.claude/logs/.vault-ingest-step2a-claude-20260713
~/.claude/logs/.vault-ingest-step2a-claude-20260714
~/.claude/logs/.vault-ingest-step2b-codex-20260709
~/.claude/logs/.vault-ingest-step2b-codex-20260711
~/.claude/logs/.vault-ingest-step2b-codex-20260712
~/.claude/logs/.vault-ingest-step2b-codex-20260713
~/.claude/logs/.vault-ingest-step2b-codex-20260714

From 07-11 through 07-14, all three of step2a-claude, step2b-codex, and step2-done are present four days in a row. On 07-10, only step2b-codex is missing. That's a day the Codex log side failed to complete for some reason, but it recovered on its own the next day. The presence or absence of a single file expresses exactly — no more, no less — whether that source has been processed today.

vault-auto-ingest.sh is currently 261 lines. Almost every line carries the memory of a failure. The PATH backfill right after set -u handles launchd's minimal environment, the caffeinate self-invoke counters freezes on battery, the 90-second network wait loop counters Wi-Fi not being connected right after wake, and the TCC preflight counters silent death in protected areas. The density of the code is itself an accumulation of operational evidence.

A state where "every morning, yesterday's exchanges with Claude are organized in Obsidian" isn't something you can build in one night of hacking. Three days of freeze, being silently killed by TCC, every slot fully hanging, self-repair deleting my .env — those experiences shaped the current design. Automation gets one line tougher with every failure. That accumulation itself is what's behind the feeling of "the environment is already ahead of me" that supports ¥1.2M/month in revenue.

The full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day procedure are collected in a paid note.
📕 Claude Code自律環境で、実際どう稼ぐか ― 仕組み・実例・始め方・サポート

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*