Every night my Mac quietly rewrites my long-term memory. Not metaphorically — a shell script drains that day's Claude Code conversation logs into an Obsidian vault, commits them to a private repo, and leaves a briefing on my desktop. It took three real outages to make it reliable. This is the script, the three failures, and the design that came out of them.
Why This Setup Works
Claude Code's "memory" disappears by default
Claude Code sessions are independent of one another. The root cause of a bug you found during a long working session today, the reason you settled on a particular architecture after trial and error, the accumulated knowledge that "this direction already failed once" — none of it is available in the next conversation once you close the session.
Even on a paid plan, even with the most capable model available, if context isn't carried over you have to explain everything from scratch every time. Many people have had the experience of thinking "I already looked this up before" or "I should have failed at this once already, and yet here I am heading down the same road again."
In a phase where you're shipping personal projects in volume, this problem is fatal. Once three or four projects are running in parallel, tracking "where each project currently stands" by hand hits a wall fast. And Claude, unable to reference previous conversations, repeats the same deliberations.
The solution is to build an environment, not a task
My first attempt at this problem was "I'll write up a summary by hand every day." It didn't last. When work has momentum you don't feel like writing a summary, and when you're tired you can write even less. A system that depends on human willpower doesn't function during a high-volume solo-dev phase.
The answer was to build an environment that automatically drains Claude's conversation logs into Obsidian every night. Once the environment is in place, willpower and motivation are irrelevant. The Mac just does it.
The reason I chose Obsidian is simple. The files are local Markdown, so Claude Code can read and write them directly. They can be version-controlled with Git. The [[link]] syntax lets you connect pieces of knowledge to each other. Logs flow in every morning and cross-project links grow naturally — from the moment this started functioning as an "external brain," the quality of my work changed.
Why a triple-layered structure became necessary
Simply "running a script at 4:55 every day" produced three distinct kinds of failure once I actually ran it. Each one only became apparent after it caused real damage.
Sleep freeze: If you close the lid on the Mac and go to bed, caffeinate -s (which only takes effect on AC power) can't prevent sleep. The script stops partway through, and that day's processing hangs in limbo until the next slot fires.
Double-execution race: An actual incident on 2026-06-10. A scheduled launchd firing overlapped with a manual run. Both tried to operate on the same vault with Git, and the result was a conflict.
Timeout: On 2026-06-13, every slot failed for the entire day. Digesting 28 hours' worth of logs via claude -p didn't fit within the 40-minute timeout window and all of it got culled. In the logs, the pattern started 04:55:00 → step2 timeout at 05:40:01 lines up across all four slots, completely uniform.
These three real failures are what forced the triple-layered structure of "multi-slot re-firing," "caffeinate sleep prevention," and "idempotent retry via step markers." It didn't come out of a design document; it accumulated from things that actually broke.
The feeling of an environment that grows
Once the system started running stably, logs pile up in Obsidian every morning. Claude reads those logs, updates per-project articles, and this week's talking points accumulate in hot.md (a summary of recent context).
When Claude reads hot.md and wiki/ in the next conversation, it can start working already knowing "last week's decisions," "the approach that failed once," and "the current state of the three projects running in parallel." That's an external long-term memory that doesn't forget when the conversation ends — a prerequisite for getting Claude Code to perform at its actual potential.
The Overall Flow
Four-layer architecture
The data the script handles flows through the following four layers.
【層1】Claude Code セッションログ（セッション終了時にStop hookが書き出し）
↓ extract_conversations.py（step1: 最新化）
【層2】~/Documents/my-knowledge-base/raw/conversations/
↓ claude -p（step2a: Claude由来ログを消化, timeout 1500s）
~/Documents/my-knowledge-base/raw/codex-conversations/
↓ claude -p（step2b: Codex由来ログを消化, timeout 1500s）
【層3】Obsidian Vault（~/Documents/claude-obsidian/wiki/）
├── hot.md（直近サマリ）
├── index.md（全体目次）
├── projects/ / learning/ / career/ ... （ドメイン別記事）
└── today-brief.md（step2.5: 今日の行動提案）
↓ git add -A && git commit && git push
【層4】private repo（安全網: 荒れてもrevert可能）
Splitting layer 2 into two lines, "Claude-derived" and "Codex-derived," is a change made on or after 2026-06-11. Originally one line processed everything, but on high-activity days it stopped fitting into the 40-minute timeout window. After the split, each has an independent 1500-second (25-minute) timeout, and if one fails the next slot can retry only what's left.
Implementing multi-slot firing
Four slots defined in a launchd plist (~/Library/LaunchAgents/com.shun.vault-auto-ingest.plist) spend the day repeatedly "retrying until success."
<key>StartCalendarInterval</key>
<array>
<dict><key>Hour</key><integer>4</integer><key>Minute</key><integer>55</integer></dict>
<dict><key>Hour</key><integer>8</integer><key>Minute</key><integer>20</integer></dict>
<dict><key>Hour</key><integer>10</integer><key>Minute</key><integer>45</integer></dict>
<dict><key>Hour</key><integer>12</integer><key>Minute</key><integer>15</integer></dict>
</array>
RunAtLoad is false. There's no auto-start at login. The four slots are the only firing sources.
At the top of the script it checks for the day's success marker and exits immediately if it's already finished.
TODAY=$(date +%Y%m%d)
DONE_MARKER="$HOME/.claude/logs/.vault-ingest-done-${TODAY}"
# 0. 本日分が既に成功していれば即終了
[ -f "$DONE_MARKER" ] && exit 0
With this, if 4:55 succeeds then 8:20, 10:45, and 12:15 become harmless no-ops that just "check the file and exit." Only if 4:55 fails (sleep freeze, no network, a transient launchd fault, etc.) does 8:20 actually take over the processing.
Sleep prevention with caffeinate
When running this on a MacBook, closing the lid is something caffeinate -s (only effective on AC power) can't fully prevent. So the script re-executes itself under both -i (prevent system sleep) and -s.
if [ -z "${CAFFEINATED:-}" ]; then
exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@"
fi
If the CAFFEINATED environment variable isn't set, it re-executes itself under caffeinate and puts everything after that under caffeinate's umbrella. Since CAFFEINATED=1 is set after the single re-execution, there's no infinite loop.
-s has no effect on battery power. If it freezes from lid-close sleep, timeout culls it and the next slot starts over. If step2 got partway through, the markers let it skip what's done and run only the rest.
Preventing double execution with a lockdir
This is a locking mechanism that uses the atomicity of mkdir (if two processes call it simultaneously, only one succeeds). It's a widely used technique for file locking in Bash.
LOCKDIR="$HOME/.claude/locks/vault-auto-ingest.lock"
if ! /bin/mkdir "$LOCKDIR" 2>/dev/null; then
oldpid=$(cat "$LOCKDIR/pid" 2>/dev/null || true)
if [ -n "${oldpid:-}" ] && kill -0 "$oldpid" 2>/dev/null; then
echo "[$(date '+%F %T')] 別インスタンス実行中(pid=${oldpid}) — skip" >> "$LOG"
exit 0
fi
rm -rf "$LOCKDIR"
/bin/mkdir "$LOCKDIR" 2>/dev/null || exit 0
fi
echo $$ > "$LOCKDIR/pid"
trap 'rm -rf "$LOCKDIR"' EXIT INT TERM
Even when an existing lockdir is present, it checks the PID, and if the process is alive it exits as "another instance is running." If the process is dead (a lock left behind by an abnormal termination), it reclaims it as stale, acquires a new lock, and continues. trap guarantees the lockdir is deleted when the script exits.
Idempotent retry via step markers
This is the part that took the most work. step2 (the ingest into the vault) is split into two lines, Claude and Codex, each with its own independent marker.
STEP2A_MARKER="$HOME/.claude/logs/.vault-ingest-step2a-claude-${TODAY}"
STEP2B_MARKER="$HOME/.claude/logs/.vault-ingest-step2b-codex-${TODAY}"
The ingest_src function looks at these markers to decide "skip if already complete, run if not."
ingest_src() {
local marker="$1" src="$2" name="$3" to="$4" extra="$5"
[ -f "$marker" ] && { echo "[...] step2($name) は本日実施済み — skip" >> "$LOG"; return 0; }
cd "$VAULT" && run_to "$to" "$CLAUDE" -p \
"...（Vaultのルールに従って wiki/ を更新するプロンプト）..." \
--dangerously-skip-permissions >> "$LOG" 2>&1 \
&& { touch "$marker"; return 0; } \
|| { echo "[...] WARN: step2($name) 失敗/timeout（次スロットで再試行）" >> "$LOG"; return 1; }
}
ingest_src "$STEP2A_MARKER" "$KB/raw/conversations/" "claude" 1500 ""
ingest_src "$STEP2B_MARKER" "$KB/raw/codex-conversations/" "codex" 1500 "Codex由来でも既存記事に統合し重複は追記でまとめろ。"
[ -f "$STEP2A_MARKER" ] && [ -f "$STEP2B_MARKER" ] && touch "$STEP2_MARKER"
The behavior pattern is as follows.
4:55 発火
├── DONE_MARKER なし → 処理続行
├── step2a 実行（claude由来, 1500s上限） → 成功 → STEP2A_MARKER 作成
├── step2b 実行（codex由来, 1500s上限） → timeout! → マーカーなし
├── brief生成 → 失敗（step2b未完でログが薄い） → DONE_MARKER 作らない
└── notify_fail で Desktop に FAILED ファイル + 通知
8:20 発火
├── DONE_MARKER なし → 処理続行
├── step2a → STEP2A_MARKER あり → skip（再実行しない）
├── step2b → マーカーなし → 実行 → 成功 → STEP2B_MARKER 作成
├── brief生成 → 成功
├── DONE_MARKER 作成 ✓
└── FAILED ファイル削除
10:45 / 12:15 発火
└── DONE_MARKER あり → exit 0（空振り）
If step2a succeeded, 8:20 runs only step2b. If both step2a and step2b are finished, 8:20 runs only the brief. Because which slot is responsible for what is determined dynamically, the guarantee that "even if 4:55 fails, it will definitely finish within the day" is preserved.
Filling in launchd PATH and the node binary
launchd's execution environment has only a minimal PATH, roughly /usr/bin:/bin:/usr/sbin:/sbin. To avoid the trap where the post-git commit hook (written in node) doesn't run and the commit itself fails, the top of the script auto-discovers the newest nvm node and adds it.
NODE_BIN=$(ls -d "$HOME"/.nvm/versions/node/*/bin 2>/dev/null | sort -V | tail -1)
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$PATH"
[ -n "$NODE_BIN" ] && export PATH="${NODE_BIN}:$PATH"
Taking the tail of sort -V (version-order sort) means it always uses the newest even when nvm has multiple versions installed. Since no version is hardcoded, adding versions with nvm install requires no changes to the script.
Also, due to macOS TCC (privacy protection), launchd can't write under ~/Documents/ unless /bin/bash has Full Disk Access. The script detects this early to prevent silent failure.
if ! ( cd "$VAULT" 2>/dev/null && git rev-parse --git-dir >/dev/null 2>&1 ); then
echo "[...] ❌ FDA未付与: launchdから '$VAULT' にアクセス不可(TCC保護)。" >> "$LOG"
notify_fail "FDA未付与: vault にアクセス不可（設定→フルディスクアクセス→/bin/bash）"
exit 1
fi
Returning exit 0 would look like "it succeeded," DONE_MARKER would be created, and the next slot wouldn't retry. TCC failure is signaled explicitly with exit 1 and handed off to the next slot.
Implementation Details
The timeout helper and designing for a guaranteed kill
One of the script's core decisions is the run_to function.
TIMEOUT_BIN="/opt/homebrew/bin/timeout"
[ -x "$TIMEOUT_BIN" ] || TIMEOUT_BIN=""
run_to() { local s=$1; shift; if [ -n "$TIMEOUT_BIN" ]; then "$TIMEOUT_BIN" --kill-after=30 "$s" "$@"; else "$@"; fi; }
It calls GNU coreutils' timeout by explicit path. macOS's standard /usr/bin doesn't have the GNU version of timeout. In environments where /opt/homebrew/bin/timeout doesn't exist, TIMEOUT_BIN is left empty and calls pass through (hang resistance drops, but it keeps working).
--kill-after=30 matters more than it looks. If the process is still alive 30 seconds after SIGTERM is sent, it's force-killed with SIGKILL. claude -p sometimes ignores SIGTERM while doing heavy work, so without --kill-after you can hit the worst case: "the process survives past the timeout while still holding the lockdir."
step2a (Claude-derived logs) and step2b (Codex-derived logs) each get a 1500-second (25-minute) limit. As mentioned earlier, it was originally a single 2400-second (40-minute) process, but as activity increased it stopped fitting. Splitting into two lines at 1500 seconds each means that if one dies, the next slot can re-run only the remainder.
Waiting for the network after wake, and the preflight detection chain
Even when the Mac wakes at 4:55, it can take tens of seconds for the Wi-Fi connection to stabilize. Both claude and git push need the network, so running before the connection is up is an immediate error.
net_ok=""
for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18; do
if /usr/bin/nc -z -G 3 1.1.1.1 443 2>/dev/null; then net_ok=1; break; fi
sleep 5
done
[ -z "$net_ok" ] && echo "[...] WARN: 網未接続のまま続行（失敗時は次スロットが再試行）" >> "$LOG"
18 attempts × 5 seconds = up to 90 seconds of waiting. nc -z -G 3 is a TCP connection test that times out in 3 seconds, lighter than curl. If it still isn't connected after 90 seconds, it "warns and continues." Using exit 1 here would also stop step1 (local processing) in an offline environment. Since the design already assumes the next slot auto-retries on failure, "record it without stopping" is enough here.
Next, preflight detection chains through three stages.
① Checking for the claude binary: This catches the case where an update removes the symlink target. Running claude update can temporarily leave the binary absent, and this is a cause of silent failure.
② macOS TCC preflight: If the vault is under ~/Documents/, a process running from launchd can't write to it without Full Disk Access.
if ! ( cd "$VAULT" 2>/dev/null && git rev-parse --git-dir >/dev/null 2>&1 ); then
echo "[...] ❌ FDA未付与: launchdから '$VAULT' にアクセス不可(TCC保護)。" >> "$LOG"
notify_fail "FDA未付与: vault にアクセス不可（設定→フルディスクアクセス→/bin/bash）"
exit 1
fi
Returning exit 0 here would create DONE_MARKER, and the next slot would misread it as "succeeded" and not retry. TCC failure must always exit with exit 1 and hand off to the next slot. This distinction is also at the root of one of the failure stories below.
Log bloat guard and brief freshness checks
Writing all of claude -p's output to the log gets you to tens of MB within a few weeks.
if [ -f "$LOG" ] && [ "$(stat -f%z "$LOG" 2>/dev/null || echo 0)" -gt 5242880 ]; then
mv "$LOG" "${LOG}.old"
fi
Once it exceeds 5 MB (5,242,880 bytes), it renames the file to .old and starts a new log. stat -f%z is the macOS file-size command. Since the options differ from GNU stat, || echo 0 provides a fallback.
The brief freshness check is another important part.
START_STAMP=$(mktemp /tmp/vault-ingest-start.XXXXXX)
This temp file is created at the top of the script, and after step2.5 (brief generation) it checks whether the brief is newer than it.
if [ -s "$BRIEF_SRC" ] && [ "$BRIEF_SRC" -nt "$START_STAMP" ]; then
# アーカイブ処理
touch "$DONE_MARKER"
else
notify_fail "ブリーフ生成が未完"
fi
Comparing with -nt (newer than) prevents the accident of "mistakenly archiving yesterday's leftover today-brief.md as today's output." If the brief isn't newer than the start of this run, it isn't treated as a success — DONE_MARKER isn't created and it hands off to the next slot.
Overwriting LLM output with something deterministic
When claude -p updates wiki/index.md, it sometimes writes an incorrect value for "total page count." The model infers it from context, so it doesn't match the actual file count.
real=$(find "$VAULT/wiki" -name '*.md' -not -path '*/.*' | wc -l | tr -d ' ')
sed -i '' -E "s/総ページ数：[0-9]+/総ページ数：${real}/" "$INDEX_FILE"
It recounts the real file count with find and force-overwrites with sed. Because this runs immediately before git commit, what gets committed has the LLM-generated number already replaced by a deterministic value. The design principle is: "numbers output by an LLM get overwritten by a deterministic check downstream."
The calendar last-known-good pattern
When writing "today's schedule" into the brief, it retains past information on days when calendar data can't be fetched.
if [ -n "$CAL_SRC" ]; then
cp "$CAL_TMP" "$CAL_SNAPSHOT"
cp "$CAL_SNAPSHOT" "$CAL_LASTGOOD" # 成功したら last-good を更新
elif [ -s "$CAL_LASTGOOD" ]; then
# 取得失敗: 前回 good を温存し stale 印を付けて出力
{ tail -n +4 "$CAL_LASTGOOD"; } > "$CAL_SNAPSHOT"
fi
It tries the Google Calendar API (ADC auth) and icalBuddy (local Apple Calendar) in order, and if both fail it uses _calendar-snapshot.md.lastgood from the last successful fetch. The principle is "never clobber the source of truth with empty." Staleness is marked explicitly with a leading ⚠️, and the brief-generation prompt is told to "note that the values are as of the fetch date if it says stale."
Where I Got Stuck
Failure ①: The day every slot died at the same spot, to the second (2026-06-13)
I noticed because FAILED-20260613.md was sitting in ~/Desktop/Daily Brief/. Opening the log, all four slots showed exactly the same pattern.
04:55:00 ===== auto-ingest 開始 =====
05:40:01 WARN: step2(claude) 失敗/timeout（次スロットで再試行）
08:20:00 ===== auto-ingest 開始 =====
09:05:01 WARN: step2(claude) 失敗/timeout（次スロットで再試行）
10:45:00 ===== auto-ingest 開始 =====
11:30:01 WARN: step2(claude) 失敗/timeout（次スロットで再試行）
12:15:00 ===== auto-ingest 開始 =====
13:00:01 WARN: step2(claude) 失敗/timeout（次スロットで再試行）
Every slot timed out at 45 minutes 01 second. The step2 timeout at the time was 2400 seconds (40 minutes). claude -p kept processing for 40 minutes straight and got timed out, the next slot retried with the same result, and that loop repeated four times.
What I suspected first was the error message after git commit. The strings "commit-msg hook" and "node not found" were visible in the log, so I thought the node binary was the cause. But running claude -p "say OK" by hand returned exit 0 immediately. Trying commit by hand worked fine.
The real cause was a transient failure on the claude -p side. 06-12 worked normally; only on 06-13 did all requests hang (probably rate limiting or a temporary service-side fault), and it was resolved the next day. The commit-msg hook theory was completely wrong.
Two lessons came out of this. One: "an error message is a suspect, not a conviction — actually run things to isolate the cause." The other concerns cleanup of FAILED markers. The cleanup at the time only "deleted that day's FAILED file on success." The 06-13 FAILED file would keep sitting on my desk the next day and beyond.
# 修正後: 過去日の FAILED も正常稼働日に掃く
find "$HOME/Desktop/Daily Brief" -maxdepth 1 -name "FAILED-*.md" ! -name "FAILED-${TODAY}.md" -delete 2>/dev/null
Adding this one line means that even if failures occur across multiple consecutive days, all the past days' files get cleared on the first day that runs normally.
Failure ②: launchd and a manual run tried to write the vault at the same time (2026-06-10)
One morning, wanting to check on the script's progress, I also ran vault-auto-ingest.sh manually. launchd's 4:55 slot was already running. Both ran git add -A && git commit on the same vault, and a conflict occurred.
error: cannot lock ref 'refs/heads/main': is at xxx but expected yyy
Because the two collided mid-processing, the vault ended up in a half-finished state and needed a manual reset.
The mkdir lock described earlier solves this, but the key point is "automatic reclamation of stale locks."
if ! /bin/mkdir "$LOCKDIR" 2>/dev/null; then
oldpid=$(cat "$LOCKDIR/pid" 2>/dev/null || true)
if [ -n "${oldpid:-}" ] && kill -0 "$oldpid" 2>/dev/null; then
echo "[...] 別インスタンス実行中(pid=${oldpid}) — skip" >> "$LOG"
exit 0
fi
rm -rf "$LOCKDIR"
/bin/mkdir "$LOCKDIR" 2>/dev/null || exit 0
fi
kill -0 "$oldpid" sends no signal; it only checks whether the process is alive. If the process is dead, it deletes the stale lockdir and takes the lock itself. This prevents the worst case: "a previous abnormal termination leaves the lockdir behind and every subsequent run skips forever."
Failure ③: The TCC trap — "it says it succeeded but nothing gets updated"
There was a period when I kept the vault at ~/Documents/claude-obsidian/ (later moved to ~/claude-obsidian/, outside TCC protection). The script at the time didn't handle TCC errors properly and exited with exit 0 even when cd "$VAULT" failed.
# 修正前の問題コード（イメージ）
cd "$VAULT" && git rev-parse --git-dir >/dev/null 2>&1 || exit 0
^^^^^^
この exit 0 が DONE_MARKER を作る前提を壊す
When the script exits with exit 0, the calling launchd treats it as "terminated normally." But since the same thing happens on the next run too, all four slots end up in a state of "appearing to succeed while doing nothing," with DONE_MARKER never created.
The symptoms were strange. The log recorded "===== 完了 =====" four times. But the vault wasn't updated. There was no DONE_MARKER either.
I noticed the cause from a message that appeared when I ran the script interactively by hand. When run from launchd, /bin/bash lacks Full Disk Access, so cd "$VAULT" was failing silently.
After the fix, TCC failure is signaled explicitly with exit 1, and it creates a FAILED file on the desktop plus a notification.
if ! ( cd "$VAULT" 2>/dev/null && git rev-parse --git-dir >/dev/null 2>&1 ); then
notify_fail "FDA未付与: vault にアクセス不可（設定→フルディスクアクセス→/bin/bash）"
exit 1 # DONE_MARKER は作られない → 次スロットが再試行
fi
The distinction between exit 0 and exit 1 is what separates "it succeeded" from "the next slot takes over." In an automation context, a silent fake success is the hardest failure mode to diagnose.
Failure ④: Trusting caffeinate -s too much
The first implementation put only caffeinate -s at the top of the script.
# 初期の誤った実装
caffeinate -s "$0" "$@"
The -s flag is described as "prevent system sleep." But in practice it only works while connected to AC power. Close the lid on battery and macOS goes to sleep, freezing the script under caffeinate -s along with it.
Checking the next morning, there were traces that the script had started but it had ended in a half-finished state. Because timeout culled it afterward the lockdir was gone, but step2a's marker existed while step2b's did not.
The fix was to switch to using both -i (prevent system sleep) and -s.
if [ -z "${CAFFEINATED:-}" ]; then
exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@"
fi
-i is effective on battery too. That said, there are situations where lid-close sleep (the suspend equivalent, not display sleep) can't be fully prevented even with -i. The multi-slot structure — "timeout culls it and the next slot retries the remainder" — is the design that assumes this. caffeinate is only an aid that delays sleep; what guarantees "the processing will complete" is the combination of markers and multiple slots.
The actual total of "time spent stuck"
Sorting through these failures, all of them stayed within one day of real damage. The sleep freeze was recovered by the next 8:20 slot. The double-execution race took 10 minutes of manual reset. The all-day outage on 06-13 auto-recovered on the next morning's 4:55 slot. The TCC trap took a few days to notice, but once noticed the fix took under 30 minutes.
The reason I could notice any of these failures is the logs and the FAILED file on the desktop. Had they failed silently, I might not have noticed that the vault hadn't been updated for days. The design of having notify_fail place a FAILED file on the desktop exists to maximize the chance of noticing. Automation isn't "run it and you're done" — it only really works once you've designed it up through "you'll reliably notice when it breaks." That's my honest takeaway from six months.
Gotchas
The earlier sections covered four real failures (all slots dying, the double-execution race, the TCC trap, misplaced faith in caffeinate). Here I'll cover the smaller points that are "not that big, but you will definitely trip on them."
The combination of set -u and launchd environment variables kills you instantly
The script has set -u at the top (it's in the actual code). It's the option that makes referencing an undefined variable exit 1 immediately. A shell running from launchd, unlike an interactive shell, has many environment variables undefined. If you reference a variable without giving it a default via the ${VAR:-} form, you get a phenomenon where it's fine in interactive runs but dies only under launchd. In this script, ${CAFFEINATED:-}, ${oldpid:-}, and ${net_ok:-} all explicitly specify an empty default. Miss one and you mass-produce unexplained exit 1s.
Forget to set RunAtLoad and it runs at every login
The plist explicitly has <key>RunAtLoad</key><false/>. Without it the default becomes true, and the script fires one extra time every time you restart or log in to the Mac. Since it exits immediately when DONE_MARKER exists there's no real harm, but the CPU load right after boot and the unintended log pollution accumulate.
Without trap, the lockdir stays behind and every subsequent run skips
trap 'rm -rf "$LOCKDIR"' EXIT INT TERM
Without this one line, the lockdir remains when the script is stopped with Ctrl+C or kill. At the next firing, trying to acquire the lock with mkdir finds the existing lockdir and misjudges it as "another instance running," so it skips. But if you've written code that checks the PID, stale locks are reclaimed automatically. Without stale reclamation, it skips forever.
Running late at night, the date rolls over and two DONE_MARKERs get generated
If the 4:55 slot starts running at 0:10 AM (via a manual start, say), date +%Y%m%d changes mid-processing and the DONE_MARKER filename changes with it. To prevent this, the script pins the date as of the start of the run with BD="$TODAY".
BD="$TODAY" # 日付跨ぎ対策: 評価はラン開始時の日付で固定
ARCH_FILE="$VAULT/wiki/briefs/daily/today-brief-${BD}.md"
TODAY is obtained exactly once at the top of the script. Even for long-running processing that crosses midnight, filenames stay unified under "the day the run started."
stat -f%z is a macOS-only option
This is the log rotation code.
[ "$(stat -f%z "$LOG" 2>/dev/null || echo 0)" -gt 5242880 ]
On GNU stat (Linux) it's -c%s. Since this is written assuming macOS it uses stat -f%z, but if you try to port it to Linux it silently falls through to echo 0 and never rotates. The || echo 0 fallback means it doesn't break, but the log keeps growing.
Without launchd's StandardErrorPath, stderr vanishes
The plist contains the following.
<key>StandardErrorPath</key>
<string>/Users/.../.claude/logs/vault-auto-ingest.launchd.log</string>
Without this, stderr from processes run via launchd is discarded to the equivalent of /dev/null. Even if step2's claude -p is writing something to stderr, you'll never see it. That creates a hard-to-diagnose situation where it's visible in interactive runs and invisible under launchd.
A node-based commit-msg hook dies without PATH augmentation
Running git commit in the vault executes the commit-msg hook. If that hook is written in node, launchd's minimal PATH (roughly /usr/bin:/bin) has no node. git commit stops with a "hook execution failed" and finishes without committing anything. The script auto-discovers nvm's node and adds it to PATH.
NODE_BIN=$(ls -d "$HOME"/.nvm/versions/node/*/bin 2>/dev/null | sort -V | tail -1)
[ -n "$NODE_BIN" ] && export PATH="${NODE_BIN}:$PATH"
The trick is taking the tail of a version-order sort with sort -V. With a string sort you get the inversion where v9.x sorts after v10.x.
Without spelling out "merge into existing articles" in step2b's prompt, you mass-produce duplicate articles
step2a (Claude-derived logs) runs first and updates the vault articles. When step2b (Codex-derived logs) runs afterward, it tries to create new articles on the same topics. Without explicit instructions in the prompt, articles with the same content split into projects/foo.md and projects/foo-2.md.
ingest_src "$STEP2B_MARKER" "$KB/raw/codex-conversations/" "codex" 1500 \
"Codex由来でも舜の知識として既存記事に統合し、Claude側と重複する話題は新記事を作らず追記でまとめろ。"
This additional prompt (the extra argument) is passed only to step2b. The instruction to "merge and append" works because it comes after seeing step2a's results.
Forget to delete the START_STAMP used for the -nt comparison and the brief is always judged "old"
START_STAMP=$(mktemp /tmp/vault-ingest-start.XXXXXX)
This temp file is deleted when the script exits. But if it remains in /tmp, it isn't created under the same name on the next run, and comparison against an old START_STAMP can judge today-brief.md as "old," entering a loop where DONE_MARKER is never created. The rm -f "$START_STAMP" at the end of the script is essential.
Marker files left for more than 7 days bloat /tmp
find "$HOME/.claude/logs" -maxdepth 1 -name '.vault-ingest-*' -mtime +7 -delete 2>/dev/null
Leave the daily-generated marker files alone and you accumulate 365 files a year in ~/.claude/logs/. -mtime +7 periodically deletes anything older than 7 days. This cleanup line is in there as a memento.
Best Practices
Twelve operating principles that solidified over six months of actually running this.
① Separate "success" from "failure that hands off to the next slot" via exit code
This is the lesson from the TCC incident. Returning exit 0 creates DONE_MARKER and the next slot won't retry. Genuine failures must always be exit 1. "Silent fake success" is the hardest-to-diagnose form of failure in automation.
② Create DONE_MARKER only after all steps are complete
If you create DONE_MARKER when step2a completes, "today is done" becomes true even with step2b and the brief unfinished. In this script, DONE_MARKER is created only when the brief is newer than START_STAMP. The ordering principle: raise the flag only after the final artifact has been produced.
③ Use intermediate markers to enable partial retry
Rather than a simple two-state "done/not done," give step2a and step2b each a half-marker. If step2a succeeded, the next slot skips step2a and starts from step2b. The more steps there are, the more valuable partial retry becomes.
④ Treat caffeinate as an aid that delays sleep, and accept that
Even using both -i and -s, suspend on a lid-closed MacBook can't be fully prevented. caffeinate is best-effort. What guarantees "it will definitely complete" is markers and multiple slots. A design that leans too hard on caffeinate will always break under battery operation.
⑤ Derive the number of slots backward from "even the slowest failure finishes within the day"
The four slots (4:55 / 8:20 / 10:45 / 12:15) are designed from the view that "if it finishes by noon, it's usable that day." Even if 4:55 fails, succeeding at 12:15 at worst still gives you a brief based on that day's activity logs. The next morning's firing is for processing the next day's material.
⑥ Always attach --kill-after=30
run_to() { "$TIMEOUT_BIN" --kill-after=30 "$s" "$@"; }
It sends SIGKILL 30 seconds after SIGTERM. claude -p sometimes ignores SIGTERM while doing heavy work. Without kill-after, you end up with "the process survives past the timeout while still holding the lockdir."
⑦ Overwrite LLM-output numbers with something deterministic downstream
real=$(find "$VAULT/wiki" -name '*.md' -not -path '*/.*' | wc -l | tr -d ' ')
sed -i '' -E "s/総ページ数：[0-9]+/総ページ数：${real}/" "$INDEX_FILE"
The "total page count" Claude writes into wiki/index.md is a guess. Count the real files and overwrite. Because this runs right before git commit, the repository always has the measured value committed. The design principle: never treat LLM-generated numbers as authoritative.
⑧ Retain the previous value even on fetch failure (last-known-good)
Calendar information has a three-stage fallback: Google Calendar API → icalBuddy → lastgood. Not "it couldn't be fetched, so blank it," but "retain the value from the last success, marked stale." The brief prompt is instructed to call it out explicitly if ⚠️stale is present. Stale information is more useful than none.
⑨ Put failures somewhere visible
FAILED_FILE="$HOME/Desktop/Daily Brief/FAILED-${TODAY}.md"
notify_fail() {
mkdir -p "$HOME/Desktop/Daily Brief"
{ echo "# Daily Brief 生成失敗 — ..."; echo "- 自動再試行: 8:20 / 10:45 / 12:15 ..."; } > "$FAILED_FILE"
/usr/bin/osascript -e "display notification ..."
}
Log files are things you don't notice unless you go look. Putting a FAILED file on the desktop and firing a macOS notification puts "the automation is broken" in front of your eyes. And on success, that file is auto-deleted. A file that "exists only while failing" is proof of normal operation.
⑩ Clean up past days' FAILED files too
This one line was added while cleaning up after the consecutive 06-13 failures.
find "$HOME/Desktop/Daily Brief" -maxdepth 1 -name "FAILED-*.md" ! -name "FAILED-${TODAY}.md" -delete 2>/dev/null
With a design of "delete only that day's file on that day's success," FAILED files keep piling up on your desk after several consecutive days of failure. Design it so that the first normally-operating day deletes all the past days' files.
⑪ Auto-discover the nvm node version instead of hardcoding an absolute path
The launchd plist's EnvironmentVariables hardcodes the v24.13.0 path (line 8 of the reference file). But the node discovery inside the script isn't pinned to an absolute version — it takes the newest via sort -V | tail -1. If you want to rely on the plist's fixed path, you need to update it every time you run nvm install. Using the script's dynamic discovery means no changes are needed after a version update. Be conscious of which one you're treating as the source of truth.
⑫ "It works" and "you'll notice when it breaks" are separate design problems
The first version only thought as far as "it works." When it broke, I might not have noticed that the vault hadn't been updated for days. Deliberately design multiple detection paths: the FAILED file, the notification, and noticing that the next day's brief is stale. Automation only becomes trustworthy once you've designed it through to "you'll reliably notice when it breaks."
Summary
Let's look back at the triple-layered structure of vault-auto-ingest.sh.
Multi-slot re-firing gives you the guarantee of "retrying over the course of a day until it succeeds." The design starts at 4:55 and runs as late as 12:15 in the worst case, but on most days it finishes at 4:55 or 8:20. The remaining two slots become no-ops that check DONE_MARKER and exit immediately.
Sleep prevention via caffeinate -i -s is an aid that prevents the process from freezing partway through. It can't fully counter a lid-closed battery-powered Mac, but a frozen run gets culled by timeout and the next slot takes over. The combination of caffeinate and the marker structure establishes a structure where "even if part of it dies, it moves forward."
Idempotent retry via step markers is the most important part. step2a and step2b each hold independent markers, and completed steps are never re-run. Splitting processing that didn't fit in a 40-minute window into two 25-minute lines means that if one dies, the next slot runs only the rest. This turns "partially timed out" into "partially finished."
What changed once this ran stably every morning? When Claude Code reads hot.md the next morning, it can start working already knowing yesterday's decisions, the reason an architecture failed once, and the current state of three projects running in parallel. Explaining everything from scratch every time becomes unnecessary. Management cost doesn't grow even when parallel projects reach three or four.
The value of automation shows not "when you get it running," but "when it recovers on its own after breaking and keeps running so routinely you don't notice it."
Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*
