Multi-Layer Caching for Dashboard Latency

The admin dashboard for the task seeding platform I work on had a latency problem that grew worse with every new feature. Every page load triggered a fan-out of storage requests—reading multiple manifest files from Google Cloud Storage to reconstruct a list of task summaries. Switching tabs refetched everything. Drilling into a specific batch job issued another round-trip per batch to load its tasks. The initial JavaScript bundle was heavy because all eight step viewers (model output, golden data, review interface, and so on) loaded upfront regardless of which step a user actually selected. The page felt sluggish—multi-second loads on a tool that operators used hundreds of times a day.

This post walks through the four layers of caching and the code-splitting strategy that brought the dashboard down to sub-second loads. Each layer addresses a different part of the problem, and they compound: the benefit of all four together is larger than the sum of any one in isolation.

Layer 1 — HTTP Cache-Control headers

The simplest wins came from telling the browser what it already knows: some data does not change often. The dashboard fetches several categories of data with very different volatility profiles, but every API route was returning default headers—effectively no-store—so the browser re-requested everything on every navigation.

The fix was straightforward. Taxonomy data like sectors and occupations changes on the order of weeks or months, so it gets a long public cache. Pool availability data changes frequently when you are looking at all available pools, but the "my pools" scope is tied to the session and needs revalidation. Run listings change when someone creates or cancels a run, so they get a short TTL with stale-while-revalidate for a snappy feel.

function setCacheHeaders(
  res: NextApiResponse,
  profile: "taxonomy" | "pool-available" | "pool-mine" | "runs"
) {
  switch (profile) {
    case "taxonomy":
      // Sectors, occupations — changes rarely
      res.setHeader("Cache-Control", "public, max-age=3600, stale-while-revalidate=600");
      break;
    case "pool-available":
      // Pool listings — short private cache
      res.setHeader("Cache-Control", "private, max-age=5, stale-while-revalidate=10");
      break;
    case "pool-mine":
      // User-specific scope — always revalidate
      res.setHeader("Cache-Control", "private, no-cache");
      break;
    case "runs":
      // Run summaries — moderate TTL
      res.setHeader("Cache-Control", "private, max-age=15, stale-while-revalidate=30");
      break;
  }
}

This is not clever engineering—it is the HTTP spec doing its job. But it eliminates a surprising number of redundant network requests, especially for taxonomy data that the browser was re-downloading on every tab switch. The key insight is matching cache policy to data volatility rather than applying a single policy everywhere.

Layer 2 — Server-side cache with tag invalidation

HTTP caching helps the browser, but it does nothing for the server-side fan-out problem. Every time a user loads the task list, the server reads multiple manifest files from cloud storage, parses them, and assembles a unified response. That fan-out was the dominant contributor to time-to-first-byte.

The solution was to wrap the storage read logic in Next.js's unstable_cache with a 30-second TTL. Reads within that window are served from the server's in-memory cache with no storage calls at all. The critical design decision was tag-based invalidation: every mutation that changes task state—creating a run, updating status, cancelling a batch—calls revalidateTag to bust the cache immediately.

import { unstable_cache, revalidateTag } from "next/cache";

const CACHE_TAG = "task-summaries";

export const getCachedTaskSummaries = unstable_cache(
  async (scope: string): Promise<TaskSummary[]> => {
    const manifests = await listManifestFiles(scope);
    const summaries = await Promise.all(
      manifests.map((m) => parseManifest(m))
    );
    return summaries.flat();
  },
  ["task-summaries"],
  { revalidate: 30, tags: [CACHE_TAG] }
);

export function invalidateTaskCache(reason: string) {
  logger.info("cache.invalidate", { tag: CACHE_TAG, reason });
  revalidateTag(CACHE_TAG);
}

Every mutation handler calls invalidateTaskCache with a structured reason string. A dedicated cache-invalidation module logs every invalidation event, so when debugging staleness issues you can trace exactly which action busted the cache and when. The pattern is simple: reads are instant (served from cache), writes trigger immediate invalidation, and the next read after a write gets fresh data. The 30-second TTL is a safety net—most invalidation happens explicitly through tags rather than by expiry.

This single change cut the median time-to-first-byte for the task list endpoint from over a second to under 50 milliseconds for cache-hit requests.

Layer 3 — React Query migration

The first two layers optimized the network. The third optimized the client. The dashboard was built with raw useEffect + fetch patterns—the kind where every component manages its own loading state, has its own error handling (or does not), and refetches on every mount. Tab switches triggered full refetches even when the data was seconds old. There was no coordination between components that needed the same data.

Migrating to React Query replaced all of that with declarative cache management. The taxonomy hook sets staleTime: Infinity because sectors and occupations genuinely do not change during a user session. The submissions view runs parallel queries for two data sources via useQueries. The expert pool table runs parallel queries for expert data and pool metadata.

The real leverage came from a centralized query key registry. Instead of scattering string keys across components, every key is defined as a hierarchical readonly tuple in one module. This gives you type-safe invalidation cascading for free—invalidating a parent key like ['tasks'] also invalidates every child key like ['tasks', taskId].

export const taskKeys = {
  all: ["tasks"] as const,
  lists: () => [...taskKeys.all, "list"] as const,
  list: (scope: string) => [...taskKeys.lists(), scope] as const,
  details: () => [...taskKeys.all, "detail"] as const,
  detail: (id: string) => [...taskKeys.details(), id] as const,
  batches: (id: string) => [...taskKeys.detail(id), "batches"] as const,
} as const;

export const taxonomyKeys = {
  all: ["taxonomy"] as const,
  sectors: () => [...taxonomyKeys.all, "sectors"] as const,
  occupations: (sectorId?: string) =>
    [...taxonomyKeys.all, "occupations", sectorId] as const,
} as const;

// Invalidating taskKeys.all cascades to every task query
// Invalidating taskKeys.lists() only affects list queries
queryClient.invalidateQueries({ queryKey: taskKeys.all });

This structure means a mutation that creates a new run can invalidate taskKeys.lists() to refresh every list view without touching cached detail data for tasks the user has already drilled into. Granular invalidation prevents the waterfall of unnecessary refetches that plagued the old approach.

The taxonomy hook is a good example of the before-and-after difference. Before: a useEffect that fetches on every mount, manages its own loading and error state, and has no cache awareness. After: a one-liner that fetches once per session and shares the result across every component that needs it.

Layer 4 — Code splitting with dynamic imports

The task detail page renders a multi-step workflow viewer. Each step has its own viewer component—model output, golden data, human review, quality audit, and so on—eight in total. The original implementation imported all eight at the top of the file, so the initial bundle included every viewer's code, dependencies, and assets regardless of which step the user selected. Most users only look at one or two steps per visit.

Switching to dynamic imports defers each viewer until the user actually selects that step. The initial bundle only includes the shell of the detail page and the tab navigation. When a user clicks a step tab, the corresponding viewer loads on demand.

import dynamic from "next/dynamic";

const StepViewers: Record<StepType, React.ComponentType<StepViewerProps>> = {
  MODEL_OUTPUT: dynamic(() => import("./viewers/model-output-viewer")),
  GOLDEN_DATA: dynamic(() => import("./viewers/golden-data-viewer")),
  HUMAN_REVIEW: dynamic(() => import("./viewers/human-review-viewer")),
  QUALITY_AUDIT: dynamic(() => import("./viewers/quality-audit-viewer")),
  FINAL_REVIEW: dynamic(() => import("./viewers/final-review-viewer")),
  CALIBRATION: dynamic(() => import("./viewers/calibration-viewer")),
  EXPERT_REVIEW: dynamic(() => import("./viewers/expert-review-viewer")),
  SUMMARY: dynamic(() => import("./viewers/summary-viewer")),
};

function TaskDetailStepContent({ step, data }: { step: StepType; data: StepData }) {
  const Viewer = StepViewers[step];
  return <Viewer data={data} />;
}

The initial JavaScript bundle for the detail page shrank substantially. Subsequent step loads are fast because each viewer chunk is small and the browser caches it after the first load. Combined with React Query's cache, drilling into a step the user has already visited is effectively instant—no network request, no code load.

Dashboard state in the URL

Caching and code splitting solved the performance problem, but the dashboard had a usability problem too: every navigation reset view state. Selecting a tab, choosing a batch, applying filters, and setting a sort order—all stored in useState, all lost on browser back/forward or page refresh. Users could not share a link to a specific dashboard view.

A custom hook replaced all transient view state with URL search parameters. Each parameter uses a namespace prefix to avoid collisions with other parts of the application.

import { useSearchParams, useRouter, usePathname } from "next/navigation";

const PREFIX = "as.";

export function useAutoSeedUrlState() {
  const searchParams = useSearchParams();
  const router = useRouter();
  const pathname = usePathname();

  const get = (key: string) => searchParams.get(`${PREFIX}${key}`);

  const set = (updates: Record<string, string | null>) => {
    const params = new URLSearchParams(searchParams.toString());
    for (const [key, value] of Object.entries(updates)) {
      const prefixed = `${PREFIX}${key}`;
      if (value === null) {
        params.delete(prefixed);
      } else {
        params.set(prefixed, value);
      }
    }
    router.replace(`${pathname}?${params.toString()}`, { scroll: false });
  };

  return {
    tab: get("tab") ?? "runs",
    batchId: get("batch"),
    sortField: get("sort") ?? "createdAt",
    sortDir: (get("dir") ?? "desc") as "asc" | "desc",
    filterStatus: get("status"),
    setTab: (tab: string) => set({ tab, batch: null }),
    selectBatch: (id: string) => set({ batch: id }),
    setSort: (field: string, dir: "asc" | "desc") => set({ sort: field, dir }),
    setFilter: (status: string | null) => set({ status }),
  };
}

With this hook, browser back/forward preserves the exact dashboard state. Operators can bookmark a filtered view or paste a URL into Slack and the recipient sees the same tab, batch, filters, and sort order. Every useState call for view state was removed in favor of the URL as the single source of truth.

Batch drill-down without a round-trip

The original batch detail view made a separate storage request per batch to load its tasks. Since the task list endpoint already reads every manifest (and the server-side cache keeps that fast), the fix was to include the runTaskIds array in each BatchSummary—a zero-cost addition since those IDs are already present in the manifest data.

When a user drills into a batch, the client filters the already-loaded task list by the batch's task IDs instead of making a new round-trip. Combined with React Query's cache, this means batch drill-down is instantaneous: no network request, no loading spinner, just an in-memory filter over data the client already has.

How the layers compound

Each layer addresses a different bottleneck, and they reinforce each other:

• HTTP Cache-Control eliminates redundant network requests for stable reference data. The browser never re-downloads taxonomy information it fetched minutes ago.
• Server-side cache reduces the cloud storage fan-out from every request to roughly once per 30 seconds (or immediately after a write, via tag invalidation). This is the biggest single improvement to time-to-first-byte.
• React Query prevents unnecessary refetches and re-renders on the client. Components share cached data, stale times are tuned to data volatility, and invalidation is granular thanks to hierarchical keys.
• Dynamic imports shrink the initial bundle so the page becomes interactive faster. Code for rarely-used step viewers never loads unless needed.

A request for taxonomy data might hit all four layers: the browser serves it from its HTTP cache (layer 1), so no request reaches the server. If the browser cache has expired, the server serves it from its own cache (layer 2) without touching storage. React Query on the client marks it as never-stale (layer 3), so even after a hard refresh, subsequent component mounts reuse the cached response. And the component that renders the taxonomy picker is only loaded when the user opens the relevant panel (layer 4).

The result: page loads dropped from multi-second to sub-second. Tab switches that previously triggered full refetches now feel instant. Batch drill-downs that required per-batch round-trips resolve from in-memory data. The initial bundle is a fraction of its former size. And because every layer has clear invalidation semantics—HTTP expiry, tag-based revalidation, React Query key invalidation—the data is never stale after a write. The dashboard went from something operators tolerated to something they trust.