Go Completely Offline: Build a Privacy-First Personal Finance Assistant with LiteRT and Gemma 4

Imagine having a personal financial planner that analyzes your sensitive transaction history, flags spending anomalies, and suggests budget optimizations—all while running 100% offline, directly inside your web browser.

In this guide, we will build an offline Retrieval-Augmented Generation (RAG) assistant powered by the LiteRT LM Web API and the highly efficient, on-device Gemma 4 E2B small language model. By caching the model weights directly in the browser's Cache Storage, your application will achieve complete privacy, zero token costs, and genuine offline capability.

1. Prerequisites

• LiteRT LM Web API
• Gemma 4 E2B model
• Angular 22
• TailwindCSS
• Node 24

While Angular and TailwindCSS provide the user interface and styling, the LiteRT LM Web API and the on-device small model power this demo's AI capabilities.

npm i --save-exact @litert-lm/core tailwindcss postcss @tailwindcss/postcss jsonrepair dexie
npm i --save-exact --save-dev angular-eslint husky lint-staged serve @commitlint/cli

We install additional dependencies for on-device AI, CSS Styling, JSON response streaming, and IndexedDB storage. The dev dependencies help automatically catch code smells, enforce code quality, and serve the production bundle.

2. Architecture

We designed this application to process transactions entirely on-device, ensuring financial data privacy inside the browser. The process operates in three key phases:

• Model and Data Storage: First, the application downloads and caches the Gemma 4 E2B model weights directly in the browser's Cache Storage so they are available offline. When users log expenses, they are saved locally in IndexedDB.

• Arithmetic Grounding: Small models hallucinate when processing arithmetic queries, so we aggregate expenses into a monthly total, a daily total, and an expense-by-category breakdown. We supply the precomputed values to the Gemma WebGPU engine as ground truth, along with the user query.

• Conversation Management: To prevent exceeding browser token limits, the application resets the conversation after three turns but retains a client-side memory of the last two queries. On reset, the application re-primes the model with immutable financial grounding data before replaying those queries. This maintains conversational continuity while keeping memory usage within on-device limits.

Next, let's look at how we instantiate this LiteRT LM engine, define the insight service, and design the Angular user interfaces to generate these insights.

3. Source Code

While the full codebase is available in the NG Personal Finance Assistant repository, our application relies on an IndexedDB database to track expenses offline. You can find the database implementation in the service file, which is injected via an injection token and initialized at startup using provideAppInitializer.

With this local database configured to log expenses, we can now focus on the AI integration. The following sections illustrate how to cache the Gemma 4 weights and leverage Angular 22's reactive features to generate secure, local insights directly from this transaction data.

4. Gemma Model Caching and Engine Initialization

Running LLMs locally in the browser requires two steps:

1. Caching the Weights: Storing the model weights (~2GB) in the browser's Cache Storage so the application can run 100% offline.

2. Engine Bootstrapping: Creating a single, reusable instance of the LiteRT LM engine so we can stream responses with WebGPU speed and avoid costly re-initialization latencies.

4.1 Caching Gemma 4 to Cache Storage in the Browser

When discussing on-device AI, people often assume offline operation, zero server dependency, and total privacy. However, this is only partially true for Web AI. Most tutorials make a fetch request to retrieve the Gemma model from HuggingFace, which works well when online. This fails if the user's connection is unstable or unavailable.

This illustrates the benefit of caching a small Gemma model in Cache Storage. The device only needs to be online once to download the model, and the application runs 100% offline.

The demo has a download button for users to download the Gemma 4 E2B model that is around 2GB.

After a successful download, go to Application > Storage > Usage to verify that the model occupies 2GB in storage.

Let's implement this logic in AiModelCacheService:

export const AI_CACHE_NAME = 'JMWebAIModels';
export const DEFAULT_MODEL_FILENAME = 'gemma-4-E2B-it-web.litertlm';
export const GEMMA_MODEL_URL = `https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/${DEFAULT_MODEL_FILENAME}`;

@Service()
export class AiModelCacheService {
  readonly #modelUrl = GEMMA_MODEL_URL;
  readonly #state = signal({ status: 'not-downloaded' });

  async downloadModel(): Promise<string> {
    if (this.#state().status === 'cached') {
      return await this.getModelUrl();
    }

    this.#state.set({ status: 'downloading' });
    const localBlobUrl = await FileProxyCache.loadFromURL(this.#modelUrl);
    this.#state.set({ status: 'cached' });

    return localBlobUrl;
  }

  async getModelUrl(): Promise<string | null> {
    return await FileProxyCache.loadFromURL(this.#modelUrl);      
  }
}

The AiModelCacheService class leverages FileProxyCache (an open-source utility by Jason Mayes designed specifically for caching large Web AI weights). Instead of using a standard browser fetch (which can crash or fail with large ~2GB assets), FileProxyCache handles chunked downloads and saves the Gemma model directly to the browser's Cache Storage. In subsequent runs, it instantly loads the cached local file and passes its blob URL to the LiteRT engine, completing the offline capability.

4.2. Lazy-Initializing the LiteRT LM Engine

The application has a single instance of the LiteRT LM engine. Initializing and destroying an engine for an individual query is an expensive operation. This introduces latency while waiting for the engine to become available, start a new conversation, and accept a message.

We lazily initialize the engine using GemmaEngineService, which we then inject into our insight service in the next section.

@Service()
export class GemmaEngineService {
  readonly #cacheService = inject(AiModelCacheService);
  #engine: Engine | null = null;

  private async initializeEngine(): Promise<Engine> {
    const localBlobUrl = await this.#cacheService.getModelUrl();
    const instance = await Engine.create({
      model: localBlobUrl,
      mainExecutorSettings: {
        maxNumTokens: 4096,
      },
    });

    this.#engine = instance;
    return instance;
  }

  getEngine(): Promise<Engine> {
    if (this.#engine) {
      return Promise.resolve(this.#engine);
    }

    return this.initializeEngine();
  }

  ... other methods and lifecycle methods ...
}

If initialized, the method returns the engine immediately. Otherwise, the initializeEngine helper method loads the model from Cache Storage, constructs the LiteRT LM engine, and returns it.

5. Insight Service Definition in Angular

We define the system, priming, and user prompts to analyze and derive deep insights into personal finance and spending. You can view our exact prompts here.

To ensure high-quality, structured output from Gemma 4, the model must return a JSON response adhering to the InsightsResponse interface. You can view the response and the Insight interface here.

Let's implement our InsightService to coordinate the streaming process:

@Service()
export class InsightService {
  readonly #engineService = inject(GemmaEngineService);

  #conversation: Conversation | null = null;
  #lastPrimedExpenses: Expense[] | null = null;
  #turnsCount = 0;
  #previousQueries: string[] = [];

  async *streamInsights(userQuery: string, expenses: Expense[]): AsyncGenerator<InsightsResponse> {

    const isContextDifferent = this.#lastPrimedExpenses !== expenses;
    const isContextExhausted = this.#turnsCount >= 3;

    // Reset & prime context when threshold is met
    if (!this.#conversation || isContextDifferent || isContextExhausted) {
      await this.primeContext(expenses, isContextExhausted ? this.#previousQueries : undefined);
    }

    const userPrompt = INSIGHTS_USER_PROMPT(userQuery);
    const stream = await this.#conversation.sendMessageStreaming(userPrompt);

    this.#previousQueries.push(userQuery);
    this.#turnsCount = this.#turnsCount + 1;

    // Yield repaired, streamed JSON chunks (using jsonrepair)       
    yield* this.processStream(stream);
  }

  /* processStream, lifecycle, and other helper methods */
}

Our primeContext method formats transaction data, computes high-level spending statistics, and appends them to the query context—guaranteeing that Gemma 4 generates a reliable, grounded JSON response.

The processStream method returns an asynchronous generator that iterates over the stream to extract each chunk's content. Because LLM response streaming delivers data fragment-by-fragment, the raw buffer is frequently incomplete (e.g., an open brace or unclosed bracket). Attempting to run standard JSON.parse() on these partial chunks would immediately throw a syntax error. To solve this, the jsonrepair library dynamically patches and closes the partial JSON string on the fly, allowing the application to successfully parse and render UI updates incrementally.

6. Reactive UI Construction in Angular

6.1 History Insights Facade

HistoryInsightsService acts as a clean facade between our UI component and the core InsightService, letting us easily query expenses and stream insights based on user questions.

Our component injects this service to trigger and render the JSON response asynchronously.

Source code of HistoryInsightsService

6.2 History Insights Component

HistoryInsightsComponent comprises child components and a pipe. However, the primary method that communicates with Gemma 4 to generate a list of insights is onAskGemma.

@Component({
  selector: 'app-history-insights',
  ...
})
export default class HistoryInsightsComponent {
  protected readonly vm = inject(HistoryInsightsService);

  readonly expenses = signal<Expense[]>([]);
  readonly streamingResponse = signal<InsightsResponse>({ insights: [] });

  async onAskGemma(query: string): Promise<void> {
    const trimmed = query.trim();
    if (!trimmed) {
      return;
    }

    this.streamingResponse.set({ insights: [] });

    // The expenses are populated after a search query
    // Consume the stream generator and update the UI signal incrementally
    const generator = this.vm.streamInsights(trimmed, this.expenses());
    for await (const response of generator) {
      this.streamingResponse.set(response);
    }
  }
}

The onAskGemma event handler invokes HistoryInsightsService's streamInsights method to retrieve the streamed response and incrementally updates the streamingResponse signal. The derived aiState signal computes the current status, captures any errors, and binds the values directly to the child component to render the streamed response in real-time.

The full listing of HistoryInsightsComponent

7. Conclusion

This concludes the journey of building an on-device personal finance assistant using the LiteRT LM Web SDK, the Gemma 4 E2B small-scale model, and Angular. This blog post shows how to build an offline RAG-based application by caching the model weights locally. Even when Wi-Fi is turned off, the personal finance assistant provides insights for user queries.

We mitigated the lack of tool-calling capability in the API and the mathematical hallucinations of small models by providing precomputed aggregated data as the LLM context. This prevents incorrect "60 + 40.5 = 200.2" expressions in the final results.

Engineers can build web applications with local models to perform financial analysis without leaking personal data to the training datasets of AI vendors.

Resources

• On-device Personal Finance Assistant
• GitHub Page
• LiteRT LM Web API
• HuggingFace LiteRT Gemma 4 E2B Model
• FileProxyCache GitHub
• Angular Documentation