This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

JaExC (Japanese Exercise Creator) is a Ruby gem that writes kanji-reading exercises with a self-hosted open-weight model, and then refuses to trust it.

My partner teaches Japanese. She asked for example sentences using specific kanji, which inspired a larger teaching project. That project predates this challenge. JaExC is a new standalone gem I built for the challenge to generate exercises.

The potential customers are Japanese teachers who repeatedly prepare exercises for students with different kanji knowledge. A teaching app built around JaExC could charge for reducing that preparation work. I have not tested willingness to pay yet.

A request might be: "Use these allowed kanji, practise 食 飲 行 見 来, and give me ten N5 sentences." JaExC returns candidates with kana readings and English and Polish translations. Here is one recorded example:

1. 毎朝水を飲みます。
   reading: まいあさみずをのみます。
   en: I drink water every morning.
   pl: Piję wodę każdego ranka.

The caller supplies the target and allowed-kanji lists. Ruby checks the actual Japanese text, not the model's declaration of what it used. The JLPT level guides generation; it is not a validated difficulty rating.

JaExC generates sentences or dictionary words without requiring Rails or a database. A teaching app can use the library to prepare candidates for review.

Demo

https://asciinema.org/a/aXq65KlZEa8knIB7

The recording shows correction attempts followed by a result with reading warnings. The CLI uses the same public Ruby API as a consuming application.

bin/jaexc-demo --type kanji_reading --target 食,飲,行,見,来 \
  --allowed 食,飲,行,見,来,茶,水,友,毎,日,人,何,今,朝 \
  --level N5 --count 10 --locales en,pl --corrections 2 --check-readings

In Ruby, the same request looks like this. Here, provider is the configured Ollama adapter and analyzer is the optional Sudachi adapter described in the README.

creator = JaExC::Creator.new(
  provider: provider, analyzer: analyzer, corrections: 2
)

result = creator.generate(
  exercise_type: :kanji_reading,
  target_kanji: %w[食 飲 行 見 来],
  allowed_kanji: %w[食 飲 行 見 来 茶 水 友 毎 日 人 何 今 朝],
  level: :N5,
  count: 10,
  translation_locales: [:en, :pl]
)

result.exercises
result.attempts
result.warnings

attempts counts model calls, including corrections. warnings contains reading differences; omit analyzer: to skip the reading check.

Code

gem "jaexc", git: "https://github.com/DominikAlberski/JaExC"

ollama pull <model>

MIT License. The README includes setup instructions, the public API and a summary of the recorded correction results.

How I Built It

I chose local inference to run generation on my own hardware, with electricity costs and no per-token API charges. Using an open model was also part of the challenge.

I tried a dense Qwen 27B model first. It exceeded my GPU's 16 GB of VRAM. With thinking enabled, it took 3 minutes 21 seconds to generate three sentences. I switched to gemma4:26b-a4b-it-qat, a mixture-of-experts model with about 4B active parameters per token. I also preferred Gemma's output in the comparison samples.

After I disabled thinking, Gemma generated ten sentences in 7–14 seconds on that machine. The requests and thinking settings differed, so these timings do not isolate the effect of model architecture. That final Gemma setup made local generation practical for this project.

JaExC::Creator talks to the model through RubyLLM with a JSON schema:

request validation → prompt → local model → JSON parse → rule validation
        → (corrections) → optional reading check → Result

When thinking ate the answer

Some early calls failed with "not valid JSON". One returned an empty answer. Replaying the requests against Ollama and inspecting the raw responses showed reasoning in a separate field. In the failed calls, the prompt and generated reasoning exhausted the 16,384-token context before the model could complete the JSON.

In the replay investigation, I made ten calls per setting, split equally between Ollama's native and OpenAI-compatible endpoints. With thinking on, seven responses were empty and one contained truncated JSON. With thinking off, all ten returned complete JSON. Calls took about seven seconds with thinking off, compared with 120–136 seconds with thinking on: roughly 20 times longer.

The adapter now disables thinking by default and distinguishes truncated output from malformed JSON.

Check the text, then ask for a correction

One early batch contained sentences like 私 は 水 を 飲 む, with spaces throughout the Japanese text. The validator accepted it at the time. I added a whitespace rule; the next spaced batch was rejected. I also updated the prompt to forbid spaces.

Ruby checks the kanji in the actual jp text against the allowed list and verifies the declared targets. It also rejects duplicates and readings containing kanji, whitespace, Latin letters or digits. The model doesn't get to mark its own answer as correct.

Prompt reminders had mixed results in the small runs I recorded. I added an optional correction loop. It sends the original instructions, the previous answer and the specific rules it violated back to the model. With corrections: 2, JaExC allows an initial attempt and up to two corrections, validating the entire batch each time.

Without corrections, the same request passed structural validation in three of six trials. With corrections: 2, all six passed in a later series. Five needed corrections. The average was 2.0 model calls per request, with total times from 8 to 25 seconds. These are small samples, and some passing batches still had reading warnings.

Only exercise-rule violations trigger corrections. Invalid JSON, truncated output, provider failures and reading warnings do not. Automatic transport retries remain disabled.

A dictionary can disagree too

The optional Sudachi analyzer gives a second reading to compare with the model's. It caught errors that passed the structural rules:

In the recorded sentence 何を食べますか, Sudachi returned nan-o for 何を; the expected reading was nani-o. A reading difference therefore produces a warning for the teacher. It does not reject the exercise or trigger a correction.

What still needs a teacher

One sentence passed the checks: 友だちを食べに来ます. Its translation said "A friend is coming to eat." The Japanese says someone is coming to eat the friend. Another passing sentence asked 水を食べますか, "Do you eat water?" That's nonsense, unless you are an alien visiting Earth.

JaExC enforces the implemented structural rules and can flag reading differences. It does not certify meaning, naturalness or translation quality.

I used AI assistance for development and for editing this post. I made the architecture and interface decisions; an AI coding agent implemented the agreed components with test-first development.

Why Does Open Innovation Matter?

Running my own Ollama server let me inspect raw responses and their reasoning fields when an answer came back empty. That exposed the context-exhaustion problem and gave me evidence for changing the adapter.

In the demonstrated configuration, generation and the optional dictionary check run on my own hardware. JaExC does not log prompts or answers. The prompt and validation code are open for other developers to inspect. The provider adapter lets me change the configured model without rewriting the exercise API.

For this challenge, I built an exercise generator with a working demo and a correction loop. It also produced some memorable bad Japanese. Teacher review remains part of using it.

Prize Categories

Best Use of Gemma. The demonstrated setup uses Gemma 4 26B-A4B through Ollama for practical local generation on my machine with one 16 GB GPU. I also preferred its word variety: Gemma included 本 and 日曜日, while Qwen returned four 日本 compounds in the comparison sample.