Why I built another Ruby test runner inspired by Playwright Test

Ruby already has great testing tools.

If you are building Rails applications today, you probably use one of these combinations:

• RSpec + Capybara
• Minitest + Capybara
• Rails system tests
• Maybe Selenium, Cuprite, Ferrum, or Playwright through Ruby bindings

These tools are mature, battle-tested, and widely used.

So the natural question is:

Why create yet another test runner?

That is exactly the question the Playwright team asked themselves when they created Playwright Test.

In the Playwright video that inspired this project, the speaker says they did not originally want to create another JavaScript testing framework. They tried existing frameworks such as Mocha and Jest, but those frameworks had historically been designed for unit testing. End-to-end testing had a different set of problems: cross-browser execution, parallelization, browser-side isolation, and a high degree of fixture flexibility.

That explanation clicked with me.

Because Ruby on Rails browser testing has a similar problem.

RSpec and Minitest are excellent general-purpose Ruby test frameworks. Capybara is an excellent browser automation DSL. But the current mainstream Rails browser testing style was not designed after learning from Playwright Test. It still feels like a unit-test-era architecture with browser automation added on top.

Playwright Test showed that browser testing deserves a test runner designed around browser testing itself.

That is the idea behind Smartest.

Repository:

class WebFixture < Smartest::Fixture
  fixture :server do
    server = TestServer.start
    on_teardown { server.stop }
    server
  end

  fixture :client do |server:|
    Client.new(base_url: server.url)
  end
end

around_suite do |suite|
  use_fixture WebFixture
  suite.run
end

test("GET /health") do |client:|
  response = client.get("/health")

  expect(response.status).to eq(200)
end

Documentation: https://smartest-rb.vercel.app/
Browser testing guide: https://smartest-rb.vercel.app/docs/playwright-browser-tests

The inspiration: Playwright Test was not just another runner

Playwright Test is not only a wrapper around browser automation.

It is a test runner designed for the real constraints of end-to-end testing:

• cross-browser support out of the box
• parallel execution
• browser-context isolation
• web-first assertions
• traceability and debugging support
• fixtures inspired by pytest

The official Playwright repository describes Playwright Test as a full-featured test runner for end-to-end testing, with Chromium, Firefox, and WebKit support, full browser isolation, auto-waiting, and web-first assertions.

The important point is not simply “Playwright controls browsers.”

The important point is:

Playwright Test changed the test runner because the target domain changed.

Browser tests are not just unit tests with a browser.

They need their own setup model.

The fixture idea I wanted in Ruby

One of the best ideas in pytest is fixture injection.

In pytest, a test can request what it needs by naming it:

def test_get_me(logged_in_client):
    response = logged_in_client.get("/me")
    assert response.status_code == 200

The test body does not say how to create logged_in_client.

The fixture system knows how to resolve it.

Fixtures can depend on other fixtures:

@pytest.fixture
def logged_in_client(client, user):
    client.login(user)
    return client

This is extremely useful for browser tests, because browser tests often have layered setup:

• app server
• browser runtime
• browser process
• browser context
• page
• user
• login state
• seeded data
• teardown

RSpec and Minitest can express all of this, of course.

But the dependency graph is often implicit.

With RSpec, we may write something like this:

let(:server) { start_server }
let(:client) { Client.new(base_url: server.url) }
let(:user) { create_user }
let(:logged_in_client) do
  client.login(user)
  client
end

it "GET /me" do
  response = logged_in_client.get("/me")
  expect(response.status).to eq(200)
end

This works.

But the fixture dependency graph is hidden inside method calls.

I wanted this instead:

test("GET /me") do |logged_in_client:|
  response = logged_in_client.get("/me")

  expect(response.status).to eq(200)
end

And fixture definitions like this:

class AppFixture < Smartest::Fixture
  fixture :client do |server:|
    Client.new(base_url: server.url)
  end

  fixture :logged_in_client do |client:, user:|
    client.login(user)
    client
  end
end

The important design decision is the keyword argument.

test("GET /me") do |logged_in_client:|

This makes the test’s dependencies explicit.

And:

fixture :logged_in_client do |client:, user:|

This makes the fixture’s dependencies explicit.

No positional argument order.
No hidden let dependency.
No separate with: [:server] declaration.

The dependency declaration and usage live in one place: the Ruby method signature.

Smartest: a small keyword-fixture-first Ruby test runner

Smartest is a small Ruby test runner with a keyword-fixture-first design.

The goal is not to replace all Ruby testing immediately.

The goal is to explore what Ruby testing could feel like if we applied the lessons of pytest fixtures and Playwright Test to Ruby browser testing.

A normal Smartest test looks like this:

test("factorial") do
  expect(1 * 2 * 3).to eq(6)
end

A fixture-driven test looks like this:

test("GET /me") do |logged_in_client:|
  response = logged_in_client.get("/me")

  expect(response.status).to eq(200)
end

Smartest is designed around three ideas:

1. Tests should be readable at the top level.
2. Fixture dependencies should be explicit.
3. Teardown should be written only when it is needed.

That third point is especially important for browser tests.

Some fixtures are just values.
Some fixtures need teardown.
Some fixtures should live for one test.
Some fixtures should live for the whole suite.

Smartest supports both regular fixtures and suite fixtures:

class PlaywrightFixture < Smartest::Fixture
  suite_fixture :playwright do
    runtime = Playwright.create
    on_teardown { runtime.stop }
    runtime
  end

  suite_fixture :browser do |playwright:|
    browser = playwright.chromium.launch
    on_teardown { browser.close }
    browser
  end

  fixture :page do |browser:|
    context = browser.new_context
    page = context.new_page

    on_teardown { context.close }

    page
  end
end

This expresses the browser lifecycle directly:

• start Playwright once
• launch a browser once
• create a fresh browser context and page per test
• close the context after each test
• close the browser after the suite
• stop Playwright after the suite

That is the kind of lifecycle that is awkward when browser testing is treated as a thin layer on top of a unit testing framework.

Why this matters for Rails browser testing

Rails browser testing has traditionally been shaped by RSpec/Minitest + Capybara.

That combination is powerful, but it also reflects an older mental model:

• test framework first
• browser automation second
• browser lifecycle hidden in configuration
• setup split across before, let, support files, shared contexts, and driver config

Playwright Test takes a different position:

• browser testing is the domain
• isolation is part of the runner
• fixture composition is part of the runner
• cross-browser execution is part of the runner

Smartest is an attempt to bring that kind of thinking to Ruby.

Not by copying Playwright Test’s TypeScript API exactly.

But by asking:

What would a Ruby-native pytest-style fixture system for browser testing look like?

The current answer is:

test("finds the smartest gem on RubyGems") do |page:|
  page.goto("https://rubygems.org/")
  page.locator("input[name='query']").fill("smartest")
  page.keyboard.press("Enter")
  page.locator("a[href='/gems/smartest']").click

  expect(page).to have_url("https://rubygems.org/gems/smartest")
  expect(page.locator("h1")).to have_text("smartest")
end

That page: fixture is the key.

It looks small.

But behind it, Smartest can manage the Playwright runtime, browser process, browser context, page creation, and teardown.

Getting started with browser tests

Smartest includes a browser-test scaffold.

Add Smartest to your project:

gem "smartest"

Then run:

bundle install
bundle exec smartest --init-browser

The browser scaffold sets up the files needed to run Playwright browser tests from Ruby.

It creates files such as:

smartest/test_helper.rb
smartest/fixtures/playwright_fixture.rb
smartest/matchers/playwright_matcher.rb
smartest/example_browser_test.rb

It also installs the browser testing dependencies:

• playwright-ruby-client
• the Playwright Node.js package
• browser binaries

Then run the generated example:

bundle exec smartest smartest/example_browser_test.rb

You can also choose a browser with environment variables:

BROWSER=firefox bundle exec smartest smartest/example_browser_test.rb

Or watch the browser while the test runs:

HEADLESS=false SLOW_MO=250 bundle exec smartest smartest/example_browser_test.rb

The generated fixture uses this lifecycle:

suite_fixture :playwright
  -> starts Playwright runtime once

suite_fixture :browser
  -> launches one browser process for the suite

fixture :page
  -> creates a fresh browser context and page per test

This is intentionally close to the Playwright Test mental model.

AI changed the feasibility of this project

There is another reason I built this now.

A few years ago, building a new test runner would have felt expensive.

You would need to design the DSL, implement the runner, handle fixture resolution, write CLI behavior, create matchers, generate scaffolding, document the behavior, and iterate for a long time.

But with OpenAI Codex, the first working version was surprisingly quick to build.

That changes the economics of experimentation.

I do not mean that AI magically makes design decisions for us.

The important design still comes from human taste:

• Should fixtures be positional or keyword-based?
• Should fixture dependencies be declared separately or via block parameters?
• Should teardown be expressed with yield, ensure, or on_teardown?
• Should browser support be built in or generated?
• Should this coexist with Minitest and RSpec?

But once the design direction is clear, AI makes it much easier to produce a working prototype and iterate.

In this case, the ideal implementation became possible much faster than I expected.

That matters for open source.

Because many useful developer tools do not start as huge polished projects.

They start as a small experiment with a strong point of view.

This is not “RSpec is bad”

I want to be clear.

RSpec is not bad.
Minitest is not bad.
Capybara is not bad.

They are important parts of the Ruby ecosystem.

But Playwright Test showed something valuable:

Sometimes a new testing domain deserves a new test runner.

Modern browser testing is one of those domains.

If JavaScript testing needed Playwright Test even though Mocha and Jest already existed, then Ruby can also explore a browser-testing-oriented runner even though RSpec and Minitest already exist.

The question is not:

Does Ruby already have testing frameworks?

Of course it does.

The better question is:

Does Ruby have a test runner designed around pytest-style fixture composition and Playwright-style browser lifecycle management?

That is the space Smartest explores.

Current status

Smartest is still young.

It is best understood as a design-stage test runner and a working experiment.

The current focus is:

• top-level test
• keyword-argument fixture injection
• fixture dependencies through keyword arguments
• fixture teardown
• suite fixtures
• around-suite and around-test hooks
• browser scaffold generation
• Playwright fixture and matcher examples

It is already enough to write Playwright-style browser tests in Ruby.

But it is not yet a mature replacement for RSpec or Minitest.

That is fine.

The goal right now is to validate the idea:

pytest-style fixtures make Ruby browser tests smarter.

Let's Try it!

Install:

gem "smartest"

Initialize browser tests:

bundle exec smartest --init-browser

Run:

bundle exec smartest smartest/example_browser_test.rb

Write:

test("my browser test") do |page:|
  page.goto("https://example.com")

  expect(page.locator("h1")).to have_text("Example Domain")
end

Closing thought

Playwright Test was created because existing JavaScript test frameworks were not built for the unique challenges of end-to-end browser testing.

That lesson should not stay only in the JavaScript world.

Ruby browser testing can also learn from it.

Smartest is my attempt to bring pytest-style fixtures and Playwright-style browser testing ergonomics into Ruby.

Smarter browser testing starts with smarter setup.