Using Playwright projects and Page extensions to run tests on multiple testing environments

The setup

We are a team of 8 and our main focus in the past years has been a multiplatform SDK developed in Kotlin Multiplatform (KMP) and a huge product with a complex architecture: Spring boot backend services, a client written in Vue.js, with all the operational tasks tied to them (eg. dependency updates, databases, service operations, on-call, and so on). I have been a member since 2021. Over the years I picked up an affinity towards frontend development and became a certified Vue.js developer. Then the decision was made to take over all of our QA responsibilities as well. Since running QA pipelines with end-to-end (e2e) tests was never in the team's scope (we have our integration and e2e tests of course, but these are not meant to serve as QA tests; also we have a dedicated QA team), and realizing that we have no measurable knowledge about the stack in which the current workflows operate (not to mention all the other stacks we have to use in our daily work), we came up with the idea to write our own e2e tests in a language and on a platform we are familiar with, then move the whole infrastructure to GitHub. As the team's frontend guy, I was asked to design the new architecture for the e2e tests of the aforementioned client. In this article I will give an overview of our goals, ideas, and solution to achieve a fast running e2e test collection that can double as UI tests for the client itself.

The goal and the challenge

With the e2e testing the following goals were set:

• stable, easily maintainable, reliable tests
• reuse part of the tests for UI testing in the client repository itself
• run the pipelines on GitHub (every other CI/CD pipeline is already there)
• make the test configurable for multiple environments (different setup, both for staging and production)
• pass the sensitive information to the tests in a secure way (eg. login information)

The client had its tests written using Cypress, and the UI tests were failing on every other run. The tests were coded hastily, with little regard to clean code principles, and every time a fix was applied, some other stuff broke. When new features were added to the client, parts of the existing test code were just copied over to provide test coverage. So in the light of the code quality, instability and the new task ahead we gave up the patching, and decided to do a whole makeover, rewriting every test case (and deleting the redundant and meaningless ones) in Playwright. We also did a huge cleanup in the test data sources by creating dedicated files and moving everything we needed into one place. The conversion went slowly, the new framework had a very different approach in interacting with components, but finally we did it. Migration done, pipeline green, job well done. Almost...during the code review we found patterns we were not happy about.

Mock service

In the new solution the mocking of the API calls was done like this:

async function mockGetListOfData(
  page: Page,
  mockResponse: SomeData[] = testData,
) {
  await page.route("**/api/v1/somedata", async (route) => {
    await route.fulfill({
      status: 200,
      contentType: "application/json",
      body: JSON.stringify(mockResponse),
    });
  });
}

Note: This is an example showing how to intercept API calls on the page in Playwright. A mocking function like this can be extended further of course, to make it return failures, different status codes and so on.

Reading through the test files we realized that every test.beforeAll() started with the same statements:

await mockApiCall1(page, testData);
await mockApiCall2(page, otherData);
await mockApiCall3(page, testError);
await mockApiCall4(page, testOtherData);
await page.goto("/");
await expect(page.getByRole("heading")).toHaveText("Heading");

More troubling was the fact that most of these methods have parameters with at most two different values: testData or []. Each and every interception requires the page as a parameter. The returned response and other metadata is also required. And are repeated at every turn. At this point people start to think:

Can we extract this somehow?

Sure you can!

Move them in a single file, wrap them in a function and call it setupApiWithResponses(page, testData, ...) and use this new function in the test.beforeAll(). Yeah, that hides all the mockApi calls, but it still must be repeated everywhere. In addition the navigation is still not solved. Those lines can be added too, but then the setup has a side effect. You remember it now, but will you remember it one year from now? Let's keep thinking!

It makes sense to create a dummy or mock service to represent the backend in cases where we do not need or have access to the real one. You either can't run it locally, or it is simply not needed for the tests. Even in case of the staging environment, you don't want to pollute everything with mountains of useless test data, right? So let's do it, create a mock service. It is a simple class, like this:

class MockService {
  private page?: Page;

  usePage(page: Page) {
    this.page = page;

    return this;
  }

  async initialize(testData: TestData) {
    if (!this.page) {
      throw new Error("Page not found! Call usePage before initialize!");
    }

    await mockApiCall1(this.page, testData.response1);
    await mockApiCall2(this.page, testData.otherData);
    await mockApiCall3(this.page, testData.testError);
    await mockApiCall4(this.page, testData.testOtherData);
  }
}

Looking good. We can even add functions to register and change the test data on the fly.

class MockService {
  private page?: Page;
  private scenario?: TestScenario;
  private api1?: ApiSetup;
  private api2?: ApiSetup;
  private api3?: ApiSetup;
  private api4?: ApiSetup;

  usePage(page: Page) {
    this.page = page;

    return this;
  }

  useScenario(scenario: TestScenario) {
    this.scenario = scenario;
    this.api1 = scenario.api1;
    this.api2 = scenario.api2;
    this.api3 = scenario.api3;
    this.api4 = scenario.api4;

    return this;
  }

  async withApi2Setup(api2Setup: Partial<Api2Setup>) {
    assertDefined(this.api2);
    this.api2 = {
      ...this.api2,
      ...(api2Setup.response && {
        response: api2Setup.response,
      }),
    };

    await this.initialize();
    return this;
  }

  async initialize() {
    if (!this.page) {
      throw new Error("Page not found! Call usePage before initialize!");
    }
    if (!this.scenario) {
      throw new Error(
        "Scenario not found! Call useScenario before initialize!",
      );
    }

    await mockApiCall1(this.page, this.scenario.api1.successResponse);
    await mockApiCall2(this.page, this.scenario.api2.response);
    await mockApiCall3(this.page, this.scenario.api3.errorResponse);
    await mockApiCall4(this.page, this.scenario.api4.emptyResponse);
  }
}

Notes:

• many of our intercepted API paths contain path parameters, like IDs, and with the route matching displayed above, all interceptors need to be re-registered if the test data changes, because the route itself will change; this is the reason for the repeated initialize() calls upon partial data overrides
• the previous issue could be solved by using less strict route matching, but then the callback needs to handle the different paths, and it can grow into a large tree of conditions pretty fast
• registering all the route interceptors can look dangerous, but in our setup every test receives a new Page and MockService instance, and Playwright by design drops every registered handler with the page after the test concludes, so no memory leaks occur
• assertDefined() is a small function that asserts that the passed parameter is defined and not null; it is used to prevent the user from overriding non-optional fields with undefined or null values; it is forced by the Partial type we use to make the method more comfortable to use
• if you return this from the use methods that help you configure the service, you will be able to chain your calls:

  const mockService = new MockService();
  await mockService.usePage(page).useScenario(defaultScenario).initialize();

So the MockService is in place. How to get it to the tests? It would be so nice to have this service available in every test, ready to be configured and used. It could be added to the init, setup, initialize steps to the test.beforeAll(), but this would not be much better than the original solution. Also it would mean that the tests would share the MockService instance, so the test isolation principle would be damaged. No, a new instance is needed in every test. To achieve this, we needed to extend the test exposed by the Playwright package.

Base test

Playwright provides a pretty flexible infrastructure that can be extended, configured, overwritten in many ways. The test is no different. Let's see if there is a way to add a MockService instance to every test. The extension we want to add as an option to the test will contain a mockService property. Since we are working in typescript, let's create a simple interface for it:

interface CustomTestOptions {
  mockService: MockService;
}

Since we want to create a new, improved version of test, let's create a dedicated file, e.g. base-test.ts for it, and add the extension:

import { test as base } from "@playwright/test";

export const test = base.extend<CustomTestOptions>({
  mockService: async ({}, use) => {
    const mockService = new MockService();
    await use(mockService);
  },

  page: async ({ page }, use) => {
    await use(page);
  },
});

After adding this extension the mockService will be available in your tests.

import { test } from "./base-test.ts"; //<- you need to import the extension here!!!

test("mockService is available", async ({ page, mockService }) => {
  await mockService.usePage(page).useScenario(testScenario).initialize();

  await page.goto("/");
  await expect(page.getByRole("heading")).toHaveText("Heading");
});

Nice and clean, but it still needs to be repeated every turn. Can we move the initialize() call into the extension? Of course!

//base-test.ts

import { test as base } from "@playwright/test";
import { defaultScenario } from "my-test-source.ts";

export const test = base.extend<CustomTestOptions>({
  mockService: async ({}, use) => {
    const mockService = new MockService();
    await use(mockService);
  },

  page: async ({ page, mockService }, use) => {
    await mockService.usePage(page).useScenario(defaultScenario).initialize();

    await use(page);
  },
});

Marvellous! But the test data is hard-coded...can it be made smarter? As it was mentioned before, the client basically has two distinct start states: empty and populated. The same setup is used for starting location and mocked APIs, the only difference is in the data. Basically two pages are required:

interface CustomTestOptions {
  mockService: MockService;
  emptyPage: Page;
  populatedPage: Page;
}

And in the extension:

//base-test.ts

import { test as base } from "@playwright/test";
import { defaultScenario, emptyScenario } from "my-test-source.ts";

export const test = base.extend<CustomTestOptions>({
  mockService: async ({}, use) => {
    const mockService = new MockService();
    await use(mockService);
  },

  page: async ({ page }, use) => {
    await use(page);
  },
  emptyPage: async ({ page, mockService }, use) => {
    await mockService.usePage(page).useScenario(emptyScenario).initialize();

    await use(page);
  },
  populatedPage: async ({ page, mockService }, use) => {
    await mockService.usePage(page).useScenario(defaultScenario).initialize();

    await use(page);
  },
});

And suddenly in our tests we have access to each of these pages (with the mockService as well):

import { test } from "./base-test.ts"; //<- you need to import the extension here!!!

test("different pages available", async ({
  page,
  emptyPage,
  populatedPage,
  mockService,
}) => {
  // test content here...
});

At this point the test data is available as an input parameter in our extension, and we get all the starting setups initialized with all the mocked endpoints with a single source of test data for each individual test. Moreover every test will have its own instance of the MockService and Page providing the isolation desired.

Starting location

Since the UI tests should mimic the user's journey on our page it became obvious that the initial navigation could be added to the base-test. This will make sure that the UI loads at the home page and instead of navigating with direct URLs, during the tests the clicks will be performed on every button to get to the destination we need. This approach provides much more lifelike behavior, adds useful constraints, and improves maintainability. To have this initial navigation, we simply added the page.goto() call to the newly defined emptyPage and populatedPage fields as well:

//base-test.ts

import { test as base } from "@playwright/test";
import { defaultScenario, emptyScenario } from "my-test-source.ts";

export const test = base.extend<CustomTestOptions>({
  mockService: async ({}, use) => {
    const mockService = new MockService();
    await use(mockService);
  },

  page: async ({ page }, use) => {
    await page.goto("/");
    await expect(page.locator("h1")).toHaveText("Client header");
    await use(page);
  },
  emptyPage: async ({ page, mockService }, use) => {
    await mockService.usePage(page).useScenario(emptyScenario).initialize();
    await page.goto("/");
    await expect(page.locator("h1")).toHaveText("Client header");

    await use(page);
  },
  populatedPage: async ({ page, mockService }, use) => {
    await mockService.usePage(page).useScenario(defaultScenario).initialize();
    await page.goto("/");
    await expect(page.locator("h1")).toHaveText("Client header");

    await use(page);
  },
});

Note: The starting location of every page can be set up separately of course, to arbitrary destinations as needed. We also inserted the simple assertion to verify that the page is loaded and ready to go. The toHaveText() method will wait for a predefined timeout duration without the need of extra delays. It completes as soon as the expectation is fulfilled.

The road so far

After completing the steps described above we were quite happy. The test code became pretty, clean, readable and much shorter. Some could argue that the test's given section is hidden this way, and one needs to know the actual setup to understand the test cases better. This is true, but in return the tests are more on point, more focused since less code is needed to actually test what you want. Basically most tests can be started at the when part. One can also counter this by giving meaningful and descriptive names to test methods. If a very specific edge case must be tested, the mockservice's response on a specific endpoint can still be altered and tailored to the requirements. then... happy as we were we turned on the end-to-end tests on the CI/CD pipeline again, pushed the changes, and to our not that great surprise (can't say as expected) half of them broke. We wrote our tests from scratch, keeping the assertions, and every step the old ones had, and we knew the UI is actually working, (no alerts, no angry customers). So what was the problem? -It must be in the setup. It was.

The environment and the window object

The UI we built is living as part of a much larger and very complex page. The host is rendered first then all the navigation links load the associated pages when needed (the user navigates to the said page). In addition, because of security reasons and industry standards, the host page provides all of the context for the sub-pages to use. Including tokens, language settings, available features, restrictions, flippers and so on. These are mostly registered as properties or methods on the global window object itself. We had to include some of these fields and callbacks in our index.html file to make our UI work in the first place. It was easy, just a few script tags and a globally defined type. Tokens and secrets are not hard-coded of course, a .env file is used to insert the values with the help of vite, but it is a different story. When a new edition of our product was released it came with new features that were not included in the old one. At this point first for development, and later for local testing several new variants of index.html (e.g. index-with-new-feature.html and index-feature-disabled.html) were created. These had different setups through the script tags, registering their own overrides on the window object. This could be done easily, resulting in four different test HTMLs, each with their own scripts to be launched with. Also these are included in the new playwright tests as landing pages. Some examples:

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <!--  component links, stylesheets, metadata-->
    <script>
      window.path.getConfig = async () => ({
        id: 1234,
        timezone: 'Europe/Athens',
        ...
      })
      window.path.getToken = async () => `%VITE_TOKEN_1`
      window.path.hasFeature = () => Promise.resolve(false)
    </script>
    <title>Awesome Client</title>
  </head>
  <body>
    <div id="app"></div>
    <script type="module" src="/src/main.ts"></script>
  </body>
</html>

<!-- index-with-new-feature.html -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <!--  component links, stylesheets, metadata-->
    <script>
      window.path.new.getConfig = async () => ({
        id: 1234,
        timezone: 'Europe/Athens',
        ...
      })
      window.path.getToken = async () => `%VITE_TOKEN_WITH_MORE_INFO`
      window.path.hasFeature = () => Promise.resolve(true)
    </script>
    <title>Awesome Client</title>
  </head>
  <body>
    <div id="app"></div>
    <script type="module" src="/src/main.ts"></script>
  </body>
</html>

Sounds perfect. But it did not work.

We test on production

Although it seems counter-intuitive, this statement has merits. We always ran our UI tests on production builds, because that is the build we ship. Our consumers (the company's host page) only needs the app.js from the build, because as it was discussed before, the host provides the place for it to render, so the index.html is not needed. The tests are a completely different story though, as they need the index.html variants to provide context. Why did it break then? We had all the index.html variants, we defined the locations, navigated to the right one when needed. Indeed, but the production build only includes the index.html file. The others are ignored and omitted. This causes 404 errors on the pipeline and in turn the tests fail. Now the root cause has been found. The solution: a single index.html file must be used and its content should be changed based on the scenario and the tested environment. But how?

First idea - Override the window in the base-test

The first idea was to simply override the necessary fields and callbacks directly in the base-test. It makes complete sense, an initScript can be registered on the page that will be executed before every test run, can add the necessary information to the window and provide the context needed:

//base-test.ts

import {windowConfig} from 'my-test-source.ts'
//other setup...

page: async ({ page }, use) => {
  await page.addInitScript((config) => {
    window.path.getConfig = async () => ({
      // config fields and callbacks from the config param
      ...config
    })
  }, windowConfig);
},

//emptyPage: ...

The idea is great, but it does not work, because:

• the init script runs before the content render: The actual index.html is loaded much much later, and since it has all its <script> tags in place for local running, it will apply its own overrides again, practically negating everything done here.
• serialization: The addInitScript() method can only handle serializable params. We had our own type for the complete window configuration, it included everything (fields and callbacks) with types, some default values. It was perfect. Just create a default and pass it as the override, and it's done, right? No, you can't do that. Callbacks are not serializable.
• not customizable: With this approach only a single variant can be created for all tests. Although one could create a separate page variant for each setup, it wouldn't scale in case more features or more editions are added later, not to mention edge cases.

Second idea - Use serializable data, override in index.html

Facing the challenges of the first idea, we looked into possible solutions for the problems, as the base concept looked viable. The use of serializable data solved one of the three problems. Let's create a defaultWindowConfig object with the override values needed in the addInitScript(). Still the problem of early running exists. Solving this requires the actual overrides to be made in the index.html itself. The addInitScript is used to put the data on the window object only.

//base-test.ts

//...

await page.addInitScript((config) => {
  window.testData = { config: config };
}, windowConfig);

//...

And in the index.html:

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <!--  component links, stylesheets, metadata-->
    <script>
      if (window.testData?.config) {
        window.path.getConfig = async () => window.testData.config.configOverride
        window.path.getToken = async () => window.testData.config.tokenOverride
        window.path.hasFeature = () => Promise.resolve(window.testData.config.hasFeature)
      } else {
        window.path.getConfig = async () => ({
          id: 1234,
          timezone: 'Europe/Athens',
          ...
        })
        window.path.getToken = async () => `%VITE_TOKEN_1`
        window.path.hasFeature = () => Promise.resolve(false)
      }
    </script>
    <title>Awesome Client</title>
  </head>
  <body>
    <div id="app"></div>
    <script type="module" src="/src/main.ts"></script>
  </body>
</html>

The example above solves all but one problem: it is still not configurable. Apart from this, it is clean, organized, has type support, readable, and uses a single html file. Notice that the changes were added only to the page and not the other variants. The reason is simple: the variants decorate the original page adding extra features, mocks, and so on. (Notice, the page is even included as an input parameter in the variants definition!) On this principle registering the windowConfig overrides only in this place is sufficient.
Solving the final and most crucial problem requires changes in the playwright.config file, because the utilization of projects is required.

Projects in Playwright

In the auto-generated Playwright config a section named projects can be seen. This field expects an array of project objects, with the most important properties being:

• name: the name of the project
• testMatch: a collection of patterns or paths to the test files / specs one wants to include
• use: additional custom data can be specified for the tests

Here is an example of creating an extended project configuration in TypeScript:

1. Extend the interface we created for the base test with the additional data and create a default object for easier use

   export interface CustomTestOptions {
     mockService: MockService;
     emptyPage: Page;
     populatedPage: Page;
     config: TestConfiguration; //<- add custom data here
   }

   interface Config {
    someKey: string
   }

   interface TestConfiguration {
     configOverride: Config;
     tokenOverride: string;
     hasFeature: boolean;
   }

   export const defaultTestConfiguration: TestConfiguration = {
     configOverride: { someKey: "someValue" },
     tokenOverride: "superSecretToken",
     hasFeature: true,
   };
   //TestConfigurations can have a factory or just defined as plain objects for all necessary cases, e.g. FeatureA - on, FeatureB - off, and so on.

Note: Every type must be serializable in the added custom TestConfiguration, no callbacks, no promises.

1. Extend the PlaywrightConfig type

   // playwright.config.ts

   export default defineConfig<CustomTestOptions>({
     // testDir: ...
     projects: [
       {
         name: "my special test case",
         testMatch: ["**/*.spec.ts"], //can be omitted to apply to all tests
         use: {
           config: defaultTestConfiguration //<- add the value you need; if you want to use a default, you can omit the field and set the default in the base-test.ts too, see below
         },
       },
       ... //additional projects for different features, staging, production environments
     ],
   });

1. Access the custom data in the base-test extension

The custom data can easily be accessed in the test extension, as the extended type from the base-test will be applied and we will see the properties appearing. In this section, the MockService related code parts are omitted for less noise, but those configurations are available as before.

   //base-test.ts

   export const test = base.extend<CustomTestOptions>({
     config: [defaultTestConfiguration, { option: true }],

     page: async ({ page, config }, use) => {
       await page.addInitScript((customConfig) => {
         window.testData = { config: customConfig };
       }, config);

       await use(page);
     },
   });

The syntax seems to be a bit weird, but it's pretty easy to understand:

• config: the name of the custom property added
• defaultTestConfiguration: the default value, in case the field is not set under project.use.config; that is the reason the repeated setting can be omitted in different projects
• { option: true }: this option must be added to make the value configurable; if it is missing, the value of the property will be the one defined here as the default for every project, overriding all other configurations.

We have customisable data

After the previous steps (test and configuration extension, mock service creation, project definition) were implemented, we reached a point where all the necessary mock answers, test tokens, feature overrides were easily manageable and clearly defined in a type-safe way. We can add any serializable data to the window object our tests need while maintaining complete isolation in the test cases. All test specs under the project have the same initial setup, saving us a lot of time and boilerplate code.

We have speed

Indeed, like greased lightning. Our 91 UI tests run around 2-3 minutes locally (4 minutes on CI/CD), averaging around 3.5 seconds per test case. We're happy with this. However, we noticed something interesting when we ran all the migrated tests together with our new configuration setup. Around 40 tests passed without problem, but then 4 or 5 failed. Every time, before we reached 40-41 all green, then red, more red, then after 10 seconds everything passed again. We ran them separately, all green. Ran them in different order, again, 40-41 green, then red. What could cause this? I ran the tests in ui mode and realised that our page had no styling, no custom components appeared from our company design system library. Those components are all served by CloudFlare. Could it be the CDN? Rate limiting? To verify our suspicion we added a listener to the page('request'), logging all the URLs and the response statuses. We ran the tests again, and there it was, plain as day: all URLs pointed to our sources on Cloudflare, all responding status code 429.
The root cause of this problem is that we initialize our index.html for each test. With test execution times around 2-3 seconds, running parallelly on multiple workers, this is just too fast. The CDN won't serve so many requests from the same source. We needed to implement some kind of caching to solve this problem. This part is not the focus of this article, it is mentioned only in the context of speed. To provide some guidelines on caching CDN data, here is a pseudo code part:

export const test = base.extend<CustomTestOptions>({
  //custom field declarations...,

  page: async ({ page, config }, use) => {
    //omitted initScript to reduce noise...

    await page.route(
      "<your-cdn-source.provider.url.com/**>",
      async (route) => {
        await yourCachingLogic(route);
      },
    );
  },
});

The function yourCachingLogic is up to reader to implement, keeping in mind all the requirements they need. In our case it handles these steps:

• fetches the content from the route URL
• generates unique file names based on the URL
• creates a dedicated folder and saves downloaded content there with the unique names
• can handle nested directories, if downloaded data is not a single file
• checks existing data and caches only what is missing
• fulfills the original route with the cached data; this is especially useful as the CDN URLs can be left in our index.html scripts, so it will work for both local running and testing

Additional considerations:

• having a cache in place calls for special handling and maintainers must make sure the data is not stale
• using the cache from tests adds additional speed: running on 2 workers our 91 tests complete under 2 minutes on local machines (worker count can be configured in the playwright.config.ts)
• it might be useful to create a teardown method to delete the cache after all tests are completed

Can we have more?

Of course! Imagine a scenario where you need to log in to your client running on staging environment, through the public entry point using a user name and a password to run your automated QA tests. Having a clean set of reusable UI tests freshly migrated from Cypress you write the steps to open the page, set the login data, click login, close all your fancy pop-ups, navigate to the correct subpage, and run the tests. These steps are easy to automate. All you need is the URL and the login information. The question is: How do we add the login information to the test? The answer: hard-code it or use environment variables. Let's see an example:

//playwright.config.ts
const testEnvLoginInfo = process.env.TEST_ENV_LOGIN_INFO ?? "{}";

//project section:
projects: [
  {
    name: "project A",
    use: {
      config: defaultTestConfiguration,
      loginInfo: testEnvLoginInfo,
    },
  },
];

To make the ends connect, the extending interface must be updated as well, making the new property accessible in the base-test.ts:

export interface CustomTestOptions {
  ...,
  config: TestConfiguration;
  loginInfo: string;
}

//base-test.ts

export const test = base.extend<CustomTestOptions>({
  config: [defaultTestConfiguration, { option: true }],
  loginInfo: ['{}', { option: true }], //<- add empty object as default value here so JSON.parse() won't break if you forget to set it;

  page: async ({ page, config }, use) => {
    await page.addInitScript((customConfig) => {
      window.testData = { config: customConfig };
    }, config);

    await use(page);
  }
});

Note: the loginInfo is not needed as a parameter to the page, but defining it is mandatory if it is to be accessed in the tests.

Finally in the tests:

//Any test spec

test("Example test", async ({ page, loginInfo }) => {
  const loginData = JSON.parse(loginInfo);

  test.step("Perform login", async () => {
    await loginToTestEnvironment(page, loginData); //<- your custom method performing input filling, clicking, scrolling, pop-up closing and other magic
  });

  //test body continues
});

Using this method enables the developers to provide any secret (tokens, certificates, passwords) to their tests while keeping the data safe. Moreover, with some additional effort these secrets can be added the GitHub repository for use in CI/CD runs.

Summary

The main takeaways of this eight-weeks journey are the value of clean test code, which prevents long refactoring overheads, enables easier addition of test cases and scenarios, and makes maintenance as easy as possible. Also the value of clear and sufficient-enough abstractions is to be mentioned. One must find the balance between short code with all the syntactic sugar imaginable, readable code with clearly defined purpose and sufficient context, and reusable code to keep things DRY. It is not easy, but doable and worths the effort. Test and use-case organization is a big find as well. The basis of organizing the tests can be the features of the tested product, or the environments themselves, as they can differ in many ways. One should also consider edge-cases, component tests, state validations and so on. The line between UI testing and QA testing should also be drawn. After looking at our case from many angles we picked the environments as the common base, and organized everything around them. However, as all the project data can be widely customzed, many different approaches can and will be viable.

Documentation

The complete Playwright documentation can be accessed on this link: Playwright documentation

The part specifically explaining projects: Playwright Projects

The section about environment variables and custom data: Custom data