Menu
Documentationbreadcrumb arrow Grafana k6breadcrumb arrow Using k6 browserbreadcrumb arrow Migrating from Playwright
Open source

Migrating from Playwright

Introduction

  • Playwright has many use cases:
    • End-to-end web testing across browsers, platforms, and languages
    • Mobile web testing
    • API testing
    • Component testing
    • general purpose browser automation
  • k6 browser use cases are to complement protocol load tests to measure how backend under load impacts frontend performance, and for synthetic monitoring using Grafana SM product. Over the last year k6 has evolved to bring more functional paradigms such as the asserts library. We are evaluating work to align with other JS test frameworks.
  • When to migrate and why:
    • want to perform load testing and FE testing at the same time – non-functional testing.
    • same script can be easily transferred over to Grafana SM – functional testing.
    • (in the future) Tighter integration with FARO for RUM vs lab data.
  • What this guide covers:
    • a basic example of migrasting a PW script to k6 browser;
    • overview of some of the config migration;
    • running the script locally and on Grafana cloud;
    • some differences that are worth mentioning.
  • What this guide covers and doesn’t cover:
    • we don’t yet provide a list of APIs between k6 browser and playwright, including future work. Check the docs to see what is available. We have feeling that we have covered a lot of the most used APIs for browser frontend testing.
  • Important terminology in k6. Because it was originally designed as a load testing tool:
    • VU: virtual user;
    • Iteration: number of times a single VU will run the iteration;
    • thresholds and check: in load testing we’re generally more interested in a more holistic view of the test run, which will have many VUs, many iterations and running for many minutes/hours. We want to ensure that the backend system behaves correctly within thresholds that we define, e.g. 99th percentile for all requests to get a response under 1 second. There is an assertions library though if you’re more interested in the functional side of testing and want to assert on specific things in your test work flow.

Migration steps

Let’s work with this Playwright test script:

js 
import { test, expect } from '@playwright/test';

test('has title', async ({ page }) => {
  await page.goto('https://playwright.dev/');

  // Expect a title "to contain" a substring.
  await expect(page).toHaveTitle(/Playwright/);
});

At the moment, k6 doesn’t implement a test framework. So we work with the export default function.

Copy paste from your Playwright test into a new file, let’s call it pw-migrated.js:

js 
import { expect } from "https://jslib.k6.io/k6-testing/0.6.0/index.js";
import { browser } from 'k6/browser';

export const options = {
  scenarios: {
    ui: {
      executor: 'shared-iterations',
      options: {
        browser: {
          type: 'chromium',
        },
      },
    },
  },
};

export default async function () {
    // paste here
}

You should now have:

js 
import { expect } from "https://jslib.k6.io/k6-testing/0.6.0/index.js";
import { browser } from 'k6/browser';

export const options = {
  scenarios: {
    ui: {
      executor: 'shared-iterations',
      options: {
        browser: {
          type: 'chromium',
        },
      },
    },
  },
};

export default async function () {
  await page.goto('https://playwright.dev/');

  // Expect a title "to contain" a substring.
  await expect(page).toHaveTitle(/Playwright/);
}

k6 doesn’t implement fixtures, so we need to work with the browser to retrieve a page within it’s own context:

js 
import { expect } from "https://jslib.k6.io/k6-testing/0.6.0/index.js";
import { browser } from 'k6/browser';

export const options = {
  scenarios: {
    ui: {
      executor: 'shared-iterations',
      options: {
        browser: {
          type: 'chromium',
        },
      },
    },
  },
};

export default async function () {
  const page = await browser.newPage(); // <- creating a new page in its own incognito context
  await page.goto('https://playwright.dev/');

  // Expect a title "to contain" a substring.
  await expect(page).toHaveTitle(/Playwright/);
}

Run the test with k6 run pw-migrated.js. Which will result in:

shell 
 > k6 run pw-migrated.js

         /\      Grafana   /‾‾/  
    /\  /  \     |\  __   /  /   
   /  \/    \    | |/ /  /   ‾‾\ 
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/ 

     execution: local
        script: pw-migrated.js
        output: -

     scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
              * ui: 1 iterations shared among 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)



  █ TOTAL RESULTS 

    EXECUTION
    iteration_duration..........: avg=1.81s    min=1.81s  med=1.81s   max=1.81s p(90)=1.81s    p(95)=1.81s  
    iterations..................: 1      0.463436/s
    vus.........................: 1      min=1       max=1
    vus_max.....................: 1      min=1       max=1

    NETWORK
    data_received...............: 0 B    0 B/s
    data_sent...................: 0 B    0 B/s

    BROWSER
    browser_data_received.......: 1.8 MB 829 kB/s
    browser_data_sent...........: 7.7 kB 3.6 kB/s
    browser_http_req_duration...: avg=120.89ms min=1.95ms med=98.73ms max=1.14s p(90)=139.67ms p(95)=263.7ms
    browser_http_req_failed.....: 0.00%  0 out of 23

    WEB_VITALS
    browser_web_vital_fcp.......: avg=1.24s    min=1.24s  med=1.24s   max=1.24s p(90)=1.24s    p(95)=1.24s  
    browser_web_vital_ttfb......: avg=1.14s    min=1.14s  med=1.14s   max=1.14s p(90)=1.14s    p(95)=1.14s  




running (00m02.2s), 0/1 VUs, 1 complete and 0 interrupted iterations
ui   ✓ [======================================] 1 VUs  00m02.2s/10m0s  1/1 shared iters

You can find the docs for assertions here: https://github.com/grafana/k6-jslib-testing.

To understand the CLI output results, go here: https://grafana.com/docs/k6/latest/results-output/end-of-test/.

Sequential vs Parallel Tests

  • Scenarios run in parallel. They’re designed that way as in most cases when working with load tests we want to run things in parallel – i.e. load test the backend with protocol based tests, and in parallel run a browser test to assert that the frontend is behaving as you’d expect.
  • Curerntly there is no way for a scenario to start after another, one way to do this is to work with the startTime (as detailed here). See an example below:
js 
import { browser } from 'k6/browser'

export const options = {
  scenarios: {
    user: {
      exec: 'userLogin',
      executor: 'shared-iterations',
      options: {
        browser: {
          type: 'chromium',
        },
      },
    },
    admin: {
      exec: 'adminLogin',
      executor: 'shared-iterations',
      startTime: '5s',  // duration + gracefulStop of the above
      options: {
        browser: {
          type: 'chromium',
        },
      },
    },
  },
}

export async function userLogin() {
  const page = await browser.newPage();

  await page.goto('https://quickpizza.grafana.com/login', {
    waitFor: 'networkidle',
  });

  await page.waitForTimeout(1000);

  // replace with these when quickpizza labels are fixed
  // await page.getByLabel('username').fill('default');
  // await page.getByLabel('password').fill('12345678');
  await page.locator('#username').fill('default');
  await page.locator('#password').fill('12345678');
  await page.getByText('Sign in').click();

  await page.getByText('Your Pizza Ratings:').waitFor();

  await page.close();
}

export async function adminLogin() {
  const page = await browser.newPage();

  await page.goto('https://quickpizza.grafana.com/admin', {
    waitFor: 'networkidle',
  });

  await page.waitForTimeout(1000);

  // replace with these when quickpizza labels are fixed
  // await page.getByLabel('username').fill('admin');
  // await page.getByLabel('password').fill('admin');
  await page.locator('#username').fill('admin');
  await page.locator('#password').fill('admin');
  await page.getByText('Sign in').click();

  await page.getByText('Latest pizza recommendations').waitFor();

  await page.close();
}
  • You could also used the experimental xk6-redis extension as a way to store locks and help synchronise scenarios.

Hybrid tests

Cloud runs

Debugging & Development Experience

Key Differences & Limitations

  • Browser context restrictions;

    Unlike in Playwright, k6 can only work with a single browserContext at a time. So in k6 you won’t be able to do:

    js 
      const bc1 = await browser.newContext();
  // This next call will result in an error "existing browser context must be closed before creating a new one"
  const bc2 = await browser.newContext();

    You’ll have to close the existing browserContext first, before creating a new one.

  • Fixtures isn’t supported in k6 browser as well as no test framework. Abstractions will need to be hand coded.

  • Test isolation patterns – in k6 there is scenarios whereas in Playwright there is a dedicated test framework. The difference stems from k6 being a load testing tool. We are evaluating a test framework, but it’s still early days.

  • File uploads/downloads

  • Screenshots and recordings

    • We have screenshot support in local runs as well in Grafana cloud. We are evaluating a session recording feature for SM, load testing and Faro.

  • not being able to share data between VUs and iterations:

    • Work with a single executed immutable sharedarray
    • Work with the xk6-redis extension which is currently experimental but will be non-experimental very soon.
    • All browserContexts are incognito. An evaluation and more feedback is needed to understand its use case issue.

  • Files can only be read during the init phase and not during the test runtime with the experimental fs module.

  • web vitals will be reported on; we are evaluating further work to bring about more measurements such as JS heap size, long task and more.

Configuration Migration

  • Playwright config → k6 options mapping
    • k6 browser doesn’t work with a work directory, so you need to supply the exact script to run relative to the current directory.
    • Scenarios are independent of each other – they are parallel by default. Add startTime to make them sequential.
    • test.only – no equivalent
    • retries: No auto retry mechanism
    • workers – depends on the number of scenarios, which is like a single playwright worker, but also very different.
    • reporter – to change the output of the result take a look here for end of test summary, here for real time, web dashboard and grafana dashboard.
    • baseURL – no such equivalent
    • trace – work with grafana k6 to see a timeline view of the test run
    • projects – no such equivalent
    • webServer – no such equivalent
    • outputDir – depends on the output you use
    • globalSetup – no such equivalent
    • globalTeardown – no such equivalent
    • timeout – use K6_BROWSER_TIMEOUT env var
    • expect.timeout – no way to set this as a config, needs to be done in the main test block
    • expect.toHaveScreenshot – no such equivalent
    • expect.toMatchSnapshot – no such equivalent
    • sharding – work with grafana k6 for easy multi region, multi load generator setup, with results automatically merged into a single report.
    • typescript is supported
  • Browser launch options