Web Performance Test: Compare Grafana k6 browser with Google Lighthouse

[ad_1]

The Grafana k6 browser module simulates how users interact with a browser page and collects web performance metrics about the interaction. Since the module’s launch in 2021, we’ve often been asked how it compares to Google Lighthouse as a tool for measuring web page performance.

This blog post compares the k6 browser and Google Lighthouse from various perspectives, including:

How these tools help with performance testing How these tools perform tests What metrics the tools collect How the tools complement each other

Note: The k6 Browser is part of Grafana k6 OSS. The goal of this blog post is to compare the k6 browser module and not the entire k6 ecosystem with Google Lighthouse.

An overview of the k6 browser compared to Google Lighthouse

When developing web applications, it is crucial to test performance from a frontend or browser perspective, a concept we explain extensively in our website load test guide. Both the k6 browser and Google Lighthouse can measure performance of web applications, but there are key differences between the two.

k6 browser: front-end test with k6

The k6 browser automates browser actions and collects front-end performance metrics as part of existing k6 tests. With the k6 browser, you can simulate a specific user journey on your website and find errors related to user experience. At the end of a k6 browser test, it reports browser metrics that you can use as guidelines for the performance of your websites. With the k6 browser, you can also combine browser-level scripts with existing protocol-level scripts to implement a hybrid approach to performance testing.

Google Lighthouse: measures the overall quality of the website

Google Lighthouse is an open source tool that you can use to improve the overall quality of your websites. In addition to measuring performance, Google Lighthouse performs audits for accessibility, SEO, and more. These checks generate scores and a report with suggested changes to improve the quality of the website.

Again, while both the k6 browser and Google Lighthouse measure web application performance, they differ in several ways.

The table below summarizes the main points of comparison between the k6 browser and Lighthouse:

Running a web performance test

To measure the performance of your website, the k6 browser provides a Playwright-like API for some familiarity. Here is a sample code snippet:

import { chromium } from ‘k6/experimental/browser’ import { check } from ‘k6’ export default async function() { const browser = chromium.launch({ headless: false }) const page = browser.newPage() // Go to demo page and login wait page.goto(‘https://test.k6.io/my_messages.php’) page.locator(‘input[name=”login”]’).type(‘administrator’); page.locator(‘input[name=”password”]’).type(‘123’); const submitButton = page.locator(‘input[type=”submit”]’); wait Promise.all([page.waitForNavigation(), submitButton.click()]); check(page, { header: page.locator(‘h2’).textContent() == ‘Welcome, admin!’, }); page.close() browser.close() }

A k6 browser test can only be done with k6 OSS. However, work is underway to make the k6 browser available as part of the Grafana Cloud k6.

On the other hand, a Lighthouse test can be generated in a number of ways, including:

Chrome DevTools Programmatically via the Google Lighthouse PageSpeed ​​Insights library

Note: This is not the complete list. You can also run Lighthouse audits with tools like WebPageTest and test frameworks like Playwright and Cypress.

Let’s look at these options briefly.

Chrome DevTools

The most convenient way to run a Lighthouse test is with Chrome DevTools:

Download Google ChromeGo to the URL you want to checkOpen DevTools and use the Lighthouse tab Google Lighthouse Library

For added flexibility or advanced usage, the Google Lighthouse library can run audits programmatically with the command line or as a Node.

Page speed information

PageSpeed ​​Insights is another free tool that lets you run Lighthouse tests. Use real-world data in addition to the lab data used by Lighthouse. To run a test, log into PageSpeed ​​Insights, enter the URL you want to check, and view a report on your websites performance.

Web performance testing strategy

A holistic performance testing strategy must take into account both front-end and back-end performance. A recommended approach is hybrid performance testing, which combines protocol-level testing with browser-level testing simultaneously.

Lighthouse only measures the front-end performance of your application, so you’ll need to use another tool to measure the back-end performance. The k6 browser, on the other hand, supports a hybrid approach by leveraging existing k6 features.

Web performance metrics

When it comes to browser performance metrics, both the k6 browser and Lighthouse report Google Web Vitals. Web Vitals are the current standard engineers used to measure user experience for a website.

Historically, the k6 browser has only reported a few browser metrics, such as `browser_loaded` and `browser_dom_content_loaded`. However, these metrics don’t accurately reflect user experience as before. As a result, they are not sufficient to analyze critical performance bottlenecks your page may have and will be removed from the k6 browser in future releases. Web Vitals are a better measure of your page performance and user experience.

Even though both k6 browser and Lighthouse report Web Vitals, there are some differences in what they report.

Fundamental vitals of the Web

Core Web Vitals are the holy trinity when it comes to web performance. These metrics apply to all web pages and measure page load performance, interactivity, and visual stability.

Lighthouse does not measure the FID because this requires real user interaction and Lighthouse by default only analyzes the initial page load. On the other hand, the k6 browser can measure the FID because it provides an API that allows you to write user interactions.

Other vital web

Both the k6 browser and Lighthouse also track metrics known as Other Web Vitals. These metrics are not required but can provide additional insight into the user experience.

Lighthouse also tracks a metric called Speed ​​Index, which is not part of Google Web Vitals.

Laboratory data and field data

The last area of ​​comparison is the data used by both tools. Lighthouse uses lab data, which is data collected from a controlled environment using predefined device and network settings. Using lab data, you can test performance upfront, which is why Lighthouse is a popular choice. While it can’t measure metrics that require real user interaction, like FID, Lighthouse can measure TBT, which is a good alternative.

To accurately measure Core Web Vitals, you need field data. Unlike lab data, field data is collected from actual users, so it more closely represents the user experience.

Lighthouse users can use PageSpeed ​​Insights, which uses field data, to get a realistic idea of ​​website performance. Similarly, the Chrome User Experience Report offers a quick way to evaluate Core Web Vitals, but it doesn’t provide a detailed assessment to quickly diagnose performance issues.

The k6 browser reports field data using the web-vitals library, a standard web API, to measure all Core Web Vitals. The web-vitals library provides a way to measure Core Web Vitals on real users that match as reported by PageSpeed ​​Insights or the Chrome User Experience Report.

When it comes to web performance, we recommend using as many tools as possible to improve the overall user experience of your websites.

Lighthouse is excellent for early feedback and diagnostics to quickly resolve any low fruit issues. Lighthouse can check a website quickly, but users will need to integrate it with other tools because it only uses lab data.

To accurately measure all of the Core Web Vitals, however, you need to use field data, and tools like the k6 browser can measure these metrics natively.

Learn more about the k6 browser

Get started with the k6 browser

browser metrics k6

Video: Browser testing with k6

[ad_2]