This content originally appeared on Level Up Coding - Medium and was authored by Guillaume Renard
My journey towards better frontend testing.
What is wrong with frontend testing?
When you look at tutorials (including mine), everything is always nice and easy when it comes to testing an app, because:
- it is a fresh install
- it uses modern tooling
- it has few dependencies
- there are only a handful of tests
But when you try to test one of your existing projects, you have to deal with the following:
- it was created with Webpack or Create-React-App (CRA)
- the tools haven’t been updated in years
- it has many dependencies, some of which are barely maintained
- it already has some tests, with variable quality, questionable relevance, and an overall lack of strategy
I work on these projects—mostly enterprise software, with a fair share of technical debt. I have been using the tools available at the time and following the best practices highlighted in the Guiding Principles of Testing Library. And I have mostly enjoyed testing so far.
But I came to the conclusion that testing, in its current form, doesn’t scale. There is always a point where:
- tests are running slow and timeouts are becoming the norm
- it is common for code that works in the browser to fail in the test suite
- some portions of code rely so much on mocks that the tests lose relevance
- debugging without “seeing” is extremely painful
- some dependencies will break in the tests, with some obscure error (usually an issue with CJS vs ESM modules)
So what can we do about this? How can we enhance the developer experience (DX) and our tests at the same time?
Let’s explore a few options.
Jest
Jest is a testing framework developed by Facebook. It aims to provide a fast, simple, zero-config, and reliable way to test web applications. If the Jest API (test, expect, beforeEach, etc.) has stood the test of time (10 years!), I’m not so sure it fully delivered what it promised… In 2024, Jest is neither fast nor easy to set up!
The other culprit, JSDOM, is the fake DOM implementation you use to render components inside the test runner (= in Node, instead of the browser). Unfortunately, the DOM implementation is partial and performance is poor.
Jest’s last major release was two years ago and the tool only has experimental support for ECMAScript Modules (the standard module system). JSDOM is barely maintained and the most downloaded versions (20.0.3 and 16.7.0) are over 2 years old.
And if you’re still using Jest through Create-React-App (CRA), I have bad news for you: you’re probably running versions of Jest and JSDOM that are over 3 years old 🙈.
Sure, you could always update Jest and JSDOM to the latest version and delay the inevitable… Even better, you could replace JSDOM with Happy DOM — another DOM implementation that is actively maintained and aims for better performance. The migration is pretty straightforward: replace one dependency (jsdom) with another (happy-dom), maybe fix a few edge cases, and off you go.
But based on my experience, you will only achieve marginal gains this way. The pain points I highlighted earlier will still exist.
Also, because of issues with third-party dependency, I’m willing to bet that you are currently using Jest’s ‘magic’ setting in your legacy projects: transformIgnorePatterns. The setting allows Jest to transpile problematic modules to a format it understands (CommonJS). While this mostly works, it is also terribly slow: the transformation happens before running the tests, which may take several seconds or even minutes.
OK, so what about Jest’s main competitor: Vitest?
Vitest
As the name suggests, Vitest is another test framework that is part of the Vite ecosystem. Its API is very close to Jest, making it easy to migrate to Vitest without rewriting test cases. And just like Vite, Vitest can run straight from your sources (with minimal transformation), making it pretty fast to kick in.
Unlike Jest, Vitest uses ESM, the latest standard for JavaScript modules. While this works great with new projects, Vitest can be picky with your imports. Packages that are bundled improperly will cost you hours looking for solutions. You might have to use some workarounds that are either slow or inconvenient, such as inlining dependencies…
While Vitest is a step forward compared with Jest, you shouldn’t expect too much from it alone. Even if you manage to run it on a legacy project, you will probably not see a big performance boost. Testing times should improve (provided you didn’t inline too many dependencies…), but only marginally. And the developer experience will broadly be the same.
I’m afraid, testing web apps with a fake DOM simply doesn’t scale, no matter the tool. Mocking is a problem. Performance is a problem. Testing without seeing is also a problem.
So, what are the other options?
Playwright Component Tests
When Playwright announced component testing I started to get excited. The promise was to write tests just like with Jest and Testing Library, but have them run inside a real browser, instead of some fake DOM.
Problem solved? Well, not so fast.
Playwright component tests suffer from CSJ/ESM module hell, meaning that it is almost impossible to get them running on a legacy project (unless the project has zero dependencies or all dependencies are perfectly bundled…).
And if you manage to get through this, I hope your project doesn’t use Mock Service Worker (MSW). For some reason, Playwright decided that MSW would always be a second-class citizen in component tests. This is such a footgun considering that MSW is one of the few tools developers are not looking to replace!
I hope the situation improves but for now, Playwright component tests are not an option.
Playwright (vanilla)
So what about using Playwright directly, without component tests? I mean: use the Playwright API to open a page and write assertions against it.
My initial thought was: “Yuck! Playwright is for end-to-end testing (E2E). These tests have a place in the testing pyramid, but they should stay out of my app”.
Past the disgust, I decided to give it a try. But with one constraint: the tests should not hit an external system or service. They should be self-contained.
One of the advantages of using Playwright directly is that whatever runs in the browser runs in your tests. So unlike in component tests, we can use MSW to mock the network requests. This way, we can write predictable tests that don’t have any side effects, while working with the real app.
I found one caveat though: mocking authentication (eg OAuth) is hard, due to the nature and the complexity of the interactions involved. You can solve this by injecting a token in the page with Playwright (eg: by setting a global variable) and pass it to your auth library. This means creating a special setup (or configuration) for your app to run inside Playwright. There isn’t a magic recipe for this: use whatever works for you and your special requirements.
This was the tedious part.
The rest is pure joy! Once your app is set up, you can load routes, interact with the page, and write assertions. And the best part: you can see what’s happening in real time! The Playwright UI is a delight. It lets you access the console, view the network requests, and inspect the exact state of the page at any time, so it’s easy to debug when your assertions are failing. Oh boy, I was not ready for this!
So is Playwright THE solution?
Unfortunately no. It may be part of it, but it is not the ultimate solution. E2E tests are slow, even when mocking the network. Imagine refreshing the page after each test case! Sure, you could always bend the rules and load each page once, then run multiple test cases, with the risk of breaking isolation. I’m not sure I’m very comfortable with the idea yet.
Also testing solely at the route level is a no-go. I want to test components in isolation too, which is not possible with Playwright — the very issue component tests were supposed to address... If you want to test components individually, you first need to install a tool such as Storybook or Ladle, then write stories for them that you can use in the browser.
I did not like the thought of using another tool initially, so I tried writing a few custom routes by hand for each component, which I then tested inside Playwright. While the approach worked, I felt I was reinventing Storybook… without a good DX.
So why not use Storybook instead?
Storybook Component Tests
Storybook is a tool used to build UI components. Whenever you write a new component, Storybook lets you add stories (= small snippets) in a file next to it, which you can visualize in the browser. This allows developers to test their components through different use cases with instant feedback.
Although my overall experience with Storybook has been positive, some developers have expressed concerns that Storybook was bloated (too many dependencies). Luckily, the latest version (8.3 to date) addresses this, so this is no longer a fair thing to say. The tool has consistently improved and become faster since version 7.
It’s also been a while since Storybook understood that the future was with browser testing, not fake DOM. It turns out that Storybook also comes with a test runner, which uses Playwright internally.
Component testing in Storybook consists of running your stories in a browser and verifying they render without errors. While stories give you some code coverage for free, they can’t replace regular unit tests alone.
So, Storybook lets you attach a play function to them, which you can use to write assertions and interact with the component. The API looks very familiar to Jest and Testing Library. This allows you to play and replay tests step by step within Storybook and run them in a test runner.
One source of truth (the story), many outputs! That’s what I call good DX.
That being said, running the test runner can be problematic in practice. The test runner requires Storybook to be up and running beforehand, which is not ideal in a CI pipeline or a pre-commit hook…
For this reason, I never regarded Storybook as a serious testing option before. However, this changed with the introduction of the new experimental Vitest plugin! This plugin substitutes the test runner with Vitest and uses browser mode to execute tests.
Despite being in the early stages, it already works remarkably well!
But what is Vitest browser mode exactly?
Vitest browser mode
Vitest browser mode is similar to Playwright component tests. It has the same API as regular tests but renders in the browser thanks to an E2E provider — typically Playwright. Unlike Playwright component testing, it’s easy to get running and you can use MSW in your tests!
With Vitest Workspaces you can also run browser tests alongside regular unit tests. This makes it very convenient to collect code coverage from both sources. Because, yes, Vitest can collect coverage metrics in browser mode too — something that is hard to achieve with Playwright (see My Playwright code coverage journey). This might not seem like a big deal, but many companies require code coverage reports to be produced for all apps (often for the wrong reasons, but that’s another story).
So oddly enough, there is not much to say about Vitest browser mode: it’s simple, it’s fast, and it just works! Using browser mode does not require new knowledge. A bit of config is all it takes. Sure, it’s still experimental, but you can already see the potential.
My ideal setup… for 2025
Alright! This has been a long and tiring journey, but I have gathered enough information to identify where I want to head with testing now.
My ultimate objectives are:
- to invest less time in testing
- to get more coverage out of it
- to have fast and reliable tests
So here is my strategy to achieve this:
- write E2E tests at route levels with Playwright to safeguard my acceptance criteria
- write stories in Storybook to test my components with different use cases, and run them using Vitest browser mode
- write regular unit tests with Vitest for the rest (mostly utility functions)
It’s worth noting that Vitest browser mode could be used to achieve point 1, but I prefer to use Playwright because it runs against the actual application, which gives me more confidence in my tests. Playwright’s GUI also offers a better developer experience than Vitest’s.
You could also use Vitest browser mode for component testing (point 2) but Storybook is well worth the extra overhead in my opinion. By leveraging the power of Storybook, you can develop components in isolation and test many use cases without breaking a sweat. Add a play function and some assertions to your stories and you no longer need regular unit tests.
For now…
The future I describe above is nearer than you might think. So if you like living on the edge, feel free to go for it now!
But here is what is missing for me:
- easy code coverage with Playwright
- Vitest browser mode is still experimental and docs are lacking—getting it to run requires trial and error (although when it works, it works very well)
- the Vitest plugin for Storybook is also experimental, and it will probably be for as long as Vitest browser mode remains so
That being said, you can start preparing for it now by upgrading your tools. If you’re in a Single Page Application (SPA), make sure to use modern tooling such as Vite or Rsbuild. Modern testing requires modern tooling. Depending on how old your project is and the amount of dependencies, this might take some time.
Then migrate your tests to Vitest. Again, you might encounter some issues with dependencies, but things are improving thanks to authors upgrading their libraries to ESM. This will not dramatically improve the speed of your tests (although you might see marginal gains when updating JSDOM or replacing it with Happy DOM). But since Vitest is a requirement for most tools I presented above, you will be ready to adopt them.
I would also start adding Storybook to projects with many components and ensure that stories are created when adding a new component. Even if the stories don’t run in your tests yet, the DX will start improving as a result.
Finally, I highly recommend getting familiar with Testing Library and its guiding principles. Modern testing tools are built with those principles in mind. Also, if you’re not using Mock Service Worker (MSW) yet, you probably should. No matter how and where you test, one constant is that you will have to mock the network. And there’s no better tool for this than MSW.
Hey! I’m Guillaume, a longtime frontend engineer. I want you to know that I will be writing a follow-up article with the actual, concrete, and step-by-step setup for my ideal testing setup. So if you’re looking forward to it, hit ‘Follow’ and stay tuned!
Are Playwright and Vitest ready to replace Jest? was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Guillaume Renard
Guillaume Renard | Sciencx (2024-10-08T16:38:39+00:00) Are Playwright and Vitest ready to replace Jest?. Retrieved from https://www.scien.cx/2024/10/08/are-playwright-and-vitest-ready-to-replace-jest/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.