Enable developers to select a unit testing framework

[mgechev]

🚀 Feature request

Allow developers to select a unit testing framework similarly to ng e2e.

Command (mark with an x)

• new
• build
• serve
• test
• e2e
• generate
• add
• update
• lint
• extract-i18n
• run
• config
• help
• version
• doc

Description

Over the past few years we stopped shipping a linter by default and in v12 we enabled a plugin-based selection of an e2e testing framework similarly to ng deploy. At the same time, we've been getting lots of requests for Jest in the CLI and even mocha. The different tools have their trade-offs and even though karma allows real browser testing it's often outperformed by jest.

Describe the solution you'd like

Meet developers where they are and enable them to use the unit testing framework they want. Remove karma by default and instead change ng test to output a list of available and supported options, such as karma and jest.

Similarly to ng e2e and ng deploy we can recommend solutions which have high adoption and are maintained by the owner of the testing tool to provide guarantees for higher reliability.

A challenge with this approach is that it gets harder to automatically generate a unit test for every generated component/directive/component/pipe. We'd like to enable testing vendors to plug into the corresponding generator so they can provide a template that we can use in ng generate to produce a dummy unit test.

That's the open question as of right now, which will unblock us to provide flexibility while continuing to enforce best practices.

[LayZeeDK]

Don't forget about Vitest 😉

[mgechev]

After talking to @alan-agius4 and @dgp1130, looks like testing frameworks can override the unit test template by reusing the current schematics for generation of directives/components/pipes/services.

I don't see any major blockers to prioritize this and put it in the queue.

Vitest should also work fine given we can use JiT in testing. Next step would be to discuss timeline and follow up.

[pumano]

@mgechev would love if it will be implemented soon :)

[damil42]

Sounds fantastic. It would be great to get rid of Karma/Jasmine. It feels old and heavy. I currently use Jest which is really great. However, problems can also occur because the runtime is Node (with JSDOM). Same for ESM packages. Maybe the simplest way is to use Babel as precompiler. But I don't know what the best solution is. Vitest also looks modern and interesting. I'll try it next, maybe. In Vue CLI/UI you can also choose the testing framework. Angular could do the same thing it already does with CSS, SCSS, etc. Or for the first step: Allow removing the testing framework (Karma/Jasmine). Option in wizard: None or Karma/Jest. And later with Jest and Vitest and Mocha, etc. One of the first things I do on a new project is remove/replace Karma/Jasmine.

[dgp1130]

I just want to note that since this issue was opened we did publish a blog post describing our general unit testing strategy: https://blog.angular.dev/moving-angular-cli-to-jest-and-web-test-runner-ef85ef69ceca. I wanted to mention this for anyone just discovering this issue who wants to understand our current plans.

It's not exactly what @mgechev described in this issue, but it's pretty similar. Also just a friendly reminder that anyone can make their own Angular CLI builder today running any arbitrary unit test runner (such as Analog's Vitest builder).

I think what would be useful for this issue is to understand what developers would like to see beyond what's mentioned in that blog post which would make it easier to adopt alternative test runners.

[jneuhaus20]

I think what would be useful for this issue is to understand what developers would like to see beyond what's mentioned in that blog post which would make it easier to adopt alternative test runners.

1. Being able to follow the recommendations and use the tools of the modern frontend testing landscape.
2. Enough built-in support that I don't ever have to figure out how to make runner X load/parse things that Angular "just can"
3. Full power for customization so that when I inevitably have to anyway, I can follow all the sans-Angular articles I find without the Angular build tooling mangling my config changes.

Elaborating on 1, I want to be good at testing and I'm willing to learn the tools, but I don't actually care what framework it is, as long as it doesn't suck, and I don't spend 30% of my time on the test runner now.

On 2, I like Angular being opinionated, but it needs to support those opinions until they're deprecated, and switch support and opinion together. E.g. Karma is still the test runner of choice in Angular's opinion (the framework, evidenced by the docs and new project defaults,) but Karma is dead and doesn't support the things Angular is doing (like esbuild instead of webpack.) I'm anchored by the corpse of Karma now, and it's Angular's responsibility to help me out of this mess, because it's the one that told me to do it this way.

Further, if Jest is the new way, cool. Except it doesn't build the way Angular does, so some of my modules are a problem for it, but not Angular. I don't want to have to fully customize the recommended framework to make it work.

For 3, there's a bunch of little examples in the karma test runner or webpack where you could make such-and-such a change to the conf to try something, but Angular actually makes its own on the fly, and either ignores your changes, or prevents you from changing an incompatible setting. E.g. doing B instead of A requires flipping 3 switches, and Angular has a death grip on 2 of them.

I just want whatever it is to work so I can write my damn tests! And every testing framework to be like Jasmine.