Performance issues with `userEvent.type()`

[Etheryte]

• @testing-library/user-event version: 12.6.0
• Testing Framework and version: @testing-library/react version 11.2.1
• DOM Environment: jsdom version 16.4.0

Relevant code or config:

describe("demo", () => {
  const text = "2001:db8:ac10:fd01::20";

  test("time type", () => {
    render(<input data-testid="input" type="text" />);
    const input = screen.getByTestId('input');
    const before = new Date();
    userEvent.type(input, text);
    console.log(`type took ${new Date() - before}ms`);
    expect(input.value).toEqual(text);
  });

  test("time delayed type", async () => {
    render(<input data-testid="input" type="text" />);
    const input = screen.getByTestId('input');
    const before = new Date();
    await userEvent.type(input, text, {
      delay: Number.MIN_VALUE,
    });
    console.log(`delayed type took ${new Date() - before}ms`);
    expect(input.value).toEqual(text);
  });

  test("time paste", () => {
    render(<input data-testid="input" type="text" />);
    const input = screen.getByTestId('input');
    const before = new Date();
    userEvent.paste(input, text);
    console.log(`paste took ${new Date() - before}ms`);
    expect(input.value).toEqual(text);
  });
});

Problem description:

Running the above test suite with yarn test --no-cache logs the following values on my machine:

type took 29ms
delayed type took 63ms
paste took 1ms

The results are fairly consistent between reruns (plus-minus a few ms).

The performance difference is starker with longer strings: replacing the input text with const text = "2001:db8:ac10:fd01::20".repeat(100);, the results are as follows:

type took 1380ms
delayed type took 5940ms
paste took 1ms

I understand that some of this difference stems from the letter-by-letter nature of type() which is crucial for certain test scenarios. For the time being however, type() is too slow for many of the tests we run and we've had to opt for paste() instead.

I haven't profiled type()'s internals so I don't know what the source of the performance penalty is.

[ph-fritsche]

I don't think there is much to gain optimizing the test utility code for performance.
After all we're talking about miliseconds in a test environment that can run multiple tests parallel.
The approach here is to trade manual labor (analyzing code) for automated labor (running test containers) by running these utilities in the first place.

If you put in the work and optimize some part of the code, a PR will be welcome.
But keep in mind we'll err on the side of maintainability.

[Etheryte]

I don't think the scale of the problem is as low as you make it to be. If you do ten type() calls within one test you're already getting close to a second and with anything larger you can quickly reach Jest's default timeout of 5 seconds. Choosing to not look into this is fair, but I don't think downplaying the problem is beneficial.

For context, in a perfect world I would expect type() with no delay to be on the same scale as paste(), or at least in the same order of magnitude. Right now it can be several orders of magnitude slower, easily reaching the default time limit.

[ph-fritsche]

Don't get me wrong. I see that you can't type a book in 5sec default timeout.
But I just checked on my local machine. I can repeat your userEvent.type(input, '2001:db8:ac10:fd01::20') 308 times before hitting the timeout.
I don't see any test that would benefit from doing a character by character and event by event simulation of typing more than that.

userEvent.paste provides a completely different abstraction. No keydown..keypress..keyup etc. for each character.
If you want to test your user typing, it is no better than calling fireEvent.input directly.

What userEvent.type does for you on the other hand is exactly this: Simulate every event that would happen if a user went on and typed every character in your input string one by one.

[nickserv]

If this is still a performance issue for anyone, a reproducible example app or performance test using User Event 13 would be helpful.

[Etheryte]

@nickmccurdy Not sure if I misunderstood you, but the example given in the opening comment still yields the same results with @testing-library/user-event 13.2.0.

[nzyoni]

@ph-fritsche

I don't think there is much to gain optimizing the test utility code for performance.
After all we're talking about miliseconds in a test environment that can run multiple tests parallel.

Hi, 😄
In my case, we run our test suites in parallel using jest and we do face time issues with the type operation.
It takes longer than what you have mentioned here, it ranges from 500 ms to 1600 ms depending on the test.

After switching from type to paste we saw a major improvement
For example

1550 ms -> 680 ms
570 ms -> 94 ms

We want to use type in order to test our code/components as a user would use, but, using type makes our tests slow 😞 and as a result, it makes our pipeline slow.

IMO This issue should be addressed.

[ph-fritsche]

As mentioned before paste is a completely different implementation with a completely different behavior. It's not interchangeable at all and comparing the execution time between those two APIs is moot.
If you want to demonstrate the performance of the implementation, you need to record the events it dispatches and replay those calling all the same event handlers.

[danny-pflughoeft]

userEvent.type(input, 'spaces not allowed'); is currently taking 5+ seconds to execute on a simple component. The same call to .paste() is instant.

I think this issue should be reopened. I get that performance of tests is not hyper-critical, but when tests are timing out literally due to a single call to .type(), that's a major, breaking bug.

[ph-fritsche]

If you can provide a reproduction, we'll look into it.
If you keep on comparing the execution time to something completely different, we'll get nowhere.

[AlonMiz]

@ph-fritsche is it reasonable to wait for 5s for typing spaces not allowed?
a regular user would type it much faster.
I experience the same behavior in my codebase with formik forms (even when using FastField), typing is significantly slow. I had to move to paste to alleviate the long test times.

[ph-fritsche]

@AlonMiz Depends.

el.addEventListener('keydown', () => {
  const t0 = (new Date()).getTime()
  while((new Date()).getTime() < t0 + 5000) true
})

And all of a sudden it's reasonable for a single keydown to take 5s.
The fact that doing something completely different does not take 5s changes nothing.

So if you think there is a problem, please help to reproduce the issue.
.paste() does something completely different than .type(). The fact that different APIs which do completely different things perform different is moot.

I'm sure there are some things inside keyboard implementation that could be optimized. If someone wants to put in the work, I'll gladly support the effort. But I don't think there is much to gain and if the difference is miniscule, we'll opt for the more maintainable variant.

I'm pretty sure that there is nothing that allows to optimize in the magnitude that you are suggesting. This is simply because I know the code and it is not that slow in any environment I examined.

So if you guys think this issue should be addressed, please address it!
The first step doing so is providing a reproduction.
Stop demanding other people to try to imagine possible scenarios for performance issues. 😞

[AlonMiz]

i agree with giving a reproducible example.
and i get your main point of "this can be something on your side",
but the main point we are trying to make is that: there are several users that report the same issue, it is not resoved, but the issue is closed. of course it can be related to 3rd party module (like the combination of formik with testing-library), but let's try get to the root cause here, and then try to fix it. 🙏

[ph-fritsche]

@AlonMiz The initial issue report states something between 0.6ms (when repeated 100 times) and 1.3ms (on the first call) execution time per char. This is in the same ballpark of the execution times I got when reproducing the example on my machine with jest+jsdom.
(Updating to v14 the execution time goes up a little bit, probably due to the additional workarounds on the element properties and selection handling, but still nothing I would not consider reasonable. The execution time per char is actually somewhere around 0.0045ms (if you type 2000 characters). The rest of the execution time is calling the API and interpreting code and input (~0.6ms), clicking and selecting (~9ms), checking for pointer-events (~40ms) and awaiting the next microtask respectively the next macrotask when using delay.)
This issue has been closed because we don't consider this an issue. Running any userEvent call is primarily about simulating behavior, so it would only be an issue if you could simulate the same behavior in a substantially shorter amount of time.

If there is another issue with some environment, I don't know. But reports about >270ms per char suggest that this is another issue that is specific to some environment.