Gather and publish CSS Typed OM performance data in an explainer?

[nainar]

Originally raised by @slightlyoff here: w3ctag/design-reviews#223 (comment)

[darrnshn]

I've done some initial benchmarking. tl;dr Allocating Typed OM objects is slow, so you have to reuse objects for performance gains.

The first benchmark just calls StylePropertyMap.set with a precomputed Typed OM object. This shows that skipping CSS parsing indeed brings performance gains [1], as long as we reuse objects. In the benchmark, you can play around with iterations to change the amount of reuse per cycle.

The second benchmark tests a common use case: getting a value from the style map, modifying it, and then putting it back. Here, strings is slightly faster. I suspect this is because we create a new object every time we call StylePropertyMap.get. Perhaps we can optimise for this case with StylePropertyMap.update (reuse the same object over multiple calls?). Also, the benchmark shows a simple case of parsing a length. If we had to parse complicated objects like transforms in JavaScript (I don't actually know to do this properly), I suspect Typed OM would be faster.

[1] On my Chrome Canary (66) build, I get:

string x 18,769 ops/sec ±1.21% (94 runs sampled)
typed x 21,627 ops/sec ±1.86% (88 runs sampled)
Fastest is typed

[2] I got:

string x 513,838 ops/sec ±1.11% (95 runs sampled)
typed x 480,403 ops/sec ±5.46% (79 runs sampled)
Fastest is string,typed

[darrnshn]

Hey @dbaron , I was wondering if you know what sort of performance gains the initial Firefox implementation of Typed OM got?

[foolip]

I suspect this is because we create a new object every time we call StylePropertyMap.get. Perhaps we can optimise for this case with StylePropertyMap.update (reuse the same object over multiple calls?)

I don't have any hands-on experience with the API to say if that makes for a good API or not, but hasten to point out that this would be observable, and so defined by the spec if it isn't already. It's a bit like [SameObject] in Web IDL, and it's pretty easy to find inconsistencies between implementations in that arena. With a single shared test we can probably avoid having such a situation for CSS Typed OM, if one doesn't already exist.

[emilio]

Hey @dbaron , I was wondering if you know what sort of performance gains the initial Firefox implementation of Typed OM got?

I don't think anybody had an initial impl on Firefox (only typed custom props were implemented, typed om is https://bugzilla.mozilla.org/show_bug.cgi?id=1278697 which doesn't have any patch or similar).

In any case probably @jyc knows better than I :)

[tabatkins]

I've done some initial benchmarking. tl;dr Allocating Typed OM objects is slow, so you have to reuse objects for performance gains.

This is expected, yeah. The fast case is indeed something like "build a CSSTransformValue, use it in multiple .set() calls across rAF() callbacks".

This is why I really really really want ECMAScript to give us Value Objects, or Typed Objects as the proposal seems to have evolved into, where they're just wrappers around a chunk of TypedArray data. These should be much faster because they're immutable and we can reuse them more widely.

(This is also why TypedOM was originally as strongly immutable as we could make it with normal JS objects, so we could replace it with Typed Objects later with minimal breakage. We gave up on this when JS appeared to not be willing to give us the thing in a reasonable timescale; we might have to re-evaluate that.)

[darrnshn]

After some improvements to performance of StylePropertyMap.get, the results for the roundtripping benchmark that I'm getting are:

string x 528,089 ops/sec ±1.33% (91 runs sampled)
typed x 531,066 ops/sec ±5.84% (77 runs sampled)
Fastest is string

Whilst typed has a higher mean result, I think benchmark.js might've penalized it for a higher variance?

The results were obtained on the new MacBook Pro. On my Linux workstation and my old MacBook Pro, the difference is greater and more consistent:

e.g. on my Linux workstation:

string x 472,318 ops/sec ±1.96% (94 runs sampled)
typed x 551,981 ops/sec ±5.77% (80 runs sampled)
Fastest is typed

Note: The numbers seem to vary a bit, so the relative difference between the two results is probably more useful than their absolute values.

[darrnshn]

Also, Blink uses a character-level "fast path" parser for common CSS values like lengths, but not for more complicated ones. So there are many cases where we can't use the fast path, and Typed OM would be significantly faster (you can try it by putting a space after "px" to not go through the fast path).

[nainar]

This issue (even closed) will capture the discussion so closing it as part of issue triaging.

[tabatkins]

In particular, my numbers on the "fast path" show an ~30% increase in ops/sec for TypedOM, agreeing with yours, and on the slow path (using 'px ', with a space, to set the string-OM case) it's nearly 100%:

string x 137,575 ops/sec ±5.04% (75 runs sampled)
typed x 225,774 ops/sec ±6.68% (75 runs sampled)
Fastest is typed

[trusktr]

These should be much faster because they're immutable and we can reuse them more widely.

Why immutable objects? Immutable patterns are widely known to be slower than mutable patterns, and immutable patterns are known to be a developer experience benefit over a performance benefit. What am I missing in this case?

[tabatkins]

It's entirely dependent on what you're doing with them. Immutable objects allow sharing, so you have lower memory pressure and less computation involved in construction. In return, when you do mutate a structure you have to reconstruct parts of it, which can be slower than just doing an in-place mutation. Which of these factors wins in any given scenario depends on your workload.