v8.deserialize() crashes with "Unable to deserialize cloned data."

[michaelKurowski]

Version

v21.7.2

Platform

Tested both on:

Darwin MacBook-Pro.local 21.3.0 Darwin Kernel Version 21.3.0: Wed Jan  5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T8101 arm64

and

Linux vps-7ec5dfe5 4.19.0-25-cloud-amd64 #1 SMP Debian 4.19.289-2 (2023-08-08) x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

1. Download the serialised file: https://mega.nz/file/2p4ViB4b#RdSp2vrxwW6kd2wp-6-ih_UcD148HMipbFoWCGwbWdo
2. Read and attempt to deserialise it:

const file = await fs.readFile(pathToTheDownloadedFile, null)
v8.deserialize(file)

3. The function will throw a similiar error the one below:

[nuxt] [request error] [unhandled] [500] Unable to deserialise cloned data.
  at Object.deserialize (node:v8:401:14)
  at load (./.output/server/chunks/nitro/node-server.mjs:51286:32)
  at async ./.output/server/chunks/load.post.mjs:37:19
  at async Object.handler (./.output/server/chunks/nitro/node-server.mjs:2332:19)
  at async Server.toNodeHandle (./.output/server/chunks/nitro/node-server.mjs:2521:7)

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior? Why is that the expected behavior?

The function should work, or there should be a workaround on how to operate with big files.

What do you see instead?

In case of my code I receive:

[nuxt] [request error] [unhandled] [500] Unable to deserialise cloned data.
  at Object.deserialize (node:v8:401:14)
  at load (./.output/server/chunks/nitro/node-server.mjs:51286:32)
  at async ./.output/server/chunks/load.post.mjs:37:19
  at async Object.handler (./.output/server/chunks/nitro/node-server.mjs:2332:19)
  at async Server.toNodeHandle (./.output/server/chunks/nitro/node-server.mjs:2521:7)

Additional information

The file is an output from v8.serialize

  const serialisedWorld = v8.serialize(world)
  ...
  fs.writeFileSync(path, serialisedWorld)

As far as I understand these two functions should be compatible with each other, so it's either deserialize that's throwing error despite receiving a correct object, or it's v8.serialize silently producing incorrect output.

Various variants of this kind of file format work just fine, the problem seems to appear once the file grows big.

[jasnell]

Without knowing more about the file you are attempting to deserialize and how it was created, it's going to be a bit difficult to diagnose here... and downloading some random binary off mega.nz is not something I'm keen to try or would recommend others to try. Can you provide a more self-contained repro case that does not involve downloading a file blindly?

[michaelKurowski]

Thank you for your response. I understand your concerns regarding downloading files from unknown sources.

The file I provided serves as a guaranteed reproduction case. Unfortunately, since the v8.deserialize function doesn't provide any meaningful error messages, I have no further insights into the root cause, so I can't create a minimal example that contains only problematic part.
The object I serialized is large and complex, and the resulting file is several megabytes long. I explicitly use this function because it contains things such as circular references to objects and I can't serialize it with other methods such as JSON. The file I've sent is a raw output from the v8.serialize function. It is, at least should be, exactly the problematic payload that causes the issue.

To clarify, the issue arises with the following straightforward flow:

1. v8.serialize is used to serialize a JavaScript object.
2. The serialized output is written to a file using fs.writeFileSync (this is the file I've attached).
3. The file is then read using fs.readFileSync.
4. Finally, v8.deserialize is used on the read data, resulting in the described error.

The hosting of the file is not critical for me, even having it as a file isn't critical for me. I sent it as a file, because I have to give to you somehow the payload that this function can't handle and this function accepts binary data.
I can provide alternative hosting options if you have preferences. For example:

• Hosting the file on a different platform
• Create some sort of a nodejs wrapper that would serve this data over http so that you can pull it with fetchinstead of downloading it as a file.
• Putting the file in a repository, including a three-line Node.js snippet that reads the file and deserializes it.

If either of these approaches would be more safe, please let me know. Alternatively, if you recommend any other methods for securely handling this situation, I’m ready to accommodate. I believe this issue showcases that v8.serialize <-> v8.deserialize flow is actually broken and can not be trusted.

I’m open to any approach you find most suitable and willing to provide additional details, such as logs, specific environment info etc. I’ve already tested this on both macOS (Darwin 21.3.0 on arm64) and Debian (4.19.0-25-cloud-amd64 on x86_64) with the same results, suggesting it isn’t tied to a single platform.

If there is anything else you’d like me to check or any steps you’d recommend for a more thorough debug, let me know.

[michaelKurowski]

Here's the reproduction:
https://github.com/michaelKurowski/deserialize-reproduction