Flaky VM crash: runtime\platform\threads.h: 55: error: expected: owner_ == kUnowned

[alexmarkov]

The following flaky crash happened on vm-aot-win-debug-arm64:

/===========================================================================\
| service/dev_fs_http_put_test/service broke (Pass -> Crash, expected Pass) |
\===========================================================================/

--- Command "vm_compile_to_kernel" (took 05.000785s):
set DART_CONFIGURATION=DebugARM64 & set DART_SUPPRESS_WER=1 & set DART_CRASHPAD_HANDLER=c:\opt\s\w\ir\cache\builder\sdk\out\DebugARM64\crashpad_handler.exe & C:\opt\s\w\ir\cache\builder\sdk\pkg\vm\tool\gen_kernel.bat --aot --platform=out/DebugARM64/vm_platform_strong.dill -o c:\opt\s\w\ir\cache\builder\sdk\out\DebugARM64\generated_compilations\vm-aot-win-debug-arm64\runtime_observatory_tests_service_dev_fs_http_put_test\out.dill C:\opt\s\w\ir\cache\builder\sdk\runtime\observatory\tests\service\dev_fs_http_put_test.dart -Dtest_runner.configuration=vm-aot-win-debug-arm64 --packages=C:\opt\s\w\ir\cache\builder\sdk\.dart_tool\package_config.json -Ddart.vm.product=false --sound-null-safety

exit code:
0

--- Command "precompiler" (took 23.000218s):
set DART_CONFIGURATION=DebugARM64 & set DART_SUPPRESS_WER=1 & set DART_CRASHPAD_HANDLER=c:\opt\s\w\ir\cache\builder\sdk\out\DebugARM64\crashpad_handler.exe & out\DebugARM64\gen_snapshot --snapshot-kind=app-aot-elf --elf=c:/opt/s/w/ir/cache/builder/sdk/out/DebugARM64/generated_compilations/vm-aot-win-debug-arm64/runtime_observatory_tests_service_dev_fs_http_put_test/out.aotsnapshot --loading-unit-manifest=c:/opt/s/w/ir/cache/builder/sdk/out/DebugARM64/generated_compilations/vm-aot-win-debug-arm64/runtime_observatory_tests_service_dev_fs_http_put_test/ignored.json --sound-null-safety -Dtest_runner.configuration=vm-aot-win-debug-arm64 --ignore-unrecognized-flags --packages=C:\opt\s\w\ir\cache\builder\sdk\.dart_tool\package_config.json c:\opt\s\w\ir\cache\builder\sdk\out\DebugARM64\generated_compilations\vm-aot-win-debug-arm64\runtime_observatory_tests_service_dev_fs_http_put_test\out.dill

exit code:
0

--- Command "remove_kernel_file" (took 44ms):
set DART_CONFIGURATION=DebugARM64 & set DART_SUPPRESS_WER=1 & set DART_CRASHPAD_HANDLER=c:\opt\s\w\ir\cache\builder\sdk\out\DebugARM64\crashpad_handler.exe & cmd.exe /c del c:\opt\s\w\ir\cache\builder\sdk\out\DebugARM64\generated_compilations\vm-aot-win-debug-arm64\runtime_observatory_tests_service_dev_fs_http_put_test\out.dill

exit code:
0

--- Command "vm" (took 296ms):
set DART_CONFIGURATION=DebugARM64 & set DART_SUPPRESS_WER=1 & set DART_CRASHPAD_HANDLER=c:\opt\s\w\ir\cache\builder\sdk\out\DebugARM64\crashpad_handler.exe & set TEST_COMPILATION_DIR=c:/opt/s/w/ir/cache/builder/sdk/out/DebugARM64/generated_compilations/vm-aot-win-debug-arm64/runtime_observatory_tests_service_dev_fs_http_put_test & out\DebugARM64\dartaotruntime.exe --sound-null-safety -Dtest_runner.configuration=vm-aot-win-debug-arm64 --ignore-unrecognized-flags --packages=C:\opt\s\w\ir\cache\builder\sdk\.dart_tool\package_config.json c:/opt/s/w/ir/cache/builder/sdk/out/DebugARM64/generated_compilations/vm-aot-win-debug-arm64/runtime_observatory_tests_service_dev_fs_http_put_test/out.aotsnapshot

exit code:
3

stdout:
00:00 �[32m+0�[0m: out.aotsnapshot (VM Service)�[0m

** Launching SERVICE_TEST_TESTEE=true out\DebugARM64\dartaotruntime.exe --disable-dart-dev --no-dds --write-service-info=file:///c:/opt/s/w/ir/x/t/dart_service40eeedfd/service_info.json --profiler --sound-null-safety -Dtest_runner.configuration=vm-aot-win-debug-arm64 --ignore-unrecognized-flags --packages=C:\opt\s\w\ir\cache\builder\sdk\.dart_tool\package_config.json --enable-vm-service:0 c:\opt\s\w\ir\cache\builder\sdk\out\DebugARM64\generated_compilations\vm-aot-win-debug-arm64\runtime_observatory_tests_service_dev_fs_http_put_test\out.aotsnapshot

>testee>out> 

>testee>out> The Dart VM service is listening on http://127.0.0.1:50716/RpAsqRZCj0E=/

** Signaled to run test queries on http://127.0.0.1:50716/RpAsqRZCj0E= (VM Service)

Loading VM...

Done loading VM

Running out.aotsnapshot [1/1]

All service tests completed successfully.

** Killing script

00:00 �[32m+1�[0m: All tests passed!�[0m

** Process exited

stderr:
c:\opt\s\w\ir\cache\builder\sdk\runtime\platform\threads.h: 55: error: expected: owner_ == kUnowned
version=3.8.0-edge.19e2a2d58c9574fbf4a3a9658eb76072c0461833 (main) (Mon Apr 7 13:48:53 2025 -0700) on "windows_arm64"
pid=11864, thread=3056, isolate_group=(nil)(0000000000000000), isolate=(nil)(0000000000000000)
os=windows, arch=arm64, comp=no, sim=no
isolate_instructions=0, vm_instructions=0
fp=8c0b6ff7d0, sp=8c0b6ff7c0, pc=7ff6d8caceec
  pc 0x00007ff6d8caceec fp 0x0000008c0b6ff7d0 Unknown symbol
  pc 0x00007ff6d8093ce8 fp 0x0000008c0b6ff7f0 Unknown symbol
  pc 0x00007ff6d8095148 fp 0x0000008c0b6ff830 Unknown symbol
  pc 0x00007ff6d80be49c fp 0x0000008c0b6ff850 Unknown symbol
  pc 0x00007ff6d80824d0 fp 0x0000008c0b6ff880 Unknown symbol
  pc 0x00007ff6d8093468 fp 0x0000008c0b6ff8a0 Unknown symbol
  pc 0x00007ff6d808f0ac fp 0x0000008c0b6ffad0 Unknown symbol
  pc 0x00007ff6d8fb9be0 fp 0x0000008c0b6ffaf0 Unknown symbol
  pc 0x00007ff6d8fb98c4 fp 0x0000008c0b6ffb20 Unknown symbol
  pc 0x00007ff6d8fb96e4 fp 0x0000008c0b6ffb60 Unknown symbol
  pc 0x00007ff6d8fb9dac fp 0x0000008c0b6ffb70 Unknown symbol
  pc 0x00007ff848762310 fp 0x0000008c0b6ffb90 Unknown symbol
  pc 0x00007ff849055aec fp 0x0000008c0b6ffbd0 Unknown symbol
-- End of DumpStackTrace

--- Re-run this test:
python3 tools/test.py -n vm-aot-win-debug-arm64 service/dev_fs_http_put_test/service

https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket/8718184469624617057/+/u/test_results/ignored_flaky_test_failure_logs

/cc @rmacnak-google

[alexmarkov]

Similar crash happened on vm-win-debug-x64 about a week ago. So it is extremely rare and not specific to a particular test.

/===========================================================================\
| co19/LibTest/io/Stdout/write_A01_t07 broke (Pass -> Crash, expected Pass) |
\===========================================================================/

--- Command "vm" (took 02.000787s):
set DART_CONFIGURATION=DebugX64 & set DART_SUPPRESS_WER=1 & set DART_CRASHPAD_HANDLER=C:\b\s\w\ir\out\DebugX64\crashpad_handler.exe & out\DebugX64\dart.exe --sound-null-safety -Dtest_runner.configuration=vm-win-debug-x64 --ignore-unrecognized-flags --packages=C:\b\s\w\ir\.dart_tool\package_config.json C:\b\s\w\ir\tests\co19\src\LibTest\io\Stdout\write_A01_t07.dart

exit code:
1073741845

stderr:
../../runtime/platform/threads.h: 55: error: expected: owner_ == kUnowned
version=3.8.0-edge (main) (Unknown timestamp) on "windows_x64"
pid=3360, thread=4808, isolate_group=(nil)(0000000000000000), isolate=(nil)(0000000000000000)
os=windows, arch=x64, comp=no, sim=no
isolate_instructions=0, vm_instructions=0
fp=a77e72f828, sp=a77e72f828, pc=7ff61a7c91c2
  pc 0x00007ff61a7c91c2 fp 0x000000a77e72f828 Unknown symbol
-- End of DumpStackTrace

--- Re-run this test:
python3 tools/test.py -n vm-win-debug-x64 co19/LibTest/io/Stdout/write_A01_t07

https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket/8718773968426090561/+/u/test_results/ignored_flaky_test_failure_logs

[alexmarkov]

One more crash happened on March 20, on vm-win-debug-x64 bot and different test:

/================================================================================\
| pkg/vm_service/test/eval_issue_49209_test broke (Pass -> Crash, expected Pass) |
\================================================================================/

--- Command "vm" (took 13.000138s):
set DART_CONFIGURATION=DebugX64 & set DART_SUPPRESS_WER=1 & set DART_CRASHPAD_HANDLER=C:\b\s\w\ir\out\DebugX64\crashpad_handler.exe & out\DebugX64\dart.exe --sound-null-safety -Dtest_runner.configuration=vm-win-debug-x64 --ignore-unrecognized-flags --packages=C:\b\s\w\ir\.dart_tool\package_config.json C:\b\s\w\ir\pkg\vm_service\test\eval_issue_49209_test.dart

exit code:
1073741845

stdout:
00:00 �[32m+0�[0m: eval_issue_49209_test.dart�[0m

** Launching SERVICE_TEST_TESTEE=true out\DebugX64\dart.exe --disable-service-auth-codes --profiler --sound-null-safety -Dtest_runner.configuration=vm-win-debug-x64 --ignore-unrecognized-flags --packages=C:\b\s\w\ir\.dart_tool\package_config.json --enable-vm-service:0 C:\b\s\w\ir\pkg\vm_service\test\eval_issue_49209_test.dart

>testee>out> The Dart VM service is listening on http://127.0.0.1:58433//
>testee>out> The Dart DevTools debugger and profiler is available at: http://127.0.0.1:58433/devtools/?uri=ws://127.0.0.1:58433/ws
** Signaled to run test queries on http://127.0.0.1:58433//

>testee>out> 
>testee>out> A<C>
Done loading VM

eval_issue_49209_test.dart [1/2]

eval_issue_49209_test.dart [2/2]

[InstanceRef id: objects/4/0, kind: String, identityHashCode: 0, classRef: [ClassRef id: classes/94, name: _OneByteString, library: [LibraryRef id: libraries/@0150898, name: dart.core, uri: dart:core]]]

All service tests completed successfully.

** Killing script

00:06 �[32m+1�[0m: All tests passed!�[0m

** Process exited: -1

stderr:
../../runtime/platform/threads.h: 55: error: expected: owner_ == kUnowned
version=3.8.0-edge (main) (Unknown timestamp) on "windows_x64"
pid=4320, thread=7332, isolate_group=(nil)(0000000000000000), isolate=(nil)(0000000000000000)
os=windows, arch=x64, comp=no, sim=no
isolate_instructions=0, vm_instructions=0
fp=1647d6f918, sp=1647d6f918, pc=7ff7aaba8702
  pc 0x00007ff7aaba8702 fp 0x0000001647d6f918 Unknown symbol
-- End of DumpStackTrace

--- Re-run this test:
python3 tools/test.py -n vm-win-debug-x64 pkg/vm_service/test/eval_issue_49209_test

https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket/8719872653958537345/+/u/test_results/ignored_flaky_test_failure_logs