Skip to content

Strange behavoir when EnableDateTimeConversion is activ #652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DavidBal opened this issue Apr 3, 2025 · 14 comments
Open

Strange behavoir when EnableDateTimeConversion is activ #652

DavidBal opened this issue Apr 3, 2025 · 14 comments
Assignees
Labels

Comments

@DavidBal
Copy link

DavidBal commented Apr 3, 2025

Hi,

i have the following sample code:

internal class Program
{
    static void Main(string[] args)
    {
        //Promise Task conversion
        using (V8ScriptEngine engine = new V8ScriptEngine(V8ScriptEngineFlags.EnableDateTimeConversion))
        {
            engine.AddHostType("TestClass", typeof(TestClass));

            object result = engine.Evaluate(new DocumentInfo() { Category = ModuleCategory.Standard },
                """
                    
                    TestClass.Output("TestStart");

                    const date1 = TestClass.GetDate(1);

                    TestClass.Output(format(date1));
                    TestClass.Output(format2(date1));

                    const date2 = TestClass.GetDate(2);                    
                    TestClass.Output(format(date2));
                    TestClass.Output(format2(date2));

                    const date3 = TestClass.GetDate(3);

                    TestClass.Output(format(date3));
                    TestClass.Output(format2(date3));

                    const date4 = TestClass.GetDate(4);

                    TestClass.Output(format(date4));
                    TestClass.Output(format2(date4));

                    TestClass.Output("TestEnd");

                    function format(date) {
                        return JSON.stringify(date);
                    }

                    function format2(date){
                        return `${date.getFullYear()}-${date.getMonth() + 1}-${date.getDate()} ${date.getHours()}:${date.getMinutes()}:${date.getSeconds()} (${date.getTimezoneOffset()})`
                    }

                """
                );

        }
    }
}

public static class TestClass
{
    public static void Output(string message)
    {
        Console.WriteLine(message);
    }

    public static DateTime GetDate(int number)
    {
        DateTime dateTime;

        switch(number)
        {
            case 1:
                dateTime = DateTime.Now;
                break;
            case 2:
                dateTime = DateTime.UtcNow;
                break;
            case 3:
                dateTime = new DateTime(1972, 11, 23, 0, 0 , 0, DateTimeKind.Local);
                break;
            case 4:
                dateTime = new DateTime(1965, 10, 09,  00, 0, 0, DateTimeKind.Local);
                break;
            default:
                dateTime = DateTime.MinValue;
                break;
        }

        return dateTime;
    }
}

The result is:

TestStart
"2025-04-03T12:00:36.717Z"
2025-4-3 14:0:36 (-120)
"2025-04-03T12:00:36.722Z"
2025-4-3 14:0:36 (-120)
"1972-11-22T23:00:00.000Z"
1972-11-23 0:0:0 (-60)
"1965-10-08T22:00:00.000Z"
1965-10-8 23:0:0 (-60)
TestEnd

For me the last value looks wrong, the expected value would be:

"1965-10-08T23:00:00.000Z"
1965-10-9 0:0:0 (-60)

EDIT:
In the last output pair the UTC value is "1965-10-08T 22 :00:00.000Z" but expected would be "1965-10-08T 23 :00:00.000Z".

Kind regards,
David

@DavidBal
Copy link
Author

DavidBal commented Apr 3, 2025

I did some deeper research and found out that it happens in all the years before 1980 and in that year in germany the summertime was introduced.

Here is my extended sample code:

    static void Main(string[] args)
    {
        //Promise Task conversion
        using (V8ScriptEngine engine = new V8ScriptEngine(V8ScriptEngineFlags.EnableDateTimeConversion))
        {
            engine.AddHostType("TestClass", typeof(TestClass));

            object result = engine.Evaluate(new DocumentInfo() { Category = ModuleCategory.Standard },
                """
                    
                    TestClass.Output("TestStart");

                    const date1 = TestClass.GetDate(1);

                    TestClass.Output(format(date1));

                    const date2 = TestClass.GetDate(2);                    
                    TestClass.Output(format(date2));
                    TestClass.Output(format2(date2));

                    const date3 = TestClass.GetDate(3);

                    TestClass.Output(format(date3));
                    TestClass.Output(format2(date3));

                    const date4 = TestClass.GetDate(4);

                    TestClass.Output(format(date4));
                    TestClass.Output(format2(date4));

                    TestClass.Output("TestEnd");

                    for(let i = 1965; i <= 2026; i++) {
                        const dateyear = TestClass.GetDateByYear(i);

                        TestClass.Output(format(dateyear));
                        TestClass.Output(format2(dateyear));
                    }

                    function format(date) {
                        return JSON.stringify(date);
                    }

                   function format2(date){
                            return `${date.getFullYear()}-${date.getMonth() + 1}-${date.getDate()} ${date.getHours()}:${date.getMinutes()}:${date.getSeconds()} (${date.getTimezoneOffset()})`
                        }

                """
                );

        }
    }
}

public static class TestClass
{
    public static void Output(string message)
    {
        Console.WriteLine(message);
    }

    public static DateTime GetDate(int number)
    {
        DateTime dateTime;

        switch(number)
        {
            case 1:
                dateTime = DateTime.Now;
                break;
            case 2:
                dateTime = DateTime.UtcNow;
                break;
            case 3:
                dateTime = new DateTime(1972, 11, 23, 0, 0 , 0, DateTimeKind.Local);
                break;
            case 4:
                dateTime = new DateTime(1965, 09, 09,  00, 0, 0, DateTimeKind.Local);
                break;
            default:
                dateTime = DateTime.MinValue;
                break;
        }

        return dateTime;
    }

    public static DateTime GetDateByYear(int year)
    {
        return new DateTime(year, 09, 09, 0, 0, 0, DateTimeKind.Local);
    }
}
```

@ClearScriptLib
Copy link
Collaborator

Hi @DavidBal,

I did some deeper research and found out that it happens in all the years before 1980 and in that year in germany the summertime was introduced.

OK, so... Is the behavior still unexpected? 😁

Thanks!

@DavidBal
Copy link
Author

DavidBal commented Apr 3, 2025

OK, so... Is the behavior still unexpected? 😁

I would say yes!

EDIT:
It works both in JS and C# like i would expect it, the problem only happens if the value is moved from c# to js via the auto conversion.

@ClearScriptLib
Copy link
Collaborator

It works both in JS and C# like i would expect it, the problem only happens if the value is moved from c# to js via the auto conversion.

Understood. Would you mind providing a minimal example with just one date-time value that demonstrates the problem? Sorry, we aren't clear on the issue, and the code produces different output here, presumably due to time zone differences. It might also be helpful to know your exact time zone.

Thanks!

@DavidBal
Copy link
Author

DavidBal commented Apr 3, 2025

Of course:

My timezone is W. Europe Standard Time / (UTC+01:00) Amsterdam, Berlin, Bern, Rom, Stockholm, Wien

        internal class Program
    {
        static void Main(string[] args)
        {
            //Promise Task conversion
            using (V8ScriptEngine engine = new V8ScriptEngine(V8ScriptEngineFlags.EnableDateTimeConversion))
            {
                engine.AddHostType("TestClass", typeof(TestClass));

                //(UTC+01:00) Amsterdam, Berlin, Bern, Rom, Stockholm, Wien
                TimeZoneInfo timeZone = TimeZoneInfo.Local;

                TimeZoneInfo germanTimeZone = TimeZoneInfo.FindSystemTimeZoneById("W. Europe Standard Time");

                object result = engine.Evaluate(new DocumentInfo() { Category = ModuleCategory.Standard },
                    """
                        
                        const date1 = TestClass.GetDate();

                        TestClass.Output(format(date1));
                        TestClass.Output(format2(date1));

                        function format(date) {
                            return JSON.stringify(date);
                        }

                        function format2(date){
                            return `${date.getFullYear()}-${date.getMonth() + 1}-${date.getDate()} ${date.getHours()}:${date.getMinutes()}:${date.getSeconds()} (${date.getTimezoneOffset()})`
                        }

                    """
                    );

            }
        }
    }

    public static class TestClass
    {
        public static void Output(string message)
        {
            Console.WriteLine(message);
        }

        public static object GetDate()
        {
            return new DateTime(1965, 09, 09,  00, 0, 0, DateTimeKind.Local);
        }
    }

Result:

"1965-09-08T22:00:00.000Z"
1965-9-8 23:0:0 (-60)

Expected:

"1965-09-08T23:00:00.000Z"
1965-9-9 0:0:0 (-60)

I hope this helps.

EDIT:
In my example in comment #652 (comment) is a error in the format2 function, sry for that. (That does not change the behavoir, that still happens.)

It should be:

                        function format2(date){
                            return `${date.getFullYear()}-${date.getMonth() + 1}-${date.getDate()} ${date.getHours()}:${date.getMinutes()}:${date.getSeconds()} (${date.getTimezoneOffset()})`
                        }

@ClearScriptLib
Copy link
Collaborator

ClearScriptLib commented Apr 3, 2025

Hi @DavidBal,

If we understand correctly, you expect new DateTime(1965, 9, 9, 0, 0, 0, DateTimeKind.Local) to convert to 1965-09-08T23:00:00.000Z when the local time zone is W. Europe Standard Time.

If that's correct, your expectation doesn't match .NET's DateTime conversion logic. Consider:

var time = new DateTime(1965, 9, 9, 0, 0, 0, DateTimeKind.Unspecified);
var zone = TimeZoneInfo.FindSystemTimeZoneById("W. Europe Standard Time");
var utc = TimeZoneInfo.ConvertTimeToUtc(time, zone);
Console.WriteLine(utc.ToString("O"));

This code produces the following output:

1965-09-08T22:00:00.0000000Z

Any idea why that's not in agreement with your expectation?

Thanks!

@DavidBal
Copy link
Author

DavidBal commented Apr 3, 2025

Now my brain is totally lost... 😄

const date = new Date(1965, 9, 9, 0, 0, 0);
console.log(date, date.getTimezoneOffset());

result:

[LOG]: Date: "1965-10-08T23:00:00.000Z",  -60 

https://www.typescriptlang.org/play/?ssl=2&ssc=45&pln=1&pc=1#code/MYewdgzgLgBAJgQygUxgXhmZB3GARJZACgEYBOANgFYAaGMuhmABjtZYEoBuAWAChQkEABtkAOmEgA5kUQo6c8VORQAKgEsAtsgBe4ZAHkAZkYgqiHbkA


Ok then the problem is between C# and js if i understand it correclty.

C# has a time offset 120
js of 60

I will do deeper research here...

I have already found a workaround, by disabling the auto conversation and providing a conversation function:

internal class Program
{
    static void Main(string[] args)
    {
        //Promise Task conversion
        using (V8ScriptEngine engine = new V8ScriptEngine(V8ScriptEngineFlags.EnableDateTimeConversion))
        {
            engine.AddHostType("TestClass", typeof(TestClass));

            //(UTC+01:00) Amsterdam, Berlin, Bern, Rom, Stockholm, Wien
            TimeZoneInfo timeZone = TimeZoneInfo.Local;

            TimeZoneInfo germanTimeZone = TimeZoneInfo.FindSystemTimeZoneById("W. Europe Standard Time");

            object result = engine.Evaluate(new DocumentInfo() { Category = ModuleCategory.Standard },
                """
                    
                    const date1 = TestClass.GetDate();

                    TestClass.Output(format(date1));
                    TestClass.Output(format2(date1));

                    function format(date) {
                        return JSON.stringify(date);
                    }

                    function format2(date){
                        return `${date.getFullYear()}-${date.getMonth() + 1}-${date.getDate()} ${date.getHours()}:${date.getMinutes()}:${date.getSeconds()} (${date.getTimezoneOffset()})`
                    }

                """
                );

        }

        using (V8ScriptEngine engine = new V8ScriptEngine())
        {
            engine.AddHostType("TestClass", typeof(TestClass));

            //(UTC+01:00) Amsterdam, Berlin, Bern, Rom, Stockholm, Wien
            TimeZoneInfo timeZone = TimeZoneInfo.Local;

            TimeZoneInfo germanTimeZone = TimeZoneInfo.FindSystemTimeZoneById("W. Europe Standard Time");

            object result = engine.Evaluate(new DocumentInfo() { Category = ModuleCategory.Standard },
                """
                    
                    const date1 = TestClass.GetDateFixed();

                    TestClass.Output(format(date1));
                    TestClass.Output(format2(date1));

                    function format(date) {
                        return JSON.stringify(date);
                    }

                    function format2(date){
                        return `${date.getFullYear()}-${date.getMonth() + 1}-${date.getDate()} ${date.getHours()}:${date.getMinutes()}:${date.getSeconds()} (${date.getTimezoneOffset()})`
                    }

                """
                );

        }
    }
}

public static class TestClass
{
    public static void Output(string message)
    {
        Console.WriteLine(message);
    }

    public static DateTime GetDate()
    {
        return new DateTime(1965, 09, 09,  00, 0, 0, DateTimeKind.Local);
    }

    public static object GetDateFixed()
    {
        DateTime date = GetDate();

        return V8ScriptEngine.Current.Evaluate(
            $"""
            new Date({date.Year}, {date.Month - 1}, {date.Day}, {date.Hour}, {date.Minute}, {date.Second}, {date.Microsecond});
            """
            );
    }
}

result:

"1965-09-08T22:00:00.000Z"
1965-9-8 23:0:0 (-60)
"1965-09-08T23:00:00.000Z"
1965-9-9 0:0:0 (-60)

As always thanks for your input, i still think there is a problem somewhere but i dont can catch it at moment.

Kind regards,
David

EDIT:
In js the there also is a timezone:

console.log(Intl.DateTimeFormat().resolvedOptions().timeZone) //Result for me: "Europe/Berlin" 

@DavidBal
Copy link
Author

DavidBal commented Apr 4, 2025

Hi @ClearScriptLib,

the problem is caused by the diffrenz in the offset in c# (120) and js (60).

That comes from diffrent timezone modells used in c# and js.

Do you see any way that could be corrected by clearscript?

Kind regards,
David

@ClearScriptLib
Copy link
Collaborator

ClearScriptLib commented Apr 4, 2025

Hi David,

the problem is caused by the diffrenz in the offset in c# (120) and js (60).

According to our (very crude) research, the JavaScript result is the correct one for October 9, 1965. Daylight Saving Time had been discontinued on October 3 of that year, but .NET doesn't seem to account for that, at least on Windows. DST was reintroduced in 1980, so JavaScript and .NET presumably agree for all dates after that, at least.

Do you see any way that could be corrected by clearscript?

As you can see, date-time conversion is a very tricky subject. Apparently, in Berlin, DST was abolished in 1949, then briefly reintroduced in the early 1960s, then abandoned again in 1965, to be adopted again in 1980. And that's just one time zone. Dealing with all that is a job for subject matter experts, and that's not us 😁.

One thing we could probably do is allow hosts to override ClearScript's DateTime-to-Date conversion somehow. Thoughts?

Cheers!

EDIT: As we suspected, .NET produces the correct result on Linux.

@DavidBal
Copy link
Author

DavidBal commented Apr 4, 2025

According to our (very crude) research, the JavaScript result is the correct one for October 9, 1965. Daylight Saving Time had been discontinued on October 3 of that year, but .NET doesn't seem to account for that, at least on Windows. DST was reintroduced in 1980, so JavaScript and .NET presumably agree for all dates after that, at least.

That matches with my research.

One thing we could probably do is allow hosts to override ClearScript's DateTime-to-Date conversion somehow. Thoughts?

That could help. Because then i can implement a workaround for this, with out searching for every place a DateTime is moved between c# and js.

EDIT: As we suspected, .NET produces the correct result on Linux.
😄

As always thank's for your support.

Kind regards,
David

@DonatJR
Copy link

DonatJR commented Apr 7, 2025

Hey @ClearScriptLib,

@DavidBal made me aware of this issue, since we are using a component of his which uses ClearScript, so we are indirectly affected as well.

First of all, thank you for your research and help confirming the conversion bug. A method to easily override the automatic conversion would be very much appreciated, as it is certainly better than not being able to do anything at all. :)

However, I would guess most ClearScript users are also not subject matter experts on this topic. Leaving the conversions to "us" might therefor produce more / different bugs down the line.
From your research (with .NET on Linux & JS having the correct result), could the behaviour not be considered an issue in .NET or Windows itself which should be fixed upstream by the appropriate team? I would assume they certainly have the experts needed for it.

Thanks!

@ClearScriptLib
Copy link
Collaborator

Hi @DonatJR,

However, I would guess most ClearScript users are also not subject matter experts on this topic. Leaving the conversions to "us" might therefor produce more / different bugs down the line.

Absolutely. The idea here is that hosts would delegate the calculation to another date-time conversion library, perhaps something like Noda Time.

From your research (with .NET on Linux & JS having the correct result), could the behaviour not be considered an issue in .NET or Windows itself which should be fixed upstream by the appropriate team?

Since the Linux version of .NET handles this correctly, it's most likely a Windows issue and probably not addressable due to long-term compatibility concerns. Nevertheless, we've posted a .NET runtime issue here.

By the way, it isn't the most efficient solution in the world, but with ClearScript it's possible to use V8 itself as a date-time conversion library. Let us know if you'd like an example.

Cheers!

@DonatJR
Copy link

DonatJR commented Apr 7, 2025

Hey, thanks for creating the issue on the dotnet repo and for keeping us updated! 👍🏼 :)

By the way, it isn't the most efficient solution in the world, but with ClearScript it's possible to use V8 itself as a date-time conversion library. Let us know if you'd like an example.

I am not sure if this is applicable to our goal. We want to pass Date(Time) values from C++ to JS, back and forth. The communication from C++ <> .NET / ClearScript is done via COM and .NET <> JS by using ClearScript. Users can set and read these Date(Time) values from both the C++ as well as the JS end and we want both sides to be consistent with each other, which is currently impacted by the conversion to .NET DateTime objects in between before passing them on to C++ or JS respectively.
If using V8 would be helpful in this scenario, we would be more than happy to get a hint or a small example on how to utilize it. Correctness beats efficiency😊

Cheers!

@ClearScriptLib
Copy link
Collaborator

ClearScriptLib commented Apr 9, 2025

Hi @DonatJR,

If using V8 would be helpful in this scenario, we would be more than happy to get a hint or a small example on how to utilize it.

Here's an example that patches .NET's ToUniversalTime and ToLocalTime methods using the Harmony patching library. The replacement methods use a private V8 instance to perform the conversion.

First, you'll need the Harmony NuGet package. Next, add this class somewhere near your application's startup code:

// using HarmonyLib;
internal static class DateTimePatcher {
    private static readonly V8ScriptEngine _engine = new();
    private static readonly ScriptObject _toUnixMs = (ScriptObject)_engine.Evaluate(@"(
        function (y, mo, d, h, mi, s, ms) {
            return new Date(y, mo - 1, d, h, mi, s, ms).valueOf();
        }
    )");
    private static readonly ScriptObject _toLocal = (ScriptObject)_engine.Evaluate(@"(
        function (ms) {
            var d = new Date(ms);
            return new Int32Array([
                d.getFullYear(), d.getMonth(), d.getDate(),
                d.getHours(), d.getMinutes(), d.getSeconds(), d.getMilliseconds()
            ]);
        }
    )");
    private static bool ToUniversalTime(DateTime __instance, out DateTime __result) {
        if (__instance.Kind == DateTimeKind.Utc) {
            __result = __instance;
        }
        else {
            var dt = DateTime.SpecifyKind(__instance, DateTimeKind.Local);
            var ms = Convert.ToInt64(_toUnixMs.InvokeAsFunction(dt.Year, dt.Month, dt.Day, dt.Hour, dt.Minute, dt.Second, dt.Millisecond));
            __result = DateTimeOffset.FromUnixTimeMilliseconds(ms).UtcDateTime;
        }
        return false;
    }
    private static bool ToLocalTime(DateTime __instance, out DateTime __result) {
        if (__instance.Kind == DateTimeKind.Local) {
            __result = __instance;
        }
        else {
            var ms = new DateTimeOffset(DateTime.SpecifyKind(__instance, DateTimeKind.Utc)).ToUnixTimeMilliseconds();
            var arr = new int[7];
            ((ITypedArray<int>)_toLocal.InvokeAsFunction(ms)).Read(0, 7, arr, 0);
            __result = new DateTime(arr[0], arr[1] + 1, arr[2], arr[3], arr[4], arr[5], arr[6], DateTimeKind.Local);
        }
        return false;
    }
    public static void Patch() {
        var harmony = new Harmony(typeof(DateTimePatcher).FullName);
        harmony.Patch(
            typeof(DateTime).GetMethod(nameof(DateTime.ToUniversalTime)),
            typeof(DateTimePatcher).GetMethod(nameof(ToUniversalTime), BindingFlags.NonPublic | BindingFlags.Static)
        );
        harmony.Patch(
            typeof(DateTime).GetMethod(nameof(DateTime.ToLocalTime)),
            typeof(DateTimePatcher).GetMethod(nameof(ToLocalTime), BindingFlags.NonPublic | BindingFlags.Static)
        );
    }
}

Then call the Patch method before you do anything else. The trick with patching a system class like DateTime is doing it before its methods are inlined or optimized away in Release mode.

Here's a simple application that demonstrates the patch:

static class Program {
    static Program() {
        DateTimePatcher.Patch();
    }
    static void Main() {
        var localTime = new DateTime(1965, 10, 9, 0, 0, 0, 0, DateTimeKind.Local);
        var utc = localTime.ToUniversalTime();
        Console.WriteLine(utc.ToString("O"));
        Console.WriteLine(utc.ToLocalTime());
    }
}

Note that DateTime.ToUniversalTime is what ClearScript uses during automatic DateTime-to-Date conversion, so the patch will affect it as well.

Good luck!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants