Delphi XE6 and up regression: “‘9999-12-31 23:59:59,1000’ is not a valid date and time” when passing a SOAP message with 9999-11-31T23:59:59.9999999; QC144171

A valid SOAP message with <urn:timeStamp>9999-11-31T23:59:59.9999999</urn:timeStamp> in a xs:dateTime field return '9999-12-31 23:59:59,1000' is not a valid date and time from a Delphi application with this SOAP response:

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/">
  <SOAP-ENV:Body>
    <SOAP-ENV:Fault>
      <faultcode>SOAP-ENV:Server</faultcode>
      <faultstring>'9999-12-31 23:59:59,1000' is not a valid date and time</faultstring>
      <faultactor/>
    </SOAP-ENV:Fault>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The reason is this exception:

exception class EConvertError with message ''9999-12-31 23:59:59,1000' is not a valid date and time'.

This is from a .NET based test case passing in timeStamp = DateTime.MaxValuewhich is handled perfectly fine by other SOAP web services tested.

I know about different resolutions of time stamps, but would never expect the 999.9999 milliseconds to be rounded up to 1000 as it is always safer to truncated away from an upper limit.

A test using Soap UI [WayBack] with this parameter finally worked (max 3 digits second fraction):

<urn:timeStamp>9999-12-31T23:59:59.999</urn:timeStamp>

The true origin of problem is in this method in the Soap.XSBuiltIns unit which has been unchanged since at least Delphi 7:

function TXSBaseTime.GetMilliSecond: Word;
begin
  Result := Round(FractionalSeconds*1000);
end;

The problem exposed itself because as of Delphi XE6 the core of function TXSBaseCustomDateTime.GetAsDateTime piece was changed from

Result := EncodeDateTime(Year, Month, Day, Hour, Minute, Second, 0);

to

Result := EncodeDateTime(Year, Month, Day, Hour, Minute, Second, Millisecond);

A combination of lack of test cases and understanding XML specifications failed to reveal this bug.

The standards specify (among others):

• '.' s+ (if present) represents the fractional seconds;
  The above is not limiting the amount of digits, not talking about milliseconds either.
• All ·minimally conforming· processors ·must· support year values with a minimum of 4 digits (i.e., YYYY) and a minimum fractional second precision of milliseconds or three decimal digits (i.e. s.sss). However, ·minimally conforming· processors ·may· set an application-defined limit on the maximum number of digits they are prepared to support in these two cases, in which case that application-defined maximum number ·must· be clearly documented.
  Delphi not only limits the fractional second precission, it changes the limit over time and does not document the limit. Three strikes…
• s -- represents a digit used in the time element "second". The two digits in a ss format can have values from 0 to 60. In the formats described in this specification the whole number of seconds ·may· be followed by decimal seconds to an arbitrary level of precision. This is represented in the picture by "ss.sss". A value of 60 or more is allowed only in the case of leap seconds.
  Given buggy the fractional second handling through milliseconds, the leap second handling is ripe for a test case as well.
  Strictly speaking, a value of 60 or more is not sensible unless the month and day could represent March 31, June 30, September 30, or December 31 in UTC. Because the leap second is added or subtracted as the last second of the day in UTC time, the long (or short) minute could occur at other times in local time. In cases where the leap second is used with an inappropriate month and day it, and any fractional seconds, should considered as added or subtracted from the following minute.

The reproduction is quite simple:

program TXSDateTime_DateTime_MaxValue_Bug;

{$APPTYPE CONSOLE}

uses
  SysUtils,
  XSBuiltIns;

type
  TXSCustomDateTimeHelper = class helper for TXSCustomDateTime
  public
    function MilliSecondFix: Word;
  end;

procedure Run;
var
  DateTime: TDateTime;
  MilliSecond: Word;
  XSDateTime: TXSDateTime;
begin
  XSDateTime := TXSDateTime.Create();
  try
    XSDateTime.XSToNative('9999-12-31T23:59:59.9999999'); // this is .NET DateTime.MaxValue passed over SOAP as xs:DateTime
    MilliSecond := XSDateTime.MilliSecond; // wrongly returns 1000
    MilliSecond := XSDateTime.MilliSecondFix; // Correctly returns 999
    DateTime := XSDateTime.AsDateTime; // works fine until Delphi XE5; throws exception in Delphi XE6 and up:
    // "exception class EConvertError with message ''9999-12-31 23:59:59,1000' is not a valid date and time"
  finally
    XSDateTime.Free();
  end;
end;

function TXSCustomDateTimeHelper.MilliSecondFix: Word;
begin
  Result := Millisecond;
  if (FractionalSeconds = 1000) then
    Result := 999;
end;

begin
  try
    Run();
  except
    on E:Exception do
      Writeln(E.Classname, ': ', E.Message);
  end;
end.

The leap second bug is also very easy to reproduce by replacing the core wit this, and – to no surprise – fails too:

    XSDateTime.XSToNative('9999-12-31T23:59:60.9999999'); // this allowed as per https://www.w3.org/TR/xmlschema-2/#dateTime
    // "s -- represents a digit used in the time element "second". The two digits in a ss format can have values from 0 to 60. In the formats described in this specification the whole number of seconds ·may· be followed by decimal seconds to an arbitrary level of precision. This is represented in the picture by "ss.sss". A value of 60 or more is allowed only in the case of leap seconds."
    DateTime := XSDateTime.AsDateTime; // breaks in any Delphi version with up until Delphi XE5 this message:
    // "exception class EConvertError with message '''9999/12/31 23:59:60.0'' is not a valid date and time'."
    // for Delphi XE6 and up with this message:
    // "exception class EConvertError with message ''9999-12-31 23:59:60,1000' is not a valid date and time'."

References:

• [Archive.is] DateTime.MaxValue Field (System)
• [Archive.is] c# – .net DateTime MaxValue is different once it is stored in database – Stack Overflow
• [Archive.is/WayBack] XML Schema Part 2: Datatypes Second Edition (not limiting the second fraction)
• [WayBack] IBM Knowledge Center – xs:dateTime (limiting the second fraction to 12 digits)

—jeroen

[Archive.is] Report No: 144171 Status: Reported
valid xs:DateTime boundaries (including second fractions close to 1 and leap seconds) are not handled correctly