The Delphi interface reference counting compiler bug that will never get fixed as it is marked “as designed”

A long time ago, I write a question [WayBack] delphi – Should the compiler hint/warn when passing object instances directly as const interface parameters? – Stack Overflow in 2010.

It was marked by (now former) compiler engineer Barry Kelly [WayBack1/WayBack2] as bug:

It’s a bug. The conversion from instance to interface reference in RunLeakCrash should be to a temporary variable, keeping it alive for the duration of RunLeakCrash.

Added to that was a comment that this has happened since at least Delphi 6, so I filed a bug WayBack QualityCentral Report #: 90482 The compiler should keep a hidden reference when passing freshly created object instances directly as const interface parameters.

Just for years later, it was marked with “As Designed” which means it is never going to be fixed, especially since in the mean time Embarcadero got rid of most the senior Delphi R&D team members and went down the path of hiring contractors.

The problem is that I run into the two manifestations every now and then, and it usually takes a long time debugging to zoom into the actual location of the spots.

First manifestation: crash

This is the bug in the first manifestation (by now the root interface is IInterface instead of IUnknown, and you usually have an interface derived from it):

procedure ReferTo(ValueReference: IUnknown);
begin
// remove the comments to reproduce it in earlier Delphi versions (I reproduced it in Delphi 5, 2007, 2009 and XE)
//  if not Assigned(ValueReference) then
//    ;
end;

procedure PrematureFreeCrash(const Reference: IUnknown);
begin
  ReferTo(Reference);
  ReferTo(Reference); // the second call will crash inside ReferTo because the object instance that Reference referes to just got destroyed
end;

procedure RunPrematureFreeCrash;
begin
  PrematureFreeCrash(TInterfacedObject.Create);
end;

The good thing is that the first manifestation will crash, especially if you run it under FastMM4 debug mode where freed memory will be filled with a $80808080 pattern, as you can see in these blog posts

• Delphi: when calling TThread.Synchronize, ensure the synchronised method handles exceptions
• Delphi: A few notes on tracking down a use-after free related issue involving interfaces crashing inside System._IntfClear.

Second manifestation: memory leak

The second manifestation is that you just access a member of the interface (like a method or property of an interface like IMyInterface) instead of calling ReferTo.

Now – since there is no reference counting, the object you created will never have _AddRef or _Release called, still have a FRefCount of zero (with which it was initialised) and live happily ever after.

In other words: now you have a memory leak that is very hard to track down.

Related but simpler bugs got solved

There have been quite a few similar bugs have been solved, for instance [WayBack] delphi – Unused interface reference is not destroyed – Stack Overflow.

I wonder what sets those apart from the above one.

Why it is hard to find the manifestations

The biggest problem on why the manifestations are hard to find is a three part thing:

• the use is often not limited to a few levels, so there can be a large stack trace you need to trace back to the cause if you get one at all
• the stack trace usually involves crashes around things like TInterfacedObject._Release which very often results in incomplete stack traces
• very often things like COM or OLE Automation are involved (hence my earlier reference to it from Links to debugging COM stuff for Office Automation in Delphi)

Workaround

Two work-arounds that both add extra reference counting::

1. Replace the const parameter with a non-const one (basically removing the const bit) which will incur automatic _AddRef and _Release reference counting in the method.
2. Store the value of the const parameter in another reference, which will incur manual _AddRef and automatic _Release reference counting in the method.

Both solutions will call the _AddRef and _Release on the interface.

Of course this incurs extra overhead, much more than the compiler based solution Barry Kelly wrote about would, but it at least introduces the reference counting

–jeroen