Store array in TQueue possible? - delphi

Question

Having problem storing an array in a TQueue. Any idea where I go wrong?
Code works fine in Delphi XE 5 but not in Delphi 10 Seattle.
(I can't decide if this is a bug or how it should work. Tried searching embarcadero for clues but failed.)
procedure TForm1.Button1Click(Sender: TObject);
var
FData: TQueue<TBytes>;
FsData: TQueue<String>;
arr: TBytes;
begin
FData := TQueue<TBytes>.Create;
FsData := TQueue<String>.Create;
try
setlength(arr, 3);
arr[0] := 1;
arr[1] := 2;
arr[2] := 3;
FData.Enqueue(arr);
Memo1.Lines.Add('Count, array:' + IntToStr(FData.Count)); // 0?
FsData.Enqueue('asada');
Memo1.Lines.Add('Count, string:' + IntToStr(FsData.Count)); // 1
finally
FData.Free;
FsData.Free;
end;
end;

Answer 1

This is a defect introduced in XE8. Here's the simplest reproduction that I can produce.
{$APPTYPE CONSOLE}
uses
System.Generics.Collections;
var
Queue: TQueue<TArray<Byte>>;
begin
Queue := TQueue<TArray<Byte>>.Create;
Queue.Enqueue(nil);
Writeln(Queue.Count);
end.
The output is 1 in XE7 and 0 in XE8 and Seattle.
This has already been reported to Embarcadero: RSP-13196.
The implementation of Enqueue looks like this:
procedure TQueue<T>.Enqueue(const Value: T);
begin
if IsManagedType(T) then
if (SizeOf(T) = SizeOf(Pointer)) and (GetTypeKind(T) <> tkRecord) then
FQueueHelper.InternalEnqueueMRef(Value, GetTypeKind(T))
else
FQueueHelper.InternalEnqueueManaged(Value)
else
case SizeOf(T) of
1: FQueueHelper.InternalEnqueue1(Value);
2: FQueueHelper.InternalEnqueue2(Value);
4: FQueueHelper.InternalEnqueue4(Value);
8: FQueueHelper.InternalEnqueue8(Value);
else
FQueueHelper.InternalEnqueueN(Value);
end;
end;
When T is a dynamic array, the FQueueHelper.InternalEnqueueMRef branch is chosen. This in turn looks like this:
procedure TQueueHelper.InternalEnqueueMRef(const Value; Kind: TTypeKind);
begin
case Kind of
TTypeKind.tkUString: InternalEnqueueString(Value);
TTypeKind.tkInterface: InternalEnqueueInterface(Value);
{$IF not Defined(NEXTGEN)}
TTypeKind.tkLString: InternalEnqueueAnsiString(Value);
TTypeKind.tkWString: InternalEnqueueWideString(Value);
{$ENDIF}
{$IF Defined(AUTOREFCOUNT)}
TTypeKind.tkClass: InternalEnqueueObject(Value);
{$ENDIF}
end;
end;
Note that there is no entry for TTypeKind.tkDynArray. Because these two methods are inlined, the inliner manages to compress it all down to nothing. No action is performed when you Enqueue a dynamic array.
Back in the good old days of XE7 the code looked like this:
procedure TQueue<T>.Enqueue(const Value: T);
begin
if Count = Length(FItems) then
Grow;
FItems[FHead] := Value;
FHead := (FHead + 1) mod Length(FItems);
Inc(FCount);
Notify(Value, cnAdded);
end;
No scope for type specific defects there.
I don't think that there's an easy workaround for you. Perhaps the most expedient way to proceed is to take the code for the XE7 TQueue and use that in place of the broken implementation from XE8 and Seattle. For the record, I've given up on the Embarcadero generic collections and use my own classes.
The back story here is that in XE8, Embarcadero decided to address a deficiency in their implementation of generics. Whenever you instantiate a generic type, copies of all the methods are created. For some methods, identical code is generated for different instantiations.
So it is quite common for TGeneric<TFoo>.DoSomething and TGeneric<TBar>.DoSomething to have identical code. Other compilers for other languages, C++ templates, .net generics, etc., recognise this duplication and merge together identical generic methods. The Delphi compiler does not. The end result is a larger executable than strictly necessary.
In XE8 Embarcadero decided to tackle this in what I regard was utterly the wrong way. Instead of attacking the root cause of the issue, the compiler, they decided to change the implementation of their generic collection classes. If you look at the code in Generics.Collections, you will see that it has been completely re-written in XE8. Where previously the code from XE7 and earlier was readable, from XE8 it is now exceedingly complex and opaque. This decision had the following consequences:
The complex code contained many errors. Many of these were found shortly after XE8 was released and have been fixed. You have stumbled upon another defect. One thing that we have learnt is that Embarcadero's internal test suite does not exercise their collection classes sufficiently. It is manifestly clear that their tests are inadequate.
By changing their library rather than the compiler, they have patched up the RTL classes. The original issue with generic code bloat remains for third party classes. Had Embarcadero fixed the issue at source then not only could they have retained the simple and correct collection class code from XE7, but all third generic code would have benefited.