I have the following structures in Delphi 2009:
type
IndiReportIndi = record
IndiName: string;
NameNum: integer;
ReportIndiName: string;
end;
var
XRefList: array of IndiReportIndi;
where XRefList is a dynamic array.
I want to save XRefList to a FileStream. How do I do that AND include all the IndiName and ReportIndiName strings so that they will all be retrievable again when I later load from that FileStream?
type
IndiReportIndi = record
IndiName: string;
NameNum: integer;
ReportIndiName: string;
procedure SaveToStream(Stream: TStream);
procedure LoadFromStream(Stream: TStream);
end;
type
TXRefList = array of IndiReportIndi;
function LoadString(Stream: TStream): string;
var
N: Integer;
begin
Result:= '';
Stream.ReadBuffer(N, SizeOf(Integer));
if N > 0 then begin
SetLength(Result, N);
// Stream.ReadBuffer(Result[1], N * SizeOf(Char));
// fast version - see comment by A.Bouchez
Stream.ReadBuffer(Pointer(Result)^, N * SizeOf(Char));
end;
end;
procedure SaveString(Stream: TStream; const S: string);
var
N: Integer;
begin
N:= Length(S);
Stream.WriteBuffer(N, SizeOf(Integer));
if N > 0 then
// Stream.WriteBuffer(S[1], N * SizeOf(Char));
// fast version - see comment by A.Bouchez
Stream.WriteBuffer(Pointer(S)^, N * SizeOf(Char));
end;
procedure IndiReportIndi.LoadFromStream(Stream: TStream);
var
S: string;
begin
IndiName:= LoadString(Stream);
Stream.ReadBuffer(NameNum, SizeOf(Integer));
ReportIndiName:= LoadString(Stream);
end;
procedure IndiReportIndi.SaveToStream(Stream: TStream);
begin
SaveString(Stream, IndiName);
Stream.WriteBuffer(NameNum, SizeOf(Integer));
SaveString(Stream, ReportIndiName);
end;
function LoadXRefList(Stream: TStream): TXRefList;
var
N: Integer;
I: Integer;
begin
Stream.ReadBuffer(N, SizeOf(Integer));
if N <= 0 then Result:= nil
else begin
SetLength(Result, N);
for I:= 0 to N - 1 do
Result[I].LoadFromStream(Stream);
end;
end;
procedure SaveXRefList(Stream: TStream; const List: TXRefList);
var
N: Integer;
I: Integer;
begin
N:= Length(List);
Stream.WriteBuffer(N, SizeOf(Integer));
for I:= 0 to N - 1 do
List[I].SaveToStream(Stream);
end;
Use: http://code.google.com/p/kblib/
type
IndiReportIndi = record
IndiName: string;
NameNum: integer;
ReportIndiName: string;
end;
TXRefList = array of IndiReportIndi;
var
XRefList: TXRefList;
To save whole XRefList to stream use:
TKBDynamic.WriteTo(lStream, XRefList, TypeInfo(TXRefList));
To load it back:
TKBDynamic.ReadFrom(lStream, XRefList, TypeInfo(TXRefList));
var
S: TStream;
W: TWriter;
A: array of IndiReportIndi;
E: IndiReportIndi;
...
begin
S := nil;
W := nil;
try
S := TFileStream.Create(...);
W := TWriter.Create(S);
W.WriteInteger(Length(A));
for E in A do
begin
W.WriteString(E.IndiName);
W.WriteInteger(E.NameNum);
W.WriteString(E.ReportIndiName);
end;
finally
W.Free;
S.Free
end;
end;
To read those data you use a TReader and ReadInteger/ReadString.
Simple TFileStreamEx to save variables
http://forum.chertenok.ru/viewtopic.php?t=4642
Work with them from 2005th year, without any problems...
Just for the answer to be accurate:
Consider taking a look at the TDynArray wrapper, which is able to serialize any record into binary, and also to/from dynamic arrays.
There are a lot of TList-like methods, including new methods (like hashing or binary search).
Very optimized for speed and disk use, works for Delphi 5 up to XE2 - and Open Source.
See also How to store dynamic arrays in a TList?
Related
I have an ini file which contains the following:
[Colours]
1 = Red
2 = Blue
3 = Green
4 = Yellow
In my app I have a TComboBox which I would like to populate with the colours in the ini file.
Does anyone know how I'd go about this?
Thanks,
You can get a list of names in a section by using TIniFile.ReadSection() and then iterate to get the values:
procedure TForm1.LoadFile(const AFilename: String);
var
I: TIniFile;
L: TStringList;
X: Integer;
N: String;
V: String;
begin
I:= TIniFile.Create(AFilename);
try
L:= TStringList.Create;
try
ComboBox1.Items.Clear;
I.ReadSection('Colours', L);
for X := 0 to L.Count-1 do begin
N:= L[X]; //The Name
V:= I.ReadString('Colours', N, ''); //The Value
ComboBox1.Items.Add(V);
end;
finally
L.Free;
end;
finally
I.Free;
end;
end;
As an alternative, you could also dump the name/value pairs within the section into a single TStringList and read each value using the string list's built-in capabilities...
procedure TForm1.LoadFile(const AFilename: String);
var
I: TIniFile;
L: TStringList;
X: Integer;
N: String;
V: String;
begin
I:= TIniFile.Create(AFilename);
try
L:= TStringList.Create;
try
ComboBox1.Items.Clear;
I.ReadSectionValues('Colours', L);
for X := 0 to L.Count-1 do begin
N:= L.Names[X]; //The Name
V:= L.Values[N]; //The Value
ComboBox1.Items.Add(V);
end;
finally
L.Free;
end;
finally
I.Free;
end;
end;
On a side-note, Ini files do not have spaces on either side of the = sign, unless of course you want that space as part of the actual name or value.
try this, without reading the file twice:
uses IniFiles;
procedure TForm1.Button1Click(Sender: TObject);
var
lIni : TIniFile;
i: Integer;
begin
lIni := TIniFile.Create('c:\MyFile.ini');
try
lIni.ReadSectionValues('Colours', ComboBox1.Items);
for i := 0 to ComboBox1.Items.Count - 1 do
ComboBox1.Items[i] := ComboBox1.Items.ValueFromIndex[i];
finally
FreeAndNil(lIni);
end;
end;
I need to save a TObjectList<TStrings> (or <TStringList>) in a TStream and then retrive it.
To be clear, how to apply SaveToStream and LoadFromStream to a TObjectList?
Try something like this:
procedure SaveListOfStringsToStream(List: TObjectList<TStrings>; Stream: TStream);
var
Count, I: Integer;
MStrm: TMemoryStream;
Size: Int64;
begin
Count := List.Count;
Stream.WriteBuffer(Count, SizeOf(Count));
if Count = 0 then Exit;
MStrm := TMemoryStream.Create;
try
for I := 0 to Count-1 do
begin
List[I].SaveToStream(MStrm);
Size := MStrm.Size;
Stream.WriteBuffer(Size, SizeOf(Size));
Stream.CopyFrom(MStrm, 0);
MStrm.Clear;
end;
finally
MStrm.Free;
end;
end;
procedure LoadListOfStringsFromStream(List: TObjectList<TStrings>; Stream: TStream);
var
Count, I: Integer;
MStrm: TMemoryStream;
Size: Int64;
SList: TStringList;
begin
Stream.ReadBuffer(Count, SizeOf(Count));
if Count <= 0 then Exit;
MStrm := TMemoryStream.Create;
try
for I := 0 to Count-1 do
begin
Stream.ReadBuffer(Size, SizeOf(Size));
SList := TStringList.Create;
try
if Size > 0 then
begin
MStrm.CopyFrom(Stream, Size);
MStrm.Position := 0;
SList.LoadFromStream(MStrm);
MStrm.Clear;
end;
List.Add(SList);
except
SList.Free;
raise;
end;
end;
finally
MStrm.Free;
end;
end;
Alternatively:
procedure SaveListOfStringsToStream(List: TObjectList<TStrings>; Stream: TStream);
var
LCount, SCount, Len, I, J: Integer;
SList: TStrings;
S: UTF8String;
begin
LCount := List.Count;
Stream.WriteBuffer(LCount, SizeOf(LCount));
if LCount = 0 then Exit;
for I := 0 to LCount-1 do
begin
SList := List[I];
SCount := SList.Count;
Stream.WriteBuffer(SCount, SizeOf(SCount));
for J := 0 to SCount-1 do
begin
S := UTF8String(SList[J]);
// or, if using Delphi 2007 or earlier:
// S := UTF8Encode(SList[J]);
Len := Length(S);
Stream.WriteBuffer(Len, SizeOf(Len));
Stream.WriteBuffer(PAnsiChar(S)^, Len * SizeOf(AnsiChar));
end;
end;
end;
procedure LoadListOfStringsFromStream(List: TObjectList<TStrings>; Stream: TStream);
var
LCount, SCount, Len, I, J: Integer;
SList: TStrings;
S: UTF8String;
begin
Stream.ReadBuffer(LCount, SizeOf(LCount));
for I := 0 to LCount-1 do
begin
Stream.ReadBuffer(SCount, SizeOf(SCount));
SList := TStringList.Create;
try
for J := 0 to SCount-1 do
begin
Stream.ReadBuffer(Len, SizeOf(Len));
SetLength(S, Len);
Stream.ReadBuffer(PAnsiChar(S)^, Len * SizeOf(AnsiChar));
SList.Add(String(S));
// or, if using Delphi 2007 or earlier:
// SList.Add(UTF8Decode(S));
end;
List.Add(SList);
except
SList.Free;
raise;
end;
end;
end;
What's in your list?
It depends on what type of objects you have in your objectlist.
You loop over the list and save each item in turn.
However the objects inside your list need to have a SaveToStream method.
For reasons unknown SaveToStream is not a method of TPersistent, instead it is implemented independently in different classes.
Test for stream support
If the VCL were built with interfaces in mind, in newer versions has been solved with the IStreamPersist interface.
If all your stuff in the list descents from a base class that has streaming built-in (e.g. TComponent) then there is no problem and you can just use TComponent.SaveToStream.
type
TStreamableClass = TStrings; //just to show that this does not depend on TStrings.
procedure SaveToStream(List: TObjectList; Stream: TStream);
var
i: integer;
begin
for i:= 0 to List.Count -1 do begin
if List[i] is TStreamableClass then begin
TStreamableClass(List[i]).SaveToStream(Stream);
end;
end; {for i}
end;
Add stream support
If you have items in your list that do not derive from a common streamable ancestor then you'll have to have multiple if list[i] is TX tests in your loop.
If the object does not have a SaveToStream method, but you have enough knowledge of the class to implement it yourself, then you have twothree options.
A: implement a class helper that adds SaveToStream to that class or B: add a descendent class that implements that option.
If these are your own objects, then see option C: below.
type
TObjectXStreamable = class(TObjectX)
public
procedure SaveToStream(Stream: TStream); virtual;
procedure LoadFromStream(Stream: TStream); virtual;
end;
procedure SaveToStream(List: TObjectList; Stream: TStream);
...
if List[i] is TObjectX then TObjectXStreamable(List[i]).SaveToStream(Stream);
...
Note that this approach fails if TObjectX has subclasses with additional data. The added streaming will not know about this extra data.
Option C: implement System.Classes.IStreamPersist
type
IStreamPersist = interface
['<GUID>']
procedure SaveToStream(Stream: TStream);
procedure LoadFromStream(Stream: TStream);
end;
//enhance your streamable objects like so:
TInterfaceBaseObject = TInterfacedObject //or TSingletonImplementation
TMyObject = class(TInterfaceBaseObject, IStreamPersist)
procedure SaveToStream(Stream: TStream); virtual;
procedure LoadFromStream(Stream: TStream); virtual;
See: Bypassing (disabling) Delphi's reference counting for interfaces
You test the IStreamPersist support using the supports call.
if Supports(List[i], IStreamPersist) then (List[i] as IStreamPersist).SaveToStream(Stream);
If you have a newer version of Delphi consider using a generic TObjectList, that way you can limit your list to: MyList: TObjectList<TComponent>;
Now you can just call MyList[i].SaveToStream, because Delphi knows that the list only contains (descendents of) TComponent.
You will need to create your own routine to do this: One for saving, the other for loading.
For saving, loop through the list, convert each pointer of the list into is hexadecimal (decimal, octal) then add a separator character like ','; When done write the string contain to the stream.
For loading, loop through the list, search for the first separator character, extract the value, convert it back as a pointer then add it to the list.
Procedure ObjListToStream(objList: TObjectList; aStream: TStream);
var
str: String;
iCnt: Integer;
Begin
if not assigned(aStream) then exit; {or raise exception}
for iCnt := 0 to objList.Count - 1 do
begin
str := str + IntToStr(Integer(objList.Items[iCnt])) + ',';
end;
aStream.Write(str[1], Length(str));
End;
Procedure StreamToObjList(objList: TObjectList; aList: String);
var
str: String;
iCnt: Integer;
iStart, iStop: Integer;
Begin
try
if not assigned(aStream) then exit; {or raise exception}
iStart := 0;
Repeat
iStop := Pos(',', aList, iStart);
if iStop > 0 then
begin
objList.Add(StrToInt(Copy(sList, iStart, iStop - iStart)));
iStart := iStop + 1;
end;
Until iStop = 0;
except
{something want wrong}
end;
End;
I haven't test it and wrote it from memory. But it should point you in the right direction.
I have a record type with name, login, external-ip, tags and a boolean. I want to send that information to other computer via UDP where I want to get it back into a variable of the same record type.
I already know how to send and receive simple strings with Indy's UDPClient/UDPServer.
But how to send a record data?
I also want to, if possible, pass this data to my encryption method codeSSL(s,k) and when received, pass to decodeSSL(s,k) but I will be very satisfied if you could answer my first question, which is more important.
On the sending side, you need to serialize your record data into a flat byte array, optionally encrypt those bytes, and then send them. On the receiving side, you would read the bytes, optionally decrypt them, and then serialize them back into a record. TIdUDPClient and TIdUDPServer has methods for reading/writing TIdBytes data, and the IdGlobal unit has functions for manipulating TIdBytes data.
For example:
Sender:
type
TMyRecord = record
Name: String;
Login: String;
ExternalIP: String;
Tags: String;
Flag: Boolean;
end;
procedure AppendStringToBuffer(var Bytes: TIdBytes; const S: String);
var
Tmp: TIdBytes;
Len: Byte;
begin
Tmp := ToBytes(S, enUTF8);
Len := Length(Tmp);
AppendByte(Bytes, Len);
AppendBytes(Bytes, Tmp);
end;
var
Rec: TMyRecord;
Buf: TIdBytes;
begin
Rec := ...;
AppendStringToBuffer(Buf, Rec.Name);
AppendStringToBuffer(Buf, Rec.Login);
AppendStringToBuffer(Buf, Rec.ExternalIP);
AppendStringToBuffer(Buf, Rec.Tags);
AppendByte(Buf, Ord(Rec.Flag));
// optionally encrypt the buffer...
MySocket.SendBuffer(TargetHost, TargetPort, Buf);
end;
Receiver:
type
TMyRecord = record
Name: String;
Login: String;
ExternalIP: String;
Tags: String;
Flag: Boolean;
end;
function ReadStringFromBuffer(const Bytes: TIdBytes; var Index: Integer): String;
var
Len: Integer;
begin
Len := Bytes[Index];
Inc(Index);
if Len > 0 then
begin
Result := BytesToString(Bytes, Index, Len, enUTF8);
Inc(Index, Len);
end else
Result := '';
end;
var
Rec: TMyRecord;
Buf: TIdBytes;
BufLen, Index: Integer;
SenderIP: String;
SenderPort: TIdPort;
begin
SetLength(Buf, 1025);
BufLen := MySocket.ReceiveBuffer(Buf, SenderIP, SenderPort);
if Buf <= 0 then Exit;
// optionally decrypt the buffer...
Index := 0;
Rec.Name := ReadStringFromBuffer(Buf, Index);
Rec.Login := ReadStringFromBuffer(Buf, Index);
Rec.ExternalIP := ReadStringFromBuffer(Buf, Index);
Rec.Tags := ReadStringFromBuffer(Buf, Index);
Rec.Flag := Buf[Index] <> $00;
...
end;
You should use a serialization library.
You can find some in this link, Delphi (win32) serialization libraries.
For a comparison benchmark between different serialization libraries, see New sample for JSON performance: mORMot vs SuperObject/XSuperObject/dwsJSON/DBXJSON.
This is an example using SuperObject for serializing/deserializing a record in a generic way.
program TestSerializer;
{$APPTYPE CONSOLE}
uses SuperObject;
type
Serializer = record
class function Serialize<T>(data: T): String; static;
class procedure Deserialize<T>(const jsonStr: String; var data: T); static;
end;
class procedure Serializer.Deserialize<T>(const jsonStr: String; var data: T);
var
ctx: TSuperRttiContext;
begin
ctx := TSuperRttiContext.Create;
try
data := ctx.AsType<T>(SO(jsonStr));
finally
ctx.Free;
end;
end;
class function Serializer.Serialize<T>(data: T): String;
var
ctx: TSuperRttiContext;
obj: ISuperObject;
begin
Result := '';
ctx := TSuperRttiContext.Create;
try
obj := ctx.AsJson<T>(data);
Result := obj.AsJson;
finally
ctx.Free;
end;
end;
type
TData = record
str: string;
int: Integer;
bool: Boolean;
flt: Double;
end;
var
data: TData;
jStr: String;
begin
data.str := 'Test';
data.int := 42;
data.bool := True;
data.flt := 3.14;
jStr := Serializer.Serialize<TData>(data);
WriteLn(jStr);
data.str := '';
Serializer.Deserialize<TData>(jStr,data);
ReadLn;
end.
If you are using Indy, there is a Send() method for sending string data.
You can find some examples of string encryption/decryption here: Delphi: simple string encryption.
To read a index file in a specific format, I cooked the following piece of code without considering byte ordering:
unit uCBI;
interface
uses
SysUtils,
Classes,
Generics.Collections;
type
TIndexList = class
private
FIndexList:TList<Cardinal>;
FOwnedStream:Boolean;
FMemoryStream: TMemoryStream;
function GetCount: Integer;
protected
public
constructor Create(AStream:TMemoryStream; OwnedStream:Boolean=True);
destructor Destroy; override;
function Add(const Value: Cardinal): Integer;
procedure Clear;
procedure SaveToFile(AFileName:TFileName);
procedure LoadFromFile(AFileName:TFileName);
property Count: Integer read GetCount;
end;
implementation
{ TIndexList }
function TIndexList.Add(const Value: Cardinal): Integer;
begin
Result := FIndexList.Add(Value)
end;
procedure TIndexList.Clear;
begin
FIndexList.Clear;
end;
constructor TIndexList.Create(AStream: TMemoryStream; OwnedStream: Boolean);
begin
FMemoryStream := AStream;
FOwnedStream := OwnedStream;
FIndexList := TList<Cardinal>.Create;
end;
destructor TIndexList.Destroy;
begin
if (FOwnedStream and Assigned(FMemoryStream)) then
FMemoryStream.Free;
FIndexList.Free;
//
inherited;
end;
function TIndexList.GetCount: Integer;
begin
Result := FIndexList.Count;
end;
procedure TIndexList.LoadFromFile(AFileName: TFileName);
var
lMemoryStream:TMemoryStream;
lCount:Cardinal;
begin
lMemoryStream := TMemoryStream.Create;
try
lMemoryStream.LoadFromFile(AFileName);
lMemoryStream.ReadBuffer(lCount,SizeOf(Cardinal));
if (lCount = Cardinal((lMemoryStream.Size-1) div SizeOf(Cardinal))) then
begin
FMemoryStream.Clear;
lMemoryStream.Position :=0;
FMemoryStream.CopyFrom(lMemoryStream,lMemoryStream.Size)
end else
raise Exception.CreateFmt('Corrupted CBI file: %s',[ExtractFileName(AFileName)]);
finally
lMemoryStream.Free;
end;
end;
procedure TIndexList.SaveToFile(AFileName: TFileName);
var
lCount:Cardinal;
lItem:Cardinal;
begin
FMemoryStream.Clear;
lCount := FIndexList.Count;
FMemoryStream.WriteBuffer(lCount,SizeOf(Cardinal));
for lItem in FIndexList do
begin
FMemoryStream.WriteBuffer(lItem,SizeOf(Cardinal));
end;
//
FMemoryStream.SaveToFile(AFileName);
end;
end.
It tested it and seems to work well as needed. Great was my suprise when I pursue extensive tests with real sample file. In fact the legacy format was devised with Amiga computer with a different byte ordering.
My Question:
How can I fix it ?
I want to keep the code unchanged and wonder wether a decorated TMemorySream will do so that I can transparently switch between big endian and little endian.
To change 'endianness' of Cardinals you can use the following:
function EndianChange(Value: Cardinal): Cardinal;
var
A1: array [0..3] of Byte absolute Value;
A2: array [0..3] of Byte absolute Result;
I: Integer;
begin
for I:= 0 to 3 do begin
A2[I]:= A1[3 - I];
end;
end;
If you want to keep your code unchanged, you can write your own TMemoryStream descendant and override its Read and Write methods using the above function, like that:
function TMyMemoryStream.Read(var Buffer; Count: Integer): Longint;
var
P: PCardinal;
I, N: Integer;
begin
inherited;
P:= #Buffer;
Assert(Count and 3 = 0);
N:= Count shr 2;
while N > 0 do begin
P^:= EndianChange(P^);
Inc(P);
Dec(N);
end;
end;
I developed the following function to convert strings to hex values.
function StrToHex(const S: String): String;
const
HexDigits: array[0..15] of Char = '0123456789ABCDEF';
var
I: Integer;
P1: PChar;
P2: PChar;
B: Byte;
begin
SetLength(Result, Length(S) * 2);
P1 := #S[1];
P2 := #Result[1];
for I := 1 to Length(S) do
begin
B := Byte(P1^);
P2^ := HexDigits[B shr 4];
Inc(P2);
P2^ := HexDigits[B and $F];
Inc(P1);
Inc(P2);
end;
end;
Now I was wondering whether there is a more efficient way to convert the strings?
Depending on your Delphi version:
D5-D2007
uses classes;
function String2Hex(const Buffer: Ansistring): string;
begin
SetLength(result, 2*Length(Buffer));
BinToHex(#Buffer[1], #result[1], Length(Buffer));
end;
D2009+
uses classes;
function String2Hex(const Buffer: Ansistring): string;
begin
SetLength(result, 2*Length(Buffer));
BinToHex(#Buffer[1], PWideChar(#result[1]), Length(Buffer));
end;
Try this one
function String2Hex(const Buffer: Ansistring): string;
var
n: Integer;
begin
Result := '';
for n := 1 to Length(Buffer) do
Result := LowerCase(Result + IntToHex(Ord(Buffer[n]), 2));
end;
I know this is a very old topic, but I feel like I kinda need to share my code regarding the question. For years I use my own HexEncode, very similar with Forlan's code up there, but just today I found a faster way to encode Hex. With my old HexEncode, encoding a 180kb binary file took about 50 seconds, while with this function it took up 6 seconds.
function getHexEncode(txt : AnsiString) : AnsiString;
var
a : integer ;
st : TStringStream;
buf : array [0..1] of AnsiChar;
tmp : ShortString;
begin
st := TStringStream.Create;
st.Size := Length(txt)*2;
st.Position := 0;
for a:=1 to Length(txt) do
begin
tmp := IntToHex(Ord(txt[a]),2);
buf[0] := tmp[1];
buf[1] := tmp[2];
st.Write(buf,2);
end;
st.Position := 0;
Result := st.DataString;
st.Free;
//Result := ''; //my old code
//for a:=1 to Length(txt) do Result := Result+IntToHex(Ord(txt[a]),2); //my old code
end;
It seems good enough, you could always have a byte->2 hex digits lookup table, but that (and similar optimizations) seems like overkill to me in most cases.
// StrToInt('$' + MyString);
Oops, did not read the question very good...