I am reading some Pascal code to debug an issue on a piece of hardware. I am not familiar with Pascal, but it doesn't seem too difficult to read with a little Googling.
However, I ran across this procedure definition:
procedure RS232ConfigTask; far;
begin
RS232Config.Init;
RS232Config.Process;
end;
What does the 'far' keyword mean?
I've tried to look it up. I found this remark on the GNU Pascal website:
The far directive can be appended to a procedure or function heading but is ignored by GPC. It is there for Borland compatibility, only. (Since the GNU compilers provide a flat memory model, the distinction between near and far pointers is void.)
I looked up near and far pointers, and found that it had something to do with segmented memory models, and referencing memory in other segments.
Okay.
This isn't a pointer though. So why would you mark a procedure 'far'?
I note that the procedure is called like so:
procedure Init;
begin
CreateTask(#RS232ConfigTask, 4096, 'Config 232');
CreateTask(#RS485ConfigTask, 4096, 'Config 485');
end;
If that helps.
Used as a procedure directive, far means that the subroutine must be called with a far call instruction which modifies the code segment and the instruction pointer. It means also that the call instruction pushes a far pointer (4 bytes, segment and offset) onto the stack. Further, to return to the caller, the subroutine uses a far ret instruction which expects a far pointer being pushed as a return address onto the stack.
For Turbo Pascal the following rule of thumb applies: Subroutines exported from a unit use implicitly the far calling convention as decribed above. Other subroutines use the near calling conventions (only offset modified by call/ret). To override this behaviour, near and far procedure directives were introduced.
'Far' is an old extension which it means - "this is a pointer that needs to be able to point at any address, not just things in the same segment as the code using it".
You can go and look at the wikipedia page and read about it if you want to know more information on it. Wikipedia page
Related
The following code should compile and does compile with many other types.
However, the compiler reports a "Constant object cannot be passed as var parameter" error - despite the variable quite obviously being a variable.
program CurrencyConstant;
{$APPTYPE CONSOLE}
var
GVar: Currency;
begin
FillChar(GVar, SizeOf(GVar), 0);
end.
Similarly, the same problem occurs with a local variable in a procedure.
procedure TestCurrency;
var
LVar: Currency;
begin
FillChar(LVar, SizeOf(LVar), 0);
end;
I suspect it has something to do with the fact that FillChar is a compiler magic procedure, and that Dest is an untyped var parameter. FillChar is the only routine I've found with this problem.
What causes this problem?
Are any other types affected?
In response to the inevitable "Why would you do that comments": We have a code generator that uses FillChar to generically initialise record structures & primitive types. It works with everything else, but unexpectedly failed with Currency. We do have workarounds, but it would be nice to understand the root cause, and know whether anything else is likely to cause us trouble.
Edit
From Jeroen's answer it is reasonable to conclude that the issue exists in all vesions of Delphi. Furthermore array's of Currency apparently exhibit a similar problem.
David's answer provides some nice workarounds.
One final workaround to consider is, modifying the generator to deal with Currency as a special case and simply set the Value := 0.
What causes the problem?
A compiler bug. Please submit a QC report.
Are any other types affected?
Maybe. Try some to find out.
As for a work around I would write it like this:
FillChar(Pointer(#LVar)^, SizeOf(LVar), 0);
or perhaps like this:
ZeroMemory(#LVar, SizeOf(LVar));
or even like this:
LVar := Default(Currency);
Personally I regard ZeroMemory as being more descriptive than FillChar.
As requested by Craig Young:
It still occurs in Delphi XE4.
Report No: 118866 Status: Reported
Cannot perform FillChar on Currency variables
https://web.archive.org/web/20150322021442/http://qc.embarcadero.com/wc/qcmain.aspx?d=118866
It is similar to
http://qc.embarcadero.com/wc/qcmain.aspx?d=87168 (not archived)
The workaround for this compiler bug for Delphi < 2009: use ZeroMemory or FillMemory from the Windows unit which works just as well as FillChar.
On the Delphi side, ZeroMemory and FillMemory use FillChar underneath which might be inlined as of Delphi 2006.
On the C++ side both use compiler macros.
It might be that this issue only happens with Currency because that is the only numeric compiler type that is scaled.
The issue does not reproduce with ordinal types, regular floating point types, and Comp.
Edit: The issue has been fixed in XE5 Update 2
I have discovered (the hard way) that if a file has a valid UTF-8 BOM but contains any invalid UTF8 encodings, and is read by any of the Delphi (2009+) encoding-enabled methods such as LoadFromFile, then the result is a completely empty file with no error indication. In several of my applications, I would prefer to simply lose a few bad encodings, even if I get no error report in this case either.
Debugging reveals that MultiByteToWideChar is called twice, first to get the output buffer size, then to do the conversion. But TEncoding.UTF8 contains a private FMBToWCharFlags value for these calls, and this is initialized with a MB_ERR_INVALID_CHARS value. So the call to get the charcount returns 0 and the loaded file is completely empty. Calling this API without the flag would 'silently drop illegal code points'.
My question is how best to weave through the nest of classes in the Encoding area to work around the fact that this is a private value (and needs to be, because it is a class var for all threads). I think I could add a custom UTF8 encoding, using the guidance in Marco Cantu's Delphi 2009 book. And it could optionally raise an exception if MultiByteToWideChar has returned an encoding error, after calling it again without the flag. But that does not solve the problem of how to get my custom encoding used instead of Tencoding.UTF8.
If I could just set this up as a default for the application at initialization, perhaps by actually modifying the class var for Tencoding.UFT8, this would probably be sufficient.
Of course, I need a solution without waiting to lodge a QC report asking for a more robust design, getting it accepted, and seeing it changed.
Any ideas would be very welcome. And can someone confirm this is still an issue for XE4, which I have not yet installed?
I ran into the MB_ERR_INVALID_CHARS issue when I first updated Indy to support TEncoding, and ended up implementing a custom TEncoding-derived class for UTF-8 handling to avoid specifying MB_ERR_INVALID_CHARS. I didn't think to use a class helper.
However, this issue is not just limited to UTF-8. Any decoding failure of any of the TEncoding classes will result in a blank result, not an exception being raised. Why Embarcadero chose that route, when most of the RTL/VCL uses exceptions instead, is beyond me. Not raising an exception on error caused a fair amount of issues in Indy that had to be worked around.
This can be done pretty simply, at least in Delphi XE5 (have not checked earlier versions). Just instantiate your own TUTF8Encoding:
procedure LoadInvalidUTF8File(const Filename: string);
var
FEncoding: TUTF8Encoding;
begin
FEncoding := TUTF8Encoding.Create(CP_UTF8, 0, 0);
// Instead of CP_UTF8, MB_ERR_INVALID_CHARS, 0
try
with TStringList.Create do
try
LoadFromFile(Filename, FEncoding);
// ...
finally
Free;
end;
finally
FEncoding.Free;
end;
end;
The only issue here is that the IsSingleByte property for the newly instantiated TUTF8Encoding is then incorrectly set to False, but this property is not currently used anywhere in the Delphi sources.
A partial workaround is to force the UTF8 encoding to suppress MB_ERR_INVALID_CHARS globally. For me, this avoids the need for raising an exception, because I find it makes MultiByteToWideChar not quite 'silent': it actually inserts $fffd characters (Unicode 'replacement character') which I can then find in the cases where this is important. The following code does this:
unit fixutf8;
interface
uses System.Sysutils;
type
TUTF8fixer = class helper for Tmbcsencoding
public
procedure setflag0;
end;
implementation
procedure TUTF8fixer.setflag0;
{$if CompilerVersion = 31}
asm
XOR ECX,ECX
MOV Self.FMBToWCharFlags,ECX
end;
{$else}
begin
Self.FMBToWCharFlags := 0;
end;
{$endif}
procedure initencoding;
begin
(Tencoding.UTF8 as TmbcsEncoding).setflag0;
end;
initialization
initencoding;
end.
A more useful and principled fix would require changing the calls to MultiByteToWideChar not to use MB_ERR_INVALID_CHARS, and to make an initial call with this flag so that an exception could be raised after the load is complete, to indicate that characters will have been replaced.
There are relevant QC reports on this issue, including 76571, 79042 and 111980. The first one has been resolved 'as designed'.
(Edited to work with Delphi Berlin)
Your "global" approach is not really global - it relies upon the assumption that all the code would only use one and the same instance of TUTF8Encoding. The same instance where you hacked the flags field.
But it would not work if one obtain TUTF8Encoding object(s) by other means than TEncoding.GetUTF8, for example in XE2 another method - TEncoding.GetEncoding(CP_UTF8) - would create a new instance of TUTF8Encoding instead of re-using FUTF8 shared one. Or some function might run TUTF8Encode.Create directly.
So i'd suggest two more approaches.
Approach with patching the class implementation, somewhat hacky. You introduce your own class for the sake of obtaining new "fixes" constructor body.
type TMyUTF8Encoding = class(TUTF8Encoding)
public constructor Create; override;
end;
This constructor would be the copycat of TUTF8Encoding.Create() implementation, except for setting the flag as you want it ( in XE2 it is done by calling another, inherited Create(x,y,z) so u would not need an access to the private field ) instead.
Then you can patch the stock TUTF8Encoding VMT overriding its virtual constructor to that new constructor of yours.
You may read Delphi documentation about "internal formats" and so forth, to get the VMT layout. You would also need calling VirtualProtect (or other platform-specific function) to remove protection from VMT memory area before patching and then to restore it.
Examples to learn from
How to change the implementation (detour) of an externally declared function
https://stackoverflow.com/a/1482802/976391
Or you may try using Delphi Detours library, hopefully it can patch virtual constructors. Then... it might be an overkill here to use that rather complex lib for that single goal.
After you hacked the TUTF8Encoding class do call the TEncoding.FreeEncodings to remove the already created shared instances (if any) if any and thus trigger recreating the UTF8 instances with your modifications.
Then, if you compile your program as a single monolithic EXE , without using runtime BPL modules, you just can copy the SysUtils.pas sources to your application folder and then to include that local copy into your project explicitly.
How to patch a method in Classes.pas
There you would change the very TUTF8Encoding implementation as you see fit in the sources and Delphi would use it.
This brain-deadly simplistic (hence - equally reliable) approach would not work though if your projects would be built to reuse rtlNNN.bpl runtime package instead of being monolithic.
I'm asking this because I'm out of good ideas...hoping for someone else's fresh perspective.
I have a user running our 32-bit Delphi application (compiled with BDS 2006) on a Windows 7 64-bit system. Our software was "working fine" until a couple weeks ago. Now suddenly it isn't: it throws an Access Violation while initializing (instancing objects).
We've had him reinstall all our software--starting all over from scratch. Same AV error. We disabled his anti-virus software; same error.
Our stack tracing code (madExcept) for some reason wasn't able to provide a stack trace to the line of the error, so we've sent a couple error logging versions for the user to install and run, to isolate the line which generates the error...
Turns out, it's a line which instances a simple TStringList descendant (there's no overridden Create constructor, etc.--basically the Create is just instancing a TStringList which has a few custom methods associated with the descendant class.)
I'm tempted to send the user yet another test .EXE; one which just instances a plain-vanilla TStringList, to see what happens. But at this point I feel like I'm flailing at windmills, and risk wearing out the user's patience if I send too many more "things to try".
Any fresh ideas on a better approach to debugging this user's problem? (I don't like bailing out on a user's problems...those tend to be the ones which, if ignored, suddenly become an epidemic that 5 other users suddenly "find".)
EDIT, as Lasse requested:
procedure T_fmMain.AfterConstruction;
begin
inherited;
//Logging shows that we return from the Inherited call above,
//then AV in the following line...
FActionList := TAActionList.Create;
...other code here...
end;
And here's the definition of the object being created...
type
TAActionList = class(TStringList)
private
FShadowList: TStringList; //UPPERCASE shadow list
FIsDataLoaded : boolean;
public
procedure AfterConstruction; override;
procedure BeforeDestruction; override;
procedure DataLoaded;
function Add(const S: string): Integer; override;
procedure Delete(Index : integer); override;
function IndexOf(const S : string) : Integer; override;
end;
implementation
procedure TAActionList.AfterConstruction;
begin
Sorted := False; //until we're done loading
FShadowList := TStringList.Create;
end;
I hate these kind of problems, but I reckon you should focus on what's happening recently BEFORE the object tries to get constructed.
The symptoms you describe sound like typical heap corruption, so maybe you have something like...
An array being written to outside bounds? (turn bounds checking on, if you have it off)
Code trying to access an object which has been deleted?
Since my answer above, you've posted code snippets. This does raise a couple of possible issues that I can see.
a: AfterConstruction vs. modified constructor:
As others have mentioned, using AfterConstruction in this way is at best not idiomatic. I don't think it's truly "wrong", but it's a possible smell. There's a good intro to these methods on Dr. Bob's site here.
b: overridden methods Add, Delete, IndexOf
I'm guessing these methods use the FshadowList item in some way. Is it remotely possible that these methods are being invoked (and thus using FShadowList) before the FShadowList is created? This seems possible because you're using the AfterConstruction methods above, by which time virtual methods should 'work'. Hopefully this is easy to check with a debugger by setting some breakpoints and seeing the order they get hit in.
You should never override AfterConstruction and BeforeDestruction methods in your programs. They are not meant for what you're doing with them, but for low-level VCL hacking (like reference adding, custom memory handling or such).
You should override the Create constructor and Destroy destructor instead and put your initialization code here, like such:
constructor TAActionList.Create;
begin
inherited;
// Sorted := False; // not necessary IMHO
FShadowList := TStringList.Create;
end;
Take a look at the VCL code, and all serious published Delphi code, and you'll see that AfterConstruction and BeforeDestruction methods are never used. I guess this is the root cause of your problem, and your code must be modified in consequence. It could be even worse in future version of Delphi.
Clearly there is nothing suspicious about what TAActionList is doing at time of construction. Even considering ancestor constructors and possible side-effects of setting Sorted := False indicate there shouldn't be a problem. I'm more interested in what's happening inside T_fmMain.
Basically something is happening that causes FActionList := TAActionList.Create; to fail, even though there is nothing wrong in the implementation of TAActionList.Create (a possibility is that the form may have been unexpectedly destroyed).
I suggest you try changing T_fmMain.AfterConstruction as follows:
procedure T_fmMain.AfterConstruction;
begin
//This is safe because the object created has no form dependencies
//that might otherwise need to be initialised first.
FActionList := TAActionList.Create;
//Now, if the ancestor's AfterConstruction is causing the problem,
//the above line will work fine, and...
inherited AfterConstruction;
//... your error will have shifted to one of these lines here.
//other code here
end;
If an environment issue with a component used by your form is causing it destroy the form during AfterConstruction, then it's the assignment of the new TAActionList.Create instance to FActionList that's actually causing the AV. Another way to test would be to first create the object to a local variable, then assign it to the class field: FActionList := LActionList.
Environment problems can be subtle. E.g. We use a reporting component which we discovered requires that a printer driver is installed, otherwise it prevents our application from starting up.
You can confirm the destruction theory by setting a global variable in the form's destructor. Also you may be able to output a stack trace from the destructor to confirm the exact sequence leading to the destruction of the form.
Our software was "working fine" until a couple weeks ago... suddenly become an epidemic that 5 other users suddenly "find".) :
Sounds like you need to do some forensic analysis, not debugging: You need to discover what changed in that user's environment to trigger the error. All the more so if you have other users with the same deployment that don't have the problem (sounds like that's your situation). Sending a user 'things to try' is one of the best ways to erode user confidence very quickly! (If there is IT support at the user site, get them involved, not the user).
For starters, explore these options:
*) If possible, I'd check the Windows Event Log for events that may have occurred on that machine around the time the problem arose.
*) Is there some kind of IT support person on the user's side that you can talk to about possible changes/problems in that user's environment?
*) Was there some kind of support issue/incident with that user around the time the error surfaced that may be connected to it, and/or caused some kind of data or file corruption particular to them?
(As for the code itself, I agree with #Warran P about decoupling etc)
Things to do when MadExcept is NOT Enough (which is rare, I must say):
Try Jedi JCL's JCLDEBUG instead. You might get a stack traceback with it, if you change out MadExcept for JCLDEBUG, and write directly the stack trace to the disk without ANY UI interaction.
Run a debug-viewer like MS/SysInternals debugview, and trace output things like the Self pointers of the objects where the problems are happening. I suspect that somehow an INVALID instance pointer is ending up in there.
Decouple things and refactor things, and write unit tests, until you find the really ugly thing that's trashing you. (Someone suggested heap corruption. I often find heap corruption goes hand in hand with unsafe ugly untested code, and deeply bound UI+model cascading failures.)
I am looking for hints on how to debugging a crash in an application that uses the MS XML wrappers in the Delphi VCL. I suspect memory corruption, or some kind of obscure evil thing happening between objects and interfaces, such as reference counting bugs, or heap corruption. The question is, in effect: how do I debug such a crash?
This particular code makes heavy internal use of and extends on the base XmlIntf Interfaces (IXMLNode). ISomethingCustom is an interface that extends IXMLNode. THe problem happens where we crash somewhere in a recursive function that is passed an ISomethingCustom which is also (or supports also, in interface terms) IXMLNode.
boolean UtilityFunction( aNode: ISomethingCustom ):Boolean;
begin
if not Assigned(aNode) then exit; // this works. great.
if not Assigned(aNode.ParentNode) then exit; // this DOES NOT WORK.
// code that blows up if aNode.ParentNode is not assigned.
end;
The situation is that the aNode is also IXMLNode, and IXMLNode.ParentNode value is assigned (not nil), and yet it points to a COM object that may have been freed, destroyed, or corrupted somehow. I am trying to figure out WHAT is going on when an interface pointer can appear to be valid, but the object behind it has been nuked somehow.
Checking Assigned(aNode.ParentNode) returns TRUE, even when, if you were to attempt a cast in the debugger (at runtime only, not in the code), like this:
inspect/evaluate aNode
inspect/evaluate TInterfacedObject(aNode).ClassName
(works in Delphi 2010, at least!)
now cast TWhateverClassNameYouGotBefore(aNode).
In the debugger I now see that this is NIL. WHich may mean that
the magic "casting interface back to
the object" feature that is new in
delphi 2010, is failing.
I believe I am trying to debug a problem where heaps are corrupted, or COM objects are corrupt on the heap, because of a reference counting problem.
I really think that nobody should ever have the situation arise where an interface appears valid, but the object underneath has been deleted. I really would like to know what to do, and what is going on.
Although you haven't shown it in your code, your comments seem to indicate that you're type-casting the interface variable to a class type. That's not allowed. I've described why:
Why can’t I cast an interface reference to an object reference?
Interface references and object references don't point to the same things. Therefore, calling a method on one when the compiler thinks you have the other will yield unexpected results. You were unlucky because the code continued to run instead of crashing with an access violation, which would have been a bigger indication that you were doing something wrong.
My article above concludes by suggesting you use the JclSysUtils.GetImplementorOfInterface function from the JCL if you have a Delphi-implemented interface and the interface offers no function of its own for revealling the underlying object.
Wild guess: Have you tried to put aNode.ParentNode in a local variable and use it in the rest of the Utilityfunction:
function UtilityFunction(aNode: ISomethingCustom): Boolean;
var
lParentNode: INode;
begin
if not Assigned(aNode) then exit; // this works. great.
lParentNode := aNode.ParentNode;
if not Assigned(lParentNode) then exit;
// code that uses lParentNode.
end;
My suggestion is to make sure that the ParentNode function is actually called in Assigned(aNode.ParentNode). There are some nasty corner-cases in Delphi where a procedure/function without arguments doesn't get called, but rather it's reference is taken when you omit the parenthesis's.
Try to change it to Assigned(Anode.ParentNode()) (which should have the same effect as François suggestion).
This seems like the simplest thing in the world and I'm ready to pull my hair out over it.
I have a unit that looks like this ;
Unit myUnit;
// ...
//normal declarations
//...
Public
//bunch of procedures including
Procedure myProcedure;
const
//bunch of constants
var
//bunch of vars including
myCounter:integer;
Implementation
Uses //(all my uses)
// All of my procedures including
Procedure myProcedure;
try
// load items from file to TListBox - this all works
except
on EReadError do begin
// handle exception
end;
end; //try
myCounter:=0; // <-- ACCESS VIOLATION HERE
while myCounter //...etc
It's a simple assignment of a variable and I have no idea why it is doing this. I've tried declaring the variable local to the unit, to the procedure, globally - no matter where I try to do it I can't assign a value of zero to an integer, declared anywhere, within this procedure without it throwing an access violation. I'm totally stumped.
I'm calling the procedure from inside a button OnClick handler from within the same unit, but no matter where I call it from it throws the exception. The crazy thing is that I do the exact same thing in a dozen other places in units all over the program without problems. Why here? I'm at a total loss.
Read of address 00000008 means that you're reading a variable at an offset of 8 bytes from a nil pointer. That doesn't fit what you're trying to do here at all, since you're writing, not reading, and you're writing a constant, not a variable read from somewhere.
Are you sure that this is the actual line that's triggering the exception? Have you put a breakpoint on this line? Have you tried moving this line to the top of the procedure?
It's hard to be sure without your actual code in front of me, but if I had to guess, I'd say that the line before this is causing the exception, and then the instruction pointer has already incremented so Delphi highlights the next line.
I don't think your error means anything like what it appears to. When you get an access violation from a piece of code like this that has no sane way of producing an access violation you're looking at trashed memory in some fashion.
Step through the offending code in the CPU window and see what's really happening.
You are using with statements and are looking at a different myCounter?
Some part of your code is writing in memory it shouldn't or freeing while it shouldn't and that by accident it results in AV's when accessing that specific local variable?
The assignment myCounter := 0 throwing an access violation suggests that either the data segment that the global vars are stored in has been removed from memory, or that the registers are hosed in your myProcedure routine.
Use the CPU view to see what registers are used to access the global var, and then work backward from there to see where that/those registers go awry.
Move MyCounter up to the top of the var list, and see if it still happens. If the problem goes away, look at the vars declared above where you have it now - you may find something interesting, such as an array that is growing past its boundary.