I start using a TFileStream and TStreamWriter to write simple text logfiles (instead of old Writeln(T,....)). And I have multiple applicatiosn writing to the same logfile.
Each appplication has its own TFileStream of course and they each open the file like this
FFileStream:=TFileStream.Create(LogName, fmOpenReadWrite+fmShareDenyNone)
FExporter:=TStreamWriter.Create(FFilestream, TEncoding.UTF8);
FExporter.NewLine:=#$0A;
FExporter.AutoFlush:=TRUE;
and write to the file with
FExporter.BaseStream.Seek(0, soFromEnd);
FExporter.Write('['+DateToStr(Now, FDateTimeFormat)+'] ['+TimeToStr(Now, FDateTimeFormat)+'] [#'+Lead0(GetCurrentThreadId, 5)+']: '+EntryText);
FExporter.WriteLine;
the result is somewhat "unsatisfactory" as the lines are displaced, empty lines in between and does not seem to work.
HOW would I do that correctly?
Writing multiples lines at the same time in multiples process may result in unexpected continue, because parallels execution.
You should assure that you are writing a block continually so WriteLine shoud be send inside the write using lineBreak at the end.
So the way you can write should be:
FExporter.BaseStream.Seek(0, soFromEnd);
FExporter.Write('['+DateToStr(Now, FDateTimeFormat)+'] ['+TimeToStr(Now, FDateTimeFormat)+'] [#'+Lead0(GetCurrentThreadId, 5)+']: '+EntryText + System.slineBreak);
//FExporter.WriteLine;
Update1:
As the link Oliver posted, sometime it can not work if the message size to be written is bigger than the OS file sector and, at that very moment, other process also try to write a message. Thus in this case the result content might be mixed.
So doing what I first purpose you would increase the probability to have the desired result, but may not be the solution in 100% of the cases.
To be 100% sure of writing continuous log in a single file, using multiples process, you should create a log process to receive a message from the others and to be the only responsible for writing synchronized log throughout threads.
Related
Hi I am using LabView 2012, Delphi XE7 and GPIB (I think 488.2), Win7 SP1 and Agilent 53131A.
I used the given NI examples.
NI Labview example - Found in LabVIEW's help - GPIB.vi.
I tried writing and reading to query frequencies from 2 channels and they are successful.
They are are sent and read in succession.
*IDN?
:FUNC 'FREQ 1'
:READ:FREQ?
If they are successful, that meant GPIB for Agilent and NI MAX and driver are successfully installed and configured.
I am also able to use KeySight Connection Expert's to write and read, Again it is also successful.
However, When I used the given NI example in Delphi. Orginally it was saved as Delphi 3 or 4.
I used the Scope Simple example for universal counter. I used it mostly for writing and reading in the simple way. All it needs initialization, read/write and cleanup
I changed the following codes as shown below, in SimpleForm.pas
The detected device is at GPIB0::3::INSTR so, at line 32,
PRIMARY_ADDR_OF_COUNTER = 3;
String to write and read so, at line 132,
CommandBox.Text := '*IDN?';
then it was compiled with no error and run.
String to write was successfully
But upon reading, it was not successfully.
The string output is supposed to be ' HEWLETT-PACKARD,53131A,0,4806'.
The error at the end of the program is as follows below:-
Unable to read from device
ibsta = SC000 <ERR TMO>
iberr = 6 <EABO>
ibcntl = 0
From these readings, I figured out as :-
EABO means abort
I am not familiar with working of GPIB. Kindly advise.
You are correct that EABO is the identifier for an abort. In addition, we can see from ibsta = SC000 <ERR TMO> that the cause of the abort was a GPIB timeout error. I am not familiar with Keysight Connection Expert or your instrument, but since the error was from GPIB timeout, the most likely causes are:
The query was improperly formatted and the instrument thought it was just a write statement with no response needed. (That's probably why the write function had no error, but the read function timed out.)
The query was improperly formatted and the instrument returned an error.
Instrument needs to have 'Talker' capability enabled to send data. (Most instruments do this automatically with queries.)
For more information on generic GPIB commands, see this reference from the folks at National Instruments.
I have a Delphi5 application which exports a file (.pdf) and a very small metadata file to a network location. The intention is that these 2 files should be processed, and then removed, by a polling .NET application.
My approach is to
Write the metadata file with the extension '.part'
Generate the .pdf
Rename the .part file to .dat
The .NET process is looking for files with the extension '.dat' only, so I would expect there to be no conflict between the 2 reader/writers. However, the .NET process is occasionally logging the following error ...
System.IO.IOException: The process cannot access the file '\\server\Path\FileName.dat' because it is being used by another process.
(I say occasionally - we are currently testing, so when volumes increase this may become much more of an issue)
The Delphi code looks like this :
AssignFile(FTextFile, Format('%s\%s.part', [DMSPath, FullFileName]));
try
try
ReWrite(FTextFile);
Writeln(FTextFile, MetaDataString);
finally
CloseFile(FTextFile);
end;
except
raise ELogFileException.Create( LOGFILEWRITEFAILURE );
end;
Then there is a separate method which performs the following lines of code
if FindFirst(Format('%s\*.part',[DMSPath]), faAnyFile, SearchRec) = 0 then begin
repeat
OldName := Format('%s\%s',[DMSPath, SearchRec.Name]);
NewName := Format('%s\%s',[DMSPath, ChangeFileExt(SearchRec.Name, '.dat')]);
RenameFile(OldName, NewName);
until FindNext(SearchRec) <> 0;
FindClose(SearchRec);
end;
I cannot see anything inherently wrong with this code and we have a couple of remedies in mind, but I have 2 questions
Should I try a different technique to more reliably protect the '.dat' file until it is fully ready
What circumstances could be causing this?
So far there has been one suggested cause - Antivirus software.
Any suggestions as to how the file might be produced differently? Note that my application is Delphi5; I wondered if there was a newer, more 'atomic' version of the 'MoveFileA' WinApi call I could use.
In the past, we had a problem with file being locked like that. Investigation pointed to Windows' Prefetch. The file being affected were not on a network directory though.
As far as I know, Prefetch only work on process startup and/or while booting (Controlled by a registry key), so it might not apply to your current situation.
You can check the "C:\"Windows"\Prefetch\" directory. If prefetch is active, it should contains multiple *.pf files. If there is one with your executable filename, it might be worth investigating.
Personally speaking, because there are multiple files involved, I'd create a separate lock file (eg, myfile.lck) that you write first. If the polling app sees it in the folder, then it stops looking for other files. Once that file is gone, then it dives deeper. I don't know if this would solve the problem you're encountering or not, but I'd give it a try. (Files with .dat extensions are frequently created by malicious evildoers, so they can raise spurious issues through other sources, like AV software. A lock file with 0 bytes in it is generally harmless and disregarded.)
I have an XQuery query intended to wipe test documents from the database before each test that is run. Essentially it looks for a certain element to be present as a top level element in a document (called 'forTestOnly') and if it finds it it deletes the document. This query is run before each test in order to ensure the tests don't interfere with one another (we have about 200 tests using this). The exact XQuery is as such:
xquery version "1.0-ml";
import module namespace dls = "http://marklogic.com/xdmp/dls" at "/MarkLogic/dls.xqy";
let $deleteNonManagedDocs := for $testDoc in /*[forTestOnly]
let $testDocUri := fn:base-uri($testDoc)
where fn:not(dls:document-is-managed($testDocUri))
return xdmp:document-delete($testDocUri)
let $deleteManagedDocs := for $testDoc in cts:search(/*[forTestOnly], dls:documents-query())
let $testDocUri := fn:base-uri($testDoc)
return dls:document-delete($testDocUri, fn:false(), fn:false())
return ($deleteManagedDocs, $deleteNonManagedDocs)
While it seems to work fine most of the time, it recently has begun to sporadically spiral out of control. At some point during the test execution it begins to run for a near indefinite amount of time (I usually stop it after 600-700 seconds), most of the time it takes less than a second though. The database used for testing is not large (it has a few basic seed documents but nothing compared to a production database), and typically each test only creates a handful of documents with the 'forTestOnly' (if not less).
The query seems simple enough, and although running it 200 times in relatively quick succession would understandably put a strain on the database I can't imagine it would cause this kind of lagging (the tests are Grails integration tests and the entire execution takes a little over two minutes). Any ideas why the long run time?
As a side note I have verified that when the tests stall it is indeed after the XQuery has begun to run and not before in some sort of test wiring/execution.
Any help is greatly appreciated.
The query might look simple, but it isn't necessarily simple to evaluate. Those dls function calls could be doing anything, so it's tricky to estimate the complexity. The use of DLS also means that we don't know how much version history has to be deleted to delete each document.
One possibility is that you've discovered a bug. It might already be fixed, which is a good reason why you should always report the full version of the software you're using. The answer may be as simple as upgrading to pick up the fix.
Another possibility is that your test suite ends up running all of this work in a single high-level evaluation, so everything's in memory until the end. That could use enough memory to drive the server into swap. That would explain the recent "spiral out of control" behavior. Check the OS and see what it says.
Next, set the group file-log-level=Debug and check ErrorLog.txt while one of these slow events is happening. If you see XDMP-DEADLOCK messages, you may have a problem where two or more copies of this delete query are running at the same time. MarkLogic has automatic deadlock detection and resolution, but it's faster to avoid the deadlock in the first place.
Some logging might also help determine where the time is spent. Something like:
let $deleteNonManagedDocs := for $testDoc in /*[forTestOnly]
let $testDocUri := fn:base-uri($testDoc)
where fn:not(dls:document-is-managed($testDocUri))
return (
xdmp:log(text { 'unmanaged', $testDocUri }),
xdmp:document-delete($testDocUri))
let $deleteManagedDocs := for $testDoc in cts:search(/*[forTestOnly], dls:documents-query())
let $testDocUri := fn:base-uri($testDoc)
let $_ := xdmp:log(text { 'managed', $testDocUri })
return dls:document-delete($testDocUri, fn:false(), fn:false())
return ()
Finally you should also be able to simplify the query a bit. Since you're deleting everything, you can just ignore DLS.
xdmp:document-delete(
cts:uris(
(), (),
cts:element-query(xs:QName('forTestOnly'), cts:and-query(())))
This would be even simpler and more efficient if you set a collection on every test document: xdmp:collection-delete('test-docs').
What tools would you recommend for unit testing in Delphi.
I've used FastMM4 for memoryleak testing.
And MadExcept, both rule, but it doesn't help me test the logic in my programs.
I would like some alternatives, so don't all rush to suggest DUnit :-).
Any suggestions?
The most commonly used is DUnit. It's actually included in modern versions of Delphi but if your version doesn't come with it you can download it from Sourceforge.
DUnit comes with D2007.
-->File -->New -->Other
Select Unit Test from the dialogue that pops up.
There are a couple of good demo videos on it's use, I'll see if I can dig one up.
This is a pretty good one and others come up on the right:
http://www.youtube.com/watch?v=nyZnfxDqThE
You could take a look at the unit testing classes available in our SynCommons open source unit. It's used in our Open-Source framework for all regression tests. It's perhaps not the best, but it's worth taking a look at it. See http://blog.synopse.info/post/2010/07/23/Unit-Testing-light-in-Delphi
In the up-to-come 1.13 version, there is also a new logging mechanism with stack trace of any raised exception and such, just like MadExcept, using .map file content as source.
It's now used by the unit testing classes, so that any failure will create an entry in the log with the source line, and stack trace:
C:\Dev\lib\SQLite3\exe\TestSQL3.exe 0.0.0.0 (2011-04-13)
Host=Laptop User=MyName CPU=2*0-15-1027 OS=2.3=5.1.2600 Wow64=0 Freq=3579545
TSynLogTest 1.13 2011-04-13 05:40:25
20110413 05402559 fail TTestLowLevelCommon(00B31D70) Low level common: TDynArray "" stack trace 0002FE0B SynCommons.TDynArray.Init (15148) 00036736 SynCommons.Test64K (18206) 0003682F SynCommons.TTestLowLevelCommon._TDynArray (18214) 000E9C94 TestSQL3 (163)
The difference between a test suit without logging and a test suit with logging is only this:
procedure TSynTestsLogged.Failed(const msg: string; aTest: TSynTestCase);
begin
inherited;
with TestCase[fCurrentMethod] do
fLogFile.Log(sllFail,'%: % "%"',
[Ident,TestName[fCurrentMethodIndex],msg],aTest);
end;
The logging mechanism can be used to trace recursive calls. It can use an interface-based mechanism to log when you enter and leave any method:
procedure TMyDB.SQLExecute(const SQL: RawUTF8);
var ILog: ISynLog;
begin
ILog := TSynLogDB.Enter(self,'SQLExecute');
// do some stuff
ILog.Log(sllInfo,'SQL=%',[SQL]);
end; // when you leave the method, it will write the corresponding event to the log
It will be logged as such:
20110325 19325801 + MyDBUnit.TMyDB(004E11F4).SQLExecute
20110325 19325801 info SQL=SELECT * FROM Table;
20110325 19325801 - MyDBUnit.TMyDB(004E11F4).SQLExecute
Here the method name is set in the code ('SQLExecute'). But if you have an associated .map file, the logging mechanism is able to read this symbol information, and write the exact line number of the event.
Note that by default you have time and date written to the log, but it's also possible to replace this timing with high-resolution timestamps. With this, you'll be able to profile your application with data coming from the customer side, on its real computer. Via the Enter method (and its auto-Leave feature), you have all information needed for this. ;)
Like this:
0000000000000B56 + TTestCompression(00AB3570).000E6C79 SynSelfTests.TTestCompression.TestLog (376)
0000000000001785 - TTestCompression(00AB3570).000E6D09 SynSelfTests.TTestCompression.TestLog (385)
I still need to write some tool to compute the profiling, but there is already a dedicated TSynLogFile class able to read the .log file, and recognize its content.
The first time the .map file is read, a .mab file is created, and will contain all symbol information needed. You can send the .mab file with the .exe to your client, or even embed its content to the .exe. This .mab file is optimized: a .map of 927,984 bytes compresses into a 71,943 .mab file.
So this unit could be recognized as the natural child of DUnit and MadExcept wedding, in pure OpenSource. :)
Additional information is available on our forum. Feel free to ask. Feedback and feature requests are welcome! Works from Delphi 6 up to XE.
I'm parsing an XML using XMLLite. I notice that when its a relatively large file, the reader's pointers fails to locate the next element(tag) of the file. When i reduced the contents of the file, it could successfully parse.
The reader continually shows node type "XmlNodeType_None" and fails to complete parsing, getting stuck in an infinite while loop.
Is it to do something with its file size? Or anything in initializing the IStream? My file has around 9000 bytes of data only.
Thanks
Do not use the SUCCEEDED macro to check if you should continue processing the value returned by IXmlReader::Read. Instead, check that the return value of IXmlReader::Read is equal to S_OK for the condition of your loop.