For the purposes of binary / source code verification, i'd like to be able to make two compiles on the same computer 2 weeks apart and have the binaries be identical and thus pass some checksum test.
So far I've found that most likely the timestamp will be written by the compiler into the binary. I can work around this by doing the compare on the dumpbin /rawdata results per this msdn article.
http://support.microsoft.com/kb/164151
However the dumpbin results still differ in a about a dozen places and the difference still appears to be some kind of timestamp (changing from A1 73 to C4 76) for example.
I assume this is the timestamp that the delphi compiler is putting into the code/data sections but i can't find where this is happening or how to turn it off. Fiddling with the various compiler/linker options has not changed this behavior.
Any help would be greatly appreciated.
We researched this question a few years ago, and as I recall the answer was simply that the people at Borland/Codegear/Embarcadero do not subscribe to the notion that the compiler should produce exactly the same output given exactly the same input (given that the current date-time is not acceptable to consider as input).
This is disappointing, but given that Delphi is upmteen major versions old, it probably will not change.
Based on your comment in the question, I think you are searching for the answer to this question:
How can I verify that a certain binary is created by a certain set of source files
One way is to have an automated built process that:
Inserts a unique verififcation tag in the source code (might be as simple as a GUID string or something else in an include file)
Compiles the binaries using that verification tag
Packs the resulting binary and source code in a delivery directory tree
You must make sure that your sourcecode actually uses that verification tag (for instance by showing in an about box, or something similar).
I use FinalBuilder to automate build processes like this.
--jeroen
Unfortunately I haven't heard of any answer to this problem.
It's not just timestamps but there are places where a few bytes of random garbage out of memory ends up in the result.
Related
Context
A friend of mine is having trouble printing source code to a human readable format.
The compiled (I assume) programs of their welding robot have the .rpg extension. They want to collect print-outs in human-readable format, possibly for backup or future reference.
Their supplier can provide the software that accomplishes this, be it at a considerable cost (and possibly: an annual license). Because of this, my friend decided to ask me if a easier/cheaper solution exists.
Examples & Pictures
The files can be read on the console of the robot, an example:
I've done some minor research and I'm fairly sure this is the Report Program Generator (RPG) language developed by IBM. The Assembly-like syntax seems to match; it might be one of the later versions of the language.
My friend has send me an example .rpg file, the contents seem binary with some string literals scattered throughout. Screenshot of the contents of an example file in hexadecimal:
The Question
There is not much, if any, clear information to be found online so I suppose I have multiple questions (for anyone that might know more about this):
Is this (first image) Report Program Generator (RPG) code?
Does the .rpg file contain compiled or processed code? Maybe an intermediate format?
Is it possible to convert files as shown in the example, back to source-code or human-readable format, kind of 'disassemble' it?
If anyone knows more, don't hesitate to give me any information or ask more details if necessary. Thanks in advance!
And maybe not an important question but still something that bugs me (and might indicate I'm on the wrong track):
If this is indeed an RPG program, why would the compiled/processed binary have the .rpg extension, shouldn't the source-file have that? This leads me to believe I'm either (a) assuming the wrong things (the language, etc...) or (b) this is an intermediate format, easier for machines to read, that has to be interpreted by some kind of runtime system.
I don't think that's any version of IBM's RPG language. RPG does have a MOVEL opcode, but it doesn't have any of the others.
Also, all the versions of the IBM language have been intended for business programming. I doubt that it would have been used for robotics.
My guess is that's a proprietary language of the company that makes the robot.
There are some similarities but it does not look like IBM RPG language.
RPG sources are in fact source physical file members. They are not stored in the "traditional" file system but in OS/400 libraries. Therefore RPG sources have no extension. They can be converted to Integrated File System stream file though.
I can't answer this question I'm afraid as it's unknown language to me.
I expect possibly that the OP misidentifies the file type/extension; that the extension is actually .prg, and the files serve as instructions for a Panasonic Industrial Welding Robot. The following forum [drilled down to Panasonic Robots] bills itself as the biggest Industrial Robots Supportforum worldwide!; perhaps a good place to ask about those images provided in the OP, and the inquiry about getting source from what appears to be a binary instruction stream.
FWiW, the first image seems to show that the Ezed utility [on the console] gives that human-readable format, so then the question might be how to get that saved and then how to transfer that elsewhere; e.g. what type of comm ports and file transfer utilities are available from whatever platform/OS.
I am looking to speed up the reading of a data file which has been converted from binary (it is my understanding that "binary" can mean a lot of different things - I do not know what type of binary file I have, just that it's a binary file) to plaintext. I looked into reading files quickly awhile ago, and was informed that reading/parsing a binary file is faster than text. So, I would like to parse/read the binary file (that was converted to plaintext) in an effort to speed up the program.
I'm using Matlab for this project (I have a Matlab "program" that needs the data in the file). I guess I need some information on the different "types" of binary, but I really want information on how to read/parse said binary file (I know what I'm looking for in plaintext, so I imagine I'll need to convert that to binary, search the file, then pull the result out into plaintext). The file is a logfile, if that helps in any way.
Thanks.
There are several issues in what you are asking -- however, you need to know the format of the file you are reading. If you can say "At position xx, I can expect to find data yy", that's what you need to know. In you question/comments you talk about searching for strings. You can also do it (much like a text file) "when I find xxxx in the file, give me the following data up to nth character, or up to the next yyyy".
You want to look at the documentation for fread. In the documentation there are snippets of code that will get you started, but as I (and others) said you need to know the format of your binary files. You can use a hex editor to ascertain some information if you are desperate, but what should be quicker is the documentation for the program that outputs these files.
Regarding different "binary files", well, there is least significant byte first or LSB last. You really don't need to know about that for this work. There are also other platform-dependent issues which I am almost certain you don't need to know about (unless you are moving the binary files from Mac to PC to unix machines). If you read to almost the bottom of the fread documentation, there is a section entitled "Reading Files Created on Other Systems" which talks about the issues and how to deal with them.
Another comment that I have to make, you say that "reading/parsing a binary file is faster than text". This is not true (or even if it is, odds are you won't notice the performance gain). In terms of development time, however, reading/parsing a textfile will save you huge amounts of time.
The simple way to store data in a binary file is to use the 'save' command.
If you load from a saved variable it should be significantly faster than if you load from a text file.
Does it happen that no one ever needs histogram in Delphi ?
Google gave me a bunch of half-baked code snippets. But it means that each time you need one - you have to invent one more ad hoc bycicle.
Torry mostly told me about some very expensive closed source Math Statistics or Financial packages, that as a subproduct have histograms. But they are very expensive and since you have no source code, each time you install update onto IDE/RTL/VCL you're probably screwed, until the vendor would make (soon ? ever?) updated packages. Given thatvendor is still does exists.
S.O. told me nothing, nil.
For what i found...
Mitov.com provides some histograms in PlotLab. which told to be free for non-commercial. Alas, it is again closed-source, and if the Histogram - quite fancy let's admit -is the onlything i need from it - why pay the whole price ?
One more example http://DSpatial.sf.net
Just few years ago i used it in Delphi 5, but even then i felt the author is loosing interest in the project. I made few enhancement, fixed some bugs, he merged them and that's all. The component was not very useful and lacked upon features, yet better than nothing. Now the project seems to be completely dead. Good old days, etc. But i do not want them back :-)
And Stack Overflow seemingly carries no single question about it. But maybe just no one bothered to create topic, after search found nothing ? I mean, Delphi was created for database access, histograms are one of basic ways to visualize data, and no one crosses them ? Something with nice style, with rich mouse tooltip like in HTML/CSS/JS on http://www.moskva.fm/stations/FM_95.2 ?
Or is this too domain-related and not ever possible to have good abstraction ?
TChart is a control that ships with most versions of Delphi. TChart can be used to make histograms (bar charts) in style. The following give you some ideas about how to use it: http://www.digitalcoding.com/tutorials/delphi/Simple-steps-to-create-Delphi-chart.html and http://delphi.about.com/od/adptips2006/qt/chart_selectbar.htm .
If you need something with code, google the pages at delphiforfun.org/programs/oscilloscope.htm . These are not controls. The oscilloscope article has a histogram with source. Some of the other projects at the site have other histogram graphs with source..not elegant but useful and free. Use them as a template to make your own control.
The link at http://delphiforfun.org/programs/Math_Topics/probability_distributions.htm shows how to make your own statistics displays with "histograms." This example makes use of TChart.
Here is some more stuff to try I found looking at my resource file:
http://wiki.lazarus.freepascal.org/TAChart, http://members.home.nl/mvanwesten/en_lazarus.html , http://www.martinole.org/TAChart.html ...some of these are GPU components that supposedly work with some versions of Delphi. Perhaps this is your lucky day as there is some source code. The first and third listed probably will work reasonably for histograms. You may have to write your own statistics algorithms.
Found this thread while doing some searching. The ImageEn component suite has a THistogramBox component. It's the NOT prettiest thing in the world, but it's the only one I've found so far.
http://www.imageen.com
I came across a histogram example in a gdiplus package available for download from code central. I don't know if it will do what you need but when I saw it I remembered your SO question.
HTH.
If you were using firemonkey, you could just created a series of TRectangles in series. They can be made unclickable by turning hittest off. Or is that too easy and straightforward?
I am trying to deserialize an old file format that was serialized in Delphi, it uses binary seralization. I know nothing about the structure of the file except some very high level records that are in it.
What steps would you take to solve this problem? Any tools etc?
A good hexeditor, and use the gray matter to identify structures.
If you get a hint what kind of file it is, you can search for more specialized tools.
Running the unix/Linux "file" command can be good too (*) See Barry's comment below for how it works. It can be a quick check for common filetypes like DBF,ZIP etc hidden by using a different extension.
(*) there are 3rd party builds for windows, but they might lag in versions. If you can do it on a recent *nix distro, it is advised to do so.
The serialization process simply loops over all published properties and streams their value to a text file. If you do not know the exact classes that were streamed to the file you will have a very hard time deserializing the file. (if not impossible)
A good hex editor is first. If the file is read without buffering (eg read directly from a TFileStream) you could gain some information when using ProcMon from SysInternals; You can see exactly what data is read in what chunks and thus determine more quickly where the boundaries are between the structures you already identified.
What's the best way to set up help (specifically HTML Help) for a Delphi application? I can see several options, all of which has disadvantages. Specifically:
I could set HelpContext in the forms designer wherever appropriate, but then I'm stuck having to track numbers instead of symbolic constants.
I could set HelpContext programmatically. Then I can use symbolic constants, but I'd have more code to keep up with, and I couldn't easily check the text DFMs to see which forms still need help.
I could set HelpKeyword, but since that does a keyword lookup (like Application.HelpKeyword) rather than a topic jump (like Application.HelpJump), I'd have to make sure that each of my help pages has a unique, non-changing, top-level keyword; this seems like extra work. (And there are HelpKeyword-related VCL bugs like this and this.)
I could set HelpKeyword, set an Application.OnHelp handler to convert HelpKeyword requests to HelpJump requests so that I can assign help by topic ID instead of keyword lookup, and add code such as my own help viewer (based on HelpScribble's code) that fixes the VCL bugs and lets HelpJump work with anchors. By this point, though, I feel like I'm working against the VCL rather than with it.
Which approach did you choose for your app?
When I first started researching how to do this several years ago, I first got the "All About help files in Borland Delphi" tutorial from: http://www.ec-software.com/support_tutorials.html
In that document, the section "Preparing a help file for context sensitive help" (which in my version of the document starts on page 28). It describes a nice numbering scheme you can use to organize your numbers into sections, e.g. Starting with 100000 for your main form and continuing with 101000 or 110000 for each secondary form, etc.
But then I wanted to use descriptive string IDs instead of numbers for my Help topics. I started using THelpRouter, which is part of EC Software's free Help Suite at: http://www.ec-software.com/downloads_delphi.html
But then I settled on a Help tool that supported string ID's directly for topics (I use Dr. Explain: http://www.drexplain.com/) so now I simply use HelpJump, e.g.:
Application.HelpJump('UGQuickStart');
I hope that helps.
We use symbolic constants. Yes, it is a bit more work, but it pays off. Especially because some of our dialogs are dynamically built and sometimes require different help IDs.
I create the help file, which gets the help topic ID, and then go around the forms and set their HelpContext values to them. Since the level of maintenance needed is very low - the form is unlikely to change help file context unless something major happens - this works just fine.
We use Help&Manual - its a wonderful tool, outputting almost any format of stuff you could want, doc, rtf, html, pdf - all from the same source. It will even read in (or paste from rtf (eg MSWord). It uses topic ID's (strings) which I just keep a list of and I manually put each one into a form (or class) as it suits me. Sounds difficult but trust me you'll spend far longer hating the wrong authouring tool. I spent years finding it!
Brian