Context
A friend of mine is having trouble printing source code to a human readable format.
The compiled (I assume) programs of their welding robot have the .rpg extension. They want to collect print-outs in human-readable format, possibly for backup or future reference.
Their supplier can provide the software that accomplishes this, be it at a considerable cost (and possibly: an annual license). Because of this, my friend decided to ask me if a easier/cheaper solution exists.
Examples & Pictures
The files can be read on the console of the robot, an example:
I've done some minor research and I'm fairly sure this is the Report Program Generator (RPG) language developed by IBM. The Assembly-like syntax seems to match; it might be one of the later versions of the language.
My friend has send me an example .rpg file, the contents seem binary with some string literals scattered throughout. Screenshot of the contents of an example file in hexadecimal:
The Question
There is not much, if any, clear information to be found online so I suppose I have multiple questions (for anyone that might know more about this):
Is this (first image) Report Program Generator (RPG) code?
Does the .rpg file contain compiled or processed code? Maybe an intermediate format?
Is it possible to convert files as shown in the example, back to source-code or human-readable format, kind of 'disassemble' it?
If anyone knows more, don't hesitate to give me any information or ask more details if necessary. Thanks in advance!
And maybe not an important question but still something that bugs me (and might indicate I'm on the wrong track):
If this is indeed an RPG program, why would the compiled/processed binary have the .rpg extension, shouldn't the source-file have that? This leads me to believe I'm either (a) assuming the wrong things (the language, etc...) or (b) this is an intermediate format, easier for machines to read, that has to be interpreted by some kind of runtime system.
I don't think that's any version of IBM's RPG language. RPG does have a MOVEL opcode, but it doesn't have any of the others.
Also, all the versions of the IBM language have been intended for business programming. I doubt that it would have been used for robotics.
My guess is that's a proprietary language of the company that makes the robot.
There are some similarities but it does not look like IBM RPG language.
RPG sources are in fact source physical file members. They are not stored in the "traditional" file system but in OS/400 libraries. Therefore RPG sources have no extension. They can be converted to Integrated File System stream file though.
I can't answer this question I'm afraid as it's unknown language to me.
I expect possibly that the OP misidentifies the file type/extension; that the extension is actually .prg, and the files serve as instructions for a Panasonic Industrial Welding Robot. The following forum [drilled down to Panasonic Robots] bills itself as the biggest Industrial Robots Supportforum worldwide!; perhaps a good place to ask about those images provided in the OP, and the inquiry about getting source from what appears to be a binary instruction stream.
FWiW, the first image seems to show that the Ezed utility [on the console] gives that human-readable format, so then the question might be how to get that saved and then how to transfer that elsewhere; e.g. what type of comm ports and file transfer utilities are available from whatever platform/OS.
Related
A friend of mine has a problem. He has hundreds of highly confidential .EBIN files for a medical study created by a person that is no longer available.
I figured that it's probably an Erlang directory - I downloaded Erlang and looked for several file type specifications, but I just can't find a way to "open" this binary file.
I feel really stupid right now as I should be able to easily access this as a long-term programmer, but I'm clueless. I don't even know what to enter into a search engine.
I'd guess that they are just containing serialized Erlang data ("terms"). Try starting Erlang and entering the following from the Erlang shell:
erlang:binary_to_term(element(2,file:read_file("YOURFILE.EBIN"))).
See http://erlang.org/doc/man/erlang.html#term_to_binary-2 for details about the term_to_binary() function and see http://erlang.org/doc/apps/erts/erl_ext_dist.html for details about the term format. If the bytes on disk don't look like this, it's likely that the binary data has also been encrypted before writing it on disk.
I'm writing a program (C#) to read, convert, display, adjust and output point cloud data.
I can make every part of the program except for one - I am required to read in a proprietary file format. The data is coming straight from a laser scanner and we cannot get any closer to the stream than what is output to the proprietary file in binary.
I have an SDK from the manufacturer/proprietor that is well outside my scope of ability to deal with.
Firstly it is written in C++, which I can read and write to some degree but this all appears incredibly complex (there are hundred of header/source files).
Secondly, the SDK documentation says that I must create my SLN using CMake which is a nightmare for me also.
Thirdly, the documentation is scarce and horrid.
Basically my question is this:
I know that after a certain amount of header information I should find thousands of lines of "lineref,x,y,z,r,g,b,time,intensity".
Can I bypass the SDK and find another way to read in this file type?
Or, must an SDK from the proprietor be used to interact with their file type due to some sort of encryption?
For the purposes of binary / source code verification, i'd like to be able to make two compiles on the same computer 2 weeks apart and have the binaries be identical and thus pass some checksum test.
So far I've found that most likely the timestamp will be written by the compiler into the binary. I can work around this by doing the compare on the dumpbin /rawdata results per this msdn article.
http://support.microsoft.com/kb/164151
However the dumpbin results still differ in a about a dozen places and the difference still appears to be some kind of timestamp (changing from A1 73 to C4 76) for example.
I assume this is the timestamp that the delphi compiler is putting into the code/data sections but i can't find where this is happening or how to turn it off. Fiddling with the various compiler/linker options has not changed this behavior.
Any help would be greatly appreciated.
We researched this question a few years ago, and as I recall the answer was simply that the people at Borland/Codegear/Embarcadero do not subscribe to the notion that the compiler should produce exactly the same output given exactly the same input (given that the current date-time is not acceptable to consider as input).
This is disappointing, but given that Delphi is upmteen major versions old, it probably will not change.
Based on your comment in the question, I think you are searching for the answer to this question:
How can I verify that a certain binary is created by a certain set of source files
One way is to have an automated built process that:
Inserts a unique verififcation tag in the source code (might be as simple as a GUID string or something else in an include file)
Compiles the binaries using that verification tag
Packs the resulting binary and source code in a delivery directory tree
You must make sure that your sourcecode actually uses that verification tag (for instance by showing in an about box, or something similar).
I use FinalBuilder to automate build processes like this.
--jeroen
Unfortunately I haven't heard of any answer to this problem.
It's not just timestamps but there are places where a few bytes of random garbage out of memory ends up in the result.
I am looking to speed up the reading of a data file which has been converted from binary (it is my understanding that "binary" can mean a lot of different things - I do not know what type of binary file I have, just that it's a binary file) to plaintext. I looked into reading files quickly awhile ago, and was informed that reading/parsing a binary file is faster than text. So, I would like to parse/read the binary file (that was converted to plaintext) in an effort to speed up the program.
I'm using Matlab for this project (I have a Matlab "program" that needs the data in the file). I guess I need some information on the different "types" of binary, but I really want information on how to read/parse said binary file (I know what I'm looking for in plaintext, so I imagine I'll need to convert that to binary, search the file, then pull the result out into plaintext). The file is a logfile, if that helps in any way.
Thanks.
There are several issues in what you are asking -- however, you need to know the format of the file you are reading. If you can say "At position xx, I can expect to find data yy", that's what you need to know. In you question/comments you talk about searching for strings. You can also do it (much like a text file) "when I find xxxx in the file, give me the following data up to nth character, or up to the next yyyy".
You want to look at the documentation for fread. In the documentation there are snippets of code that will get you started, but as I (and others) said you need to know the format of your binary files. You can use a hex editor to ascertain some information if you are desperate, but what should be quicker is the documentation for the program that outputs these files.
Regarding different "binary files", well, there is least significant byte first or LSB last. You really don't need to know about that for this work. There are also other platform-dependent issues which I am almost certain you don't need to know about (unless you are moving the binary files from Mac to PC to unix machines). If you read to almost the bottom of the fread documentation, there is a section entitled "Reading Files Created on Other Systems" which talks about the issues and how to deal with them.
Another comment that I have to make, you say that "reading/parsing a binary file is faster than text". This is not true (or even if it is, odds are you won't notice the performance gain). In terms of development time, however, reading/parsing a textfile will save you huge amounts of time.
The simple way to store data in a binary file is to use the 'save' command.
If you load from a saved variable it should be significantly faster than if you load from a text file.
I am trying to deserialize an old file format that was serialized in Delphi, it uses binary seralization. I know nothing about the structure of the file except some very high level records that are in it.
What steps would you take to solve this problem? Any tools etc?
A good hexeditor, and use the gray matter to identify structures.
If you get a hint what kind of file it is, you can search for more specialized tools.
Running the unix/Linux "file" command can be good too (*) See Barry's comment below for how it works. It can be a quick check for common filetypes like DBF,ZIP etc hidden by using a different extension.
(*) there are 3rd party builds for windows, but they might lag in versions. If you can do it on a recent *nix distro, it is advised to do so.
The serialization process simply loops over all published properties and streams their value to a text file. If you do not know the exact classes that were streamed to the file you will have a very hard time deserializing the file. (if not impossible)
A good hex editor is first. If the file is read without buffering (eg read directly from a TFileStream) you could gain some information when using ProcMon from SysInternals; You can see exactly what data is read in what chunks and thus determine more quickly where the boundaries are between the structures you already identified.