Sometimes in s/w companies, customers provide data in multiple formats. There are linkable and executable data that are said to be "Rehosted" and compiled object files that are said to be "Retargeted". I am trying to understand what rehosting and retargeting mean in this area. Is it similar to the Bootstrap theory in computer science? I have the understanding of the following process (if not incorrect):
PROBLEM:
I need to write a compiler for a new language called "MyLang" to run on PowerPC
Solution:
1. I need to write a compiler for a language "MyLang-Mini"; a subset of "MyLang" to run on PowerPC.
2. I need to write a compiler for "MyLang" using "MyLang-Mini" to run on PowerPC.
3. I run the compiler obtained from no. 1 through the compiler obtained from no. 2 to
obtain the compiler for MyLang to run on PowerPC.
IN BESPOKE "T" DIAGRAM (...ISH):
MyLang PowerPC MyLang PowerPC
MyLangMini MyLangMini PowerPC PowerPC(instr.)
PowerPC(instr.)
What I am getting confused about is rehosting and retargeting. How are they coonected to this concept? What am I rehosting and retargeting if I have some binary data such as .exe or .obj? I would appreciate some detailed explanation if possible please!
I know that this will embark onto "CROSS-COMPILERS", but would prefer expert opinions to be sure.
Thanks in advance.
I now know that in s/w engineering:
REHOSTING - If you have a third-party application linkable/executable that requires usage on your host machine, you do rehosting. The target in this case are most often the same (OS platform, processor, etc.). In worst case, there is a virtualisation required. The rehosted application will run as if it was one of the application running in the host machine
RETARGETTING - If you have a third-party source code, you might need to recompile that to match with your target environment. It may also be that you have third-party .o or .obj compiled models and you want to link them with your source code (retargeted) in order to host it on a host machine. Just like REHOSTED application, it will be as if the application was installed on the host machine.
It will be good to know how this is similar to the compiler rehosting and retargeting. Sorry, I am a newbee is this area and will appreciate even a slap on the wrist.
Related
I need to create a Lua binary for Windows that is fully portable, meaning that I can have it on a flash drive and it will work on any Windows computer (well, Windows 7 computer). I need at least a few additional libraries including Lua socket, a library that allows the proper storage (without rounding at all) and computation of large numbers (around 81 digits in total), and the rs232 library.
The problem is that I don't know how to compile them together into a binary, or if I can use some method to just use a plain Lua binary and use require to add the others. I've been researching this for a long time (a few weeks now) and haven't been able to find a solution. If anyone can help, it would be greatly appreciated.
If it makes the process any easier, I do have a Linux operating system I can use if necessary.
Answer: if you are using Windows, you can get the .dll files for each library and add them to your Lua directory. I now have a version of Lua with all of the libraries I wanted, plus a few more. Thank you everyone for your help.
How can I build and compile my own Lua files on Windows? And make them executable.
I am reading Beginning Lua programming, and I have Windows 7 and MacOS Lion both installed. I am having the hard time to follow the instructions. They do not work for me.
On MacOS I open the terminal and put these in:
export LUA_DIR=/usr/local/lib/lua/5.1
mkdir -p /usr/local/lib/lua/5.1 (it tells me, mkdir: illegal option) and I can not follow from here
SET LUA_DIR=”c:\program files\lua\5.1”
As for Windows I do this according to the book.
This what I see in my shell c:\Users\bd>
mkdir "c:\program files\utility" and it tells me access is denied
I have tried to right click on this folder and check off read only, but it does not work.
Any clues would be appreciated, this part has been really confusing for me.
To package your Lua files into an executable on Windows you have several options. There is srlua, there is wxLuaFreeze from wxLua (available as a binary for Windows), and there are more options in this SO answer.
Essentially, the main two options are: (1) append your Lua code to a precompiled exe file, such that it will be loaded and executed when that exe file is run, and (2) convert your Lua code into real executable by compiling it to bytecode, then to C, and then to your target platform.
As to your MacOS issue, mkdir -p means that mkdir is asked to create intermediate directories (for example, you asked to create /a/b/c, it will also create /a/b if those don't exist). As you don't say which version of MacOS you run, it's difficult to provide more detailed answer.
For now the standard distribution of Lua does not compile a script to native executable code; it execute your scripts by first compiling it to bytecode, then by interpreting the bytecode with a reasonnably fast static interpret (this also means that it is easily portable across native or virtual systems, and very resistant to attacks (that could be targetting bugs in the native compiler itself).
Also Lua still does not feature a runtime JIT compiler like Java and .Net: Lua still does not features a VM to produce a safe sandbox.
There exists Lua packages that convert your bytecode (or directly a source script) to a C source that can be used to convert a Lua library into native mode via the same C compiler used to compile the Lua engine itself (this is how the builtin libraries are produced, though they are slightly optimized manually in some time-critical parts).
However it is possible to compile Lua to a javascript source, and run it with fast performance using Javascript, because today's Javascript interprets do have good performance with their implemented VM featuring a JIT compiler for their own bytecodes.
It is also possible by converting it the Lua bytecode to a .Net or Java source that can then be executed directly from Lua (for that you need a version of Lua that has been ported to .Net or Java or Javascript, something that is not so complicate than developing in C/C++ directly a VM with a JIT compiler (a moderately complex part is the bytecode verifier, but the really complex part is the memory manager its garbage collector and its sandbox so that your Lua script will be fully isolated from the Lua engine itself for itw own memory, but the most complex part if the runtime optimizer and collection of profiling statistics: this has been done in the modern VMs for Java, .Net, Javascript, PHP/Zend, Python, Perl...).
I dont know which other language VM would offer the best performance to port Lua and implement on it a compiler to their own bytecode running at near native speed in their VM. But my own small experience with programs (in a much simpler language) self-generating a bytecode that they can run themselves, has always shown me Java winning in performance over .Net and Javascript. This is most probably because Java features an profiling-based dynamic code optimizer
(On the opposite the .Net optimizer runs only once during program installation, using some profiling data collected during the installation of the .Net VM itself, or at first instanciation of the script, without really knowing any profiling data collected during execution of the compiled program itself, and based on some cheked assumptions about the platform capabilities).
I also don't if would be faster in PHP, Python or Perl; the comparison with newer Javascript engines was never attempted though. Porting/compiling a Lua program to Javascript is relatively easy because it implements closures relatively easy for the resolution of linkages. Then the generated Javascript will compile to native code with the excellent Javascript's JIT compilers we have today (and never cease to improve in performance, so much that I've seen various appliactions running now faster in Javascript than before when they were written in C++ or plain C; as well the memory footprint has largely been reduced, we no longer have memory leaks, and even if there's a garbage collector, today's Javascript VM have a very efficient one, which is even better than the GC implemented in the native Lua).
But Lua remains useful as it is easy to secure and sandbox and offers various security benefits (but there are security issues in Lua as well for some kinds of applications, where Javascript offers some solutions, notably for side-channel attacks based on variation of time of execution; but these side-channel attacks are very hard to solve and can affect any system, any program, any programming language, and this starts becoming a critical issue because they are now more esily exploitable; the reason of that comes from hardware optimizations that we depend more and more today when we want to maximize the performances). And with Lua you may be more immune to these problems that a sandboxing sofware environment cannot solve alone.
Probably later we'll see a true VM implementation of Lua with a JIT and self-generating code and the possibility to instanciate new sandboxed VMs to run their self-generated code. It will take more time to generate an EXE file for distribution; notably because it generally requires adding also an installer and a distribution manager.
So for now we could imagine distributing Lua applications compiled to the bytecode of another JIT-capable VM: this generated bytecode would be faster than the Lua bytecode, and would then be extremely complex to reverse-engineer to the semantics of Lua because it would require two separate reverse engineering first from the bytecode of the other VM to the bytecode of Lua, both bytecodes loosing some easiy inferable rules and options tested and foll, and then again to sme Lua source
For the OSX terminal issue:
This command should work
export LUA_DIR=/usr/local/lib/lua/5.1
This command will probably give you permission problems:
mkdir -p /usr/local/lib/lua/5.1
You may try this to solve that. You will be prompted for your password:
sudo mkdir -p /usr/local/lib/lua/5.1
This command has nothing to do with OSX and will not work. This is a windows command:
SET LUA_DIR=”c:\program files\lua\5.1”
You have a permissions problem with Windows- try creating your cmd or PowerShell in Administrator mode. C:\Program Files is a protected directory that a regular user account doesn't have permission to write to.
As for the OS X issue, check out the mkdir OS X manual page to make sure you have the command correct.
So, if I understood your question correctly, you are trying to build Lua on Windows.
This is of course possible, but not easy for beginners. I would highly recommend you to use a binary distribution, which is much easier to install, unless you have special requirements.
Here are several Windows distributions :
Lua Binaries (Lua 5.1 and 5.2)
LuaForWindows (Lua 5.1)
LuaDist (Lua 5.2)
I have been asked to update a program written in 1987 in Delphi (I guess). I have no documentation about this program only a few side notes the programmer took that don't make too much sense to make.
The cd show this files:
Size | Filename
19956 VP.DTA
142300 VP.LEX
404 VP.NDX
126502 VP.RCS
131016 VP.SCR
150067 VP.XEL
101791 vp.exe
Is anyone of this files a database? If so can I access it's data?
I tried several code decompilers but they show a message saying it was not a Win32 compatible application.
The program run in MS-DOS.
Is it possible to obtain the source code? Can I use this code in any way to build a new application?
Update01: I can run the program in MS-DOS. The program conjugate verbs and shows an example sentence where the verb can be used. The GUI is a little bit confusing and there is no help menu so I can't see all the capabilities of the program.
Update02: In conversation with the owner of the program we found another solution. He ask me if it was possible to have the program in a server and the clients could login in with a user and a password and execute the program in a terminal. I have an account in my university server, which I can access throughout ssh and compile and execute c programs in it. The server is in linux so I couldn't try the program in it. If I set up a windows server, can I have multiple people accessing and executing the program in a terminal? The program is an exe. Doesn't this raise some security issues?
Delphi is from mid nineties, so that probably means Delphi's ancestor Turbo Pascal, not Delphi.
Some extensions sound familiar, as shortened versions of words:
ndx = index
dta = data
scr = screen (?)
lex = lexicon (list of words or deduped strings in general) (?)
Screen was sometimes used for e.g. helpscreens, a medieval form of helpfiles, they are typicall ansi screens that can be loaded directly into screen memory
There is a fair chance that this is something handcrafted, specially if that date of 1987 and the general assumption "pascal" is true, and not generated by some known database package at all.
Reverseengineering the fileformat might be a more worthwhile way than trying to reverseengineering the app.
A good start would to be to take a the unix "file" command to see if it can recognize the file types. (the file command searches for signatures inside files, and there are windows ports. I use Cygwin's)
A devel experienced in such matters can also see a lot from a hexdump (specially the first parts of a file)
Is it possible to obtain the source code?
Probably not, you may want to look at something like IDA Pro which can disassemble applications to C using something like Hex-Rays.
Do you know what the application is supposed to be?
If it's ms-dos, you're probably better off just drawing up new requirements and doing new development.
Look for DeDe to reverse engineering a delphi compiled program. But as far as i know, delphi is a real compiler. So there is no way to de-compiled it. If you are able to read assembler code then you can try de-compile it. Clipper and Foxpro (dos version) are another stories cause they not real compiler.
This is definitely not Delphi. It might be one of the database centric languages like Clipper 1. .SCR probably means "screen" and defines I/O masks. .NDX is a table index and .DTA means "data".
If it is clipper, you might actually be lucky, because as far as I remember these programs were P code, so it could be possible to decompile it.
It looks like CLipper (NDX and SCR). If you have a DBF file then it's Clipper for sure. But some people renamed the DBF to something like DAT. If it is Clipper, I believe there was a decompile named Valkyrie.
So here is the problem: Recently someone bought a new PC for server to replace an older dating from before 1985 (i wonder how it is possible to work daily from then) .
He wants to put there the old COBOL software and he isnt willing in any means to rewrite it to something better..
So is there any compiler for 1985 cobol? For nowadays red hat linux? Googling it found opencobol and other few but all converted the code to c... Seems too compilacted too me..
UPDATE AS REQUESTED
AIX was the old system
What's the problem with converting the COBOL to C and then compiling? As long as it works. Early C++ environments were implemented in the same way: they converted the C++ to C, and then invoked the C compiler.
Converting the COBOL to C allows them to use high-level abstractions that implement the COBOL equivalents in C. They can leverage the standard C libraries, and also convert the COBOL data access code into calls to widely available databases like MySQL. Finally, converting to C and then compiling leverages the vast amount of development effort that went into code generation. Were they to try compiling directly to object code, they'd have to generate the intermediate code expected by the GNU compiler subsystem, or they'd have to go directly to object code. Either one of those would be much more complicated than converting to C, meaning that the likelihood of bugs in the COBOL compiler would be much higher.
From where I sit, I'd say OpenCOBOL is worth looking into. Note that they say they implement "a substantial part of the COBOL 85 and COBOL 2002 standards." You probably want to make sure that they implement the parts that you need.
I would also suggest that you look into TinyCOBOL.
You don't mention when the application, or AIX was last updated. If these were updated in the last few years, you may be able to port the application, without re-compiling. You should check to see what COBOL compiler was used originally, e.g IBM, RM/COBOL, AcuCOBOL, etc. It might be possible to buy a run-time only version (will execute, but not compile), which would be cheaper than buying a compiler.
A company called Micro Focus make a cobol compiler for Windows but I can assure you it is not cheap at all!
Standard method for doing this is called migrating and involves a number of steps including converting source file to a textfile format or a filetype compatible with the target computer, using an approved method of converting to a file and writing to magtape with compatible recording method of Phase encoding or to disk or other data medium possibly in the ASN.xx mode, transferring to the new computer to then read in the file (through ASN.yy) and store it in a native or import file format, then either use a utility to convert it to the sourcefile format or by running the program development environment to access the native text file or import file and saving the content as a native sourcefile. Perform manual checks and amendments to the source or script code and then compile the program and repeat alterations until a working version is achieved. Create test data files on the new computer and create a new jobfile or macro to run the job in the development environment. When fully tested the program can be run live using data files and live macros or jobfiles migrated over from the old system or newly created in more or less the same way as bringing over the source code. An important point is that the live data must be read into a specialized data takeon or loading program to achieve a populated database before any new transactions occur in the case of a structured datafile being necessary. When moving from AIX or other versions of Unix to an entirely different operating system the characters for end of line and linefeed and end of record may need specific conversion if they are not handled by a file format convertor or exporter utility.
I'd like to see if I can reduce VM latencies by using Mono's AOT (ahead of time compiler). How does one run the .so files? (Sorry this is question is so rudimentary; I couldn't find a simple answer on Mono's documentation).
I figured out how to use mkbundle (mkbundle2 in this case), but are the resulting files being optimized? The Mono AOT documentation mentions "-O=all,-shared" flags, but I don't see these for mkbundle.
Though it's probably a much different answer, if there are good ways of keeping the VM open (with something more natural / platform neutral than a fifo special file), that might be best.
Thanks in advance!
If the AOT .so files exist beside the exe/dll files, Mono will use them when you run the exe. But you still need the dll/exe files for metadata and things that cannot be AOT-compiled.
Not sure if it's exactly related to your question, but see the final note here
Link
about the --resident flag to the compiler, to make fsc.exe faster on Mono.