I wonder whether there's a way to program a resource DLL for Windows Clusters in Delphi. I want to write a program that observes cluster state changes.
Judging from the MSDN API reference, all relevant functions are located in the ClusAPI.h and ResAPI.h headers, but unfortunately it seems as if they haven't been ported yet and HeaderConv seems to fail on them.
Does anyone has experience in doing such a thing?
It's a very C-oriented API, with mandatory Dll entry points, tables of callback function pointers, and binary data blocks. I've been able to wrap a C#/.NET library with a C++/CLI interop layer, but it's definitely non-trivial.
This would require a manual C header translation to Pascal code. If the API deals in pointers, you wouldn't have to translate any structures... this would make it comparatively simple to perform the translation as you'd only need to translate the function declarations themselves. If it passes structures (as opposed to pointers there-to) you would have to translate all of those structures as Delphi Records (or Packed Records as appropriate). Structure translations are far more time-consuming.
Sorry I can't offer an "instant solution", but at the info I've provided should help (if you didn't already know it).
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
Why decompiling a delphi exe, is so easy, compared to others executables built with other programming languages/compilers?
There are a few things that help with reversing delphi programs:
You get the full form data including the name of event handler methods
All members with published visibility have metadata used with RTTI
The compiler is pretty bad at optimizing. It does no whole program optimization and the assembly is usually a straight forward translation of the original source with only minor optimizations. (At least it was in the versions I used, might have improved since then)
All classes, even those compiled with RTTI off have some level of metadata available. In particular it's possible to get the name and inheritance structure of classes. And for any instance of a class you happen to see in the debugger you can get its VMT and thus its class name.
Delphi uses textfiles describing the content of your form and hooks up event handlers by name. This approach obviously needs enough metadata to deserialize that textual representation of a from and hook up the eventhandlers by name.
An alternative some other GUI toolkits use is auto-generating code that initializes the form and hooks up the event handler with code. Since this code directly uses pointers to the eventhandlers and directly assigns to properties/calls setters it doesn't need any metadata. Which has the side-effect that reversing becomes a bit harder.
It shouldn't be too hard to create a program that transforms a dfm file into a series of hardcoded instructions that creates the form instead. So a tool like DeDe won't work that well anymore. But that doesn't gain you much in practice.
But figuring out which evenhandler corresponds to which control/event is still rather easy. Especially since stuff like FLIRT identifies most library functions. So you just need to breakpoint the one you're interested in and then step into the user code.
The statement you make is false. Delphi is not particularly more easy to decompile than code produced by other mainstream compilers.
For .net languages there is Reflector.
C++ is covered in this Stack Overflow question.
Python/Perl/Ruby etc. are interpreted.
If you were able to prove that the results of decompiling a Delphi executable were of significantly higher quality than in other widely used languages then your question would carry more weight.
Story from the trenches: Decompiling a tiny Delphi DLL
I've been through a Delphi decompiling session myself. It was one of those fake-sounding "I lost my sources" thing, I really did lose the sources for a tiny Firebird UDF library. Now I do no better, I didn't jump right into decompiling because the library was so small and I knew a rewrite would be much faster.
This DLL exports a function that looks like this:
function udf_do_some_math(Number1, Number2:Currency): Currency;
After doing the sane thing and rewriting the function and doing some regression tests I discovered some obscure corner-cases where the new function's result wasn't the same as the old function's result! The trouble was, the new function's result was the correct result, the old DLL contained a BUG and I had to reproduce the BUG - with this function consistency is more important then accuracy.
Again, did the sane thing and tried to "guess" at the BUG. I knew it was a rounding issue but simply couldn't figure out what it was. Finally I decided to give decompilers I try. After all this was a small library, the entry-point was straight-forward and I didn't really need re-compilable code, nor 100% decompilation: I only needed enough to figure out the old BUG so I can reproduce it!
Decompiling failed! I tried lots of different decompilers, including a couple of "commercial" ones. Most produced what on the surface looked like good data, but not enough to figure out the old bug. The most promising one, the one with version specific knowledge of the VCL and RTL gave the worst failure: sure, it figured out the RTL calls, gave them names, but failed to locate the exported function! The one function I was interested in wasn't shown int the list of entry points, and it should have been straight forward since it's an exported function.
This decompiling attempt should have been easy because:
The code was fairly simple and not a lot of it.
It was a DLL with an exported function, none of the complexity you'd expect from an event-driven exe.
I wasn't interested in re-compilable code, I simply wanted to find an old bug so I can reproduce it.
I didn't ask for Pascal code, assembler would've been good enough.
I knew precisely what the code was doing and how it was doing it. It wasn't cryptic 3rd party code.
My solution
After decompilers failed me I turned to my own trusty Delphi IDE for debugging. I wrote a small Delphi program that directly imports the function from the DLL, created a fake Firbird memory manager DLL so my DLL can load, called my old function with the parameters I knew would give bad results, steped into the code using the debugger and closely watched the FPU registers. After a few failed attempts I finally noticed a value was popped from the FPU stack as integer where it shouldn't have been Integer so I had my BUG: I mistakenly defined an Integer local variable where I should have used Currency. Armed with that knowledge I was able to reproduce the bug.
Only thing that is easier in Delphi is retrieving VCLs.
After using decompilers like DeDe you will get application user interface but without any logic.
So if you want to retrieve only forms and buttons - Delphi is easier than other compilers, but if you want to know what is going on after clicking on the button you'll need to use ollydbg or other (debugger/disassembler) as for other languages that creates executables.
There are pros and cons. I am not sure what angle your referring to as being easier. There is also a huge difference in a 1 form simple application, versus a very in-depth application that has many forms and tons of classes and functions. It's like Notepad versus Office 2013 (given they were coded in delphi, just an example comparing complexity not language).
In a small app, having the extra information that Delphi apps "usually" contain can make it a breeze. However, in a large application it may "help", but you have a million calls to dig through. They may help you get in the near vicinity, but calls inside of calls inside of calls, then multiple returns used as jumps... makes you dizzy. Then if the app "was" packed or protected, some things can still be a garbled mess. While it may work programming wise, reading it can be a lot harder. I was in one the other day, where all of the strings were encrypted, so "referenced text strings" were no help, and the encryption was not a simple md5 or base64, it was some custom algorithm. Maybe an MD5 with a salt, then base64 encoded? I never could get to the exact method on the strings. I knew what some of them were supposed to be, but couldn't reproduce the method, even though it looked like it was base64, it was the base64 of the string already encrypted some how... I dont rely on text strings, but in a large large app, every little bit helps.
Of course, my interpretation of this question, was looking at a Delphi exe in OllyDbg. I could be off base on where you guys were going with this topic, but I feel in regards to Olly and reversing, I am on point (if that was what you were talking about) lol.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
Why decompiling a delphi exe, is so easy, compared to others executables built with other programming languages/compilers?
There are a few things that help with reversing delphi programs:
You get the full form data including the name of event handler methods
All members with published visibility have metadata used with RTTI
The compiler is pretty bad at optimizing. It does no whole program optimization and the assembly is usually a straight forward translation of the original source with only minor optimizations. (At least it was in the versions I used, might have improved since then)
All classes, even those compiled with RTTI off have some level of metadata available. In particular it's possible to get the name and inheritance structure of classes. And for any instance of a class you happen to see in the debugger you can get its VMT and thus its class name.
Delphi uses textfiles describing the content of your form and hooks up event handlers by name. This approach obviously needs enough metadata to deserialize that textual representation of a from and hook up the eventhandlers by name.
An alternative some other GUI toolkits use is auto-generating code that initializes the form and hooks up the event handler with code. Since this code directly uses pointers to the eventhandlers and directly assigns to properties/calls setters it doesn't need any metadata. Which has the side-effect that reversing becomes a bit harder.
It shouldn't be too hard to create a program that transforms a dfm file into a series of hardcoded instructions that creates the form instead. So a tool like DeDe won't work that well anymore. But that doesn't gain you much in practice.
But figuring out which evenhandler corresponds to which control/event is still rather easy. Especially since stuff like FLIRT identifies most library functions. So you just need to breakpoint the one you're interested in and then step into the user code.
The statement you make is false. Delphi is not particularly more easy to decompile than code produced by other mainstream compilers.
For .net languages there is Reflector.
C++ is covered in this Stack Overflow question.
Python/Perl/Ruby etc. are interpreted.
If you were able to prove that the results of decompiling a Delphi executable were of significantly higher quality than in other widely used languages then your question would carry more weight.
Story from the trenches: Decompiling a tiny Delphi DLL
I've been through a Delphi decompiling session myself. It was one of those fake-sounding "I lost my sources" thing, I really did lose the sources for a tiny Firebird UDF library. Now I do no better, I didn't jump right into decompiling because the library was so small and I knew a rewrite would be much faster.
This DLL exports a function that looks like this:
function udf_do_some_math(Number1, Number2:Currency): Currency;
After doing the sane thing and rewriting the function and doing some regression tests I discovered some obscure corner-cases where the new function's result wasn't the same as the old function's result! The trouble was, the new function's result was the correct result, the old DLL contained a BUG and I had to reproduce the BUG - with this function consistency is more important then accuracy.
Again, did the sane thing and tried to "guess" at the BUG. I knew it was a rounding issue but simply couldn't figure out what it was. Finally I decided to give decompilers I try. After all this was a small library, the entry-point was straight-forward and I didn't really need re-compilable code, nor 100% decompilation: I only needed enough to figure out the old BUG so I can reproduce it!
Decompiling failed! I tried lots of different decompilers, including a couple of "commercial" ones. Most produced what on the surface looked like good data, but not enough to figure out the old bug. The most promising one, the one with version specific knowledge of the VCL and RTL gave the worst failure: sure, it figured out the RTL calls, gave them names, but failed to locate the exported function! The one function I was interested in wasn't shown int the list of entry points, and it should have been straight forward since it's an exported function.
This decompiling attempt should have been easy because:
The code was fairly simple and not a lot of it.
It was a DLL with an exported function, none of the complexity you'd expect from an event-driven exe.
I wasn't interested in re-compilable code, I simply wanted to find an old bug so I can reproduce it.
I didn't ask for Pascal code, assembler would've been good enough.
I knew precisely what the code was doing and how it was doing it. It wasn't cryptic 3rd party code.
My solution
After decompilers failed me I turned to my own trusty Delphi IDE for debugging. I wrote a small Delphi program that directly imports the function from the DLL, created a fake Firbird memory manager DLL so my DLL can load, called my old function with the parameters I knew would give bad results, steped into the code using the debugger and closely watched the FPU registers. After a few failed attempts I finally noticed a value was popped from the FPU stack as integer where it shouldn't have been Integer so I had my BUG: I mistakenly defined an Integer local variable where I should have used Currency. Armed with that knowledge I was able to reproduce the bug.
Only thing that is easier in Delphi is retrieving VCLs.
After using decompilers like DeDe you will get application user interface but without any logic.
So if you want to retrieve only forms and buttons - Delphi is easier than other compilers, but if you want to know what is going on after clicking on the button you'll need to use ollydbg or other (debugger/disassembler) as for other languages that creates executables.
There are pros and cons. I am not sure what angle your referring to as being easier. There is also a huge difference in a 1 form simple application, versus a very in-depth application that has many forms and tons of classes and functions. It's like Notepad versus Office 2013 (given they were coded in delphi, just an example comparing complexity not language).
In a small app, having the extra information that Delphi apps "usually" contain can make it a breeze. However, in a large application it may "help", but you have a million calls to dig through. They may help you get in the near vicinity, but calls inside of calls inside of calls, then multiple returns used as jumps... makes you dizzy. Then if the app "was" packed or protected, some things can still be a garbled mess. While it may work programming wise, reading it can be a lot harder. I was in one the other day, where all of the strings were encrypted, so "referenced text strings" were no help, and the encryption was not a simple md5 or base64, it was some custom algorithm. Maybe an MD5 with a salt, then base64 encoded? I never could get to the exact method on the strings. I knew what some of them were supposed to be, but couldn't reproduce the method, even though it looked like it was base64, it was the base64 of the string already encrypted some how... I dont rely on text strings, but in a large large app, every little bit helps.
Of course, my interpretation of this question, was looking at a Delphi exe in OllyDbg. I could be off base on where you guys were going with this topic, but I feel in regards to Olly and reversing, I am on point (if that was what you were talking about) lol.
Just curious - are protocol buffers usable with F#? Any caveats etc.?
I'm just trying to answer this question myself.
Marc Gravell's protobuf-net project worked out-of-the-box with F# because it uses standard .NET idioms. You can use attributes to get serializing without having to write .proto files or do any two-stage compilation, or you can generate the necessary code from standard .proto files. Performance is good for .NET but a lot slower than alternatives like OCaml's built-in Marshal module. However, this library forces you to make every field in every message type mutable. This is really counter-productive because messages should be immutable. Also, the documentation leaves a lot to be desired but, then, this is free software.
I haven't managed to get Jon Skeet's protobuf-csharp-port library to work at all yet.
Ideally, you'd be able to serialize all of the built-in F# types (tuples, records, unions, lists, sets, maps, ...) to this wire format out-of-the-box but none of the existing open source solutions are capable of this. I'm also concerned by the complexity of these solutions: Jon Skeet's is 88,000 lines of C# code and comments (!).
As an aside, I am disappointed to see that Google protocol buffers do not specify standard formats for DateTime or decimal numbers.
I haven't looked at Proto# yet and cannot even find a download for Froto. There is also ProtoParser but it just parses .proto files and cannot actually serialize anything.
There isn't an F# specific one listed here, but there is an OCaml one, or there is a .NET "general" one (protobuf-net).
In all honesty, I simply haven't gotten around to trying protobuf-net with F# objects, in part because I simply don't know enough F#, but if you can create POCOs they should work. They would need to have some kind of mutability (perhaps even just private mutability) to work with protobuf-net, though.
If you are happy to generate a C# DTO and just consume that from F#, then protobuf-net or Jon's port should work just fine.
I'd expect both my own port and Marc Gravell's to work just fine with F#, to the same extent that any other .NET library does. In other words, neither port is written in a way which is likely to produce idiomatic F# code, but they should work.
My port will generate C# code, so you'll need to build that as a separate project for your serialization model - but that should interoperate with F# without any problems. The generated types are immutable (with mutable builders) so that should help in an F# context.
Of course, you could always take the core parts of either project and come up with an idiomatic F# solution too - whether you port the whole project to F# or use the existing libraries with an F# code generator and helper functions, or something like that.
I've heard a lot about the new/improved RTTI capabilities of Delphi 2010, but I must admit my ignorance...I don't understand it. I know every version of Delphi has supported RTTI...and I know that RTTI (Runtime Type Information) allows me to access type information while my application is running.
But what exactly does that mean? Is Delphi 2010's RTTI support the same thing as reflection in .NET?
Could someone please explain why RTTI is useful? Pretend I'm your pointy haired boss and help me understand why RTTI is cool. How might I use it in a real-world application?
RTTI in Delphi is still not quite as full-featured as Reflection in .NET or other managed languages, because it is operating on compiled code, not an Intermediate Language (bytecode). However, it is a very similar concept, and the new RTTI system in Delphi 2010 brings it a lot closer to reflection, exposing an entire object-oriented API.
Pre-D2010, the RTTI was pretty limited. About the only thing I ever remember doing with it was converting an enumerated type to a string (or vice versa) for use in drop-down lists. I may have used it at one point for control persistence.
With the new RTTI in D2010 you can do a lot more things:
XML Serialization
Attribute-based metadata (TCustomAttribute). Typical use cases would be automatic validation of properties and automated permission checks, two things that you normally have to write a lot of code for.
Adding Active Scripting support (i.e. using the Windows script control)
Building a plug-in system; you could do this before, but there were a lot of headaches. I wasn't able to find a really good example of someone doing this from top to bottom, but all of the necessary functions are available now.
It looks like someone's even trying to implement Spring (DI framework) for Delphi 2010.
So it's definitely very useful, although I'm not sure how well you'd be able to explain it to a PHB; most of its usefulness is probably going to be realized through 3rd-party libraries and frameworks, much the same way it works in the .NET community today - it's rare to see reflection code sitting in the business logic, but a typical app will make use of several reflection-based components like an Object-Relational Mapper or IoC Container.
Have I answered the question?
D2010's extended RTTI is a lot like C#'s reflection. It gives you the ability to get at any field of an object, or inspect its methods. This has all sorts of potential uses. For example, if you can read any field of an object, you can write serialization code that can work with any object. And the ability to inspect methods and obtain their name and signature makes a class much easier to register with a scripting engine.
To me, that's the primary advantage of extended RTTI: The ability to write code that works with any class by examining its members, instead of writing different versions of the same code over and over, tailored to each individual class.
Most people probably won't use it in a real world application.
The people who will use it are the framework builders. Frameworks such as DUnit make extensive use of RTTI.
With the new RTTI capabilities we should expect to start seeing more advanced frameworks and tools appearing, similar to what is available for .NET. These frameworks will revolutionise your development moreso than RTTI will on it's own.
RTTI in Delphi has always been important since version 1.0. Classic RTTI features include the "published" properties section of Classes, which allowed the Object Inspector and the component designtime features to work. For my purposes, I would often use Published class properties to allow for enumeration of those properties at runtime. To store things from my objects to disk, for persistence.
The Delphi 2010 RTTI extends this classic RTTI massively, so much so that you could be forgiven for thinking Delphi did not even have RTTI until delphi 2010.
I would say the #1 most useful applications of "The New RTTI" are (as several other answers already state) going to be in Frameworks written by the gurus, that:
Handle persistence to files or databases. Database and configuration or document saving/loading frameworks and components would use this under the hood.
Handle pickling/marshalling/encoding/decoding to and from various
over-the-wire formats, like JSON, XML, EDI, and other things.
Unit testing was mentioned by someone else (JUnit), but I think perhaps the same frameworks could be really handy for debug and error reporting tools. Given an object passed as a parameter, on the stack, why not have bug reports that can dump all the data that was passed along to a function that failed, and not just a list of functions?
As you can see, some creative people are likely to think of even more uses for this. You could say, that though it does not bring parity to .NET reflection (which another answer speaks more about), it does bring a lot of "dynamic language" features (Think of Perl, Python, JavaScript) to an otherwise strongly typed static-type systems world of Delphi.
For me, personally, extended RTTI gave a possibility to retrieve calling convention from the method pointer. However, currently, that code is under conditional directive, because i am not satisfied with it.
(Critics and suggestions on workarond with basic RTTI are welcome, tho)
Look for TMS Aurelius and you will see that RTTI Attributes are very useful in terms of creating an ORM DataBase Framework and XML Serialization into pure objects and the opposite too.
You are supposed to care because they put it on the box. Clearly they think that some people will care.
Whether you actually have a use for it is entirely dependent on the nature of your projects. Since you didn't have it before and don't understand why having it now is a benefit, this would suggest to me that you don't have a use for it. It's then up to you whether to spend the time researching the subject further in order to discover whether you might be able to find a use for it.
Whether that is the most productive use of your time in relation to your projects, again is something only you can know.
I need to return objects from a DLL made in Delphi, to an app made in Delphi, too. The objective is to do a subsystem that can be modify in the future without to modify the main app. So, I imagine developing the subsystem in a DLL is a (good??) idea... i am programming in Windows XP, Delphi 7. I did read DLLs only return basic data type, but there must to be a way to do that...
Best regards.
I prefer to apply COM to such a model, which allows you to create external "objects" which you then can direct from within your application. Creating COM objects in Delphi is extremely simple, use your ActiveX creation methods to create an ActiveX library and then create COM objects. You then use the interface unit in your main application, and when you CoCreate an instance of the object it loads the appropriate DLL. The only tricky part of this is that your com objects must be registered on the system to function properly... which in the Win7/Vista world requires elevated access... although once this is done, it is seamless.
The BEST way is to use a COM wrapper as suggested by skamradt.
It is possible, but not a good idea to pass object references as pointers to DLL's. Refer particularly to Peter Haas's comments.
If you do pass an object from a Delphi DLL to a Delphi app you will have the following problems:
You must use the same version of Delphi for the app and DLL.
Both app and DLL MUST have the same implementation of the object - or at least identical layout of all fields in the class - OK if you are using standard objects such as TStringList.
You should use shared memory or runtime packages, otherwise you will get weird access violations.
I have had to maintain code where this was done - it was nightmarish as you couldn't change classes without a lot of re-compiling.
You can use interfaces and most of your problems, wrt compiler/rtl versions or even other languages just go away.
Delphi's interfaces are always IUnknown-compatible, which makes them compatible with most OO-enabled languages on Windows.
One thing to keep in mind, though: Don't use AnyString, stick to WideString, which is the Stringtype used by COM.
A platform- and language independent way could be to exchange serialized objects.
It has a performance impact, but it has advantages too: the DLL works without modifications with other languages and platforms like .Net or Java (via JNA Java Native Access). It does not depend on any special features of the operating system so it can as well be used on Linux or MacOS if you compile the library with Free Pascal.
For the serialization, you could use JSON or XML. There are open source libraries for Delphi, for example SuperObject and OmniXML.