Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I have a few large (~600k lines of code) Delphi projects. They include some custom components which our team has developed.
Often, when I call up code completion with ctrl+space or just by pressing ".", the IDE locks up and thinks really hard for a long time. Sometimes the delay can be a full minute, or more. Other times, it pops up immediately with suggestions.
What factors influence the performance of intellisense in Delphi? Is there any way I can improve this performance?
My best solution so far is to turn off the automatic completion, and use ctrl+space when I need to meditate quietly for a minute or so.
I can't help but mention that VS2005, VS2008, and XCode all seem to give virtually instant intellisense feedback (although I've never tried it on a project this large).
As an alternative, I've offered this suggestion.
Delphi Code Insight invokes the compiler dll to do a custom compile when the user requests Code Insight (Ctrl+Space, '.', etc). This custom compile does a build in the unit and skips over codegen, linking, etc until it reaches your current offset in the file buffer. With this in mind, the unit list that the compiler sees before it gets to your current position will play a large factor in determining the speed of the Code Insight operation. There may be a unit (or multiple units) that are causing a hefty file system dependency, etc. It's quite possible that reordering the uses clause, refactoring the uses clause to be in multiple files, or removing units in the uses clause that aren't necessary for your current unit to compile may improve performance. Additionally, using packages or shortening your unit search path may improve CI response time.
Be sure to explicitly include all the units(*) used by your project in the dpr.
Do not rely on the search path to find a unit called from another unit, add it to the dpr. The dpr will be much longer but all the compilation related things will be faster, including code-insight.
(*) not the units of the installed components.
I don't know which version you are using, but much faster code completion is one of the things I like most about Delphi 2009.
This is a long-standing issue with Delphi, and I had to resort to turning off automatic completion. After working that way for a while, I was very happy with it. Even if it only takes a fraction of a second, having the IDE lag my typing was disconcerting and interrupted my flow. Much nicer with the automatics off, IMO.
I just came across this problem myself, I fixed it by removing a dead network link from my environment library path. Solved my issue 100%.
Do you include the sources directories for your teams custom components to be in the library path? It would be interesting to see the speed difference if only the component DCU files are in the library path, versus having the source files there too.
Related
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
We are a little team working on a Rails 2.3 project. A short description: this project has currently
460 000 lines of Ruby, CSS, JS and YML (including some plugins and libraries)
350 ActiveRecord classes (only the project's classes)
some Rspec tests in the recent classes and controllers (not enough)
45 required gems
A database with 2 Go data.
The project is divided in approximately 15 'modules' (not as Rails module), with each module talking to others modules. This project exists from several years, and has been improved and maintained with several people, not necessary Rails experts.
The current main problem is that several parts of the code are difficultly maintainable, are unoptimized and not enough 'thoughful' (there is hack everywhere). The CSS files are nearly unreadable.
Our team is cogitating about refactoring this project. We have a few solutions :
Create a new project from scratch, and include functionnalities one by one. The advantage of this solution is that we have all the facets of the projects, and we can take prettier decitions about code design. An other advantage is that we can update to Rails 3. A lack of this method is that we have to maintain 2 projects in parrallel, when we have to include new functionnalities.
Update existing code, module by module. The advantage of this solution is that we keep only one project to maintain. But there is several lacks... How to go from an old module to a new module ? How to proceed between the new and old class names ? How to know where old code is inused and must be deleted ?... and even more.
Does anyone already have done a big refactoring like this ?
Does anyone have experience feedback about this ?
Does anyone have an other solution ?
Thanks for reading !
The answer here really depends how much autonomy you have and how much time you can devote to the refactor. With a project of this size, almost certainly you can't just stop all work and devote 100% of your time towards the cleanup ROR 3 migration (cough cough, migration!).
If my assumption is correct, then my advice would be to tackle this in small chunks using the second method (e.g. update existing code). In addition to what you've outlined, I would encourage a few other things:
Write tests before you refactor a piece of code. This will help you be confident that you're not introducing problems (emphasis on the help... some breakage is of course possible).
Where possible, consider breaking modules into separate, independent gems. This will help you to track external dependencies and figure out where all 45 of those external gems are getting used. It will also help limit the scope of your tests.
Frankly the second method is much safer for your job as a developer, since it allows you to improve the codebase while simultaneously responding to other business needs. If a portion of the refactor is going too slowly, you can also just stop and pick up elsewhere, since the rest of the codebase is still intact.
Refactoring is a lot of work, and will take a long time and a lot of thought.
In comparison with a totally new project, the problem is that you have to maintain a working project, and probably will have to support some bug fixes and feature requests as you go.
on the other hand you have a much better understanding of the problem you are solving, and you can use the existing software as your defacto "functional spec"
identify the Interfaces between your "modules", by listing each inter-dependency.
Clean up these "interfaces" if necessary
Formalize interfaces by Writing unit tests to test the input and output of each module
create totally new modules as drop in replacements. architect from scratch, and do your ruby upgrade here. Make sure your new modules pass the same tests as your old modules.
If necessary (as an interim solution) wrap various incompatible sections to get them working together. this may be slow, but it's a stopgap.
good luck.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
Why decompiling a delphi exe, is so easy, compared to others executables built with other programming languages/compilers?
There are a few things that help with reversing delphi programs:
You get the full form data including the name of event handler methods
All members with published visibility have metadata used with RTTI
The compiler is pretty bad at optimizing. It does no whole program optimization and the assembly is usually a straight forward translation of the original source with only minor optimizations. (At least it was in the versions I used, might have improved since then)
All classes, even those compiled with RTTI off have some level of metadata available. In particular it's possible to get the name and inheritance structure of classes. And for any instance of a class you happen to see in the debugger you can get its VMT and thus its class name.
Delphi uses textfiles describing the content of your form and hooks up event handlers by name. This approach obviously needs enough metadata to deserialize that textual representation of a from and hook up the eventhandlers by name.
An alternative some other GUI toolkits use is auto-generating code that initializes the form and hooks up the event handler with code. Since this code directly uses pointers to the eventhandlers and directly assigns to properties/calls setters it doesn't need any metadata. Which has the side-effect that reversing becomes a bit harder.
It shouldn't be too hard to create a program that transforms a dfm file into a series of hardcoded instructions that creates the form instead. So a tool like DeDe won't work that well anymore. But that doesn't gain you much in practice.
But figuring out which evenhandler corresponds to which control/event is still rather easy. Especially since stuff like FLIRT identifies most library functions. So you just need to breakpoint the one you're interested in and then step into the user code.
The statement you make is false. Delphi is not particularly more easy to decompile than code produced by other mainstream compilers.
For .net languages there is Reflector.
C++ is covered in this Stack Overflow question.
Python/Perl/Ruby etc. are interpreted.
If you were able to prove that the results of decompiling a Delphi executable were of significantly higher quality than in other widely used languages then your question would carry more weight.
Story from the trenches: Decompiling a tiny Delphi DLL
I've been through a Delphi decompiling session myself. It was one of those fake-sounding "I lost my sources" thing, I really did lose the sources for a tiny Firebird UDF library. Now I do no better, I didn't jump right into decompiling because the library was so small and I knew a rewrite would be much faster.
This DLL exports a function that looks like this:
function udf_do_some_math(Number1, Number2:Currency): Currency;
After doing the sane thing and rewriting the function and doing some regression tests I discovered some obscure corner-cases where the new function's result wasn't the same as the old function's result! The trouble was, the new function's result was the correct result, the old DLL contained a BUG and I had to reproduce the BUG - with this function consistency is more important then accuracy.
Again, did the sane thing and tried to "guess" at the BUG. I knew it was a rounding issue but simply couldn't figure out what it was. Finally I decided to give decompilers I try. After all this was a small library, the entry-point was straight-forward and I didn't really need re-compilable code, nor 100% decompilation: I only needed enough to figure out the old BUG so I can reproduce it!
Decompiling failed! I tried lots of different decompilers, including a couple of "commercial" ones. Most produced what on the surface looked like good data, but not enough to figure out the old bug. The most promising one, the one with version specific knowledge of the VCL and RTL gave the worst failure: sure, it figured out the RTL calls, gave them names, but failed to locate the exported function! The one function I was interested in wasn't shown int the list of entry points, and it should have been straight forward since it's an exported function.
This decompiling attempt should have been easy because:
The code was fairly simple and not a lot of it.
It was a DLL with an exported function, none of the complexity you'd expect from an event-driven exe.
I wasn't interested in re-compilable code, I simply wanted to find an old bug so I can reproduce it.
I didn't ask for Pascal code, assembler would've been good enough.
I knew precisely what the code was doing and how it was doing it. It wasn't cryptic 3rd party code.
My solution
After decompilers failed me I turned to my own trusty Delphi IDE for debugging. I wrote a small Delphi program that directly imports the function from the DLL, created a fake Firbird memory manager DLL so my DLL can load, called my old function with the parameters I knew would give bad results, steped into the code using the debugger and closely watched the FPU registers. After a few failed attempts I finally noticed a value was popped from the FPU stack as integer where it shouldn't have been Integer so I had my BUG: I mistakenly defined an Integer local variable where I should have used Currency. Armed with that knowledge I was able to reproduce the bug.
Only thing that is easier in Delphi is retrieving VCLs.
After using decompilers like DeDe you will get application user interface but without any logic.
So if you want to retrieve only forms and buttons - Delphi is easier than other compilers, but if you want to know what is going on after clicking on the button you'll need to use ollydbg or other (debugger/disassembler) as for other languages that creates executables.
There are pros and cons. I am not sure what angle your referring to as being easier. There is also a huge difference in a 1 form simple application, versus a very in-depth application that has many forms and tons of classes and functions. It's like Notepad versus Office 2013 (given they were coded in delphi, just an example comparing complexity not language).
In a small app, having the extra information that Delphi apps "usually" contain can make it a breeze. However, in a large application it may "help", but you have a million calls to dig through. They may help you get in the near vicinity, but calls inside of calls inside of calls, then multiple returns used as jumps... makes you dizzy. Then if the app "was" packed or protected, some things can still be a garbled mess. While it may work programming wise, reading it can be a lot harder. I was in one the other day, where all of the strings were encrypted, so "referenced text strings" were no help, and the encryption was not a simple md5 or base64, it was some custom algorithm. Maybe an MD5 with a salt, then base64 encoded? I never could get to the exact method on the strings. I knew what some of them were supposed to be, but couldn't reproduce the method, even though it looked like it was base64, it was the base64 of the string already encrypted some how... I dont rely on text strings, but in a large large app, every little bit helps.
Of course, my interpretation of this question, was looking at a Delphi exe in OllyDbg. I could be off base on where you guys were going with this topic, but I feel in regards to Olly and reversing, I am on point (if that was what you were talking about) lol.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
Why decompiling a delphi exe, is so easy, compared to others executables built with other programming languages/compilers?
There are a few things that help with reversing delphi programs:
You get the full form data including the name of event handler methods
All members with published visibility have metadata used with RTTI
The compiler is pretty bad at optimizing. It does no whole program optimization and the assembly is usually a straight forward translation of the original source with only minor optimizations. (At least it was in the versions I used, might have improved since then)
All classes, even those compiled with RTTI off have some level of metadata available. In particular it's possible to get the name and inheritance structure of classes. And for any instance of a class you happen to see in the debugger you can get its VMT and thus its class name.
Delphi uses textfiles describing the content of your form and hooks up event handlers by name. This approach obviously needs enough metadata to deserialize that textual representation of a from and hook up the eventhandlers by name.
An alternative some other GUI toolkits use is auto-generating code that initializes the form and hooks up the event handler with code. Since this code directly uses pointers to the eventhandlers and directly assigns to properties/calls setters it doesn't need any metadata. Which has the side-effect that reversing becomes a bit harder.
It shouldn't be too hard to create a program that transforms a dfm file into a series of hardcoded instructions that creates the form instead. So a tool like DeDe won't work that well anymore. But that doesn't gain you much in practice.
But figuring out which evenhandler corresponds to which control/event is still rather easy. Especially since stuff like FLIRT identifies most library functions. So you just need to breakpoint the one you're interested in and then step into the user code.
The statement you make is false. Delphi is not particularly more easy to decompile than code produced by other mainstream compilers.
For .net languages there is Reflector.
C++ is covered in this Stack Overflow question.
Python/Perl/Ruby etc. are interpreted.
If you were able to prove that the results of decompiling a Delphi executable were of significantly higher quality than in other widely used languages then your question would carry more weight.
Story from the trenches: Decompiling a tiny Delphi DLL
I've been through a Delphi decompiling session myself. It was one of those fake-sounding "I lost my sources" thing, I really did lose the sources for a tiny Firebird UDF library. Now I do no better, I didn't jump right into decompiling because the library was so small and I knew a rewrite would be much faster.
This DLL exports a function that looks like this:
function udf_do_some_math(Number1, Number2:Currency): Currency;
After doing the sane thing and rewriting the function and doing some regression tests I discovered some obscure corner-cases where the new function's result wasn't the same as the old function's result! The trouble was, the new function's result was the correct result, the old DLL contained a BUG and I had to reproduce the BUG - with this function consistency is more important then accuracy.
Again, did the sane thing and tried to "guess" at the BUG. I knew it was a rounding issue but simply couldn't figure out what it was. Finally I decided to give decompilers I try. After all this was a small library, the entry-point was straight-forward and I didn't really need re-compilable code, nor 100% decompilation: I only needed enough to figure out the old BUG so I can reproduce it!
Decompiling failed! I tried lots of different decompilers, including a couple of "commercial" ones. Most produced what on the surface looked like good data, but not enough to figure out the old bug. The most promising one, the one with version specific knowledge of the VCL and RTL gave the worst failure: sure, it figured out the RTL calls, gave them names, but failed to locate the exported function! The one function I was interested in wasn't shown int the list of entry points, and it should have been straight forward since it's an exported function.
This decompiling attempt should have been easy because:
The code was fairly simple and not a lot of it.
It was a DLL with an exported function, none of the complexity you'd expect from an event-driven exe.
I wasn't interested in re-compilable code, I simply wanted to find an old bug so I can reproduce it.
I didn't ask for Pascal code, assembler would've been good enough.
I knew precisely what the code was doing and how it was doing it. It wasn't cryptic 3rd party code.
My solution
After decompilers failed me I turned to my own trusty Delphi IDE for debugging. I wrote a small Delphi program that directly imports the function from the DLL, created a fake Firbird memory manager DLL so my DLL can load, called my old function with the parameters I knew would give bad results, steped into the code using the debugger and closely watched the FPU registers. After a few failed attempts I finally noticed a value was popped from the FPU stack as integer where it shouldn't have been Integer so I had my BUG: I mistakenly defined an Integer local variable where I should have used Currency. Armed with that knowledge I was able to reproduce the bug.
Only thing that is easier in Delphi is retrieving VCLs.
After using decompilers like DeDe you will get application user interface but without any logic.
So if you want to retrieve only forms and buttons - Delphi is easier than other compilers, but if you want to know what is going on after clicking on the button you'll need to use ollydbg or other (debugger/disassembler) as for other languages that creates executables.
There are pros and cons. I am not sure what angle your referring to as being easier. There is also a huge difference in a 1 form simple application, versus a very in-depth application that has many forms and tons of classes and functions. It's like Notepad versus Office 2013 (given they were coded in delphi, just an example comparing complexity not language).
In a small app, having the extra information that Delphi apps "usually" contain can make it a breeze. However, in a large application it may "help", but you have a million calls to dig through. They may help you get in the near vicinity, but calls inside of calls inside of calls, then multiple returns used as jumps... makes you dizzy. Then if the app "was" packed or protected, some things can still be a garbled mess. While it may work programming wise, reading it can be a lot harder. I was in one the other day, where all of the strings were encrypted, so "referenced text strings" were no help, and the encryption was not a simple md5 or base64, it was some custom algorithm. Maybe an MD5 with a salt, then base64 encoded? I never could get to the exact method on the strings. I knew what some of them were supposed to be, but couldn't reproduce the method, even though it looked like it was base64, it was the base64 of the string already encrypted some how... I dont rely on text strings, but in a large large app, every little bit helps.
Of course, my interpretation of this question, was looking at a Delphi exe in OllyDbg. I could be off base on where you guys were going with this topic, but I feel in regards to Olly and reversing, I am on point (if that was what you were talking about) lol.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 9 years ago.
How to improve our productivity when starting new project using Delphi?
For me, I create a template project including splash screen, primary data module with default component and all data-aware subroutines (open, close, query, default when failed to open DB, etc), and save the template in my repository. (I use all my own frames and unit in the project, so everything always be created automatically when I choose : New Project and choosing my template.)
Questions:
Is there any other way to improve reusability using Delphi?
What free open source framework (like Jedi) / IDE PLugins (like GExpert) you think is best to use using Delphi?
Pardon me if this question has been asked before.
For Q1: Use mature libraries which do for you the grunt work: DevExpress -or- TMS for UI (JEDI's JVCL also is very good), FastReport for reporting. Choose also a good connectivity framework (it depends on your db back-end) in order to have many day-to-day tasks solved OOTB. Also, if you're familiar with OPF have a look at InstantObjects. I heard that's very nice.
For Q2: See the IDE enchancements from cnpack.org
Also it will help to slowly build your own specialized framework/set of tools. Not something very big but as a quick way to get some things done in your way. Always try to design for reuse, even if it takes a little bit longer in the beginning.
For larger applications, the key to productivity is to work at a higher abstraction level than data module and database.
The database has a small set of types. In your domain you are likely to have default mappings from domain types to database types. You'll also have validators and formatters for them.
You'll have default reports, filters and search (windows, panels, sql queries) for your domain entities. You'll have role-based access control to them.
Take a look at domain-driven development.
One of my big gains in reusability in recent years has been moving to an ORM layer (I use tiopf) to separate out my business objects from the database.
As an example, I have db persisted, background threaded email objects. To add them to another application, I add in the required units and add a few lines of configuration (table/field mapping). Similarly I have user objects, generic lookup lists and the like all of which can be added to different projects for the price of initial configuration. This works on different databases without any changes other than configuration.
tiopf is my ORM of choice (read my overview here) but there are numerous others.
Defining the exact range of applications you want to make is a good first step. Improving productivity is nearly always related to specialization. General tools is just an extremely small percentile of productivity. I'd rather search (or make) specialized frameworks in my line of work if I really wanted to boost productivity.
I don't use GExperts, or another plugin. I don't benefit greatly (being able to grep just fine on the cmdline) in productivity, and any crash that can be avoided due to problems in the plugin is then a pre.
I used ModelMaker with Delphi. It is really nice tool that lets you draw objects and then can generate code based on templates you write.
It can speed up things when there is number of similar classes in project.
For Q2: GExperts is useful. The grep search comes in particularly handy, although the search / replace can cause odd side affects (inserting characters it shouldn't!).
I know you mentioned free / open source, but Castalia is very good. The refactoring methods work well and I like the structural highlighting, which makes it much easier to work with code (I did originally use CodeRush for this but it went Visual Studio only). The Bookmark stack is also handy for quick going back and forth through code.
Also, if you are a team, look for ways to improve your development process. Besides using Delphi as your implementation language, what are your project management methods? What source code management system do you use? What is your build system? Do you use automated testing methods? etc.
When I introduced Scrum at a previous employer, we got an almost immediate 50% improvement in team productivity. So check out the various Agile methodologies.
For true reusability, try to think in interfaces and try to black box as much as possible. Patterns are everywhere, research them and put them into practice.
When dealing with objects, as much as possible use the abstract, or an interface rather than a concrete implementation. Just be careful of taking this too far. Too many abastractions can add complexity and make debugging harder.
Units containing your business rules should be used by your gui. Units containing business rules should never themselves directly use gui units.
When I ask questions like this about "what is the best thing to do or to use" they are quickly removed from the site by the moderators.
In my opinion, you are right to use the sample template. Also #John Thomas is correct. But you can go beyond:
Use a template for database procedures and functions to work with requests;
Try to put the most of the database logic inside the database using stored procedures and functions. This way you don't have to worry about selecting what is common use and what is specific to current project;
Use a template for main menu and main screen;
Instead of crating many windows, create one TFrame for each module of your application and reserve a place in the main screen to load them. Create an object of this frame in memory only when the user clicks it's menu. This way, your system gets faster and more memory economic;
Reuse those frames by creating base frames with the common functionality and layout and them create new frames descendants from it.
Just great!
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 14 years ago.
Improve this question
I am really wondering why FogBugz when installed locally insists that DEP is turned off?
FogBugz 6 (and earlier) requires that Data Execution Prevention (DEP) be disabled on versions of Windows that have DEP, because of a third-party COM component that we use for parsing email. We will fix this in the next major release of FogBugz: FogBugz will no longer use this third-party component (in fact, the next version of FogBugz will not use any COM components).
Turn it on and see where it crashes with a debugger :) I ran across some COM components that would execute some code from a data block that triggered a DEP exceptions. I would be willing to guess FogBugz is also accessing some native components somewhere that are doing the same.
Code that attempts to patch or insert hook into other modules within it's address space often won't work unless DEP is disabled, or the appropriate memory protection options are set for the installed hook.
This is a common technique with some frameworks (eg Delphi) where 'patches' are applied dynamically ar run-time to fix bugs that the vendor has not yet address.
I just don't like the idea to have DEP turned OFF in a Server environment because a modern state of the art software just can't handle it. Specially since it is the only software that I have tried over the years that have required it.
Its during the installation that I came across the DEP alert.
As the FogBugz link above says, " Be warned, however, that FogBugz will not function properly with DEP enabled.".
Not knowing the specifics of FogBugz, but...
The most common reason for turning off DEP is programmatically generated thunks on the stack or the heap. The Windows Kernel emulates the most common ones but "most common" wasn't very good coverage.
The second most common reason for turning off DEP is incorrectly linked code segments that appear to be data segments.
The third most common reason is machine code in strings. In general, this is really bad style but sometimes on Windows it cannot be helped.
The fourth most common reason is some algorithm in the code assumed the stack layout. DEP messes with that.
Or perhaps the program really is running code on a heap buffer.