We have large commercial app that we want to convert from Delphi 6 to 2010. Approx 10 3rd party component sets, all with source code... I have heard warnings about Unicode with 2010 - Does anyone have experience and or suggestions?
There are many resources available that you can read and that you will assist in the migration from Delphi 6 to Delphi 2009/2010 (Unicode).
You can use these articles as a guide.
Unicode Migration Statistics Tool (This utility will hopefully assist you in collecting useful statistics
on how hard (or not) it would be to migrate your older applications to
Unicode.)
Delphi 2009 and Unicode
Delphi 2009 strings explained by example
Upgrading a major project to Delphi 2009
Delphi and Unicode
Dr. Bob Delphi 2009 Unicode
Delphi 2009 - Unicode in Type Libraries
On Strings and Unicode in Delphi 2009
Delphi in a Unicode World Part I: What is Unicode, Why do you need it, and How do you work with it in Delphi?
Delphi in a Unicode World Part II: New RTL Features and Classes to Support Unicode
Delphi in a Unicode World Part III: Unicodifying Your Code
CodeRage 4 : Using Unicode and Other Encodings in your Programs
Bye.
You'll find some useful answers in these StackOverflow questions:
Move project from Delphi 3 to Delphi 2010
When and Why Should I Use TStringBuilder?
Convert function to delphi 2009/2010 (unicode)
Unicode problems with Delphi 2009 / 2010 and windows API calls
Also, for what it is worth, I purchased Marco Cantu's Delphi 2009 Handbook. It was all I needed to make a relatively smooth converstion from Delphi 4 to Delphi 2009 in only a few weeks.
I do, however, recommend that you ensure your 3rd party packages have a Delphi 2009 upgrade, or you may have some real difficulties. Converting your own code is one thing. Converting someone else's is another.
I use two 3rd party packages, both with source code. Both had upgrades available, and the developer of one of them wrote that he had a lot of trouble upgrading his very complex component to the Unicode of Delphi 2009. It took him a few months, but he completed it. And as a result, I had little trouble with my implementation of his component when I did my upgrade.
i've been in the same circumstance recently. you mostly need to pay attention to the "edges" of the app. INI files, file I/O, log files, etc. win API calls from delphi work since they've now connected the unicode API calls instead. check each 3rd party component set to make sure they're at least ready for Delphi 2009...better yet 2010. even my use of databases simply wasn't an issue...nearly everything worked right away. it just wasn't a big deal. anything that relies on the size of a character should be reviewed.
really the transition of most concern is 2007_or_earlier --> 2009_or_later.
there are plenty of discussions/blog entries about it. you could read, read, read...or you could get started & see what happens. (i did some of both). i'm sure there are "stack overflow" issues discussing your question. i'm not pretending to give a detailed description of what could happen.
it's simply not as scary as it sounds.
Approx 10 3rd party component sets, all with source code.
One thing I'd add is if the component doesn't support Delphi 2009/2010, don't try to upgrade it by hacking the code.
Following is what I posted on How do the new string types work in Delphi 2009/2010?:
See Delphi and Unicode, a white paper written by Marco Cantù and I guess
The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!), written by Joel.
One pitfall is that the default Win32 API call has been mapped to use the W (wide string) version instead of the A (ANSI) version, for example ShellExecuteA If your code is doing tricky pointer code assuming internal layout of AnsiString, it will break. A fallback is to substitute PChar with PAnsiChar, Char with AnsiChar, string with AnsiString, and append A at the end of Win32 API call for that portion of code. After the code actually compiles and runs normally, you could refactor your code to use string (UnicodeString).
I must add this article from Carey Jensen to others mentioned. It is labeled: "Delphi Unicode Migration for Mere Mortals: Stories and Advice from the Front Lines" (in english).
http://www.danysoft.com/free/delphiunicodemigration.pdf
As you can see in the title of it, you will find many experiences and hints and tips. I think it is the answer to your question. After carefully read it, you will sure knows what to do next. :)
Found in: http://www.danysoft.com/productos/migrar-aplicaciones-a-delphi-xe-o-cbuilder-xe.html
Another point to take care of, is the usage of Variant types with strings and the VarType function testing for strings: one needs to use varUString instead of varString.
Assuming AValue is of type Variant and has being assigned a Unicode string value, the following won't work:
if VarType(AValue) = varString then ...
and needs to be changed to
if VarType(AValue) = varUString then ...
I am currently converting a legacy system to a new platform and need to extract strings from the old systems resource files.
The old system was written in Delphi and the strings are kept in files called .dfm. I have no trouble locating the strings and for English and other European languages there is no problem. The trouble comes when I try to extract strings in Japanese. I have used Notepad++ and it seems to me that the program don´t recognice the correct encoding. I get Japanese symbols but they don´t seem to match what is in the GUI. Notepad++ shows signs in something called GB2312(Simplified Chinese). But it looks weird.
My question is, does anyone have any tips on programs/text editors that are good at operations like this?
Also I'm grateful for any tips that might help me along the way.
Assuming that your issue is simply that Notepad++ is incorrectly guessing the encoding you can solve the problem by manually setting the encoding in Notepad++, like this:
Notepad++ itself already handles encoding issues. To make it to desired encoding, like Unicode;
first, copy all the contents of the file,
choose Unicode without BOM in the menu,
last, replace all contents with copied contents
save the file
Your contents will then be in your desired encoding.
Strings are kept not [just] in DFMs in Delphi. Only forms and associated text are. So you would to review all the code as well.
As for DFMs - before Delphi 2009 DFMs didn't use Unicode so you must know what charset was used. That was one of big problems with localization and internationalization of Delphi applications.
Delphi 7 and ShellListView it is possible with unicode?
I tried ShellListView, LMD, bsSkinFileListView and TMS Unicode Component. In list i see unicode filenames as i.e.: "????????.txt". Noting works.
Anyone have some idea how to list files/dirs as Windows Explorer with normal visible filenames?
Thanks
Delphi 7 doesn't support Unicode VCL components. You need to upgrade to the latest version of Delphi where all the available shell components have good support for Unicode. Trying to do this in Delphi 7 is simply impractical and nobody is going to invest time doing that nowadays.
Delphi 2009 and above support unicode. I have few legacy pascal source files that I wish to make it compile in Delphi 2009/2010 as well as Delphi 2007 and below.
A quick and safe way is replace
String to AnsiString
PChar to PAnsiChar
Char to AnsiChar
Is there any utility available that able to parse .pas file and make such replacement?
There is a tool for pointing out areas that might need attention:
http://cc.embarcadero.com/Item/27398
It doesn't convert it automatically, grep would do that but as mghie said it's not that simple.
You can use sed for that.
sed -i bak -e "s/string/AnsiString/g" *.pas
It would be a very bad idea, though. There's no reason your code shouldn't compile in all Delphi versions. The meaning of "string" has changed, but so what? Your Delphi 2007 code doesn't need to be used with your Delphi 2009 code. The DCU file formats are different, so you'd have to recompile anything you change anyway.
By changing everything to AnsiString, you're essentially rejecting everything new that Delphi 2009 offers. If that's what you want to do, you could have saved yourself a lot of money by simply not upgrading to Delphi 2009 at all. Why buy a product and then not use any of its features? Since everything else in the product is Unicode, your program's performance will go down the tubes as it continually converts between string formats. You'll also drown in compiler warnings from all the conversions.
Don't force square pegs into round holes, especially when you have a perfectly good set of round pegs sitting right next to you.
My company's main application is mostly written in C++ (with some Delphi code and components). We are upgrading from RAD Studio 2007 to 2010 for the next release, starting in about a week. What do I need to know to ensure this upgrade goes smoothly?
Points I have thought of so far are:
Unicode. This one looks really complicated. Our app contains a horrible mix of std::string-s and AnsiString-s with casts to and from them. I have lots of questions about this, such as "is wstring capable of holding everything a UnicodeString can, and should we just do a search/replace", or "should we avoid all C++ string types altogether and use UnicodeString", "can we change all event handlers to use String though the existing .HPPs event handler method prototypes were compiler-translated to AnsiString", right down to basics such as "should we prefix all strings with L, or is the compiler smart enough with Unicode enabled to use Unicode strings", etc. Any insight on this would be really appreciated.
We also need backwards compatibility. Our app uses its own binary tuple format that currently stores strings as an array of bytes. I need to upgrade this to read old files and, presumably, write new Unicode strings as well. How do I handle Unicode strings embedded in a binary format? Is there any generic way where I can point a UnicodeString at an array of bytes, that may be originally written as either ANSI bytes or Unicode, and it will figure out what they are?
Third-party components. We use SpTBX mainly, and it appears to be compatible.
Project upgrades. The standard advice in the Codegear forums seems to be to manually recreate all project files when upgrading. This is an awful lot of work (7 projects (mostly libs) in our main app, plus half a dozen DLLs, a lot of files.) Is there any way to automate this?
How's the linker look? We traditionally have a lot of trouble with the linker randomly crashing or running out of resources, though it got a lot better in 2007. This is one reason our main application is split into several libs - the linker cannot (hopefully, "could not, but now can"?) handle it otherwise.
I know there's a new type library editor and format (it stores the IDL, ie text, and generates the TLB dynamically?) How well does this handle upgrading existing COM projects with a TLB? We have Delphi code and TLB that are built into the C++ application.
Is there anything else I should be considering or be aware of?
I have found:
2007 and 2010 co-existing. I'm not sure I trust this answer since I have had issues with 2006 and 2007 on the same machine before.
several answers about Unicode: writing strings with 2009 and generic transition to Unicode text but none are answers for concerns, or the C++Builder-specific parts at all.
This question about guidelines upgrading to 2009 but though the answers are helpful, they don't answer all the Unicode-related issues above.
[Edit: added] Codegear documents for Unicode in RAD Studio and things to look for when converting to Unicode
Project upgrades. The standard advice in the Codegear forums seems to be to manually recreate all project files when upgrading. This is an awful lot of work (7 projects (mostly libs) in our main app, plus half a dozen DLLs, a lot of files.) Is there any way to automate this?
There is: just use the IDE's project importer :)
Seriously, I would just try importing the projects, and then go investigate if it doesn't seem to work.
How's the linker look? We traditionally have a lot of trouble with the linker randomly crashing or running out of resources, though it got a lot better in 2007. This is one reason our main application is split into several libs - the linker cannot (hopefully, "could not, but now can"?) handle it otherwise.
I've had almost no trouble with ILINK anymore since C++Builder 2009. I've occasionally read that others experienced out-of-memory errors, but someone in the newsgroups has discovered a workaround:
https://forums.embarcadero.com/thread.jspa?messageID=140012&tstart=0#140012
Also, as you can read here, the compiler got a new option (-Cx) to control the maximal amount of memory it allocates.
I know there's a new type library editor and format (it stores the IDL, ie text, and generates the TLB dynamically?) How well does this handle upgrading existing COM projects with a TLB?
Should work without a hitch.
I have lots of questions about this, such as "is wstring capable of holding everything a UnicodeString can, and should we just do a search/replace"
Yes, on Windows platforms wchar_t usually is 16 bit large, which means it suffices for holding UTF-16 which UnicodeString is.
or "should we avoid all C++ string types altogether and use UnicodeString"
Depends on how portable your code needs to be. In any case, whenever you just need a string type, use "String", not "UnicodeString".
"can we change all event handlers to use String though the existing .HPPs were compiler-translated to AnsiString"
First, you should NEVER re-use .hpp files generated by older versions of DCC!
For event handlers that use the String type in Delphi, you must use UnicodeString. As above, simply use "String", and your code will work for both the ANSI and Unicode versions of C++Builder.
right down to basics such as "should we prefix all strings with L, or is the compiler smart enough with Unicode enabled to use Unicode strings"
The compiler doesn't convert your strings (it would conflict with the language standards), but both AnsiString and UnicodeString do have copy constructor overloads for both char* and wchar_t* string literals. I.e., the following will work:
AnsiString as = L"foo";
UnicodeString us = "bar";
What will not work this way, though, is the whole bunch of printf()/scanf() functions; AnsiString::sprintf() takes const char*, UnicodeString::sprintf() takes const wchar_t*.
If you are using sprintf() a lot, you may find my CbdeFormat library useful; just read my article on the subject.
You do not say what the data strings in your binary tuple format are for: is it necessary for them to store Unicode? When I transitioned from D2007 to D2009 I was able to keep some parts of the system ANSI-string only.
If storing Unicode is required, then you need to check if your existing data is compatible with a format such as UTF-8. If the range of values stored in existing data files present a problem, then I would make your next upgrade do a one-time conversion of any old data files, reading in the old AnsiString data and writing it back as UTF-8 to a different file name or extension, or by modifying appropriate file header data. I have been versioning data files for a long time, just to allow this sort of processing change.
I am only just starting a BCB2010 project, so cannot comment on your other questions, but I certainly had difficulty upgrading a Delphi project from D2007 to D2009 - though I was able to fix this by editing the project file, which is just XML.
Good luck with the conversion ;-)
Unicode. This one looks really
complicated. Our app contains a
horrible mix of std::string-s and
AnsiString-s with casts to and from
them. I have lots of questions about
this, such as "is wstring capable of
holding everything a UnicodeString
can, and should we just do a
search/replace"
std::wstring contains wchar_t* strings, just like System::UnicodeString does.
should we avoid all C++ string
types altogether and use
UnicodeString
That is up to you to decide. char* strings are still supported. You are not forced to migrate everything to Unicode.
can we change all event handlers to
use String though the existing .HPPs
were compiler-translated to AnsiString
No, you cannot change auto-managed event handlers to use the System::String alias. All IDE versions will complain about that. You will have to manually update your event handler declarations and implementations to use UnicodeString parameters instead of AnsiString parameters when appropriate. That also means you cannot share DFMs and Unit .h files across multiple IDE versions, either (which you should not be doing anyway).
should we prefix all strings with L,
or is the compiler smart enough with
Unicode enabled to use Unicode strings
No. If you declare a string constant or character constant without an L prefix, the data will still be interpretted as Ansi. That has not changed. You can, however, pass Ansi data to System::UnicodeString (but not to std::wstring), and it will convert to Unicode automatically. But you have to be careful because it will use the OS's default Ansi codepage to interpret the data. As long as your Ansi data is only using ASCII characters only, then you will be OK. Otherwise, if you are using non-ASCII characters, then you are better off putting the data into a System::AnsiStringT or System::RawByteString (both were introduced in CB2009) that has been assigned the correct codepage, and then assign that to your System::UnicodeString variable. The associated codepage will be used instead of the OS default codepage for the conversion.
We also need backwards compatibility.
Our app uses its own binary tuple
format that currently stores strings
as an array of bytes. I need to
upgrade this to read old files and,
presumably, write new Unicode strings
as well. How do I handle Unicode
strings embedded in a binary format?
If your tuple is expecting 8-bit characters, then you will have to make sure that any struct declarations and such are using char and not wchar_t characters. If you need to store Unicode strings, but need to maintain the 8-bit compatibility, then you should encode your Unicode strings to UTF-8 first (you can use the System::UTF8String string type to help you - starting in CB2009, it is a true UTF-8 string now). As long as you do not use non-ASCII characters, then your old apps will not know the difference, as ASCII characters are encoded as-is in UTF-8. If you want to store raw Unicode data, however, then your tuple would need a flag somewhere (if it does not already have one) indicating whether the string data is stored as Ansi or Unicode, and your apps would have to look for that flag.
Is there any generic way where I can
point a UnicodeString at an array of
bytes, that may be originally written
as either ANSI bytes or Unicode, and
it will figure out what they are?
No. You have to know the actual encoding of the bytes beforehand. If you pass a memory address to System::AnsiString or std::string, it is going to assume Ansi characters. If you pass the same memory address to System::UnicodeString or std::wstring, it is going to assume Unicode characters instead.
Third-party components. We use SpTBX
mainly, and it appears to be
compatible.
Just like with all prior versions (except for the migration from 2006 to 2007), any third-party components you have will need to be re-compiled for 2010, either manually (if you have the source code for them) or by their respective vendors.
Project upgrades. The standard advice
in the Codegear forums seems to be to
manually recreate all project files
when upgrading.
Yes. That still applies.
I know there's a new type library
editor and format (it stores the IDL,
ie text, and generates the TLB
dynamically?)
.TLB files are not used at all anymore. The new system operates on .ridl (Reduced IDL) files now. During compiling, the .ridl produces the correct TypeLibrary information in the executable's binary resources directly. No .tlb files are generated.
How well does this handle upgrading
existing COM projects with a TLB? We
have Delphi code and TLB that are
built into the C++ application.
I do not remember whether CB2010 (or CB2009, for that matter) can consume pre-existing .tlb files directly. I don't think they can. You can, however, run the .tlb file through tlibimp.exe and it will export a .ridl file. Or you can copy the IDL text from the TLB editor in a past version and paste it into a new .ridl file manually. Either way, you can then add that .ridl ile to your CB2010 project.
2007 and 2010 co-existing. I'm not
sure I trust this answer since I have
had issues with 2006 and 2007 on the
same machine before.
That is why I use virtual machines when installing multiple IDE versions on the same physical machine.
Is the cost of upgrading in line with the benefits?
Why not start a gradual upgrade where new components would be developed on the new platform. Integrate the new components to the old version via different interop helpers.
This approach was suggested to vb6 developers who were thinking about upgrading to vb.net.