I have some 32-bit DLLs that don't have matched 64-bit DLLs. How can I invoke these DLLs from a 64-bit application written in Delphi XE2?
No, you cannot directly do this. A 64 bit process can only execute 64 bit code, and a 32 bit process can only execute 32 bit code.
The trick is to use multiple processes.... (Note this can be done for non visual code, and even for GUI elements, though there can be some small but problematic behaviors for visual elements.)
The most common solution is to wrap the 32 bit dll in an out of process COM server, which you can call across the 64/32 bit barrier. (This goes both ways, you can create a 64 bit out of process COM server and call it from a 32 bit application also.)
Yes, there are other ways to conceive of this, but the most common is to use COM:
Create a new 32 bit out of process COM server that hosts your 32 bit
DLL and exposes the needed functionality from the 32 bit dll.
Call this COM server from your 64 bit code
I should add that it is also possible to create the new 32 bit COM server as an in-process COM server, and then configure COM+ to run it. COM+ will run it out of process, and magically run your 32 bit in process COM server out of process, where you can call it from 32 and 64 bit code transparently, as if it was in process. (Note, if the COM server is a GUI control, going out of process may or may not work. The team I work with has done it successfully, but there are complexities -- some of which cannot be surmounted -- related to hooking parent windows and controls that cannot be done across the process boundary.)
You can use the same exact technique used to call 64 bit dlls from 32 bit code.
See http://cc.embarcadero.com/Item/27667
"Just" make the contrary: run a background 32 bit process, the communicate from your 64 bit process with it using a memory mapped buffer.
But this is definitively not an easy task. You'll have to rewrite some asm code. I wrote some article about how it works.
The out-of-process COM option is perhaps the easiest to implement. Or use a more simple IPC - like WM_COPYDATA message or any other mean. But you'll definitively need another 32 bit process to link to the 32 bit libraries.
I had the same issue some time back and found this link:32-bit DLLs in 64-bit environment
The 32-bit DLL was written in Delphi, ages ago, and we now had a need to call it from a 64-bit platform- but we don't have a 64-bit Delphi.
I've made it work- though it seems a bit of a kludge, it was better than getting the DLL rewritten in 64-bit (we'd have had to purchase a 64-bit version of Delphi, or start from scratch in something else).
NB while this needs some hacking, no programming is required- it uses components that came with Windows. Works in (at least) Windows 7, Windows 2008.
Related
Is there a way to use a 32bit NIF from a 64 bit Erlang (under Windows)?
Seems impossible, but maybe there is a way to achieve this?
My only suggestion (too silly) - you can write 32-bit out-of-proc port driver as wrapper of your NIF and run it in separate process. Of course, erlang node in this case should interact not with NIF but with external port program. But you can consider this as joke ;-)
I have a Delphi XE4 application, which can act as an out-of-process COM component.
The application is sometimes started as 32 and sometimes as 64 bit application on the same computer.
The currently running instance should act as out-of-process server for a 32 bit COM client. The application registers[1] itself as "per user" COM component during start. Later the 32 bit COM client can connect to the running instance.
This works fine for the 32 bit version of the application.
This also works fine for the 64 bit version as long as only the 64 bit version is registered[2].
This doesn't work anymore for the 64 bit version if the 32 bit version is registered too[3]. In this case the COM client always try to start a 32 bit application instead of using the running 64 bit instance.
Is there a way for a 64 bit Delphi application to register itself for both 64 and 32 bit clients?
(Editing the registry directly doesn't seem a good idea, but I imagine that at the end the 64 bit application should have removed the 32 bit version from the Wow6432Node branch or should have replaced it there).
Notes:
[1] = Code used by the application to registers itself during start:
ComServer.PerUserRegistration := True;
ComServer.UpdateRegistry(True);
ComServer.Initialize;
[2] Only 64 bit registered = path to the 64 bit exe can be found under HKCR/CLSID/{...}/LocalServer32 and there is nothing or a non existing path under HKCR/Wow6432Node/CLSID/{...}/LocalServer32
[3] 32 bit registered too = path to the 32 bit exe can be found under HKCR/Wow6432Node/CLSID/{...}/LocalServer32
First of all, I should say that I remain unconvinced of the wisdom of having your COM server register itself rather than having an installer do so. However, you have clearly stated that is your goal so let us proceed on that basis.
It looks to me as though you want the running application to write to both 32 and 64 bit views of the registry. I personally would write there directly. Use KEY_WOW64_32KEY and KEY_WOW64_64KEY flags to access the 32 and 64 bit views of the registry.
In the comments and in chat, you indicate that you regard direct registry writes as hacking. But that is the officially documented way to register COM servers. And that is of course all that UpdateRegistry does. Since you have special requirements that are not handled by UpdateRegistry I cannot see a better way to achieve your goals.
This MSDN article about the CLSCTX enumeration describes two values (CLSCTX_ACTIVATE_32_BIT_SERVER and CLSCTX_ACTIVATE_64_BIT_SERVER) with which a COM client can say its preference about the architecture of the COM server.
Instead of registering the running instance of my application both as 32 and 64 bit COM server, I could let the client express a preference about which version it wants to have so that Windows uses the running one. It is possible only under following conditions:
The COM client can be modified,
The COM client can detect which version (32 or 64 bit) of the out-of-process COM application is currently running.
By the way the MSDN article explains quite well which version of Windows will prefer which version of the out-of-process COM component (Starting with Windows Server 2003 SP1, a 32 bit client prefers a 32 bit COM server).
I would like to know what's the difference betweenmach_vm_allocate and vm_allocate. I know mach_vm_allocate is only available in OS X and not iOS, but I'm not sure why. The file that has all the function prototypes in for the mach_vm_... functions (mach/mach_vm.h) only has #error mach_vm.h unsupported. in iOS.
the new Mach VM API that is introduced in Mac OS X 10.4. The new API is essentially the same as the old API from the programmer's standpoint, with the following key differences.
-Routine names have the mach_ prefixfor example, vm_allocate() becomes mach_vm_allocate() .
-Data types used in routines have been updated to support both 64-bit and 32-bit tasks. Consequently, the new API can be used with any task.
The new and old APIs are exported by different MIG subsystems:
mach_vm and vm_map , respectively. The corresponding header files are
<mach/mach_vm.h> and <mach/vm_map.h> , respectively.
The new Mach VM API that is introduced in Mac OS X 10.4. The new API is essentially the same as the old API from the programmer's standpoint, with the following key differences:
Routine names have the mach_ prefixfor example, vm_allocate() becomes mach_vm_allocate();
Data types used in routines have been updated to support both 64-bit and 32-bit tasks. Consequently, the new API can be used with any task.
The new and old APIs are exported by different MIG subsystems: mach_vm and vm_map respectively. The corresponding header files are <mach/mach_vm.h> and <mach/vm_map.h> respectively.
The information is derived from the book OS X Internals A System Approach.
Those would be /usr/include/mach/mach_vm.h and /usr/include/mach/vm_map.h. You can grep in /usr/include/mach to get those.
In the kernel, the APIs are basically using the same implementation. On iOS, you can use the "old" APIs perfectly well.
I'm not happy with the other answers. Two seem to quote from the same source (one doesn't correctly attribute it) and this source rather misleading.
The data types in the old API are variable size. If you are compiling for i386 CPU architecture, they are 32 bit, if you compile for x86_64 CPU architecture, they are 64 bit.
The data type in the new API are fixed size and always 64 bit.
As a 32 bit process only requires 32 bit data types and a 64 bit process already had 64 bit types even with the old API, it isn't obvious why the new API was required at all. But it will become obvious if you look at Apples kernel design:
The real kernel of macOS is a Mach 3 micro kernel. Being a micro kernel, it only takes care of process and thread management, IPC, virtual memory management and process/thread scheduling. That's it. Usually everything else would be done in userspace when using a micros kernel but that is slow and thus Apple took a different approach.
Apple took the FreeBSD monolithic kernel and wrapped it around the micro kernel which executes the FreeBSD kernel as a single process (a task) with special privileges (despite being a task, the code runs within kernel space not userspace as all the other tasks in the system). The combined kernel has the name XNU.
And in the past, Apple supported any combination of mixing 32 and 64 bits within a system: Your system could run a 32 bit kernel and run 32 or 64 bit processes on it or it could run a 64 bit kernel and run 32 or 64 bit processes on it. You might already be seeing where this going, aren't you?
If process and kernel are of different bit width, the old API is of no use. E.g., a 32 bit kernel cannot use it to interact with the memory of a 64 bit process because all data types of the API calls would only be 32 bit in the 32 bit kernel task but a 64 bit process has a 64 bit memory space even if the kernel itself has not.
Actually there is even a 3rd version of that API. To quote a comment from Apple's kernel source:
* There are three implementations of the "XXX_allocate" functionality in
* the kernel: mach_vm_allocate (for any task on the platform), vm_allocate
* (for a task with the same address space size, especially the current task),
* and vm32_vm_allocate (for the specific case of a 32-bit task). vm_allocate
* in the kernel should only be used on the kernel_task. vm32_vm_allocate only
* makes sense on platforms where a user task can either be 32 or 64, or the kernel
* task can be 32 or 64. mach_vm_allocate makes sense everywhere, and is preferred
* for new code.
As long as you are only using that API in userspace and only for memory management of your old process, using the old API is still fine, even for a 64 bit process as all data types will be 64 bit in that case.
The new API is only required when working across process boundaries and only if it's not certain that both processes are both 32 or 64 bit. However, Apple has dropped 32 bit kernel support long ago and meanwhile they have also dropped 32 bit userspace support as all system libraries only ship as 64 bit library starting with 10.15 (Catalina).
On iOS the new API was never required as you could never write kernel code for iOS and the security concept of iOS forbids direct interaction with other process spaces. On iOS you can only use that API to interact with your own process space and for that it will always have correct data types.
What are the side effects of using IMAGE_FILE_LARGE_ADDRESS_AWARE flag (to use more than 2GB of RAM) in my program?
I am using Delphi 7 with FastMM4.
You need to make sure you don't use the built-in memory manager and use something that can support addresses >2GB. e.g. FastMM.
You may have other code, typically 3rd party code, in your codebase that will fall foul of addresses >2GB. I personally deal with this by running under 64 bit Windows and forcing the system to use top-down memory allocation via a registry setting.
When you do this you may run into some bugs in Windows. For example GetCursorPos on Vista fails when the address of its parameter is >2GB. I work around that by patching Windows.GetCursorPos with a version that goes through GetCursorInfo. This bug is fixed in Windows 7 but MS elected not to back-port it to Vista.
I can't stress enough how important it is to run with top-down memory allocation enabled.
I need some help to describe, in technical words, why a 64-bit application prompts a "Not a valid Win32 application" in Windows 32-bit on a 32-bit machine? Any MSDN reference is greatly appreciate it (I couldn't google a reliable source). I know it shouldn't run, but I have no good explanation for this.
A 32 bit OS runs in 32-bit protected mode. A 64 bit OS runs in long mode (i.e. 64-bit protected mode). 64 bit instructions (used by 64 bit programs) are only available when the CPU is in long mode; so you can't execute them in 32-bit protected mode, in which a 32 bit OS runs.
(The above statement applies to x86 architecture)
By the way, the reason for "Not a valid Win32 application" error message is that 64 bit executables are stored in PE32+ format while 32 bit executables are stored in PE32 format. PE32+ files are not valid executables for 32 bit Windows. It cannot understand that format.
To expand on what others have said in a more low-level and detailed way:
When a program is compiled, the instructions are written for a specific processor instruction set. What we see as "x = y + z" usually amounts to something along the lines of copying one value into a register, passing the add command with the memory location of the other value, etc.
Specific to this one question, a 64 bit application is expecting 64 bits of address space to work with. When you pass a command to the processor in a 32 bit system, it works on those 32 bits of data at once.
The point of it all? You can't address more than 4 gigabytes (232) of memory on a 32 bit system without creativity. Some tasks that would take multiple operations (say, dealing with simple math on numbers > 4 billion unsigned) can be done in a single operation. Bigger, faster, but requires breaking compatibility with older systems.
A 64-bit application requires a 64-bit CPU because it makes use of 64-bit instructions.
And a 32-bit OS will put your CPU into 32-bit mode, which disables said instructions.
Why would you expect it to work? You're compiling your program to take advantage of features that don't exist on a 32-bit system.
A 64-bit architecture is built to invoke hardware CPU instructions that aren't supported by a 32-bit CPU, which your CPU is emulating in order to run a 32-bit OS.
As it would be theorically possible to run in some kind of emulation mode for the instruction set, you get into troubles as soon as your application needs more memory than 32 bits can address.
The primary reason why 64-bit apps don't run on 32-bit OSes has to do with the registers being used in the underlying assembly (presuming IA-32 here) language. In 32-bit operating systems and programs, the CPU registers are capable of handle a maximum of 32 bits of data (DWORDS) in the registers. In 64-bit OSes, the registers must handle double the bits (64) and hence the QWORD data size in assembly. Compiling for x86 (32 bit) means the compiler is limiting the registers to handling 32 bits of data max. Compiling for x64 means the compiler is allowing the registers to handle up to 64 bits of data. Stepping down from QWORDs (64) to DWORDs (32) means you end up losing half the information stored in the registers. You're most likely getting the error because the underlying compiled machine code is trying to move more than 32 bits of data into a register which the 32-bit operating system can't handle since the maximum size of its registers are 32 bits.
Because they're fundamentally different. You wouldnt expect a Frenchman to understand Mandarin Chinese, so why would you expect a 32bit CPU to understand 64bit code?