F# P/Invoke Marshaling Recursive Structures - f#

Question

None of the examples I have seen thus far appear to address the problem of marshaling a structure containing a union of structures that contain recursive references. I am attempting to write a marshaler for a structure which contains these and have so far failed.
For example:
typedef enum {
My_StructA = 0x7878,
My_StructB
} MyStructTag;
typedef struct _MyStruct MyStruct;
struct _MyStruct {
MyStructTag discriminator;
union {
struct {
int a;
int b;
} StructA;
struct {
int c;
MyStruct* d;
} StructB;
} MyUnion;
};
I attempted to define the structures as follows:
type MyStructTag =
| My_StructA = 0x7878
| My_StructB = 0x7879
[<Struct; StructLayout(LayoutKind.Sequential)>]
type StructA =
val mutable a : int
val mutable b : int
[<Struct; StructLayout(LayoutKind.Sequential)>]
type StructB =
val mutable c : int
val mutable d : MyStruct
[<Struct; StructLayout(LayoutKind.Explicit)>]
type MyStruct =
[<FieldOffset(0)>] val discriminator : MyStructTag
[<FieldOffset(4)>] val structA : StructA
[<FieldOffset(4)>] val structB : StructB
Note that the reason I bothered to define MyStruct explicitly is to allow myself to make use of Marshal.OffsetOf() and Marshal.SizeOf() when writing the custom marshaler for this structure. From what I have seen, writing a custom marshaler is the only way to handle unions. If I am wrong about that, references would be greatly appreciated!
The error I receive when writing the above code is:
error FS0039: The type 'MyStruct' is not defined
I assume this is because only discriminated union types can be defined recursively. However, I am not aware of any other way to represent these structures in F#.
Thank you in advance for your time.

Answer 1

You've got two problems. First of all, any mutually recursive types (whether discriminated unions, classes, or structs) need to be defined using
type A = ...
and B = ...
rather than
type A = ...
type B = ...
(and note that attributes can come before or after the word type, but only after the word and...). However, if you try this, you'll see that you just get a different error, because structs can't be directly recursive as fields of each other. If struct A had a field which was a struct B and struct B had a field which was a struct A (and either of them had any other fields), then the size would be infinite. Note that this is true of your C code as well - StructB contains a pointer to a MyStruct, not a MyStruct itself. In .NET, you can use an IntPtr for this - in F# you can use the nativeint alias or nativeptr<MyStruct>. Try this:
open System.Runtime.InteropServices
type MyStructTag =
| My_StructA = 0x7878
| My_StructB = 0x7879
[<Struct; StructLayout(LayoutKind.Sequential)>]
type StructA =
val mutable a : int
val mutable b : int
[<Struct; StructLayout(LayoutKind.Sequential)>]
type StructB =
val mutable c : int
val mutable d : nativeptr<MyStruct>
and [<Struct; StructLayout(LayoutKind.Explicit)>]MyStruct =
[<FieldOffset(0)>] val discriminator : MyStructTag
[<FieldOffset(4)>] val structA : StructA
[<FieldOffset(4)>] val structB : StructB