This question already has answers here:
How do you actually use dynamically sized types in Rust?
(4 answers)
Closed 2 years ago.
My mental model of data layout in Rust was that all structs' sizes have to be known at compile-time, which means all of their properties have to be known at compile-time recursively. This is why you can't have a struct member that's simply a trait (and why enums have to be used in place of union types): the size can't be known, so instead you have to use either
A generic, so the trait gets "reified" to a size-known struct at usage time
An enum, which has a finite set of possible size-known layouts
A Box, whose size is known because it's just a pointer
But in the docs for Path, it says:
This is an unsized type, meaning that it must always be used behind a pointer like & or Box. For an owned version of this type, see PathBuf.
Yet Path is neither a trait nor a generic struct, it's just a plain struct.
What's wrong with my mental model that this can be possible?
I found this explanation of what dynamically-sized types are, but I still don't understand, how I would make one of my own. Is doing so a special privilege reserved for the language itself?
A struct is allowed to contain a single unsized field, and this makes the struct itself unsized. Constructing a value of an unsized type can be very difficult, and can usually only be done by using unsafe to cast a reference to a sized variant to the unsized variant.
For example, one might use #[repr(transparent)] to make it possible to cast an &[u8] to a &MySlice like this. This attribute makes the type guaranteed to be represented in the same way as its single field.
#[repr(transparent)]
struct MySlice {
inner: [u8],
}
It would then be sound to convert slices like this:
impl MySlice {
pub fn new(slice: &[u8]) -> &MySlice {
// SAFETY: This is ok because MySlice is #[repr(transparent)]
unsafe {
&*(slice as *const [u8] as *const MySlice)
}
}
}
You could e.g. refuse to perform the conversion if the slice was not valid ascii, and now you would have a byte array that was guaranteed to point to valid ascii, just like how &str is an &[u8] that is guaranteed to be valid utf-8.
Related
Is int type in Dart a value type or reference type?
What is "reference type","value type","canonicalized" in Dart? (concrete definition)
I was looking into the specific definitions of "reference type" and "value type", but in the end, I thought it would be good to understand the code that represents the difference and the behavior (result) of that code.
If possible, I would like Dart code, but if it has the same structure as Dart regarding the above issues, there is no problem in other languages, so I would appreciate it if you could give me sample code.
The meaning of "reference type" and "value type" used here is the distinction made by C#.
A reference type is a traditional object-oriented object. It has identity which it preserves over time. You can have two objects with the same state, but they are different objects.
You don't store reference type values directly into variables. Instead you store a reference to them. When you access members of the object, you do so through the reference (x.foo means "evaluate x to a reference, then access the foo member of the object referenced by that reference).
When you check for identity, identical(a, b), you check whether two different references refer to the same object.
A value type does not have identity. It's defined entirely in terms of its state, its contents. When you assign a value type to a variable, you make a copy of it. It's the "value" in call-by-value.
C# allows you to declare value types with that behavior.
(Reference types do not correspond to call-by-reference, rather to call-by-sharing.)
In Dart, all objects are reference objects. This differs from, say, Java, where an int is a value-type ("primitive type" in their parlance), and they have a separate Integer class which can wrap the int values and is a reference type. That's needed because Integer is a subtype of Object, but int isn't, so you can't store int values in a generic container which expects a subtype of Object.
In Dart, some reference types are still more value-like than others. In particular numbers.
A Dart number, like the integer 1, has precisely one object representing that value. If you write 1 in different places of the program, or even calculate 4 ~/ 4 or 3 - 2, you always get the same object representing the value 1. It's like it's not an object at all, and identical only compares values. And that's precisely what happens, because it allows the compiler to treat integers as primitive values internally, and not worry about sometimes having to give you the same 1 object back that you put in.
This is sometimes expressed as integers being "canonicalized": There is only one canonical instance representing each state.
Dart also canonicalizes other objects.
Constant expressions are canonicalized, so that two different constant expressions generating instances of the same type, with identical states, are made to return the same, canonical, instance representing that value.
Since the value is guaranteed to be immutable, there is no need to have equal objects with different identities. (If they were mutable, it matters very much which one you mutate and which one you don't).
In C# I know that an underlying type for each enumeration type can be defined explicitly and individually, for example:
enum CoffeeSize : byte
{
None,
Tall,
Venti,
Grande
};
Now I would like to do the same in Delphi.
Does there exist some option to define an underlying type for each enumeration individually, other than by using the minimum enumeration size {$Z} or {$MINENUMSIZE}?
Does there exist some option to define an underlying type for each enumeration individually, other than by using the minimum enumeration size {$Z} or {$MINENUMSIZE}?
No.
I need to assign return value of a fetch:
try! c.fetch(fr)
to a to-many relationship (itemsWithoutRatingOfLoggedInUser):
Utility.app(c: c).itemsWithoutRatingOfLoggedInUser = try! c.fetch(fr2)
And got following error. What can I do? I tried to cast with as! operator, did not help.
Casting (with as) is for cases where a value of one type can be interpreted as a value of some other, explicitly related type. This interpretation is defined in terms of language features: either type relationships (like inheritance or protocol conformance), or the handful of special cases where Swift can bridge Foundation or CF types to/from their Swift Standard Library equivalents.
When you cast, you're asking the Swift compiler/runtime to squint its eyes a little and pretend that one type is another. (Maybe even do a little twiddling behind the scenes so that pretending works.) Most often, casting happens when you have a value where you know more about its type than Swift (in the context of the function you're writing) does.
When you have types that aren't explicitly related though language features — that is, one isn't a subclass of the other, or one isn't a protocol adopted by the other, or they aren't one of the special sets of equivalent types from Foundation/CF that Swift knows how to bridge — you have to convert, not cast. (Swift also makes you convert instead of casting when types are semantically different enough that you need to think about the consequences of conversion, like when going between signed and unsigned integers.)
Array and NSOrderedSet are semantically different enough types that you have to convert, not cast.
Conversion between types requires that the type being converted to know about the type being converted from, so to perform a conversion you use an initializer of the target type.
let stuff = managedObjectContext.fetch(request)
let orderedSet = NSOrderedSet(array: stuff)
Note that NSOrderedSet is not a generic type in Swift 3, so even if you create it from a typed array ([MyObject]), the elements come out as Any. That's a use case for casting:
let item = orderedSet as! MyObject
I'm trying to achieve a static cast like coercion that doesn't result in copying of any data.
A naive static cast does not work
let pkt = byte_buffer :> PktHeader
FS0193: Type constraint mismatch. The type byte[] is not compatible with type PktHeader The type 'byte[]' is not compatible with the type 'PktHeader' (FS0193) (program)
where the packet is initially held in a byte array because of the way System.Net.Sockets.Socket.Receive() is defined.
The low level packet struct is defined something like this
[<Struct; StructLayout(LayoutKind.Explicit)>]
type PktHeader =
[<FieldOffset(0)>] val mutable field1: uint16
[<FieldOffset(2)>] val mutable field2: uint16
[<FieldOffset(4)>] val mutable field3: uint32
.... many more fields follow ....
Efficiency is important in this real world scenario because wasteful copying of data could rule out F# as an implementation language.
How do you achieve zero copy efficiencies in this scenario?
EDIT on Nov 29
my question was predicated on the implicit belief that a C/C++/C# style unsafe static cast is a useful construct, as if this is self evident. However, on 2nd thought this kind of cast is not idiomatic in F# since it is inherently an imperative language technique fraught with peril. For this reason I've accepted the answer by V.B. where SBE/FlatBuffers data access is promulgated as best practice.
A pure F# approach for conversion
let convertByteArrayToStruct<'a when 'a : struct> (byteArr : byte[]) =
let handle = GCHandle.Alloc(byteArr, GCHandleType.Pinned)
let structure = Marshal.PtrToStructure (handle.AddrOfPinnedObject(), typeof<'a>)
handle.Free()
structure :?> 'a
This is a minimum example but I'd recommend introducing some checks on the length of the byte array because, as it's written there, it will produce undefined results if you give it a byte array which is too short. You could check against Marshall.SizeOf(typeof<'a>).
There is no pure F# solution to do a less safe conversion than this (and this is already an approach prone to runtime failure). Alternative options could include interop with C# to use unsafe and fixed to do the conversion.
Ultimately though, you are asking for a way to subvert the F# type system which is not really what the language is designed for. One of the principle advantages of F# is the power of the type system and it's ability to help you produce statically verifiable code.
F# and very low-level performance optimizations are not best friends, but then... some smart people do magic even with Java, which doesn't have value types and real generic collections for them.
1) I am a big fan of a flyweight pattern lately. If you architecture allows for it, you could wrap a byte array and access struct members via offsets. A C# example here. SBE/FlatBuffers even have tools to generate a wrapper automatically from a definition.
2) If you could stay within unsafe context in C# to do the work, pointer casting is very easy and efficient. However, that requires pinning the byte array and keeping its handle for later release, or staying within fixed keyword. If you have many small ones without a pool, you could have problems with GC.
3) The third option is abusing .NET type system and cast a byte array with IL like this (this could be coded in F#, if you insist :) ):
static T UnsafeCast(object value) {
ldarg.1 //load type object
ret //return type T
}
I tried this option and even have a snippet somewhere if you need, but this approach makes me uncomfortable because I do not understand its consequences to GC. We have two objects backed by the same memory, what would happen when one of them is GCed? I was going to ask a new question on SO about this detail, will post it soon.
The last approach could be good for arrays of structs, but for a single struct it will box it or copy it anyway. Since structs are on the stack and passed by value, you will probably get better results just by casting a pointer to byte[] in unsafe C# or using Marshal.PtrToStructure as in another answer here, and then copy by value. Copying is not the worst thing, especially on the stack, but allocation of new objects and GC is the enemy, so you need byte arrays pooled and this will add much more to the overall performance than you struct casting issue.
But if your struct is very big, option 1 could still be better.
The following example fails with:
FAIL: MyClass tests getClassReturnsConstructorForDouble
Expected: ?:<double>
Actual: ?:<double>
Example:
test("getClassReturnsConstructorForDouble", () {
double object = 10.1;
Type objectClass = reflect(object).type.reflectedType;
expect(objectClass, equals(object.runtimeType));
});
Is it not possible to accurately reflect int/double types?
reflectedType gets you the true implementation type. The runtimeType can lie, and does, by design.
There can be varying implementations for numbers of various sorts of numbers, for strings and other built in types, but the they are exposed as the publicly known ones.
For example, we might have different implementation types for integers of varying size, but they will all say their runtimeType is int. If you reflect on them, you can see the difference.
Another example might be String. There may be specialized classes for strings that are pure ASCII for example, because they can be represented more compactly. This is not exposed at the base level: the runtimeType is String. You cannot detect this unless you dig for it via reflection.
I think reflection is returning the correct types to you, but your test fails because objectClass and object.runtimeType are two seperate objects and therefore not equal to each other. If you convert them to string first they will equal each other:
expect(objectClass.toString(), equals(object.runtimeType.toString()));