How can I dynamically allocate a memory space and get a pointer to a list of structs in Zig.
Like in C :
struct Foo* my_array_of_foo = (struct Foo*) malloc(10*sizeof(Foo));
const allocator: *std.mem.Allocator = std.heap.page_allocator; // this is not the best choice of allocator, see below.
const my_slice_of_foo: []Foo = try allocator.alloc(Foo, 10);
defer allocator.free(my_slice_of_foo);
This will allocate a slice with len 10. It can later be freed with allocator.free(my_slice_of_foo)
In zig, arrays are usually represented as slices which hold a pointer and the number of items (struct {ptr: [*]type, len: usize}). Allocators have a function .create(type) to allocate space for a single value and return a pointer, and a function .alloc(type, count) to allocate a contiguous array and return a slice.
std.heap.page_allocator is not the best choice of an allocator for tasks like this. I would recommend using the general purpose allocator that will catch memory leaks for you, make it easier to find use-after-free errors, and use memory more efficiently:
pub fn main() !void {
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer std.debug.assert(!gpa.deinit());
const allocator = &gpa.allocator;
}
In tests, it is good practice to use the testing allocator which offers the same safety features as the general purpose allocator but is handled for you by the test harness:
test "allocate stuff" {
const allocator = std.testing.allocator;
}
It is also often useful to create arena allocators. .free() does nothing on an arena allocator, instead everything put into an arena allocator gets freed all at once when the arena allocator is destroyed. This can make memory management easier and faster for sections of your application this works for.
const allocator = ... pick an allocator;
var arena_allocator = std.heap.ArenaAllocator.init(allocator);
defer arena_allocator.deinit();
const arena = &arena_allocator.allocator;
Related
Let's say I have a function defined in Rust, which looks like this:
#[no_mangle]
pub unsafe extern "C" fn do_something(
my_value: *mut MyStruct,
some_param: c_uint,
content: *mut *mut u8,
length: *mut c_uint,
capacity: *mut c_uint,
) -> *mut MyStruct {
// Do something and obtain an `ffi_result`...
let heap_data = ffi_result.as_mut_ptr();
// These values are passed as "out" parameters.
*length = ffi_result.len() as c_uint;
*capacity = ffi_result.capacity() as c_uint;
*content = heap_data;
// We intentionally "leak" this data to the heap.
// The caller is responsible for cleaning it up by calling another function.
std::mem::forget(ffi_result);
std::boxed::Box::into_raw(value_of_type_my_struct)
}
It takes in a pointer to a struct, a simple integer parameter, several out parameters that can later be used to create an Array and it returns a pointer to a struct.
Now I compile the rust library into a static library for the target aarch64-apple-ios. I set up a XCode project, add the static library as a dependency as explained here with an "Objective-C Bridging Header" where I import the following header file
#ifndef libmy_project_h
#define libmy_project_h
#include <stdint.h>
struct myStruct;
struct myStruct *do_something(struct myStruct *state, int someParam, char **content, int *length, int *capacity);
#endif
Up until this point everything seems to work fine and I have already successfully used this procedure for a whole bunch of other functions. However in this special case I can not figure out how to call this function from swift. I need to call the function from swift and pass content, length and capacity as out parameters so that I can later use the pointers to create an Array in Swift like so.
This is what I tried so far:
var content = UnsafeMutablePointer<UnsafeMutablePointer<CChar>?>(UnsafeMutablePointer(bitPattern: 0))
var length = UnsafeMutablePointer<Int32>(bitPattern: 0)
var capacity = UnsafeMutablePointer<Int32>(bitPattern: 0)
let my_struct = do_something(my_struct, Int32(some_param), content, length, capacity)
let buffer = UnsafeRawBufferPointer(start: content?.pointee, count: Int(length!.pointee))
var data = Array(repeating: UInt8(0), count: Int(length!.pointee))
data.withUnsafeMutableBytes { arrayPtr in
arrayPtr.copyBytes(from: buffer)
}
However now when I execute this swift snippet, I get an EXC_BAD_ACCESS error, which I think occurs because the pointers I manually created do not belong to the adress space of my application. How can I create pointers that I can use as out parameters?
P.S. For reference here is the same interop code in C#:
[DllImport("my_dll")]
private static extern IntPtr do_something(IntPtr state, uint someParam, out IntPtr content, out uint length, out uint capacity);
Which can be called like so:
IntPtr contentPointer;
uint length, capacity;
IntPtr my_struct = do_something(state, myParam, out contentPointer, out length, out capacity);
byte[] rawContent = new byte[length];
Marshal.Copy(contentPointer, rawContent, 0, (int)length);
// Free the data owned by rust with another FFI call:
free_do_something_result(contentPointer, length, capacity);
var length = UnsafeMutablePointer<Int32>(bitPattern: 0)
You need to pass storage for your out-parameters. This is defining a null pointer. When Rust tries to write the result to address 0, it crashes, since you don't have access to write there.
Instead of creating two layers of pointers, create a value of the type you want, and then pass the address (&) of that value; this will add the extra layer of pointer automatically.
// Create storage
var content: UnsafeMutablePointer<CChar>? // Could be NULL, so Optional
var length: Int32 = 0
var capacity: Int32 = 0
// Pass as references
do_something(&content, &length, &capacity)
// Copy the data
let data = Array(UnsafeRawBufferPointer(start: content, count: Int(length)))
content is still a pointer here because the thing being updated is a pointer. You're not providing storage for content, Rust is. But you do need to provide storage for the pointer (and that's what this does).
I can't compile your code because it's missing a lot (an MCVE would be much better here), so I can't test that this is doing exactly what you mean, but it should be close.
In your example, you're leaking the memory, but since your C# calls free_do_something_result (which I assume cleans it up), I assume you're actually doing the same in the Swift.
I'm trying to optimize as much as possible an operation done on slices of u32 from arrays of u8. As such, I'm testing different options (for loops, iterators, using ByteOrder crate, etc.)
As part of these tests, I also wanted to check out if I could improve it using from_raw_parts standard function.
Here is my code:
use byteorder::{ByteOrder, BigEndian, LittleEndian};
use std::io::Read;
fn main(){
let random_bytes = (0..4).map(|_| { rand::random::<u8>() }).collect::<Vec<u8>>();
let random_bytes = random_bytes.as_slice();
let view = &random_bytes as *const _ as *const u32;
let slice: &[u32] = unsafe { std::slice::from_raw_parts(view, 1) };
println!("{:x?}", slice);
println!("{:x?}", LittleEndian::read_u32(&random_bytes[0..4]));
println!("{:x?}", BigEndian::read_u32(&random_bytes[0..4]));
println!("{:x?}", &random_bytes[0..4]);
}
I would've expected at least one one of the two Little or Big endian to be equal to the first print, but instead this does not seem to be the case, e.g. an example output
[d951db30]
143600ff
ff003614
[ff, 0, 36, 14]
What am I doing wrong?
The problem is here:
let view = &random_bytes as *const _ as *const u32;
A slice is a 2-machine-word struct that contains the pointer to data and the element count as members.
By doing &random_bytes, you are taking a reference to this slice structure (which itself contains a pointer and length), not acquiring the underlying pointer.
Slices have an as_ptr method that returns the pointer to data itself. When you use it, your code functions correctly:
use byteorder::{ByteOrder, BigEndian, LittleEndian};
use std::io::Read;
fn main(){
let random_bytes = (0..4).map(|_| { rand::random::<u8>() }).collect::<Vec<u8>>();
let random_bytes = random_bytes.as_slice();
let view = random_bytes.as_ptr() as *const u32;
let slice: &[u32] = unsafe { std::slice::from_raw_parts(view, 1) };
println!("{:x?}", slice);
println!("{:x?}", LittleEndian::read_u32(&random_bytes[0..4]));
println!("{:x?}", BigEndian::read_u32(&random_bytes[0..4]));
println!("{:x?}", &random_bytes[0..4]);
}
Output in playground:
[684a2f5b]
684a2f5b
5b2f4a68
[5b, 2f, 4a, 68]
recently I was trying to see the address of my variable and I have this question.
var age: Int = 5
withUnsafePointer(to: age) {
print($0) // 0x00007ffee3362750
print($0.pointee) // 5
}
withUnsafePointer(to: &age) {
print($0) // 0x000000010d226330
print($0.pointee) // 5
}
why it shows different memory address and why it shows the same value for the pointee?
var strarr = [1, 2]
withUnsafePointer(to: strarr[0]) {
print("\($0)") // 0x00007ffee3362750
print("\($0.pointee)") // 1
}
withUnsafePointer(to: strarr[1]) {
print("\($0)") // 0x00007ffee3362750
print("\($0.pointee)") // 2
}
withUnsafePointer(to: &strarr[0]) {
print("\($0)") // 0x0000600002755760
print("\($0.pointee)") // 1
}
withUnsafePointer(to: &strarr[1]) {
print("\($0)") // 0x0000600002755768
print("\($0.pointee)") // 2
}
for the array, why it shows the same memory address for index 1 and 2, when I'm not passing the address of the variable and why it shows a different memory address for index 1 and 2 when I'm passing the memory address?
Appreciate your answer and looking forward to understand this
The differences you see come from the fact that you are using two different overloads of withUnsafePointer, I'll list them in the same order you used them:
func withUnsafePointer<T, Result>(to value: T, _ body: (UnsafePointer<T>) throws -> Result) rethrows -> Result
, and
func withUnsafePointer<T, Result>(to value: inout T, _ body: (UnsafePointer<T>) throws -> Result) rethrows -> Result
The difference between the two is the inout qualifier used for the value parameter.
Now, let's try to understand what happens behind the scenes.
First, 0x00007ffee3362750 looks like a stack pointer, while 0x000000010d226330 looks like a heap pointer. Stack addresses start at the top of the allocated memory for the program, and decrease with every function call (and increase when the function returns).
This indicates that the first overload of withUnsafePointer create a temporary writable variable from the one passed as the argument. This is needed as UnsafePointer needs an inout reference to work with, and a regular parameter is readonly.
This means that the implementation of the non-inout overload of withUnsafePointer looks something like this:
func withUnsafePointer<T, Result>(to value: T, _ body: (UnsafePointer<T>) throws -> Result) rethrows -> Result {
var value = value // shadow the argument, create a readwrite location
return try withUnsafePointer(&value, body)
}
Thus, the first call needs to allocate an intermediate memory location, and this is why you see two different addresses, it's because the addresses are not pointing to the same memory location. However, since both memory locations start with the same value, printing the pointee results in identical output.
Now, let's talk about the array example. The behaviour you see has the same cause: stack allocations. What happens is:
program starts executing, stack pointer has value P (random name)
non-inout withUnsafePointer is called, this is a function call, and stack is reserved for the function; stack pointer is P - N
withUnsafePointer creates the temporary writable variable, and executes
withUnsafePointer returns, and frees the stack memory, at this stack pointer gets back to P
second non-inout withUnsafePointer is called, the stack pointer is back to P - N, however since there were no other function calls between the two, the same stack address is reserved for the temporary writable variable, thus the UnsafePointer instance has the same address
Here, even if the UnsafePointer points to the same address, the values at that address are different, corresponding to values of arr[0] and arr[1].
As for the inout calls, the UnsafePointer point to the actual addresses of the items in the array buffer.
This is how you can get different values for the non-inout calls too:
withUnsafePointer(to: strarr[0]) {
print("\($0)")
print("\($0.pointee)")
}
// this adds a nested function call, which also decreases the stack pointer
// resulting in the temporary location not being on the same stack address
func test() {
withUnsafePointer(to: strarr[1]) {
print("\($0)")
print("\($0.pointee)")
}
}
test()
My understanding is that Vala and Genie have reference counting rather than garbage collection.
Per Valadoc.org, this:
string path = Path.build_filename ("my", "full", "path/to.txt");
yields this:
a newly-allocated string that must be freed with g_free
Is this correct or is g_free not required due to reference counting?
If string is wrapped within an object will g_free on string be actioned on object destruction?
Valadoc contains the documentation from the original GLib, GObject, Gtk+, etc. libraries. It also contains additional Vala specific documentation on top of that.
You often see sentences which only makes sense in the context of the C programming language.
The Vala compiler does the memory management for you (most of the time). In this example a Vala string is actually translated by the Vala compiler to a gchar * in C where it has to be deallocated using g_free which the Vala compiler also does for you.
As a matter of fact strings are a bit different than objects as they are not reference counted, but copied instead.
Take this example in Vala:
int main () {
Object obj1 = new Object ();
Object obj2 = obj1;
string s1 = "";
string s2 = s1;
return 0;
}
The (manually cleaned up) code that valac produces in C looks like this:
int main (void) {
GObject* obj1 = g_object_new (G_TYPE_OBJECT, NULL);
GObject* obj2 = g_object_ref (obj1);
gchar* s1 = g_strdup ("");
gchar* s2 = g_strdup (s1);
g_free (s2);
g_free (s1);
g_object_unref (obj2);
g_object_unref (obj1);
return 0;
}
As you can see obj2 is a reference to the same object as in obj1 and the object is only destroyed when both references are unrefed.
The string s2 on the other side is a copy of the string stored in s1 (which is in turn a copy of the string literal "").
As you can also see the compiler does take care of such details for you. It makes the manual reference counting and manual string copying automatic.
var cpuInfo: processor_info_array_t = nil
var numCpuInfo: mach_msg_type_number_t = 0
var coresTotalUsage: Float = 0.0
var numCPUsU: natural_t = 0
let err = host_processor_info(mach_host_self(), PROCESSOR_CPU_LOAD_INFO, &numCPUsU, &cpuInfo, &numCpuInfo);
assert(err == KERN_SUCCESS, "Failed call to host_processor_info")
Hi, I am calling the above C API host_processor_info to get process load informations from swift, no problem there.
cpuInfo is a inout parameter (pointer) that, on return, will point to a structure containing the CPU information allocated by that API.
The caller is reponsible for deallocating the memory; I can do that easily from objective C but haven't had any luck in swift. I know I could wrap that call into an objective C extension but I'm trying to learn swift and would like, if possible, avoid the obj-c solution.
in obj-c I would deallocate with:
size_t cpuInfoSize = sizeof(integer_t) * numCpuInfo;
vm_deallocate(mach_task_self(), (vm_address_t) cpuInfo, cpuInfoSize)
cpuInfo in swift is an UnsafeMutablePointer not convertible into a vm_address_t.
Any help appreciated, thanks.
processor_info_array_t is a pointer type, and vm_address_t is an integer type
(ultimately an alias for UInt). (Judging from the comments in <i386/vm_types.h>
this might to be for historical reasons.)
The only way to convert a pointer to an integer (of the same size) in Swift is unsafeBitCast.
mach_init.h defines
extern mach_port_t mach_task_self_;
#define mach_task_self() mach_task_self_
Only the extern variable is visible in Swift, not the macro.
This gives:
let cpuInfoSize = vm_size_t(sizeof(integer_t)) * vm_size_t(numCpuInfo)
vm_deallocate(mach_task_self_, unsafeBitCast(cpuInfo, vm_address_t.self), cpuInfoSize)
In Swift 4, the equivalent code appears to be:
let cpuInfoSize = vm_size_t(MemoryLayout<integer_t>.stride * Int(numCpuInfo))
vm_deallocate(mach_task_self_, vm_address_t(bitPattern: cpuInfo), cpuInfoSize)
In particular, the initializer UInt(bitPattern:) is now apparently preferred to unsafeBitCast() to initialize an unsigned integer with the bit pattern of a pointer (I guess this usage is not longer considered "unsafe"). It correctly handles a nil pointer, returning 0 in this case.