When I read The Swift Programming Language: Memory Safety, I was confused by the section Conflicting Access to Properties:
The code below shows that the same error appears for overlapping write
accesses to the properties of a structure that’s stored in a global
variable.
var holly = Player(name: "Holly", health: 10, energy: 10)
balance(&holly.health, &holly.energy) // Error
In practice,
most access to the properties of a structure can overlap safely. For
example, if the variable holly in the example above is changed to a
local variable instead of a global variable, the compiler can prove
that overlapping access to stored properties of the structure is
safe:
func someFunction() {
var oscar = Player(name: "Oscar", health: 10, energy: 10)
balance(&oscar.health, &oscar.energy) // OK
}
In the example above, Oscar’s health and energy are passed as the two in-out parameters to balance(_:_:). The compiler can prove that memory
safety is preserved because the two stored properties don’t interact
in any way.
How the compiler can prove that memory safety?
Being inside a function scope gives the compiler the certainty of which operations will be executed on the struct and when. The compiler knows how structs work, and how and when (relative to the time the function is called) the code inside a function is executed.
In a global or larger scope, the compiler loses visibility over what could be modifying the memory, and when, so it cannot assure safety.
It's because of multiple threads. When "holly" is a global variable, multiple threads could access the global variable at the same time, and you are in trouble. In the case of a local variable, that variable exists once per execution of the function. If multiple threads run someFunction() simultaneously, each thread has its own "oscar" variable, so there is no chance that thread 1's "oscar" variable access thread 2's oscar variable.
An answer from Andrew_Trick on Swift Forums:
"don't interact" is a strong statement. The compiler simply checks that each access to 'oscar' in the call to 'balance' can only modify independent pieces of memory ('heath' vs 'energy'). This is a special case because any access to a struct is generally modeled as a read/write to the entire struct value. The compiler does a little extra work to detect and allow this special case because copying the 'oscar' struct in situations like this would be inconvenient.
Related
I would like to get the lua state from inside lua so I can pass it too an external program that cannot be hooked up using ffi or dll. I just need a pointer to it and the ability to share it(shared memory across program boundaries.
That or I can create a lua state in my program and then pass that so I would simply need to set the lua state to it inside lua(and it would have to work with shared memory).
I've thought about sharing data using json but ideally I would like to directly access objects.
Lua is pretty good about avoiding heap allocation and global pointers to allocated memory. lua_newstate takes an allocator function as a parameter. The provided function will be used to allocate/deallocate all memory associated with the lua_State object. Including the pointer returned by lua_newstate.
So hypothetically, you could provide an allocator function that allocates/deallocates interprocess shared memory. And then, you can just pass the lua_State to some other process and access it.
First, you clearly cannot do this "from inside lua"; that kind of low-level thing just ain't happening. You cannot access the lua_State object from within Lua. You must be in control of the lua_State creation process for that to be a possibility. So we're talking about C (equivalent) code here, not in-Lua code.
Now, you can expose a C function to Lua which returns a light userdata that just so happens to be the exact lua_State* in question. But Lua can't really do much with light userdata other than pass it to other C function APIs.
Second, while the Lua system provides a guarantee that it will only allocate memory through the allocator, the system does not provide a guarantee that what you're trying to do will work. It is entirely possible that the Lua implementation does use process global memory, so long as it does it in such a way that different threads can access that global memory without breaking threading guarantees.
Obviously, you can inspect the Lua implementation to see if it does anything of the kind. But my point is that the guarantees are that each independent lua_State will be thread-isolated from each other and that each lua_State will only allocate memory through the given allocator. There is no guarantee that Lua's implementation doesn't have some global storage that it uses for some purpose.
So simply sharing the memory allocated by the Lua state may not be enough.
Also, even if this works, the two processes cannot access the same lua_State object at the same time, just like two threads in the same process cannot access the lua_State at the same time.
The lua state is not designed to leave the program / thread it is executing in.
Doing a query on a running lua_state could result in a crash, because it is only notionally consistent when a lua call returns, or a C api function is called. During execution, some un-locked modifications could cause uninitialized memory access, or ininite loops due to lists being inconsistent.
In another question, I found out that the Assigned() function is identical to Pointer <> nil. It has always been my understanding that Assigned() was detecting these dangling pointers, but now I've learned it does not. Dangling Pointers are those which may have been created at one point, but have since been free'd and haven't been assigned to nil yet.
If Assigned() can't detect dangling pointers, then what can? I'd like to check my object to make sure it's really a valid created object before I try to work with it. I don't use FreeAndNil as many recommend, because I like to be direct. I just use SomeObject.Free.
Access Violations are my worst enemy - I do all I can to prevent their appearance.
If you have an object variable in scope and it may or may not be a valid reference, FreeAndNil is what you should be using. That or fixing your code so that your object references are more tightly managed so it's never a question.
Access Violations shouldn't be thought of as an enemy. They're bugs: they mean you made a mistake that needs fixed. (Or that there's a bug in some code you're relying on, but I find most often that I'm the one who screwed up, especially when dealing with the RTL, VCL, or Win32 API.)
It is sometimes possible to detect when the address a pointer points to resides in a memory block that is on the heap's list of freed memory blocks. However, this requires comparing the pointer to potentially every block in the heap's free list which could contain thousands of blocks. So, this is potentially a computationally intensive operation and something you would not want to do frequently except perhaps in a severe diagnostic mode.
This technique only works while the memory block that the pointer used to point to continues to sit in the heap free list. As new objects are allocated from the heap, it is likely that the freed memory block will be removed from the heap free list and put back into active play as the home of a new, different object. The original dangling pointer still points to the same address, but the object living at that address has changed. If the newly allocated object is of the same (or compatible) type as the original object now freed, there is practically no way to know that the pointer originated as a reference to the previous object. In fact, in this very special and rare situation, the dangling pointer will actually work perfectly well. The only observable problem might be if someone notices that the data has changed out from under the pointer unexpectedly.
Unless you are allocating and freeing the same object types over and over again in rapid succession, chances are slim that the new object allocated from that freed memory block will be the same type as the original. When the types of the original and the new object are different, you have a chance of figuring out that the content has changed out from under the pointer. However, to do that you need a way to know the type of the original object that the pointer referred to. In many situations in native compiled applications, the type of the pointer variable itself is not retained at runtime. A pointer is a pointer as far as the CPU is concerned - the hardware knows very little of data types. In a severe diagnostic mode it's conceivable that you could build a lookup table to associate every pointer variable with the type allocated and assigned to it, but this is an enormous task.
That's why Assigned() is not an assertion that the pointer is valid. It just tests that the pointer is not nil.
Why did Borland create the Assigned() function to begin with? To further hide pointerisms from novice and occasional programmers. Function calls are easier to read and understand than pointer operations.
The bottom line is that you should not be attempting to detect dangling pointers in code. If you are going to refer to pointers after they have been freed, set the pointer to nil when you free it. But the best approach is not to refer to pointers after they have been freed.
So, how do you avoid referring to pointers after they have been freed? There are a couple of common idioms that get you a long way.
Create objects in a constructor and destroy them in the destructor. Then you simply cannot refer to the pointer before creation or after destruction.
Use a local variable pointer that is created at the beginning of the function and destroyed as the last act of the function.
One thing I would strongly recommend is to avoid writing if Assigned() tests into your code unless it is expected behaviour that the pointer may not be created. Your code will become hard to read and you will also lose track of whether the pointer being nil is to be expected or is a bug.
Of course we all do make mistakes and leave dangling pointers. Using FreeAndNil is one cheap way to ensure that dangling pointer access is detected. A more effective method is to use FastMM in full debug mode. I cannot recommend this highly enough. If you are not using this wonderful tool, you should start doing so ASAP.
If you find yourself struggling with dangling pointers and you find it hard to work out why then you probably need to refactor the code to fit into one of the two idioms above.
You can draw a parallel with array indexing errors. My advice is not to check in code for validity of index. Instead use range checking and let the tools do the work and keep the code clean. The exception to this is where the input comes from outside your program, e.g. user input.
My parting shot: only ever write if Assigned if it is normal behaviour for the pointer to be nil.
Use a memory manager, such as FastMM, that provides debugging support, in particular to fill a block of freed memory with a given byte pattern. You can then dereference the pointer to see if it points at a memory block that starts with the byte pattern, or you can let the code run normallly ad raise an AV if it tries to access a freed memory block through a dangling pointer. The AV's reported memory address will usually be either exactly as, or close to, the byte pattern.
Nothing can find a dangling (once valid but then not) pointer. It's your responsibility to either make sure it's set to nil when you free it's content, or to limit the scope of the pointer variable to only be available within the scope it's valid. (The second is the better solution whenever possible.)
The core point is that the way how objects are implemented in Delphi has some built-in design drawbacks:
there is no distinction between an object and a reference to an object. For "normal" variables, say a scalar (like int) or a record, these two use cases can be well told apart - there's either a type Integer or TSomeRec, or a type like PInteger = ^Integer or PSomeRec = ^TSomeRec, which are different types. This may sound like a neglectable technicality, but it isn't: a SomeRec: TSomeRec denotes "this scope is the original owner of that record and controls its lifecycle", while SomeRec: PSomeRec tells "this scope uses a transient reference to some data, but has no control over the record's lifecycle. So, as dumb it may sound, for objects there's virtually no one who has denotedly control over other objects' lifecycles. The result is - surprise - that the lifecycle state of objects may in certain situations be unclear.
an object reference is just a simple pointer. Basically, that's ok, but the problem is that there's sure a lot of code out there which treats object references as if they were a 32bit or 64bit integer number. So if e.g. Embarcadero wanted to change the implementation of an object reference (and make it not a simple pointer any more), they would break a lot of code.
But if Embarcadero wanted to eliminate dangling object pointers, they would have to redesign Delphi object references:
when an object is freed, all references to it must be freed, too. This is only possible by double-linking both, i.e. the object instance must carry a list with all of the references to it, that is, all memory addresses where such pointers are (on the lowest level). Upon destruction, that list is traversed, and all those pointers are set to nil
a little more comfortable solution were that the "one" holding such a reference can register a callback to get informed when a referenced object is destroyed. In code: when I have a reference FSomeObject: TSomeObject I would want to be able to write in e.g. SetSomeObject: FSomeObject.OnDestruction := Self.HandleDestructionOfSomeObject. But then FSomeObject can't be a pointer; instead, it would have to be at least an (advanced) record type
Of course I can implement all that by myself, but that is tedious, and isn't it something that should be addressed by the language itself? They also managed to implement for x in ...
According to The Apple Swift documentation: Global functions are closures that have a name and do not capture any values. But I ran into an example domenstrating use of closures in the book IOS 11 Programming Fundamentals with Swift, which passes a global function A as a parameter into another global function B to modify the value of a global variable x. The book states that A captures X, which contradicts what the Swift documentation says.
The code example:
func pass100(_ f: (Int) -> ()) {
f(100)
}
var x = 0
print(x) // output 0
func setX(newX: Int) {
x = newX
}
pass100(setX)
print(x) //output 100
The above code snippet works in Xcode and I am confused. Could anyone explain what's going on here?
This is what "capturing" means:
A closure can capture constants and variables from the surrounding context in which it is defined. The closure can then refer to and modify the values of those constants and variables from within its body, even if the original scope that defined the constants and variables no longer exists.
Global functions are said to not capture anything because you can never leave the global scope while your program is running, so there is no point in capturing anything. If it did capture something, that means global variables can still be changed even after you somehow "left" the global scope (maybe the program finishes running), which would be weird.
The global function in your code is not capturing anything. It can change the value of x simply because x is still in scope. You've not left the global scope.
A closure can capture constants and variables from the surrounding
context in which it is defined.
Global, like local, is a context. A is a global function and X is a global variable; therefore, A captures X. Technically, all of the methods in the app capture X because it's exposed in the global context. Conventionally speaking, however, "capture" usually refers to a local scope.
To demonstrate this, if you moved A and X into a local scope, such as a method within a class, nothing changes—A still captures X. All you've done in your example is placed them into a global scope, and because they're in the same scope (or context), the rules of capturing have not changed.
It's a matter of semantics and convention and I would argue that this is one area where Swift documentation may be slightly misleading (if I were Apple, I'd reword it). Moving a function from a local scope to a global scope in no way alters its ability to capture. I think what Apple means is that by default, a local method always captures something (even if it's self), whereas a global function is not guaranteed to capture anything if there isn't anything around to be captured.
Just wondering, since closure is a function that has references to variables/methods outside it's definition.
Every function closes over program's global variables (basically in every mainstream language, be it javascript/python/c/c+/whatever).
So, consequently, every function is a closure?
Edit: let me reemphasize, I'm not talking only about closures in javascript but in a more general context
Yes, exactly. As you've identified, every function in JavaScript is a closure over at least one context: The global context. That's how/why global variables work in JavaScript.
We don't normally call them closures unless they close over some other context and actually make use of the fact that they do, but you're quite right that at a technical level, they all are.
Every function closes over program's global variables (basically in every mainstream language, be it javascript/c/c+/whatever).
I wouldn't generalize that far, no. Different languages have different ways of implementing global variables. Whether functions in those languages are all "closures" is probably open for debate, so I've restricted my answer above to JavaScript.
closure is a function that has references to variables/methods outside its definition
No, this is a "function with free variables", not a "closure".
To quote wikipedia
...a closure is only distinct from a function with free variables when outside of the scope of the non-local variables, otherwise the defining environment and the execution environment coincide and there is nothing to distinguish these (static and dynamic binding can't be distinguished because the names resolve to the same values).
In other words, in some context, a closure is a reference to a function that binds variables from another context. Otherwise, it wouldn't make sense to call it a "closure".
I'm new to Delphi, and I've been running some tests to see what object variables and stack variables are initialized to by default:
TInstanceVariables = class
fBoolean: boolean; // always starts off as false
fInteger: integer; // always starts off as zero
fObject: TObject; // always starts off as nil
end;
This is the behaviour I'm used to from other languages, but I'm wondering if it's safe to rely on it in Delphi? For example, I'm wondering if it might depend on a compiler setting, or perhaps work differently on different machines. Is it normal to rely on default initialized values for objects, or do you explicitly set all instance variables in the constructor?
As for stack (procedure-level) variables, my tests are showing that unitialized booleans are true, unitialized integers are 2129993264, and uninialized objects are just invalid pointers (i.e. not nil). I'm guessing the norm is to always set procedure-level variables before accessing them?
Yes, this is the documented behaviour:
Object fields are always initialized to 0, 0.0, '', False, nil or whatever applies.
Global variables are always initialized to 0 etc as well;
Local reference-counted* variables are always initialized to nil or '';
Local non reference-counted* variables are uninitialized so you have to assign a value before you can use them.
I remember that Barry Kelly somewhere wrote a definition for "reference-counted", but cannot find it any more, so this should do in the meantime:
reference-counted == that are reference-counted themselves, or
directly or indirectly contain fields (for records) or elements (for
arrays) that are reference-counted like: string, variant, interface
or dynamic array or static array containing such types.
Notes:
record itself is not enough to become reference-counted
I have not tried this with generics yet
Global variables that don't have an explicit initializer are allocated in the BSS section in the executable. They don't actually take up any space in the EXE; the BSS section is a special section that the OS allocates and clears to zero. On other operating systems, there are similar mechanisms.
You can depend on global variables being zero-initialized.
Class fields are default zero. This is documented so you can rely on it.
Local stack varaiables are undefined unless string or interface, these are set to zero.
Just as a side note (as you are new to Delphi): Global variables can be initialized directly when declaring them:
var myGlobal:integer=99;
Here's a quote from Ray Lischners Delphi in a Nutshell Chapter 2
"When Delphi first creates an object, all of the fields start out empty, that is, pointers are initialized to nil, strings and dynamic arrays are empty, numbers have the value zero, Boolean fields are False, and Variants are set to Unassigned. (See NewInstance and InitInstance in Chapter 5 for details.)"
It's true that local-in-scope variables need to be initialised... I'd treat the comment above that "Global variables are initialised" as dubious until provided with a reference - I don't believe that.
edit...
Barry Kelly says you can depend on them being zero-initialised, and since he's on the Delphi compiler team I believe that stands :) Thanks Barry.
Global variables and object instance data (fields) are always initialized to zero.
Local variables in procedures and methods are not initialized in Win32 Delphi; their content is undefined until you assign them a value in code.
Even if a language does offer default initializations, I don't believe you should rely on them. Initializing to a value makes it much more clear to other developers who might not know about default initializations in the language and prevents problems across compilers.
From Delphi 2007 help file:
ms-help://borland.bds5/devcommon/variables_xml.html
"If you don't explicitly initialize a global variable, the compiler initializes it to 0."
I have one little gripe with the answers given. Delphi zeros out the memory space of the globals and the newly-created objects. While this NORMALLY means they are initialized there is one case where they aren't: enumerated types with specific values. What if zero isn't a legal value??
Newly introduced (since Delphi 10.3) inline variables are making the control of initial values easier.
procedure TestInlineVariable;
begin
var index: Integer := 345;
ShowMessage(index.ToString);
end;