In most managed languages (that is, the ones with a GC), local variables that go out of scope are inaccessible and have a higher GC-priority (hence, they'll be freed first).
Now, C is not a managed language, what happens to variables that go out of scope here?
I created a small test-case in C:
#include <stdio.h>
int main(void){
int *ptr;
{
// New scope
int tmp = 17;
ptr = &tmp; // Just to see if the memory is cleared
}
//printf("tmp = %d", tmp); // Compile-time error (as expected)
printf("ptr = %d\n", *ptr);
return 0;
}
I'm using GCC 4.7.3 to compile and the program above prints 17, why? And when/under what circumstances will the local variables be freed?
The actual behavior of your code sample is determined by two primary factors: 1) the behavior is undefined by the language, 2) an optimizing compiler will generate machine code that does not physically match your C code.
For example, despite the fact that the behavior is undefined, GCC can (and will) easily optimize your code to a mere
printf("ptr = %d\n", 17);
which means that the output you see has very little to do with what happens to any variables in your code.
If you want the behavior of your code to better reflect what happens physically, you should declare your pointers volatile. The behavior will still be undefined, but at least it will restrict some optimizations.
Now, as to what happens to local variables when they go out of scope. Nothing physical happens. A typical implementation will allocate enough space in the program stack to store all variables at the deepest level of block nesting in the current function. This space is typically allocated in the stack in one shot at the function startup and released back at the function exit.
That means that the memory formerly occupied by tmp continues to remain reserved in the stack until the function exits. That also means that the same stack space can (and will) be reused by different variables having approximately the same level of "locality depth" in sibling blocks. The space will hold the value of the last variable until some other variable declared in some sibling block variable overrides it. In your example nobody overrides the space formerly occupied by tmp, so you will typically see the value 17 survive intact in that memory.
However, if you do this
int main(void) {
volatile int *ptr;
volatile int *ptrd;
{ // Block
int tmp = 17;
ptr = &tmp; // Just to see if the memory is cleared
}
{ // Sibling block
int d = 5;
ptrd = &d;
}
printf("ptr = %d %d\n", *ptr, *ptrd);
printf("%p %p\n", ptr, ptrd);
}
you will see that the space formerly occupied by tmp has been reused for d and its former value has been overriden. The second printf will typically output the same pointer value for both pointers.
The lifetime of an automatic object ends at the end of the block where it is declared.
Accessing an object outside of its lifetime is undefined behavior in C.
(C99, 6.2.4p2) "If an object is referred to outside of its lifetime, the behavior is undefined. The value of a pointer becomes indeterminate when the object it points to reaches the end of its lifetime."
Local variables are allocated on the stack. They are not "freed" in the sense you think about GC languages, or memory allocated on the heap. They simply go out of scope, and for builtin types the code won't do anything - and for objects, the destructor is called.
Accessing them beyond their scope is Undefined Behaviour. You were just lucky, as no other code has overwritten that memory area...yet.
Related
I am working on dynamic memory analysis using stack painting/foot print analysis method.
dynamic-stack-depth-determination-using-footprint-analysis
basically the idea is to fill the entire amount of memory allocated to the stack area with a dedicated fill value, for example 0xABABABAB, before the application starts executing. Whenever the execution stops, the stack memory can be searched upwards from the end of the stack until a value that is not 0xABABABABis found, which is assumed to be how far the stack has been used. If the dedicated value cannot be found, the stack has consumed all stack space and most likely has overflowed.
I want a c code to fill the stack from top to bottom with a pattern.
void FillSystemStack()
{
extern char __stack_start,_Stack_bottom;
}
NOTE
I am using STM32F407VG board emulated with QEMU on eclipse.
stack is growing from higher address to lower address
start of the stack is 0x20020000
bottom of the stack is Ox2001fc00
You shouldn't completely fill the stack after main() begins, because the stack is in use once main() begins. Completely filling the stack would overwrite the bit of stack that has already been used and could lead to undefined behavior. I suppose you could fill a portion of the stack soon after main() begins as long as you're careful not to overwrite the portion that has been used already.
But a better plan is to fill the stack with a pattern before main() is called. Review the startup code for your tool chain. The startup code initializes variable values and sets the stack pointer before calling main(). The startup code may be in assembly depending on your tool chain. The code that initializes variables is probably a simple loop that copies bytes or words from the appropriate ROM to RAM sections. You can probably use this code as an example to write a new loop that will fill the stack memory range with a pattern.
This is a Cortex M so it gets the stack pointer set out of reset. Meaning it's pretty much instantly ready to go for C code. If your reset vector is written in C and performs stacking/C function calls, it will be too late to fill the stack at a very early stage. Meaning you shouldn't do it from application C code.
The normal way to do the trick you describe is through an in-circuit debugger. Download the program, hit reset, fill the stack with the help of the debugger. There will be some convenient debugger command available to do that. Execute the program, try to use as much of it as possible, observe the stack in the memory map of your debugger.
With the insights from #kkrambo answer, I tried to paint the stack just after the start of main by taking care that I do not overwrite the portion that has been used already.My stack paint and stack count functions are given below:
StackPaint
uint32_t sp;
#define STACK_CANARY 0xc5
#define WaterMark 0xc9
void StackPaint(void)
{
char *p = &__StackLimit; // __StackLimit macro defined in linker script
sp=__get_MSP();
PRINTF("stack pointer %08x \r\n\n",sp);
while((uint32_t)p < sp)
{
if(p==&__StackTop){ // __StackTop macro defined in linker script
*p = WaterMark;
p++;
}
*p = STACK_CANARY;
p++;
}
}
StackCount
uint16_t StackCount(void)
{
PRINTF("In the check address function in main file \r\n\n");
const char *p = &__StackLimit;
uint16_t c = 0;
while(*p == WaterMark || (*p == STACK_CANARY && p <= &__StackTop))
{
p++;
c++;
}
PRINTF("stack used:%d bytes \n",1024-c);
PRINTF("remaining stack :%d bytes\n",c);
return c;
}
For the past few days am trying to learn if pass by value and pass by reference impact the memory differently. Googling this query, people kept repeating themselves about a copy being created in terms of pass by value and how the original value is affected in terms pass by reference. But I was wondering if someone could zero in on the memory part.
This question actually depends heavily on the particular language as some allow you to be explicit and define when you want to pass a variable by value and when by reference and some do it always the same way for different types of variables.
A quite popular type of behavior is to use passing by value (by default) for simple times: like int, string, long, float, double, bool etc.
Let us show the memory impact on a theoretical language:
int $myVariable = 5;
at this moment you have created a one variable in memory which takes the size required to store an integer (let us say 32 bits).
Now you want to pass it to a function:
function someFunction(int parameter)
{
printOnScreen(parameter);
}
so your code would look like:
function someFunction(int $parameter)
{
printOnScreen($parameter);
}
int $myVariable = 5; //Position A
someFunction($myVariable); //Position B
...rest of the code //Position C
Since simple types are passed by value the value is copied in memory to another storage place - therefore:
during Position A you have memory occupied by ONE int (with value 5);
during Position B you have memory occupied by TWO ints (with values of 5) as your $myVariable was copied in memory
during Position C you have again memory occupied by ONE int (with value of 5) as the second one was already destroyed as it was needed only for the time of execution of the function
This has some other implications: modifications on a variable passed by value DO NOT affect the original variable - for example:
function someFunction(int $parameter)
{
$parameter = $parameter + 1;
printOnScreen($parameter);
}
int $myVariable = 5; //Position A
someFunction($myVariable); //Position B
printOnScreen($myVariable); //Position C
During position A you set value of 5 under variable $myVariable.
During position B you pass it BY VALUE to a function which adds 1 to your passed value. YET since it was a simple type, passed by value, it actually operates on a LOCAL variable, a COPY of your variable. Therefore position C will again write just 5 (your original variable as it was not modified).
Some languages allow you to be explicit and inform that you want to pass a reference and not the value itself using a special operator -for example &. So let us again follow the same example but with explicit info that we want a reference (in function's arguments
- note the &):
function someFunction(int &$parameter)
{
$parameter = $parameter + 1;
printOnScreen($parameter);
}
int $myVariable = 5; //Position A
someFunction($myVariable); //Position B
printOnScreen($myVariable); //Position C
This time operation and memory implications will be different.
During Position A an int is created (every variable is always consisted of two elements: place in memory and a pointer, an identifier which place is it. For ease of the process let us say that pointer is always one byte). So whenever you create a variable you actually create two things:
reserved place in memory for the VALUE (in this case 32 bits as it was an int)
pointer (8 bits [1 byte])
Now during position B, the function expects A POINTER to a memory place. Which means that it will locally, for itself create only a copy of the pointer (1 byte) and not copy the actual reserved place as the new pointer WILLL POINT to the same place as the original one. This means that during operation of the function you have:
TWO POINTERS to an int in memory
ONE place reserved for VALUE of the int
Both of those pointer POINT to the same VALUE
Which means that any modification of the value will affect both.
So looking at the same example position C will not print out also 6 as inside the function we have modified the value under the SAME POINTER as $myVariable.
For COMPLEX TYPES (objects) the default action in most programming environments is to pass the reference (pointer).
So for example - if you have a class:
class Person {
public string $name;
}
and create an instance of it and set a value:
$john = new Person();
$john->name = "John Malkovic";
and later pass it to a function:
function printName(Person $instanceOfPerson)
{
printOnScreen($instanceOfPerson);
}
in terms of memory it will again create only a new POINTER in memory (1 byte) which points to the same value. So having a code like this:
function printName(Person $instanceOfPerson)
{
printOnScreen($instanceOfPerson);
}
$john = new Person(); // position A
printName($john); // position B
...rest of the code // position C
during position A you have: 1 Person (which means 1 pointer [1 byte] to a place in memory which has size to store an object of class person)
during position B you have: 2 pointers [2 bytes] but STILL one place in memory to store an object of class person's value [instance]
during position C you have again situation from position A
I hope that this clarifies the topic for you - generally there is more to cover and what I have mentioned above is just a general explanation.
Pass-by-value and pass-by-reference are language semantics concepts; they don't imply anything about the implementation. Usually, languages that have pass-by-reference implement it by passing a pointer by value, and then when you read or write to the variable inside the function, the compiler translates it into reading or writing from a dereference of the pointer. So you can imagine, for example, if you have a function that takes a parameter by reference in C++:
struct Foo { int x; }
void bar(Foo &f) {
f.x = 42;
}
Foo a;
bar(a);
it is really syntactic sugar for something like:
struct Foo { int x; }
void bar(Foo *f_ptr) {
(*f_ptr).x = 42;
}
Foo a;
bar(&a);
And so passing by reference has the same cost as passing a pointer by value, which does involve a "copy", but it's the copy of a pointer, which is a few bytes, regardless of the size of the thing pointed to.
When you talk about pass-by-value doing a "copy", that doesn't really tell you much unless you know what exactly the variable or value passed represents in the language. For example, Java only has pass-by-value. But every type in Java is either a primitive type or a reference type, and the values of reference types are "reference", i.e. pointers to objects. So you can never have a value in Java (what a variable holds or what an expression evaluates to) which "is" an "object"; objects in Java can only be manipulated through these "references" (pointers to objects). So when you ask the cost of passing a object in Java, it's actually wrong because you cannot "pass" an object in Java; you can only pass references (pointers to objects), and the copy the happens for pass-by-value, is the copy of the pointer, which is a few bytes.
So the only case where you would actually copy a big structure when passing, is if you have a language where objects or structs are values directly (not behind a reference), and you do pass-by-reference of that object/struct type. So for example, in C++, you can have objects which are values directly, or you can have pointers to them, and you can pass them by value or by reference:
struct Foo { int x; }
void bar1(Foo f1) { } // pass Foo by value; this copies the entire size of Foo
void bar2(Foo *f2) { } // pass pointer by value; this copies the size of a pointer
void bar3(Foo &f3) { } // pass Foo by reference; this copies the size of a pointer
void bar4(Foo *&f4) { } // pass pointer by reference; this copies the size of a pointer
(Of course, each of those have different semantic meanings; for example, the last one allows the code inside the function to modify the pointer variable passed to point to somewhere else. But if you are concerned about the amount copied. Only the first one is different. In Java, effectively only the second one is possible.)
I came across this strange phenomenon while playing with unsafe Rust. I think this code should make a segmentation fault but it does not. Am I missing something?
I tried to set a pointer to a variable with a shorter lifetime and then dereference it.
// function that sets a pointer to a variable with a shorter lifetime
unsafe fn what(p: &mut *const i32) {
let a = 2;
*p = &a;
//let addr = *p; // I will talk about this later
println!("inside: {}", **p);
}
fn main() {
let mut p: *const i32 = 0 as *const i32;
unsafe {
what(&mut p);
// I thought this line would make a segfault because 'a' goes out of scope at the end of the function making the address invalid
println!("segfault? {}", *p);
// Even more unsettling: I can increment the address and still dereference it.
p = ((p as usize) + 1) as *const i32;
println!("I'm definitely missing something: {}", *p);
}
}
This program outputs:
inside: 2
segfault? {random number around 20000. probably uninitialized memory but why?}
I'm definitely missing something: {uninitialized memory}
If I uncomment the line
let addr = *p;
the second row becomes
segfault? 2
Why is there no segfault? Can the compiler extend the lifetime of a or the address p points at for safety? Am I missing some basic information about pointers in Rust?
This isn't unique to Rust. See:
Why doesn't the following code produce a segmentation fault?
Accessing an array out of bounds gives no error, why?
Can a local variable's memory be accessed outside its scope?
TL;DR: you have lied to the compiler. Your unsafe block does not uphold the requirements of safe code. This means you have created undefined behavior and the program is allowed to do whatever it wants. That could mean:
it crashes (such as via a segfault)
it runs perfectly
it erases your hard drive
it empties your bank account
it generates nasal demons
it eats your laundry
etc.
Segfaults are never a guaranteed outcome. A segmentation fault occurs when you access memory that is outside of the chunk of memory your thread/process has. The memory of the stack is well inside of that chunk, so it's unlikely to trigger the case.
See also:
Why doesn't this Rust program crash?
The following is base on my guess. Someone please point out the parts that I understand incorrectly.
If I have a class, of which an instance occupies 128 bits, called Class128Bits. And my program runs on a 64 bits computer.
First, I call let pointer = UnsafeMutablePointer<Calss128Bits>.allocate(capacity: 2)
the memory layout should look like this:
000-063 064 bits chaos
064-127 064 bits chaos
128-255 128 bits chaos
256-383 128 bits chaos
If I call pointer.pointee = aClass128Bits, it crashes because the pointers in the first two grids have not been initialized yet. Accessing to what they point to leads to unpredictable results.
But if I call pointer.initialize(to: aClass128Bits, count: 2), the pointers could be initialized like this:
000-063 address to offset 128
064-127 address to offset 256
128-255 a copy of aClass128Bits
256-383 a copy of aClass128Bits
Then any accesses will be safe.
However this cannot explain why UnsafeMutablePointer<Int> does not crash.
Original
The case I am facing:
The pointer to Int works fine, but the one to String crashes.
I know that I need to initialize it like this:
But I can't see the reason why I need to pass "42" twice.
In C, I might do something similar like this:
char *pointer = (char *)malloc(3 * sizeof(char));
memcpy(pointer, "42", 3);
free(pointer)
If allocate equals malloc, free equals deallocate, memcpy equals pointee{ set },
then what do initialize and deinitialize actually do?
And why does my code crash?
let pointer0 = UnsafeMutablePointer<String>.allocate(capacity: 1)
let pointer1 = UnsafeMutablePointer<Int>.allocate(capacity: 1)
let check the size of both
MemoryLayout.size(ofValue: pointer0) // 8
MemoryLayout.size(ofValue: pointer1) // 8
let check the value of .pointee
pointer0.pointee // CRASH!!!
while
pointer1.pointee // some random value
Why? The answer is as simple, as it can be. We allocated 8 bytes, independently from "associated" Type. Now is clear, that 8 bytes in memory are not enough to store any String. the underlying memory must be referenced indirectly. But there are some 8 random bytes there ... Loading what is in the memory with address represented by 8 random bytes as a String will most likely crash :-)
Why didn't it crash in the second case? Int value is 8 bytes long and the address can be represented as Int value.
let's try in the Playground
import Foundation
let pointer = UnsafeMutablePointer<CFString>.allocate(capacity: 1)
let us = Unmanaged<CFString>.passRetained("hello" as CFString)
pointer.initialize(to: us.takeRetainedValue())
print(pointer.pointee)
us.release()
// if this playground crash, try to run it again and again ... -)
print(pointer.pointee)
look what it prints to me :-)
hello
(
"<__NSCFOutputStream: 0x7fb0bdebd120>"
)
There is no miracle behind. pointer.pointee is trying to represent what is in the memory, which address is stored in our pointer, as a value of its associated type. It never crashes for Int because every 8 continues bytes somewhere in the memory can be represented as Int.
Swift use ARC, but creating the Unsafe[Mutable]Poiner doesn't allocate any memory for the instance of T, destroying it doesn't deallocate any memory for it.
Typed memory must be initialized before use and deinitialized after use. This is done using initialize and deinitialize methods respectively. Deinitialization is only required for non-trivial types. That said, including deinitialization is a good way to future-proof your code in case you change to something non-trivial
Why doesn't assignment to .pointee with Int value crash?
Initialize store the address of value
Assignment to pointee update the value at stored address
Without initializing It most likely will crash, only the probability is less by modifying only 8 bytes in memory at some random address.
trying this
import Darwin
var k = Int16.max.toIntMax()
typealias MyTupple = (Int32,Int32,Int8, Int16, Int16)
var arr: [MyTupple] = []
repeat {
let p = UnsafeMutablePointer<MyTupple>.allocate(capacity: 1)
if k == 1 {
print(MemoryLayout.size(ofValue: p), MemoryLayout.alignment(ofValue: p),MemoryLayout.stride(ofValue: p))
}
arr.append(p.pointee)
k -= 1
defer {
p.deallocate(capacity: 1)
}
} while k > 0
let s = arr.reduce([:]) { (r, v) -> [String:Int] in
var r = r
let c = r["\(v.0),\(v.1),\(v.2),\(v.3)"] ?? 0
r["\(v.0),\(v.1),\(v.2),\(v.3)"] = c + 1
return r
}
print(s)
I received
8 8 8
["0,0,-95,4104": 6472, "0,0,0,0": 26295]
Program ended with exit code: 0
It doesn't look very random, is it? That explains, why the crash with the typed pointer to Int is very unlikely.
One reason you need initialize(), and the only one as for now maybe, is
for ARC.
You'd better think with local scope variables, when seeing how ARC works:
func test() {
var refVar: RefType = initValue //<-(1)
//...
refVar = newValue //<-(2)
//...
//<-(3) just before exiting the loacl scope
}
For a usual assignment as (2), Swift generates some code like this:
swift_retain(_newValue)
swift_release(_refVar)
_refVar = _newValue
(Assume _refVar and _newValue are unmanaged pseudo vars.)
Retain means incrementing the reference count by 1, and release means decrementing the reference count by 1.
But, think what happens when the initial value assignment as at (1).
If the usual assignment code was generated, the code might crash at this line:
swift_release(_refVar)
because newly allocated region for a var may be filled with garbages, so swift_release(_refVar) cannot be safely executed.
Filling the newly region with zero (null) and release safely ignoring the null could be one solution, but it's sort of redundant and not effective.
So, Swift generates this sort of code for initial value assignment:
(for already retained values, if you know ownership model, owned by you.)
_refVar = _initValue
(for unretained values, meaning you have no ownership yet.)
swift_retain(_initValue)
_refVar = _initValue
This is initialize.
No-releasing the garbage data, and assign an initial value, retaining it if needed.
(The above explanation of "usual assignment" is a little bit simplified, Swift omits swift_retain(_newValue) when not needed.)
When exiting the local scope at (3), Swift just generates this sort of code:
swift_release(_refVar)
So, this is deinitialize.
Of course, you know retaining and releasing are not needed for primitive types like Int, so initialize and deinitialize may be donothing for such types.
And when you define a value type which includes some reference type properties, Swift generates initialize and deinitialize procedures specialized for the type.
The local scope example works for the regions allocated on the stack, and initialize() and deinitialize() of UnsafeMutablePointer works for the regions allocated in the heap.
And Swift is evolving so swift, that you might find another reason for needing initialize() and deinitialize() in the future, you'd better make it a habit to initialize() and deinitialize() all allocated UnsafeMutablePointers of any Pointee types.
From the documentation it is possible to conclude that .initialize() is a method that :
Initializes memory starting at self with the elements of source.
And .deinitialize() is a method that :
De-initializes the count Pointees starting at self, returning their
memory to an uninitialized state.
We should understand that when we are using UnsafeMutablePointer we should manage memory on our own. And methods that are described above help us to do this.
So in your case lets analyze example that you provide:
let pointer = UnsafeMutablePointer<String>.allocate(capacity: 1)
// allocate a memory space
pointer.initialize(to: "42")
// initialise memory
pointer.pointee // "42"
// reveals what is in the pointee location
pointer.pointee = "43"
// change the contents of the memory
pointer.deinitialize()
// return pointer to an unintialized state
pointer.deallocate(1)
// deallocate memory
So your code crashes because you do not initialize memory and try to set value.
Previously in objective-c when we are working with objects we always use [[MyClass alloc] init]].
In this case :
alloc:
allocates a part of memory to hold the object, and returns the
pointer.
init:
sets up the initial parameters of the object and returns it.
So basically .initialize() sets the value to the allocated memory part. When you create an object only with alloc you only set reference to empty memory part in the heap. When you call .initialize() you set value to this memory allocation in the heap.
Nice article about the pointers.
I just had Apple's C/C++ compiler initialize a float to a non-zero value (approx "-0.1").
That was a big surprise - and only happened occasionally (but 100% repeatably, if you ran through the same function calls / args beforehand). It took a long time to track down (using assertions).
I'd thought floats were zero-initialized. Googling suggests that I was thinking of C++ (which of course is much more precise about this stuff - c.f. SO: What are primitive types default-initialized to in C++? ).
But maybe Apple's excuse here is that their compiler was running in C mode ... so: what about C? What should happen, and (more importantly) what's typical?
(OF COURSE I should have initialized it manually - I normally do - but in this one case I failed. I didn't expect it to blow up, though!)
(Google is proving worse than useless for any discussion of this - their current search refuses to show "C" without "C++". Keeps deciding I'm too stupid, and ignoring even my input even when running in advanced mode)
Here's the actual source example where it happened. At first I thought there might be a problem with definitions of MAX and ABS (maybe MAX(ABS,ABS) doesnt always do what you'd expect?) ... but digging with assertions and debugger, I eventually found it was the missing initialization - that float was getting init'd to non-zero value VERY occasionally):
float crossedVectorX = ... // generates a float
float crossedVectorY = ... // generates a float
float infitesimal; // no manual init
float smallPositiveFloat = 2.0 / MAX( ABS(crossedVectorX), ABS(crossedVectorY));
// NB: confirmed with debugger + assertions that smallPositiveFloat was always positive
infitesimal += smallPositiveFloat;
NSAssert( infitesimal >= 0.0, #"This is sometimes NOT TRUE" );
Only objects with static storage duration are initialized to 0 if there is no explicit initializer.
#include <stdio.h>
float f; // initialized to 0, file scope variables have static storage
static float g; // initialized to 0
int main(void)
{
float h; // not initialized to 0, automatic storage duration
static float i; // initialized to 0
return 0;
}
Objects with automatic storage duration (like h in the example above) that are not explicitly initialized have an indeterminate value. Reading their value is undefined behavior.
EDIT: for the sake of completeness, since C11 objects with thread storage duration are also initialized to 0 if there is no explicit initializer.
The relevant part of the standard is §6.7.9 paragraph 10:
If an object that has automatic storage duration is not initialized explicitly, its value is indeterminate.
If your variable had thread or static storage duration instead, then the next part of the paragraph would take effect:
If an object that has static or thread storage duration is not initialized explicitly, then:
-- if it has pointer type, it is initialized to a null pointer;
-- if it has arithmetic type, it is initialized to (positive or unsigned) zero;
...
I would also note that you should turn on your compiler's warnings (specifically the warning for uninitialized variables), as that should have identified the problem for you immediately.
Static variable would be initialized to zero, but I'm guessing you are talking about a local variable (i.e. stack, or automatic) - these are not initialized for you, but get whatever value is at that memory on the stack.
I had to pull out my K&R for this answer:
In the absence of explicit initialization, external and static variables are guaranteed to be initialized to zero; automatic and register variables have undefined (i.e., garbage) initial values.
I don't believe that any of the standards for C define initial values for variables in general. This would be in accord with the general philosophy of and application domain for C -- programming for grown-ups who may, one day, have reason to want their compiler to not initialise a variable for them and who know that it is their responsibility to initialise their own variables.