In the Keil RTX RTOS configuration file, user could configure the default user thread stack size.
In general, the stack holds auto/local variables. The "ZI data" section holds uninitialized global variables.
So if I change the user thread stack size in the RTX configuration file, the stack size will increase and the "ZI data" section size will not increase.
I test it, the test result shows that if I increase user thread stack size. The "ZI data" section size will increase synchronously with the same size.
In my test program, there is 6 threads and each has 600 bytes stack. I use Keil to build the program and it shows me that:
Code (inc. data) RO Data RW Data ZI Data Debug
36810 4052 1226 380 6484 518461 Grand Totals
36810 4052 1226 132 6484 518461 ELF Image Totals (compressed)
36810 4052 1226 132 0 0 ROM Totals
==============================================================================
Total RO Size (Code + RO Data) 38036 ( 37.14kB)
Total RW Size (RW Data + ZI Data) 6864 ( 6.70kB)
Total ROM Size (Code + RO Data + RW Data) 38168 ( 37.27kB)
But if I changed each thread stack size to 800 bytes. Keil shows me as follows:
==============================================================================
Code (inc. data) RO Data RW Data ZI Data Debug
36810 4052 1226 380 7684 518461 Grand Totals
36810 4052 1226 132 7684 518461 ELF Image Totals (compressed)
36810 4052 1226 132 0 0 ROM Totals
==============================================================================
Total RO Size (Code + RO Data) 38036 ( 37.14kB)
Total RW Size (RW Data + ZI Data) 8064 ( 7.88kB)
Total ROM Size (Code + RO Data + RW Data) 38168 ( 37.27kB)
==============================================================================
The "ZI data" section size increase from 6484 to 7684 bytes. 7684 - 6484 = 1200 = 6 * 200. And 800 - 600 = 200.
So I see the thread stack is put in "ZI Data" section.
My question is:
Does it mean auto/local variables in the thread will be put in "ZI Data" section, when thread stack is put in "ZI data" section in RAM ?
If it's true, it means there is no stack section at all. There are only "RO/RW/ZI Data" and heap sections at all.
This article gives me the different answer. And I am a little confused about it now.
https://developer.mbed.org/handbook/RTOS-Memory-Model
The linker determines what memory sections exist. The linker creates some memory sections by default. In your case, three of those default sections are apparently named "RO Data", "RW Data", and "ZI Data". If you don't explicitly specify which section a variable should be located in then the linker will assign it to one of these default sections based on whether the variable is declared const, initialized, or uninitialized.
The linker is not automatically aware that you are using an RTOS. And it has no special knowledge of which variables you intend to use as thread stacks. So the linker does not automatically create independent memory sections for your thread stacks. Rather, the linker will treat the stack variables like any other variable and include them in one of the default memory sections. In your case the thread stacks are apparently being put into the ZI Data section by the linker.
If you want the linker to create special independent memory sections for your thread stacks then you have to explicitly tell the linker to do so via the linker command file. And then you also have to specify that the stack variables should be located in your custom sections. Consult the linker manual for details on how to do this.
Tasks stacks have to come from somewhere - in RTX by default they are allocated statically and are of fixed size.
The os_tsk_create_user() allows the caller to supply a stack that could be allocated in any manner (statically or from the heap; allocating from the caller stack is possible, but unusual, probably pointless and certainly dangerous) so long as it has 8 byte alignment. I find RTX's automatic stack allocation almost useless and seldom appropriate in all but the most trivial application.
Related
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 7680 bytes) in /home/admin/web/original-peptide.com/public_html/tools/smarty/sysplugins/smarty_internal_templatelexer.php on line 473
try this
ini_set('memory_limit', '512M');
look here for more info
A support customer just modified a product from "normal" product to "pack", and added the same product as a variant of the new "pack", resulting in a endless loop, using up as much RAM as the system would allow.
Since the product page didn't open anymore neither in the backend and on the catalog page, we had to undo the "packing" via database.
delete from ps_pack where id_product_item=1;
delete from ps_pack where id_product_item=2;
update ps_product set cache_is_pack=0 where id_product=1;
update ps_product set cache_is_pack=0 where id_product=2;
Why does this code not work?
TIB 10 ACCEPT
TIB SP# 1 cells - 10 cmove
In that code I tried to enter a string and store it in the terminal input buffer and later store it on the parameter stack.
But with .S I see that does not work.
The parameter stack grows towards low memory
The main problem with the sample code is that the parameter stack grows towards low memory. So the starting point for the destination of the copy should be at a higher memory address (inside the existing/defined parameter stack). So instead of
TIB SP# 1 cells - 10 cmove
it should be:
TIB SP# 1 cells + 10 cmove
Memory allocation for the string
The next problem is that there is not enough storage for the string on the parameter stack. ACCEPT has left over one cell (four bytes on a 32-bit system), the actual number of characters. With sample input "user10181" (9 characters),
TIB 10 ACCEPT
results in:
.S <1> 9 ok
Forgetting about that extra element for the moment1, for the purpose of this elaboration, we allocate four cells on the parameter stack (the actual value, for example, 235, does not matter), 16 bytes on a 32-bit system:
235 DUP DUP DUP
The result of TIB SP# 1 cells + 10 cmove is then:
.S <5> 9 235 8241 541085779 541215060 ok
We see that three of the four cells (each with four bytes) have been overwritten by cmove (as expected).
TIB is overwritten by subsequent input from the terminal
Unfortunately, our copied bytes are not as expected. Decoding the output for the three changed cells (that are in decimal), first from
8241 541085779 541215060
to hexadecimal:
2031 20405053 20424954
And then decoding as ASCII:
20 31 20 40 50 53 20 42 49 54
1 # P S B I T
And reversing (we had the high memory first and the test platform is little endian):
"TIB SP# 1 "
That is the first ten characters of our second line, TIB SP# 1 cells + 10 cmove. Thus it is clear that the terminal input buffer (TIB) is too temporary to be used in this case.
The solution
The solution to the third problem is to have all the code compiled before we ask for user input. For instance, put it into a word, inputOnStack:
: inputOnStack TIB 10 ACCEPT 235 DUP DUP DUP TIB SP# 1 cells + 10 cmove ;
The result is then:
inputOnStack user10181 ok
.S <5> 9 235 24881 942747697 1919251317 ok
That corresponds to "user10181" and the tenth character is "a" (most likely from the "a" in inputOnStack).
Test platform:
Raspberry Pi, model B.
Operating system: Raspbian, as installed by NOOBS 1.3.10, released 2014-09-09.
Gforth: version 0.7.0 (installed with sudo apt-get update; sudo apt-get install gforth)
1. A more advanced version of the code could use the actual number of characters. In any case, it should DROPped one way or the other to balance the stack if this code is be integrated into other code.
Consider very carefully what happens to the stack after each word. I have reproduced your code below, and annotated it with the stack depth at every point.
( 0 ) TIB ( 1 ) 10 ( 2 ) ACCEPT ( 1 )
( 1 ) TIB ( 2 ) SP# ( 3 ) 1 ( 4 ) cells ( 4 ) - ( 3 ) 10 ( 4 ) cmove ( 1 )
So when SP# is executed, it returns a pointer to stack element 2. The pointer is then decremented by one cell, resulting in a pointer to stack element 3 (because the stack grows downwards). cmove then overwrites 10 bytes, i.e. 2 stack elements (guessing you're running a 64-bit forth). So stack elements 3 and 2 are changed. Finally, cmove pops three elements from the stack, leaving only one. Which is unchanged.
MU!
Basically what you do here is a copy CMOVE to the area at the stack, on a Forth where SP# is defined (it is not a standard word). So you destroy the stack. The answer is: such code is not supposed to work.
The question should not be " why doesn't this work?" but "under what circumstance has this the intended effect?"
Let us assume:
this Forth has an addressable and writable stack (if you think that's the same read up about Intel segment descriptors and the particulars of POP and MOV instructions)
SP# points to the top of the stack at the moment is called.
the stack grows up. (if it grows down the situation is totally different)
at the start of the call the stack was empty.
The TIB and ACCEPT are red herrings and will be ignored.
1 cells - \ The pointer to the stack is changed opening up one cell
100 cmove \ you overwrite one cell below the stack,
\ then the (32-bit) cell place where the result of SP# resides *and*
\ then TIB and then a couple of more bytes
Assuming a protected Forth, the last bytes are beyond the stack, and lead to a segmentation fault.
So you must readjust your thinking to low level if you want to use Forth. Think of memory as stack of letterboxes and start from there.
I'm trying to run the following and I receive an error saying that ERROR: The SAS System stopped processing this step because of insufficient memory.
The dataset has about 1170(row)*90(column) records. What are my alternatives here?
The error infor. is below:
332 proc assoc data=want1 dmdbcat=dbcat pctsup=0.5 out=frequentItems;
333 id tid;
334 target item_new;
335 run;
----- Potential 1 item sets = 188 -----
Counting items, records read: 19082
Number of customers: 203
Support level for item sets: 1
Maximum count for a set: 136
Sets meeting support level: 188
Megs of memory used: 0.51
----- Potential 2 item sets = 17578 -----
Counting items, records read: 19082
Maximum count for a set: 119
Sets meeting support level: 17484
Megs of memory used: 1.54
----- Potential 3 item sets = 1072352 -----
Counting items, records read: 19082
Maximum count for a set: 111
Sets meeting support level: 1072016
Megs of memory used: 70.14
Error: Out of memory. Memory used=2111.5 meg.
Item Set 4 is null.
ERROR: The SAS System stopped processing this step because of insufficient memory.
WARNING: The data set WORK.FREQUENTITEMS may be incomplete. When this step was stopped there were
1089689 observations and 8 variables.
From the documentation (http://support.sas.com/documentation/onlinedoc/miner/em43/assoc.pdf):
Caution: The theoretical potential number of item sets can grow very
quickly. For example, with 50 different items, you have 1225 potential
2-item sets and 19,600 3-item sets. With 5,000 items, you have over 12
million of the 2-item sets, and a correspondingly large number of
3-item sets.
Processing an extremely large number of sets could cause your system
to run out of disk and/or memory resources. However, by using a higher
support level, you can reduce the item sets to a more manageable
number.
So - provide a support= option make sure it's sufficiently high, e.g.:
proc assoc data=want1 dmdbcat=dbcat pctsup=0.5 out=frequentItems support=20;
id tid;
target item_new;
run;
Is there a way to frame the data mining task so that it requires less memory for storage or operations? In other words, do you need all 90 columns or can you eliminate some? Is there some clear division within the data set such that PROC ASSOC wouldn't be expected to use those rows for its findings?
You may very well be up against software memory allocation limits here.
For my program delphi-code-coverage-wizard, I need to verify that a (detailed) mapping file .map matches a Delphi application .exe
Of course, this verification should be realized with Delphi.
Is there a way to check it ? Maybe by verifying some information from the EXE ?
I think a quite simple heuristic would be to check that the various sections in the PE file start and finish at the same place:
For example, here's the top of a map file.
Start Length Name Class
0001:00401000 000A4938H .text CODE
0002:004A6000 00000C9CH .itext ICODE
0003:004A7000 000022B8H .data DATA
0004:004AA000 000052ACH .bss BSS
0005:00000000 0000003CH .tls TLS
I also looked at what dumpbin /headers had to say about these sections:
SECTION HEADER #1
.text name
A4938 virtual size
1000 virtual address (00401000 to 004A5937)
A4A00 size of raw data
400 file pointer to raw data (00000400 to 000A4DFF)
0 file pointer to relocation table
0 file pointer to line numbers
0 number of relocations
0 number of line numbers
60000020 flags
Code
Execute Read
SECTION HEADER #2
.itext name
C9C virtual size
A6000 virtual address (004A6000 to 004A6C9B)
E00 size of raw data
A4E00 file pointer to raw data (000A4E00 to 000A5BFF)
0 file pointer to relocation table
0 file pointer to line numbers
0 number of relocations
0 number of line numbers
60000020 flags
Code
Execute Read
...truncated
Look at the .text section. According to dumpbin it starts at 00401000 and finishes at 004A5937 which is a length of 000A4938, exactly as in the .map file. Naturally you'd read the PE file directly rather than running dumpbin, but this illustrates the point.
I'd expect a vanishingly small number of false positives with this approach.
Segments of memory - BSS, Stack, Heap, Data, Code/Text (Are there any more?).
Say I have a 128MB RAM, Can someone tell me:
How much memory is allocated for each of these memory segments?
Where do they start? Please specify the address range or something like that for better clarity.
What factors influence which should start where?
That question depends on the number of variables used. Since you did not specify what compiler or language or even operating system, that is a difficult one to pin down on! It all rests with the operating system who is responsible for the memory management of the applications. In short, there is no definite answer to this question, think about this, the compiler/linker at runtime, requests the operating system to allocate a block of memory, that allocation is dependent on how many variables there are, how big are they, the scope and usage of the variables. For instance, this simple C program, in a file called simpletest.c:
#include <stdio.h>
int main(int argc, char **argv){
int num = 42;
printf("The number is %d!\n", num);
return 0;
}
Supposing the environment was Unix/Linux based and was compiled like this:
gcc -o simpletest simpletest.c
If you were to issue a objdump or nm on the binary image simpletest, you will see the sections of the executable, in this instance, 'bss', 'text'. Make note of the sizes of these sections, now add a int var[100]; to the above code, recompile and reissue the objdump or nm, you will find that the data section has appeared - why? because we added a variable of an array type of int, with 100 elements.
This simple exercise will prove that the sections grows, and hence the binary gets bigger, and it will also prove that you cannot pre-determine how much memory will be allocated as the runtime implementation varies from compiler to compiler and from operating system to operating system.
In short, the OS calls the shot on the memory management!
you can get all this information compiling your program
# gcc -o hello hello.c // you might compile with -static for simplicity
and then readelf:
# readelf -l hello
Elf file type is EXEC (Executable file)
Entry point 0x80480e0
There are 3 program headers, starting at offset 52
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x000000 0x08048000 0x08048000 0x55dac 0x55dac R E 0x1000
LOAD 0x055dc0 0x0809edc0 0x0809edc0 0x01df4 0x03240 RW 0x1000
NOTE 0x000094 0x08048094 0x08048094 0x00020 0x00020 R 0x4
Section to Segment mapping:
Segment Sections...
00 .init .text .fini .rodata __libc_atexit __libc_subfreeres .note.ABI-tag
01 .data .eh_frame .got .bss
02 .note.ABI-tag
The output shows the overall structure of hello. The first program header corresponds to the process' code segment, which will be loaded from file at offset 0x000000 into a memory region that will be mapped into the process' address space at address 0x08048000. The code segment will be 0x55dac bytes large and must be page-aligned (0x1000). This segment will comprise the .text and .rodata ELF segments discussed earlier, plus additional segments generated during the linking procedure. As expected, it's flagged read-only (R) and executable (X), but not writable (W).
The second program header corresponds to the process' data segment. Loading this segment follows the same steps mentioned above. However, note that the segment size is 0x01df4 on file and 0x03240 in memory. This is due to the .bss section, which is to be zeroed and therefore doesn't need to be present in the file. The data segment will also be page-aligned (0x1000) and will contain the .data and .bss ELF segments. It will be flagged readable and writable (RW). The third program header results from the linking procedure and is irrelevant for this discussion.
If you have a proc file system, you can check this, as long as you get "Hello World" to run long enough (hint: gdb), with the following command:
# cat /proc/`ps -C hello -o pid=`/maps
08048000-0809e000 r-xp 00000000 03:06 479202 .../hello
0809e000-080a1000 rw-p 00055000 03:06 479202 .../hello
080a1000-080a3000 rwxp 00000000 00:00 0
bffff000-c0000000 rwxp 00000000 00:00 0
The first mapped region is the process' code segment, the second and third build up the data segment (data + bss + heap), and the fourth, which has no correspondence in the ELF file, is the stack. Additional information about the running hello process can be obtained with GNU time, ps, and /proc/pid/stat.
example taken from:
http://www.lisha.ufsc.br/teaching/os/exercise/hello.html
memory depend on the global variable and local variable