I want to declare 4 large arrays of each can store 48000 float_32 constants.
I am using STM32F4, so i figured out ram isnt enough and i should use flash
instead. Upon my research, seems like everyone is talking about one should never use flash for write/read processes on runtime, go for eeprom instead. I am aware of the risks, but why do 1mb flash memory is in there if i should never use it? What is it used for?
On my real problem, i just want to be able to write and generate a sampling array and write it on flash for once the device is initialized, and never be touched again. It is safe this way, but I can't find any useful and easy to understand tutorials anywhere for writing on flash. Is it forbidden? Should we not use 1mb of huge space available on the chip? Please point me to a nice tutorial on writing/reading STM32F4 flash.
Related
In STM32F7, The code is running from the internal flash (default), we can read/write data from/to internal flash.
My problem is: I want to use external QSPI flash for my code execution (Memory mapped mode).
Also during this mode, I would like to use the same QSPI flash for my data storage (ie. saving some settings) while code is executing in QSPI flash. But this is not possible since ST states that in its reference document (AN4760):
In Memory-mapped mode the QUADSPI allows the access to the external
memory for read operation through the memory mapped address region
(from 0x9000 0000 to 0x9FFF FFFF) and allows the external memory to
be seen just like an internal memory.
Is there any solution to my problem (write data to QSPI flash) without exiting from memory mapped mode?
Is it possible to partition QSPI flash into two parts? One is used for memory mapped mode and the other used as just read/write data into it.
Note: I don't want to jump from external flash to internal flash for write data, then jump to external flash again for executing the code.
Any help would be appreciated.
Thanks.
No it is not possible. FLASH memory, if was written before, has to be erased, then you need to enter the wirte mode and write the memory. FLASH memory is always slow to write.
The memory mapped mode is usually used to run the code from the QSPI flash, or to simplify the the read access.
I know this is an older post but for future reference:
You need to stop executing out of external flash to write to it. Maybe by copying a small code block to RAM or executing from internal flash. Then jump to executing that code. That code could disable the external flash from memory mapped mode, write your data to it and then switch it back to memory mapped mode. Obviously during this time you would need to disable any relevant interrupts and make sure there was no accesses to memory mapped flash. Also take special care not to erase and overwrite your code in external flash, unless you want to of course!
This is a similar process to writing to internal flash when you cannot execute from it while writing to it.
You can write to flash by writing to the quadspi registers. See qspi_write_cmd_addr_data() in https://github.com/micropython/micropython/blob/master/ports/stm32/qspi.c
How does the iOS platform handle memory-mapped files during low-memory scenarios? By low-memory scenarios, I mean when the OS sends the UIApplicationDidReceiveMemoryWarningNotification notification to all observers in the application.
Our files are mapped into memory using +[NSData dataWithContentsOfMappedFile:], the documentation for which states:
A mapped file uses virtual memory techniques to avoid copying pages of the file into memory until they are actually needed.
Does this mean that the OS will also unmap the pages when they're no longer in use? Is it possible to mark pages as being no longer in use? This data is read-only, if that changes the scenario. How about if we were to use mmap() directly? Would this be preferable?
Memory-mapped files copy data from disk into memory a page at a time. Unused pages are free to be swapped out, the same as any other virtual memory, unless they have been wired into physical memory using mlock(2). Memory mapping leaves the determination of what to copy from disk to memory and when to the OS.
Dropping from the Foundation level to the BSD level to use mmap is unlikely to make much difference, beyond making code that has to interface with other Foundation code somewhat more awkward.
(This is not an answer, but it would be useful information.)
From #ID_AA_Carmack tweet,
#ID_AA_Carmack are iOS memory mapped files automatically unmapped in low memory conditions? (using +[NSData dataWithContentsOfMappedFile]?)
ID_AA_Carmack replied for this,
#KhrobEdmonds yes, that is one of the great benefits of using mapped files on iOS. I use mmap(), though.
I'm not sure that is true or not...
From my experiments NSData does not respond to memory warnings. I tested by creating a memory mapped NSData and accessing parts of the file so that it would be loaded into memory and finally sending memory warnings. There was no decrease in memory usage after the memory warning. Nothing in the documentation says that a memory will cause NSData to reduce real memory usage in low memory situations so it leads me to believe that it does not respond to memory warnings. For example NSCache documentation says that it will try and play nice with respect to memory usage plus I have been told it responds to the low memory warnings the system raises.
Also in my simple tests on an iPod Touch (4th gen) I was able to map about 600 megs of file data into virtual memory use +[NSData dataWithContentsOfMappedFile:]. Next I started to access pages via the bytes property on the NSData instance. As I did this real memory started to grow however it stopped growing at around 30 megs of real memory usage. So the way it is implemented it seems to cap how much real memory will be used.
In short if you want to reduce memory usage of NSData objects the best bet is to actually make sure they are completely released and not relying on anything the system automagically does on your behalf.
If iOS is like any other Unix -- and I would bet money it is in this regard -- pages in an mmap() region are not "swapped out"; they are simply dropped (if they are clean) or are written to the underlying file and then dropped (if they are dirty). This process is called "evicting" the page.
Since your memory map is read-only, the pages will always be clean.
The kernel will decide which pages to evict when physical memory gets tight.
You can give the kernel hints about which pages you would prefer it keep/evict using posix_madvise(). In particular, POSIX_MADV_DONTNEED tells the kernel to feel free to evict the pages; or as you say, "mark pages as being no longer in use".
It should be pretty simple to write some test programs to see whether iOS honors the "don't need" hint. Since it is derived from BSD, I bet it will.
Standard virtual memory techniques for file-backed memory says that the OS is free to throw away pages whenever it wants because it can always get them again later. I have not used iOS, but this has been the behavior of virtual memory on many other operating systems for a long time.
The simplest way to test it is to map several large files into memory, read through them to guarantee that it pages them into memory, and see if you can force a low memory situation. If you can't, then the OS must have unmapped the pages once it decided that they were no longer in use.
The dataWithContentsOfMappedFile: method is now deprecated from iOS5.
Use mmap, as you will avoid these situations.
I was reading an article on memory fragmentation when I recalled that there are several examples of software that claim to defragment memory. I got curious, how does it work? Does it work at all?
EDIT:
xappymah gave a good argument against memory defragmentation in that a process might be very surprised to learn that its memory layout suddenly changed. But as I see it there's still the possibility of the OS providing some sort of API for global memory control. It does seem a bit unlikely however since it would give rise to the possibility of using it in malicious intent, if badly designed. Does anyone know if there is an OS out there that supports something of the sort?
The real memory defragmentation on a process level is possible only in managed environments such as, for example, Java VMs when you have some kind of an access to objects allocated in memory and can manage them.
But if we are talking about the unmanaged applications then there is no possibility to control their memory with third-party tools because every process (both the tool and the application) runs in its own address space and doesn't have access to another's one, at least without help from OS.
However even if you get access to another process's memory (by hacking your OS or else) and start modifying it I think the target application would be very "surprised".
Just imagine, you allocated a chunk of memory, got it's starting address and on the next second this chunk of memory is moved somewhere else because of "VeryCoolMemoryDefragmenter" :)
In my opinion memory it's a kind of Flash Drive, and this chip don't get fragmented because there aren't turning disks pins recording and playing information, in a random way, like a lie detector. This is the way that Hard Disk Fragmentation it's done. That's why SSD drives are so fast, effective, reliable and maintenance free. SSD it's a BIG piece of memory and it kind of look alike.
When developing an OpenGL program, is there a way to poll from the system to find out just how many megabytes are available to store textures, etc?
Or is the standard approach these days just allocate memory and forget about everything?
Although the official stance remains "you don't need to know, you don't want to know, and it would not help you anyway", luckily at least two IHVs have shown a little more insight lately and offer extensions to query that information:
NVX_gpu_memory_info
ATI_meminfo
One nice thing about these extensions is that they have a least common denominator which is just what most people need, and you don't need to query extension support or do anything special, as they both work via glGetIntegerv.
In the easiest case, you can just initialize an array of 4 integers to zero (or some minimum default value that you'll assume in case the extensions don't work), then you call glGetIntegerv twice (with GPU_MEMORY_INFO_CURRENT_AVAILABLE_VIDMEM_NVX and TEXTURE_FREE_MEMORY_ATI, respectively), and finally call glGetError to clear the error state. glGetIntegerv does not modify the pointed-to memory if it fails, nor does it crash or any other bad thing -- it merely sets the error state to GL_INVALID_ENUM.
Both extensions return a value in the first array position, the ATI one returns some values in the other 3 too.
n.b.: glAreTexturesResident has not been supported for almost a decade on mainstream hardware, in the same manner as texture priorities. The common mantra is that the driver writer knows much better than you anyway.
OpenGL doesn't give you this information. And frankly: There's only little benefit, simply because today we have multitasking operating systems. The OpenGL driver is responsible for swapping in texture data to/from system memory, if there's demand for it.
What OpenGL can do for you, is tell, if the textures you've uploaded are still resident in fast memory. The function is called "glAreTexturesResident". You can use this to gradually upload stuff to the GPU until you've filled up the GPU's memory. But keep in mind that you're not the only user of the GPU.
I mostly work on C language for my work. I have faced many issues and spent lot time in debugging issues related to dynamically allocated memory corruption/overwriting. Like malloc(A) A bytes but use write more than A bytes. Towards that i was trying to read few things when i read about :-
1.) An approach wherein one allocates more memory than what is needed. And write some known value/pattern in that extra locations. Then during program execution that pattern should be untouched, else it indicated memory corruption/overwriting. But how does this approach work. Does it mean for every write to that pointer which is allocated using malloc() i should be doing a memory read of the additional sentinel pattern and read for its sanity? That would make my whole program very slow.
And to say that we can remove these checks from the release version of the code, is also not fruitful as memory related issues can happen more in 'real scenario'. So can we handle this?
2.) I heard that there is something called HEAP WALKER, which enables programs to detect memory related issues? How can one enable this.
thank you.
-AD.
If you're working under Linux or OSX, have a look at Valgrind (free, available on OSX via Macports). For Windows, we're using Rational PurifyPlus (needs a license).
You can also have a look at Dmalloc or even at Paul Nettle's memory manager which helps tracking memory allocation related bugs.
If you're on Mac OS X, there's an awesome library called libgmalloc. libgmalloc places each memory allocation on a separate page. Any memory access/write beyond the page will immediately trigger a bus error. Note however that running your program with libgmalloc will likely result in a significant slowdown.
Memory guards can catch some heap corruption. It is slower (especially deallocations) but it's just for debug purposes and your release build would not include this.
Heap walking is platform specific, but not necessarily too useful. The simplest check is simply to wrap your allocations and log them to a file with the LINE and FILE information for your debug mode, and most any leaks will be apparent very quickly when you exit the program and numbers don't tally up.
Search google for LINE and I am sure lots of results will show up.