I am trying to create a Gtk Widget that you can pass an OpenCV image to that will then show it. I have created a class that is inherited from Gtk.Image that is used to show the image. You pass the OpenCV image to this class using the show_frame method, which then updates the Gtk.Image so it shows that image.
I have tested this and it works fine, i.e the image is correctly shown and updated when the show_frame method is called. However every time the image is updated, the memory used increases, until there is not enough memory and the program crashes.
I believe this is due to the memory that image is not being freed correctly. I cannot however work out how to fix this. I have tried unreferencing the gbytes once a new frame is received but this does not help. The memory only builds up when the set_from_pixbuf function is called. If this is commented out the memory usage stays at a constant level.
class OpenCVImageViewer(Gtk.Image):
def __init__(self):
Gtk.Image.__init__(self)
def show_frame(self, frame):
# Convert to opencv BGR to Gtk RGB
rgb_image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
# Get details about frame in order to set up pixbuffer
height = rgb_image.shape[0]
width = rgb_image.shape[1]
nChannels = rgb_image.shape[2]
gbytes = GLib.Bytes.new(rgb_image.tostring())
pixbuf = GdkPixbuf.Pixbuf.new_from_bytes(gbytes, GdkPixbuf.Colorspace.RGB, False,
8, width, height, width*nChannels)
# Add Gtk to main thread loop for thread safety
GLib.idle_add(self.set_from_pixbuf, pixbuf)
GLib.idle_add(self.queue_draw)
Well,
I found a solution, but I do not understand why it works: Set the image with a copy of the pixbuffer.
imageWidget.set_from_pixbuf(pixbuffer.copy())
I came to this solution after observing that the memory leak disapeared for scaled pixbuffers (i.e. result of pixbuffer.scale_simple).
Excerpt from the PyGTK FAQ, section 5.17:
There is a reference cycle between the python wrapper and its underlying C object; this means that the object will not be automatically deallocated when there are no more user references, and you will need the garbage collector to kick in (which may take a few cycles). This occasionally causes the odd problem, such as with pixbufs described in FAQ 8.4
And from section 8.4:
The answer is "Interesting GC behaviour" in Python. Apparently finalizers are not necessarily called as soon as an object goes out of scope. My guess is that the python memory manager doesn't directly know about the storage allocated for the image buffer (since it's allocated by the gdk) and that it therefore doesn't know how fast memory is being consumed.
The solution is to call gc.collect() at some appropriate place.
For example, I had some code that looked like this:
for image_path in images:
pb = gtk.gdk.pixbuf_new_from_file(image_path)
pb = pb.scale_simple(thumb_width, thumb_height, gtk.gdk.INTERP_BILINEAR)
thumb_list_model.set_value(thumb_list_model.append(None), 0, pb)
This chewed up an unacceptably large amount of memory for any reasonable image set. Changing the code to look like this fixed the problem:
import gc
for image_path in images:
pb = gtk.gdk.pixbuf_new_from_file(image_path)
pb = pb.scale_simple(thumb_width, thumb_height, gtk.gdk.INTERP_BILINEAR)
thumb_list_model.set_value(thumb_list_model.append(None), 0, pb)
del pb
gc.collect()
I am not exactly sure where you should call the garbage collector in your code (since I don't really know that much Python), but I believe this is the way to solve it.
Related
I hope this question isn't too open-ended. I ran into a memory issue with Rust, where I got an "out of memory" from calling next on an Iterator trait object. I'm unsure how to debug it. Prints have only brought me to the point where the failure occurs. I'm not very familiar with other tools such as ltrace, so although I could create a trace (231MiB, pff), I didn't really know what to do with it. Is a trace like that useful? Would I do better to grab gdb/lldb? Or Valgrind?
In general I would try to do the following approach:
Boilerplate reduction: Try to narrow down the problem of the OOM, so that you don't have too much additional code around. In other words: the quicker your program crashes, the better. Sometimes it is also possible to rip out a specific piece of code and put it into an extra binary, just for the investigation.
Problem size reduction: Lower the problem from OOM to a simple "too much memory" so that you can actually tell the some part wastes something but that it does not lead to an OOM. If it is too hard to tell wether you see the issue or not, you can lower the memory limit. On Linux, this can be done using ulimit:
ulimit -Sv 500000 # that's 500MB
./path/to/exe --foo
Information gathering: If you problem is small enough, you are ready to collect information which has a lower noise level. There are multiple ways which you can try. Just remember to compile your program with debug symbols. Also it might be an advantage to turn off optimization since this usually leads to information loss. Both can be archived by NOT using the --release flag during compilation.
Heap profiling: One way is too use gperftools:
LD_PRELOAD="/usr/lib/libtcmalloc.so" HEAPPROFILE=/tmp/profile ./path/to/exe --foo
pprof --gv ./path/to/exe /tmp/profile/profile.0100.heap
This shows you a graph which symbolizes which parts of your program eat which amount of memory. See official docs for more details.
rr: Sometimes it's very hard to figure out what is actually happening, especially after you created a profile. Assuming you did a good job in step 2, you can use rr:
rr record ./path/to/exe --foo
rr replay
This will spawn a GDB with superpowers. The difference to a normal debug session is that you can not only continue but also reverse-continue. Basically your program is executed from a recording where you can jump back and forth as you want. This wiki page provides you some additional examples. One thing to point out is that rr only seems to work with GDB.
Good old debugging: Sometimes you get traces and recordings that are still way too large. In that case you can (in combination with the ulimit trick) just use GDB and wait until the program crashes:
gdb --args ./path/to/exe --foo
You now should get a normal debugging session where you can examine what the current state of the program was. GDB can also be launched with coredumps. The general problem with that approach is that you cannot go back in time and you cannot continue with execution. So you only see the current state including all stack frames and variables. Here you could also use LLDB if you want.
(Potential) fix + repeat: After you have a glue what might go wrong you can try to change your code. Then try again. If it's still not working, go back to step 3 and try again.
Valgrind and other tools work fine, and should work out of the box as of Rust 1.32. Earlier versions of Rust require changing the global allocator from jemalloc to the system's allocator so that Valgrind and friends know how to monitor memory allocations.
In this answer, I use the macOS developer tool Instruments, as I'm on macOS, but Valgrind / Massif / Cachegrind work similarly.
Example: An infinite loop
Here's a program that "leaks" memory by pushing 1MiB Strings into a Vec and never freeing it:
use std::{thread, time::Duration};
fn main() {
let mut held_forever = Vec::new();
loop {
held_forever.push("x".repeat(1024 * 1024));
println!("Allocated another");
thread::sleep(Duration::from_secs(3));
}
}
You can see memory growth over time, as well as the exact stack trace that allocated the memory:
Example: Cycles in reference counts
Here's an example of leaking memory by creating an infinite reference cycle:
use std::{cell::RefCell, rc::Rc};
struct Leaked {
data: String,
me: RefCell<Option<Rc<Leaked>>>,
}
fn main() {
let data = "x".repeat(5 * 1024 * 1024);
let leaked = Rc::new(Leaked {
data,
me: RefCell::new(None),
});
let me = leaked.clone();
*leaked.me.borrow_mut() = Some(me);
}
See also:
Why does Valgrind not detect a memory leak in a Rust program using nightly 1.29.0?
Handling memory leak in cyclic graphs using RefCell and Rc
Minimal `Rc` Dependency Cycle
In general, to debug, you can use either a log-based approach (either by inserting the logs yourself, or having a tool such a ltrace, ptrace, ... to generate the logs for you) or you can use a debugger.
Note that ltrace, ptrace or debugger-based approaches require that you be able to reproduce the problem; I tend to favor manual logs because I work in an industry where bug reports are generally too imprecise to allow immediate reproduction (and thus we use logs to create the reproducer scenario).
Rust supports both approaches, and the standard toolset that one uses for C or C++ programs works well for it.
My personal approach is to have some logging in place to quickly narrow down where the issue occurs, and if logging is insufficient to fire up a debugger for a more fine-combed inspection. In this case I would recommend going straight away for the debugger.
A panic is generated, which means that by breaking on the call to the panic hook, you get to see both the call stack and memory state at the moment where things go awry.
Launch your program with the debugger, set a break point on the panic hook, run the program, profit.
all
My program maybe have a memory issue, so I try to find information about memory usage provided by various tools. In order to find the cause, I do simple experiments as well.
In release mode, I add the following code,
pChar = new char[((1<<30)/2)];
for(int i; i < ((1<<30)/2); i++)
{
pChar[i] = i % 256;
}
When the code is executed, the available physical memory in Windows task manager doesn't change. In my view, the compiler may remove the code to boost performance. I declare the variable as one global variable. It doesn't work. But in debug mode, the available physical memory in Windows task manager changes as expected. I can't understand that.
I have another question. Will the new operation allocate memory from virtual memory if the physical memory runs out. Or one exception will be thrown?
It's indeed quite possible that the compiler detects a "write-only" variable. Since it's non-volatile, the writes can be safely eliminated, and then there's no need for the OS to actually allocate RAM.
new just allocates address space, on modern systems. Physical RAM is allocated when needed. Typically this happens when the ctor runs, as it initializes the members. But in new char there's of course no ctor.
I'm developing an application with adobe air 3 for ios and having low memory errors frequently.
After ios 5 update os started to kill my app after some low memory warnings.
But the thing is profiler says app uses 4 to 9 megs of memory.
There are a lot of bitmap copy operations around and sometimes instantiates new bitmaps from embedded bitmaps.
I highly optimized everything and look for leaks etc.
I watch profiler for memory status and seems like GC clears everything. everything looks perfect but app continues to get low memory errors and gets killed by os.
Is there anything wrong with this code below. Because my assumption is this ClassReference never gets off from memory even the profiles says memory is cleared.
I used clone method to pass value instead of pass by ref. so I guess GC can collect that local variable. I tried with and without clone nothing changes.
If the code below runs 10-15 times with different tile Id's app crashes but with same ID's it continues working.
Is there anyone who is familiar with this kind of thing?
tmp is bitmapData
if (isMoving)
{
tmp=getProxyImage(x,y); //low resolution tile image
}
else
{
strTmp="main_TILE"+getTileID(x,y);
var ClassReference:Class = getDefinitionByName(strTmp) as Class; //full resolution tile image //something wrong here
tmp=new ClassReference().bitmapData.clone(); //something wrong here
ClassReference=null;
}
return tmp.clone();
Thanks for reading. I hope some one has a solution for this.
You are creating three copies of your bitmapdata with this. They will likely get garbage collected eventually, but you probably run out of memory before that happens.
(Here I assume you have embedded your bitmapdata using the [Embed] tag)
tmp = new ClassReference()
// allocates no new memory, class reference already exists
var ClassReference:Class = getDefinitionByName(strTmp) as Class;
// creates a new BitmapAsset from the class reference including it's BitmapData.
// then you clone this bitmapdata, giving you two
tmp = new ClassReference().bitmapData.clone();
// not really necessary since ClassReference goes out of scope anyway, but no harm done
ClassReference=null;
// Makes a third copy of your second copy and returns it.
return tmp.clone();
I would recommend this (assuming you need unique bitmapDatas for each tile)
var ClassReference:Class = getDefinitionByName(strTmp) as Class;
return new ClassReference().bitmapData.clone();
If you don't need unique bitmapDatas, keep static properties with the bitmapDatas on some class and use the same ones all over. That will minimize memory usage.
Some time ago i posted a question related to a WriteableBitmap memory leak, and though I received wonderful tips related to the problem, I still think there is a serious bug / (mistake made by me) / (Confusion) / (some other thing) here.
So, here is my problem again:
Suppose we have a WPF application with an Image and a button. The image's source is a really big bitmap (3600 * 4800 px), when it's shown at runtime the applicaton consumes ~90 MB.
Now suppose i wish to instantiate a WriteableBitmap from the source of the image (the really big Image), when this happens the applications consumes ~220 MB.
Now comes the tricky part, when the modifications to the image (through the WriteableBitmap) end, and all the references to the WriteableBitmap (at least those that I'm aware of) are destroyed (at the end of a method or by setting them to null) the memory used by the writeableBitmap should be freed and the application consumption should return to ~90 MB. The problem is that sometimes it returns, sometimes it does not.
Here is a sample code:
// The Image's source whas set previous to this event
private void buttonTest_Click(object sender, RoutedEventArgs e)
{
if (image.Source != null)
{
WriteableBitmap bitmap = new WriteableBitmap((BitmapSource)image.Source);
bitmap.Lock();
bitmap.Unlock();
//image.Source = null;
bitmap = null;
}
}
As you can see the reference is local and the memory should be released at the end of the method (Or when the Garbage collector decides to do so). However, the app could consume ~224 MB until the end of the universe.
Any help would be great.
Is it necessary to render the Bitmap image at the same resolution and pixels? You could create the writeablebitmap at a much lower set of pixels and call the render method. Since the writeablebitmap carries a reference to the original uielements when calling render, in this case, you are going to have 3 chunks: 1) original uielement, 2) pixels in writeablebitmap, 3) reference to copied original.
I had a similar issue with the writeablebitmap in terms of memory leaks and I fixed it from checking out this link:
http://www.wintellect.com/CS/blogs/jprosise/archive/2009/12/17/silverlight-s-big-image-problem-and-what-you-can-do-about-it.aspx
If you create another writeablebitmap and copy the pixels over, then dispose of the first writeablebitmap you should see some memory release - at least I did in my scenario.
I'm not so new to the concepts present on J2ME, but I'm sort of lazy in ways I shouldn't:
Lately my app has been loading images into memory as they were candy...
Sprite example = new Sprite(Image.createImage("/images/example.png"), w, h);
and I'm not really sure it's the best way, but it worked fine in my Motorola Z6, until last night, when I tested the app in a old Samsung cellphone and the images wont even load and require several attemps of starting the thread to show up. The screen was left on white, so I realized that it has to be something about Image loading that I'm not doing quite well... Is there anyone who can tell me how to properly make a loading routine in my app?.
I'm not sure exactly what you are looking for, but the behavior you describe very much sounds like you are experiencing an OutOfMemory exception. Try reducing the dimensions of your images (heap usage is based on dimension) and see if the behavior ceases. This will let you know if it is truly an OutOfMemory issue or something else.
Other tips:
Load images largest to smallest. This helps with heap fragmentation and allows the largest heap space for the largest images.
Unload (set to null) in reverse order of how you loaded and garbage collect after doing so. Make sure to Thread.yield() after you call the GC.
Make sure you only load the images that you need. Unload images from a state that the application is no longer in.
Since you are creating sprites you may have multiple sprites for one image. Consider creating an image pool to make sure you only load the image once. Then just point each Sprite object to the image within the pool that it requires. Your example in your question seems like you would more than likely load the same image into memory more than once. That's wasteful and could be part of the OutOfMemory issue.
Using a film image(a set of images by a defined dimension in one image) and use logic to pull them out one at a time.
Because they a grouped into one image, you are saving header space per image and thus can reduce the memory used.
This techniques was first used in MIDP 1.0 memory constrained devices.
Using the Fostah approach of not loading images over and over, I made the following class:
public class ImageLoader {
private static Hashtable pool = new Hashtable();
public static Image getSprite(String source){
if(pool.get(source) != null) return (Image) pool.get(source);
try {
Image temp = Image.createImage(source);
pool.put(source, temp);
return temp;
} catch (IOException e){
System.err.println("Error al cargar la imagen en "+source+": "+e.getMessage());
}
return null;
}
}
So, whenever I need an image I first ask the pool for it, or just load it into the pool.