What is the relation between render passes, command buffers and clearing of attachments in vulkan? - renderer

I am currently learning vulkan and came across above question that I can't seem to answer by reading the spec.
When looking at the spec and code samples, it's always the same simplified workflow:
begin commandbuffer
begin renderpass
bind stuff (pipeline, buffers, descriptor sets)
draw
end renderpass
end commandbuffer
create_submit_info
submit_to_graphics_queue
Now the first thing that's a little unclear to me is when to clear the attachments. If I create my attachments with LOAD_OP_CLEAR then I have to supply clear values for VkRenderPassBeginInfo, but every command buffer contains a vkBeginRenderpass.
So does that mean that every submitted command buffer starts a new renderpass and so clears the attachments? That does not sound right.
If I specify LOAD_OP_DONT_CARE then I have to use vkCmdClear which kind of asks for a seperate command buffer just for clearing the attachments. Which can't be right either.
So can someone please clearify the relation between command buffers and render passes for me?
Or is it a misunderstanding of vkCmdBeginRenderPass? Does it not actually begin a new render pass when there is one running already?!
Thank you for helping.

Command buffers are responsible for storing commands that later get submitted to queue(s) and processed by the hardware. It is the only way to perform operations in Vulkan - record them and then submit. But the important thing is that each command buffer is totally independent from all other command buffers. If You want to perform a specific job, You need to record all the necessary commands that set appropriate state into the command buffer. If You want to perform similar job in another command buffer, You need to record the same set of commands in this other command buffer because there is no state sharing between command buffers. They are all independent. (There are some exceptions but they are not relevant for this discussion as they involve only secondary command buffers).
Next, in Vulkan rendering can only occur inside render passes. Render pass is a general definition of steps (called subpasses) drawing commands are divided into and of rendering resources (and relationships between them) needed for these drawing commands (attachments). But this is just description, metadata. You define how these attachments are used (as color attachments, as depth attachment, as input attachments) and what are their layouts in each subpass. You also define what to do with each attachment before the render pass (load op) and after the render pass (store op).
Now the actual resources used for these attachments are defined through framebuffers. This way You can perform similar rendering operations on various sets of images (by using various but compatible framebuffers) without the need to recreate render passes. And which framebuffer should be used during rendering is defined when You start a render pass.
Now when we combine the above information, render pass and command buffer, we get this: each command buffer is independent, so rendering must start and end during a single command buffer (excluding secondary command buffers). So that also means that You need to start and end a render pass within a single command buffer (but of course You can also start and end multiple render passes in a single command buffer).
So does that mean that every submitted command buffer starts a new
renderpass and so clears the attachments?
If You want a command buffer that only performs math calculations by executing compute shaders, then You don't need render passes. So such command buffer doesn't need to start and end any render pass and it doesn't need to clear any attachments. But if You want to render in a command buffer, then yes, each such command buffer must start (and end) a render pass. If rendering involves clearing, then each command buffer that needs to perform such operation will also need to clear attachments.
Or is it a misunderstanding of vkCmdBeginRenderPass? Does it not
actually begin a new render pass when there is one running already?!
vkCmdBeginRenderPass() starts a new render pass, yes, but in a single command buffer You cannot start another render pass until previous render pass is ended. You must explicitly start a render pass and end it. Only after that You can start another render pass (in the same command buffer).
As for clears, they are also defined during beginning of a render pass, which allows You to clear attachments with different colors. This way You don't have to create a separate render pass every time You want to change "background" color.
And one more thing about attachment clearing: unless really necessary, use render pass clears (LOAD_OP_CLEAR) instead of explicit clears (vkCmdClear()) as they may hurt performance (render pass clears are recommended by most of the vendors as far as I know).
I hope this hope clarifies this topic.

To answer your questions:
Normaly you clear when the frame begins, before you render something.
beginRenderpass can only be called by the primary commandbuffer, secondary commandbuffer must not invoke this call.
Basically you are starting a renderpass instance in the primary commandbuffer(only commandbuffer that you can submit to the queue),
This is what you will be doing on each frame.
but you can clear one or more regions of color and depth/stencil attachments inside a render pass instance, by calling
vkCmdClearAttachments regardless whether it is LOAD_OP_CLEAR / LOAD_OP_DONT_CARE,
or if you want to do it out side the renderpass you use vkCmdClearColor/DepthStencilImage.
this can be called by either the primary command buffer or secondary commandbuffer.
Tip: using LOAD_OP_DONT_CARE may optimized on some drivers, if you are sure that you will overwrite the entire the screen written by the previous frame.
so the drivers don't have to load/copy the memory from the presenation buffer for the current renderpass to clear it.
you can use vkCmdClearAttachment command in the secondary comandbuffer to clear any attachment.
But you cannot submit them by itself, it has to be put inside the primary commandbuffer.
So does that mean that every submitted command buffer starts a new renderpass and so clears the attachments?
is it a misunderstanding of vkCmdBeginRenderPass?
yes every submit commandbuffer starts the renderpass and clear the fbo attachments.
Does it not actually begin a new render pass when there is one running already?
Renderpass contains the executation order of the framebuffer. it has states.
the states are reused on each frame. you can use another renderpass (different state) with the same fbo.
so the first renderpass can clear it and the second renderpass don't clear at the beginning of the frame.
you cannot call another renderpass inside renderpass instance.
what you see below is invalid
Commandbuffer.beginRenderpass(renderpass1, fbo, ClearValues);
Commandbuffer.beginRenderpass(renderpass2, fbo, ClearValues);// ERROR
Commandbuffer.end;
you have to end the renderpass1 instance before you begin the second renderpass.
should be like this
Commandbuffer.beginRenderpass(renderpass1, fbo, ClearValues);
// draw scene
Commandbuffer.end;// renderpass is ended
Commandbuffer.beginRenderpass(renderpass2, fbo, ClearValues);
// draw full screen quad
Commandbuffer.end;
Commandbuffer.beginRenderpass(renderpass3, fbo, ClearValues);
// draw full screen quad
Commandbuffer.end;
let say that in the above example the fbo has 3 attachment. we are using 3 renderpass here.
First: Render the scene in to the attachment 1 using the renderpass1.
Second: Read from the attachment 1 and do a Vertical Blur and write into attachment 2 using renderpass2.
third: Read from the attachment 2 and do a Horinzontal blur and write in to the swapchain image using renderpass3.
(Note: for this particular techniqueue we cannnot use multiple subpass thats why I'm using 3 renderpasses for the same fbo.)

Related

Apply Effect to one or group of buses in a render callback

I have a render callback with 20 buses routed to a mixer and need to apply an effect to only some of these buses.
Do I need different callbacks for this or I can get it with the same render ?
There are a few ways to do this.
You can manually add effects in-between your mixer and your callbacks. So instead of setting the callback on the mixer input directly, you would create an effect unit for each bus that needs it, connect the effect to the mixer input, then set the input callback on the effect. (Most straightforward)
Example:
AUGraphConnectNodeInput(inGraph, sourceEffect, inSourceOutputNumber, mixerNode, inDestInputNumber);
AUGraphSetNodeInputCallback (inGraph, sourceEffect,0,&inputCallbackStruct);
You can keep a reference to each effect, then within your callback manually call AudioUnitRender on the effect. (Kind of complicated)
If you wish to use a single effect for this processing across multiple busses, you could use two mixers. All the busses that need effect would be routed to the "Effect Mixer", then the effect can be on the output of the "Effect Mixer", and the output of the effect can go to your main mixer. (Most efficient)

Python-fu/gimpfu parameters - What does "image" mean?

I've been trying to switch from script-fu to python-fu, and I can't figure out what I'm supposed to pass for the "image" parameter. In script-fu it was just an integer, but when I put the same integer into python-fu, it just says it's the wrong type. Same with strings and floats... what am I supposed to put in here? The documentation just says the parameter it takes is "IMAGE" but what does that mean? Where do I find it? How do I get it?
pdb.gimp_image_get_layers(image)
Here's a picture of the interpreter.
You pass an Image object -
Most of the parameters that are only integers in script-fu are not to be used - in their place, you have to pass an actual object reference. ANd how do you get those?
Simply all PDB functions and methods return these objects already - not their numeric IDs, like it happens in script-fu.
So, for Image, you either get the Image as parameter for your python-fu function, or on a call to pdb.gimp_image_new.
If, as it is often the case, you are making tests on the interactive console, you have to get references to the active images. In that case, call
gimp.image_list() to get a Python list with the currently open images - the image at index 0 on this list is the rightmost (newest) open image on the screen -so just do image = gimp.image_list()[0] to get a reference to it.
While you are at it, explore the image object with dir(image) you willfind it is populated with attributes and methods that are handy shortcuts to otherwise hard-to-type pdb calls. For example, image.layers gives you a Python list with reference to all layers on the image (as actual Layer objects, not their IDs), image.width gives you the width, and calling img.new_layer() creates a new layer and adds it to the image - this call has optional parameters to specify its name, width, height and so on.
As a last resort, there are reserved methods on the gimp module that can convert a numeric ID to the actual objects: '_id2display', '_id2drawable', '_id2image', '_id2vectors' - you should not need to use those. If you ever find your self needing to use those in the body of a script, due to some pdb function returning IDs instead of objects, please fill a bug report on GIMP's bugzilla.

Get Direct3D device from Direct2D render target

I'm using Direct2D to render my user interface.
What I would like is to be more easily able to profile my ui rendering (since I'm using several panels using Graphics debugger is a bit cumbersome).
Since I know that Direct2D uses a Direct3D device (exactly d3d11 device using 10_0 feature level) under the hood, I'd like to know if it is possible to retrieve either a ID310Device or ID3D11Device instance from ID2D1RenderTarget or ID2D1Factory object.
In that case I would easily be able to attach a timestamp query on the BeginDraw/EndDraw calls.
I tried several QueryInterface calls, but none of them have been sucessful so far.
An interesting undocumented secret is that any ID2D1RenderTarget you get from ID2D1Factory will also be an ID2D1DeviceContext (it seems to be intentional from what I've gathered, just accidentally undocumented?). Just call IUnknown::QueryInterface() to snag it. From there you can toy around with methods like GetDevice() and GetTarget(). If you can get the target then you may be able to weasel your way to obtaining the IDXGISurface which supports IDXGIDeviceSubObject::GetDevice() https://msdn.microsoft.com/en-us/library/windows/desktop/bb174529(v=vs.85).aspx (I haven't verified this part)
And in Win10 it looks like ID2D1Device2 gives you precisely what you want: GetDxgiDevice() https://msdn.microsoft.com/en-us/library/windows/desktop/dn917489(v=vs.85).aspx . So in that case, your ID2D1RenderTarget is cast to an ID2D1DeviceContext via IUnknown::QueryInterface(), and then you get an ID2D1Device via ID2D1DeviceContext::GetDevice() and then cast it to an ID2D1Device2 via another call to IUnknown::QueryInterface().

How to get the Process Environment Block (PEB) from extern process?

I want to get the PEB from the "notepad.exe" process. Does someone know how to do it?
I tried the GetModuleHandle API, but it doesn't return a valid pointer (it return zero every time) because I have to be the caller process of that module.
For that reason, I want to know how to get it to work with EnumProcessModules or CreateToolhelp32Snapshot.
Matt Pietrek described how to do that in a 1994 Under the Hood column. It was about how to get the environment variables of another process, where the first step is to get a pointer to the PEB. To do that, he says, call NtQueryInformationProcess. The PROCESS_BASIC_INFORMATION structure it fills contains the base address of the PEB structure. (You'll need to use ReadProcessMemory to read it since the address will be in the context of the external process's address space, not yours.)
To call NtQueryInformationProcess, you'll need a handle to the process. If you started the process yourself (by calling CreateProcess), then you already have a handle. Otherwise, you'll need to find the process ID and then call OpenProcess. To get the process ID, search for the process you want with EnumProcesses or Process32First/Process32Next. (I prefer the latter because it provides more information with less work.)

cvRetrieveFrame intricacies - openCV

The Documentation of OpenCV mentions that "the returned image (by cvRetrieveFrame) should not be released or modified by the user" ...
Link: http://opencv.willowgarage.com/documentation/c/highgui_reading_and_writing_images_and_video.html#retrieveframe
I am trying to debug my code, which involves the following steps:
Retrieve frame from video using cvRetrieveFrame()
Do some processing on the frame
output results
My instinct says that something is wrong with cvRetrieveFrame() because if I manually input frames using cvLoadImage, the program works fine. But I am not getting same results while using cvRetrieveFrame().
Since the documentation mentions such a restriction, any reason for such a restriction ? And, any alternatives ?
Have a great day
Before you call this function, you should have used another function which is cvGrabFrame() in order to be able to use the mentioned function, which you can use it for doing any necessary processing on the frame (such as the decompression stage in
the codec) and then return an IplImage* pointer that points to another internal buff er
(so do not rely on this image, because it will be overwritten the next time you call
cvGrabFrame()).

Resources