Given an MTLTexture, defined as follows.
// Create device.
id<MTLDevice> dev = MTLCreateDefaultSystemDevice();
// Size of texture.
const unsigned int W = 640;
const unsigned int H = 480;
// Define texture.
MTLTextureDescriptor *desc = [[MTLTextureDescriptor alloc] init];
desc.pixelFormat = MTLPixelFormatBGRA8Unorm;
desc.width = W;
desc.height = H;
// Create texture.
id<MTLTexture> tex = [device newTextureWithDescriptor:desc];
It is my understanding that at this point I should have a texture as defined in desc allocated on device dev and accessible through tex.
Now, given another texture tex2 (known to be allocated and accessible) and a Metal compute kernel defined as follows.
kernel void foo(texture2d<float, access::read> in [[texture(0)]],
texture2d<float, access::write> out [[texture(1)]],
uint2 pix [[thread_position_in_grid]]) {
// Out of bounds check.
if (pix.x >= out.get_width() || pix.y >= out.get_height()) {
return;
}
// Do some processing on the input texture.
// ... All fine up to here.
// Write out a pixel to the output buffer.
const float4 p = abc; // abc is computed above.
out.write(p, pix);
}
It is my understanding that when the pixel p is written out to out, the values of p will be converted to conform to the pixel format of tex, in this case MTLPixelFormatBGRA8Unorm.
However, when launching the kernel as follows, the line in which p is written to out (above defined as tex) triggers a critical error (SIGABRT).
// Create a Metal library.
id<MTLLibrary> lib = [dev newDefaultLibrary];
// Load the kernel.
id<MTLFunction> kernel = [lib newFunctionWithName:#"foo"];
// Create a pipeline state.
id<MTLComputePipelineState> pipelineState = [dev newComputePipelineStateWithFunction:kernel error:NULL];
// Create a command queue.
id<MTLCommandQueue> cmdQueue = [dev newCommandQueue];
// Create command buffer.
id<MTLCommandBuffer> cmdBuff = [cmdQueue commandBuffer];
// Create compute encoder.
id<MTLComputeCommandEncoder> enc = [cmdBuff computeCommandEncoder];
// Set the pipeline state.
[enc setComputePipelineState:pipelineState];
// Set the input textures (tex2 is read only in the kernel, as above).
[enc setTexture:tex2 atIndex:0];
[enc setTexture:tex atIndex:1];
// 2D launch configuration.
const MTLSize groupDim = MTLSizeMake(16, 16, 1);
const MTLSize gridDim = MTLSizeMake((int)ceil((float)(W / (float)groupDim.width)),
(int)ceil((float)(H / (float)groupDim.height)),
1);
// Launch kernel.
[enc dispatchThreadgroups:gridDim threadsPerThreadgroup:groupDim];
[enc endEncoding];
[enc commit];
[cmdBuff waitUntilCompleted];
My question is that under the scenario outlined above, is my understanding of how one allocates a MTLTexture correct? Or, is the example above merely defining a wrapper around some texture that I need to separately allocate?
The above texture allocation and compute kernel launch are correct. Upon further digging in the documentation, the part that was missing was the usage property of MTLTextureDescriptor. In the documentation, the following is stated.
The default value for this property is MTLTextureUsageShaderRead.
As such, in the example given in the question, the following additional property assignment on MTLTextureDescriptor is required.
desc.usage = MTLTextureUsageShaderWrite;
Related
I’m working on a Metal, MTKView based app that takes advantage of the A11 TBDR architecture to do deferred shading in a single render pass. I used Apple’s Deferred Lighting sample code as reference, and it works great.
I’d like to try changing the geometry buffer pass to be GPU-driven, using the Indirect Command Buffer feature of Metal 2 on A11 hardware.
I’ve been using Apple’s Encoding Indirect Command Buffers on the GPU sample code as my main point of reference for this. I’m able to run this sample on my iPhone XR (although, probably off-topic, the scrolling is not smooth, it judders).
I’m running into difficulties however with my own code, when I try to move my geometry buffer pass into an indirect command buffer. When I set supportIndirectCommandBuffers to true on the MTLRenderPipelineDescriptor of the Geometry Buffer pipeline, device.makeRenderPipelineState fails with the error
AGXMetalA12 Code=3 "Fragment shader cannot be used with indirect command buffers"
I’ve not been able to find any information in the documentation on this error. I’m wondering, are there certain kinds of fragment operation that are not allowed in indirect pipelines, or some kind of limit to GPU-driven drawing that I've overlooked (the number of color attachments perhaps)?
SharedTypes.h
Header shared by Metal and Swift
#ifndef SharedTypes_h
#define SharedTypes_h
#ifdef __METAL_VERSION__
#define NS_CLOSED_ENUM(_type, _name) enum _name : _type _name; enum _name : _type
#define NSInteger metal::int32_t
#else
#import <Foundation/Foundation.h>
#endif
#include <simd/simd.h>
typedef struct {
uint32_t meshId;
matrix_float3x3 normalViewMatrix;
matrix_float4x4 modelMatrix;
matrix_float4x4 shadowMVPTransformMatrix;
} InstanceData;
typedef struct {
vector_float3 cameraPosition;
float voxelScale;
float blockScale;
vector_float3 lightDirection;
matrix_float4x4 viewMatrix;
matrix_float4x4 projectionMatrix;
matrix_float4x4 projectionMatrixInverse;
matrix_float4x4 shadowViewProjectionMatrix;
} VoxelUniforms;
typedef NS_CLOSED_ENUM(NSInteger, BufferIndex)
{
BufferIndexInstances = 0,
BufferIndexVertices = 1,
BufferIndexIndices = 2,
BufferIndexVoxelUniforms = 3,
};
typedef NS_CLOSED_ENUM(NSInteger, RenderTarget)
{
RenderTargetLighting = 0,
RenderTargetNormal_shadow = 1,
RenderTargetVoxelIndex = 2,
RenderTargetDepth = 3,
};
#endif /* SharedTypes_h */
GBuffer shader
#include <metal_stdlib>
using namespace metal;
#include "../SharedTypes.h"
struct VertexIn {
packed_half3 position;
packed_half3 texCoord3D;
half ambientOcclusion;
uchar normalIndex;
};
struct VertexInOut {
float4 position [[ position ]];
half3 worldPos;
half3 eyeNormal;
half3 localPosition;
half3 localNormal;
float eyeDepth;
float3 shadowCoord;
half3 texCoord3D;
};
vertex VertexInOut gBufferVertex(device InstanceData* instances [[ buffer( BufferIndexInstances ) ]],
device VertexIn* vertices [[ buffer( BufferIndexVertices ) ]],
constant VoxelUniforms &uniforms [[ buffer( BufferIndexVoxelUniforms ) ]],
uint vid [[ vertex_id ]],
ushort iid [[ instance_id ]])
{
InstanceData instance = instances[iid];
VertexIn vert = vertices[vid];
VertexInOut out;
float4 position = float4(float3(vert.position), 1);
float4 worldPos = instance.modelMatrix * position;
float4 eyePosition = uniforms.viewMatrix * worldPos;
out.position = uniforms.projectionMatrix * eyePosition;
out.worldPos = half3(worldPos.xyz);
out.eyeDepth = eyePosition.z;
half3 normal = normals[vert.normalIndex];
out.eyeNormal = half3(instance.normalViewMatrix * float3(normal));
out.shadowCoord = (instance.shadowMVPTransformMatrix * position).xyz;
out.localPosition = half3(vert.position);
out.localNormal = normal;
out.texCoord3D = half3(vert.texCoord3D);
return out;
}
fragment GBufferData gBufferFragment(VertexInOut in [[ stage_in ]],
constant VoxelUniforms &uniforms [[ buffer( BufferIndexVoxelUniforms ) ]],
texture3d<ushort, access::sample> voxelMap [[ texture(0) ]],
depth2d<float> shadowMap [[ texture(1) ]],
texture3d<half, access::sample> fogOfWarMap [[ texture(2) ]]
) {
// voxel index
half3 center = round(in.texCoord3D);
uchar voxIndex = voxelMap.read(ushort3(center)).r - 1;
// ambient occlusion
half3 neighborPos = center + in.localNormal;
half3 absNormal = abs(in.localNormal);
half2 texCoord2D = tc2d(in.localPosition / uniforms.voxelScale, absNormal);
half ao = getAO(voxelMap, neighborPos, absNormal.yzx, absNormal.zxy, texCoord2D);
// shadow
constexpr sampler shadowSampler(coord::normalized,
filter::linear,
mip_filter::none,
address::clamp_to_edge,
compare_func::less);
float shadow_sample = ambientLightingLevel;
for (short i = 0; i < shadowSampleCount; i++){
shadow_sample += shadowMap.sample_compare(shadowSampler, in.shadowCoord.xy + poissonDisk[i] * 0.002, in.shadowCoord.z - 0.0018) * shadowContributionPerSample;
}
shadow_sample = min(1.0, shadow_sample);
//fog-of-war
half fogOfWarSample = fogOfWarMap.sample(fogOfWarSampler, (float3(in.worldPos) / uniforms.blockScale) + float3(0.5, 0.4, 0.5)).r;
half notVisible = max(fogOfWarSample, 0.5h);
// output
GBufferData out;
out.normal_shadow = half4(in.eyeNormal, ao * half(shadow_sample) * notVisible);
out.voxelIndex = voxIndex;
out.depth = in.eyeDepth;
return out;
};
Pipeline setup
extension RenderTarget {
var pixelFormat: MTLPixelFormat {
switch self {
case .lighting: return .bgra8Unorm
case .normal_shadow: return .rgba8Snorm
case .voxelIndex: return .r8Uint
case .depth: return .r32Float
}
}
static var allCases: [RenderTarget] = [.lighting, .normal_shadow, .voxelIndex, .depth]
}
public final class GBufferRenderer {
private let renderPipelineState: MTLRenderPipelineState
weak var shadowMap: MTLTexture?
public init(depthPixelFormat: MTLPixelFormat, colorPixelFormat: MTLPixelFormat, sampleCount: Int = 1) throws {
let library = try LibraryMonad.getLibrary()
let device = library.device
let descriptor = MTLRenderPipelineDescriptor()
descriptor.vertexFunction = library.makeFunction(name: "gBufferVertex")!
descriptor.fragmentFunction = library.makeFunction(name: "gBufferFragment")!
descriptor.depthAttachmentPixelFormat = depthPixelFormat
descriptor.stencilAttachmentPixelFormat = depthPixelFormat
descriptor.sampleCount = sampleCount
for target in RenderTarget.allCases {
descriptor.colorAttachments[target.rawValue].pixelFormat = target.pixelFormat
}
// uncomment below to trigger throw
// descriptor.supportIndirectCommandBuffers = true
renderPipelineState = try device.makeRenderPipelineState(descriptor: descriptor) // throws "Fragment shader cannot be used with indirect command buffers"
}
public convenience init(mtkView: MTKView) throws {
try self.init(depthPixelFormat: mtkView.depthStencilPixelFormat, colorPixelFormat: mtkView.colorPixelFormat, sampleCount: mtkView.sampleCount)
}
}
The above works great when triggering draws from the CPU in the usual way, but when setting supportIndirectCommandBuffers in preparation for GPU drawing it throws the error.
I've tried stripping down the fragment shader to just return constant values for the GBuffers, and then makeRenderPipelineState succeeds, but when I add texture sampling back in it begins complaining again. I can't seem to pin down what exactly it doesn't like about the frag shader.
Looking through the code and through Metal documentation and Metal Shading Language specification, I think I know why you get this error.
If you look through render_command interface that is present in metal_command_buffer header in Metal, you'll find that to pass parameters to indirect render commands, you only have these functions: set_vertex_buffer and set_fragment_buffer, there is no set_vertex_texture or set_vertex_sampler like you have in MTLRenderCommandEncoder.
But, since your pipeline uses shader that in turn uses textures as arguments and you indicate by using supportIndirectCommandBuffers that you would like to use this pipeline in indirect commands, Metal has no choice but to fail pipeline creation.
Instead if you want to pass textures or samplers to indirect render commands, you should use argument buffers, that you will pass to the shader that issues indirect render commands, which in turn will bind them using set_vertex_buffer and set_fragment_buffer for each render_command.
Specification: Metal Shading Language Specification (Section 5.16)
I'm puzzled why this isn't working.
I'm trying to add texture data to each of the cube textures faces. For some reason, only the first(+x) works. The MSDN documentation is quite sparse, but it looks like this should do the trick:
// mip-level 0 data
// R8G8B8A8 texture
uint32_t sizeWidth = textureWidth * sizeof(uint8_t) * 4;
if (isCubeTexture)
{
for (uint32_t index = 0; index < gCubemapNumTextures; ++index)
{
const uint32_t subResourceID = D3D11CalcSubresource(0, index, 1);
context->UpdateSubresource(mTexture, subResourceID, NULL, &textureData.at(sizeWidth * textureHeight * index), sizeWidth, 0);
}
}
When debugging and looking at the faces its all just black except the first face, which seems to load fine. So obivously I am doing somerhing wrong, how do you properly upload cubetexture data to all the faces?
EDIT: follow parameters used to create the texture:
D3D11_TEXTURE2D_DESC textureDesc;
ZeroMemory(&textureDesc, sizeof(D3D11_TEXTURE2D_DESC));
textureDesc.Width = textureWidth;
textureDesc.Height = textureHeight;
textureDesc.ArraySize = isCubeTexture ? gCubemapNumTextures : 1;
if (isSRGB)
textureDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM_SRGB;
else
textureDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
textureDesc.SampleDesc.Count = 1;
textureDesc.Usage = D3D11_USAGE_DEFAULT;
textureDesc.BindFlags = D3D11_BIND_RENDER_TARGET | D3D11_BIND_SHADER_RESOURCE;
textureDesc.MiscFlags = D3D11_RESOURCE_MISC_GENERATE_MIPS;
if (isCubeTexture)
textureDesc.MiscFlags |= D3D11_RESOURCE_MISC_TEXTURECUBE;
DXCALL(device->CreateTexture2D(&textureDesc, NULL, &mTexture));
Then after uploading the data I generate mip chain like this:
context->GenerateMips(mShaderResourceView);
And again, it works fine but only for the first (+x) face.
You create the texture with "0" mip levels by virtue of zero'ing out the texture description. Zero means "full mip chain please", which means more than 1 mip (unless your texture is 1x1).
Your arguments to D3D11CalcSubresource has a third argument of '1', suggesting only one mip, which appears not to be true. Be sure to pass in the correct number of mips to this helper function or it won't calculate the correct subresource index.
You can get the mip count by calling GetDesc() after the texture has been created.
ok i tried everything at this point and I'm really lost....
ID3D11Texture2D* depthStencilTexture;
D3D11_TEXTURE2D_DESC depthTexDesc;
ZeroMemory (&depthTexDesc, sizeof(D3D11_TEXTURE2D_DESC));
depthTexDesc.Width = set->mapSettings["SCREEN_WIDTH"];
depthTexDesc.Height = set->mapSettings["SCREEN_HEIGHT"];
depthTexDesc.MipLevels = 1;
depthTexDesc.ArraySize = 1;
depthTexDesc.Format = DXGI_FORMAT_D32_FLOAT;
depthTexDesc.SampleDesc.Count = 1;
depthTexDesc.SampleDesc.Quality = 0;
depthTexDesc.Usage = D3D11_USAGE_DEFAULT;
depthTexDesc.BindFlags = D3D11_BIND_DEPTH_STENCIL;
depthTexDesc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE | D3D11_CPU_ACCESS_READ;
depthTexDesc.MiscFlags = 0;
mDevice->CreateTexture2D(&depthTexDesc, NULL, &depthStencilTexture);
D3D11_DEPTH_STENCIL_DESC dsDesc;
// Depth test parameters
dsDesc.DepthEnable = true;
dsDesc.DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ALL;
dsDesc.DepthFunc = D3D11_COMPARISON_LESS;//LESS
// Stencil test parameters
dsDesc.StencilEnable = false;
dsDesc.StencilReadMask = 0xFF;
dsDesc.StencilWriteMask = 0xFF;
// Stencil operations if pixel is front-facing
dsDesc.FrontFace.StencilFailOp = D3D11_STENCIL_OP_KEEP; //KEEP
dsDesc.FrontFace.StencilDepthFailOp = D3D11_STENCIL_OP_INCR; //INCR
dsDesc.FrontFace.StencilPassOp = D3D11_STENCIL_OP_KEEP; //KEEP
dsDesc.FrontFace.StencilFunc = D3D11_COMPARISON_ALWAYS;
// Stencil operations if pixel is back-facing
dsDesc.BackFace.StencilFailOp = D3D11_STENCIL_OP_KEEP; //KEEP
dsDesc.BackFace.StencilDepthFailOp = D3D11_STENCIL_OP_DECR; //DECR
dsDesc.BackFace.StencilPassOp = D3D11_STENCIL_OP_KEEP; //KEEP
dsDesc.BackFace.StencilFunc = D3D11_COMPARISON_ALWAYS;
// Create depth stencil state
mDevice->CreateDepthStencilState(&dsDesc, &mDepthStencilState);
D3D11_DEPTH_STENCIL_VIEW_DESC depthStencilViewDesc;
ZeroMemory (&depthStencilViewDesc, sizeof(depthStencilViewDesc));
depthStencilViewDesc.Format = depthTexDesc.Format;
depthStencilViewDesc.ViewDimension = D3D11_DSV_DIMENSION_TEXTURE2D;
depthStencilViewDesc.Texture2D.MipSlice = 0;
mDevice->CreateDepthStencilView(depthStencilTexture, &depthStencilViewDesc, &mDepthStencilView);
mDeviceContext->OMSetDepthStencilState(mDepthStencilState, 1);
and then afterwards i call
mDeviceContext->OMSetRenderTargets(1, &mTargetView, mDepthStencilView);
obviously i clean before every frame
mDeviceContext->ClearRenderTargetView(mTargetView, D3DXCOLOR(0.0f, 0.0f, 0.0f, 1.0f));
mDeviceContext->ClearDepthStencilView(mDepthStencilView, D3D11_CLEAR_DEPTH, 1.0f, 0 );
and still it just keeps the last pixel drawn with no testing....
screenshot
PS i've checked the rasterizer and it is correctly drawing only the front faces
any help anyone?
Check your HRESULTs - the call to CreateTexture2D is almost certainly failing because you have specified CPU_ACCESS flags on a DEFAULT texture. Since you never check any errors or pointers, this just propagates NULL to all your depth objects, effectively disabling depth testing.
You can also catch errors like this by enabling D3D debug layers, by adding D3D11_CREATE_DEVICE_DEBUG to the flags on D3D11CreateDevice. If you had done this, you would see the following debug spew:
D3D11 ERROR: ID3D11Device::CreateTexture2D: A D3D11_USAGE_DEFAULT
Resource cannot have any CPUAccessFlags set. The following
CPUAccessFlags bits cannot be set in this case: D3D11_CPU_ACCESS_READ
(1), D3D11_CPU_ACCESS_WRITE (1). [ STATE_CREATION ERROR #98:
CREATETEXTURE2D_INVALIDCPUACCESSFLAGS]
I'm using the function cvHaarDetectObjects to do face detection and there is a memory leak checking with valgrind even though I think I freed all the memories. I really don't know how to fix the memory leak. Here is my code:
int Detect(MyImage* Img,MyImage **Face)
{
Char* Cascade_name = new Char[1024];
strcpy(Cascade_name,"/usr/share/OpenCV/haarcascades/haarcascade_frontalface_alt.xml");
// Create memory for calculations
CvMemStorage* Storage = 0;
// Create a new Haar classifier
CvHaarClassifierCascade* Cascade = 0;
int Scale = 1;
// Create two points to represent the face locations
CvPoint pt1, pt2;
int Loop;
// Load the HaarClassifierCascade
Cascade = (CvHaarClassifierCascade*)cvLoad( Cascade_name, 0, 0, 0 );
// Check whether the cascade has loaded successfully. Else report and error and quit
if( !Cascade )
{
fprintf( stderr, "ERROR: Could not load classifier cascade\n" );
exit(0);
}
// Allocate the memory storage
Storage = cvCreateMemStorage(0);
// Clear the memory storage which was used before
cvClearMemStorage( Storage );
// Find whether the cascade is loaded, to find the faces. If yes, then:
if( Cascade )
{
// There can be more than one face in an image. So create a growable sequence of faces.
// Detect the objects and store them in the sequence
CvSeq* Faces = cvHaarDetectObjects( Img->Image(), Cascade, Storage,
1.1, 2, CV_HAAR_DO_CANNY_PRUNING,
cvSize(40, 40) );
int MaxWidth = 0;
int MaxHeight = 0;
if(Faces->total == 0)
{
cout<<"There is no face."<<endl;
return 1;
}
//just get the first face
for( Loop = 0; Loop <1; Loop++ )
{
// Create a new rectangle for drawing the face
CvRect* Rect = (CvRect*)cvGetSeqElem( Faces, Loop );
// Find the dimensions of the face,and scale it if necessary
pt1.x = Rect->x*Scale;
pt2.x = (Rect->x+Rect->width)*Scale;
if(Rect->width>MaxWidth) MaxWidth = Rect->width;
pt1.y = Rect->y*Scale;
pt2.y = (Rect->y+Rect->height)*Scale;
if(Rect->height>MaxHeight) MaxHeight = Rect->height;
cvSetImageROI( Img->Image(), *Rect );
MyImage* Dest = new MyImage(cvGetSize(Img->Image()),IPL_DEPTH_8U, 1);
cvCvtColor( Img->Image(), Dest->Image(), CV_RGB2GRAY );
MyImage* Equalized = new MyImage(cvGetSize(Dest->Image()), IPL_DEPTH_8U, 1);
// Perform histogram equalization
cvEqualizeHist( Dest->Image(), Equalized->Image());
(*Face) = new MyImage(Equalized->Image());
if(Equalized)
delete Equalized;
Equalized = NULL;
if(Dest)
delete Dest;
Dest = NULL;
cvResetImageROI(Img->Image());
}
if(Cascade)
{
cvReleaseHaarClassifierCascade( &Cascade );
delete Cascade;
Cascade = NULL;
}
if(Storage)
{
cvClearMemStorage(Storage);
cvReleaseMemStorage(&Storage);
delete Storage;
Storage = NULL;
}
if(Cascade_name)
delete [] Cascade_name;
Cascade_name = NULL;
return 0;
}
In the code, MyImage is a wrapper class of IplImage containing IplImage* p as a member. if the constructor takes a IplImage* ppara as parameter, then the member p will create memory using cvCreateImage(cvGetSize(ppara), ppara->depth, ppara->nChannels) and cvCopy(ppara, p). if it takes size,depth and channels as parameter, then only do cvCreateImage. Then the destructor do cvReleaseImage(&p). The function int Detect(MyImage *Img, MyImage **Face) is called like:
IplImage *Temp = cvLoadImage(ImageName);
MyImage* Img = new MyImage(Temp);
if(Temp)
cvReleaseImage(&Temp);
Temp = NULL;
MyImage * Face = NULL;
Detect(Img, &Face);
I released Img and Face in the following code once the operations on them is done. And the memory leak is happened inside the Detect function. I'm using OpenCV 2.3.1 on 64 bit OS fedora 16. The whole program can terminate normally except for the memory leak.
Thanks a lot.
I found out why there is a memory leak. The reason is:
In the MyImage class constructor, I passed in a IplImage* p pointer, and do the following:
mp = cvCloneImage(p);
where mp is a IplImage* member of the MyImage class. I free the IplImage* pointer that I passed in after creating a new MyImage class object, since cvCloneImage() will create some memories. However I free the member pointer mp in the class destructor when it actually doesn't new any memory. It just points to memories that created by cvCloneImage(). So the memories created by cvCloneImage() isn't freed. This is where the memory leak came from.
Thus I do the following in the constructor given a IplImage* p passed in as parameter:
mp = cvCreateImage(cvGetSize(p), p->depth, p->nChannels);
cvCopy(p, mp);
And free the mp pointer in the class destructor will free the memory that it creates.
After doing this, definitely lost and indirectly lost memories are turned to 0, but there are still possibly lost memories, and valgrind points all the lost record to the cvHaarDetectObjects() function which is from OpenCV. And mostly are caused by some "new threads" issues. Thus I googled this problem and found out valgrind does give possibly lost memories sometimes when new thread is involved. So I monitored the memory usage of the system. The result shows no memory usage building up as the program executed repeatedly.
That's what I found.
Instead of any additional blob detection library, how do I use the cv::SimpleBlobDetector class and its function detectblobs()?
Python: Reads image blob.jpg and performs blob detection with different parameters.
#!/usr/bin/python
# Standard imports
import cv2
import numpy as np;
# Read image
im = cv2.imread("blob.jpg")
# Setup SimpleBlobDetector parameters.
params = cv2.SimpleBlobDetector_Params()
# Change thresholds
params.minThreshold = 10
params.maxThreshold = 200
# Filter by Area.
params.filterByArea = True
params.minArea = 1500
# Filter by Circularity
params.filterByCircularity = True
params.minCircularity = 0.1
# Filter by Convexity
params.filterByConvexity = True
params.minConvexity = 0.87
# Filter by Inertia
params.filterByInertia = True
params.minInertiaRatio = 0.01
# Create a detector with the parameters
# OLD: detector = cv2.SimpleBlobDetector(params)
detector = cv2.SimpleBlobDetector_create(params)
# Detect blobs.
keypoints = detector.detect(im)
# Draw detected blobs as red circles.
# cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS ensures
# the size of the circle corresponds to the size of blob
im_with_keypoints = cv2.drawKeypoints(im, keypoints, np.array([]), (0,0,255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
# Show blobs
cv2.imshow("Keypoints", im_with_keypoints)
cv2.waitKey(0)
C++: Reads image blob.jpg and performs blob detection with different parameters.
#include "opencv2/opencv.hpp"
using namespace cv;
using namespace std;
int main(int argc, char** argv)
{
// Read image
#if CV_MAJOR_VERSION < 3 // If you are using OpenCV 2
Mat im = imread("blob.jpg", CV_LOAD_IMAGE_GRAYSCALE);
#else
Mat im = imread("blob.jpg", IMREAD_GRAYSCALE);
#endif
// Setup SimpleBlobDetector parameters.
SimpleBlobDetector::Params params;
// Change thresholds
params.minThreshold = 10;
params.maxThreshold = 200;
// Filter by Area.
params.filterByArea = true;
params.minArea = 1500;
// Filter by Circularity
params.filterByCircularity = true;
params.minCircularity = 0.1;
// Filter by Convexity
params.filterByConvexity = true;
params.minConvexity = 0.87;
// Filter by Inertia
params.filterByInertia = true;
params.minInertiaRatio = 0.01;
// Storage for blobs
std::vector<KeyPoint> keypoints;
#if CV_MAJOR_VERSION < 3 // If you are using OpenCV 2
// Set up detector with params
SimpleBlobDetector detector(params);
// Detect blobs
detector.detect(im, keypoints);
#else
// Set up detector with params
Ptr<SimpleBlobDetector> detector = SimpleBlobDetector::create(params);
// Detect blobs
detector->detect(im, keypoints);
#endif
// Draw detected blobs as red circles.
// DrawMatchesFlags::DRAW_RICH_KEYPOINTS flag ensures
// the size of the circle corresponds to the size of blob
Mat im_with_keypoints;
drawKeypoints(im, keypoints, im_with_keypoints, Scalar(0, 0, 255), DrawMatchesFlags::DRAW_RICH_KEYPOINTS);
// Show blobs
imshow("keypoints", im_with_keypoints);
waitKey(0);
}
The answer has been copied from this tutorial I wrote at LearnOpenCV.com explaining various parameters of SimpleBlobDetector. You can find additional details about the parameters in the tutorial.
You may store the parameters for the blob detector in a file, but this is not necessary. Example:
// set up the parameters (check the defaults in opencv's code in blobdetector.cpp)
cv::SimpleBlobDetector::Params params;
params.minDistBetweenBlobs = 50.0f;
params.filterByInertia = false;
params.filterByConvexity = false;
params.filterByColor = false;
params.filterByCircularity = false;
params.filterByArea = true;
params.minArea = 20.0f;
params.maxArea = 500.0f;
// ... any other params you don't want default value
// set up and create the detector using the parameters
cv::SimpleBlobDetector blob_detector(params);
// or cv::Ptr<cv::SimpleBlobDetector> detector = cv::SimpleBlobDetector::create(params)
// detect!
vector<cv::KeyPoint> keypoints;
blob_detector.detect(image, keypoints);
// extract the x y coordinates of the keypoints:
for (int i=0; i<keypoints.size(); i++){
float X = keypoints[i].pt.x;
float Y = keypoints[i].pt.y;
}
Note: all the examples here are using the OpenCV 2.X API.
In OpenCV 3.X, you need to use:
Ptr<SimpleBlobDetector> d = SimpleBlobDetector::create(params);
See also: the transition guide: http://docs.opencv.org/master/db/dfa/tutorial_transition_guide.html#tutorial_transition_hints_headers
// creation
cv::SimpleBlobDetector * blob_detector;
blob_detector = new SimpleBlobDetector();
blob_detector->create("SimpleBlobDetector");
// change params - first move it to public!!
blob_detector->params.filterByArea = true;
blob_detector->params.minArea = 1;
blob_detector->params.maxArea = 32000;
// or read / write them with file
FileStorage fs("test_fs.yml", FileStorage::WRITE);
FileNode fn = fs["features"];
//blob_detector->read(fn);
// detect
vector<KeyPoint> keypoints;
blob_detector->detect(img_text, keypoints);
fs.release();
I do know why, but params are protected. So I moved it in file features2d.hpp to be public:
virtual void read( const FileNode& fn );
virtual void write( FileStorage& fs ) const;
public:
Params params;
protected:
struct CV_EXPORTS Center
{
Point2d loc
If you will not do this, the only way to change params is to create file (FileStorage fs("test_fs.yml", FileStorage::WRITE);), than open it in notepad, and edit. Or maybe there is another way, but I`m not aware of it.