Overlay smaller image in a larger image in OpenCV - opencv

I would like to replace a part of the image with my image in Opencv
I used
cvGetPerspectiveMatrix() with a warpmatrix and using cvAnd() and cvOr()
but could not get it to work
This is the code that is currently displaying the image and a white polygon for the replacement image. I would like to replace the white polygon for a pic with any dimension to be scaled and replaced with the region pointed.
While the code is in javacv I could convert it to java even if c code is posted
grabber.start();
while(isDisp() && (image=grabber.grab())!=null){
if (dst_corners != null) {// corners of the image to be replaced
CvPoint points = new CvPoint((byte) 0,dst_corners,0,dst_corners.length);
cvFillConvexPoly(image,points, 4, CvScalar.WHITE, 1, 0);//white polygon covering the replacement image
}
correspondFrame.showImage(image);
}
Any pointers to this will be very helpful.
Update:
I used warpmatrix with this code and I get a black spot for the overlay image
cvSetImageROI(image, cvRect(x1,y1, overlay.width(), overlay.height()));
CvPoint2D32f p = new CvPoint2D32f(4);
CvPoint2D32f q = new CvPoint2D32f(4);
q.position(0).x(0);
q.position(0).y(0);
q.position(1).x((float) overlay.width());
q.position(1).y(0);
q.position(2).x((float) overlay.width());
q.position(2).y((float) overlay.height());
q.position(3).x(0);
q.position(3).y((float) overlay.height());
p.position(0).x((int)Math.round(dst_corners[0]);
p.position(0).y((int)Math.round(dst_corners[1]));
p.position(1).x((int)Math.round(dst_corners[2]));
p.position(1).y((int)Math.round(dst_corners[3]));
p.position(3).x((int)Math.round(dst_corners[4]));
p.position(3).y((int)Math.round(dst_corners[5]));
p.position(2).x((int)Math.round(dst_corners[6]));
p.position(2).y((int)Math.round(dst_corners[7]));
cvGetPerspectiveTransform(q, p, warp_matrix);
cvWarpPerspective(overlay, image, warp_matrix);
I get a black spot for the overlay image and even though the original image is a polygon with 4 vertices the overlay image is set as a rectangle. I believe this is because of the ROI. Could anyone please tell me how to fit the image as is and also why I am getting a black spot instead of the overlay image.

I think cvWarpPerspective(link) is what you are looking for.
So instead of doing
CvPoint points = new CvPoint((byte) 0,dst_corners,0,dst_corners.length);
cvFillConvexPoly(image,points, 4, CvScalar.WHITE, 1, 0);//white polygon covering the replacement image
Try
cvWarpPerspective(yourimage, image, M, image.size(), INTER_CUBIC, BORDER_TRANSPARENT);
Where M is the matrix you get from cvGetPerspectiveMatrix

One way to do it is to scale the pic to the white polygon size and then copy it to the grabbed image setting its Region of Interest (here is a link explaining the ROI).
Your code should look like this:
resize(pic, resizedImage, resizedImage.size(), 0, 0, interpolation); //resizedImage should have the points size
cvSetImageROI(image, cvRect(the points coordinates));
cvCopy(resizedImage,image);
cvResetImageROI(image);
I hope that helps.
Best regards,
Daniel

Related

How to remove blank space from cropped image om opencv?

I am trying to crop a specific part of a frame in opencv to get a cropped image of the detections from mobilenet ssd model. The code to crop the image is like this
for box_id in boxes_ids:
x,y,w,h,id = box_id
crop=frame[y:h,x:w]
cv2.imshow("d",crop)
cv2.waitKey(5)
This code is producing a blank space towards the right of all the images that I extract :
Please tell me how can i fix this.
try using Pillow that helps
def trim(im, color):
bg = Image.new(im.mode, im.size, color)
diff = ImageChops.difference(im, bg)
diff = ImageChops.add(diff, diff)
bbox = diff.getbbox()
if bbox:
return im.crop(bbox)
This function will probably take it out, just be carefull that this will only work if the segment of image has consistent pixels
as said before in the comments, there is a minimum window width, and smaller crops will be drawn on some neutral background.
but maybe it's more intuitive to draw the crop into an empty image, conserving its original position:
for box_id in boxes_ids:
x,y,w,h,id = box_id
draw = np.zeros(frame.shape, np.uint8)
draw[y:h,x:w] = frame[y:h,x:w]
cv2.imshow("d",draw)
cv2.waitKey(5)

How to add transparent border OpenCV copymakeborder

I have 26 PNG files, each with an image of a letter of the alphabet. They've all been fully cropped to the letter shape with the result that when I insert them into an image, letters with tails all 'sit on the line'
Each letter is in black, with a transparent background. Each PNG has different dimensions, because of the differing letter shapes
I thought I'd remediate this by adding a transparent border of a different size depending on the source file, to make common datum for all the letters, so that 'a' for example would have some transparent space at the bottom.
I've coded up the calculcation for each letter, but I have two issues:
1) Even before applying the operation, I can't seem to read the file in and write it to a new unchanged file in OpenCV. The transparency in the image is replaced with black.
2) While I can add a colour border, I can't seem to add a transparent border.
Original Image:
Read in, and written out:
Apparenly with a blue border, but maximum transparancy:
I have a feeling that if I can sort out the first problem, the second might fall in line. Here is my code:
img = cv2.imread(file)
img_with_border = cv2.copyMakeBorder(img, top, bottom, left, right, cv2.BORDER_CONSTANT, value=[-255,0,0,255])
#img_with_border = img
cv2.imwrite(newfile, img_with_border, [cv2.IMWRITE_JPEG_QUALITY, 100])
I'd appreciate some help on what is going on here with transparancy. Is OpenCV the right tool to use?
Thanks,
Jeff.
To load a PNG image with 4 channels in OpenCV, use im = cv2.imread(file, cv2.IMREAD_UNCHANGED). You will obtain a BGRA image.
To change the alpha value, you have to change the fourth channel of the image. This means that to create your transparent border you have to pass a value (B, G, R, 0) and not [-255, 0, 0, 255]. (What is that -255 by the way ?). B, G and R can be 0, it doesn't matter.
Also, make sure you write to a PNG image to keep the transparency. You seem to be writing your result as JPEG.

Wrong result using function fillPoly in opencv for very large images

I have a hard time solving the issue with mask creation.My image is large,
40959px X 24575px and im trying to create a mask for it.
I noticed that i dont have a problem for images up to certain size(I tested about 33000px X 22000px), but for dimensions larger than that i get an error inside my mask(Error is that it gets black in the middle of the polygon and white region extends itself to the left edge.Result should be without black area inside polygon and no white area extending to the left edge of image).
So my code looks like this:
pixel_points_list = latLonToPixel(dataSet, lat_lon_pairs)
print pixel_points_list
# This is the list im getting
#[[213, 6259], [22301, 23608], [25363, 22223], [27477, 23608], [35058, 18433], [12168, 282], [213, 6259]]
image = cv2.imread(in_tmpImgFilePath,-1)
print image.shape
#Value of image.shape: (24575, 40959, 4)
mask = np.zeros(image.shape, dtype=np.uint8)
roi_corners = np.array([pixel_points_list], dtype=np.int32)
print roi_corners
#contents of roi_corners_array:
"""
[[[ 213 6259]
[22301 23608]
[25363 22223]
[27477 23608]
[35058 18433]
[12168 282]
[ 213 6259]]]
"""
channel_count = image.shape[2]
ignore_mask_color = (255,)*channel_count
cv2.fillPoly(mask, roi_corners, ignore_mask_color)
cv2.imwrite("mask.tif",mask)
And this is the mask im getting with those coordinates(minified mask):
You see that in the middle of the mask the mask is mirrored.I took those points from pixel_points_list and drawn them on coordinate system and im getting valid polygon, but when using fillPoly im getting wrong results.
Here is even simpler example where i have only 4(5) points:
roi_corners = array([[ 213 6259]
[22301 23608]
[35058 18433]
[12168 282]
[ 213 6259]])
And i get
Does anyone have a clue why does this happen?
Thanks!
The issue is in the function CollectPolyEdges, called by fillPoly (and drawContours, fillConvexPoly, etc...).
Internally, it's assumed that the point coordinates (of integer type int32) have meaningful values only in the 16 lowest bits. In practice, you can draw correctly only if your points have coordinates up to 32768 (which is exactly the maximum x coordinate you can draw in your image.)
This can't be considered as a bug, since your images are extremely large.
As a workaround, you can try to scale your mask and your points by a given factor, fill the poly on the smaller mask, and then re-scale the mask back to original size
As #DanMaĆĄek pointed out in the comments, this is in fact a bug, not fixed, yet.
In the bug discussion, there is another workaround mentioned. It consists on drawing using multiple ROIs with size less than 32768, correcting coordinates for each ROI using the offset parameter in fillPoly.

Eliminating various backgrounds from image and segmenting object?

Let say I have this input image, with any number of boxes. I want to segment out these boxes, so I can eventually extract them out.
input image:
The background could anything that is continuous, like a painted wall, wooden table, carpet.
My idea was that the gradient would be the same throughout the background, and with a constant gradient. I could turn where the gradient is about the same, into zero's in the image.
Through edge detection, I would dilate and fill the regions where edges detected. Essentially my goal is to make a blob of the areas where the boxes are. Having the blobs, I would know the exact location of the boxes, thus being able to crop out the boxes from the input image.
So in this case, I should be able to have four blobs, and then I would be able to crop out four images from the input image.
This is how far I got:
segmented image:
query = imread('AllFour.jpg');
gray = rgb2gray(query);
[~, threshold] = edge(gray, 'sobel');
weightedFactor = 1.5;
BWs = edge(gray,'roberts');
%figure, imshow(BWs), title('binary gradient mask');
se90 = strel('disk', 30);
se0 = strel('square', 3);
BWsdil = imdilate(BWs, [se90]);
%figure, imshow(BWsdil), title('dilated gradient mask');
BWdfill = imfill(BWsdil, 'holes');
figure, imshow(BWdfill);
title('binary image with filled holes');
What a very interesting problem! Here's my solution in an attempt to solve this problem for you. This is assuming that the background has the same colour distribution throughout. First, transform your image from RGB to the HSV colour space with rgb2hsv. The HSV colour space is an ideal transform for analyzing colours. After this, I would look at the saturation and value planes. Saturation is concerned with how "pure" the colour is, while value is the intensity or brightness of the colour itself. If you take a look at the saturation and value planes for the image, this is what is shown:
im = imread('http://i.stack.imgur.com/1SGVm.jpg');
out = rgb2hsv(im);
figure;
subplot(2,1,1);
imshow(out(:,:,2));
subplot(2,1,2);
imshow(out(:,:,3));
This is what I get:
By taking a look at some locations in the gray background, it looks like the majority of the saturation are less than 0.2 as well as the elements in the value plane are greater than 0.3. As such, we want to find the opposite of those pixels to get our objects. As such, we find those pixels whose saturation is greater than 0.2 or those pixels with a value that is less than 0.3:
seg = out(:,:,2) > 0.2 | out(:,:,3) < 0.3;
This is what we get:
Almost there! There are some spurious single pixels, so I'm going to perform an opening with imopen with a line structuring element.
After this, I'll perform a dilation with imdilate to close any gaps, then use imfill with the 'holes' option to fill in the gaps, then use erosion with imerode to shrink the shapes back to their original form. As such:
se = strel('line', 3, 90);
pre = imopen(seg, c);
se = strel('square', 20);
pre2 = imdilate(pre, se);
pre3 = imfill(pre2, 'holes');
final = imerode(pre3, se);
figure;
imshow(final);
final contains the segmented image with the 4 candy boxes. This is what I get:
Try resizing the image. When you make it smaller, it would be easier to join edges. I tried what's shown below. You might have to tune it depending on the nature of the background.
close all;
clear all;
im = imread('1SGVm.jpg');
small = imresize(im, .25); % resize
grad = (double(imdilate(small, ones(3))) - double(small)); % extract edges
gradSum = sum(grad, 3);
bw = edge(gradSum, 'Canny');
joined = imdilate(bw, ones(3)); % join edges
filled = imfill(joined, 'holes');
filled = imerode(filled, ones(3));
imshow(label2rgb(bwlabel(filled))) % label the regions and show
If you have a recent version of MATLAB, try the Color Thresholder app in the image processing toolbox. It lets you interactively play with different color spaces, to see which one can give you the best segmentation.
If your candy covers are fixed or you know all the covers that are coming into the scene then Template matching is best for this. As it is independent of the background in the image.
http://docs.opencv.org/doc/tutorials/imgproc/histograms/template_matching/template_matching.html

Crop an Image to the shape of a Vector or Overlay a Shape

I imagine this is a shot in the dark, but is it possible to have a vector file of a shape (in this case a hexagon with rounded corners), and pass an image through some code and have it coming out cropped in the shape of that vector?
I'm attempting to utilize hexagons in my design and have gone through every page I possibly can. I've seen the many HTML and CSS solutions, but none of them achieve what I'm looking for flawlessly.
Another idea I have is maybe overlaying a transparent hexagon shape with white corners on top of the existing image with imagemagick, and then going through and making any white transparent. Thoughts?
I don't have any code for cropping in the shape of a vector file, but here's what I have for overlaying an outline of the shape I want on top of the other picture:
imgfile = "public/" + SecureRandom.uuid + ".png"
SmartCropper.from_file(art.url(:original)).smart_square.resize(225,225).write(imgfile)
overlay = Magick::Image.read("app/assets/images/overlay.png")
img = Magick::Image.read(imgfile)
img.composite(overlay,0,0, Magick::OverCompositeOp)
Right now it's giving me an undefined method error for composite, which is strange because I've followed some other stack overflow questions using the same thing in their models.
Any help is appreciated!
You have fallen for a common ImageMagick trap - the objects you get from the .read method are not Magick::Image objects but Magick::ImageList ones, and for most image types you want the first item from the list.
Without knowing how you have set up your overlay.png file, it is difficult to tell what the best composite option is. However, in a similar situation I found CopyOpacityCompositeOp to be useful, and to have the overlay's transparency control the transparency in the final image.
I tested the following code and it looks like it would do what you want if overlay.png was set up that way:
require 'smartcropper'
imgfile = "test_square.png"
SmartCropper.from_file( 'test_input.png' ).
smart_square.resize( 225, 225 ).write( imgfile )
overlay = Magick::Image.read( 'overlay.png' ).first
img = Magick::Image.read( imgfile ).first
img.composite( overlay, 0, 0, Magick::CopyOpacityCompositeOp ).
write( "test_result.png" )
Instead of reading overlay from a file, you could create it using Magick::Draw like this:
overlay = Magick::Image.new( 225, 225 ) do |i|
i.background_color= "Transparent"
end
gc = Magick::Draw.new
gc.stroke('white').stroke_width(10)
gc.fill('white')
gc.polygon(97.5, 26.25, 178.5, 73.125, 178.5, 167,
97.5, 213.75, 16.5, 167, 16.5, 73.125)
gc.draw( overlay )
NB That's a hexagon, but I've not bothered centering it.

Resources