I'm trying to remove the background of an UIImage and ended up using iOS port of MagicWand.
I was able to successfully remove the selected pixels using the flood fill algorithm by invoking the method MagickFloodfillPaintImage. However the challenge now is softening the sharp edges left by the Flood fill removal process.
I'm using the code below to remove the background.
PixelIterator* iterator = NULL;
PixelWand** pixels = NULL;
size_t x;
iterator = NewPixelRegionIterator(magick_wand, _touchCoords.x*scale, _touchCoords.y*scale, 1, 1);
pixels = PixelGetNextIteratorRow(iterator,&x);
bc_wand = pixels[0];
channel = ParseChannelOption("rgba");
status = MagickFloodfillPaintImage(magick_wand, channel, fc_wand, _tolerance, bc_wand, _touchCoords.x*scale, _touchCoords.y*scale, MagickFalse);
if (status == MagickFalse) {
ThrowWandException(magick_wand);
}
Any help would be greatly appreciated!
Sounds like you need anti-aliasing in MagickFloodfillPaintImage. However MagickFloodfillPaintImage do not support it.
But fortunately, you can use traditional multisampling technique to simulate it.
And it is quite easy:
Resize your image to a larger resolution (For example 100x100 -> 200x200)(Up sampling).
Apply MagickFloodfillPaintImage
Resize the image to the original size (Down sampling).
The quality depends on the resize ratio.
Consider this example (remove the white background):
Original algorithm would produce hard edges:
With resize, it could be much better:
Related
Summary:
I've been having issues correcting white balance and color casts on images specifically when testing on an iPhone 7. So far everything runs fine on an iPhone 6, but the camera on a 7 creates sometimes a purple or yellow or blue tint in any image with the flash on. The app I'm developing relies heavily on color detection using OpenCV, so I'm trying different methods to correct the color cast. The scenario I'm running into is this: the user has a piece of paper and some items on the paper to identify by color, but when an iPhone 7 is used close to the paper with the flash on the entire image takes on a tint. The paper is used to make it easier to separate images from the background, as well as used in white-balance to know what part of the image should be white to potentially fix white-balance/color cast problems.
Details:
I'm able to correct slight tints in color using a background adjust method with OpenCV:
- (void)adjustBackground:(Mat &)inputROI image:(Mat &)imageInBG{
Scalar meanCol = mean(inputROI);
// original
Mat labOrig, labFloat, ROIfloat;
std::vector<Mat>planes(3);
inputROI.convertTo(ROIfloat, CV_32FC3, 1.0/255.0f);
cvtColor(ROIfloat,labFloat,CV_BGR2Lab);
split(labFloat,planes);
double l_v,a_v,b_v;
rgb2lab(meanCol(2), meanCol(1), meanCol(0), l_v, a_v, b_v);
add(planes[1], -a_v, planes[1]);
add(planes[2], -b_v, planes[2]);
merge(planes,labFloat);
cvtColor(labFloat, ROIfloat, CV_Lab2BGR);
ROIfloat.convertTo(inputROI , CV_8UC3, 255.0f);
planes.clear();
labOrig.release();
labFloat.release();
ROIfloat.release();
}
Where rgb2lab does just what it implies, converting rgb to the lab color space. I also convert the image to float for better precision. This is able to correct small color casts but if the image is heavily tinted it still results in slightly tinted colors and color detection with OpenCV still results in too much of the tint color being detected.
What I tried next was more of a direct adjustment of the camera settings, which I feel is a better approach to fixing the problem initially instead of after the fact with a sort of post-processing color correction. I found some documentation for modifying the camera's temperature and tint values, but it just results in the user having to manually adjust sliders to get the desired white-balanced image:
Class captureDeviceClass = NSClassFromString(#"AVCaptureDevice");
if (captureDeviceClass != nil) {
AVCaptureDevice *device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
if ([device isWhiteBalanceModeSupported: AVCaptureWhiteBalanceModeLocked]){
if ([device lockForConfiguration:nil]){
AVCaptureWhiteBalanceTemperatureAndTintValues temperatureAndTint = {
.temperature = tempVal,
.tint = tintVal,
};
AVCaptureWhiteBalanceGains wbGains = [device deviceWhiteBalanceGainsForTemperatureAndTintValues:temperatureAndTint];
if((NSLocationInRange(wbGains.redGain, NSMakeRange(1, (device.maxWhiteBalanceGain - 1.0))))&&(NSLocationInRange(wbGains.greenGain, NSMakeRange(1, (device.maxWhiteBalanceGain - 1.0))))&&(NSLocationInRange(wbGains.blueGain, NSMakeRange(1, (device.maxWhiteBalanceGain - 1.0)))))
{
NSLog(#"Good values");
[device deviceWhiteBalanceGainsForTemperatureAndTintValues:temperatureAndTint];
[device setWhiteBalanceModeLockedWithDeviceWhiteBalanceGains:[device deviceWhiteBalanceGainsForTemperatureAndTintValues:temperatureAndTint] completionHandler:^(CMTime syncTime) {
}];
[device unlockForConfiguration];
}
else{
NSLog(#"Bad values");
}
}
}
Where tempVal and tintVal are inputs from sliders.
Is there any way to turn off auto adjustments on the iPhone camera, or is there a better way in OpenCV to adjust for more extreme color casts?
Edit:
Here are some examples. Disregard the graphs in the middle, I was trying something with histograms. The one image shows a blue tint on the entire image, and the other shows my color cast correction working in the ROI in the middle, but it changes the colors on the image too much (I need the color bands on the resistors to be as accurate as possible).
http://i.imgur.com/jlc4MDa.jpg and http://i.imgur.com/PG81pAl.jpg
For anyone this might help, I found a decent solution to adjust the color cast. It turns out the way I was adjusting in the first place with OpenCV was pretty close, and I just needed to adjust the color ranges I was looking for to resemble the kind of colors I was getting after adjustment. I also stopped using the rgb2lab function I found and just directly transformed the mat itself from bgr (float for precision) to lab. This seems like its closer to true white-balanced or color-corrected images, but it could just be the adjusted color ranges I came up with that really made it better.
The rest is pretty much the same, just find the mean of the A and B channel and split the mat into LAB planes to adjust them "back to center". Here is the code for using float LAB mats and getting the offset in A and B:
Mat labFloat, ROIfloat;
std::vector<Mat>lab_planes(3);
input.convertTo(ROIfloat, CV_32FC3, 1.0/255.0f);
cvtColor(ROIfloat,labFloat,CV_BGR2Lab);
split(labFloat,lab_planes);
Scalar meanVals = mean(labFloat);
// valA is the mean of the A channel (offset to add/subtract)
// valB is the mean of the B channel (offset to add/subtract)
double valA = meanVals(1);
double valB = meanVals(2);
EDIT:
I just want to add that I also started only color-cast correcting spot areas that I needed instead of the entire image. Different areas of an image have a different cast based on lighting etc. so it makes sense to just use small areas at a time for correcting. This turned out to be a lot more accurate and gave better results.
When layering multiple I have to combine them at least once with an "empty" image or background. But there is nothing like "empty". It's transparent black oder transparent white. Even being completely transparent has a color which is normally ignored when displaying.
I'd like to have the folloing:
Destination = Image1 + Image2
To do this I do in fact this:
Clear(Destination, 0); // Clear it with transparent black
Blend Image1 over Destination
Blend Image2 over Destination
Although Destination is transparent black the black contributes to the final result which is what I'd like to avoid. I could clear the destination with transparent white, but it would only work in this example and produces the same problems with darker images.
I understand why the result is like it is: The directx blend formular is set like that:
BlendOp = Add;
SrcBlend = SrcAlpha;
DestBlend = InvSrcAlpha;
The alpha of the destination (DestAlpha) isn't accounted at all in this formular. What can I do to implement this everyday problem of blending two images together before rendering them somewhere else?
Afaik you can't achieve this with normal blending operations, but maybe you could write a appropiate shader to do the blending for you. You render on an extra texture as a rendertarget and set this texture in the pixelshader. Now you branch your writed alpha value, if the background-alpha is zero you write your imagecolor, otherwise there is already an image written on this pixel and you can do the normal alphablending in your shader. With this branch you detect, whether the pixel is empty or written before and then prevent the black color to have any impact. Hope this helps :)
A little example to visualize what I mean:
float4 targetColor = tex2D(backgroundSampler,screen.TexCoord);
float4 diffuse = tex2D(imageSampler,image.TexCoord);
// overwrite black background
if (targetColor.a == 0.0) {
return diffuse;
}
// do alphablending
return targetColor*(1-diffuse.a)+diffuse*(diffuse.a);
Using XNA, I'm trying to make an adventure game engine that lets you make games that look like they fell out of the early 90s, like Day of the Tentacle and Sam & Max Hit the Road. Thus, I want the game to actually run at 320x240 (I know, it should probably be 320x200, but shh), but it should scale up depending on user settings.
It works kind of okay right now, but I'm running into some problems where I actually want it to look more pixellated that it currently does.
Here's what I'm doing right now:
In the game initialization:
public Game() {
graphics = new GraphicsDeviceManager(this);
graphics.PreferredBackBufferWidth = 640;
graphics.PreferredBackBufferHeight = 480;
graphics.PreferMultiSampling = false;
Scale = graphics.PreferredBackBufferWidth / 320;
}
Scale is a public static variable that I can check anytime to see how much I should scale my game relative to 320x240.
In my drawing function:
spriteBatch.Begin(SpriteSortMode.BackToFront, BlendState.NonPremultiplied, SamplerState.PointClamp, DepthStencilState.Default, RasterizerState.CullNone, null, Matrix.CreateScale(Game.Scale));
This way, everything is drawn at 320x240 and blown up to fit the current resolution (640x480 by default). And of course I do math to convert the actual coordinates of the mouse into 320x240 coordinates, and so forth.
Now, this is great and all, but now I'm getting to the point where I want to start scaling my sprites, to have them walk into the distance and so forth.
Look at the images below. The upper-left image is a piece of a screenshot from when the game is running at 640x480. The image to the right of it is how it "should" look, at 320x240. The bottom row of images is just the top row blown up to 300% (in Photoshop, not in-engine) so you can see what I'm talking about.
In the 640x480 image, you can see different "line thicknesses;" the thicker lines are how it should really look (one pixel = 2x2, because this is running at 640x480), but the thinner lines (1x1 pixel) also appear, when they shouldn't, due to scaling (see the images on the right).
Basically I'm trying to emulate a 320x240 display but blown up to any resolution using XNA, and matrix transformations aren't doing the trick. Is there any way I could go about doing this?
Render everything in the native resolution to a RenderTarget instead of the back buffer:
SpriteBatch targetBatch = new SpriteBatch(GraphicsDevice);
RenderTarget2D target = new RenderTarget2D(GraphicsDevice, 320, 240);
GraphicsDevice.SetRenderTarget(target);
//perform draw calls
Then render this target (your whole screen) to the back buffer:
//set rendering back to the back buffer
GraphicsDevice.SetRenderTarget(null);
//render target to back buffer
targetBatch.Begin();
targetBatch.Draw(target, new Rectangle(0, 0, GraphicsDevice.DisplayMode.Width, GraphicsDevice.DisplayMode.Height), Color.White);
targetBatch.End();
My usual method of 100% contrast and some brightness adjusting to tweak the cutoff point usually works reasonably well to clean up photos of small sub-circuits or equations for posting on E&R.SE, however sometimes it's not quite that great, like with this image:
What other methods besides contrast (or instead of) can I use to give me a more consistent output?
I'm expecting a fairly general answer, but I'll probably implement it in a script (that I can just dump files into) using ImageMagick and/or PIL (Python) so if you have anything specific to them it would be welcome.
Ideally a better source image would be nice, but I occasionally use this on other folk's images to add some polish.
The first step is to equalize the illumination differences in the image while taking into account the white balance issues. The theory here is that the brightest part of the image within a limited area represents white. By blurring the image beforehand we eliminate the influence of noise in the image.
from PIL import Image
from PIL import ImageFilter
im = Image.open(r'c:\temp\temp.png')
white = im.filter(ImageFilter.BLUR).filter(ImageFilter.MaxFilter(15))
The next step is to create a grey-scale image from the RGB input. By scaling to the white point we correct for white balance issues. By taking the max of R,G,B we de-emphasize any color that isn't a pure grey such as the blue lines of the grid. The first line of code presented here is a dummy, to create an image of the correct size and format.
grey = im.convert('L')
width,height = im.size
impix = im.load()
whitepix = white.load()
greypix = grey.load()
for y in range(height):
for x in range(width):
greypix[x,y] = min(255, max(255 * impix[x,y][0] / whitepix[x,y][0], 255 * impix[x,y][1] / whitepix[x,y][1], 255 * impix[x,y][2] / whitepix[x,y][2]))
The result of these operations is an image that has mostly consistent values and can be converted to black and white via a simple threshold.
Edit: It's nice to see a little competition. nikie has proposed a very similar approach, using subtraction instead of scaling to remove the variations in the white level. My method increases the contrast in the regions with poor lighting, and nikie's method does not - which method you prefer will depend on whether there is information in the poorly lighted areas which you wish to retain.
My attempt to recreate this approach resulted in this:
for y in range(height):
for x in range(width):
greypix[x,y] = min(255, max(255 + impix[x,y][0] - whitepix[x,y][0], 255 + impix[x,y][1] - whitepix[x,y][1], 255 + impix[x,y][2] - whitepix[x,y][2]))
I'm working on a combination of techniques to deliver an even better result, but it's not quite ready yet.
One common way to remove the different background illumination is to calculate a "white image" from the image, by opening the image.
In this sample Octave code, I've used the blue channel of the image, because the lines in the background are least prominent in this channel (EDITED: using a circular structuring element produces less visual artifacts than a simple box):
src = imread('lines.png');
blue = src(:,:,3);
mask = fspecial("disk",10);
opened = imerode(imdilate(blue,mask),mask);
Result:
Then subtract this from the source image:
background_subtracted = opened-blue;
(contrast enhanced version)
Finally, I'd just binarize the image with a fixed threshold:
binary = background_subtracted < 35;
How about detecting edges? That should pick up the line drawings.
Here's the result of Sobel edge detection on your image:
If you then threshold the image (using either an empirically determined threshold or the Ohtsu method), you can clean up the image using morphological operations (e.g. dilation and erosion). That will help you get rid of broken/double lines.
As Lambert pointed out, you can pre-process the image using the blue channel to get rid of the grid lines if you don't want them in your result.
You will also get better results if you light the page evenly before you image it (or just use a scanner) cause then you don't have to worry about global vs. local thresholding as much.
I'm just starting game development and I thought a game like Tank wars or Worms would be nice.
The hardest part I can think of so far is making the terrain destructible and I want to know how it's done before doing the easy parts.
I thought that explosion could have a mask texture which could be scaled for different weapons. Then using that mask I should make underlying terrain transparent (and optionally draw a dark border).
(source: mikakolari.fi)
How do I achieve that?
Do I have to change the alpha value pixel by pixel or can I use some kind of masking technique? Drawing a blue circle on top of the terrain isn't an option.
I have versions 3.1 and 4.0 of XNA.
This tutorial is what you are searching:
http://www.riemers.net/eng/Tutorials/XNA/Csharp/series2d.php
Capter 20: Adding explosion craters
In short:
You have 2 textures: 1 Color Texture (visible), 1 Collision Texture (invisible)
You substract the explosion image from your collision texture.
To get the dark border: expand the explosion texture and darken the color in this area.
Now you generate a new Color Texture (old color - collison = new color).
This is a difficult question to answer - because there are many ways you could do it. And there are pros and cons to each method. I'll just give an overview:
As an overall design, you need to keep track of: the original texture, the "darkness" applied, and the "transparency" applied. One thing I can say almost for sure is you want to "accumulate" the results of the explosions somewhere - what you don't want to be doing is maintaining a list of all explosions that have ever happened.
So you have surfaces for texture, darkness and transparency. You could probably merge darkness and transparency into a single surface with a single channel that stores "normal", "dark" (or a level of darkness) and "transparent".
Because you probably don't want the dark rings to get progressively darker where they intersect, when you apply an explosion to your darkness layer with the max function (Math.Max in C#).
To produce your final texture you could just write from the darkness/transparency texture to your original texture or a copy of it (you only need to update the area that each explosion touches).
Or you could use a pixel shader to combine them - the details of which are beyond the scope of this question. (Also a pixel shader won't work on XNA 4.0 on Windows Phone 7.)
You should Make a new Texure2D with the Color of desired pixels.Alpha = 0.
Color[] bits = new Color[Texture.Width * Texture.Height];
Texture.GetData(bits);
foreach(Vector2D pixel in overlapedArea)
{
int x = (int)(pixel.X);
int y = (int)(pixel.Y);
bits[x + y * texture.Width] = Color.FromNonPremultiplied(0,0,0,0));
}
Texture2D newTexture = new Texture2D(texture.GraphicsDevice, texture.Width, texture.Height);
newTexture.SetData(bits);
Now replace the new Texture2D with the Last Texture and you're good to go!
For more code about Collision, or changing texture pixels color go to this page for codes:
http://www.codeproject.com/Articles/328894/XNA-Sprite-Class-with-useful-methods