Is there any way to get a layer image data, like the one that custom filters receive as a param? Yes, I know I can use getImageData and get it, but I was having a problems with it. In draggable canvases, it only captures visible part of the layer. Maybe I can create new canvas, then place a layer there and get image data? But I was wondering if there is any way to do it that is already built into konva?
If you want to get image data of layer's canvas you can do this:
const ctx = layer.getContext();
const imageData = ctx.getImageData(0, 0, width, height);
If your stage is too small and you need bigger image data, you have to export the layer into a bigger canvas and then get its image data:
const canvas = layer.toCanvas({ x: 0, y: 0, width, height});
const imageData = canvas.getContext('2d').getImageData(0, 0, width, height);
Related
I'm using the Vision framework to detect rectangular documents in a captured photo. Detecting and drawing a path around the document is working perfectly. I then want to crop the image to be only the detected document. I'm successfully cropping the image, but it seems the coordinates don't line up and the cropped image is only part of the detected document and the rest is just the desk behind the document. I'm using the following cropping code:
private UIImage CropImage(UIImage image, CGRect rect, float scale)
{
var drawRect = new CGRect(rect.X, rect.Y, rect.Size.Width, rect.Size.Height);
using (var cgImage = image.CGImage.WithImageInRect(drawRect))
{
var croppedImage = UIImage.FromImage(cgImage);
return croppedImage;
};
}
Using the following parameters:
image is the same UIImage that i successfully drew the rectangle path on.
rect is the VNRectangleObservation.BoundingBox. This is normalized so i'm scaling it using the image.size. it's the same scaling i do when drawing the rectangle path.
scale is 1f, but i'm currently ignoring this.
The cropped image generally seems to be the right size, but it is shifted up and to the left which cuts off the lower and right side of the document. Any help would be appreciated.
for anyone else that finds this, the issue seemed to be CGImage rotating when cropping the image which caused the VNRectangleObservation to not line up anymore. I used this article, Tracking and Altering Images, to get a working solution using CIFilter. Cropping code follows:
var ciFilter = CIFilter.FromName("CIPerspectiveCorrection");
if (ciFilter == null) continue;
var width = inputImage.Extent.Width;
var height = inputImage.Extent.Height;
var topLeft = new CGPoint(observation.TopLeft.X * width, observation.TopLeft.Y * height);
var topRight = new CGPoint(observation.TopRight.X * width, observation.TopRight.Y * height);
var bottomLeft = new CGPoint(observation.BottomLeft.X * width, observation.BottomLeft.Y * height);
var bottomRight = new CGPoint(observation.BottomRight.X * width, observation.BottomRight.Y * height);
ciFilter.SetValueForKey(new CIVector(topLeft), new NSString("inputTopLeft"));
ciFilter.SetValueForKey(new CIVector(topRight), new NSString("inputTopRight"));
ciFilter.SetValueForKey(new CIVector(bottomLeft), new NSString("inputBottomLeft"));
ciFilter.SetValueForKey(new CIVector(bottomRight), new NSString("inputBottomRight"));
var ciImage = inputImage.CreateByApplyingOrientation(CGImagePropertyOrientation.Up);
ciFilter.SetValueForKey(ciImage, CIFilterInputKey.Image);
var outputImage = ciFilter.OutputImage;
var uiImage = new UIImage(outputImage);
imageList.Add(uiImage);
imageList is a List<UImage> since i'm handling multiple detected rectangles.
observation is a single observation of type VNRectangleObservation.
The cropped image generally seems to be the right size, but it is shifted up and to the left which cuts off the lower and right side of the document.
From Apple documentation CGImageCreateWithImageInRect , there is a discussion about the cropped size .
CGImageCreateWithImageInRect performs the following tasks to create the subimage:
It calls the CGRectIntegral function to adjust the rect parameter to integral bounds.
It intersects the rect with a rectangle whose origin is (0,0) and size is equal to the size of the image specified by the image parameter.
It reads the pixels within the resulting rectangle, treating the first pixel within as the origin of the subimage.
If W and H are the width and height of image, respectively, then the point (0,0) corresponds to the first pixel of the image data. The point (W–1, 0) is the last pixel of the first row of the image data, while (0, H–1) is the first pixel of the last row of the image data and (W–1, H–1) is the last pixel of the last row of the image data.
Then you can check in your local project with an image (size is : 1920 * 1080) as follow:
UIImageView imageView = new UIImageView(new CGRect(0, 400, UIScreen.MainScreen.Bounds.Size.Width, 300));
UIImage image = new UIImage("th.jpg");
imageView.Image = CropImage(image, new CGRect(0, 0, 1920, 1080), 1);
View.AddSubview(imageView);
The CropImage Method :
private UIImage CropImage(UIImage image, CGRect rect, float scale)
{
var drawRect = new CGRect(rect.X, rect.Y, rect.Size.Width, rect.Size.Height);
using (var cgImage = image.CGImage.WithImageInRect(drawRect))
{
if(null != cgImage)
{
var croppedImage = UIImage.FromImage(cgImage);
return croppedImage;
}
else
{
return image;
}
};
}
This will show the Original Size of Image :
Now you can modify the cropped size as follow :
UIImageView imageView = new UIImageView(new CGRect(0, 400, UIScreen.MainScreen.Bounds.Size.Width, 300));
UIImage image = new UIImage("th.jpg");
imageView.Image = CropImage(image, new CGRect(0, 0, 1920, 100), 1);
View.AddSubview(imageView);
Here I set x = 0 , y = 0 , that means from (0,0) to start , and width is 1920 ,height is 100 . I just crop the height of the original Image . The effect as follow :
Then if you modify the x/y ,the cropped image will move to other area to crop .As follow:
UIImageView imageView = new UIImageView(new CGRect(0, 400, UIScreen.MainScreen.Bounds.Size.Width, 300));
UIImage image = new UIImage("th.jpg");
imageView.Image = CropImage(image, new CGRect(0, 100, 1920, 100), 1);
View.AddSubview(imageView);
Then you will see it's different with the second effect :
So when cropping an image , you should understand the drawRect of image.CGImage.WithImageInRect(drawRect) clearly .
Note from doc :
Be sure to specify the subrectangle's coordinates relative to the original image's full size, even if the UIImageView shows only a scaled version.
I'm currently implementing some sort of coloring book and I'm curious about the best way to change pixels in UIImage. Here is my code:
self.context = CGContext(data: nil, width: image.width, height: image.height, bitsPerComponent: 8, bytesPerRow: image.width * 4, space: colorSpace, bitmapInfo: CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.premultipliedFirst.rawValue)!
self.context?.draw(image.cgImage, in: CGRect(x: 0, y: 0, width: CGFloat(image.width), height: CGFloat(image.height)))
let ptr = context.data
self.pixelBuffer = ptr!.bindMemory(to: UInt32.self, capacity: image.width * image.height)
And change pixels using this function:
#inline (__always) func fill(matrixPosition: MatrixPosition, color: UInt32) {
pixelsBuffer?[self.linearIndex(for: matrixPosition)] = color
}
The problem is that every time when I change pixels I have to invoke makeImage on context to generate new image and it takes a lot of time:
func generateImage() -> UIImage {
let cgImage = context.makeImage()!
let uiimage = UIImage(cgImage: cgImage)
return uiimage
}
Does my approach is correct? What are better and faster ways to implement it? Thanks.
Manipulating individual pixels and then copying the entire memory buffer to a CGContext and then creating a UIImage with that context is going to end up being inefficient, as you are discovering.
You can continue to improve and optimize a CoreGraphics canvas approach by being more efficient about what part of your offscreen is copied onto screen. You can detect the pixels that have changed and only copy the minimum bounding rectangle of those pixels onto screen. This approach may be good enough for your use case where you are only filling in areas with colors.
Instead of copying the entire offscreen, copy just the changed area:
self.context?.draw(image.cgImage, in: CGRect(x: diffX, y: diffY, width: diffWidth, height: diffHeight))
It is up to you to determine the changed rectangle and when to update the screen.
Here is an example of a painting app that uses CoreGraphics, CoreImage and CADisplayLink. The code is a bit old, but the concepts are still valid and will serve as a good starting point. You can see how the changes are accumulated and drawn to the screen using a CADisplayLink.
If you want to introduce various types of ink and paint effects, a CoreGraphics approach is going to be more challenging. You will want to look at Apple's Metal API. A good tutorial is here.
is it possible to get back original image from image ROI? for example say we have
cv::Mat image = imread("image.jpg", 0);
cv::Mat imageROI = image(0, 0, 100, 100);
myFunction(imageROI);
and in myFunction I want to work with original image. is there any way to convert imageROI to original image when we don't access the original image?
I don't know if I understood the question exactly like you think, but if you ask if let's say we have header
void myFunc(cv::Mat &m);
// .... later on
cv::Mat image = imread("image.jpg", 0);
cv::Mat imageROI = image(0, 0, 100, 100);
myFunction(imageROI);
// .... later on myFuncDefinition
void myFunc(cv::Mat &m) {
// some code
// here you would like to have an original image, right?
}
So the answer for that is no and the proof is by simplicity: why want you to design opencv api in such way to make it possible store unnecessary data? If you do
cv::Mat imageROI = image(0, 0, 100, 100);
by purpose you would like to forgot about entire image and you are particulary interested in some ROI. Mat container is designed in such way to copy only matrix 'headers' and not matrix content. So if you do cv::Mat imageROI = image(0, 0, 100, 100) perhaps the matrix content (ie image data) might be stored somewhere in memory (because roi is the part of it, so by optimalization purposes it might no be deleted even is original image variable went out of scope), but your matrix header changed. Namely, from pointing to (0, 0, imageWisth, imageHeight) to (0, 0, 100, 100) and there's no way to bring it back just using variable m.
Why don't pass additional parameter as a reference?
Just incase anybody looks at this question you can actually do this
cv::Mat mat = ...
cv::Size size;
cv::Point offset;
// find original image size, and get offset of roi
mat.locateROI(size, offset);
// put image back to original size;
mat.adjustROI(offset.y, size.height - mat.rows, offset.x, size.width- mat.cols);
I'm try to create application where I'm use a UIScrolView and I can draw different jpeg images on the appropriate point for each of this images. It is must looks like a video, for example:
I have a first jpeg image (background), and this image (for example image has coordinates: x = 0, y = 0, and dimentions: width = 1024, height = 768), I send to UIScrolView for showing. Then I get enother jpeg image with their own coordinatates and dimentions(for example x = 10, y = 10, width = 100, height = 100) and I need show this "smal" image over "background".
How I can di that?
Or another way of this issue. Example:
I have a BMP data of first image(bacground). How I can send this BMP data to the UIScrolView for showing?\
P.S. One more question: Which of presented above ways will run faster for processing and presentation of data?
WBR
Maxim Tartachnik
if I want to draw a filled rectangle, I could do
g.fillRect(0, 0, 100, 100)
But if I want to draw the same filled rectangle with drawImage, what do I do? I tried the below:
g.drawImage(null, 0,0,100, 100, Color.BLACK, null);
And it does not work. I am not going to use an image, but must the Image still be non-null? What would be the correct way to do this?