I'm learning OpenGL according to this tutorial, and trying to write simple function that draws colored rectangle.
Input params:
size: CGSize,
scale: CGFloat,
This part of code works ok:
// Setup context
let api = EAGLRenderingAPI.OpenGLES3
let context = EAGLContext(API: api)
EAGLContext.setCurrentContext(context)
// Setup render buffer
var renderBuffer = GLuint()
glGenRenderbuffers(1, &renderBuffer)
glBindRenderbuffer(GLenum(GL_RENDERBUFFER), renderBuffer)
let bufferWidth = GLsizei(size.width * scale)
let bufferHeight = GLsizei(size.height * scale)
let bufferFormat = GLenum(GL_RGBA8)
glRenderbufferStorage(GLenum(GL_RENDERBUFFER), bufferFormat, bufferWidth, bufferHeight)
// Setup frame buffer
var frameBuffer = GLuint()
glGenFramebuffers(1, &frameBuffer)
glBindFramebuffer(GLenum(GL_FRAMEBUFFER), frameBuffer)
glFramebufferRenderbuffer(GLenum(GL_FRAMEBUFFER), GLenum(GL_COLOR_ATTACHMENT0), GLenum(GL_RENDERBUFFER), renderBuffer)
// Draw
glClearColor(1, 0.5, 0.5, 1)
glClear(GLbitfield(GL_COLOR_BUFFER_BIT))
Most problematic part is to create an UIImage:
// Get bytes
let byteLength = Int(bufferWidth * bufferHeight) * 4;
let bytes = malloc(byteLength)
glReadPixels(0, 0, bufferWidth, bufferHeight, bufferFormat, GLenum(GL_UNSIGNED_BYTE), bytes)
print( glGetError() ) // prints 1280 (GL_INVALID_ENUM) !!!
// Create a CGImage
let dataProvider = CGDataProviderCreateWithData(nil, bytes, byteLength, nil)
let colorspace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo: CGBitmapInfo = [.ByteOrder32Little, CGBitmapInfo(rawValue: CGImageAlphaInfo.Last.rawValue)]
let aCGImage = CGImageCreate(
Int(bufferWidth),
Int(bufferHeight),
8,
4,
4 * Int(bufferWidth),
colorspace,
bitmapInfo,
dataProvider,
nil,
false,
.RenderingIntentDefault
)!
let anUIImage = UIImage(CGImage: aCGImage, scale: scale, orientation: .Up)
While glReadPixels generates GL_INVALID_ENUM error, I'm not even sure if my way to create UIImage is correct.
OpenGL ES has very limited formats that are accepted. There is an excellent website with OpenGL documentations http://docs.gl
You are interested in http://docs.gl/es2/glReadPixels or http://docs.gl/es3/glReadPixels. Buffer format should be GL_RGBA or GL_BGRA.
Maybe better approach would be https://stackoverflow.com/a/9704392/1351828.
Related
TL;DR: In legacy Obj-C code, the color space param value was NULL. That is not allowed in the Swift equivalent. What value to use?
I have inherited code that reads:
unsigned char pixel[1] = {0};
CGContextRef context = CGBitmapContextCreate(
pixel,1, 1, 8, 1, NULL, (CGBitmapInfo)kCGImageAlphaOnly
);
The port to Swift 4 CGContext is straightforward, except for that NULL color space value. Using a plausible value, I am getting nil back from CGContext.init?(). My translation is:
var pixelValue = UInt8(0)
var pixel = Data(buffer: UnsafeBufferPointer(start:&pixelValue, count:1))
let context = CGContext(
data : &pixel,
width : 1,
height : 1,
bitsPerComponent: 8,
bytesPerRow : 1,
space : CGColorSpace(name:CGColorSpace.genericRGBLinear)!,
bitmapInfo : CGImageAlphaInfo.alphaOnly.rawValue
)! // Returns nil; unwrapping crashes
Q: What is the appropriate value for space? (The value I provide is not returning nil; it's the CGContext() call itself.
Setting the environment variable CGBITMAP_CONTEXT_LOG_ERRORS yields an error log like this:
Assertion failed: (0), function get_color_model_name,
file /BuildRoot/Library/Caches/com.apple.xbs/Sources/Quartz2D_Sim/
Quartz2D-1129.2.1/CoreGraphics/API/CGBitmapContextInfo.c, line 210.
For some more backstory, the context was used to find the alpha value of a single pixel in a UIImage in the following way:
unsigned char pixel[1] = {0};
CGContextRef context = CGBitmapContextCreate(pixel,1, 1, 8, 1, NULL, (CGBitmapInfo)kCGImageAlphaOnly);
UIGraphicsPushContext(context);
[image drawAtPoint:CGPointMake(-point.x, -point.y)];
UIGraphicsPopContext();
CGContextRelease(context);
CGFloat alpha = pixel[0]/255.0;
(I do have possible alternatives for finding alpha, but in the interest of leaving legacy code alone, would like to keep it this way.)
I recently worked with similar topic, maybe this code sample will help someone:
let image = UIImage(named: "2.png")
guard let cgImage = image?.cgImage else {
fatalError()
}
let width = cgImage.width
let height = cgImage.height
//CGColorSpaceCreateDeviceGray - 1 component, 8 bits
//i.e. 1px = 1byte
let bytesPerRow = width
let bitmapByteCount = width * height
let bitmapData: UnsafeMutablePointer<UInt8> = .allocate(capacity: bitmapByteCount)
defer {
bitmapData.deallocate()
}
bitmapData.initialize(repeating: 0, count: bitmapByteCount)
guard let context = CGContext(data: bitmapData, width: width, height: height,
bitsPerComponent: 8, bytesPerRow: bytesPerRow,
space: CGColorSpaceCreateDeviceGray(), bitmapInfo: CGImageAlphaInfo.alphaOnly.rawValue) else {
fatalError()
}
//draw image to context
var rect = CGRect(x: 0, y: 0, width: width, height: height)
context.draw(cgImage, in: rect)
// Enumerate through all pixels
for row in 0..<height {
for col in 0..<width {
let alphaValue = bitmapData[row * width + col]
if alphaValue != 0 {
//visible pixel
}
}
}
Here’s how to determine whether a pixel is transparent:
let info = CGImageAlphaInfo.alphaOnly.rawValue
let pixel = UnsafeMutablePointer<UInt8>.allocate(capacity:1)
defer {
pixel.deinitialize(count: 1)
pixel.deallocate()
}
pixel[0] = 0
let sp = CGColorSpaceCreateDeviceGray()
let context = CGContext(data: pixel,
width: 1, height: 1, bitsPerComponent: 8, bytesPerRow: 1,
space: sp, bitmapInfo: info)!
UIGraphicsPushContext(context)
im.draw(at:CGPoint(-point.x, -point.y))
UIGraphicsPopContext()
let p = pixel[0]
let alpha = Double(p)/255.0
let transparent = alpha < 0.01
For the record, here is how I wound up doing it. It hasn't (yet) misbehaved, so on the principle of "If it ain't broke, don't fix it" I'll leave it. (I have added self for clarity.) But you can be sure that I will paste Matt's code right in there, in case I need it in the future. Thanks Matt!
// Note that "self" is a UIImageView; "point" is the point under consideration.
let im = self.image!
// TODO: Why is this clamping necessary? We get points outside our size.
var x = point.x
var y = point.y
if x < 0 { x = 0 } else if x > im.size.width - 1 { x = im.size.width - 1 }
if y < 0 { y = 0 } else if y > im.size.height - 1 { y = im.size.height - 1 }
let screenWidth = self.bounds.width
let intrinsicWidth = im.size.width
x *= im.scale * intrinsicWidth/screenWidth
y *= im.scale * intrinsicWidth/screenWidth
let pixData = im.cgImage?.dataProvider?.data
let data = CFDataGetBytePtr(pixData!)
let pixIndex = Int(((Int(im.size.width*im.scale) * Int(y)) + Int(x)) * 4)
let r = data?[pixIndex]
let g = data?[pixIndex + 1]
let b = data?[pixIndex + 2]
let α = data?[pixIndex + 3]
let red = CGFloat(r!)/255
let green = CGFloat(g!)/255
let blue = CGFloat(b!)/255
let alpha = CGFloat(α!)/255
I'm using a CGBitMapContext() to convert colour spaces to ARGB and get the pixel data values, I malloc space for bit map context and free it after I'm done but am still seeing a Memory Leak in Instruments I'm thinking I'm likely doing something wrong so any help would be appreciated.
Here is the ARGBBitmapContext function
func createARGBBitmapContext(width: Int, height: Int) -> CGContext {
var bitmapByteCount = 0
var bitmapBytesPerRow = 0
//Get image width, height
let pixelsWide = width
let pixelsHigh = height
bitmapBytesPerRow = Int(pixelsWide) * 4
bitmapByteCount = bitmapBytesPerRow * Int(pixelsHigh)
let colorSpace = CGColorSpaceCreateDeviceRGB()
// Here is the malloc call that Instruments complains of
let bitmapData = malloc(bitmapByteCount)
let context = CGContext(data: bitmapData, width: pixelsWide, height: pixelsHigh, bitsPerComponent: 8, bytesPerRow: bitmapBytesPerRow, space: colorSpace, bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue)
// Do I need to free something here first?
return context!
}
Here is where I use the context to retrieve all the pixel values as a list of UInt8s (and where the memory leaks)
extension UIImage {
func ARGBPixelValues() -> [UInt8] {
let width = Int(self.size.width)
let height = Int(self.size.height)
var pixels = [UInt8](repeatElement(0, count: width * height * 3))
let rect = CGRect(x: 0, y: 0, width: width, height: height)
let context = createARGBBitmapContext(inImage: self.cgImage!)
context.clear(rect)
context.draw(self.cgImage!, in: rect)
var location = 0
if let data = context.data {
while location < (width * height) {
let arrOffset = 3 * location
let offset = 4 * (location)
let R = data.load(fromByteOffset: offset + 1, as: UInt8.self)
let G = data.load(fromByteOffset: offset + 2, as: UInt8.self)
let B = data.load(fromByteOffset: offset + 3, as: UInt8.self)
pixels[arrOffset] = R
pixels[arrOffset+1] = G
pixels[arrOffset+2] = B
location += 1
}
free(context.data) // Free the data consumed, perhaps this isn't right?
}
return pixels
}
}
Instruments reports a malloc error of 1.48MiB which is right for my image size (540 x 720) I free the data but apparently that is not right.
I should mention that I know you can pass nil to CGContext init (and it will manage memory) but I'm more curious why using malloc creates an issue is there something more I should know (I'm more familiar with Obj-C).
Because CoreGraphics is not handled by ARC (like all other C libraries), you need to wrap your code with with an autorelease, even in Swift. Particularly if you are not on the main thread (which you should not be, if CoreGraphics is involved... .userInitiated or lower is appropriate).
func myFunc() {
for _ in 0 ..< makeMoneyFast {
autoreleasepool {
// Create CGImageRef etc...
// Do Stuff... whir... whiz... PROFIT!
}
}
}
For those that care, your Objective-C should also be wrapped like:
BOOL result = NO;
NSMutableData* data = [[NSMutableData alloc] init];
#autoreleasepool {
CGImageRef image = [self CGImageWithResolution:dpi
hasAlpha:hasAlpha
relativeScale:scale];
NSAssert(image != nil, #"could not create image for TIFF export");
if (image == nil)
return nil;
CGImageDestinationRef destRef = CGImageDestinationCreateWithData((CFMutableDataRef)data, kUTTypeTIFF, 1, NULL);
CGImageDestinationAddImage(destRef, image, (CFDictionaryRef)options);
result = CGImageDestinationFinalize(destRef);
CFRelease(destRef);
}
if (result) {
return [data copy];
} else {
return nil;
}
See this answer for details.
I'm trying implement GLKit texture painting, and it's looks too jagged
Creating texture from .png, got this code from Apple's GLPaing example
private func texture(fromName name: String) -> textureInfo_t {
var texId: GLuint = 0
var texture: textureInfo_t = (0, 0, 0)
let brushImage = UIImage(named: name)!.cgImage!
let width: size_t = brushImage.width
let height: size_t = brushImage.height
var brushData = [GLubyte](repeating: 0, count: width * height * 4)
let bitmapInfo = CGImageAlphaInfo.premultipliedLast.rawValue
let brushContext = CGContext(data: &brushData, width: width, height: height, bitsPerComponent: 8, bytesPerRow: width * 4, space: (brushImage.colorSpace!), bitmapInfo: bitmapInfo)
brushContext?.draw(brushImage, in: CGRect(x: 0.0, y: 0.0, width: width.g, height: height.g))
glGenTextures(1, &texId)
// Bind the texture name.
glBindTexture(GL_TEXTURE_2D.ui, texId)
// Set the texture parameters to use a minifying filter and a linear filer (weighted average)
glTexParameteri(GL_TEXTURE_2D.ui, GL_TEXTURE_MIN_FILTER.ui, GL_LINEAR)
// Specify a 2D texture image, providing the a pointer to the image data in memory
glTexImage2D(GL_TEXTURE_2D.ui, 0, GL_RGBA, width.i, height.i, 0, GL_RGBA.ui, GL_UNSIGNED_BYTE.ui, brushData)
// Release the image data; it's no longer needed
texture.id = texId
texture.width = width.i
texture.height = height.i
return texture
}
and rendering while painting
private func renderLine(from _start: CGPoint, to _end: CGPoint) {
struct Static {
static var vertexBuffer: [GLfloat] = []
}
var count = 0
EAGLContext.setCurrent(context)
glBindFramebuffer(GL_FRAMEBUFFER.ui, viewFramebuffer)
// Convert locations from Points to Pixels
let scale = self.contentScaleFactor
var start = _start
start.x *= scale
start.y *= scale
var end = _end
end.x *= scale
end.y *= scale
// Allocate vertex array buffer
// Add points to the buffer so there are drawing points every X pixels
count = max(Int(ceilf(sqrtf((end.x - start.x).f * (end.x - start.x).f + (end.y - start.y).f * (end.y - start.y).f) / kBrushPixelStep.f)), 1)
Static.vertexBuffer.reserveCapacity(count * 2)
Static.vertexBuffer.removeAll(keepingCapacity: true)
for i in 0..<count {
Static.vertexBuffer.append(start.x.f + (end.x - start.x).f * (i.f / count.f))
Static.vertexBuffer.append(start.y.f + (end.y - start.y).f * (i.f / count.f))
}
// Load data to the Vertex Buffer Object
glBindBuffer(GL_ARRAY_BUFFER.ui, vboId)
glBufferData(GL_ARRAY_BUFFER.ui, count*2*MemoryLayout<GLfloat>.size, Static.vertexBuffer, GL_DYNAMIC_DRAW.ui)
glEnableVertexAttribArray(ATTRIB_VERTEX.ui)
glVertexAttribPointer(ATTRIB_VERTEX.ui, 2, GL_FLOAT.ui, GL_FALSE.ub, 0, nil)
// Draw
glUseProgram(program[PROGRAM_POINT].id)
glDrawArrays(GL_POINTS.ui, 0, count.i)
// Display the buffer
glBindRenderbuffer(GL_RENDERBUFFER.ui, viewRenderbuffer)
context.presentRenderbuffer(GL_RENDERBUFFER.l)
}
How could I improve texture quality?
UPDATE:
Even with bigger resolution .png result same. What is wrong am I doing?
There 1024x1024 .png with transparent background that I'm using:
I need to count all the black pixels in UIImage. I have found a code that could work however it is written in Objective-C. I have tried to convert it in swift but I get lots of errors and I cannot find the way of fix them.
Whats the best way to do this using Swift?
Simple Image
Objective-C:
/**
* Structure to keep one pixel in RRRRRRRRGGGGGGGGBBBBBBBBAAAAAAAA format
*/
struct pixel {
unsigned char r, g, b, a;
};
/**
* Process the image and return the number of pure red pixels in it.
*/
- (NSUInteger) processImage: (UIImage*) image
{
NSUInteger numberOfRedPixels = 0;
// Allocate a buffer big enough to hold all the pixels
struct pixel* pixels = (struct pixel*) calloc(1, image.size.width * image.size.height * sizeof(struct pixel));
if (pixels != nil)
{
// Create a new bitmap
CGContextRef context = CGBitmapContextCreate(
(void*) pixels,
image.size.width,
image.size.height,
8,
image.size.width * 4,
CGImageGetColorSpace(image.CGImage),
kCGImageAlphaPremultipliedLast
);
if (context != NULL)
{
// Draw the image in the bitmap
CGContextDrawImage(context, CGRectMake(0.0f, 0.0f, image.size.width, image.size.height), image.CGImage);
// Now that we have the image drawn in our own buffer, we can loop over the pixels to
// process it. This simple case simply counts all pixels that have a pure red component.
// There are probably more efficient and interesting ways to do this. But the important
// part is that the pixels buffer can be read directly.
NSUInteger numberOfPixels = image.size.width * image.size.height;
while (numberOfPixels > 0) {
if (pixels->r == 255) {
numberOfRedPixels++;
}
pixels++;
numberOfPixels--;
}
CGContextRelease(context);
}
free(pixels);
}
return numberOfRedPixels;
}
Much faster is to use Accelerate's vImageHistogramCalculation to get a histogram of the different channels in your image:
let img: CGImage = CIImage(image: image!)!.cgImage!
let imgProvider: CGDataProvider = img.dataProvider!
let imgBitmapData: CFData = imgProvider.data!
var imgBuffer = vImage_Buffer(data: UnsafeMutableRawPointer(mutating: CFDataGetBytePtr(imgBitmapData)), height: vImagePixelCount(img.height), width: vImagePixelCount(img.width), rowBytes: img.bytesPerRow)
let alpha = [UInt](repeating: 0, count: 256)
let red = [UInt](repeating: 0, count: 256)
let green = [UInt](repeating: 0, count: 256)
let blue = [UInt](repeating: 0, count: 256)
let alphaPtr = UnsafeMutablePointer<vImagePixelCount>(mutating: alpha) as UnsafeMutablePointer<vImagePixelCount>?
let redPtr = UnsafeMutablePointer<vImagePixelCount>(mutating: red) as UnsafeMutablePointer<vImagePixelCount>?
let greenPtr = UnsafeMutablePointer<vImagePixelCount>(mutating: green) as UnsafeMutablePointer<vImagePixelCount>?
let bluePtr = UnsafeMutablePointer<vImagePixelCount>(mutating: blue) as UnsafeMutablePointer<vImagePixelCount>?
let rgba = [redPtr, greenPtr, bluePtr, alphaPtr]
let histogram = UnsafeMutablePointer<UnsafeMutablePointer<vImagePixelCount>?>(mutating: rgba)
let error = vImageHistogramCalculation_ARGB8888(&imgBuffer, histogram, UInt32(kvImageNoFlags))
After this runs, alpha, red, green, and blue are now histograms of the colors in your image. If red, green, and blue each only have count in the 0th spot, while alpha only has count in the last spot, your image is black.
If you want to not even check multiple arrays, you can use vImageMatrixMultiply to combine your different channels:
let readableMatrix: [[Int16]] = [
[3, 0, 0, 0]
[0, 1, 1, 1],
[0, 0, 0, 0],
[0, 0, 0, 0]
]
var matrix: [Int16] = [Int16](repeating: 0, count: 16)
for i in 0...3 {
for j in 0...3 {
matrix[(3 - j) * 4 + (3 - i)] = readableMatrix[i][j]
}
}
vImageMatrixMultiply_ARGB8888(&imgBuffer, &imgBuffer, matrix, 3, nil, nil, UInt32(kvImageNoFlags))
If you stick this in before the histograming, your imgBuffer will be modified in place to average the RGB in each pixel, writing the average out to the B channel. As such, you can just check the blue histogram instead of all three.
(btw, the best description of vImageMatrixMultiply I've found is in the source code, like at https://github.com/phracker/MacOSX-SDKs/blob/2d31dd8bdd670293b59869335d9f1f80ca2075e0/MacOSX10.7.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImage.framework/Versions/A/Headers/Transform.h#L21)
I ran into a similar issue now, where I needed to determine if an image was 100% black. The following code will return the number of pure black pixels it finds in an image.
However, if you want to bump the threshold up, you can change the compare value, and allow it to tolerate a wider range of possible colors.
import UIKit
extension UIImage {
var blackPixelCount: Int {
var count = 0
for x in 0..<Int(size.width) {
for y in 0..<Int(size.height) {
count = count + (isPixelBlack(CGPoint(x: CGFloat(x), y: CGFloat(y))) ? 1 : 0)
}
}
return count
}
private func isPixelBlack(_ point: CGPoint) -> Bool {
let pixelData = cgImage?.dataProvider?.data
let pointerData: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
let pixelInfo = Int(((size.width * point.y) + point.x)) * 4
let maxValue: CGFloat = 255.0
let compare: CGFloat = 0.01
if (CGFloat(pointerData[pixelInfo]) / maxValue) > compare { return false }
if (CGFloat(pointerData[pixelInfo + 1]) / maxValue) > compare { return false }
if (CGFloat(pointerData[pixelInfo + 2]) / maxValue) > compare { return false }
return true
}
}
You call this with:
let count = image.blackPixelCount
The one caveat is that this is a very slow process, even on small images.
I'm using OpenCV in my swift project to detect faces on pictures from an iPhone's gallery.
When I try to process a UIImage picked from the gallery, I'm first doing that :
+ (UIImage *)processImageWithOpenCV:(UIImage*)inputImage {
...
cv::Mat img = [inputImage CVGrayscaleMat];
...
}
Here is my CVGrayscaleMat method :
-(cv::Mat)CVGrayscaleMat
{
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceGray();
CGFloat cols = self.size.width;
CGFloat rows = self.size.height;
cv::Mat cvMat((int)rows, (int)cols, CV_8UC1); // 8 bits per component, 1 channels
CGContextRef contextRef = CGBitmapContextCreate(cvMat.data, // Pointer to data
(int)cols, // Width of bitmap
(int)rows, // Height of bitmap
8, // Bits per component
4896, // Bytes per row
colorSpace, // Colorspace
kCGImageAlphaNone |
kCGBitmapByteOrderDefault); // Bitmap info flags
CGContextDrawImage(contextRef, CGRectMake(0, 0, (int)cols, (int)rows), self.CGImage);
CGContextRelease(contextRef);
CGColorSpaceRelease(colorSpace);
return cvMat;
}
When I try to process a picture, I got a EXC_BAD_ACCESS on that line :
CGContextDrawImage(contextRef, CGRectMake(0, 0, (int)cols, (int)rows), self.CGImage);
I'm still not super familiar with OpenCV... Any idea ?
Thanks in advance !
SOLVED :
Using the standard function - (cv::Mat)cvMatFromUIImage:(UIImage *)image