I want to draw a 2d array on iOS in fast time. The 2d array is like heat map, depth map or segmentation map, etc.
In my case, with UIKit framework, drawing big size array, like 500x500 shape, is too slow.
// my solution, but it's too slow
override func draw(_ rect: CGRect) {
guard let ctx = UIGraphicsGetCurrentContext(),
let heatmap = self.heatmap else { return }
ctx.clear(rect);
let size = self.bounds.size
let heatmap_w = heatmap.count
let heatmap_h = heatmap.first?.count ?? 0
let w = size.width / CGFloat(heatmap_w)
let h = size.height / CGFloat(heatmap_h)
for j in 0..<heatmap_w {
for i in 0..<heatmap_h {
let value = heatmap[i][j]
let alpha: CGFloat = CGFloat(value)
guard alpha > 0 else { continue; }
let rect: CGRect = CGRect(x: CGFloat(i) * w, y: CGFloat(j) * h, width: w, height: h)
let color: UIColor = UIColor(red: 1.0, green: 0.0, blue: 0.0, alpha: alpha*0.58)
let bpath: UIBezierPath = UIBezierPath(rect: rect)
color.set()
bpath.stroke()
bpath.fill()
}
}
} // end of draw(rect:)
I think Metal or CoreGraphics framework are related to this problem, but I couldn't find the proper example or material. Are there any recommended ways?
heatmap example
updated:
Here is segmentation post-processing implementation example with MetalKit. The latency of post-processing was down to 1 ms from 240 ms on iPhone 11 Pro.
I referred MetalCamera a lot.
new updated:
Here is depth prediction post-processing implementation example with MetalKit and Accelerate framework. The latency of post-processing was down to 1 ms from 15 ms on iPhone 11 Pro.
The best way is to use Metal Compute Function to achieve maximum performance.
Apple has a good tutorial that shows the basics. Additionally you can use Metal Performance Shaders to implement custom rendering functions or to take advantage of a large library of existing functions.
Related
I've got a pixel art game app that uses UIKit for its menus and SpriteKit for the gameplay scene. The pixels are getting blurred due to anti-aliasing.
With the sprites I can turn off the anti-aliasing using...
node.texture?.filteringMode = .nearest
but in UIKit I can't find a way to turn off the anti-aliasing in the UIImageView's.
I saw this post but there's no example and the answer wasn't accepted. Not sure how to turn it off using CGContextSetShouldAntialias, or where to call it.
Based on the example I found here, tried using this subclass but it didn't seem to make a difference; according to my breakpoints the method is never called:
class NonAliasingView: UIImageView {
override func draw(_ rect: CGRect) {
guard let ctx = UIGraphicsGetCurrentContext() else { return }
// fill background with black color
ctx.addRect(bounds)
ctx.setFillColor(UIColor.black.cgColor)
ctx.fillPath()
if let img = image {
let pixelSize = CGSize(width: img.size.width * layer.contentsScale, height: img.size.height * layer.contentsScale)
UIGraphicsBeginImageContextWithOptions(pixelSize, true, 1)
guard let imgCtx = UIGraphicsGetCurrentContext() else { return }
imgCtx.setShouldAntialias(false)
img.draw(in: CGRect(x: 0, y: 0, width: pixelSize.width, height: pixelSize.height))
guard let cgImg = imgCtx.makeImage() else { return }
ctx.scaleBy(x: 1, y: -1)
ctx.translateBy(x: 0, y: -bounds.height)
ctx.draw(cgImg, in: CGRect(x: (bounds.width - img.size.width) / 2, y: (bounds.height - img.size.height) / 2, width: img.size.width, height: img.size.height))
}
}
}
Here's the code from my view controller where I tried to implement the subclass (modeImage is an IBOutlet to a UIImageView):
// modeImage.image = gameMode.displayImage
let modeImg = NonAliasingView()
modeImg.image = gameMode.displayImage
modeImage = modeImg
If I try to use UIGraphicsGetCurrentContext in the view controller it is nil and never passes the guard statement.
I've confirmed view.layer.allowsEdgeAntialiasing defaults to false. I don't need anti-aliasing at all, so if there's a way to turn off anti-aliasing app wide, or in the whole view controller, I'd be happy to use it.
How do you disable anti-aliasing with a UIImageView in UIKit?
UPDATE
Added imgCtx.setShouldAntialias(false) to method but still not working.
To remove all antialiasing on your image view and just use nearest-neighbor filtering, set the magnificationFilter and minificationFilter of the image view's layer to CALayerContentsFilter.nearest, as in:
yourImageView.layer.magnificationFilter = .nearest
yourImageView.layer.minificationFilter = .nearest
I would like to achieve the effect shown in this gif.
Currently I do this with a series of ~7 png images with a red background and a white line, these are animated through the sprite with an SKAction.
There are a few others additions I would like to make to the sprite, that can change depending on situation, and also I would like to repeat this with a number of colours.
This results in: 6 colours, 7 shine effects, 5 edge effects and 4 corner effects resulting in 136 combinations of textures I would need to create and store.
I feel like there has to be a way to superimpose png's with transparent backgrounds when setting the texture of a sprite but I cannot seem to find a way to do this anywhere.
Is this possible so that I can reduce the number of assets to 22 or do I have to make all 136 and build in logic to the class to decide which to use?
I wanted an effect like this for my game, I tried a lot of options. I tried using particles for performance but couldn't even get close. I know you can accomplish it with Shaders, but i didn't go that route and in iOS 12 Shaders won't be support Open GL anyway. In the end I opted to go with CropNodes.
This is my glare image, it is hard to see because it slightly transparent whiteish image.
This is the results I achieved using CropNodes
class Glare: SKSpriteNode {
var glare = SKSpriteNode()
private var cropNode = SKCropNode()
init(color: UIColor, size: CGSize) {
super.init(texture: nil, color: color, size: size)
alpha = 0.7
zPosition = 10
setupGlare()
}
required init?(coder aDecoder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
func setupGlare() {
let buffer: CGFloat = 10
let mask = SKSpriteNode(texture: nil, color: .black, size: self.size)
let texture = SKTextureAtlas(named: "Sprites").textureNamed("glare")
glare = SKSpriteNode(texture: texture)
glare.position = CGPoint(x: 0 - (self.size.width / 2 + buffer), y: self.size.height / 2 + buffer)
glare.setScale(3.50)
glare.zPosition = 1
cropNode.zPosition = 2
cropNode.maskNode = mask
cropNode.addChild(glare)
let random = Double(CGFloat.random(min: 0, max: 1))
let pause = SKAction.wait(forDuration: random)
let move = SKAction.moveBy(x: self.size.width + buffer * 2, y: 0 - (self.size.height + buffer * 2), duration: 0.5)
let wait = SKAction.wait(forDuration: 1.0)
let reset = SKAction.moveBy(x: 0 - (self.size.width + buffer * 2), y: self.size.height + buffer * 2, duration: 0.0)
let seq = SKAction.sequence([move, wait, reset])
let repeater = SKAction.repeatForever(seq)
let repeaterSeq = SKAction.sequence([pause, repeater])
glare.run(repeaterSeq)
}
func showGlare(texture: SKTexture) {
let mask = SKSpriteNode(texture: texture)
cropNode.maskNode = mask
glare.isHidden = false
if cropNode.parent == nil { self.addChild(cropNode)}
}
func hideGlare() {
glare.isHidden = true
//remove cropnode from the node tree
cropNode.removeFromParent()
}
}
and then in my GameScene...
I add my glares to a glare layer but that isn't necessary. I also go through when the game loads and create my glares for all 15 slots ahead of time and put them in an array. This way I do not have to create them on the fly and I can just turn on slot 10 any time I want and turn it off as well.
private var glares = [Glare]()
let glare = Glare(color: .clear, size: CGSize(width: kSymbolSize, height: kSymbolSize))
glare.position = CGPoint(x: (CGFloat(x - 1) * kSymbolSize) + (kSymbolSize / 2), y: 0 - (CGFloat(y) * kSymbolSize) + (kSymbolSize / 2))
glare.zPosition = 100
glareLayer.addChild(glare)
glares.append(glare)
When I want to show the glare on a slot
EDIT texture here for you would just be a blank square texture the size of your tile.
glares[index].showGlare(texture: symbol.texture!)
When I want to hide it
glares[index].hideGlare()
is there a way to improve the speed / performance of drawing pixel per pixel into a UIView?
The current implementation of a 500x500 pixel UIView, is terribly slow.
class CustomView: UIView {
public var context = UIGraphicsGetCurrentContext()
public var redvalues = [[CGFloat]](repeating: [CGFloat](repeating: 1.0, count: 500), count: 500)
public var start = 0
{
didSet{
self.setNeedsDisplay()
}
}
override func draw(_ rect: CGRect
{
super.draw(rect)
context = UIGraphicsGetCurrentContext()
for yindex in 0...499{
for xindex in 0...499 {
context?.setStrokeColor(UIColor(red: redvalues[xindex][yindex], green: 0.0, blue: 0.0, alpha: 1.0).cgColor)
context?.setLineWidth(2)
context?.beginPath()
context?.move(to: CGPoint(x: CGFloat(xindex), y: CGFloat(yindex)))
context?.addLine(to: CGPoint(x: CGFloat(xindex)+1.0, y: CGFloat(yindex)))
context?.strokePath()
}
}
}
}
Thank you very much
When drawing individual pixels, you can use a bitmap context. A bitmap context takes raw pixel data as an input.
The context copies your raw pixel data so you don't have to use paths, which are likely much slower. You can then get a CGImage by using context.makeImage().
The image can then be used in an image view, which would eliminate the need to redraw the whole thing every frame.
If you don't want to manually create a bitmap context, you can use
UIGraphicsBeginImageContext(size)
let context = UIGraphicsGetCurrentContext()
// draw everything into the context
let image = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
Then you can use a UIImageView to display the rendered image.
It is also possible to draw into a CALayer, which does not need to be redrawn every frame but only when resized.
That's how it looks now, are there any optimizations possible or not?
public struct rgba {
var r:UInt8
var g:UInt8
var b:UInt8
var a:UInt8
}
public let imageview = UIImageView()
override func viewDidLoad() {
super.viewDidLoad()
let width_input = 500
let height_input = 500
let redPixel = rgba(r:255, g:0, b:0, a:255)
let greenPixel = rgba(r:0, g:255, b:0, a:255)
let bluePixel = rgba(r:0, g:0, b:255, a:255
var pixelData = [rgba](repeating: redPixel, count: Int(width_input*height_input))
pixelData[1] = greenPixel
pixelData[3] = bluePixel
self.view.addSubview(imageview)
imageview.frame = CGRect(x: 100,y: 100,width: 600,height: 600)
imageview.image = draw(pixel: pixelData,width: width_input,height: height_input)
}
func draw(pixel:[rgba],width:Int,height:Int) -> UIImage
{
let colorSpace = CGColorSpaceCreateDeviceRGB()
let data = UnsafeMutableRawPointer(mutating: pixel)
let bitmapContext = CGContext(data: data,
width: width,
height: height,
bitsPerComponent: 8,
bytesPerRow: 4*width,
space: colorSpace,
bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue)
let image = bitmapContext?.makeImage()
return UIImage(cgImage: image!)
}
I took the answer from Manuel and got it working in Swift 5. The main sticking point here was to clear the dangling pointer warning now in Xcode 12.
var image:CGImage?
pixelData.withUnsafeMutableBytes( { (rawBufferPtr: UnsafeMutableRawBufferPointer) in
if let rawPtr = rawBufferPtr.baseAddress {
let bitmapContext = CGContext(data: rawPtr,
width: width,
height: height,
bitsPerComponent: 8,
bytesPerRow: 4*width,
space: colorSpace,
bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue)
image = bitmapContext?.makeImage()
}
})
I did have to move away from the rgba struct approach for front loading the data and moved to direct UInt32 values derived from rawValues in the enum. The 'append' or 'replaceInRange' approach to updating an existing array took hours (my bitmap was LARGE) and ended up exhausting swap space on my computer.
enum Color: UInt32 { // All 4 bytes long with full opacity
case red = 4278190335 // 0xFF0000FF
case yellow = 4294902015
case orange = 4291559679
case pink = 4290825215
case violet = 4001558271
case purple = 2147516671
case green = 16711935
case blue = 65535 // 0x0000FFFF
}
With this approach I was able to quickly build a Data buffer with that data amount via:
func prepareColorBlock(c:Color) -> Data {
var rawData = withUnsafeBytes(of:c.rawValue) { Data($0) }
rawData.reverse() // Byte order is reveresed when defined
var dataBlock = Data()
dataBlock.reserveCapacity(100)
for _ in stride(from: 0, to: 100, by: 1) {
dataBlock.append(rawData)
}
return dataBlock
}
With that I just appended each of these blocks into my mutable Data instance 'pixelData' and we are off. You can tweak how the data is assembled, as I just wanted to generate some color bars in a UIImageView to validate the work. For a 800x600 view, it took about 2.3 seconds to generate and render the whole thing.
Again, hats off to Manuel for pointing me in the right direction.
I need your advice, I'm new in SpriteKit I need to make animation strips. I have 3 solutions to make it, but I need advice that better and less costly for CPU.
1 solution: each stripe - SKSpriteNode with animation and texture
2 solution: background video
3 solution: each stripe - SKShapeNode with animation
This is a simple task , you don't need to build an atlas animation or use SKShapeNode, you can use SKSpriteNode as this code:
var bar = SKSpriteNode(color: SKColor.greenColor(), size: CGSizeMake(40, 200))
barra.position = CGPoint(x: CGRectGetMidX(self.frame), y: CGRectGetMidY(self.frame))
self.addChild(barra)
Build n bars with random size, and use SKAction to move them.
Whith this approach your animation will be different everytime you launch it.
Code in details:
class GameScene: SKScene {
var bars: [SKSpriteNode]!
var totBars : Int = 50
override func didMoveToView(view: SKView) {
self.backgroundColor = SKColor(red: 131/255, green: 190/255, blue: 177/255, alpha: 1)
let redBarColor = SKColor(red: 204/255, green: 75/255, blue: 75/255, alpha: 1)
let yellowBar = SKColor(red: 253/255, green: 242/255, blue: 160/255, alpha: 1)
// add here your colors
var colorSelected:SKColor = redBarColor
bars = [SKSpriteNode]()
for i in 0..<totBars-1 {
let colorNum = randomNumber(1...2)
switch (colorNum) {
case 1:
colorSelected = redBarColor
case 2:
colorSelected = yellowBar
default:
break
}
let randomWidth = randomCGFloat(5,max:40)
let randomHeight = randomCGFloat(30,max:400)
let bar = SKSpriteNode.init(color: colorSelected, size: CGSizeMake(randomWidth, randomHeight))
bar.zRotation = -45 * CGFloat(M_PI / 180.0)
bar.name = "bar\(i)"
self.addChild(bar)
bars.append(bar)
}
animateBars()
}
func animateBar(bar:SKSpriteNode) {
print("- \(bar.name) start!")
let deltaX = self.randomCGFloat(0,max:self.frame.maxX)
let deltaY:CGFloat = self.frame.maxY/2
let rightPoint = CGPointMake(self.frame.maxX + deltaX,self.frame.maxY + deltaY)
let leftPoint = CGPointMake(-self.frame.maxX + deltaX,-self.frame.maxY + deltaY)
bar.position = rightPoint
let waitBeforeExit = SKAction.waitForDuration(Double(self.randomCGFloat(1.0,max:2.0)))
let speed = self.randomCGFloat(150,max:300)
let move = SKAction.moveTo(leftPoint, duration: self.getDuration(rightPoint, pointB: leftPoint, speed: speed))
bar.runAction(SKAction.sequence([waitBeforeExit,move]), completion: {
print("\(bar.name) reached position")
self.animateBar(bar)
})
}
func animateBars() {
for bar in bars {
animateBar(bar)
}
}
func getDuration(pointA:CGPoint,pointB:CGPoint,speed:CGFloat)->NSTimeInterval {
let xDist = (pointB.x - pointA.x)
let yDist = (pointB.y - pointA.y)
let distance = sqrt((xDist * xDist) + (yDist * yDist));
let duration : NSTimeInterval = NSTimeInterval(distance/speed)
return duration
}
func randomCGFloat(min: CGFloat, max: CGFloat) -> CGFloat {
return CGFloat(Float(arc4random()) / Float(UINT32_MAX)) * (max - min) + min
}
func randomNumber(range: Range<Int> = 1...6) -> Int {
let min = range.startIndex
let max = range.endIndex
return Int(arc4random_uniform(UInt32(max - min))) + min
}
}
If your stripes are simply plain rectangles, you can use SKSpriteNodes and only give them dimensions and a color, then use actions to animate them. you can rotate the rectangles to give the effect you show in the picture.
You could actually build the whole animation in Xcode using the editor, save it in its own SKS file and load the animation using a SKReferenceNode.
To answer your question, Solution 3 is the least costly method for your CPU. Here're several reasons why this is true:
1. The first solution that you're suggesting SKSpriteNode with animation and texture" require the computer to load the texture to the view. If you have multiple .png files, this would mean that the computer would need to load all of these files. But if you were to use SKShapeNode, you would avoid having to do this and you would cheaply create the same looks as the SKShapeNode is not based on an image.
2. Solution 2 is the most costly to your CPU because running a video. This takes a lot of space in your memory which might even create lags if you run it on your phone.
Also, just another thing to note: If you were to extensively use Solution 1 and 2, you would end up using so much memory. But if you use the Solution 3, you will not deal with this.
Another option, in addition to the ones put forth already, would be to write a GLSL shader for the background. If you just want a cool backdrop that doesn't interact with the rest of your game, this would probably be the most performant option, since it would all happen on the GPU.
You could even, conceivably, render the rectangles without requiring any images at all, since they are just areas of color.
There's a decent introduction to the topic here: Battle of Brothers GLSL Introduction.
I am trying to add a vertical scrolling background to my project. From what I scene on the internet. My background consists of 8 images, each [320x1000px].png files. So what I ended up doing for it was this:
//Layered Nodes
var backgroundNode: SKNode!
override init(size: CGSize) {
super.init(size: size)
scaleFactor = self.size.width / 320.0
// Background
backgroundNode = createBackgroundNode()
addChild(backgroundNode)
}
func createBackgroundNode() -> SKNode {
let backgroundNode = SKNode()
let ySpacing = 1000.0 * scaleFactor
for index in 0...3 {
let node = SKSpriteNode(imageNamed:String(format: "bg%d", index + 1))
node.setScale(scaleFactor)
node.anchorPoint = CGPoint(x: 0.5, y: 0.0)
node.position = CGPoint(x: self.size.width / 2, y: ySpacing * CGFloat(index))
backgroundNode.addChild(node)
}
return backgroundNode
}
Problem is, they use up to 50Mb of the project. I am trying to find a way to do it so it would take much less memory off my game but I can't seem to find it. Is there anything wrong with this? If not, should I best focus on other parts of the project and keep it this way?
You can create background on the go from vector drawing using paint-code app.
Another point. Are your images optimized? If no you can use ImageOptim for free.