Add shader for SKShapeNode - ios

I want to add drop shadow effect for a SKShapeNode. I found a Emboss shader here.
Here is my code:
let node = SKShapeNode(rectOf: CGSize(width: w, height: h),cornerRadius: w / 4.0)
node.position = CGPoint(x: x, y: frame.size.height + h / 2)
node.fillColor = color
node.strokeColor = .clear
node.fillShader = createColorEmboss()
let v = simd_make_float2(Float(w),Float(h))
node.setValue(SKAttributeValue(vectorFloat2: v), forAttribute: "a_size")
func createColorEmboss() -> SKShader {
let source = "void main() {" +
"vec4 current_color = SKDefaultShading();" +
"if (current_color.a > 0.0) {" +
"vec2 pixel_size = 1.0 / a_size;" +
"vec4 new_color = current_color;" +
"new_color += texture2D(u_texture, v_tex_coord + pixel_size) * u_strength;" +
"new_color -= texture2D(u_texture, v_tex_coord - pixel_size) * u_strength;" +
"gl_FragColor = vec4(new_color.rgb, 1) * current_color.a * v_color_mix.a;" +
"} else {" +
"gl_FragColor = current_color;" +
"}" +
"}"
let shader = SKShader(source: source, uniforms: [SKUniform(name: "u_strength", float: 1)])
shader.attributes = [SKAttribute(name: "a_size", type: .vectorFloat2)]
return shader
}
But nothing happened. Any idea?

There are some big differences between a textured (*) SKShapeNode and SKSpriteNode here:
If the SKSpriteNode has a texture and a custom shader (shader property), then the shader will have access to the original texture via the u_texture uniform, and it will be able use it for rendering the node, i.e. by applying various effects to it like emboss, blur, etc.
SKShapeNode may have a fillTexture and a fillShader, but the way they work is different:
fillTexture is a rectangular texture that's converted to grayscale and used to set the luminosity of fillColor in the areas of the shape node which are filled.
fillShader can only see fillTexture as this rectangular image before it gets used for the final render, with no way to modify, or even see, which areas of the rectangle will be visible (part of the fill), and how the final render will look.
If there is no explicit fillTexture set, the u_texture uniform seems to pretend that it's an endless expanse of whiteness: whatever coordinate you ask about, it will return the color white, even if you go out of its bounds.
(*) You can also create a SKSpriteNode with just a color; its behavior will be a weird mix of SKShapeNode and a textured SKSpriteNode, and it's not helpful for our discussion, so I'll be ignoring this kind.
What you can do is to rasterize your SKShapeNode into a texture, and create an SKSpriteNode from it. It's actually very simple:
let node = SKShapeNode(rectOf: CGSize(width: w, height: h),cornerRadius: w / 4.0)
node.fillColor = color
node.strokeColor = .clear
let theTexture = view.texture(from:node)
let spriteNode = SKSpriteNode(texture:theTexture)
spriteNode.position = CGPoint(x: x, y: frame.size.height + h / 2)
spriteNode.shader = createColorEmboss()
let v = simd_make_float2(Float(w),Float(h))
spriteNode.setValue(SKAttributeValue(vectorFloat2: v), forAttribute: "a_size")
As you can see, you don't even have to add the node to the scene; the view will render it into a texture, with a clear background. You can also use view.texture(from:crop:) to specify a larger crop that can accommodate a drop shadow if it extends beyond the original shape.
I must warn you, however, that this particular shader you are trying to use may not be what you need for a drop shadow…

Related

How do I tell a CIKernel to only process specific regions of an image?

I have a simple CIKernel that tints pixel red:
extern "C" float4 BasicFilter (coreimage::sampler input, coreimage::destination dest) {
float4 inputPixel = input.sample(input.transform(dest.coord()));
float4 color = float4(1.0, 0.0, 0.0, 0.5);
float r = (inputPixel.r * (1 - color.a)) + (color.r * color.a);
float g = (inputPixel.g * (1 - color.a)) + (color.g * color.a);
float b = (inputPixel.b * (1 - color.a)) + (color.b * color.a);
float a = 1.0;
return float4(r,g,b,a);
}
By default, this kernel runs on ever single pixel, returning a red, tinted image. I'd like it to work in real time, every frame. For this reason, I think it's important to limit it to processing only the pixels in a specific region of an image. How do I do that?
For example:
Input image: 8192 x 8192
Region to tint red: CGRect(x: 20, y: 20, width: 10, height: 10)
I could write an if-then statement in the kernel to only tint the pixels contained in the CGRect. However, all the other pixels would still pass through the kernel.
In other words, no matter how small the CGRect, the above kernel will process all 67,108,864 pixels in the 8192 x 8192 image, instead of 100 pixels in the CGRect above.
Note, I'm not looking for a cropped image. I'd still like the output to be 8192 x 8129 pixels, with a tinted 10 x 10 square at x:20, y:20.
I thought the ROI callback function might be the solution, but it still appears the entire image is being tinted red, not just the region I specified. This is the ROI Callback I supplied:
let roiCallback: CIKernelROICallback = { index, rect -> CGRect in
return CGRectCGRect(x: 20, y: 20, width: 10, height: 10)
}
I have also tried:
Step 1: Crop the input image to the region needed and processing it;
Step 2: Composite output of step 1 over original input image.
This did not result in higher FPS. It appears as though in Step 2 all 67,108,164 pixels still have to be processed by CISourceOverCompositing anyway, so not an improvement.
I'm rendering the final output in a MTKView.
Is there a more efficient solution?
Edit 01:
I tried Frank Rupprecht's suggestion below. This was the output:
If it helps, this is the what the rendering stage looks like:
Render output to CVPixelBuffer
Convert CVPixelBuffer to CIImage and store CImage for next pass through kernel in the next frame.
Scale and translate CIImage from step 2 to display in MTKView drawable.
The reason to make an intermediate render in step 2 is to hold on to the image in its original resolution, and is not changed by the scaling and translating that happens when rendering to the drawable.
You can specify the region the kernel is applied to using the extend parameter in the apply(...) method:
kernel.apply(extent: CGRect(x: 20, y: 20, width: 10, height: 10), roiCallback:...)
This should only run the kernel on the specified region of the image. You don't need to change the ROICallback since this only specifies what part of the input image is needed to produce the output of the rendered region, so just returning the input rect in the roiCallback should be fine.
Another optimization: Since you only ever read the single pixel at the output coordinate from your input (and thereby have a 1:1 mapping from input to output), you can use a CIColorKernel instead by changing your kernel code like this:
extern "C" float4 BasicFilter (coreimage::sample input) {
float4 inputPixel = input; // this is already just a single pixel value
float4 color = float4(1.0, 0.0, 0.0, 0.5);
float r = (inputPixel.r * (1 - color.a)) + (color.r * color.a);
float g = (inputPixel.g * (1 - color.a)) + (color.g * color.a);
float b = (inputPixel.b * (1 - color.a)) + (color.b * color.a);
float a = 1.0;
return float4(r,g,b,a);
}
And then just initialize the kernel as a CIColorKernel instead of a CIKernel. On apply, you also don't need to specify a roiCallback then since Core Image knows it's a 1:1 mapping.

SpriteKit SKShader flipping after SKTransition completes

I'm creating a space-themed game in SpriteKit and to simulate a starfield I'm using an GLSL fragment shader from ShaderToy, converted to work in SpriteKit.
To use it I simply init a clear SKSpriteNode of the same size as my scene (2048 x 1536) and apply the shader to this node.
let starField = SKSpriteNode(color: UIColor.clear, size: CGSize(width: 2048, height: 1536))
let shader = SKShader(fileNamed: "Starfield.fsh")
starField.shader = shader
The shader renders just fine and shows a nice starfield. So far, so good.
The problem occurs when I transition from a different scene using SKTransition. During the transition, the shader appears to be rasterising, and as soon as the transition is complete the whole thing flips upside down.
My transition code (doesn't appear to matter what transition or duration I use):
let gameScene = GameScene(level: level)
gameScene.scaleMode = self.scaleMode
let transition = SKTransition.fade(withDuration: 3.0)
transition.pausesIncomingScene = true
self.view?.presentScene(gameScene, transition:transition)
I've tried with a different shader and the same occurs - the starfield is one way up during the transition, and instantly 'flips' as soon as the previous scene has been cleared up. Has anyone experienced the same and knows what is going on?
I have a video of the problem, which you can see occurring at around 7 seconds:
https://youtu.be/l1lLv6MwKYU
The shader code is as follows:
#define M_PI 3.1415926535897932384626433832795
float rand(vec2 co);
float rand(vec2 co)
{
return fract(sin(dot(co.xy ,vec2(12.9898,78.233))) * 43758.5453);
}
void main()
{
float size = 50.0;
float prob = 0.95;
vec2 pos = floor(1.0 / size * gl_FragCoord.xy);
float color = 0.0;
float starValue = rand(pos);
if (starValue > prob)
{
vec2 center = size * pos + vec2(size, size) * 0.5;
float t = 0.9 + 0.2 * sin(u_time + (starValue - prob) / (1.0 - prob) * 45.0);
color = 1.0 - distance(gl_FragCoord.xy, center) / (0.9 * size);
color = color * t / (abs(gl_FragCoord.y - center.y)) * t / (abs(gl_FragCoord.x - center.x));
}
else if (rand(v_tex_coord) > 0.996)
{
float r = rand(gl_FragCoord.xy);
color = r * (0.25 * sin(u_time * (r * 5.0) + 720.0 * r) + 0.75);
}
gl_FragColor = vec4(vec3(color), 1.0);
}
:

Using shaders from Shadertoy in Interface Builder (Xcode)

I'm attempting to see what shaders look like in Interface Builder using sprite kit, and would like to use some of the shaders at ShaderToy. To do it, I created a "shader.fsh" file, a scene file, and added a color sprite to the scene, giving it a custom shader (shader.fsh)
While very basic shaders seem to work:
void main() {
gl_FragColor = vec4(0.0,1.0,0.0,1.0);
}
Any attempt I make to convert shaders from ShaderToy cause Xcode to freeze up (spinning color ball) as soon as the attempt is made to render them.
The shader I am working with for example, is this one:
#define M_PI 3.1415926535897932384626433832795
float rand(vec2 co)
{
return fract(sin(dot(co.xy ,vec2(12.9898,78.233))) * 43758.5453);
}
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
float size = 30.0;
float prob = 0.95;
vec2 pos = floor(1.0 / size * fragCoord.xy);
float color = 0.0;
float starValue = rand(pos);
if (starValue > prob)
{
vec2 center = size * pos + vec2(size, size) * 0.5;
float t = 0.9 + 0.2 * sin(iGlobalTime + (starValue - prob) / (1.0 - prob) * 45.0);
color = 1.0 - distance(fragCoord.xy, center) / (0.5 * size);
color = color * t / (abs(fragCoord.y - center.y)) * t / (abs(fragCoord.x - center.x));
}
else if (rand(fragCoord.xy / iResolution.xy) > 0.996)
{
float r = rand(fragCoord.xy);
color = r * (0.25 * sin(iGlobalTime * (r * 5.0) + 720.0 * r) + 0.75);
}
fragColor = vec4(vec3(color), 1.0);
}
I've tried:
Replacing mainImage() with main(void) (so that it will be called)
Replacing the iXxxxx variables (iGlobalTime, iResolution) and fragCoord variables with their related variables (based on the suggestions here)
Replacing some of the variables (iGlobalTime)...
While changing mainImage to main() and swapping out the variables got it to work without error in TinyShading realtime tester app - the outcome is always the same in Xcode (spinning ball, freeze). Any advice here would be helpful as there is a surprisingly small amount of information currently available on the topic.
I managed to get this working in SpriteKit using SKShader. I've been able to render every shader from ShaderToy that I've attempted so far. The only exception is that you must remove any code using iMouse, since there is no mouse in iOS. I did the following...
1) Change the mainImage function declaration in the ShaderToy to...
void main(void) {
...
}
The ShaderToy mainImage function has an input named fragCoord. In iOS, this is globally available as gl_FragCoord, so your main function no longer needs any inputs.
2) Do a replace all to change the following from their ShaderToy names to their iOS names...
fragCoord becomes gl_FragCoord
fragColor becomes gl_FragColor
iGlobalTime becomes u_time
Note: There are more that I haven't encountered yet. I'll update as I do
3) Providing iResolution is slightly more involved...
iResolution is the viewport size (in pixels), which translates to the sprite size in SpriteKit. This used to be available as u_sprite_size in iOS, but has been removed. Luckily, Apple provides a nice example of how to inject it into your shader using uniforms in their SKShader documentation.
However, as stated in Shader Inputs section of ShaderToy, the type of iResolution is vec3 (x, y and z) as opposed to u_sprite_size, which is vec2 (x and y). I am yet to see a single ShaderToy that uses the z value of iResolution. So, we can simply use a z value of zero. I modified the example in the Apple documentation to provide my shader an iResolution of type vec3 like so...
let uniformBasedShader = SKShader(fileNamed: "YourShader.fsh")
let sprite = SKSpriteNode()
sprite.shader = uniformBasedShader
let spriteSize = vector_float3(
Float(sprite.frame.size.width), // x
Float(sprite.frame.size.height), // y
Float(0.0) // z - never used
)
uniformBasedShader.uniforms = [
SKUniform(name: "iResolution", vectorFloat3: spriteSize)
]
That's it :)
Here is the change to the shader that works when loaded as a shader with swift:
#define M_PI 3.1415926535897932384626433832795
float rand(vec2 co);
float rand(vec2 co)
{
return fract(sin(dot(co.xy ,vec2(12.9898,78.233))) * 43758.5453);
}
void main()
{
float size = 50.0; //Item 1:
float prob = 0.95; //Item 2:
vec2 pos = floor(1.0 / size * gl_FragCoord.xy);
float color = 0.0;
float starValue = rand(pos);
if (starValue > prob)
{
vec2 center = size * pos + vec2(size, size) * 0.5;
float t = 0.9 + 0.2 * sin(u_time + (starValue - prob) / (1.0 - prob) * 45.0); //Item 3:
color = 1.0 - distance(gl_FragCoord.xy, center) / (0.9 * size);
color = color * t / (abs(gl_FragCoord.y - center.y)) * t / (abs(gl_FragCoord.x - center.x));
}
else if (rand(v_tex_coord) > 0.996)
{
float r = rand(gl_FragCoord.xy);
color = r * (0.25 * sin(u_time * (r * 5.0) + 720.0 * r) + 0.75);
}
gl_FragColor = vec4(vec3(color), 1.0);
}
Play with Item 1: to increase the number of stars in the sky the smaller the number the more stars I like the number to be around 50 not too dense
Item 2: changes the randomness or how close together the stars will appear 1 = none, 0.1 = side by side around 0.75 gives a nice feel.
Item 3 is where most of the magic happens this is the size and pulse of the stars.
float t = 0.9
Changing 0.9, will increase the initial star sign up or down a nice value is 1.4 not too big not too small.
float t = 0.9 + 0.2
Changing the second value in this equation 0.2, will increase the pulse effect width of the stars proportionally to the original size I like with 1.4 a value of 1.2.
To add the shader to your swift project add a sprite to the scene the size of the screen then add the shader like this:
let backgroundImage = SKSpriteNode()
backgroundImage.texture = textureAtlas.textureNamed("any )
backgroundImage.size = screenSize
let shader = SKShader(fileNamed: "nightSky.fsh")
backgroundImage.shader = shader

iOS: transforming a view into cylindrical shape

With Quartz 2D we can transform our views on the x, yand z axis.
In some cases we could even make them look 3D by changing the values of the matrixes.
I was wondering if it could be possible to transform a view into a cylinder shape like in the following picture?
Please ignore the top part of the cylinder. I am more curious to know whether it would be possible warping an UIView around like the side of the cylinder as in the image.
Is that possible only making use of Quartz 2D, layers and transformations (not OpenGL)? If not, is it possible to at least draw it in CGContext to make a view appear like so?
You definitely can't do this with a transform. What you could do is create your UIView off-screen, get the context for the view, get an image from that, and then map the image to a new image, using a non-linear mapping.
So:
Create an image context with UIGraphicsBeginImageContext()
Render the view there, with view.layer.renderInContext()
Get an image of the result with CGBitmapContextCreateImage()
Write a mapping function that takes the x/y screen coordinates and maps them to coordinates on the cylinder.
Create a new image the size of the screen view, and call the mapping
function to copy pixels from the source to the destination.
Draw the destination bitmap to the screen.
None of these steps is particularly-difficult, and you might come up with various ways to simplify. For example, you can just render strips of the original view, offsetting the Y coordinate based on the coordinates of a circle, if you are okay with not doing perspective transformations.
If you want the view to actually be interactive, then you'd need to do the transform in the opposite direction when handling touch events.
No you can't bend a view using a transform.
The transform can only manipulate the four corners of the view so no matter what you do it will still be a plane.
I realize this goes beyond Quartz2D... You could try adding SceneKit.
Obtain the view's image via UIGraphicsBeginImageContext(), view.layer.renderInContext(), CGBitmapContextCreateImage().
Create a SCNMaterial with the diffuse property set to the image of your view
Create an SCNCylinder and apply the material to it.
Add the cylinder to an SCNScene.
Create an SCNView and set its scene.
Add the SCNView to your view hierarchy.
Reference : Using OpenGL ES 2.0 with iOS, how do I draw a cylinder between two points?
I have also used the same code for one of my project:
Check this one where it is mentioned to draw cone shape; it's dated but after adapting the algorithm, it works.
See code below for solution. Self represents the mesh and contains the vertices, indices, and such.
- (instancetype)initWithOriginRadius:(CGFloat)originRadius
atOriginPoint:(GLKVector3)originPoint
andEndRadius:(CGFloat)endRadius
atEndPoint:(GLKVector3)endPoint
withPrecision:(NSInteger)precision
andColor:(GLKVector4)color
{
self = [super init];
if (self) {
// normal pointing from origin point to end point
GLKVector3 normal = GLKVector3Make(originPoint.x - endPoint.x,
originPoint.y - endPoint.y,
originPoint.z - endPoint.z);
// create two perpendicular vectors - perp and q
GLKVector3 perp = normal;
if (normal.x == 0 && normal.z == 0) {
perp.x += 1;
} else {
perp.y += 1;
}
// cross product
GLKVector3 q = GLKVector3CrossProduct(perp, normal);
perp = GLKVector3CrossProduct(normal, q);
// normalize vectors
perp = GLKVector3Normalize(perp);
q = GLKVector3Normalize(q);
// calculate vertices
CGFloat twoPi = 2 * PI;
NSInteger index = 0;
for (NSInteger i = 0; i < precision + 1; i++) {
CGFloat theta = ((CGFloat) i) / precision * twoPi; // go around circle and get points
// normals
normal.x = cosf(theta) * perp.x + sinf(theta) * q.x;
normal.y = cosf(theta) * perp.y + sinf(theta) * q.y;
normal.z = cosf(theta) * perp.z + sinf(theta) * q.z;
AGLKMeshVertex meshVertex;
AGLKMeshVertexDynamic colorVertex;
// top vertex
meshVertex.position.x = endPoint.x + endRadius * normal.x;
meshVertex.position.y = endPoint.y + endRadius * normal.y;
meshVertex.position.z = endPoint.z + endRadius * normal.z;
meshVertex.normal = normal;
meshVertex.originalColor = color;
// append vertex
[self appendVertex:meshVertex];
// append color vertex
colorVertex.colors = color;
[self appendColorVertex:colorVertex];
// append index
[self appendIndex:index++];
// bottom vertex
meshVertex.position.x = originPoint.x + originRadius * normal.x;
meshVertex.position.y = originPoint.y + originRadius * normal.y;
meshVertex.position.z = originPoint.z + originRadius * normal.z;
meshVertex.normal = normal;
meshVertex.originalColor = color;
// append vertex
[self appendVertex:meshVertex];
// append color vertex
[self appendColorVertex:colorVertex];
// append index
[self appendIndex:index++];
}
// draw command
[self appendCommand:GL_TRIANGLE_STRIP firstIndex:0 numberOfIndices:self.numberOfIndices materialName:#""];
}
return self;
}

Using OpenGL ES 2.0 with iOS, how do I draw a cylinder between two points?

I am given two GLKVector3's representing the start and end points of the cylinder. Using these points and the radius, I need to build and render a cylinder. I can build a cylinder with the correct distance between the points but in a fixed direction (currently always in the y (0, 1, 0) up direction). I am not sure what kind of calculations I need to make to get the cylinder on the correct plane between the two points so that a line would run through the two end points. I am thinking there is some sort of calculations I can apply as I create my vertex data with the direction vector, or angle, that will create the cylinder pointing the correct direction. Does anyone have an algorithm, or know of one, that will help?
Are you drawing more than one of these cylinders? Or ever drawing it in a different position? If so, using the algorithm from the awesome article is a not-so-awesome idea. Every time you upload geometry data to the GPU, you incur a performance cost.
A better approach is to calculate the geometry for a single basic cylinder once — say, one with unit radius and height — and stuff that vertex data into a VBO. Then, when you draw, use a model-to-world transformation matrix to scale (independently in radius and length if needed) and rotate the cylinder into place. This way, the only new data that gets sent to the GPU with each draw call is a 4x4 matrix instead of all the vertex data for whatever polycount of cylinder you're drawing.
Check this awesome article; it's dated but after adapting the algorithm, it works like a charm. One tip, OpenGL ES 2.0 only supports triangles so instead of using GL_QUAD_STRIP as the method does, use GL_TRIANGLE_STRIP instead and the result is identical. The site also contains a bunch of other useful information regarding OpenGL geometries.
See code below for solution. Self represents the mesh and contains the vertices, indices, and such.
- (instancetype)initWithOriginRadius:(CGFloat)originRadius
atOriginPoint:(GLKVector3)originPoint
andEndRadius:(CGFloat)endRadius
atEndPoint:(GLKVector3)endPoint
withPrecision:(NSInteger)precision
andColor:(GLKVector4)color
{
self = [super init];
if (self) {
// normal pointing from origin point to end point
GLKVector3 normal = GLKVector3Make(originPoint.x - endPoint.x,
originPoint.y - endPoint.y,
originPoint.z - endPoint.z);
// create two perpendicular vectors - perp and q
GLKVector3 perp = normal;
if (normal.x == 0 && normal.z == 0) {
perp.x += 1;
} else {
perp.y += 1;
}
// cross product
GLKVector3 q = GLKVector3CrossProduct(perp, normal);
perp = GLKVector3CrossProduct(normal, q);
// normalize vectors
perp = GLKVector3Normalize(perp);
q = GLKVector3Normalize(q);
// calculate vertices
CGFloat twoPi = 2 * PI;
NSInteger index = 0;
for (NSInteger i = 0; i < precision + 1; i++) {
CGFloat theta = ((CGFloat) i) / precision * twoPi; // go around circle and get points
// normals
normal.x = cosf(theta) * perp.x + sinf(theta) * q.x;
normal.y = cosf(theta) * perp.y + sinf(theta) * q.y;
normal.z = cosf(theta) * perp.z + sinf(theta) * q.z;
AGLKMeshVertex meshVertex;
AGLKMeshVertexDynamic colorVertex;
// top vertex
meshVertex.position.x = endPoint.x + endRadius * normal.x;
meshVertex.position.y = endPoint.y + endRadius * normal.y;
meshVertex.position.z = endPoint.z + endRadius * normal.z;
meshVertex.normal = normal;
meshVertex.originalColor = color;
// append vertex
[self appendVertex:meshVertex];
// append color vertex
colorVertex.colors = color;
[self appendColorVertex:colorVertex];
// append index
[self appendIndex:index++];
// bottom vertex
meshVertex.position.x = originPoint.x + originRadius * normal.x;
meshVertex.position.y = originPoint.y + originRadius * normal.y;
meshVertex.position.z = originPoint.z + originRadius * normal.z;
meshVertex.normal = normal;
meshVertex.originalColor = color;
// append vertex
[self appendVertex:meshVertex];
// append color vertex
[self appendColorVertex:colorVertex];
// append index
[self appendIndex:index++];
}
// draw command
[self appendCommand:GL_TRIANGLE_STRIP firstIndex:0 numberOfIndices:self.numberOfIndices materialName:#""];
}
return self;
}

Resources