I am very new to the concept and use of shaders in SpriteKit.
I found this tutorial on how to render a Mandelbrot fractal with a custom shader file - Fractal.fsh - attached to a Color Sprite's Custom Shader property.
https://www.weheartswift.com/fractals-Xcode-6/
It works fine and I thought to my self that learning about OpenGL ES and custom shaders in SpriteKit would be a fun exercise.
According to Apple though, OpenGL ES is deprecated as of iOS 12.
https://developer.apple.com/library/archive/documentation/3DDrawing/Conceptual/OpenGLES_ProgrammingGuide/BestPracticesforShaders/BestPracticesforShaders.html
My question is this:
Does this mean that custom shaders for use in SpriteKit should be written in Metal as of now?
I have tried to figure out how to rewrite the fractal.fsh shader code, referred to in the first link, in Metal but I have not - yet - been able to find any resources on how to convert existing custom SKShader's from OpenGL ES to Metal. However, I am NOT looking for someone to rewrite that code to use Metal, only a pointer in the right direction.
UPDATE:
Based on the answer from #Knight0fDragon I will try to clarify my question:
The documentation on the SKShader class states that:
"An SKShader object holds a custom OpenGL ES fragment shader."
https://developer.apple.com/documentation/spritekit/skshader
So if a SKShader object holds a custom OpenGL ES fragment shader, what will it hold after the support for OpenGL ES is deprecated?
How would one go on about creating a custom fragment shader to use in SpriteKit if one cannot use OpenGL ES as of iOS 12?
First I thought that the *.fsh file containing the GLSL code could be replaced with a *.metal file containing equivalent metal code but that assessment was clearly too naive (because I tried and I couldn't assign the *.metal file to the Color Sprite's Custom Shader property)
From the documentation on "Executing Shaders in Metal and OpenGL
":
On devices that support it, the GLSL code you provide to SKShader is automatically converted to Metal shading language and run on a Metal renderer.
So, from my understanding, SpriteKit will use Metal as a backend where it is available and convert your shaders for you when compiling them. I did not find an option to directly write the shaders in Metal.
According to Apple (see near bottom of the page at this link)...
Apps built using OpenGL ES will continue to run in iOS 12, but OpenGL
ES is deprecated in iOS 12. Games and graphics-intensive apps that
previously used OpenGL ES should now adopt Metal [emphasis added].
If you are starting from scratch, I suggest you write shaders in Metal.
Related
So I've got a weird issue. I'm porting a modern source port of an old game engine to iOS and tvOS. It was written using OpenGL and I found a fork of it that uses OpenGL ES and I've been able to Frankenstein combine the two to where it will actually now successfully run on iOS and tvOS devices.
Here's the weird part - parts of the game do not render correctly on my iPhone X but they do render correctly on my iPad Air 2 and Apple TV (4th gen).
I notice in the flurry of messages in the output window that the spot where the engine outputs renderer information, on the iPhone X it says
OpenGL version: OpenGL ES 2.0 Metal 58.4
whereas on, say, the iPad Air 2 it says
OpenGL version: OpenGL ES 2.0 A8X GPU - 130.1
"OpenGL ES Metal" sounds like "Jumbo Shrimp" to me since those are obviously not the same thing. If I were to guess, I'd say that on the iPhone X the OpenGL ES drivers are running on top of some sort of Metal translation layer, which may be Apple's long term plan for having some sort of future-proofing in the wake of the OpenGL ES deprecation.
But for whatever reason it's breaking my game engine and although I'm decent at making code work together, I don't know enough about graphics coding to know even where to look to change things to work.
Obviously the right answer is to fix whatever is causing the issue but as a short term fix I'm curious if there's any way to get a game on iOS to not use OpenGL ES on top of Metal? (if that is indeed what is happening)
So, like Brad Larson says below, the answers were: yes OpenGL ES is running on top of Metal and no, it can't be avoided. However for future reference if anyone else is running into this problem I solved the real underlying issue with the help of another SO answer:
WebGL GLSL fragment shader not working on iOS
Basically the floating point precision of the shaders need to be upgraded from lowp and mediump to highp
Yes, OpenGL ES is effectively an interface to Metal on recent iOS versions (since I believe 10, if I'm not mistaken).
From the Roblox Metal retrospective:
It is also worth noting that there is no GL driver on iOS 10 on the newest iPhones, apparently. GL is implemented on top of Metal, which means using OpenGL is only really saving you a bit of development effort – not that much, considering that the promise of “write once, run anywhere” that OpenGL has does not really work out on mobile.
You can verify this yourself by profiling OpenGL ES code on modern iOS versions:
In addition to the above, you'll see Metal-specific operations like -[MTLIOAccelCommandQueue submitCommandBuffers:count:] appearing in profiling of your OpenGL ES applications.
Near as I can tell, there's no way to circumvent this, it's how rendering is architected on modern iOS versions. That said, I've not seen this alter the behavior of my OpenGL ES code. I have seen different iOS GPUs produce slightly different rendering behavior due to their hardware, so it's possible you're encountering something that is device-specific.
Check for the usual suspects involving precision qualifiers, Z-fighting, division-by-zero in shaders, etc. and see if one of those is messing up your rendering. Those are all places where I've seen new device generations disrupt my rendering.
Apple says that apps built using OpenGL ES will continue to run in iOS 12, but Open GL ES is deprecated in iOS 12. Games and graphics-intensive apps that previously used OpenGL ES should now adopt Metal.
But I don't want to migrate my app from OpenGL to Metal because it's the same code for Android and I don't want to create two separate branches, so what can I do to continue to use OpenGL in future iOS releases?
You more or less do not.
Apple has made it abundantly clear that they are not interested in supporting OpenGL ES further on any platform they control. In the future, you will use Metal or you will not have accelerated 3D graphics. Those will be your options.
There are projects that expose OpenGL ES on such platforms by implementing an ES layer on top of Metal. MoltenGL, for example, but that one isn't free apparently.
I use MetalANGLE, which is free and open-source with a non-viral license, in my commercial product, as an (almost) drop-in replacement for GLKit. I have had to make no changes at all to the part of my code that uses OpenGL ES, and only minor changes to the way my View and ViewController classes were initialised.
It is now six months since I moved from GLKit to MetalANGLE and the only problem that arose was one of scaling, which was really a bug in my app: in MetalANGLE I needed to retrieve the UI scale from the View's contentScaleFactor rather than [[UIScreen mainScreen] scale].
There have been no display glitches, crashes, slowdowns, or any problems whatever with the graphics rendering.
I can therefore see no reason to pay for MoltenGL instead of using the free MetalANGLE.
I recommend that OpenGL and Metal are developed together on iOS, and new features are used with Metal.
I recently started building a game for iOS using my iPhone 5s for testing.
However, recently I have decided to try my iPhone SE to see how it looks; all of the materials on all of the objects appear to be missing, but the color of the trails for the particle effects and the line renderers still have their color.
It could be a shader issue or a problem with the graphics API Unity is using. If you're not using the standard shader then make sure that you're shader is compatible with mobile devices. Also make sure that it's included in the project by creating a folder named Resources and moving your shader into that.
If you're using one of the standard shaders that comes with Unity then the issue is likely not a shader one but the Graphics API selected. It's likely using Metal which is causing that issue. Use OpenGL ES instead of Metal.
Disable Auto Graphics API then change iOS Graphics API to OpenGLES2 or OpenGLES3 in Unity's Player Settings.
Im trying to make render to texture for reflection and refration texture for water shader... but glClipPlane is not defined in kivy opengl.. Here are some ScreenShots
Test with PyOpengl
Test with kivy Opengl
From Clipping-planes in OpenGL ES 2.0, it looks like this wasn't part of OpenGL ES 2.0, which Kivy nominally targets. If you do want to use it, you probably can, but it isn't part of Kivy's exposed API (these low level opengl calls are usually considered internal to Kivy).
My app currently uses OpenGL and i set polygon offset using glPolygonOffset and GL_POLYGON_OFFSET_FILL. I am in the process of transforming my code to Metal and looking for the equivalent methods in Metal. What are they?
MTLRenderCommandEncoder setDepthBias:slopeScale:clamp: