I have just delved into the world of Metal, and I thought that I'd got the hang of it! But then it occurred to me that if I wanted to make a game, then static objects moving around a screen wouldn't suffice. So my question is, 'Is it possible to create animations for models with Metal?'
I have looked at using other APIs, such as SpriteKit, and SceneKit, but I found that they do not support shaders, and are not as powerful as Metal.
The only way that I can think about how I would go about this, is by creating 60 different models, and then loading each one one after the other, to give a 'stop-motion' kind of effect, but I think that this would probably be incredibly inefficient, and was hoping that there was an easier answer?
Thanks a lot!
Yes, there are other, more efficient ways to do animation. But before getting into that, a warning: it really looks like you're barking up the wrong tree here.
Metal is a (conceptually) very low-level interface. You use Metal to talk (almost) directly to the GPU, so to work with it you need to think (sort of) like a GPU: in terms of data buffers, vertex transformations, etc. You seem to be working at a much higher conceptual level, so you're probably better served by one of the high-level game engines: SpriteKit for 2D or SceneKit for 3D. (Or a third party engine like Cocos or Unity.) Metal, on the other hand, is better suited for building those game engines.
SpriteKit and SceneKit do support shaders. Look at SKShader and SCNShadable in the docs (and be sure to click the "More" links to read the full overviews). SceneKit also supports character animations (aka skeletal animation aka skinning): typically one designs and rigs a model for animation in an external authoring tool (Maya, Blender, etc), then uses SceneKit to work with the animations at run time.
It is possible to do things like GPU-based skeletal animation in Metal. But I haven't seen any tutorials or similar written about it yet, probably because Metal is such a new technology. Fundamentally, though, it'd be based on the same sorts of techniques you'd use for skeletal animation in OpenGL or Direct3D — and much has been written about animation for those technologies. If you're willing to invest the time and energy to work at a low level, adapting the subject matter from GL/D3D tutorials is relatively easy.
You can do skeletal animation in Metal, SCNKit would be using the GPU to deform the mesh as well. But to do it in Metal you would need to pass skin weights, along with bone matrices for the bind pose and the transformations of the bones as they animate then calculate the new vertex positions based on these. In fact I think you need the inverse of the bind pose matrices. Each mesh vertex is then transformed by a weighted sum of transformations dictated by the skin weights.
I tried it but screwed it up somehow it didn’t deform properly, I don’t know if I’d obtained the wrong matrices from my custom script to grab animation data from blender or a bug in my shader maths or from the weights.
It was probably close, but with all the possible things that I may have got wrong in the process it was difficult to fix so I abandoned it in the end.
Probably easier to stick with SceneKit and let apple take care of the rest or use an existing game engine such as Unity.
Then again if you want a challenge, I’m sure it’s possible, just a little tricky. You could try CPU first to make sure the maths is ok then port it to the GPU to make it faster?
SceneKit do support shaders. And an object that manages the relationship between skeletal animations and the nodes and geometries they animate is SCNSkinner from SceneKit.
Typically, you need to create a skinned model using, for example, Autodesk Maya, save it along with animations that use the skeleton, in a scene file. You load the model from the scene file and pose or animate it in your app, either by using animation objects also loaded from the scene file or by directly manipulating the nodes in the skeleton. That's it.
Watch this 7-parts video about Blender's skeletal system and how to use it in SceneKit.
convenience init(baseGeometry: SCNGeometry?, //character
bones: [SCNNode], //array of bones
boneInverseBindTransforms: [NSValue]?, //ibt of matrix4
boneWeights: SCNGeometrySource, //influence on geometry
boneIndices: SCNGeometrySource //index mapping
)
Related
So.
After many years of iOS development I said it's time to try to do a little game for myself. Now I chose to do it using Apple's SceneKit since it looks like it provides everything I need.
My problem is that I've stumbled upon a huge problem (for me) and searching on Google doesn't yeld any results.
Any idea how do I go about having an object (a sphere for that matter) that deforms itself, say, because of a gravitational force. So basically it should squash on impact with the ground.
Or, how do I go about deforming it when it collides with other spheres, like a soft beach ball would?
Any starting point along those lines would be helpful.
I can post my code here, but I'm afraid it has nothing to do with my problem since I really don't know where to start.
Thanks!
Update
After doing a bit more reading I think that what I want could be doable with Vertex Shaders. Is that a right path to follow?
For complicated animations, you'll generally be better off using a 3D modeling tool like Blender, Maya, or Cheetah3D to build the body and construct the animation. Those tools let you think at a higher level of abstraction. Then you can export that model to Collada (DAE) format and then import it into SceneKit.
https://en.wikibooks.org/wiki/Blender_3D:_Noob_to_Pro/Basic_Animation/Bounce has a tutorial on building a deforming, bouncing ball using Blender.
SceneKit only does physics using rigid bodies. If you want something to deform, you would have to do it yourself.
It is probably because SceneKit has no way of knowing how an object should be deformed. Should it just compress, should it compress in one direction and expand in all others to preserve it's volume, should only part of the model compress and the rest stay rigid (like the tires on a car).
What you could try is wait for a collision to occur and do the following
calculate and store the velocity after the bounce
disable collision checking on the object
run an animation for the "squash"
enable collision checking on the object
apply the calculated velocity
It will be entirely up to you how real or cartoony you want to make the bounce look.
I am looking for a tool to start with simple 3d objects for Scene Kit. I know there are a lot professional tools out there, but just buying blind a program to try a bit how things work would be wasted money.
How do I create 3D content for a Scene Kit scene, beyond using the builtin geometry and geometry creation tools within Scene Kit?
Short answer: Blender.
Long answer: Blender, because it's free and as good as any other 3D app (they are unfortunately all quite bad) but take this to the Apple Developer forums if you'd like more info, as this question is not appropriate for Stack Overflow. The way you're thinking about this is also not appropriate; no app is a "Colada editor". That would be like saying Photoshop is a JPEG editor.
The Unity forum is also a great place to hear a bunch of opinions; unlike any of Apple's tools other than Logic or perhaps Final Cut, it has a wide-ranging user base of technically-minded artists from whom to gather opinions.
Use Polygon Modelling.
COLLADA content is created in 3D modelling programs. There are three main ways of creating 3D content: polygon modelling, clay/brush style push/pull modelling of highly complex meshes, and what's most commonly known as NURBS modelling.
Polygon modelling involves creating shapes by articulating and adding primitive pieces of geometry to primitive pieces of geometry and progressively building up complex shapes in this manner. It has several massive advantages for game engine content creation that have seen it become (far and away) the most popular manner of making content for game engines.
Namely:
Strict and absolute control of the amount and type of geometry (performance)
Strict and absolute seam and inter-object connectivity control (animation)
Absolute texture control and material/shader control via geometry and bitmaps with 1:1 UVW mapping (ideal for expressing textures onto geometry with great performance)
Well designed and commonly used smoothing and tessellation algorithms nondestructive to, and considerate of, the above points
Given these massive benefits for game content creation, it's imperative you learn Polygon modelling first, for any and all game content creation. Most nearly all polygon modellers output a format compatible with COLLADA.
Understanding Polygon modelling will also give you understanding of some of how Geometry Shaders work and what they act on. The addition of Geometry Shaders is a relatively new feature of Scene Kit that provides vastly more interesting ways to manipulate geometry than the basic geometry creation tools provided within Scene Kit.
For SCNFloor, if the reflective is set to 1 and reflectionFallOffEnd is big enough, it will be like a mirror.
My question is how to apply this to other geometries (say plane or box)? As I want to have a mirror in my game.
I have done quite a bit of research on how to make reflections using Scenekit.
Here are the different leads I found (sadly, they will all need a serious amount of code and research):
Screen-Space reflections
Pros :
Cheap
Easy to make
Cons
Doesn't always look great
I'm not sure how to output a normal pass with SCNTechnique
Parallax-mapped cubemaps
Pros :
Cheap
Looks amazing
Cons
No real time objects unless using an image proxy
No good code sample online, will need research
Not quite sure how to use it with SCNProgram
Two cameras + Stencil
Pros :
Realistic
Real time
Almost built in
Cons
No documentation of the pointOfView of SCNTechnique
No documentation on Stencils
Needs to render the scene twice
OpenGL mirror
Pros :
Actually duplicates the geometry, so very accurate
This is the technique used by SCNFloor (I think)
Cons
Geometry can clip with the mirror plane (happens with SCNFloor)
Unusable on anything other than a plane
Needs OpenGL Code
4 Cameras linked to a Cubemap
Pros :
Easy to set up
Real time
Works on any object
Very popular technique in modern Video Games
Cons
I have no idea if this would work
Will need to render the scene 5 times for a single mirror
Not very accurate depending on object
My conclusion is that we need more help on using SCNTechnique. We could build amazing things with it but the lack of documentation and examples is a big problem.
If you could specify what kind of mirror you have in mind, I'll be happy to help you choose the best way to go.
I know this is an old question, but I wanted to share what I have done. I created a gist on GitHub that contains the code and explains how it works.
It basically attaches six cameras to a node and automatically creates a cubemap that is then used as the reflective property of the object. The main downside is that it won't work with physically based materials, but in order to simulate roughness, it blurs the cubemap to whatever you set the roughness property to. It works well in real time and you can set how quickly the cubemaps update so that you are not affecting the framerate of your game too much. It can also handle many different reflective objects and automatically stops updating nodes that you can't see.
This is currently not supported on other geometry types. Please file a request to Apple.
I understand that cocos2d it's really simple API, and that I can use it to do simple and huge 2D or even sometimes 3D games/applications. As well I understand that OpenGL it's more complicated, it's lower level API etc.
Question: What is better for implementing 2D/3D games? Why do we need to learn OpenGL if we have simple frameworks like cocos2d? What you can do with OpenGL that you can't do with cocos2d?
Thanks in advance!
What is better for implementing 2D/3D games?
Hard to tell, but a higher level API is always there to make things easier for you. For example you are writing a 2D shootem up. You will likely use a game loop, you will want to use sprites and make those move on the screen. You may want animations like explosions taking place. You'll end up writing your own higher level API to do those things. Cocos2D has solved those problems for you already. Any other frameworld should have solved it.
Why do we need to learn OpenGL if we have simple frameworks like cocos2d?
In case you like to cusomize the standard behaviour of a framework, especially the drawing part, you should get into openGL. If there is something you like to have which doesn't come out of the box you may find yourself reimplementing a base framework class. For example, look at the shaders used in Cocos2D 2.0. If you like some special blending mode, like a tinting effect, you won't get it for free. There is a colour attribute for a CCSprite but this may not be the result you're expecting. So you'll have to write your own shader and plug it into the sprite you like to be displayed in a different way.
What you can do with OpenGL that you can't do with cocos2d?
This comparison doesn't really work out, since cocos2d facilitates opengGL for the drawing part to build up that higher level api and make your life easier as a game developer.
Cocos2d is a wrapper around the 2D features of OpenGL (as of this: http://www.cocos2d-iphone.org/about) . Under the hood it itself uses OpenGL ES to implement its features. This is good because it means that there will be minimal performance overhead so you can start using its simpler API without having to get immersed to the definitely bigger learning path of OpenGL.
It has however only strong 2D support and if you plan to write later 3d games you loose all benefits of Cocos2d: why would you rewrite a 3d rendering engine with a 2d framework that under the hood uses a very strong 3d engine? You loose performance for a lot of unnecessary work.
So the simpler answer is: for 2d Cocos2d, for 3d OpenGL.
If you want to start OpenGL ES, this is a very good tutorial for beginners: http://iphonedevelopment.blogspot.it/2009/05/opengl-es-from-ground-up-table-of.html
I am trying to write a strategy game using XNA 4.0, with a dynamically generating map, and it's really difficult to create all the ground textures, having to distort them individually in photoshop.
So what I want to do is create a flat image, and then apply the distortion programatically to simulate perspective, by moving the corners of the image.
Here is an example done in photoshop:
How can I do that in XNA?
My answer isn't XNA-specific as I've never actually used the library; however the concept should still apply.
In general, the best way to get a good perspective effect is to actually give 3d coordinates and transformations and let DirectX/OpenGL handle the rest. This has great benefits over attempting to do it yourself - specifically, ease of use, performance (much of the work is passed on to your graphics card), and perspective-correct texturing. And nothing's stopping you from doing 3d and 2d in the same scene, if that's a concern. There are numerous tutorials online for getting set up in the third dimension with XNA. I'd suggest heading over to MSDN.