I'm currently working on loading 3D scene data (vertices and faces) for a mobile game.
My first thought was to load all data into system memory, and transfer only visible parts into video memory while traveling in scene.
I learned that in mobile, system memory and video memory both share the same memory. I thought it might be better to load all scene data into video memory directly, because no transferring is needed when traveling in the scene, Am I right?
Related
In my application I create textures, render to them and delay reading from them until absolutely needed by the CPU.
I'd like to know (and I still don't know how) given that I guess and monitor the consumed GPU memory would the call to readPixels() alleviate the memory consumption of the GPU by transferring data to the CPU? Or would that memory still be occupied until I destroy the texture?
readPixels just copies the data. It does not remove it from the GPU.
Textures that you don't pass data to (you passed null) take the same amount of memory as textures you pass no data to. The browser just fills the texture with zeros for you.
The only way for a texture to stop using memory is for you to delete it with gl.deleteTexture. You also need to remove every reference to it (unbind it from any texture units it's still on and remove it from any framebuffer attachments or delete the framebuffer's it's attached to).
I'm new to ios game development field.
I have been going through the following apple tutorial multiple times but not getting the points
https://developer.apple.com/library/ios/documentation/GraphicsAnimation/Conceptual/SpriteKit_PG/Sprites/Sprites.html
thanks
Directly from the SKTextureAtlas class reference :
Texture atlases can improve memory usage and rendering performance. For example, if you have a scene with sprites drawn with different textures, Sprite Kit performs one drawing pass for each texture. However, if all of the textures were loaded from the same texture atlas, then Sprite Kit can render the sprites in a single drawing pass—and use less memory to do so. Whenever you have textures that are always used together, you should store them in an atlas.
SKTextureAtlas Class Reference
animation can be hundred of files. atlas puts them in one large file.
reading one file (large with lots of images) is much more efficient then reading hundreds of files (with one image each)
rendering of one large image and then showing only part of it is much more efficient then rendering each file separately.
I use SKScenes for different screens of my main menu. Each of these screens is rather complicated, sporting dozens of elements.
So each time user moves between those screens there is like 1-2 second lag before going next scene or previous scene as I recreate them each time.
Should I just store my scenes in some singleton after they are created so that next calls are faster?
Is it a good thing to do? Won't it be too memory-heavy?
Some optimization tips:
1) Your biggest memory hog is going to be the graphics. To keep this under control, use SKTexture object with a texture atlas. Always use a texture atlas because:
Texture atlases can improve memory usage and rendering performance. For example, if you have a scene with sprites drawn with different textures, Sprite Kit performs one drawing pass for each texture. However, if all of the textures were loaded from the same texture atlas, then Sprite Kit can render the sprites in a single drawing pass—and use less memory to do so. Whenever you have textures that are always used together, you should store them in an atlas.
2) Create your texture atlases based on levels so you do not use up memory for nothing. In other words, create an atlas for level 1, another one for level 2 and so on. According to the docs:
For a larger game, you may need to split the textures into levels or themes. Each level or theme’s textures are designed to fit in a specific amount of memory. When the player starts a new level, you preload all of that level’s texture objects. When the player finishes playing the level, the textures not needed for the next level are discarded.
3) You can use a (SKSpriteNode) singleton if you have a lot of the same animations used in different scenes. For example, an enemy present in all levels who has animations for idle, walking, running, jumping, attacking, etc... The same singleton can load all of your texture alases for your animations. You can also use a singleton to store all of your game sounds and thus further reduce load times but keep an eye on memory usage if you have a lot of sounds/music.
4) Setting the skView.ignoresSiblingOrder = YES; also gives an optimization boost. According to the documentation:
The default value is NO, which means that when multiple nodes share the same z position, those nodes are sorted and rendered in a deterministic order. Parents are rendered before their children, and siblings are rendered from eldest to youngest. When this property is set to YES, the position of the nodes in the tree is ignored when determining the rendering order. The rendering order of nodes at the same z position is arbitrary and may change every time a new frame is rendered. When sibling and parent order is ignored, Sprite Kit applies additional optimizations to improve rendering performance. If you need nodes to be rendered in a specific and deterministic order, you must set the z position of those nodes.
5) If your images do not absolutely require alpha, do not include it in your images.
6) Remove nodes from the parent if you no longer need them. Also make sure you remove any reference to them such as inclusion in any arrays. The rest is taken care of by SpriteKit as stated in the docs:
An SKTexture object is created and attached to the sprite. This texture object automatically loads the texture data whenever the sprite node is in the scene, is visible, and is necessary for rendering the scene. Later, if the sprite is removed from the scene or is no longer visible, Sprite Kit can delete the texture data if it needs that memory for other purposes. This automatic memory management simplifies but does not eliminate the work you need to do to manage art assets in your game.
In other words, delete a SKTexture object by removing any strong references to it, including:
All texture references from SKSpriteNode and SKEffectNode objects in your game
Any strong references to the texture in your own code
An SKTextureAtlas object that was used to create the texture object
In my application i should play video in unusual way.
Something like interactive player for special purposes.
Main issues here:
video resolution can be from 200*200px up to 1024*1024 px
i should have ability to change speed from -60 FPS to 60 PFS (in this case video should be played slower or faster depending on selected speed, negative means that video should play in back direction)
i should draw lines and objects over the video and scale it with image.
i should have ability Zoom image and pan it if its content more than screen size
i should have ability to change brightness, contrast and invert colors of this video
Now im doing next thing:
I splited my video to JPG frames
created timer for N times per seconds (play speed control)
each timer tick im drawing new texture (next JPG frame) with OpenGL
for zoom and pan im playing with OpenGL ES transformations (translate, scale)
All looks fine until i use 320*240 px, but if i use 512*512px my play rate is going down. Maybe timer behavour problem, maybe OpenGL. Sometimes, if im trying to open big textures with high play rate (more than 10-15 FPS), application just crash with memory warnings.
What is the best practice to solve this issue? What direction should i dig? Maybe cocos2d or other game engines helps me? Mb JPG is not best solution for textures and i should use PNG or PVR or smth else?
Keep the video data as a video and use AVAssetReader to get the raw frames. Use kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange as the colorspace, and do YUV->RGB colorspace conversion in GLES. It will mean keeping less data in memory, and make much of your image processing somewhat simpler (since you'll be working with luma and chroma data rather than RGB values).
You don't need to bother with Cocos 2d or any game engine for this. I strongly recommend doing a little bit of experimenting with OpenGL ES 2.0 and shaders. Using OpenGL for video is very simple and straightforward, adding a game engine to the mix is unnecessary overhead and abstraction.
When you upload image data to the textures, do not create a new texture every frame. Instead, create two textures: one for luma, and one for chroma data, and simply reuse those textures every frame. I suspect your memory issues are arising from using many images and new textures every frame and probably not deleting old textures.
JPEG frames will be incredibly expensive to uncompress. First step: use PNG.
But wait! There's more.
Cocos2D could help you mostly through its great support for sprite sheets.
The biggest help, however, may come from packed textures a la TexturePacker. Using PVR.CCZ compression can speed things up by insane amounts, enough for you to get better frame rates at bigger video sizes.
Vlad, the short answer is that you will likely never be able to get all of these features you have listed working at the same time. Playing video 1024 x 1024 video at 60 FPS is really going to be a stretch, I highly doubt that iOS hardware is going to be able to keep up with those kind of data transfer rates at 60FPS. Even the h.264 hardware on the device can only do 30FPS at 1080p. It might be possible, but to then layer graphics rendering over the video and also expect to be able to edit the brightness/contrast at the same time, it is just too many things at the same time.
You should focus in on what is actually possible instead of attempting to do every feature. If you want to see an example Xcode app that pushes iPad hardware right to the limits, please have a look at my Fireworks example project. This code displays multiple already decoded h.264 videos on screen at the same time. The implementation is built around CoreGraphics APIs, but the key thing is that Apple's impl of texture uploading to OpenGL is very fast because of a zero copy optimization. With this approach, a lot of video can be streamed to the device.
I am building an app that uses Cocos2d to create a map of content (technically a big Tree in the graph theory sense). Each node is represented by a sprite with it's own texture, with the leaf nodes all having a second "button pressed" texture. In addition, some of the nodes are PNG animations. I have one sprite sheet containing the 130 or so node textures, and 3 other sprite sheets for the animations. All in all those 4 files add up to only about 18.4 MB.
If I run the app and DON'T run the cocos2d part (i.e. Never launch the framework at all and therefore never load the textures) the app runs at around 10 MB. When I load Cocos2d and those sprite sheets, the memory shoots up past 90 MB.
All my research seems to say I am doing things in the most efficient way possible, but with the memory usage already so high I get a lot of memory related crashes on iPad 1.
Is this normal for cocos2d? Since the images themselves aren't so large, is it using so much memory because of the number of sprites? Are there ways to bring the memory footprint down? I am at a loss, and very much under the gun...
It turns out that the efficiency you achieve by packing the sprites together into big sprite sheets is measured in rendering time, not in memory saved. No matter in a UIImage or into an OpenGL framework like Cocos2d, when you load an image the amount of memory it takes up is width x height x bytes per pixel. So a 2048x2048 png image takes up 2048x2048x4 bytes in memory (12 MB).
To keep memory down in this case I simply have to load things as needed, and unload them as soon as they are not needed. Now the trick is finding out where and when to load, so the user never sees a stutter. FUN!