I need to move large amounts of pixels on the screen on an iOS device. What is the most efficient way of doing this?
So far I'm using glTexSubImage2D(), but I wonder if this can be done any faster. I noticed that OpenGL ES 2.0 does not support pixel buffers, but there seems to be a pixel buffer used by Core Video. Can I use that? Or maybe there's an Apple extension for OpenGL that could help me achieve something similar (I think saw a very vague mention about a client storage extension in one of the WWDC 2012 videos, but I can't find any documentation about it)? Any other way that I can speed this up?
My main concern is that glTexSubImage2D() copies all the pixels that I send. Ideally, I'd like to skip this step of copying the data, since I already have it prepared...
The client storage extension you're probably thinking of is CVOpenGLESTextureCacheCreateTextureFromImage; a full tutorial is here. That's definitely going to be the fastest way to get data to the GPU.
Frustratingly the only mention I can find of it in Apple's documentation is the iOS 4.3 to 5.0 API Differences document — do a quick search for CVOpenGLESTextureCache.h.
Related
Is it possible to perform OCR on image (for example, from assets) instead of live video with Anyline, microblink or other SDKs?
Tesseract is not an option due to my limited time.
I've tested it but the results are very inappropriate. I know that it can be improved with OpenCv or something but I have to keep a deadline.
EDIT:
This is an example of what the image looks like when it arrives to the OCR SDK.
I am not sure for the others, but you can use microblink SDK for reading from a single image. It is documented here.
Reading from a video stream will give much better results, but it all depends on what you are trying to do exactly. What are you trying to read?
For reading barcodes or MRZ from i.e. identity documents, it works pretty well. For raw text OCR, not quite as good but it is not really intended for that anyway.
https://github.com/garnele007/SwiftOCR
Machine learning based, Trainable on different font, chars, etc.
and free
My goal is as follows: I have to read in a video that is stored on the sd card, process it frame for frame and then store it in a new file on the SD card again，In each image to do image processing.
At first I wanted to use opencv for android but I did not seem to be able to read the video
here.
I am guessing you already know that doing this on a mobile device or any compute limited devices is not ideal, simply because video manipulation is very computer intensive which translates to slow execution and heavy battery usage on many devices. If you do have the option to do the processing on the server side it is definitely worth considering.
Assuming that for your use case you need to do it on the mobile device, then OpenCV on Android will now allow you to read in a video and access each frame - #StephenG mentions this in his answer to the question you refer to above.
In the past, functionality like this did not get ported to the Android OpenCv as the guidance was to use ffmpeg for frame grabbing on Android devices.
According to more recent documentation, however, this should be available for Android now using the VideoCapture class (note I have not used this myself...):
http://docs.opencv.org/java/2.4.11/org/opencv/highgui/VideoCapture.html
It is worth noting that OpenCV Android examples are all currently based around Eclipse and if you want to use Studio, getting things up an running initially can be quite tricky. The following worked for me recently, but as both studio and OpenCV can change over time you may find you have to do some forum hunting if it does not work for you:
https://stackoverflow.com/a/35135495/334402
Taking a different approach, you can use ffmpeg itself, in a wrapper in Android, for tasks like this.
The advantage of the wrapper approach is that you can use all the usual command line syntax and there is a lot of info on the web to help you get the right parameters.
The disadvantage is that ffmpeg was not really designed to be wrapped in this way so you do sometimes see issues. Having said that it is a common approach now and so long as you choose a well used wrapper library you should at least have a good community to discuss any issues you come across with. I have used this approach in a hand crafted way in the past but if I was doing it again I would use one of the popular examples such as:
https://github.com/WritingMinds/ffmpeg-android-java
I am looking for the best tool to achieve something like this (this is Blender's game engine, no real reflections, etc.) in an webgl viewer.
http://youtu.be/9-n12ZH5O6k
The idea is to prepare several basic scenes like this and then for the user to upload his design and have it previewed on a car (or other far more basic objects).
While p3d is nice, I don't think it does the job. There's no API for these cases yet. What are some options to pull this off? The requirement would be to have a library that doesn't have a too large footprint, since the feature/product is planned for the Asian market, so internet speed has to be considered.
you should look into three.js/babylon.js maybe? But you surely won't achieve that app just by a fingersnap, so read the tutorials as well, but these libs will surely ease your task by much.
I'm creating a simple Breakout style game and would like a simple way to display the score.
I've been doing some research and found several ways to do text in OpenGL ES, but most methods look fairly complicated.
This looks like it would do the trick, but I couldn't get it to work.
I've looked into FTGL and FreeType, but they look complicated.
I've also read one can display a UILabel over the EAGLContext, but not sure how that would be in the performance department.
I could probably get any of these options to work, I'm just wondering what the best solution is for this situation.
For simple use cases like you're describing, on even vaguely modern hardware (i.e. iPhone 3GS and later, I think), the compositing penalty for layering UIKit/CoreAnimation content on top of OpenGL ES content is negligible. (You can see this if you run your app in Instruments with the "Color OpenGL ES fast path blue" option turned on.)
They say premature optimization is the root of all evil — it's pretty easy to try UILabel, see if it makes a significant difference to your app's performance, and look into third-party libraries and more complicated solutions only if it does.
(Also, it sounds like you might be trying to manage your own CAEAGLLayer. For common use cases, it's a lot easier to use GLKView, plus GLKViewController for animation.)
I'd recommend checking out the Print3D functionality of the PowerVR SDK's PVRTools framework. Print3D is free to use, cross-platform (iOS, Android, Linux, Windows, OS X etc.) and it efficiently renders text within OpenGL ES 1.x, 2.0 & 3.0 applications. The SDK includes an example application with source that demonstrates how to use the Print3D framework (IntroducingPrint3D).
The PowerVR Graphics SDK can be downloaded for free from Imagination's website: http://www.imgtec.com/powervr/insider/sdkdownloads/index.asp
An overview of the source included in the SDK can be found here: http://www.imgtec.com/powervr/insider/sdkdownloads/learn_more.asp
I am working on an image processing app for the iOS, and one of the various stages of my application is a vector based image posterization/color detection.
Now, I've written the code that can, per-pixel, determine the posterized color, but going through each and every pixel in an image, I imagine, would be quite difficult for the processor if the iOS. As such, I was wondering if it is possible to use the graphics processor instead;
I'd like to create a sort of "pixel shader" which uses OpenGL-ES, or some other rendering technology to process and posterize the image quickly. I have no idea where to start (I've written simple shaders for Unity3D, but never done the underlying programming for them).
Can anyone point me in the correct direction?
I'm going to come at this sideways and suggest you try out Brad Larson's GPUImage framework, which describes itself as "a BSD-licensed iOS library that lets you apply GPU-accelerated filters and other effects to images, live camera video, and movies". I haven't used it and assume you'll need to do some GL reading to add your own filtering but it'll handle so much of the boilerplate stuff and provides so many prepackaged filters that it's definitely worth looking into. It doesn't sound like you're otherwise particularly interested in OpenGL so there's no real reason to look into it.
I will add the sole consideration that under iOS 4 I found it often faster to do work on the CPU (using GCD to distribute it amongst cores) than on the GPU where I needed to be able to read the results back at the end for any sort of serial access. That's because OpenGL is generally designed so that you upload an image and then it converts it into whatever format it wants and if you want to read it back then it converts it back to the one format you expect to receive it in and copies it to where you want it. So what you save on the GPU you pay for because the GL driver has to shunt and rearrange memory. As of iOS 5 Apple have introduced a special mechanism that effectively gives you direct CPU access to OpenGL's texture store so that's probably not a concern any more.