I have created a simple Direct2D Effect, which flips the incoming image (can flip either horizontally or vertically or both at the same time). The custom effect seems to work fine. The problem starts when I try to chain two instances of the effect one after another:
ID2D1Effect *flip1; // initialized
ID2D1Effect *flip2; // initialized
ID2D1Bitmap1 *bmp; // initialized
flip1->SetInput(0, bmp);
flip2->SetInputEffect(0, flip1);
// ...
ID2D1DeviceContext *pContext; // initialized
pContext->BeginDraw();
pContext->DrawImage(flip2);
pContext->EndDraw();
As result I'm sometimes getting a "junk" image as the output. I noticed that as long as the second flip configured to leave the image as is, this chain works file. When the second flip modifies the image, I get whole or part of the target image "garbaged".
My suspicion is that, as the flip effect uses a complex sampling (target color of pixel xy depends on original pixel at different location), somehow the second flip effect tries to access an output pixel of the first flip, which is not ready yet.
Does this assumption make sense? Is there a way to avoid it? I always have a fallback of rendering each effect on a different target Bitmap, but I assumed this will take longer time than simply chaining the effects together.
Related
What is the best way to draw a whole CALayer hierarchy to a context (hence a UIImage, PDF)?
My hierarchy contains CATransformLayer, CATextLayers, CAReplicatorLayers and CAShapeLayers.
It looks fine on the screen but I never get it to look the same in the UIImage (and I don't want to use a screenshot-like rendering, since I want the CAShapeLayers and CATextLayers to be rendered as a vertex graphic to PDF)
These are some problems that I'm facing:
The rendering ignores the zPosition property and the translation.z in
the transform, it renders strictly by the order in the sublayers
array.
My layers have frames/position/transforms mixed and often
layers get moved around (pseudo-randomly) when rendering.
Sometimes CAReplicatorLayers are simply ignored and only the first instance gets rendered to the CGContext
As soon as a translation.z value of the transform is set to a non-zero value, that layer completely ignores its transfrom and renders itself at (0, 0), I worked around that by manually ordering the layers and setting the zTranslation to 0 but it's not very comfortable.
I hope somebody of you knows an easy and safe way to render everything to a context exactly as it is shown on the screen!
I am working on a tracking algorithm and one of the earliest steps it does is background subtraction. The algorithm gets a series of frames that represent the video with a moving object and static background. The object is in every frame.
In my first version of this process I computed a median image from all the frames and got a very good background scene approximation. Then I subtracted the resulting image from every frame in video sequence to get foreground (moving objects).
The above method worked well, but then I tried to replace it by using OpenCV's background subtractors MOG and MOG2.
What I do not understand is how these two classes perform the "precomputation of the background model"? As far as I understood from dozens of tutorials and documentations, these subtractors update the background model every time I use the apply() method and return a foreground mask.
But this means thet the first result of the apply() method will be a blank mask. And the later images wil have initial object's position ghost in it (see example below):
What am I missing? I googled a lot and seem to be the only one with this problem... Is there a way to run background precomputation that I am not aware of?
EDIT: I found a "trick" to do it: Before using OpenCV's MOG or MOG2 I first compute median background image, then I use it in first apply() call. The following apply() calls produce the foreground mask without the initial position ghost.
But still, is this how it should be done or is there a better way?
If your moving objects are present right from the start, all updating background estimators will place them in the background initially. A solution to that is to initialize your MOG on all frames and then run MOG again with this initialization (as with your median estimate). Depending on the number of frames you might want to adjust the update parameter of MOG (learningRate) to make sure its fully initialized (if you have 100 frames it probably needs to be higher at least 0.01):
void BackgroundSubtractorMOG::operator()(InputArray image, OutputArray fgmask, double **learningRate**=0)
If your moving objects are not present right from the start, make sure that MOG is fully initialized when they appear by setting a high enough value for the update parameter learningRate.
I currently have a free hand drawing iPad app, that adds lines to a mutable path via quad curves in the touches methods then calls setNeedsDisplayInRect on the new area.
Problem is when the drawing (path) gets rather large, it takes longer to redraw, and begins to bog down. As well as whenever the user changes the brush size or color, it applies this to overlapping parts of the previously drawn path on redraw.
To counter this, I call renderInContext in a background thread in touchesEnded, and merge this with another UIImage in an imageview behind the draw view. Then clear the draw view.
This also helps so when the user hits save, the drawing is usually already rendered in a single UIImage - ready to go.
This works fine on other devices, but on he iPad 3 retina display, the performance is really awful and tends to crash whenever the user lifts his finger multiple times when drawing quickly.
I am seeking any type of advice for best practice in handling this type of situation? Aside from adding additional views to render off of in the background to prevent the main and background thread from accessing the same view at a time - which sounds rather hack-ish - I feel like I'm beating a dead horse?
In my current app, I made a working implementation that works fine on iPad 2 as well as 3, regardless of path length or number of paths. It seems that the graphics card is better at drawing lots of small paths then a few large paths, and either one is faster than rendering an image into a context. So, what I do is even if the user is continuously drawing, I break the path into many smaller paths and add those to an array. This approach gives me one advantage, and one disadvantage.
Advantage: The ability to zoom and redraw the image crisply
Disadvantage: Can't do pixel perfect erasing
As far as multiple colors, I made a subclass of UIBezierPath that includes a color property. Since colors are now serializable via NSCoding, they are easily saveable. In addition, I have a "stroke" object, which holds all of the paths the user created in one continuous stroke. This way I can handle undo / redo correctly.
Hope this info helps.
I am creating a custom button that needs to be able to glow to a varying degree
How would I use these pictures to make a button that 'glows' the diamond when it is pressed, and have this glow gradually fade back to inert state?
I want to churn out several different colours of diamond as well... I am hoping to generate all different coloured diamonds from the same stock images presented here.
I would like to get my head around the basic methods available, in enough detail that I can see each one through and make a decision which path to take...
My tangled efforts so far... ( I will delete all of this, or move it into possibly several answers as a solution unfolds... )
I can see 3 potential solution paths:
GL
it looks as though GL has everything it takes to get complete fine-grained control over the process, although functions exposed by core graphics come tantalisingly close, and that would save several hundred lines of code spread over a bunch of source files, which seems a bit ridiculous for such a basic task.
core graphics, and core animation to accomplish the blending
documentation goes on to say
Anything underneath the unpainted samples, such as the current fill color or other drawing, shows through.
so I can chroma-key mask the left image, setting {0,0,0} ie Black as the key.
this at least secures a transparent background, now I have to work on making it yellow instead of grey.
so maybe I could have started instead with setting a yellow back colour for my image context, then use some CGContextSetBlendMode(...) to imprint the diamond on the yellow, THEN use chroma-key masking to get a transparent background
ok, this covers at least getting the basic unlit image on-screen
now I could overlay the sparkly image, using some blend mode, maybe I could keep it in its current greyscale state, and that would just boost the colours of the original
only problem with this is that it is a lot of heavy real-time blending
so maybe I could pre-calculate every image in the animation... this is looking increasingly mucky...
Cocos2D
if this allows me to set the blend mode to additive blending then I could just composite the glowing image over the original image with an appropriate Alpha setting.
After digging through a lot of documentation, the optimal solution seems to be to use core graphics functions to get the source images into a single 2-component GL texture, and then use GL to blend between them.
I will need to pass a uniform value glow_factor into the shader
The obvious solution might seem to simply use
r,g,b = in_r,g,b * { (1 - glow_factor) * inertPixel + glow_factor * shinyPixel }
(where inertPixel is the appropriate pixel of the inert diamond etc)...
it looks like I would also do well to manufacture my own sparkles and add them over the top; a gem should sparkle white irrespective of its characteristic colour.
After having looked at this problem a little more, I can see several solutions
Solution A -- store the transition from glow=0 to glow=1 as 60 frames in memory, then load the appropriate frame into a GL texture every time it is required.
this has an obvious benefit that a graphic designer could construct the entire sequence and I could load it in as a bunch of PNG files.
another advantage is that these frames wouldn't need to be played in sequence... the appropriate frame can be chosen on-the-fly
however, it has a potential drawback of a lot of sending data RAM->VRAM
this can be optimised by using glTexSubImage2D; several frames can be sent simultaneously and then unpacked from within GL... in fact maybe the entire sequence. if this is so, then it would make sense to use PVRT texture compression.
iOS: playing a frame-by-frame greyscale animation in a custom colour
Solution B -- load glow=0 and glow=1 images as GL textures, and manually write shader code that takes in the glow factor as a uniform and performs the blend
this has an advantage that it is close to the wire and can be tweaked in all sorts of ways. Also it is going to be very efficient. This advantage is that it is a big extra slice of code to maintain.
Solution C -- set glBlendMode to perform additive blending.
then draw the glow=0 image image, setting eg alpha=0.2 on each vertex.
then draw the glow=1 image image, setting eg alpha=0.8 on each vertex.
this has an advantage that it can be achieved with a more generic code structure -- ie a very general ' draw textured quad / sprite ' class.
disadvantage is that without some sort of wrapper it is a bit messy... in my game I have a couple of dozen diamonds -- at any one time maybe 2 or 3 are likely to be glowing. so first-pass I would render EVERYTHING ( just need to set Alpha appropriately for everything that is glowing ) and then on the second pass I could draw the glowing sprite again with appropriate Alpha for everything that IS glowing.
it is worth noting that if I pursue solution A, this would involve creating some sort of real-time movie player object, which could be a very useful reusable code component.
I have written a 2D Jump&Run Engine resulting in a 320x224 (320x240) image. To maintain the old school "pixely"-feel to it, I would like to scale the resulting image by 2 or 3 or 4, according to the resolution of the user.
I don't want to scale each and every sprite, but the resulting image!
Thanks in advance :)
Bob's answer is correct about changing the filtering mode to TextureFilter.Point to keep things nice and pixelated.
But possibly a better method than scaling each sprite (as you'd also have to scale the position of each sprite) is to just pass a matrix to SpriteBatch.Begin, like so:
sb.Begin(/* first three parameters */, Matrix.CreateScale(4f));
That will give you the scaling you want without having to modify all your draw calls.
However it is worth noting that, if you use floating-point offsets in your game, you will end up with things not aligned to pixel boundaries after you scale up (with either method).
There are two solutions to this. The first is to have a function like this:
public static Vector2 Floor(Vector2 v)
{
return new Vector2((float)Math.Floor(v.X), (float)Math.Floor(v.Y));
}
And then pass your position through that function every time you draw a sprite. Although this might not work if your sprites use any rotation or offsets. And again you'll be back to modifying every single draw call.
The "correct" way to do this, if you want a plain point-wise scale-up of your whole scene, is to draw your scene to a render target at the original size. And then draw your render target to screen, scaled up (with TextureFilter.Point).
The function you want to look at is GraphicsDevice.SetRenderTarget. This MSDN article might be worth reading. If you're on or moving to XNA 4.0, this might be worth reading.
I couldn't find a simpler XNA sample for this quickly, but the Bloom Postprocess sample uses a render target that it then applies a blur shader to. You could simply ignore the shader entirely and just do the scale-up.
You could use a pixelation effect. Draw to a RenderTarget2D, then draw the result to the screen using a Pixel Shader. There's a tool called Shazzam Shader Editor that let's you try out pixel shaders and it includes one that does pixelation:
http://shazzam-tool.com/
This may not be what you wanted, but it could be good for allowing a high-resolution mode and for having the same effect no matter what resolution was used...
I'm not exactly sure what you mean by "resulting in ... an image" but if you mean your end result is a texture then you can draw that to the screen and set a scale:
spriteBatch.Draw(texture, position, source, color, rotation, origin, scale, effects, depth);
Just replace the scale with whatever number you want (2, 3, or 4). I do something similar but scale per sprite and not the resulting image. If you mean something else let me know and I'll try to help.
XNA defaults to anti-aliasing the scaled image. If you want to retain the pixelated goodness you'll need to draw in immediate sort mode and set some additional parameters:
spriteBatch.Begin(SpriteBlendMode.AlphaBlend, SpriteSortMode.Immediate, SaveStateMode.None);
GraphicsDevice.SamplerStates[0].MagFilter = TextureFilter.Point;
GraphicsDevice.SamplerStates[0].MinFilter = TextureFilter.Point;
GraphicsDevice.SamplerStates[0].MipFilter = TextureFilter.Point;
It's either the Point or the None TextureFilter. I'm at work so I'm trying to remember off the top of my head. I'll confirm one way or the other later today.