How to draw on the fly 2D pixel-by-pixel in MonoGame/XNA? - xna

I was wondering how to draw on the fly pixel by pixel in XNA/MonoGame and could only find this. Problem is, the question wasn't centered about how to actually draw pixel by pixel but rather manage resources and updating stuff.
Because I couldn't find any simple and short answer and was wondering how to do for some time now, I'll share a snippet I stumbled upon (see answer).
NB: If anyone having a better approach or an alternative method to do it, feel free to post or comment!

You need to use the SetData() method of a Texture2D object you previously initialized with a 1 by 1 size.
Texture2D pixel; //ie. in class declaration
And later, for instance in the Initialize method:
pixel = new Texture2D(GraphicsDevice, 1, 1);
pixel.SetData(new Color[] {Color.White});
Edit:
As Strom mentionned in his answer, the Texture2D object must be declared at the class level. You should avoid declaring or modifying a texture in the Draw() method as it'll slow the game loop.

Declare a class level variable for your texture and sizes:
Texture2D Tex, Tex1, Tex2;
const int WIDTH = 32;
const int HEIGHT = 32;
Color[] TexSource = new Color[WIDTH * HEIGHT];
Initialize the variables in LoadContent()
int radius = 5;
point center = new Point(WIDTH >> 2, HEIGHT >> 2);
Tex = new Texture2D(GraphicsDevice, WIDTH, HEIGHT);
for(int x = 0 ; x < WIDTH; x++)
for(int y = 0 ; y < HEIGHT; y++)
if(Math.Sqrt((x - center.x)* (x - center.x)) < radius && Math.Sqrt((y - center.y) * (y - center.y) < radius))
TexSource[x + y * WIDTH] = Color.Red;
else
TexSource[x + y * WIDTH] = Color.White;
Tex = SetData<Color>(TexSource);
//The resulting texture is a red circle on a white background.
//If you want to draw on top of an existing texture Tex1 as Tex2:
Tex1 = Content.Load<Texture2D>(Tex1Name);
Tex2 = new Texture2D(GraphicsDevice, Tex1.Width, Tex1.Height);
TexSource = Tex1.GetData<Color>();
// Draw a white \ on the image, we will assume tex1 is square for this task
for(int x = 0 ; x < Tex1.Width; x++)
for(int y = 0 ; y < Tex1.Width; y++)
if(x==y) TexSource[x + y * Tex1.Width] = Color.White;
You may change the textures in Update()
But, never create or modify a texture in Draw() ever.
The SpriteBatch.Draw() call expects the GraphicsDevice buffer to contain all of the texture data (by extension to the GPU buffer), any changes still happening(from Update and IsRunningSlowly == true) at this point will cause tearing when rendered.
The workaround of GraphicsDevice.Textures[0] = null; blocks the Draw call and the game until transfer to the GPU is complete, thus slowing the entire game loop.

Related

Convolution of Image Processing in Processing language

Since the Corona situation characterizes my studies as self-study, as a Processing-Language newbie I don't have an easy time getting into the subject of image processing , more specifically convolution. Therefore I hope that you can help me.
My lecturer, who unfortunately is nearly never reachable, left me the following conv code. The theory behind convolution is clear to me, but I have many gaps in understanding related to the code. Could someone leave a line comment so that I can get into the code a bit more fluently?
The Code is following
color convolution (int x, int y, float[][] matrix, int matrix_size, PImage img){
float rtotal = 0.0;
float gtotal = 0.0;
float btotal = 0.0;
int offset = matrix_size / 2;
for (int i = 0; i < matrix_size; i++){
for (int j= 0; j < matrix_size; j++){
int xloc = x+i-offset;
int yloc = y+j-offset;
int loc = xloc + img.width*yloc;
rtotal += (red(img.pixels[loc]) * matrix[i][j]);
gtotal += (green(img.pixels[loc]) * matrix[i][j]);
btotal += (blue(img.pixels[loc]) * matrix[i][j]);
}
}
rtotal = constrain(rtotal, 0, 255);
gtotal = constrain(gtotal, 0, 255);
btotal = constrain(btotal, 0, 255);
return color(rtotal, gtotal, btotal);
}
I have to do a bit of guesswork since I'm not positive about all of the functions you're using and I'm not familiar with the Processing 3+ library, but here's my best shot at it.
color convolution (int x, int y, float[][] matrix, int matrix_size, PImage img){
// Note: the 'matrix' parameter here will also frequently be referred to as
// a 'window' or 'kernel' in research
// I'm not certain what your PImage class is from, but I'll assume
// you're using the Processing 3+ library and work off of that assumption
// how much of each color we see within the kernel (matrix) space
float rtotal = 0.0;
float gtotal = 0.0;
float btotal = 0.0;
// this offset is to zero-center our kernel
// the fact that we use matrix_size / 2 sort of implicitly
// alludes to the fact that our matrix_size should be an odd-number
// so that we can have a middle-pixel
int offset = matrix_size / 2;
// looping through the kernel. the fact that we use 'matrix_size'
// as our end-condition for both dimensions means that our 'matrix' kernel
// must always be a square
for (int i = 0; i < matrix_size; i++){
for (int j= 0; j < matrix_size; j++){
// calculating the index conversion from 2D to the 1D format that PImage uses
// refer to: https://processing.org/tutorials/pixels/
// for a better understanding of PImage indexing (about 1/3 of the way down the page)
// WARNING: by subtracting the offset it is possible to hit negative
// x,y values here if you pick an x or y position less than matrix_size / 2.
// the same index-out-of-bounds can occur on the high end.
// When you convolve using a kernel of N x N size (N here would be matrix_size)
// you can only convolve from [N / 2, Width - (N / 2)] for x and y
int xloc = x+i-offset;
int yloc = y+j-offset;
// this is the final 1D PImage index that corresponds to [xloc, yloc] in our 2D image
// really go back up and take a look at the link if this doesn't make sense, it's pretty good
int loc = xloc + img.width*yloc;
// I have to do some speculation again since I'm not certain what red(img.pixels[loc]) does
// I'll assume it returns the red red channel of the pixel
// this section just adds up all of the pixel colors multiplied by the value in the kernel
rtotal += (red(img.pixels[loc]) * matrix[i][j]);
gtotal += (green(img.pixels[loc]) * matrix[i][j]);
btotal += (blue(img.pixels[loc]) * matrix[i][j]);
}
}
// the fact that no further division or averaging happens after the for-loops implies
// that the kernel you feed in should have balanced values for your kernel size
// for example, a kernel that's designed to average out the color over the 3 x 3 area
// it covers (this would be like blurring the image) would be filled with 1/9
// in general: the kernel you're using should have a sum of 1 for all of the numbers inside
// this is just 'in general' you can play around with not doing that, but you'll probably notice a
// darkening effect for when the sum is less than 1, and a brightening effect if it's greater than 1
// for more info on kernels, read this: https://en.wikipedia.org/wiki/Kernel_(image_processing)
// I don't have the code for this constrain function,
// but it's almost certainly just your typical clamp (constrains the values to [0, 255])
// Note: this means that your values saturate at 0 and 255
// if you see a lot of black or white then that means your kernel
// probably isn't balanced as mentioned above
rtotal = constrain(rtotal, 0, 255);
gtotal = constrain(gtotal, 0, 255);
btotal = constrain(btotal, 0, 255);
// Finished!
return color(rtotal, gtotal, btotal);
}

How to calculate camera ray position for use with XMVector3Unproject(), DirectX11?

I'm trying to create a ray-casting camera in DirectX11 using XMVector3Unproject(). From my understanding, I will be passing in the (Vector3)position of the pixel on the near plane, and in separate call, a corresponding position on the far plane. Then I would subtract these vectors to get the direction of the ray. The origin would then be the Unprojected coordinate on the near plane. My problem here is calculating the origin of the ray to be passed in.
Example
// assuming screenHeight and screenWidth are the number of pixels.
const uint32_t screenHeight = 768;
const uint32_t screenWidth = 1024;
struct Ray
{
XMFLOAT3 origin;
XMFLOAT3 direction;
};
Ray rays[screenWidth * screenHeight];
for (uint32_t i = 0; i < screenHeight; ++i)
{
for (uint32_t j = 0; j < screenWidth; ++j)
{
// 1. ***calculate and store the current pixel position on the near plane***
// 2. ***calculate the corresponding point on the far plane***
// 3. ***pass both positions separately into XMVector3Unproject() (2 total calls to the function)***
// 4. ***store the returned vectors' difference into rays[i * screenWidth + j].direction***
// 5. ***store the near plane pixel position's returned vector into rays[i * screenWidth + j].origin***
}
}
Hopefully I'm understanding this correctly. Any help in determining the ray origins, or corrections would be greatly appreciated.
According to the documentation, the XMVector3Unproject function gives you the coordinates of a ray you have provided in camera space (Normalized-device coordinates), in object space (given your model matrix).
To generate your camera rays, you consider your camera pinhole (all the light passes through one point, which is your camera (0, 0, 0), then you choose your ray direction. Let say you want to generate W*H camera rays, your loop might look like this
Vector3 ray_origin = Vector3(0, 0, 0);
for (float x = -1.f; x <= 1.f; x += 2.f / W) {
for (float y = -1.f; y <= 1.f; y += 2.f / H) {
Vector3 ray_direction = Normalize(Vector3(x, y, -1.f)) - ray_origin;
Vector3 ray_in_model = Unproject(ray_direction, 0.f, 0.f,
width, height, znear, zfar,
proj, view, model);
}
}
You might also want to have a look at this link which sounds interesting

(MATH ISSUE) Creating a SPIRAL out of points: How do I change "relative" position to absolute position

Recently I had the idea to make a pendulum out of points using Processing, and with a little learning I solved it easily:
int contador = 0;
int curvatura = 2;
float pendulo;
void setup(){
size(300,300);
}
void draw(){
background(100);
contador = (contador + 1) % 360; //"CONTADOR" GOES FROM 0 TO 359
pendulo = sin(radians(contador))*curvatura; //"PENDULO" EQUALS THE SIN OF CONTADOR, SO IT GOES FROM 1 TO -1 REPEATEDLY, THEN IS MULTIPLIED TO EMPHASIZE OR REDUCE THE CURVATURE OF THE LINE.
tallo(width/2,height/3);
println(pendulo);
}
void tallo (int x, int y){ //THE FUNTION TO DRAW THE DOTTED LINE
pushMatrix();
translate(x,y);
float _y = 0.0;
for(int i = 0; i < 25; i++){ //CREATES THE POINTS SEQUENCE.
ellipse(0,0,5,5);
_y+=5;
rotate(radians(pendulo)); //ROTATE THEM ON EACH ITERATION, THIS MAKES THE SPIRAL.
}
popMatrix();
}
So, in a brief, what I did was a function that changed every point position with the rotate fuction, and then I just had to draw the ellipses in the origin coordinates as that is the real thing that changes position and creates the pendulum ilussion.
[capture example, I just need 2 more points if you are so gentile :)]
[capture example]
[capture example]
Everything was OK that far. The problem appeared when I tried to replace the ellipses for a path made of vertices. The problem is obvious: the path is never (visually) made because all vertices would be 0,0 as they move along with the zero coordinates.
So, in order to make the path possible, I need the absolute values for each vertex; and there's the question: How do I get them?
What I know I have to do is to remove the transform functions, create the variables for the X and Y position and update them inside the for, but then what? That's why I cleared this is a maths issue, which operation I have to add in the X and Y variables in order to make the path and its curvature possible?
void tallo (int x, int y){
pushMatrix();
translate(x,y);
//NOW WE START WITH THE CHANGES. LET'S DECLARE THE VARIABLES FOR THE COORDINATES
float _x = 0.0;
float _y = 0.0;
beginShape();
for(int i = 0; i < 25; i++){ //CREATES THE DOTS.
vertex(_x,_y); //CHANGING TO VERTICES AND CALLING THE NEW VARIABLES, OK.
//rotate(radians(pendulo)); <--- HERE IS MY PROBLEM. HOW DO I CONVERT THIS INTO X AND Y COORDINATES?
//_x = _x + ????;
_y = _y + 5 /* + ???? */;
}
endShape();
popMatrix();
}
We need to have in mind that pendulo's x and y values changes in each iteration of the for, it doesn't has to add the same quantity each time. The addition must be progressive. Otherwise, we would see a straight line rotating instead of a curve accentuating its curvature (if you increase curvatura's value to a number greater than 20, you will notice the spiral)
So, rotating the coordinates was a great solution to it, now it's kind of a muddle to think the mathematical solution to the x and y coordinates for the spiral, my secondary's knowledges aren't enough. I know I have to create another variable inside the for in order to do this progression, but what operation should it have?
I would be really glad to know, maths
You could use simple trigonometry. You know the angle and the hypotenuse, so you use cos to get the relative x position, and sin to the y. The position would be relative to the central point.
But before i explain in detail and draw some explanations, let me propose another solution: PVectors
void setup() {
size(400,400);
frameRate(60);
center = new PVector(width/2, height/3); //defined here because width and height only are set after size()
}
void draw() {
background(255);
fill(0);
stroke(0);
angle = arc_magn*sin( (float) frameCount/60 );
draw_pendulum( center );
}
PVector center;
float angle = 0;
float arc_magn = HALF_PI;
float wire_length = 150;
float rotation_angle = PI/20 /60 ; //we divide it by 60 so the first part is the rotation in one second
void draw_pendulum(PVector origin){
PVector temp_vect = PVector.fromAngle( angle + HALF_PI);
temp_vect.setMag(wire_length);
PVector final_pos = new PVector(origin.x+temp_vect.x, origin.y+temp_vect.y );
ellipse( final_pos.x, final_pos.y, 40, 40);
line(origin.x, origin.y, final_pos.x, final_pos.y);
}
You use PVector class static method fromAngle( float angle ) that returns a unity vector of the given angle, then use .setMag() to define it's length.
Those PVector methods will take care of the trigonometry for you.
If you still want to know the math behind it, i can make another example.

Convert an image to a SceneKit Node

I have a bit-map image:
( However this should work with any arbitrary image )
I want to take my image and make it a 3D SCNNode. I've accomplished that much with this code. That takes each pixel in the image and creates a SCNNode with a SCNBox geometry.
static inline SCNNode* NodeFromSprite(const UIImage* image) {
SCNNode *node = [SCNNode node];
CFDataRef pixelData = CGDataProviderCopyData(CGImageGetDataProvider(image.CGImage));
const UInt8* data = CFDataGetBytePtr(pixelData);
for (int x = 0; x < image.size.width; x++)
{
for (int y = 0; y < image.size.height; y++)
{
int pixelInfo = ((image.size.width * y) + x) * 4;
UInt8 alpha = data[pixelInfo + 3];
if (alpha > 3)
{
UInt8 red = data[pixelInfo];
UInt8 green = data[pixelInfo + 1];
UInt8 blue = data[pixelInfo + 2];
UIColor *color = [UIColor colorWithRed:red/255.0f green:green/255.0f blue:blue/255.0f alpha:alpha/255.0f];
SCNNode *pixel = [SCNNode node];
pixel.geometry = [SCNBox boxWithWidth:1.001 height:1.001 length:1.001 chamferRadius:0];
pixel.geometry.firstMaterial.diffuse.contents = color;
pixel.position = SCNVector3Make(x - image.size.width / 2.0,
y - image.size.height / 2.0,
0);
[node addChildNode:pixel];
}
}
}
CFRelease(pixelData);
node = [node flattenedClone];
//The image is upside down and I have no idea why.
node.rotation = SCNVector4Make(1, 0, 0, M_PI);
return node;
}
But the problem is that what I'm doing takes up way to much memory!
I'm trying to find a way to do this with less memory.
All Code and resources can be found at:
https://github.com/KonradWright/KNodeFromSprite
Now you drawing each pixel as SCNBox of certain color, that means:
one GL draw per box
drawing of unnecessary two invisible faces between adjancent boxes
drawing N of same 1x1x1 boxes in a row when one box of 1x1xN can be drawn
Seems like common Minecraft-like optimization problem:
Treat your image is 3-dimensional array (where depth is wanted image extrusion depth), each element representing cube voxel of certain color.
Use greedy meshing algorithm (demo) and custom SCNGeometry to create mesh for SceneKit node.
Pseudo-code for meshing algorithm that skips faces of adjancent cubes (simplier, but less effective than greedy meshing):
#define SIZE_X = 16; // image width
#define SIZE_Y = 16; // image height
// pixel data, 0 = transparent pixel
int data[SIZE_X][SIZE_Y];
// check if there is non-transparent neighbour at x, y
BOOL has_neighbour(x, y) {
if (x < 0 || x >= SIZE_X || y < 0 || y >= SIZE_Y || data[x][y] == 0)
return NO; // out of dimensions or transparent
else
return YES;
}
void add_face(x, y orientation, color) {
// add face at (x, y) with specified color and orientation = TOP, BOTTOM, LEFT, RIGHT, FRONT, BACK
// can be (easier and slower) implemented with SCNPlane's: https://developer.apple.com/library/mac/documentation/SceneKit/Reference/SCNPlane_Class/index.html#//apple_ref/doc/uid/TP40012010-CLSCHSCNPlane-SW8
// or (harder and faster) using Custom Geometry: https://github.com/d-ronnqvist/blogpost-codesample-CustomGeometry/blob/master/CustomGeometry/CustomGeometryView.m#L84
}
for (x = 0; x < SIZE_X; x++) {
for (y = 0; y < SIZE_Y; y++) {
int color = data[x][y];
// skip current pixel is transparent
if (color == 0)
continue;
// check neighbour at top
if (! has_neighbour(x, y + 1))
add_face(x,y, TOP, );
// check neighbour at bottom
if (! has_neighbour(x, y - 1))
add_face(x,y, BOTTOM);
// check neighbour at bottom
if (! has_neighbour(x - 1, y))
add_face(x,y, LEFT);
// check neighbour at bottom
if (! has_neighbour(x, y - 1))
add_face(x,y, RIGHT);
// since array is 2D, front and back faces is always visible for non-transparent pixels
add_face(x,y, FRONT);
add_face(x,y, BACK);
}
}
A lot of depends on input image. If it is not big and without wide variety of colors, it I would go with SCNNode adding SCNPlane's for visible faces and then flattenedClone()ing result.
An approach similar to the one proposed by Ef Dot:
To keep the number of draw calls as small as possible you want to keep the number of materials as small as possible. Here you will want one SCNMaterial per color.
To keep the number of draw calls as small as possible make sure that no two geometry elements (SCNGeometryElement) use the same material. In other words, use one geometry element per material (color).
So you will have to build a SCNGeometry that has N geometry elements and N materials where N is the number of distinct colors in your image.
For each color in you image build a polygon (or group of disjoint polygons) from all the pixels of that color
Triangulate each polygon (or group of polygons) and build a geometry element with that triangulation.
Build the geometry from the geometry elements.
If you don't feel comfortable with triangulating the polygons yourself your can leverage SCNShape.
For each polygon (or group of polygons) create a single UIBezierPath and a build a SCNShape with that.
Merge all the geometry sources of your shapes in a single source, and reuse the geometry elements to create a custom SCNGeometry
Note that some vertices will be duplicated if you use a collection of SCNShapes to build the geometry. With little effort you can make sure that no two vertices in your final source have the same position. Update the indexes in the geometry elements accordingly.
I can also direct you to this excellent GitHub repo by Nick Lockwood:
https://github.com/nicklockwood/FPSControls
It will show you how to generate the meshes as planes (instead of cubes) which is a fast way to achieve what you need for simple scenes using a "neighboring" check.
If you need large complex scenes, then I suggest you go for the solution proposed by Ef Dot using a greedy meshing algorithm.

Replicate OpenCV resize with bilinar interpolation in C (shrink only)

I'm trying to make a copy of the resizing algorithm of OpenCV with bilinear interpolation in C. What I want to achieve is that the resulting image is exactly the same (pixel value) to that produced by OpenCV. I am particularly interested in shrinking and not in the magnification, and I'm interested to use it on single channel Grayscale images. On the net I read that the bilinear interpolation algorithm is different between shrinkings and enlargements, but I did not find formulas for shrinking-implementations, so it is likely that the code I wrote is totally wrong. What I wrote comes from my knowledge of interpolation acquired in a university course in Computer Graphics and OpenGL. The result of the algorithm that I wrote are images visually identical to those produced by OpenCV but whose pixel values are not perfectly identical (in particular near edges). Can you show me the shrinking algorithm with bilinear interpolation and a possible implementation?
Note: The code attached is as a one-dimensional filter which must be applied first horizontally and then vertically (i.e. with transposed matrix).
Mat rescale(Mat src, float ratio){
float width = src.cols * ratio; //resized width
int i_width = cvRound(width);
float step = (float)src.cols / (float)i_width; //size of new pixels mapped over old image
float center = step / 2; //V1 - center position of new pixel
//float center = step / src.cols; //V2 - other possible center position of new pixel
//float center = 0.099f; //V3 - Lena 512x512 lower difference possible to OpenCV
Mat dst(src.rows, i_width, CV_8UC1);
//cycle through all rows
for(int j = 0; j < src.rows; j++){
//in each row compute new pixels
for(int i = 0; i < i_width; i++){
float pos = (i*step) + center; //position of (the center of) new pixel in old map coordinates
int pred = floor(pos); //predecessor pixel in the original image
int succ = ceil(pos); //successor pixel in the original image
float d_pred = pos - pred; //pred and succ distances from the center of new pixel
float d_succ = succ - pos;
int val_pred = src.at<uchar>(j, pred); //pred and succ values
int val_succ = src.at<uchar>(j, succ);
float val = (val_pred * d_succ) + (val_succ * d_pred); //inverting d_succ and d_pred, supposing "d_succ = 1 - d_pred"...
int i_val = cvRound(val);
if(i_val == 0) //if pos is a perfect int "x.0000", pred and succ are the same pixel
i_val = val_pred;
dst.at<uchar>(j, i) = i_val;
}
}
return dst;
}
Bilinear interpolation is not separable in the sense that you can resize vertically and the resize again vertically. See example here.
You can see OpenCV's resize code here.

Resources