Here I created one simple OpenGLES iPad sample.
//----------------------------------------
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glFrustumf(-1, 1, -1, 1, 0, 20);
glMatrixMode(GL_MODELVIEW);
static const GLfloat squareVertices[] = {
-0.5f, -0.33f,1.6,
0.5f, -0.33f,1.6,
-0.5f, 0.33f,1.6,
0.5f, 0.33f,1.6
};
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
. . . . . . .
. . . . . . .
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
//----------------------------------------
I used gluLookAt in mac sample, but it is not found in iPad. What is the basic mistake in above code? why square is not visible?
if I change z value in vertex array then works fine.
static const GLfloat squareVertices[] = {
-0.5f, -0.33f,0.5,
0.5f, -0.33f,0.5,
-0.5f, 0.33f,0.5,
0.5f, 0.33f,0.5
};
I thought z value range should be 0-20, here 1.6, its in range. I don't like to change vertex value now. Help me to set glFrustumf.
The near and far values passed to glFrustum are not absolute coordinates in object space. They are the distances from the camera to the near and far planes. That means that you can't say that a z coordinate of 1.6 is necessarily in your frustum unless you know where the camera position is and what direction it is pointing in.
For example if your camera is at (0,0,-19) pointed at the origin and you pass 0 and 20 to glFrustum for near and far then in object space the near plane is at -19 (-19+0) and the far plane is at 1 (-19+20). In such a case the square with z=1.6 would be past the far plane but the one with z=0.5 would be inside the frustum. Similarly if your camera was at (0,0,1) with the same values for glFrustum then in object space the near plane would be at 1 and the far would be at -19 so 0.5 would be in and 1.6 would be out.
Related
I've got problem with obtaining proper camera pose from iPad camera using OpenCV.
I'm using custom made 2D marker (based on AruCo library ) - I want to render 3D cube over that marker using OpenGL.
In order to recieve camera pose I'm using solvePnP function from OpenCV.
According to THIS LINK I'm doing it like this:
cv::solvePnP(markerObjectPoints, imagePoints, [self currentCameraMatrix], _userDefaultsManager.distCoeffs, rvec, tvec);
tvec.at<double>(0, 0) *= -1; // I don't know why I have to do it, but translation in X axis is inverted
cv::Mat R;
cv::Rodrigues(rvec, R); // R is 3x3
R = R.t(); // rotation of inverse
tvec = -R * tvec; // translation of inverse
cv::Mat T(4, 4, R.type()); // T is 4x4
T(cv::Range(0, 3), cv::Range(0, 3)) = R * 1; // copies R into T
T(cv::Range(0, 3), cv::Range(3, 4)) = tvec * 1; // copies tvec into T
double *p = T.ptr<double>(3);
p[0] = p[1] = p[2] = 0;
p[3] = 1;
camera matrix & dist coefficients are coming from findChessboardCorners function, imagePoints are manually detected corners of marker (you can see them as green square in the video posted below), and markerObjectPoints are manually hardcoded points that represents marker corners:
markerObjectPoints.push_back(cv::Point3d(-6, -6, 0));
markerObjectPoints.push_back(cv::Point3d(6, -6, 0));
markerObjectPoints.push_back(cv::Point3d(6, 6, 0));
markerObjectPoints.push_back(cv::Point3d(-6, 6, 0));
Because marker is 12 cm long in real world, I've chosed the same size in the for easier debugging.
As a result I'm recieving 4x4 matrix T, that I'll use as ModelView matrix in OpenCV.
Using GLKit drawing function looks more or less like this:
- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect {
// preparations
glClearColor(0.0, 0.0, 0.0, 0.0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
float aspect = fabsf(self.bounds.size.width / self.bounds.size.height);
effect.transform.projectionMatrix = GLKMatrix4MakePerspective(GLKMathDegreesToRadians(39), aspect, 0.1f, 1000.0f);
// set modelViewMatrix
float mat[16] = generateOpenGLMatFromFromOpenCVMat(T);
currentModelMatrix = GLKMatrix4MakeWithArrayAndTranspose(mat);
effect.transform.modelviewMatrix = currentModelMatrix;
[effect prepareToDraw];
glDrawArrays(GL_TRIANGLES, 0, 36); // draw previously prepared cube
}
I'm not rotating everything for 180 degrees around X axis (as it was mentioned in previously linked article), because I doesn't look as necessary.
The problem is that it doesn't work! Translation vector looks OK, but X and Y rotations are messed up :(
I've recorded a video presenting that issue:
http://www.youtube.com/watch?v=EMNBT5H7-os
I've tried almost everything (including inverting all axises one by one), but nothing actually works.
What should I do? How should I properly display that 3D cube? Translation / rotation vectors that come from solvePnP are looking reasonable, so I guess that I can't correctly map these vectors to OpenGL matrices.
Thanks to Djo1509 from http://answers.opencv.org/ I've found out that the problem was unnecessary transposed rotation matrix R matrix used as part of matrix T, and unnecessary tvec = -R * tvec operation.
For more info look there
I'm looking for a simple implementation for arcball rotation on 3D models with quaternions, specifically using GLKit on iOS. So far, I have examined the following sources:
Arcball rotation with GLKit
How to rotate a 3D object with touches using OpenGL
I've also been trying to understand source code and maths from here and here. I can rotate my object but it keeps jumping around at certain angles, so I fear gimbal lock is at play. I'm using gesture recognizers to control the rotations (pan gestures affect roll and yaw, rotate gestures affect pitch). I'm attaching my code for the quaternion handling as well as the modelview matrix transformation.
Variables:
GLKQuaternion rotationE;
Quaternion Handling:
- (void)rotateWithXY:(float)x and:(float)y
{
const float rate = M_PI/360.0f;
GLKVector3 up = GLKVector3Make(0.0f, 1.0f, 0.0f);
GLKVector3 right = GLKVector3Make(1.0f, 0.0f, 0.0f);
up = GLKQuaternionRotateVector3(GLKQuaternionInvert(self.rotationE), up);
self.rotationE = GLKQuaternionMultiply(self.rotationE, GLKQuaternionMakeWithAngleAndVector3Axis(x*rate, up));
right = GLKQuaternionRotateVector3(GLKQuaternionInvert(self.rotationE), right);
self.rotationE = GLKQuaternionMultiply(self.rotationE, GLKQuaternionMakeWithAngleAndVector3Axis(y*rate, right));
}
- (void)rotateWithZ:(float)z
{
GLKVector3 front = GLKVector3Make(0.0f, 0.0f, -1.0f);
front = GLKQuaternionRotateVector3(GLKQuaternionInvert(self.rotationE), front);
self.rotationE = GLKQuaternionMultiply(self.rotationE, GLKQuaternionMakeWithAngleAndVector3Axis(z, front));
}
Modelview Matrix Transformation (Inside Draw Loop):
// Get Quaternion Rotation
GLKVector3 rAxis = GLKQuaternionAxis(self.transformations.rotationE);
float rAngle = GLKQuaternionAngle(self.transformations.rotationE);
// Set Modelview Matrix
GLKMatrix4 modelviewMatrix = GLKMatrix4Identity;
modelviewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, -0.55f);
modelviewMatrix = GLKMatrix4Rotate(modelviewMatrix, rAngle, rAxis.x, rAxis.y, rAxis.z);
modelviewMatrix = GLKMatrix4Scale(modelviewMatrix, 0.5f, 0.5f, 0.5f);
glUniformMatrix4fv(self.sunShader.uModelviewMatrix, 1, 0, modelviewMatrix.m);
Any help is greatly appreciated, but I do want to keep it as simple as possible and stick to GLKit.
There seem to be a few issues going on here.
You say that you're using [x,y] to pan, but it looks more like you're using them to pitch and yaw. To me, at least, panning is translation, not rotation.
Unless I'm missing something, it also looks like your replacing the entire rotation everytime you try to update it. You rotate a vector by the inverse of the current rotation and then create a quaternion from that vector and some angle. I believe that this is equivalent to creating the quaternion from the original vector and then rotating it by the current rotation inverse. So you have q_e'*q_up. Then you multiply that with the current rotation, which gives q_e*q_e'*q_up = q_up. The current rotation is canceled out. This doesn't seem like it's what you want.
All you really need to do is create a new quaternion from axis-and-angle and then multiply it with the current quaternion. If the new quaternion is on the left, the orientation change will use the eye-local frame. If the new quaternion is on the right, the orientation change will be in the global frame. I think you want:
self.rotationE =
GLKQuaternionMultiply(
GLKQuaternionMakeWithAngleAndVector3Axis(x*rate, up),self.rotationE);
Do this, without the pre-rotation by inverse for all three cases.
I've never used the GLKit, but it's uncommon to extract axis-angle when converting from Quaternion to Matrix. If the angle is zero, the axis is undefined. When it's near zero, you'll have numeric instability. It looks like you should be using GLKMatrix4MakeWithQuaternion and then multiplying the resulting matrix with your translation matrix and scale matrix:
GLKMatrix4 modelviewMatrix =
GLKMatrix4Multiply( GLKMatrix4MakeTranslation(0.0f, 0.0f, -0.55f),
GLKMatrix4MakeWithQuaternion( self.rotationE ) );
modelviewMatrix = GLKMatrix4Scale( modelviewMatrix, 0.5f, 0.5f, 0.5f );
I was recently asked a bit more about my resulting implementation of this problem, so here it is!
- (void)rotate:(GLKVector3)r
{
// Convert degrees to radians for maths calculations
r.x = GLKMathDegreesToRadians(r.x);
r.y = GLKMathDegreesToRadians(r.y);
r.z = GLKMathDegreesToRadians(r.z);
// Axis Vectors w/ Direction (x=right, y=up, z=front)
// In OpenGL, negative z values go "into" the screen. In GLKit, positive z values go "into" the screen.
GLKVector3 right = GLKVector3Make(1.0f, 0.0f, 0.0f);
GLKVector3 up = GLKVector3Make(0.0f, 1.0f, 0.0f);
GLKVector3 front = GLKVector3Make(0.0f, 0.0f, 1.0f);
// Quaternion w/ Angle and Vector
// Positive angles are counter-clockwise, so convert to negative for a clockwise rotation
GLKQuaternion q = GLKQuaternionIdentity;
q = GLKQuaternionMultiply(GLKQuaternionMakeWithAngleAndVector3Axis(-r.x, right), q);
q = GLKQuaternionMultiply(GLKQuaternionMakeWithAngleAndVector3Axis(-r.y, up), q);
q = GLKQuaternionMultiply(GLKQuaternionMakeWithAngleAndVector3Axis(-r.z, front), q);
// ModelView Matrix
GLKMatrix4 modelViewMatrix = GLKMatrix4Identity;
modelViewMatrix = GLKMatrix4Multiply(modelViewMatrix, GLKMatrix4MakeWithQuaternion(q));
}
Hope you put it to good use :)
I'm trying to use OpenCV to do some basic augmented reality. The way I'm going about it is using findChessboardCorners to get a set of points from a camera image. Then, I create a 3D quad along the z = 0 plane and use solvePnP to get a homography between the imaged points and the planar points. From that, I figure I should be able to set up a modelview matrix which will allow me to render a cube with the right pose on top of the image.
The documentation for solvePnP says that it outputs a rotation vector "that (together with [the translation vector] ) brings points from the model coordinate system to the camera coordinate system." I think that's the opposite of what I want; since my quad is on the plane z = 0, I want a a modelview matrix which will transform that quad to the appropriate 3D plane.
I thought that by performing the opposite rotations and translations in the opposite order I could calculate the correct modelview matrix, but that seems not to work. While the rendered object (a cube) does move with the camera image and seems to be roughly correct translationally, the rotation just doesn't work at all; it on multiple axes when it should only be rotating on one, and sometimes in the wrong direction. Here's what I'm doing so far:
std::vector<Point2f> corners;
bool found = findChessboardCorners(*_imageBuffer, cv::Size(5,4), corners,
CV_CALIB_CB_FILTER_QUADS |
CV_CALIB_CB_FAST_CHECK);
if(found)
{
drawChessboardCorners(*_imageBuffer, cv::Size(6, 5), corners, found);
std::vector<double> distortionCoefficients(5); // camera distortion
distortionCoefficients[0] = 0.070969;
distortionCoefficients[1] = 0.777647;
distortionCoefficients[2] = -0.009131;
distortionCoefficients[3] = -0.013867;
distortionCoefficients[4] = -5.141519;
// Since the image was resized, we need to scale the found corner points
float sw = _width / SMALL_WIDTH;
float sh = _height / SMALL_HEIGHT;
std::vector<Point2f> board_verts;
board_verts.push_back(Point2f(corners[0].x * sw, corners[0].y * sh));
board_verts.push_back(Point2f(corners[15].x * sw, corners[15].y * sh));
board_verts.push_back(Point2f(corners[19].x * sw, corners[19].y * sh));
board_verts.push_back(Point2f(corners[4].x * sw, corners[4].y * sh));
Mat boardMat(board_verts);
std::vector<Point3f> square_verts;
square_verts.push_back(Point3f(-1, 1, 0));
square_verts.push_back(Point3f(-1, -1, 0));
square_verts.push_back(Point3f(1, -1, 0));
square_verts.push_back(Point3f(1, 1, 0));
Mat squareMat(square_verts);
// Transform the camera's intrinsic parameters into an OpenGL camera matrix
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
// Camera parameters
double f_x = 786.42938232; // Focal length in x axis
double f_y = 786.42938232; // Focal length in y axis (usually the same?)
double c_x = 217.01358032; // Camera primary point x
double c_y = 311.25384521; // Camera primary point y
cv::Mat cameraMatrix(3,3,CV_32FC1);
cameraMatrix.at<float>(0,0) = f_x;
cameraMatrix.at<float>(0,1) = 0.0;
cameraMatrix.at<float>(0,2) = c_x;
cameraMatrix.at<float>(1,0) = 0.0;
cameraMatrix.at<float>(1,1) = f_y;
cameraMatrix.at<float>(1,2) = c_y;
cameraMatrix.at<float>(2,0) = 0.0;
cameraMatrix.at<float>(2,1) = 0.0;
cameraMatrix.at<float>(2,2) = 1.0;
Mat rvec(3, 1, CV_32F), tvec(3, 1, CV_32F);
solvePnP(squareMat, boardMat, cameraMatrix, distortionCoefficients,
rvec, tvec);
_rv[0] = rvec.at<double>(0, 0);
_rv[1] = rvec.at<double>(1, 0);
_rv[2] = rvec.at<double>(2, 0);
_tv[0] = tvec.at<double>(0, 0);
_tv[1] = tvec.at<double>(1, 0);
_tv[2] = tvec.at<double>(2, 0);
}
Then in the drawing code...
GLKMatrix4 modelViewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, 0.0f);
modelViewMatrix = GLKMatrix4Translate(modelViewMatrix, -tv[1], -tv[0], -tv[2]);
modelViewMatrix = GLKMatrix4Rotate(modelViewMatrix, -rv[0], 1.0f, 0.0f, 0.0f);
modelViewMatrix = GLKMatrix4Rotate(modelViewMatrix, -rv[1], 0.0f, 1.0f, 0.0f);
modelViewMatrix = GLKMatrix4Rotate(modelViewMatrix, -rv[2], 0.0f, 0.0f, 1.0f);
The vertices I'm rendering create a cube of unit length around the origin (i.e. from -0.5 to 0.5 along each edge.) I know with OpenGL translation functions performed transformations in "reverse order," so the above should rotate the cube along the z, y, and then x axes, and then translate it. However, it seems like it's being translated first and then rotated, so perhaps Apple's GLKMatrix4 works differently?
This question seems very similar to mine, and in particular coder9's answer seems like it might be more or less what I'm looking for. However, I tried it and compared the results to my method, and the matrices I arrived at in both cases were the same. I feel like that answer is right, but that I'm missing some crucial detail.
You have to make sure the axis are facing the correct direction. Especially, the y and z axis are facing different directions in OpenGL and OpenCV to ensure the x-y-z basis is direct. You can find some information and code (with an iPad camera) in this blog post.
-- Edit --
Ah ok. Unfortunately, I used these resources to do it the other way round (opengl ---> opencv) to test some algorithms. My main issue was that the row order of the images was inverted between OpenGL and OpenCV (maybe this helps).
When simulating cameras, I came across the same projection matrices that can be found here and in the generalized projection matrix paper. This paper quoted in the comments of the blog post also shows some link between computer vision and OpenGL projections.
I'm not an IOS programmer, so this answer might be misleading!
If the problem is not in the order of applying the rotations and the translation, then suggest using a simpler and more commonly used coordinate system.
The points in the corners vector have the origin (0,0) at the top left corner of the image and the y axis is towards the bottom of the image. Often from math we are used to think of the coordinate system with the origin at the center and y axis towards the top of the image. From the coordinates you're pushing into board_verts I'm guessing you're making the same mistake. If that's the case, it's easy to transform the positions of the corners by something like this:
for (i=0;i<corners.size();i++) {
corners[i].x -= width/2;
corners[i].y = -corners[i].y + height/2;
}
then you call solvePnP(). Debugging this is not that difficult, just print the positions of the four corners and the estimated R and T, and see if they make sense. Then you can proceed to the OpenGL step. Please let me know how it goes.
I am reading sample code GLVideoFrame from WWDC2010. In this sample, it has code like below:
static const GLfloat squareVertices[] = {
-0.5f, -0.33f,
0.5f, -0.33f,
-0.5f, 0.33f,
0.5f, 0.33f,
};
...
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glTranslatef(0.0f, (GLfloat)(sinf(transY)/2.0f), 0.0f);
transY += 0.075f;
...
glVertexPointer(2, GL_FLOAT, 0, squareVertices);
Notice that this code does not call any function like glFrustum or glOrtho for openGL projection setting.
By only calling gLoadIdentity(), what will be the "default" view volume?
it wil be a perspective project or orthographic projection?
edited:
to be more specific,
is the view volume a cube that "ranging from -1 to 1 in all three axes" ?
OpenGL assumes that after the ModelView and Projection transform, all visible elements are in clip space (or NDC space); it uses the cube [-1;+1]^3. The matrices contents is entirely your responsibility. Since you load the identity matrix, there is no Projection at all.
I'm learning how to use OpenGL ES 2.0 on iOS. Right now I want to just do some basic 2D animation (e.g. move a rectangle around the screen and change its size). I've started out with the project template for OpenGL ES provided by Apple in Xcode. My drawing code looks like this:
static GLfloat squareVertices[] = {
-0.5f, -0.33f,
0.5f, -0.33f,
-0.5f, 0.33f,
0.5f, 0.33f
};
// Update attribute values.
glVertexAttribPointer(VERTEX_ATTR, 2, GL_FLOAT, 0, 0, squareVertices);
glEnableVertexAttribArray(VERTEX_ATTR);
glVertexAttribPointer(COLOR_ATTR, 4, GL_UNSIGNED_BYTE, 1, 0, squareColors);
glEnableVertexAttribArray(COLOR_ATTR);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 5);
Now this will draw a nice rectangle in the middle of the screen. But if I start to change the rectangle by adding the following code, it starts to look funky:
squareVertices[5] -= .001;
squareVertices[7] -= .001;
It is as if part of the rectangle is attached to the center of the screen. I am completely new to OpenGL ES so I'm sure my problem is obvious. I also assume this has something to do with OpenGL ES being a 3D graphics library and I'm trying to treat it as a 2D space. So my question is: What is the best way to draw and animate 2D objects in OpenGL ES 2.0? I've seen some stuff online for OpenGL ES 1.1, but that is not much help to me. Are their special techniques for 2D drawing in OpenGL ES 2.0, or is there some sort of 2D drawing mode?
Any guidance would be greatly appreciated.
#macinjosh: This is a response to your updated question for those who are interested in the answer. I'm guessing you've gained further knowledge since Dec '10 when you posted!
OpenGL vertices are in 3D, not 2D. To be completely truthful, they're actually in 4D since they include a 'w' component as well, but for now just consider that as value 1.0 like a homogenous vector.
Due to their 3D-ness, unless you add a 'z' component in a shader you must specify one as a vertex attribute.
Because you have a '3' as the third parameter of glVertexAttribPointer. I believe you can set it to 2 but I haven't tried this in GL. My guess is the missing z axis would be filled in internally as '0' (but again, try it and see!). Internally, it's probably all going to be 4 float vectors with the 4th ('w') parameter used for homogeneous matrix multiplication gubbins.
If you're doing this on a mobile device you may also wish to look into fixed point maths (faster on some devices that don't have a floating point co-pro) and also Vertex Buffer Objects, which are more efficient on many machines. Also, a fixed vertex format such as used by
glInterleavedArrays(format_code, stride, data)
may prove to be more efficient as the device may have optimized code paths for whatever 'format_code' you decide to go with.
I'm new with OpenGL, but vertex doesn't require 3 float, one every axis : X,Y,Z?
So, the first vertex array will be :
( -0.50f, -0.33f, 0.50f)
( -0.33f, -0.50f, 0.33f)
( 0.50f, 0.33f, ?????)
the second will be :
( -0.50f, -0.33f, 0.00f )
( 0.50f, -0.33f, 0.00f )
( -0.50f, 0.33f, 0.00f )
( 0.50f, 0.33f, 0.00f )
After some playing around I changed the drawing code to this:
static GLfloat squareVertices[12] = {
-0.5f, -0.33f, 0.0,
0.5f, -0.33f, 0.0,
-0.5f, 0.33f, 0.0,
0.5f, 0.33f, 0.0
};
// Update attribute values.
glVertexAttribPointer(VERTEX_ATTR, 3, GL_FLOAT, 0, 0, squareVertices);
glEnableVertexAttribArray(VERTEX_ATTR);
glVertexAttribPointer(COLOR_ATTR, 4, GL_UNSIGNED_BYTE, 1, 0, squareColors);
glEnableVertexAttribArray(COLOR_ATTR);
squareVertices[7] -= .001;
squareVertices[10] -= .001;
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
Adding the third 0.0 float to each vertex seemed to do the trick. I am unclear on why this is so if any one could shed some light I would apreciate it.