Using matrices to transform the Three.js scene graph - webgl

I'm attempting to load a scene from a file into Three.js (custom format, not one that Three.js supports). This particular format describes a scene graph where each node in the tree has a transform specified as a 4x4 matrix. The process for pushing it into Three.js looks something like this:
// Yeah, this is javascript-like psuedocode
function processNodes(srcNode, parentThreeObj) {
for(child in srcNode.children) {
var threeObj = new THREE.Object3D();
// This line is the problem
threeObj.applyMatrix(threeMatrixFromSrcMatrix(child.matrix));
for(mesh in child.meshes) {
var threeMesh = threeMeshFromSrcMesh(mesh);
threeObj.add(threeMesh);
}
parentThreeObj.add(threeObj);
processNodes(child, threeObj); // And recurse!
}
}
Or at least that's what I'd like it to be. As I pointed out, the applyMatrix line doesn't work the way that I would expect. The majority of the scene looks okay, but certain elements that have been rotated aren't aligned properly (while other are, it's strange).
Looking through the COLLADA loader (which does approximately the same thing I'm trying to do) it appears that they decompose the matrix into a translate/rotate/scale and apply each individually. I tried that in place of the applyMatrix shown above:
var props = threeMatrixFromSrcMatrix(child.matrix).decompose();
threeObj.useQuaternion = true;
threeObj.position = props[ 0 ];
threeObj.quaternion = props[ 1 ];
threeObj.scale = props[ 2 ];
This, once again, yields a scene where most elements are in the right place but meshes that previously were misaligned have now been transformed into oblivion somewhere and no longer appear at all. So in the end this is no better than the applyMatrix from above.
Looking through several online discussions about the topic it seems that the recommended way to use matrices for your transforms is to apply them directly to the geometry, not the nodes, so I tried that by manually building the transform matrix like so:
function processNodes(srcNode, parentThreeObj, parentMatrix) {
for(child in srcNode.children) {
var threeObj = new THREE.Object3D();
var childMatrix = threeMatrixFromSrcMatrix(child.matrix);
var objMatrix = THREE.Matrix4();
objMatrix.multiply(parentMatrix, childMatrix);
for(mesh in child.meshes) {
var threeMesh = threeMeshFromSrcMesh(mesh);
threeMesh.geometry.applyMatrix(objMatrix);
threeObj.add(threeMesh);
}
parentThreeObj.add(threeObj);
processNodes(child, threeObj, objMatrix); // And recurse!
}
}
This actually yields the correct results! (minus some quirks with the normals, but I can figure that one out) That's great, but the problem is that we've now effectively flattened the scene hierarchy: Changing the transform on a parent will yield unexpected results on the children because the full transform stack is now "baked in" to the meshes. In this case that's an unacceptable loss of information about the scene.
So how might one go about telling Three.js to do the same logic, but at the appropriate point in the scene graph?
(Sorry, I would dearly love to post some live code examples but that's unfortunately not an option in this case.)

You can use matrixAutoUpdate = false to skip the Three.js scenegraph position/scale/rotation stuff. Then set object.matrix to the matrix you want and all should be dandy (well, it still gets multiplied by parent node matrices, so if you're using absolute modelview matrices you need to hack updateMatrixWorld method on Object3D.)
object.matrixAutoUpdate = false;
object.matrix = myMatrix;
Now, if you'd like to have a custom transformation matrix applied on top of the Three.js position/scale/rotation stuff, you need to edit Object3D#updateMatrix to be something like.
THREE.Object3D.prototype._updateMatrix = THREE.Object3D.prototype.updateMatrix;
THREE.Object3D.prototype.updateMatrix = function() {
this._updateMatrix();
if (this.customMatrix != null)
this.matrix.multiply(this.customMatrix);
};
See https://github.com/mrdoob/three.js/blob/master/src/core/Object3D.js#L209

Sigh...
Altered Qualia pointed out the solution on Twitter within minutes of me posting this.
It's a simple one-line fix: Just set matrixAutoUpdate to false on the Object3D instances and the first code sample works as intended.
threeObj.matrixAutoUpdate = false; // This fixes it
threeObj.applyMatrix(threeMatrixFromSrcMatrix(child.matrix));
It's always the silly little things that get you...

Related

Printing an image to a dye based application

I am learning about fluid dynamics (and Haxe) and have come across this awesome project and thought I would try to extend to it to help me learn. A demo of the original project in action can be seen here.
So far, I have created a side menu of items containing different shapes. When the user clicks on one of the shapes, then, clicks onto the canvas, the image selected should be imprinted onto the dye. The user will then move the mouse and explore the art etc.
To try and achieve this I did the following:
import js.html.webgl.RenderingContext;
function imageSelection(): Void{
document.querySelector('.myscrollbar1').addEventListener('click', function() {
// twilight image clicked
closeNav();
reset();
var image:js.html.ImageElement = cast document.querySelector('img[src="images/twilight.jpg"]');
gl.current_context.texSubImage2D(cast fluid.dyeRenderTarget.writeToTexture, 0, Math.round(mouse.x), Math.round(mouse.y), RenderingContext.RGB, RenderingContext.UNSIGNED_BYTE, image);
TWILIGHT = true;
});
After this call, inside the update function, I have the following:
override function update( dt:Float ){
time = haxe.Timer.stamp() - initTime;
performanceMonitor.recordFrameTime(dt);
//Smaller number creates a bigger ripple, was 0.016
dt = 0.090;//#!
//Physics
//interaction
updateDyeShader.isMouseDown.set(isMouseDown && lastMousePointKnown);
mouseForceShader.isMouseDown.set(isMouseDown && lastMousePointKnown);
//step physics
fluid.step(dt);
particles.flowVelocityField = fluid.velocityRenderTarget.readFromTexture;
if(renderParticlesEnabled){
particles.step(dt);
}
//Below handles the cycling of colours once the mouse is moved and then the image should be disrupted into the set dye colours.
}
However, although the project builds, I can't seem to get the image imprinted onto the canvas. I have checked the console log and I can see the following error:
WebGL: INVALID_ENUM: texSubImage2D: invalid texture target
Is it safe to assume that my cast for the first param is not allowed?
I have read that the texture target is the first parameter and INVALID_ENUM in particular means that one of the gl.XXX parameters are just flat out wrong for that particular function.
Looking through to the file writeToTexture is declared as so: public var writeToTexture (default, null):GLTexture;. WriteToTexture is a wrapper around a regular webgl handle.
I am using Haxe version 3.2.1 and using Snow to build the project. WriteToTexture is defined inside HaxeToolkit\haxe\lib\gltoolbox\git\gltoolbox\render
writeToTexture in gltoolbox is a GLTexture. With snow and snow_web, this is defined in snow.modules.opengl.GL as:
typedef GLTexture = js.html.webgl.Texture;
So we're simply dealing with a js.html.webgl.Texture here, or WebGLTexture in native JS.
Which means that yes, this is definitely not a valid value for texSubImage2D()'s target, which is specified to take one of the gl.TEXTURE_* constants.
A GLenum specifying the binding point (target) of the active texture.
From this description it's obvious that the parameter isn't actually for the texture itself - it merely gives some info on how the active texture should be used.
The question then becomes how the "active" texture can be set. bindTexture() can be used for this.

Cesium path onto terrain: line connecting 2 points goes under the terrain

I have a path moving over time.
I use Cesium.sampleTerrain to get positions elevation and drape them on the terrain.
The problem is that, even if all points are on the terrain, the line connecting 2 points sometimes goes under the terrain.
How can I do to drape also connecting lines on the terrain?
Here is my code:
var promise = Cesium.sampleTerrain(terrainProvider, 14, positions);
Cesium.when(promise, function(updatedPositions) {
var cartesianPositions = Cesium.Ellipsoid.WGS84.cartographicArrayToCartesianArray(updatedPositions);
var sample = new Cesium.SampledPositionProperty();
sample.setInterpolationOptions({
interpolationDegree : 3,
interpolationAlgorithm : Cesium.HermitePolynomialApproximation
});
$(cartesianPositions).each(function(index, cartPosition) {
var time = Cesium.JulianDate.addSeconds(start, index*10, new Cesium.JulianDate());
sample.addSample(time, cartPosition);
})
var target = viewer.entities.add({
position: sample,
path: {
resolution: 60,
material:Cesium.Color.BLUE,
width: 4,
trailTime: 422*10,
leadTime: 0
}
});
});
So like Matthew says; Cesium doesn't currently support a 'polyline' type entity with draping over terrain.
If you find that the Entity API isn't giving you what you need, it might be worth digging into the lower-level Primitives API to gain finer control - more specifically the GroundPrimitive geometry.
Among others; GroundPrimitives currently support the CorridorGeometry.
I have no experience with temporal data plotting within Cesium, but I would suggest you consider this approach rather than the async promise approach, which (IMO) seems like more of a hack born from the absence of a GroundPrimitive-type solution at the time.
Here's a crude example of a GroundPrimitive in action (note we don't need any z values):
var viewer = new Cesium.Viewer('cesiumContainer');
var corridorInstance = new Cesium.GeometryInstance({
geometry : new Cesium.CorridorGeometry({
vertexFormat : Cesium.VertexFormat.POSITION_ONLY,
positions : Cesium.Cartesian3.fromDegreesArray([
-122.26, 46.15,
-122.12, 46.26,
]),
width : 100
}),
id : 'myCorridor',
attributes : {
color : new Cesium.ColorGeometryInstanceAttribute(0.0, 1.0, 1.0, 0.5)
}
});
var corridorPrimitive = new Cesium.GroundPrimitive({
geometryInstance : corridorInstance
});
viewer.scene.primitives.add(corridorPrimitive);
viewer.camera.setView({
destination: Cesium.Cartesian3.fromDegrees(-122.19, 46.20, 10000.0)
});
Which will give you this:
Cesium does not currently support draping lines on terrain, but it is on our road map and really important to us. This is actually an extremely complicated problem to handle correctly in all cases (and is even more complicated because of the limitations of WebGL). It will require a lot of research and experimentation and there's no hard timeline for when it will be finished. We should have a version of it for static lines by spring as part of our 3D Tiles work, but dynamic lines are probably further out.
If you're interested in following development of this feature, keep your eye on issue #2172 in our GitHub repository. We'll also make announcements on our blog/twitter/forum when this feature is part of an official release.

implementing a boundary listener in box2dweb

I am relatively new to box2d, I'm using the box2dweb version. I want to be able to detect when kinematic bodies have exited the boundaries of my stage. The Box2D.Dynamics.b2BoundaryListener is the obvious answer, but it is apparently not supported with box2dweb build.
Is there an efficient way to create a listener, rather than primitively checking the coordinates of each body during every call to update()? I tried using static bodies as sensors, but they don't detect kinematic bodies.
As far as I know the b2BoundaryListener was part of earlier versions of Box2D and was removed since the Box2D world does not have boundaries anymore.
You could use dynamic sensors since these would also detect kinematic bodies.
If your world uses gravity you would have to fix the bodies though.
You could either do that by applying a force that counteracts the gravity or by fixing these dynamic sensor bodies to static bodies by a joint (e.g. weld joint).
Newer Box2D Javascript ports also include the setGravityScale method that will allow you to create dynamic bodies that are not influenced by gravity (e.g. JSBox2D)
But I think doing this manually in update is probably a better idea and should be pretty straight forward if you use the b2AABB class. This will remove objects when their bounding box is outside of your bounds.
You could do something like the following (untested code):
var body = world.GetBodyList();
while (body != null) {
var fixture = body.GetFixtureList();
var bodyIsInBounds = false;
while (fixture != null) {
// bounds AABB are your boundaries (as b2AABB object)
if (fixture.GetAABB().TestOverlap(boundsAABB)) {
bodyIsInBounds = true;
break;
}
fixture = fixture.GetNext();
}
if (!bodyIsInBounds) {
world.DestroyBody(body);
}
body = body.GetNext();
}
You might also have to check for b2AABB.Contains. I am not sure whether or not TestOverlap will return true if boundsAABB contains the other b2AABB object completely.

Pass null values to SVG path (using d3.js) to suppress missing data

Using jQuery Flot, I can pass a null value to the plotting mechanism so it just won't draw anything on the plot. See how the missing records are suppressed:
I'm looking to move to d3js, so that I can have deeper low level control of the graphics using SVG. However, I have yet to find out how to do that same process of suppressing missing records. The image below is an attempt to do this, using a value of 0 instead of null (where the d3 package breaks down). Here is some code to give you an idea of how I produced the graph below:
var line = d3.svg.line()
.x(function(d) {
var date = new Date(d[0]);
return x(date);
})
.y(function(d) {
var height = d[1];
if (no_record_exists) {
return y(0);
}
return y(height) + 0.5;
});
I looked up the SVG path element at the Mozilla Developer Network, and I found out that there is a MoveTo command, M x y, that only moves the "pen" to some point without drawing anything. Has this been implemented in the d3js package, so that I won't have to create several path elements every time I encounter a missing record?
The defined function of d3.svg.line() is the way to do this
Let's say we want to include a break in the chart if y is null:
line.defined(function(d) { return d.y!=null; })
Use line.defined or area.defined, and see the Area with Missing Data example.

How do I set up DirectX 9 so that backface culling is off, z-buffering is on, and gouraud shading works, for triangle meshes without normals data?

I've been having difficulty identifying the correct parameters for the PresentParameters and DirectX device, so that there can be both vertex-level gouraud shading and the use of a z buffer. Some triangle meshes work fine, others have background triangles appearing in front of triangles which are closer-to-camera.
An example of this is found here: http://gallery.me.com/robert.perkins/100045/zBufferGone. The input data is a simple list of vertices in facets. The winding order of the vertices in each facet is nondeterministic (comes from various CAD software export functions) and there is no normals data.
The PresentParameters are being set up right now as follows. I realize this is C# instead of C++ but I think it's descriptive enough, and the parameters pass through to C++ code. This produces the image in the picture; the behavior is the same on the Reference device:
pParams = new PresentParameters()
{
BackBufferWidth = this.ClientSize.Width,
BackBufferHeight = this.ClientSize.Height,
AutoDepthStencilFormat = Format.D16,
EnableAutoDepthStencil = true,
SwapEffect = SwapEffect.Discard,
Windowed = true
};
_engineDX9 = new EngineDX9(this, SlimDX.Direct3D9.DeviceType.Hardware, SlimDX.Direct3D9.CreateFlags.SoftwareVertexProcessing, pParams);
_engineDX9.DefaultCamera.NearPlane = 0;
_engineDX9.DefaultCamera.FarPlane = 10;
_engineDX9.D3DDevice.SetRenderState(RenderState.Ambient, false);
_engineDX9.D3DDevice.SetRenderState(RenderState.ZEnable, ZBufferType.UseZBuffer);
_engineDX9.D3DDevice.SetRenderState(RenderState.ZWriteEnable, true);
_engineDX9.D3DDevice.SetRenderState(RenderState.ZFunc, Compare.Always);
_engineDX9.BackColor = Color.White;
_engineDX9.FillMode = FillMode.Solid;
_engineDX9.CullMode = Cull.None;
_engineDX9.DefaultCamera.AspectRatio = (float)this.Width / this.Height;
All of my other setup attempts, even on the reference device, return a COM error code ({"D3DERR_INVALIDCALL: Invalid call (-2005530516)"}). What are the correct setup parameters?
EDIT: The C++ class which interfaces with DirectX9 sets defaults like this:
PresentParameters::PresentParameters()
{
BackBufferWidth = 640;
BackBufferHeight = 480;
BackBufferFormat = Format::X8R8G8B8;
BackBufferCount = 1;
Multisample = MultisampleType::None;
MultisampleQuality = 0;
SwapEffect = SlimDX::Direct3D9::SwapEffect::Discard;
DeviceWindowHandle = IntPtr::Zero;
Windowed = true;
EnableAutoDepthStencil = true;
AutoDepthStencilFormat = Format::D24X8;
PresentFlags = SlimDX::Direct3D9::PresentFlags::None;
FullScreenRefreshRateInHertz = 0;
PresentationInterval = PresentInterval::Immediate;
}
Where does it return an invalid call?
Edit: I'm assuming in the new EngineDX9 call? Have you tried setting a device window handle in the present parameters?
Edit 2: Have you turned on the debug spew in the DirectX control panel to see whether it tells you what the error is?
Edit3: You have tried setting backbufferWidth and Height to 0? What is backbuffercount set to? Might also be worth trying "Format.D24S8" on the backbuffer? Its "possible" your graphics card doesn't support 16-bit (unlikely though). Have you checked in the caps that the mode you are trying to create is valid? I asssume, btw, that the CLR language you are using automagically sets the parameters you don't set to 0? I,personally, always prefer to be explicit in such cases ....
PS I'm guessing here because im a native C++ DX9 coder not a CLR SlimDX coder ...
Edit4: I'm sure its the lack of window handle ... I'm probably wrong but thats the only thing i can see REALLY wrong with your setup. A windowed DX9 device requires a window. Btw set width and height to 0 to just use the window you are setting the device too's size ...
Edit 5: I've really been heading down the wrong route here. There is nothing wrong with the creation of the device that produced your "incorrect" device. Do not mess with the present parameters they are fine. The main reason you'll have problems with your Z-Buffering is that you set the compare function to always. This means that, regardless of what the z-buffer contains, pas the pixel and write its z into the z-buffer overwriting whatever is there already. I'd wager therein lies your Z-buffering problem.

Resources