Shader code optimization - directx

Question

I have this code snippet (for cubemap PCF filtering). I would like to optimize it for shader model 2. I tried eliminating the branches with permutation matrices stored in uniforms, but it requires too much (2x24).
float3 l = normalize(ldir);
float3 al = abs(l);
float3 off2, off3, off4;
if( al.x < al.y )
{
if( al.y < al.z )
{
// z is dominant
off2 = CubeOffset(l.zxy, float2(0, 1), texelsize).yzx;
off3 = CubeOffset(l.zxy, float2(1, 0), texelsize).yzx;
off4 = CubeOffset(l.zxy, float2(1, 1), texelsize).yzx;
}
else
{
// y is dominant
off2 = CubeOffset(l.yxz, float2(0, 1), texelsize).yxz;
off3 = CubeOffset(l.yxz, float2(1, 0), texelsize).yxz;
off4 = CubeOffset(l.yxz, float2(1, 1), texelsize).yxz;
}
}
else
{
if( al.x < al.z )
{
// z is dominant
off2 = CubeOffset(l.zxy, float2(0, 1), texelsize).yzx;
off3 = CubeOffset(l.zxy, float2(1, 0), texelsize).yzx;
off4 = CubeOffset(l.zxy, float2(1, 1), texelsize).yzx;
}
else
{
// x is dominant
off2 = CubeOffset(l, float2(0, 1), texelsize);
off3 = CubeOffset(l, float2(1, 0), texelsize);
off4 = CubeOffset(l, float2(1, 1), texelsize);
}
}
Perhaps a mathematical relation can be found between the comparisons (al.xyy < al.yzz) and the swizzles.
UPDATE: definition of cubeoffset
float3 CubeOffset(float3 swiz, float2 off, float2 texelsize)
{
float3 ret;
ret.yz = swiz.yz + 2.0f * off * texelsize;
ret.x = sqrt(1.0f - dot(ret.yz, ret.yz));
if( swiz.x < 0 )
ret.x *= -1.0f;
return ret;
}
And the HLSL error when compiling SM 2.0:
error X5608: Compiled shader code uses too many arithmetic instruction slots (107).
Max. allowed by the target (ps_2_0) is 64.
error X5609: Compiled shader code uses too many instruction slots (111).
Max. allowed by the target (ps_2_0) is 96.
GLSL handles it fine. The goal is backward compatibility.
(btw. the algorithm is faulty, but that's not an issue right now)

Answer 1

Not really an optimization, but consider to test this.
Obvious, not always desired, and rarely best solution in such cases is to move to CPU extra code that not fits up (due to instructions count for example). In case of branching you can:
move branch condition check to CPU
split and move to separate shaders branch bodies
bind appropriate shader by results of condition check
It is a simplest thing you can do. And no need to poking around in assembler. Problem is when your condition is calculated inside shader.
Hope it helps somehow.

Answer 2

While I don't know whether it can be solved with SM 2.0, considering the advancements in GPU power I provide a SM 3.0 solution.
Please keep in mind, that this code is a snippet from my own shader language (but similar to HLSL):
template <int samples>
float PCFIrregularCUBE(sampler shadowmap, sampler noisetex, float3 ldir, float2 sloc, float2 texelsize)
{
const float kernelradius = 2.0f;
float3 l = normalize(ldir);
float3 al = abs(l);
float2 noise;
float2 rotated;
float sd, t, s;
float d = length(ldir);
noise = tex2D(noisetex, sloc);
noise = normalize(noise * 2.0f - 1.0f);
float2 rotmat0 = float2(noise.x, noise.y);
float2 rotmat1 = float2(-noise.y, noise.x);
float3 off;
s = 0;
for( int i = 0; i < samples; ++i ) {
rotated.x = dot(irreg_kernel[i], rotmat0) * kernelradius;
rotated.y = dot(irreg_kernel[i], rotmat1) * kernelradius;
if( al.x < al.y ) {
if( al.y < al.z )
off = CubeOffsetZXY(l, rotated, texelsize);
else
off = CubeOffsetYXZ(l, rotated, texelsize);
} else {
if( al.x < al.z )
off = CubeOffsetZXY(l, rotated, texelsize);
else
off = CubeOffsetXYZ(l, rotated, texelsize);
}
sd = texCUBE(shadowmap, off).r;
t = ((d > sd) ? 0.0f : 1.0f);
s += ((sd < 0.001f) ? 1.0f : t);
}
return s * (1.0f / samples);
}
With CubeOffsetXXX being like:
float3 CubeOffsetZXY(float3 swiz, float2 off, float2 texelsize)
{
float3 ret;
ret.xy = swiz.xy + 2.0f * off * texelsize * swiz.z;
ret.z = sqrt(1.0f - dot(ret.xy, ret.xy));
if( swiz.z < 0 )
ret.z *= -1.0f;
return ret;
}
For more details you should google for irregular PCF. The result in its worst (as in "the camera is close") is:
Notice the "salt and pepper" noise caused by irregular PCF. From a distance it is perfectly acceptable (Crysis 1 method).