Picking, Mouse Projections, Screen to Ray

 The Problem:

When the user clicks on the 2D screen, how do we know in 3D coordinates where they actually clicked, so we can select the object they clicked on?

The Solution:

Obviously a simple geometric / linear algebra problem. It is commonly done by querying the graphics card for the projection, modelview, and viewport. I despise asking the card for anything, as it should be a one way pipe. (minus CUDA, which is still new).

Our special case is, if we know how we constructed the projection matrix, we can easily invert the process to convert a screen ray into world coordinates. Then we can make those world coordinates relative to our "camera" matrix, and you can then perform a ray to scene collision detection;

This is also called "picking", "selection", and has a variety of other names. I highly reccomend never using any method but your own geometric methods, since you can't rely on this type of functionality to be consistent, without ensuring your own code is.

In layman terms, "You can click things in 3D" / "What you see is what you get"

The orange dot is the actual 3D position of the Ray -> AABB intersection
(A Ray is defined by a point and a direction, a AABB is a axis aligned bounding box, so it has a minimum position and a maximum position, or a center and a half-size for each axis)

Here's some code snippet:

class matrix
{
    public:

    //+X = left, +Y = up, +Z = forward
    float Xx, Xy, Xz, Xw;    //Order from spec; Colum major 0..16. Have to transpose all the mathematics (internally)
    float Yx, Yy, Yz, Yw;
    float Zx, Zy, Zz, Zw;
    float x, y, z, w;

    ...


void mingl::matrix::frustum_get_ray( int umouse_x, int umouse_y, int window_w, int window_h, float distnear, float distfar, float fovangle, float & Rx, float & Ry, float & Rz, float & Rdx, float & Rdy, float & Rdz )
{

    float scrn_x =  (float)(2*umouse_x - window_w) / ((float)window_w);
    float scrn_y =  -(float)(2*umouse_y - window_h) / ((float)window_h);

    float td_dx, td_dy;

    float td_tan = tan( fovangle * (M_PI/360.0f) );

    if( window_w > window_h ){

        td_dx = td_tan * scrn_x * ((float)window_w/(float)window_h);
        td_dy = td_tan * scrn_y;
    }else{

        td_dx = td_tan * scrn_x;
        td_dy = td_tan * scrn_y * ((float)window_h/(float)window_w);
    }

    //Generate points on the projection viewport
    float p1[3] = { td_dx*distnear, td_dy*distnear, distnear };
    float p2[3] = { td_dx*distfar, td_dy*distfar, distfar };
    float dv[3] = { (p2[0] - p1[0]), (p2[1] - p1[1]), (p2[2] - p1[2]) };

    //Note this is a specialized "To Global" operation (because of the inversion on axes)
    //Ray does not start from center of camera.
    Rx = x - Zx * p1[2] + Xx * p1[0] + Yx * p1[1];
    Ry = y - Zy * p1[2] + Xy * p1[0] + Yy * p1[1];
    Rz = z - Zz * p1[2] + Xz * p1[0] + Yz * p1[1];

    //note the conversion.
    Rdx = -Zx * dv[2] + Xx * dv[0] + Yx * dv[1];
    Rdy = -Zy * dv[2] + Xy * dv[0] + Yy * dv[1];
    Rdz = -Zz * dv[2] + Xz * dv[0] + Yz * dv[1];
}

 From reading that code, you'll notice the projection matrix is defined by:

    //3D projection matrix via symmetrical frustum; the most common projection.
    float fov = 60.0;
    float aspect = 1.0;
    float unear = 0.125;
    float ufar = 1024.0;

    float top = tan(fov*0.00872664625997f) * unear;    //0.00872665f = pi / 360 = (pi / 180) * 0.5
    if( (window_h > 0) && (window_w > 0) ){

        if( window_w > window_h ){

            aspect = float(window_w)/float(window_h);
            projection.frustum( aspect * -top, aspect * top, -top, top, unear, ufar );
        }else{

            aspect = float(window_h)/float(window_w);
            projection.frustum( -top, top, aspect * -top,aspect * top, unear, ufar );
        }
    }


void mingl::matrix::frustum( float ul, float ur, float ub, float ut, float un, float uf )
{
    float dx = (ur-ul);
    float dy = (ut-ub);
    float dz = (uf-un);
    dx = (dx <= 0) ? 1.0 : dx;
    dy = (dy <= 0) ? 1.0 : dy;
    dz = (dz <= 0) ? 1.0 : dz;

    identity();

    Xx = (2.0*un)/dx;
    Yy = (2.0*un)/dy;
    Xz = (ur + ul)/dx;
    Yz = (ut + ub)/dy;
    Zz = -(uf + un)/dz;
    Zw = -1.0;
    z = -(2.0*uf*un)/dz;
    w = 0;

Works beautifully! Onward with my megaman project.

-Z