[three.js] OBJECT PICKING : select object by mouse

In this tutorial we’ll learn what is object picking, how projection and unprojection work, and how to use those with three.js to build this scene where gray cubes turn reddish when the mouse hovers them–the closer to the camera they are, the redder they turn!

Get the source code for this tutorial if you want to dive straight into the code!

Here are a couple more examples:

• interactive cubes, where you can click on cubes and a black spot will appear in the exact point where you clicked, and…
• interactive voxel painter, which allows you to add voxels and remove them too

Did you play with them? Notice a common pattern? We are detecting objects in 3D space using our 2D coordinate world (the screen). That’s what object picking is about!

How does it work?

Before we write a single line of code, it’ll be really helpful to understand how computer 3D graphics work, even if in a very rough manner. How do we go from an abstract, perfect 3D scene to a 2D image in our screens?

When you use a camera to render a scene, a whole bunch of math machinery starts munching and processing your 3D scene, in order to produce a 2D representation of the fragment of your scene that can be seen from the camera. There are many steps involved but the one that interests us is the projection, which is what turns abstract 3D stuff into somewhat convincing 2D entities in your screen.

And that is great, but… how do we go backwards?

Why would we want to go backwards?, you might be asking. Well, if you want to find out which object is underneath your mouse pointer, you need to bring those 2D coordinates back into the 3D world before you try to find out if there’s something there. That is unprojecting!

경축! 아무것도 안하여 에스천사게임즈가 새로운 모습으로 재오픈 하였습니다.
어린이용이며, 설치가 필요없는 브라우저 게임입니다.
https://s1004games.com

See? Now everything fits:

• Going from 3D to 2D? PROJECTION!
• Going from 2D to 3D? UNPROJECTION!

There’s still a missing piece: how do we find the intersections once we’re back in 3D land? It always sounds funny to me, but here’s the answer: we cast rays! Yes, I’m being serious! We cast a ray from the position of our mouse in 3D space, following the current direction of the camera, and see if we hit any object(s) on the way. If we do, then we’ve found what we were looking for. If we don’t, then there’s nothing underneath the mouse.

I know it can be confusing, so please look at this drawing:

There’s an abstract 3D scene on the left, with two cubes and a pyramid. Then there’s a representation of our screen –i.e. how we see the 3D scene–. You can also see the mouse cursor, over the cubes. On the right, the camera from where we’re looking, and in blue, the ray we’re using to pick objects.

And now that you know how object picking works in theory, let’s see how we do that with three.js!

Object picking in action

Thankfully, this is very easy to implement with three.js. We’ll start with the basics: setting up the renderer, a scene and a camera.

var container = document.getElementById( 'container' ),
    containerWidth, containerHeight,
    renderer,
    scene,
    camera;

containerWidth = container.clientWidth;
containerHeight = container.clientHeight;

renderer = new THREE.CanvasRenderer();
renderer.setSize( containerWidth, containerHeight );
container.appendChild( renderer.domElement );

renderer.setClearColorHex( 0xeeeedd, 1.0 );

scene = new THREE.Scene();

camera = new THREE.PerspectiveCamera( 45, containerWidth / containerHeight, 1, 10000 );
camera.position.set( 0, 0, range * 2 );
camera.lookAt( new THREE.Vector3( 0, 0, 0 ) );

Nothing specially surprising!

Now let’s add some objects. We’ll create the gray cubes, and place them randomly in the scene, but they will be children of a cubes object. NOTE: if you want to learn more about adding objects to other objects, readthis tutorial