Fluffy predator with THREE.js & instanced geometry

For a recent project I had to manipulate a potentially large number of meshes. I chose to use THREE.InstancedBufferGeometry as it is very efficient ; it allows to draw the same mesh many times, with different attributes, in a single drawcall.

Working with instances is slightly different from using regular meshes. In a nutshell, you give the blueprint of the mesh you want to draw, adding attributes as you would with a THREE.Buffergeometry and set some special THREE.InstancedBufferAttribute that will affect each instance of the blueprint.

If you understand the concept of blueprint (the actual geometry), instance (the placeholder for a blueprint) and InstancedBufferAttributes (the variables applied to each instance of the blueprint), you’re good to go.

The big difference with regular meshes is that instead of setting the meshes’ translation/rotation/scale (TRS for short) directly on the mesh object, you have to either, manipulate all the instances’ BufferAttributes on the CPU then re-upload them to the GPU or perform procedural transforms directly on the vertex shader.

Anyhow, a vertex shader is involved to transform the blueprint. This may seem scary but it’s not, especially as brilliant people already did most of it :)

There is a limitation when using Instances ; in WebGL, you can’t pass more than 4 floats per attribute. This may not sound like much but it has a big impact on how you manipulate the meshes. You can’t pass a Matrix4 (4*4 floats) to transform an instance. This was a surprise for me, a Matrix4 is the way meshes’ store their transformations internally, it would be convenient to pass the elements of a matrix and use this matrix InstancedAttribute it to transform each instance of the blueprint but it is not technically feasible.

When life gives you lemons…

Therefore, the easiest way to transform an instance is to use 3 InstancedBufferAttributes to represent the TRS :

  • translation: a Vector3, equivalent to the mesh.position
  • rotation: a Vector4, equivalent to the mesh.quaternion (not the mesh.rotation!)
  • scale: a Vector3, equivalent to the mesh.scale

here’s a method that instantiates count isosceles triangles and randomizes them:

For the vertex shader, you can do something like this (picked from the THREE examples):

and the fragment uses the world position as the fragment color:

With the code above, the instances should be properly positioned, rotated and scaled and the mesh should look something like this:

so with this, you can already do a lot of funny stuff, anything that needs many objects to move individually. My project was to animate a wall of panels, much like how Portal levels build up. I ended up with this mesmerizing *ahem* demo:

Then I tried to assign uvs as InstancedBufferAttributes (related to each instance) rather than BufferAttributes (related to the blueprint).

Each instance now has it’s own uvs instead of using the blueprint‘s uvs.
You may wonder how the instances are animated, that’s quite simple ; you need two sets of positions (two meshes), then create a target attribute on the blueprint geometry (as it will be the same target for all the instances) and in the vertex shader, instead of writing:

you’d write :

where ratio is a value between 0 and 1. When the ratio is 0, the instance is in the original position, when the ratio is 1, it is in the target position, if ratio is between 0 and 1, the values are interpolated between the 2.

The ratio can be passed as an InstancedBufferAttribute so that each instance opens gradually and/or with a delay and of course, the ratio can be a noise function to get fancy animations.

It is also trivial to use a distribution object – the translation InstanceBufferAttributes are the vertices of the distribution mesh, the rotation can be computed with a lookAt() – which gives this kind of things:

Btw the background music was recorded randomly, it is “I bet you look good on the dancefloor” by the arctic monkeys :)

The next natural step for me was to try to cover a mesh with instances of triangles so that any mesh with a triangular basis would fit nicely and wrap the distribution mesh.

The source triangle should be scaled, skewed, rotated and translated so that a single matrix can transform it into any triangle in space. Little did I know that aligning a triangle to another triangle using a transform matrix requires a special type of transform called an affine transform (I couldn’t explain the difference between homothetic and affine really).

When life gives you affine transforms, run.

After creating a source unit triangle (like the one used in the basic demo), my algorithm went as follows for each face of the mesh:

  1. find its center and compute its normal
  2. transform an Object3D so that it is positioned and oriented like the face
  3. use the inverse of the object’s matrix to transform the triangle’s vertices so that they lie on the ‘XY’ plane
  4. multiply the object’s transform matrix with the 2D affine transform (the one that aligns 2 triangles)
  5. use this matrix to transform the instance

steps 1 to 3 are really straight forward ; THREE has everything you need to do this, especially as, when using an old school THREE.Geometry, you can access the object’s faces’ list where every Face object holds the face normal. step 4 implied finding the affine transform… After 2 days of research (and headaches), I found this great JavaScript example and after some refactoring, I ended up with this glorious piece of code:

and to use it:

nowcontains transform from our source triangle to the current face.

I thought it would work but nope! In fact, I obtained a completely valid Matrix4 but, as we need to decompose the Matrix4 into the T,R & S components to pass them as attributes to the GPU, I suspect some data, required for the affine transform, went missing. In other words, it got schwifty and the triangles wouldn’t fit.

After some further research, I found this StackOverflow thread explaining how to pass a Matrix4 as 4 vec4, then recompose it on the vertex shader. I followed the example and got it to work.

And there you have it: a fluffy Predator!

the HI-RES mesh is fairly big (250K faces, 8Mo) so it may turn your computer into a toaster.
UPDATE: I’ve added a LO-RES setting (65k faces, 2Mo ) that should work better on regular computers, click to load:

On  a side note, if your mesh disappears when you zoom in, it probably gets frustrum culled, try to disable culling on the Mesh:

another reason why it would disappear is if the geometry’s bounding sphere is too small, you can (hackily) fix this by doing so:

note that this will force the render of your mesh, use with caution :)

I can think of many silly use cases for this, instanced geometry (finally) allows control over hundreds of thousands of meshes, not particles, actual meshes! I put the files on GIT repo: https://github.com/nicoptere/FluffyPredator/

enjoy :)

15 Comments

  1. tlecoz

    Very interesting !
    I ‘m in love with your glsl-function called “transform”

    Until now, I used to do it like that (the rotation part) :

    void rotateXYZAroundCenter(inout vec3 pos,in vec3 rotations){

    vec3 s = sin(rotations);
    vec3 c = cos(rotations);

    // rotation around x
    float xy = c.x * pos.y – s.x * pos.z;
    float xz = s.x * pos.y + c.x * pos.z;
    // rotation around y
    float yz = c.y * xz – s.y * pos.x;
    float yx = s.y * xz + c.y * pos.x;
    // rotation around z
    float zx = c.z * yx – s.z * xy;
    float zy = s.z * yx + c.z * xy;

    pos.x = zx ;
    pos.y = zy ;
    pos.z = yz ;
    }

    But your function is probably much more optimised.

    Thanks !

    • nico

      instanciation is quite powerful :)

      I don’t know if this TRS transform is very different from your function (apart from the fact that it adds Translation and Scale :)) or if it is more efficient (it depends on whether or not the cross product is hardware accelerated) but it’s simpler to read and write.

      the thing is that it needs a Quaternion instead of Euler rotations (XYZ) and that’s a touch more complex (for people like me at least ^^)

  2. tlecoz

    Hello
    Every glsl-functions is optimized compared to those you write by yourself, thats why I said yours is probably more efficient.
    I didnt had the time to go more into it but I will

    Thank you again

  3. Douglas

    Hey!

    I posted the same question on one of your youtube videos, but what method did you use to assign UV’s for each instanced mesh? :o

    Best

    /Douglas

    • nico

      hey,
      sorry I’m not much of a youtube person… :/
      so for the uvs, you need to pass a scale and offset attribute to each instance ; the scale will ‘crop’ a portion of the texture – say you have a 4 * 3 grid, your scale will be 1/4 on the X axis and 1/3 on the Y axis – and the offset will tell which cell of the ‘grid’ you want to display – again with a 4 * 3 grid, to get the bottom right corner, the offset should be 3 * 1/4 = .75 / 2 * 1/3 = .66.
      then in the vertex shader, you compute the uvs like so: vUv = uv * scale + offset; and you’ll probably need to flip the Y axis so adding a vUv.y = 1. - vUv.y; could help.
      I haven’t tried the code but I hope it helps :)

  4. Douglas

    Heya!

    I might have missed something, could you ellaborate on the formula you mentioned?

    and also, should I pass the blueprint geometry’s UV’s, or should I, as you mentioned in the article, create an instanced bufferattribute and assign UV coordinates by hand?

    If so, I tried doing that and it worked when I assigned the array as a buffer attribute, but when I used the same array as an instanced bufferattribute, my meshes are getting different colors that are similar to the image that I use as a texture!

  5. Excellent article! I’m feeling rather stupid as I’m new to coding and three js, but do I understand correctly that this is the only way to place instances at the center of faces? I’ve worked out how to place instances on a surface but I’d like to place them at the center of each face. Currently, the techniques I’m using place an instance randomly meaning some faces get several, some get none. The goal is effectively a cube with one instance at the center of each of the 6 faces. Do I need to implement the code described in this article or is there an easier way? Thanks!

    • nico

      hey, sorry for the very late reply.
      if you want to place items on each face’s center, the easiest is to compute the center of each face on the CPU ( it’s the sum of the face’s 3 vertices XYZ divided by 3).
      then use this to place the instances.

  6. Nameless

    Hey Nico!

    I just have to say that you are an absolute CODE WIZARD!!!

    This entire article is just magic spell after magic spell holy fricking moly, feels like pure liquid golden knowledge got poured into my brain.

    Maybe I’m overreacting a bit but my god this is some good sauce right here, I thought I had something by pulling off FBO particles for a project but this is on a whole other level.

    Thanks so much for writing it up AND providing us with the code.

    I hope to be like you when I grow up,
    Cheers

    • nico

      hey, thanks a lot for your enthusiasm :)
      I’m happy that this articles could help you and I’m sure you’ll do great when you grow up.
      many thanks, you made my day!

Leave a Reply

Your email address will not be published. Required fields are marked *