Warning: Declaration of Jetpack_IXR_Client::query() should be compatible with IXR_Client::query(...$args) in /home/clients/7267bc096562fcdb78c0ab60d3ac51fb/web/blog/wp-content/plugins/jetpack/class.jetpack-ixr-client.php on line 91
{"id":621,"date":"2016-01-11T11:41:28","date_gmt":"2016-01-11T11:41:28","guid":{"rendered":"http:\/\/barradeau.com\/blog\/?p=621"},"modified":"2021-05-25T15:55:08","modified_gmt":"2021-05-25T15:55:08","slug":"fbo","status":"publish","type":"post","link":"https:\/\/barradeau.com\/blog\/?p=621","title":{"rendered":"FBO particles"},"content":{"rendered":"

update 210525:<\/strong>
\nMario Carrillo<\/a>\u00a0was kind enough to port the code samples to ES6, something I’ve been willing to do for years.
\nso check out his repo:\u00a0 https:\/\/github.com\/marioecg\/gpu-party\/<\/a>\u00a0( and check out his work while you’re at it )<\/p>\n

I can’t tell how many particle engines I’ve written for the past 15 years but I’d say a lot.\u00a0one reason is that it’s easy to implement and quickly gives good looking \/ complex results.<\/p>\n

in august 2014, I started a year-long project that never shipped (which I playfully codenamed “the silent failure”), the first thing they asked for was a particle engine to emulate a shitload of particles.<\/p>\n

the way to go in this case is to use a GPGPU approach a.k.a. FBO particles.\u00a0it is a fairly well documented technique,\u00a0there were\u00a0working examples of FBO particles running in THREE.js<\/a> and especially this one<\/a> by Joshua Koo<\/a>\u00a0& Ricardo Cabello<\/a><\/p>\n

in a nutshell, 2 passes are required:<\/p>\n

\n
simulation: uses a Data\u00a0Texture as an input, updates the particles’ positions and writes them back to a RenderTarget<\/li>\n
render:\u00a0uses the RenderTarget to distribute\u00a0the particles in space and renders the particles to screen<\/li>\n<\/ul>\n
the first pass requires a bi-unit square, an orthographic camera and the ability to render to a texture.\u00a0the second pass is your regular particles rendering routine with a twist on how to retrieve the particle’s position.<\/p>\n
the FBO class goes like:<\/p>\n
var FBO = function( exports ){\r\n\r\n var scene, orthoCamera, rtt;\r\n exports.init = function( width, height, renderer, simulationMaterial, renderMaterial ){\r\n\r\n gl = renderer.getContext();\r\n\r\n \/\/1 we need FLOAT Textures to store positions\r\n \/\/https:\/\/github.com\/KhronosGroup\/WebGL\/blob\/master\/sdk\/tests\/conformance\/extensions\/oes-texture-float.html\r\n if (!gl.getExtension(\"OES_texture_float\")){\r\n throw new Error( \"float textures not supported\" );\r\n }\r\n\r\n \/\/2 we need to access textures from within the vertex shader\r\n \/\/https:\/\/github.com\/KhronosGroup\/WebGL\/blob\/90ceaac0c4546b1aad634a6a5c4d2dfae9f4d124\/conformance-suites\/1.0.0\/extra\/webgl-info.html\r\n if( gl.getParameter(gl.MAX_VERTEX_TEXTURE_IMAGE_UNITS) == 0 ) {\r\n throw new Error( \"vertex shader cannot read textures\" );\r\n }\r\n\r\n \/\/3 rtt setup\r\n scene = new THREE.Scene();\r\n orthoCamera = new THREE.OrthographicCamera(-1,1,1,-1,1\/Math.pow( 2, 53 ),1);\r\n\r\n \/\/4 create a target texture\r\n var options = {\r\n minFilter: THREE.NearestFilter,\/\/important as we want to sample square pixels\r\n magFilter: THREE.NearestFilter,\/\/\r\n format: THREE.RGBFormat,\/\/could be RGBAFormat \r\n type:THREE.FloatType\/\/important as we need precise coordinates (not ints)\r\n };\r\n rtt = new THREE.WebGLRenderTarget( width,height, options);\r\n\r\n\r\n \/\/5 the simulation:\r\n \/\/create a bi-unit quadrilateral and uses the simulation material to update the Float Texture\r\n var geom = new THREE.BufferGeometry();\r\n geom.addAttribute( 'position', new THREE.BufferAttribute( new Float32Array([ -1,-1,0, 1,-1,0, 1,1,0, -1,-1, 0, 1, 1, 0, -1,1,0 ]), 3 ) );\r\n geom.addAttribute( 'uv', new THREE.BufferAttribute( new Float32Array([ 0,1, 1,1, 1,0, 0,1, 1,0, 0,0 ]), 2 ) );\r\n scene.add( new THREE.Mesh( geom, simulationMaterial ) );\r\n\r\n\r\n \/\/6 the particles:\r\n \/\/create a vertex buffer of size width * height with normalized coordinates\r\n var l = (width * height );\r\n var vertices = new Float32Array( l * 3 );\r\n for ( var i = 0; i < l; i++ ) {\r\n\r\n var i3 = i * 3;\r\n vertices[ i3 ] = ( i % width ) \/ width ;\r\n vertices[ i3 + 1 ] = ( i \/ width ) \/ height;\r\n }\r\n\r\n \/\/create the particles geometry\r\n var geometry = new THREE.BufferGeometry();\r\n geometry.addAttribute( 'position', new THREE.BufferAttribute( vertices, 3 ) );\r\n\r\n \/\/the rendermaterial is used to render the particles\r\n exports.particles = new THREE.Points( geometry, renderMaterial );\r\n exports.renderer = renderer;\r\n \r\n };\r\n \r\n \/\/7 update loop\r\n exports.update = function(){\r\n\r\n \/\/1 update the simulation and render the result in a target texture\r\n exports.renderer.render( scene, orthoCamera, rtt, true );\r\n\r\n \/\/2 use the result of the swap as the new position for the particles' renderer\r\n exports.particles.material.uniforms.positions.value = rtt;\r\n\r\n };\r\n return exports;\r\n}({});<\/pre>\n
I left the comments so it should be easy to understand. step by step, it\u00a0unrolls as follow:<\/p>\n
\n
we need to determine if the hardware is capable of rendering the shaders. for the simulation pass, we’ll need to use float textures, if the hardware doesn’t support them, throw an error.<\/li>\n
for the render pass, we’ll have to access the textures in the vertex shader which isn’t always supported\u00a0by the hardware, if unsupported, bail out & throw error.<\/li>\n
create a scene and a\u00a0bi-unit\u00a0orthographic camera (bi-unit = left:-1,right:1, top:1, bottom:-1)\u00a0near and far are not relevant as there is no depth so to speak in the simulation.<\/li>\n
create the\u00a0RenderTarget\u00a0that will allow the data transfer between the simulation and the render shaders. as this is not a “regular” texture, it’s important to set the filtering to NearestFilter (crispy pixels). also the format can be either RGB (to store the XYZ coordinates) or RGBA if you need to store an extra\u00a0value for each\u00a0particle.<\/li>\n
straight forward: we create a bi-unit square geometry & mesh and associate the simulation\u00a0shader to it, it will be rendered with the orthographic camera.<\/li>\n
we create the render mesh, this time we need as many vertices as the pixel count in the float texture: width * height &\u00a0to make things easy, we normalize the vertices’ coordinates.\u00a0then we initialize a Points object( a.k.a Particles,\u00a0a.k.a PointCloud depending on which version of THREE.js you use)<\/li>\n
initialization is over now the update loop does 2 things:
\n1 render the simulation into the renderTarget
\n2 pass the result to the renderMaterial (assigned to the partciles object)<\/li>\n<\/ol>\n
that’s all good and sound, now a basic\u00a0instantiation would look like this (I’ll skip the scene setup you can find it here<\/a>):<\/p>\n
\/\/initializes the FBO particles object\r\nfunction init()\r\n{\r\n \/\/width \/ height of the FBO\r\n var width = 256;\r\n var height = 256;\r\n\r\n \/\/populate a Float32Array of random positions\r\n var data = getRandomData( width, height, 256 );\r\n \/\/convertes it to a FloatTexture\r\n var positions = new THREE.DataTexture( data, width, height, THREE.RGBFormat, THREE.FloatType );\r\n positions.needsUpdate = true;\r\n\r\n \/\/simulation shader used to update the particles' positions\r\n var simulationShader = new THREE.ShaderMaterial({\r\n uniforms:{ positions: { type: \"t\", value: positions } },\r\n vertexShader: \"simulation_vs\",\r\n fragmentShader: \"simulation_fs\"\r\n });\r\n\r\n \/\/render shader to display the particles on screen\r\n \/\/the 'positions' uniform will be set after the FBO.update() call\r\n var renderShader = new THREE.ShaderMaterial( {\r\n uniforms: {\r\n positions: { type: \"t\", value: null },\r\n pointSize: { type: \"f\", value: 2 }\r\n },\r\n vertexShader: \"render_vs\",\r\n fragmentShader: \"render_fs\"\r\n } );\r\n\r\n \/\/init the FBO\r\n FBO.init( width,height, renderer, simulationShader, renderShader );\r\n scene.add( FBO.particles );\r\n update();\r\n}\r\n[...]\r\nfunction update()\r\n{\r\n requestAnimationFrame(update);\r\n FBO.update();\r\n renderer.render( scene, camera );\r\n}<\/pre>\n
now, you may be wondering what do the shaders look like, here they are:<\/p>\n
\/\/-------------- SIMULATION -----------------\/\/\r\n\r\n\/\/vertex shader\r\nvarying vec2 vUv;\r\nvoid main() {\r\n vUv = vec2(uv.x, uv.y);\r\n gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );\r\n}\r\n\r\n\/\/fragment Shader\r\nuniform sampler2D positions;\/\/DATA Texture containing original positions\r\nvarying vec2 vUv;\r\nvoid main() {\r\n\r\n \/\/basic simulation: displays the particles in place.\r\n vec3 pos = texture2D( positions, vUv ).rgb;\r\n \/*\r\n we can move the particle here \r\n *\/\r\n gl_FragColor = vec4( pos,1.0 );\r\n}\r\n\r\n\/\/-------------- RENDER -----------------\/\/\r\n\r\n\/\/vertex shader\r\nuniform sampler2D positions;\/\/RenderTarget containing the transformed positions\r\nuniform float pointSize;\/\/size\r\nvoid main() {\r\n\r\n \/\/the mesh is a nomrliazed square so the uvs = the xy positions of the vertices\r\n vec3 pos = texture2D( positions, position.xy ).xyz;\r\n \/\/pos now contains a 3D position in space, we can use it as a regular vertex\r\n\r\n \/\/regular projection of our position\r\n gl_Position = projectionMatrix * modelViewMatrix * vec4( pos, 1.0 );\r\n\r\n \/\/sets the point size\r\n gl_PointSize = pointSize;\r\n}\r\n\r\n\/\/fragment shader\r\nvoid main()\r\n{\r\n gl_FragColor = vec4( vec3( 1. ), .25 );\r\n}<\/pre>\n
ok this was a long explanation, time to\u00a0do something with it,\u00a0the above will probably look somehow like this (click for live version):<\/p>\n
$\"basic\"$ <\/a><\/p>\n
which is a bit dry\u00a0I’ll admit, but at least it works :)<\/p>\n
the benefit of this system is its ability to\u00a0support lots of particles, I mean lots of them<\/strong>, while preserving a rather light memory footprint. the above uses\u00a0a 256^2 texture or\u00a065536 particles, 512^2 = 262144, 1024^2 = 1048576 etc…. and as many vertices which is often more than what is needed to display …well anything (imagine a mesh with 1+ Million vertices).<\/p>\n
on the other \u00a0hand particles often cause buffer overdraw which can\u00a0slow down the render a lot if you render\u00a0many particles on the same location for instance.<\/p>\n
it’s trivial to create random or geometric position buffers, it’s straight forward to display an image of course (vertex position = normalized pixel position + elevation),\u00a0it’s easy to\u00a0create buffer describing 3D objects as we don’t need the connection information (the faces)\u00a0and as we shall see,\u00a0it’s also easy to animate this massive amount of particles.<\/p>\n
let’s start with a random buffer:<\/p>\n
\/\/returns an array of random 3D coordinates\r\nfunction getRandomData( width, height, size ){\r\n\r\n var len = width * height * 3;\r\n var data = new Float32Array( len );\r\n while( len-- )data[len] = ( Math.random() * 2 - 1 ) * size ;\r\n return data;\r\n}\r\n\r\n\/\/then you convert it to a Data texture:\r\nvar data = getRandomData( width, height, 256 );\r\nvar positions = new THREE.DataTexture( data, width, height, THREE.RGBFormat, THREE.FloatType );\r\npositions.needsUpdate = true;<\/pre>\n
this is what I used for the first demo not\u00a0very useful apart from debug,\u00a0an image is a touch more interesting, how would we do that?<\/p>\n
an image is a grid of values (pixels) so given an image, its width, height and an arbitrary elevation,\u00a0the buffer creation goes as follow:<\/p>\n
function getImage( img, width, height, elevation ){\r\n\r\n var ctx = getContext( null, width, height );\r\n ctx.drawImage(img, 0, 0);\r\n\r\n var imgData = ctx.getImageData(0,0,width,height);\r\n var iData = imgData.data;\r\n\r\n var l = (width * height );\r\n var data = new Float32Array( l * 3 );\r\n for ( var i = 0; i < l; i++ ) {\r\n\r\n var i3 = i * 3;\r\n var i4 = i * 4;\r\n data[ i3 ] = ( ( i % width ) \/ width -.5 ) * width;\r\n data[ i3 + 1 ] = ( iData[i4] \/ 0xFF * 0.299 +iData[i4+1]\/0xFF * 0.587 + iData[i4+2] \/ 0xFF * 0.114 ) * elevation;\r\n data[ i3 + 2 ] = ( ( i \/ width ) \/ height -.5 ) * height;\r\n }\r\n return data;\r\n}<\/pre>\n
the getContext() method creates a 2D context to\u00a0access the image’s pixels values and the loooooong line \u00a0computes the greyscale value for that pixel.<\/p>\n
which should give this (can’t view online because of CORS restrictions, it’s on the repo though):<\/span>
\nthanks to @makc<\/a> for pointing out in the comments that images need a crossOrigin, in this case, img.crossOrigin = “anonymous”; solved the problem, so enjoy the live demo :)
\n $\"image\"$ <\/a><\/p>\n
it uses\u00a0this\u00a0256 * 256 greyscale image:<\/p>\n
$\"noise\"$ <\/p>\n
loading a mesh is even easier:<\/p>\n
function parseMesh(g){\r\n\r\n var vertices = g.vertices;\r\n var total = vertices.length;\r\n var size = parseInt( Math.sqrt( total * 3 ) + .5 );\r\n var data = new Float32Array( size*size*3 );\r\n for( var i = 0; i < total; i++ ) {\r\n data[i * 3] = vertices[i].x;\r\n data[i * 3 + 1] = vertices[i].y;\r\n data[i * 3 + 2] = vertices[i].z;\r\n }\r\n return data;\r\n}<\/pre>\n
the method takes the geometry of the loaded mesh and the trick here is\u00a0to determine the size of the\u00a0texture from the amount of vertices. this total<\/em> is simply the square root of the vertices count.<\/p>\n
click the picture for\u00a0a\u00a0live version<\/p>\n
$\"mesh\"$ <\/a><\/p>\n
in the render shader, I compute the depth and the size of the particles is indexed on it which gives the illusion of faces but those are only particles (47516 particles, less than the first example).<\/p>\n
what about\u00a0animation?<\/p>\n
say we want to morph a cube into a sphere, first\u00a0we\u00a0need a sphere:<\/p>\n
\/\/returns a Float32Array buffer of spherical 3D points\r\nfunction getPoint(v,size)\r\n{\r\n v.x = Math.random() * 2 - 1 ;\r\n v.y = Math.random() * 2 - 1 ;\r\n v.z = Math.random() * 2 - 1 ;\r\n if(v.length()>1)return getPoint(v,size);\r\n return v.normalize().multiplyScalar(size);\r\n}\r\nfunction getSphere( count, size ){\r\n var len = count * 3;\r\n var data = new Float32Array( len );\r\n var p = new THREE.Vector3();\r\n for( var i = 0; i < len; i+=3 )\r\n {\r\n getPoint( p, size );\r\n data[ i ] = p.x;\r\n data[ i + 1 ] = p.y;\r\n data[ i + 2 ] = p.z;\r\n }\r\n return data;\r\n}<\/pre>\n
note that it\u00a0uses the “discard” approach ;\u00a0a point is generated randomly in the range [-1,1] and if it’s length > 1, it’s discarded. it is quite inefficient but\u00a0prevents points from being stuck<\/em> in the corners of a normalized cube. there’s also an exact way to prevent this problem that goes:<\/p>\n
Math.cbrt = Math.cbrt || function(x) {\r\n var y = Math.pow(Math.abs(x), 1\/3);\r\n return x < 0 ? -y : y;\r\n};\r\nfunction getPoint(v,size)\r\n{\r\n var phi = Math.random() * 2 * Math.PI;\r\n var costheta = Math.random() * 2 -1;\r\n var u = Math.random();\r\n\r\n var theta = Math.acos( costheta );\r\n var r = size * Math.cbrt( u );\r\n\r\n v.x = r * Math.sin( theta) * Math.cos( phi );\r\n v.y = r * Math.sin( theta) * Math.sin( phi );\r\n v.z = r * Math.cos( theta );\r\n return v;\r\n}<\/pre>\n
it\u00a0needs\u00a0 cubic roots<\/a> and involves more computations so all in all the discard method is not that bad and easier to understand (for people like me at least).<\/p>\n
now back to morphing, the way to go is to create 2 DataTextures, one for the cube, one for the sphere and pass them to the simulation shader.\u00a0the simulation shader will perform the animation between the 2 and render the result to the RenderTarget. then the render target will be used to draw the particles.<\/p>\n
\/\/first model\r\nvar dataA = getRandomData( width, height, 256 );\r\nvar textureA = new THREE.DataTexture( dataA, width, height, THREE.RGBFormat, THREE.FloatType, THREE.DEFAULT_MAPPING, THREE.RepeatWrapping, THREE.RepeatWrapping );\r\ntextureA.needsUpdate = true;\r\n\r\n\/\/second model\r\nvar dataB = getSphere( width * height, 128 );\r\nvar textureB = new THREE.DataTexture( dataB, width, height, THREE.RGBFormat, THREE.FloatType, THREE.DEFAULT_MAPPING, THREE.RepeatWrapping, THREE.RepeatWrapping );\r\ntextureB.needsUpdate = true;\r\n\r\nsimulationShader = new THREE.ShaderMaterial({\r\n\r\n uniforms: {\r\n textureA: { type: \"t\", value: textureA },\r\n textureB: { type: \"t\", value: textureB },\r\n timer: { type: \"f\", value: 0}\r\n },\r\n vertexShader: ShaderLoader.get( \"simulation_vs\"),\r\n fragmentShader: ShaderLoader.get( \"simulation_fs\")\r\n\r\n});<\/pre>\n
the simulation’s fragment shader is a bit more complex:<\/p>\n
\/\/ simulation\r\nuniform sampler2D textureA;\r\nuniform sampler2D textureB;\r\nuniform float timer;\r\n\r\nvarying vec2 vUv;\r\nvoid main() {\r\n\r\n \/\/origin\r\n vec3 origin = texture2D( textureA, vUv ).xyz;\r\n\r\n \/\/destination\r\n vec3 destination = texture2D( textureB, vUv ).xyz;\r\n\r\n \/\/lerp\r\n vec3 pos = mix( origin, destination, timer );\r\n gl_FragColor = vec4( pos,1.0 );\r\n\r\n}<\/pre>\n
that wasn’t too scary right? we have our 2 DataTextures (the cube and the sphere) passed to the simulation material along with a timer<\/em>\u00a0value which is a float in the range [0,1]. we sample the coordinates of the first model, store them in origin<\/em>, the coordinates of the second model, store them in destination<\/em>\u00a0then use the mix(a,b, T)\u00a0<\/em>to blend between the two.<\/p>\n
here’s a preview of the timer value being set at 0, 0.5 and 1:<\/p>\n
$\"morph_0\"$ <\/a> $\"morph_1\"$ <\/a> $\"morph_2\"$ <\/a><\/p>\n
of course we can create more sophisticated\u00a0animations, the idea is the same, anything that should alter the particles’ positions will happen in the simulation shader.<\/p>\n
for instance this uses a curl noise to move particles around:<\/p>\n
$\"animation\"$ <\/a><\/p>\n
the size is set\u00a0like this:<\/p>\n
gl_PointSize = size = ( step( 1. - ( 1. \/ 512. ), position.x ) ) * pointSize;<\/pre>\n
it reads, if the current “uv.x” is lower\u00a0than 0,998046875, it’s a small particle, otherwise it’s big.\u00a0if you’re interested in the simulation shader, it’s here<\/a>, I don’t think it’s an example of good practices though.<\/p>\n
to wrap it up, this technique allows to easily control insane amounts of particles, it is well supported (as compared to when I started using it at least) and – when using smaller texture sizes – it performs relatively well on most platforms ;\u00a0usually GPUs optimize nearby texture sampling, which is what the FBO is based on.<\/p>\n
I’ll leave you here, all the sources are on github<\/a>.
\nenjoy<\/p>\n","protected":false},"excerpt":{"rendered":"
update 210525: Mario Carrillo\u00a0was kind enough to port the code samples to ES6, something I’ve been willing to do for years. so check out his repo:\u00a0https:\/\/github.com\/marioecg\/gpu-party\/\u00a0( and check out his work while you’re at it ) particles are awesome. I can’t tell how many particle engines I’ve written for the past 15 years but I’d … [Read more…]<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":639,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"sharing_disabled":false,"spay_email":"","jetpack_publicize_message":""},"categories":[3],"tags":[],"jetpack_featured_media_url":"https:\/\/barradeau.com\/blog\/wp-content\/uploads\/2016\/01\/animation.jpg","jetpack_publicize_connections":[],"jetpack_shortlink":"https:\/\/wp.me\/s4oXhx-fbo","_links":{"self":[{"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/621"}],"collection":[{"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=621"}],"version-history":[{"count":21,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/621\/revisions"}],"predecessor-version":[{"id":623,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/621\/revisions\/623"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/media\/639"}],"wp:attachment":[{"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=621"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=621"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=621"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}