\n http:\/\/barradeau.com\/blog\/wp-content\/uploads\/2021\/02\/L3RHeYxkqe.mp4<\/a><\/video><\/div>\n
images are also very easy to work with so the\u00a0bulge<\/em>\u00a0effect was practically free.
\nit\u00a0sort of<\/em>\u00a0worked, at least it was not uninteresting and even worked with very thin lines (the roads)<\/p>\n
$\"\"$ <\/a><\/p>\n
I liked the version above, it came from using binary ( black or white ) values to draw the buildings and add a small blur, very crisp, just not the intended aesthetics.<\/p>\n
below is a “merged” tileset test<\/p>\n
$\"\"$ <\/a><\/p>\n
the tile combines 3 layers: the buildings in the red channel, the roads in the green channel and the altitude in the blue channel, the altitude is subtle but shows on the horizon.<\/p>\n
raster tiles proved to be limited, especially as it sneakily re-introduced the precision issue ; as the geometric grid doesn’t correspond to the the pixels, it gives the impression that the buildings “slide” over the geometry instead of being solidly anchored in the geometry, not what we wanted.<\/p>\n
I tried a couple of things to create meshes, was interested in TIN<\/a>\u00a0but it takes a Geotiff as an input and I didn’t manage to create it, try later.<\/p>\n
now the nice thing with having vector data is that we can apply various operations onto it. I pushed the pre-process a bit further and simplified, triangulated each building Polygon with OPENCV<\/a>, then extruded and stored the tiles as binary objects.
\nand voil\u00e0! 3D tiles!
\n $\"\"$ it’s not perfect and there are some holes in the caps but it’s good enough when seen from afar and it looks cool which is the only relevant metric.<\/p>\n
behold! the embodied nightmare of tile providers and GPU makers alike: a WebGL map of Marseille at zoom 19 ( ~3.5M triangles ) $\"\"$ <\/a><\/p>\n
so dense, so yummy.<\/p>\n
$\"\"$ <\/a>
\nand the good part is that some buildings are well documented like this stadium<\/p>\n
$\"\"$ <\/p>\n
which brings a great sense of detail and creates some happy accidents<\/em> while navigating the city.
\nby the way, the OSM data contain a unique ID per feature so when making the tiles, I made sure to store them to prevent overlapping buildings as they may belong to various tiles. it’s even more important with roads that can cross many tiles and appear in each of the tiles. if it gives lighter tiles and prevents z-fighting, the tradeoff is that they’re not sorted : a tile may load with some “missing” buildings that appear later when the neighbouring tiles load.<\/p>\n
a further optimisation would consist in merging the tiles ; for now each tile contains roughly 2K vertices, I could have merged them 4 by 4 or up to a fixed size but this would have induced different loading strategies. I could also have simplified the paths more aggressively but as such, it’s not perfect but it works<\/p>\n
<\/h2>\n
<\/p>\n
navigation & tile management<\/h2>\n
a quick note about navigation: everything is located with a regular tile map engine, as if they were raster tiles.
\nrecomputing individual GPS coordinates of each building polygon would be prohibitively expensive for the CPU, so\u00a0the 3D tiles coordinates were computed in a 256\u00b2 space to match the size of a raster tile.\u00a0the engine then only places the 3D tiles according to the top left lat\/lng of the tile, at zoom level 16.\u00a0this still gives very high position values (in the tenth of millions units) so\u00a0everything is offset by “minus the center” of the map, this prevents very high positions values that cause the renderer to glitch because of the depth map precision (yes, even with a logarithmic depth buffer).<\/p>\n
there’s a second benefit in offseeting everything by minus the center of the map: the camera target is always at 0, 0, 0 !<\/p>\n
http:\/\/barradeau.com\/blog\/wp-content\/uploads\/2021\/02\/xF8LrTCsAS.mp4<\/a><\/video><\/div>\n
the map has a “display radius” that manages the tiles’ visibility and there’s a vertex shader that handles the tiles’ apparition, I used the rather crypitc<\/em> yet so handy `onBeforeCompile` method of three.js’ materials.<\/p>\n
\/\/material is a MeshStandardMaterial instance\r\nlet uniforms = { radius: { value: 1 } };\r\nmaterial.onBeforeCompile = (m) => {\r\n m.uniforms = Object.assign(m.uniforms, uniforms);\r\n let vs = m.vertexShader;\r\n vs = vs.replace(\r\n \"#include <common>\",\r\n `#include <common>\r\n uniform float radius;\r\n float noise(vec2 p){\r\n vec2 ip = floor(p);\r\n vec2 u = fract(p);\r\n u = u*u*(3.0-2.0*u);\r\n float res = mix(\r\n mix(rand(ip),rand(ip+vec2(1.0,0.0)),u.x),\r\n mix(rand(ip+vec2(0.0,1.0)),rand(ip+vec2(1.0,1.0)),u.x),u.y);\r\n return res*res;\r\n }`\r\n );\r\n vs = vs.replace(\r\n \"#include <displacementmap_vertex>\",\r\n `#include <displacementmap_vertex>\r\n float r = length( (modelMatrix * vec4( transformed, 1. ) ).xyz ) \/ radius;\r\n r = 1. - smoothstep( .6, .75, r + .5 * ( noise(transformed.xz * 0.01 ) - noise(transformed.xz * 0.0001 ) ) );\r\n transformed.y *= min( 1., r );`\r\n );\r\n m.vertexShader = vs;\r\n};<\/pre>\n
I control the height of the buildings by adding noise depending on the distance to the center of the world (which is … 0, 0, 0 ! thank you offset :))<\/p>\n
<\/h2>\n
<\/p>\n
terrain and water<\/h2>\n
for the terrain, I thought I’d use a tileset, there’s a 1:1 match with the building tiles’ locations, the data is readily available through the\u00a0 amazon LTS<\/a> and it’s easy to compute.<\/p>\n
$\"\"$ <\/p>\n
well, think again tiger! couldn’t figure out exactly why but some seams appeared in 3D when they don’t show on a 2D map. tyro<\/a> <\/strong>(<check this out!) suggests to blend the seams by using the values of the neighbouring tiles. given the tile based nature of my map, it was too complex to set up (need to recompute up to 9 textures each time a tile was loaded).<\/p>\n
so instead I collected all the altitudes with this excruciatingly slow python method:<\/p>\n
import cv2\r\n# convert the elevetion to an altitude in meters\r\ndef decodeUint8Altitude(pixel):\r\n b,g,r = pixel\r\n return ( r * 256 + g + b \/ 256) - 32768\r\n\r\n#this parses each pixel of an image\r\ndef filter( img ):\r\n height, width, color = img.shape\r\n res = np.zeros( (img.shape[1], img.shape[0] ), dtype=np.float )\r\n for y in range(height):\r\n for x in range(width):\r\n for c in range(color):\r\n res[y, x] = max(0, decodeUint8Altitude(img[y, x]) )\r\n return res\r\n#call \r\nres = filter( cv2.imread( \"tile.png\", cv2.IMREAD_UNCHANGED )\r\n#res now contains the altitudes as floats\r\n<\/pre>\n
I saved the altitude tiles as 8bit pngs with a 3 meters increment (not very precise but ok-ish) and computed a big png file of the altitudes, hoping to use it as a displacement map in blender.<\/p>\n
meanwhile I also triangulated and exported the land and water from my vector data, and merged them into OBJ files. in Blender, I cleaned them up manually, there are some insanely cool tools to decimate edges and create faces! ( edit mode > edge mode : select > select similar, mesh > cleanup > decimate geometry, you’re welcome :) )<\/p>\n
$\"\"$ <\/a><\/p>\n
it worked nicely for the water but unfortunately, the grid of the landmass mesh was too regular and not well defined enough so I gave up.<\/p>\n
first I though I’d use the altitudes texture as a displacement map in blender but instead, I used the\u00a0 https:\/\/elevationapi.com\/<\/a>\u00a0to produce a mesh, manually aligned it to the coastline and decimated it to reach a reasonable triangle count.\u00a0I used a smaller version of the altitudes’ PNG to locate the surface and prevent the camera from going through the ground. as the ground mesh is much coarser than a tileset, I couldn’t get the roads to “stick” to the ground nicely so they float above the terrain, guess I’ll have to live with it.<\/p>\n
<\/h2>\n
360\u00b0 panoramas & titles<\/h2>\n
the map was fun already but there was a second important feature: 360\u00b0 panoramas.<\/p>\n
I happened to work on Hopper the Explorer<\/a>\u00a0and already knew a thing or two about photospheres, this time they needed a fresh take on panoramas. I suggested to use a Machine Learning model<\/a> to infer depth and make the sphere appear using a depth map.<\/p>\n
the first transition is seen from outside the sphere to give an idea of how the geometry is distorted when using the depth map, the last bit of the video shows how it looks from inside, the intended effect,\u00a0the red ribbon is the text that was supposed to be partially occluded by the foreground.\u00a0this is the test I did to convince them it was a good idea:<\/p>\n
http:\/\/barradeau.com\/blog\/wp-content\/uploads\/2021\/02\/ikA8OxngSD.mp4<\/a><\/video><\/div>\n
and the least I can say is that they were not<\/strong> convinced ^^’<\/p>\n
so we dumped the depth maps and changed for a much more qualitative approach, closer to the art direction: Axel created 2D masks for the foreground to partially hide the text and did some transition mockups using an\u00a0optics compensation<\/strong> effect (aka a pincushion effect<\/strong>).<\/p>\n
the end result looks somewhat like this (better check it live<\/a> though).<\/p>\n
http:\/\/barradeau.com\/blog\/wp-content\/uploads\/2021\/02\/SmeaaugOHO.mp4<\/a><\/video><\/div>\n
here’s the fragment shader used during the post processing, it takes 2 textures as input, tDiffuse the map, tSphere, the panorama, + a `mixer` value.<\/p>\n
uniform float mixer;\r\nuniform sampler2D tDiffuse;\r\nuniform sampler2D tSphere;\r\nvarying vec2 vUv;\r\n\/\/idea\r\n\/\/https:\/\/www.decarpentier.nl\/lens-distortion\r\n\/\/distort method from \r\n\/\/ https:\/\/www.imaginationtech.com\/blog\/speeding-up-gpu-barrel-distortion-correction-in-mobile-vr\/\r\nvec2 distort(vec2 st, float alpha, float expo ){\r\n vec2 p1 = vec2(2.0 * st - 1.0);\r\n vec2 p2 = p1 \/ (1.0 - alpha * length( p1 ) * expo );\r\n return (p2 + 1.0) * 0.5;\r\n}\r\nvoid main() {\r\n float t = mixer;\r\n float tr = smoothstep( 0.75,1., t );\r\n float e = t * 4.;\r\n \r\n \/\/stretch out city scene\r\n vec2 uv = distort( vUv, -10. * pow( .5 + t * .5, 32. ), e );\r\n vec4 city = texture2D( tDiffuse, uv );\r\n \r\n \/\/pinch in sphere scene\r\n uv = distort( vUv, -10. * ( 1. - tr ), e );\r\n vec4 sphere = texture2D( tSphere, uv );\r\n \/\/blend \r\n gl_FragColor = mix( city, sphere, tr );\r\n}<\/pre>\n
the titles needed to be animated letter by letter and localized, Axel pointed me to the JS library they use at Fonts Ninja<\/a>\u00a0and I was impressed!
\n opentype.js<\/a>\u00a0will give you access to the vector data of a font, with a proper kerning, it allowed to write arbitrary texts on the fly (I should probably start using SDF Fonts at some point).<\/p>\n
here’s a test of the split letters word<\/p>\n
$\"\"$ <\/p>\n
and a quick animation test.<\/p>\n
http:\/\/barradeau.com\/blog\/wp-content\/uploads\/2021\/02\/DH8KauTbFE.mp4<\/a><\/video><\/div>\n
one of my regrets is not to have had more time to polish the text animation.<\/p>\n
<\/p>\n
<\/p>\n
Post processing LUT<\/h2>\n
at some point Axel told me he used Photoshop LUTs (Look Up Table) to beautify the C4D renders and asked if we could have this on the website. little did I know how much of a pain it would be. first, a LUT is a tool to transform a pixel value, like a filter. to disambiguate, what most people call a LUT is often a palette or gradient mapping ; using a greyscale image input, we fetch the corresponding value in a palette (often a gradient) and replace the greyscale value with the color. here’s a three.js example:\u00a0 https:\/\/threejs.org\/examples\/?q=lookuptable#webgl_geometry_colors_lookuptable<\/a>\u00a0so it’s usually limited to 256 discrete values and gives a very cartoon look to the final image ( it works very well for cell shading ).<\/p>\n
photoshop LUTs map each pixel of an image to a\u00a032768 (32^3) RGB values and give much more subtle results.<\/p>\n
FUN FACT<\/strong> three.js does have a 3D LUT post processing pass<\/a> since r122<\/a>, I was simply using the r120 and by the time I realized it was in there, I had a working alternative. I’m leaving the code here for “learning purpose” or in case the 3D LUT uses 3D textures (I haven’t checked, I’m still too pissed!)<\/p>\n
to illustrate, the image below is the LUT I used, as rendered by photoshop.\u00a0on the left hand side, it’s a series of 64* ( 64 * 64 pixels ) quads that gradually brighten up. think of it as the slices of a box made of 63^3 cubes. on the right hand side, the LUT was applied to the same grid and – hopefully – you see that the colors behave differently. used on an image, this will produce the (in)famous Instagram filter look.<\/p>\n
$\"\"$ <\/a><\/p>\n
in the app, we’ll use the right hand texture as our LUT.<\/p>\n
I asked some creative dev friends for help, they pointed to Matt DesLauriers implementation<\/a>\u00a0which helped a lot, I then used it as part of the post process pipeline. for the record here’s the code.
\nfirst a python script that generates the grid above:<\/p>\n
import cv2\r\nimport numpy as np\r\nsize = 64\r\nbuffer = np.zeros( (512, 512,3), dtype=np.uint8)\r\nfor b in range( 0, size ):\r\n x = ( b % 8 ) * size\r\n y = ( int( b \/ 8 ) ) * size\r\n for r in range( 0, size ):\r\n for g in range( 0, size ):\r\n buffer[ y+g:y+g+1, x+r:x+r+1, 0 ] = int( b * 4 )\r\n buffer[ y+g:y+g+1, x+r:x+r+1, 1 ] = int( g * 4 )\r\n buffer[ y+g:y+g+1, x+r:x+r+1, 2 ] = int( r * 4 )\r\ncv2.imwrite( \"lut_src.png\", buffer )<\/pre>\n
this is not required by the app but it can help checking in photoshop and\/or debug in WebGL.<\/p>\n
the ShaderPass goes like:<\/p>\n
let LUTShader = {\r\n uniforms: {\r\n tDiffuse: { value: null },\r\n tLut: { value: null },\r\n opacity: { value: 1.0 },\r\n },\r\n vertexShader: `\r\n varying vec2 vUv;\r\n void main() {\r\n \tvUv = uv;\r\n \tgl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );\r\n }`,\r\n fragmentShader: `\r\n uniform float opacity;\r\n uniform sampler2D tDiffuse;\r\n uniform sampler2D tLut;\r\n varying vec2 vUv;\r\n vec4 lookup(in vec4 textureColor, in sampler2D lookupTable) {\r\n \/\/square index by blue level (linear ; it gives the cell index in a 'flat' array)\r\n float b = ( textureColor.b ) * 63.;\r\n \/\/ current cell id : we remove the fractional part to \"snap\" to the previous integer (ie 12.3564546 > 12)\r\n float fb = floor( b ); \r\n \/\/ current cell uv\r\n \/\/ the x position of the cell in the grid is given by : mod( cellId, columns)\r\n \/\/ the y position of the cell in the grid is given by : floor( cellId \/ columns)\r\n vec2 uv0 = vec2( mod( fb, 8. ), floor( fb \/ 8. ) );\r\n \/\/ and everything is downscaled by : 1 \/ columns ( 1. \/ .8 in this case as 64 px cell size = 512 px texture size \/ 8 )\r\n float grid_cell = 1. \/ 8.;\r\n \/\/this is the top left corner of the cell\r\n uv0 *= grid_cell;\r\n \/\/we need to \"pad\" the cell to make sure we hit the center of the cell's corner pixels\r\n \/\/padding = half a pixel so: .5 \/ 512.\r\n float padding = .5 \/ 512.;\r\n vec2 cell_offset = vec2( padding );\r\n \/\/the must be a little smaller too: the width = 2\r\n vec2 cell_size = vec2( grid_cell - 2. * padding );\r\n \/\/we add the offset to the uv and finally we lerp the uv between the top left and bottom right corners of the cell\r\n \/\/we use the r an g value of the texel\r\n uv0 += cell_offset + textureColor.rg * cell_size;\r\n \/\/ next cell id and uv (same as above with the next index in the 'flat list' )\r\n float cb = ceil( b );\/\/min( fb+1., 63. ); \r\n vec2 uv1 = grid_cell * vec2( floor( mod( cb, 8. ) ), floor( cb \/ 8. ) ); \r\n uv1 += cell_offset + textureColor.rg * cell_size;\r\n \/\/flip Y\r\n uv0.y = 1. - uv0.y;\r\n uv1.y = 1. - uv1.y;\r\n \/\/now we sample the 2 colors from the lookup table\r\n vec4 c1 = texture2D(lookupTable, uv0);\r\n vec4 c2 = texture2D(lookupTable, uv1);\r\n \/\/and mix them using the fractional part of the ( blue value * cell count )\r\n return mix(c1, c2, fract( b ) );\r\n }\r\n void main() {\r\n vec4 texel = texture2D( tDiffuse, vUv );\r\n gl_FragColor = lookup( texel, tLut );\r\n }\r\n `,\r\n};\r\nexport { LUTShader };<\/pre>\n
and finally, to use it with three.js effectComposer:<\/p>\n
renderer.autoClear = false;\r\nlet composer = new EffectComposer(renderer);\r\nconst renderPass = new RenderPass(scene, camera);\r\ncomposer.addPass(renderPass);\r\nlet LUTPass = new ShaderPass(LUTShader);\r\nlet loader = new TextureLoader();\r\nloader.load(\"\/textures\/lut.png\", (t) => {\r\n LUTPass.uniforms.tLut.value = t;\r\n t.magFilter = t.minFilter = LinearFilter; \/\/ important!\r\n});\r\ncomposer.addPass(LUTPass);<\/pre>\n
I wanted to add a bokeh filter too but it was deemed to CPU intensive and was removed (still available in the debug panel<\/a>\u00a0under\/params\/postprocess) .<\/p>\n
<\/h2>\n
<\/h2>\n
wrap up<\/h2>\n
what a “fun little project”!\u00a0I learnt a lot and it was so nice to work with very skilled people, Axel<\/a> (AD, 2D\/3D motion & visuals…) & Emmanuel<\/a> (Back, Front, Photos…) are very talented and very demanding. I like that very much.
\nby the way the debug panel is still active, you can try :\u00a0https:\/\/marseille.laphase5.com\/?debug=true and play with some of the stuff, overshoot the LUT, add bokeh, change the colors…<\/p>\n
enjoy! :)<\/p>\n","protected":false},"excerpt":{"rendered":"
last year I was contacted by\u00a0la phase 5 , they needed help with the WebGL part of their 2021 wish card. they’re based in Marseille, the second biggest french city, by the mediterranean sea. it’s quite a unique city, both classy and cheap, packed and desert,\u00a0La Phase 5’s plan\u00a0was to write a love letter to … [Read more…]<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":1264,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"sharing_disabled":false,"spay_email":"","jetpack_publicize_message":""},"categories":[3],"tags":[8,10,9],"jetpack_featured_media_url":"https:\/\/barradeau.com\/blog\/wp-content\/uploads\/2021\/02\/brave_2W4vG1m2mf.jpg","jetpack_publicize_connections":[],"jetpack_shortlink":"https:\/\/wp.me\/p4oXhx-ju","_links":{"self":[{"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1208"}],"collection":[{"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1208"}],"version-history":[{"count":37,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1208\/revisions"}],"predecessor-version":[{"id":1269,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1208\/revisions\/1269"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=\/wp\/v2\/media\/1264"}],"wp:attachment":[{"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1208"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1208"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/barradeau.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1208"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

<\/h2>\n <\/p>\n

<\/h2>\n

<\/h2>\n
<\/p>\n