Warning: Declaration of Jetpack_IXR_Client::query() should be compatible with IXR_Client::query(...$args) in /home/clients/7267bc096562fcdb78c0ab60d3ac51fb/web/blog/wp-content/plugins/jetpack/class.jetpack-ixr-client.php on line 91
{"id":1165,"date":"2017-09-16T09:46:30","date_gmt":"2017-09-16T09:46:30","guid":{"rendered":"http:\/\/barradeau.com\/blog\/?p=1165"},"modified":"2017-09-16T09:48:42","modified_gmt":"2017-09-16T09:48:42","slug":"generating-things-with-code-1-of-2","status":"publish","type":"post","link":"http:\/\/barradeau.com\/blog\/?p=1165","title":{"rendered":"Generating things with\u00a0code\u00a0(1 of 2)\u00a0"},"content":{"rendered":"

this article was initially published on Medium<\/a>, I’m posting it here for the record.<\/p>\n

\n

\n

\n

<\/h1>\n

PCG, data and\u00a0space<\/h1>\n<\/div>\n<\/div>\n<\/section>\n
\n
\n
\n
1. PROCEDURAL CONTENT GENERATION (PCG)<\/h3>\n
The essence of generative design is to use\u00a0functions\u200a\u2014\u200asimple, independent actions\u200a\u2014\u200a<\/em>and\u00a0algorithms\u200a\u2014\u200aseries of actions required to solve a given task\u200a\u2014\u200a<\/em>to build systems that create series of non-repeating results.\u00a0PCGs encompass the programmatic generation of content, they often rely on pseudo-random data and are essentially the task of\u00a0designing a process<\/em><\/span>\u00a0rather than an\u00a0object.\u00a0<\/em>PCGs are massively used in object design, games, art, vfx, architecture\u2026<\/p>\n
DATA \/ SPACE \/\u00a0PROCESS<\/h4>\n
The coding platform, the language, the tools & tool-chain, the capture devices & data sources are irrelevant. This will obviously have an impact on the final result if chosen poorly but it is of no importance when designing a generative system. Generating things with code is language & platform agnostic.<\/p>\n
On the other hand, there are 3 important things to\u00a0take into account when generating something with code: the\u00a0data<\/strong>\u00a0you\u2019ll be using, the\u00a0space<\/strong>\u00a0in which they\u2019ll be displayed and the\u00a0process<\/strong>\u00a0that will transform the data.<\/span><\/p>\n
A generative system should allow you to perform the following tasks:
\nmodelisation:<\/strong>\u00a0create the model of a system\u00a0
\nsimulation:\u00a0<\/strong>use the model to process data
\nexploration:<\/strong>\u00a0alter the model\u2019s and or the simulation\u2019s parameter
\nvisualization:<\/strong>\u00a0rendering the simulation as 2D or 3D graphics, video, tangible objects, installations\u2026<\/p>\n
DOWNSIDES<\/h4>\n
\n
the mathematical uniqueness is not enough to create significant differences between outcomes<\/li>\n
it is hard to maintain consistent variations while using numerous variables<\/li>\n
it is hard to assess \u201cgood\u201d settings, human curation is often required<\/li>\n<\/ul>\n
UPSIDES<\/h4>\n
\n
PCGs are very good at serial content<\/li>\n
the computer does the boring part<\/li>\n
few parameters give many variations<\/li>\n
emergent<\/em>\u00a0behaviors can produce surprising or unexpected results<\/li>\n<\/ul>\n
Kate Compton has written an extensive post<\/a>\u00a0about PCG, you\u2019ll find lots of practical advices from a seasoned professional.<\/p>\n
EXAMPLES<\/h4>\n<\/div>\n
\n
\n
\n
<\/div>\n
\n
Hello Games\u200a\u2014\u200ano man\u2019s sky<\/a><\/p>\n
model<\/strong>: universe generator
\nsimulation<\/strong>: update the model with game logic
\nexploration<\/strong>: interact with the simulation
\nvisualization<\/strong>: rendering a 3D scene<\/p>\n<\/div>\n
\n
\n
\n
<\/div>\n
\n
nervous systems\u200a\u2014\u200aKINEMATICS FOLD<\/a><\/p>\n
model<\/strong>: triangle based patch modeler
\nsimulation<\/strong>: apply physics rules to the model
\nexploration<\/strong>: change the input 3D model
\nvisualization<\/strong>: 3D renders & 3D print<\/p>\n<\/div>\n
\n
\n
\n
<\/div>\n
\n
We Are Chopchop\u200a\u2014\u200aunnamed soundsculpture<\/a><\/p>\n
model<\/strong>: represent a 3D object in time and space
\nsimulation<\/strong>: playback of a recorded sequence
\nexploration<\/strong>: move a virtual camera through space
\nvisualization<\/strong>: render particles in 3D<\/p>\n
In this example, the simulation\u2019s rules\u200a\u2014\u200athe algorithm or the system\u200a\u2014\u200acan be described as follows:<\/p>\n
\n
spawn a particle at a random location on a 3D mesh at a given time<\/li>\n
make the particle fall over time and chek if it hits the floor<\/li>\n
if the particle hits the floor, make it disappear after a while and respawn<\/li>\n<\/ul>\n
The particles move & disappear\u00a0procedurally<\/em>, we know the initial position and the simulation will determine where the particle ends. This generative system can work with any 4 dimensional data set (3D position over time).<\/p>\n
\n<\/div>\n<\/div>\n<\/section>\n
\n
\n
\n
2. DATA<\/h3>\n
The universe is a\u00a0continuous<\/em>\u00a0system yet computers need to process\u00a0discrete<\/em>data. In a nutshell, the difference is that continuous data are\u00a0measured<\/strong>\u00a0while discrete data are\u00a0counted<\/strong>. The reason for that is simple\u00a0; the precision of the data that a computer works with are limited. For instance, the continuous\u00a0time\u00a0<\/em>must become an\u00a0interval<\/em>\u00a0for us to use it in a generative system. The\u00a0light\u00a0<\/em>must become a\u00a0color palette<\/em>, a chemical\u00a0reaction\u00a0<\/em>must be described as a\u00a0process<\/em>, a\u00a0phenomenon\u00a0<\/em>will have to become a\u00a0simulation etc.<\/em><\/p>\n
Any given object can be described by a number of\u00a0dimensions<\/em>, in code we often refer to these dimensions as\u00a0properties\u00a0<\/em>or\u00a0fields<\/em>\u00a0but that\u2019s somewhat restrictive.<\/em>\u00a0The values of each dimension of an object can vary without affecting the others. In the case of light, the\u00a0direction<\/em>,\u00a0speed<\/em>,\u00a0wavelength\u00a0<\/em>and\u00a0energy<\/em>\u00a0can vary independently\u00a0; they\u2019re all distinct\u00a0dimensions\u00a0<\/em>of the light and we can model and simulate them in code as the properties of an object.<\/p>\n
computer representation of uni-dimensional data<\/h4>\n
Once a\u00a0dimension\u00a0<\/em>is discrete, we can use it in our models and simulations. Depending on the\u00a0granularity<\/em>\u00a0of the dimension, we can use various\u00a0data-holders<\/em>\u00a0the simplest of which is the\u00a0bit<\/em>\u00a0that can only hold 2 values\u00a0: 0 \/ 1. An obvious limitation is that it can only describe whether a given property of the object \u201cis\u201d or \u201cisn\u2019t\u201d. If we need a finer grain, we should use different data types, what I like to call the \u201c1D extension pack\u201d which contains:<\/p>\n
\n
byte \/ octet: 8 bits & 256 integer values<\/li>\n
float, uint, double: 16, 32, 64 bits<\/li>\n
character: 7 bits (US ASCII ), 32 bits (UTF-32)<\/li>\n<\/ul>\n
acknowledged that in the computers\u2019s memory, a number or a character is stored as an array of bits, we could argue that a number or a character are themselves N-dimensional objects.<\/p>\n
\n
\n
<\/div>\n
<\/p>\n<\/div>
https:\/\/en.wikipedia.org\/wiki\/Single-precision_floating-point_format#\/media\/File:Float_example.svg<\/a><\/figcaption><\/figure>\n
data in 2 dimensions<\/h4>\n
The next step towards complexity is to use series of uni-dimensional data, for instance:<\/p>\n
\n
a string: \u201cGenerating things with code\u201d<\/li>\n
a typed linear array: [\u201ca\u201d, \u201cb\u201d, \u201cc\u201d] \/ [ 0, 1, 2 ]<\/li>\n
an (empty) image: [ width * height ]<\/li>\n<\/ul>\n
the object is already multi dimensional\u00a0; each dimension can be altered without affecting the whole. The string can be longer or shorter without altering the letters, and the letters can change without altering the length of the sentence, the image\u2019s width can change without altering its height, a.s.o.<\/p>\n
n-dimensional data<\/h3>\n
If we extrapolate further, N-dimensional data are a combinatorics of many uni-dimensional data, here are some 3D objects:<\/p>\n
\n
an opaque color: { R, G, B }<\/li>\n
a vector: { X, Y, Z }<\/li>\n
a greyscale pixel: { X, Y, greyscale }<\/li>\n<\/ul>\n
As the name suggests, N-dimensional data can represent any number of dimensions, here are some more complex (yet fairly common) data-holders:<\/p>\n
\n
pixel + color: { X, Y, R, G, B }<\/li>\n
pixel + color & alpha: { X, Y, R, G, B, A }<\/li>\n
stereo sound: { left, right, time, sampling rate }<\/li>\n
video: size, color depth, sound, time<\/li>\n
3D object: positions, indices, normals, colors, uvs, textures\u2026<\/li>\n<\/ul>\n
From the\u00a0dimensions\u2019\u00a0<\/em>perspective, a single tweet can potentially be represented in ~72 dimensions, often themselves N-dimensional. The underlying idea is that we often overlook the complexity of the simplest things\u00a0; a short text, a limited set of colors, an IP address are complex N-dimensional objects.<\/p>\n
data acquisition<\/h3>\n
We are also surrounded by devices that allow us to capture rich data from our surroundings. A\u00a0smartphone<\/strong>\u00a0can capture photos, videos, sounds & GPS positions but also has a set of \u201chidden\u201d sensors:<\/p>\n
TYPE_ACCELEROMETER Motion detection (shake, tilt, etc.).\r\nTYPE_AMBIENT_TEMPERATURE Monitoring air temperatures.\r\nTYPE_GRAVITY Motion detection (shake, tilt, etc.).\r\nTYPE_GYROSCOPE Rotation detection (spin, turn, etc.).\r\nTYPE_LIGHT Controlling screen brightness.\r\nTYPE_LINEAR_ACCELERATION Monitoring acceleration along an axis.\r\nTYPE_MAGNETIC_FIELD Creating a compass.\r\nTYPE_ORIENTATION Determining device position.\r\nTYPE_PRESSURE Monitoring air pressure changes.\r\nTYPE_PROXIMITY Phone position during a call.\r\nTYPE_RELATIVE_HUMIDITY Monitoring absolute & relative humidity.<\/pre>\n
All these sensors (and more) are readily accessible and can be recorded from the smartphones as an input to our generative system.<\/p>\n
The\u00a0open data\u00a0<\/strong>available to us have grown exponentially for the past years, they are rather oriented towards statistical analysis, mostly \u201cgeo-social\u201d and often geolocated. This is especially true for cartography but not only.<\/p>\n
We can also use some\u00a0tracking devices<\/strong>\u00a0such as a webcam or a microphone as with a smartphone but also mainstream devices like a Kinect (video, depth, 3D point clouds, infrared video, skeletal information), the Leap Motion that precisely scans a smaller area and returns a well defined point cloud and descriptors for hands and fingers positions. More recently the VR devices like the Oculus Rift (mostly 3 DOF) and the HTC Vive (room scale 6 DOF) that allow to precisely capture the orientation and position of a headset and hand held devices in space and time.<\/p>\n
If this is\u2019nt enough, it\u2019s possible to build\u00a0custom devices<\/strong>\u00a0starting with Arduino boards and Processing and adding all manners of cheap sensors: gesture detection, heat detector, 360\u00b0 scanners, wind, rain, smoke sensors\u2026<\/p>\n
And finally, let\u2019s not forget that we can\u00a0synthesize\u00a0<\/em>data. The most basic form of\u00a0synthetic data<\/strong>\u00a0is obviously the random, noise, fractals and turbulence generators, but other techniques have emerged in the past years that deserve a mention.\u00a0Photogrammetry\u00a0<\/strong>is one of them, it consists in re-building a 3D model from a set of 2D pictures (see\u00a0visualSFM<\/a>\u00a0for more information). Convolutional Neural Networks, Generative Adversarial Networks, what we usually call\u00a0AI\u00a0<\/strong>that gained a huge momentum since deepdream and partially solves one of the downsides of PCGs I mentioned above ( \u2018it is hard to assess \u201cgood\u201d settings\u2019 )<\/p>\n
In short, there are more and more data available and they become more and more complex. Reduced to sets of\u00a0uni-dimensional<\/em>\u00a0variables, these data can\u00a0shift<\/em>\u00a0between various\u00a0spaces.<\/em><\/p>\n<\/div>\n<\/div>\n<\/section>\n
\n
<\/div>\n
\n
\n
<\/h3>\n
3. SPACES<\/h3>\n
Spaces are fields of representation, a place where the data can exist. They have specific qualities that can help you highlight the traits of the data you\u2019re using. The important thing to acknowledge is their umber of dimensions\u00a0; N-dimensional data require N-dimensional space to be fully displayed.<\/p>\n
2D euclidean spaces<\/h4>\n<\/div>\n
\n
\n
\n
<\/div>\n