| Home | Pictures of 3D objects |
Technical issuesHere are collected some discussions of technical aspects of GlowScript, which may be of interest to other software developers. Pixel-level transparency based on depth peeling Handling transparency correctly in WebGL is challenging. A standard technique is to order the centers of objects according to their z depth from the camera and render back to front. This approach can make serious errors in the case of intersecting or enclosing transparent objects, where some parts of one object are in front of a second object, and other parts are behind the second object. A better approach is called "depth peeling", in which transparency is dealt with at the pixel level rather than at the object level. Consider for example a transparent sphere enclosing an opaque box, and carry out the following operations: 1) Render the pixels of the opaque box to a texture, which we'll call a color texture. This of course takes into consideration the lights in the scene. Call this color texture C0. Note that all opacity values (the "a" in rgba) are 1.0 for this opaque object. 2) Render the z depths (distance from the camera) to a texture, which we'll call a depth texture. That is, given the depth of a pixel, the fragment shader stores a false color into the texture, a false color whose 4 bytes represents in some form the depth. Call this depth texture D0. 3) Render the transparent object (the sphere) to a texture C1, but using information in the depth texture D0. In the fragment shader, read the (false color) depth from D0, and if the z depth of the transparent sphere pixel is not in front of the opaque box pixel, discard the pixel (using the "discard" statement in the fragment shader). Upon completion of this render step, C1 contains color and opacity information (rgba) for the front-most transparent surface, a hemisphere. This is called a "depth peel". 4) We now have two color textures, C0 and C1, which we can merge to form a scene with an opaque box and a transparent hemisphere in front of the box. Render a simple quad object (two triangles) that fill the canvas. In the fragment shader, read the color information from C0 and C1. Store a pixel color that is determined like this, where (1.0-C1.a) is the transparency of the transparent layer: vec3 color = C1.rgb*C1.a + These four steps illustrate the basic idea. In GlowScript, four transparent depth peels are performed rather than one, and 10 separate renders are carried out: C0 -- opaque color texture MERGE -- The final render is a merge of C0, C1, C2, C3, and C4: vec3 color = C1.rgb*C1.a + It may be that C2, C3, and C4 are all empty, but there is no obvious inexpensive way to get the information needed to tell the CPU to avoid scheduling those extra renders, because readPixels is very expensive. If there are more than four transparent layers, this algorithm will not treat them properly. However, note that the fifth and later peels will contribute little to the final pixel color, being partially occluded by four transparent layers in front. Before starting the many renders the objects are sorted into opaque and transparent lists, and if there are no transparent objects a simple C0 render is all that is needed. Moreover, this simple render can exploit antialiasing, whereas the storage into textures for depth peeling unfortunately turns off antialiasing. It is remarkable that doing 10 separate renders runs adequately fast for real-time rendering of moderately complicated scenes. For example, displaying a 10x10x10 grid of rotating transparent boxes can run at around 20 frames per second on ordinary computers, depending on the graphics card. In OpenGL it is possible to create several textures in one render, but WegGL permits attaching just one texture to a framebuffer object, hence a large number of separate renders are needed. To see this algorithm in action, run this Transparency example program. Note that the code for this transparency demo is remarkably short. GlowScript is aimed at making it feasible for nonexpert programmers to exploit WebGL to generate navigable real-time 3D animations. At https://github.com/BruceSherwood/glowscript, the key files dealing with depth peeling are lib/glow/WebGLRender.js and the shader programs in the shaders folder. It is possible to implement "Fast Approximate Anti-Aliasing" (FXAA) to get around the problem that the use of framebuffer objects turns off anti-aliasing: This has not been tried in GlowScript. Mouse picking Here is how mouse picking has been implemented in GlowScript. For an object with id number N, create a false color like this: function id_to_falsecolor(N) { We want to render the false colors of the objects to a texture the size of the canvas (need not be powers of 2): var pick_texture = gl.createTexture() gl.texParameteri(gl.TEXTURE_2D, gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, Set up a framebuffer to render to: var pickFramebuffer = gl.createFramebuffer() // create depth buffer: gl.bindRenderbuffer(gl.RENDERBUFFER, null) At the start of a render cycle execute this: gl.bindFramebuffer(gl.FRAMEBUFFER, After rendering all the objects, with no lighting calculations (so that all pixels of an object have the same false color), get the id number of the object under the mouse like this: var pixels = new Uint8Array(4) A non-power-of-two texture has restrictions; see Note that antialiasing being turned off is actually what one wants for mouse picking because you don't want an object with id of 10 that is next to an object of id 30 to get picked with an id of 20, which could happen with antialiasing turned on, due to the averaging of neighboring colors. |