More advanced uses of Texturing, Multi-Pass Rendering

  We want to discuss advanced uses of texture mapping, and discuss a few texture mapping techniques. The core concepts will be in the final exam, though. The most advanced sides have to do with shadows and reflections of moving objects. 

Using multiple textures is called decal texturing and this is a prominent element of many games. Other uses is the texture to depart the illusion of some embossing of objects.

We create an illusion and in order to fake differences in the reflection of light, we found changing the real geometry, which means pushing and pulling the surface. We use diffuse and specular reflections to be dependent on the surface normal.

A bump map is a texture that encodes the "height offset" of a surface relative to the baseline of an underlying flat model. 

A color is indicates how much a surface is raised off a plane. We want to create an estimation of directional normal vector, creating higher specular highlights, and create the illusion of raised surfaces and depressed surfaces. 

A normal map stores 2 components of the normal at each texture pixel location.  

Certain perturbations create a delusion of graininess by playing with the interaction of light and the geometry. 

The color is a mixture of underlying base color plus the white color, which is color of the light source.

Parallex mapping emulates even more realistic. The bricks have a sum of depths, and you cannot capture them by a simple normal or dot map. 

Something gives you the delusions of the spatial distribution. Normal mapping is different than parallax mapping, to give you the delusion of something having a substantial or high depth. There is a big difference, and you can see if there are any additional artifacts that parallex mapping try to introduce, so there are side effects.  

If you want to observe a surface from a grazing angle, then when you are rendering something from a camera point of reference, it "sees" what would be on a straight line between the eye and point A, where the straight line from the camera will hit the location of the fragment. I am rendering the black surface and emulate it as if it were the red surface. We lift the "plate" up perpendicularly at every individual location. Parallax mapping tries to compute an approximation of the height map of the first point to emboss, and then an offset to look at the underlying color texture, and this is what creates the additional depth. 

This is used to make the oceans shiny and the land more dull. 

 As always this is just a visual illusion. As I turn to the side, I cannot see the stuff raised but I can look at the scenes to create the illusion of embossing. 

Imagine somebody has mounted 6 projection screens and show some background, and it makes you think you are in some virtual environment, that has based the images in your surroundings in 6 panels, 4 surroundings, where there is 1 on the top and 1 on the bottom. 

We texture a cube with 1 image, and render it as if it's with everything else, and we want to emulate the process but with the assumption that the cube is infinitely far away from you. You are also given the option to query that cube. 

You can use a data structure called a cube map that allows you to query a coloration using solely a direction. 

The downside is there is no parallax, and the distances of the distances of motions cannot be shrunken down in perspective, especially with a sky box. Your motion cannot reveal that your entire world is planar. 

The only camera control that I have is pivoting around the single point. There is a big cube with 6 faces with a data structures to query the color.

The world is not necessarily animated yet. We cannot bump into an object, or walk in a tree, because that tree is presumed to be too far away (which means infinitely).

The next type is called environment mapping, and this is where we allow a reflective surface to be textured in a way to have a  colorful environment and see the textures back.

At every point in the object, I can go ahead and compute the direction. The cubemap texture provides the reflected color information on certain objects. We are trying to compute the I vector, and that indicates we can use a color as a specular reflection color, used to incorporate a lighting model?

We can map many background to create an illusion of billboard-style parallex, and then have a bumpmap that lets you select between different levels. 

The cubemap texture also provides reflected color information on objectsm (the same teacup shiny object and use specular reflection, render a triangle and find camera space vector).

Let's move along and see how environment maps are captured.

 The general idea is to have a reflective object. You see almost half the environment distorted and compressed.

We use an image to do a specular mapping and reflective mapping and texturing of an object as actually being reflected. This is an illustration of fantastic ingenuity.

Today sphere maps and storing the value of a surrounding texture was actually the way, today we are using cube maps because they are significantly more accelerate. Acquiring from spherical maps would often translate them back to cubic maps.

Projective textures essentially try to emulate the idea of having a light projector where we put a slight transparency held up to a light source and colored and casts a colored image on different objects of your environment.

The key idea is to place the camera in front of a projector and compute the coordinates of a coordinate system. For every point affected by light, compute the NDC coordinates and use them to retrieve a colored light. I can compute the world coordinates, then compute the normalized device coordinates . Place a camera at the location of the texture and compute the NDC coordinates of "projector system" and use them to retrieve a colored light.

There is multi-text rendering and texture, which can be multi-effect dynamic rendering. 

So far, we have used WebGL to draw the image of a world onto our screen. Actually write yourself to a special image on the side. 

The frame buffer is a memory that is a replica of what you see on a screen. We can subsequently store a memory image for future use, redirecting it into a texture, as a temporary store of a rendering operation.

 

 

We can use that image that a second visualization pass can read and use that rendering to detect a vertex shader. 

Instead of taking the output of a program, we go ahead and save it in a texture. We then have every one of the faces be a flat screen display that display what this window displays on its own. 

 We create a texture to go into the cube facet and then redraw the cube.

The most direct effect is called dynamic environment mapping.

There are multiple passes and we redirect them into a texture. 

Then I can stash to texture, all of the results of what a camera see, and use this newly updated texture as skybox texture.  

I would keep the original skybox and for the purposes of doing the reflection mapping I would use the updated units in order to store and replicate these effects. 

Illuminating light source would cast shadows of anything that is in the line of a particular object. The locations that are lit are visible from a source's viewpoint.

To shadow map, 

1. Place a camera at light's location.

Render and store depth map of the image. Create the image that has some value that corresponds to how far away the object is from the current display.

I can do

gl_FragColor = vec4(-0.2 * fPosition.zzz-0.8, 1.0);

This make points closer darker and points farther more illuminated, which is a depth map, an image using color in order to signify distances.

When rendering every point, Compare distance to the light and shadow map value, and fragment should be lit with its respective light color.

We can try to render a depth image in a cube map, and render a depth image in all possible other directions in order to create a depth reconstruction.

Additional issues are sampling and resolution of the shadow map, and it does show up as artifacts, and sometimes shadows do show up as pixelated since the shadow map is itself kind of a pixelized image to have some granularity to the shadow shapes themselves.

Another possibility is thin objects.

With  multiple light sources, this can be an issue as well . One alternative methodology used in modern games is to create outlines in the world that are being illuminated vs shadows. 

One question is "which of the appearance require render to texture"? Dynamic environment maps, bump rendering. Qualitative high-level problems, and we will look for stuff for practice exams.