Whitepapers
WHY A 128-FACED POLYHEDRON
Let's take a look at the principle of chromakeying using a three-dimensional model.
Suppose there exists a monitor that shows color information in a three-dimensional space. A pixel within a color video picture is represented by a tiny point located within the 3D space. Pixels with different colors are represented by points each located at a different position within the same 3D color space. Pixels with colors similar to each other are shown as adjoining points. The primary function of chromakey equipment is to produce masks that isolate the target object and the surrounding blue background where the target object is represented by white and the blue background by black.
In the conventional chromakey equipment, a mask is produced in the following manner:
- Input a blue background image.
- Set aside memory space to store the resulting masks.
- Place several partitions around the blue background portion(Figure 8).
- Pick out each pixel starting from the upper left corner and perform the following process:
- Plot the pixel picked out from the blue background picture as a point within the color space.
- Find out if the point is located within the partition, or outside of it.
- Assign black if it is within the partition, and white if it is outside.
- Store a pixel, black or white (as determined by step c., above), at a relative position in the memory space (Figure 9).
- Repeat the above procedure on all pixels.
- Output the resultant mask picture.
Figure: 8 Color information viewed from a certain angle (back)
Figure: 9 Color information viewed from a certain angle (back)
This technique was called a hard-edged chromakey. In the conventional chromakey procedure, a box like a matchbox as shown in Figure 10 is used.
Figure: 10 Color information viewed from a certain angle (back)
The hard-edged chromakey did not produce attractive composited pictures. The reason was that the technique forced all the pixels into the foreground object or the blue background, nothing in between. In the actual world, however, blue-backed pictures had colors subtly varying between blue and the foreground object colors, especially in the edge regions where human hair often appeared. If the pixels are forced to be either blue or target object, the details of the wisp of hair was either lost or showed up with blue-spill around the object in the finished picture, failing to render natural-looking results.
In the real world, hair has its own color and blue background has it's own blueness, and these two colors can very often co-exist in a single pixel at varying proportions. This produces a final color for the pixel representing some average of the two colors involved. In the 3D color monitor, such in-between colors are physically positioned somewhere between the target object color and the blue background color (see Figure 11). These pixels should not be forced into the target object color OR the blue background color, but should rather be handled as "colors between the target object color and blue" using a technique allowing 'fuzzyness', which is the basis of Primatte's "soft chromakey" technique.
Figure: 11 Color information viewed from a certain angle (back)
To process pixels that are not quite complete foreground or background, two different sets of partitions are used. The partition sets, one larger than the other, are placed as double matchboxes (Figure 12). Hereafter, they will be referred to as the larger matchbox and the smaller matchbox:
Figure: 12 Color information viewed from a certain angle (back)
With the double matchboxes, the separation process on a pixel is similar to the process as described in the above steps 4b through 4c, but some additional, more complex processing, must be done:
b-1 Assign black to the mask if the target pixel is within smaller matchbox.
b-2 Assign white to the mask if it is outside the larger matchbox.
b-3 Assign gray to the mask if the pixel is somewhere between the two
matchboxes. The gray should be given a brightness level in proportion to the pixels distance from the boxes: the closer to the smaller box, a darker gray, and the closer to the larger box, a lighter gray.
This process is repeated on all involved pixels from the original blue-background image. Depending on the brightness level of the resultant gray of the pixel, it is determined how much (percentage-wise) the original pixel belongs to the object (Figure 13). This double-box approach has given a great improvement to the chromakey process, and is the basis of the soft chromakey functionality in Primatte.
Figure: 13 Color information viewed from a certain angle (back)
In the soft chromakey process, the placement of the two partitions, larger and smaller, is the most important part of the technique. The target object should well be outside of the larger box, and the blue background should be well inside the smaller box, with all the other pixels in-between the two boxes, for the best possible result.
Figure: 14 Color information viewed from a certain angle (back)
Figure: 15 Color information viewed from a certain angle (back)
Figure: 16 Color information viewed from a certain angle (back)
Figure: 17 Color information viewed from a certain angle (back)
Figure: 18 Color information viewed from a certain angle (back)
Figure: 19 Color information viewed from a certain angle (back)
A sample of an actual blue background picture is shown in Figure 14. Note that the picture, as displayed on the 3D color monitor, shows itself as a triangular fan shape with its pivoting point on the blue background point. The structure of the dispersed points in the fan is shown in Figure 15. This clearly suggests that the partitions should ideally be placed as shown in Figure 16. As explained earlier, the conventional chromakey technique used partitions (matchboxes) with their edges parallel to the RGB axes, which made it impossible to place partitions in an ideal manner (Figure 17). The result of doing it that way was poor handling of in-between colors.
Other approaches have been tried, such as an 8-faced polyhedron or plane-slicing as shown in Figure 18. Though the plane-slicing technique seems to show a better fit, and is employed in the most popular system used today, satisfactory results are rarely obtained.
As a result of such analysis, the 128-faced polyhedron technique was determined to give the best results and was implemented in the proprietary technology used for Primatte. As shown in Figure 19, a partitioning set with many surfaces of different orientation can best be made to fit the varied color dispersion. This results in a better approximation of the complex shape generated by a real-world image. Each apex of the 128-faced polyhedron has a special relative position to each other allowing very fast generation of masks. In the implemented Primatte system, triple 128-faced polyhedrons are used to even better advantage to control blue spills. Primatte is the world's only chromakey system that employs the 128-faced polyhedron approach.
