At Colorcom, we have made great progress toward this Holy Grail. We have reached a milestone with the development of a raster to vector converter technology called IFR (Indirect Formularizing Resolution). This technology represents the solution to many critical problems currently presented by unintelligible graphic information.

In Figure A, we as humans, would see an outline of a diamond in the picture, but a computer would have a difficult time recognizing and explaining the picture.

As shown by the two images in Figures B (1&2), computers have a difficult time understanding and working with raster data.

For example, we can combine them together with an AND gate; (1 AND 1 = 1) and (1 AND 0 = 0). There are also OR and NOT gates. With these three basic logic tools, AND, OR and NOT, complex logical problems can be handled such as an adder, subtracter, or arrhythmic logic unit.  For this reason, computers are efficient at mathematics and can provide a wide array of mathematical tools.

One set of these tools is an array of algorithms that work on pictures held in vector or mathematical format.  In Figure C, the X is illustrated as a raster image on the left and a vector image on the right.

The reason the problem is so difficult is that there are so many combinations of patterns that raster data can take. For example, a 3 by 3 pixel picture matrix with nine colors has about 21,000 possibilities. A raster to vector converter has to convert each of these possibilities into an equation.

If just one more pixel is added, the number of possibilities increases exponentially. A picture can have millions of pixels, making the level of complexity very extreme.

As shown in Figure E, the vector data, in most instances, is limited to horizontal lines.  In a drawing comprised of gray lines on white paper, the horizontal vector line starts on the left side of the gray and extends to the right side of the gray.

Wavelets sift through the raster data to find each place where the image changes color. These color changes are called borders. Each part of a raster image that is comprised of the same color is called a blob. If a map of the United States depicted Texas as red and Louisiana as blue, each state would be a blob and their perimeters would be borders. Wavelets represent the borders as mathematical equations.

As shown in Figure F, wavelets would have no trouble with the north part of the Texas-Louisiana border because it is a straight line, but the Sabine River defines the rest of the border. Therefore, wavelets have to try to generate math that would define the Sabine River. After a few twists and turns, the math gets too complicated to be of much use.

Wavelet designers have tried to get around this problem by defining the river with small sections of independent equations, but this solution comes up short.

Fractal solutions are restricted because the library of shapes used is very limited. This is in comparison to all of the possible shapes required to accurately create a mathematical picture. Thus, as shown in Figure H, images created with fractals are fragmented and made up of non-related objects.

The Bezier method, as shown in Figure J, identifies the borders of each blob in the image.  These borders, though somewhat arbitrary, are located and tagged.  These tags then become the new representation of the image.  When the output image is rasterzied for display, Bezier curves are used to connect the tags when reconstructing the image in vector form.

Through IFR, raster data is translated into a more abstract data form. These data representations are then transformed into several other representations and then finally converted into vector form.

Since people perform raster to vector conversions quickly and easily in their heads, the challenge was how to learn from this process and reproduce the conversion. This translation could not be accomplished if humans had perfect recognition. IFR development is based on the fact that humans make mistakes in their perception. We have translated this understanding into a new and revolutionary technology called IFR.

From a philosophical viewpoint, if the technology works with visual perception, then it should work when applied to the other senses. Experimentation with IFR supports this philosophy. We believe that IFR is equally applicable with many digitized sensors such as sound, touch, humidity, acceleration, etc. The possibilities seem endless.

Looking at a complex raster picture where every dot is a different color from any of its neighbors, we would need to see similarities between the selected dots and their neighbors before we could decipher artifacts or images in the picture.

Even if a theoretical "super-human" could see all of the pixel to pixel color changes in a picture, he would need to be able to ignore the small color changes between pixels to understand or decipher the picture as well as mere mortal. Intelligence demands that the pixels be aggregated before artifacts can be extracted from a picture.

In order to fully understand the picture, the "super-human" would need to generalize similar colors and classify them by lumping them into categories. This process is the first step toward computers gaining intelligent understanding of the world around them.

Our internal system of generalization has much to do with why certain colors seem to clash with others. By looking at clashing colors, we can learn about how humans generalize and classify colors. The visual information in pictures gives us clues that allow us to lump together some adjacent pixels and make it easier to decipher the picture.

A strategy that is critical to the overall success of IFR is to keep machine cognizance relevant to human perception. Humans only see somewhere between 17 and 19 bits of color. The last 5 or 6 bits of a 24-bit color palette are wasted except for various interpolation schemes that increase resolution.

If a picture is perceived in a low noise environment such as scanning, there is not much fluctuation that exceeds 4 to 8 (i.e. 2 or 3 bits) color levels. Therefore, in the case of scanning, we can aggregate at perhaps 20 bits of color where we would be above the color noise and beyond human ability.  If a worst case analysis is done from a theoretical noise perspective, about eight color levels are needed to get around the noise.

As a test, we filmed video using a poor quality VHS camcorder. This was sampled at 4 times the color burst which added a great deal of noise. Over-sampling of clocked data can cause bias due to the violation of the Nyquist sampling theorem. There is at most one color per quadrant of the wave cycle. We found that the video noise peaked at about six color levels at 24-bit color. As a sanity check, we did the same test on a weak aerial TV signal. Theoretically, this noise would be less than that of the camcorder because of the poor color-burst phase lock on old VHS systems. Our suspicions were correct. The noise on this picture was 4 to 5 color levels.

Slew rate distortion (e.g., a slow video amp) can enter into this picture. There are cases where color trends, like an intended smear, can exist without adjacent pixels being the same; however, it is easy to distinguish between invalid slews and valid smears.

IFR includes algorithms that handle all of these concerns. They are implemented as simple state machine functions that do not require a math library. 

IFR eliminates the color problem with a color generalization algorithm that lumps adjacent color pixels together. In some ways this could be considered lossy; however, it should be considered intelligent. In the section entitled "Artifact Aggregation," we pointed out that a picture must have artifacts to have any visual intelligence. This does not occur unless adjacent pixels are lumped together to decipher the artifacts.

In the various video pictures that were tested, there were no smears to be found except for the intentional smear that was part of the video color pattern generator test, and this smear was not adjacent to another one.

Many pictures contain insignificant details that are not needed or desired. When IFR compression is taken into the lossy mode, aggregation can actually become an effective advantage. For example, a high-resolution scan of a 35mm photograph might show creases and wrinkles in the backdrop. These slight shade differences are easily eliminated when the picture is over-aggregated.

Demo 6:   The blob compressor is complete with the following charcteristics.

Demo 7:   The  auto-set feature of the color segmenter is
implemented.

Demo 8:   The glider is completed.

Demo 9:   Initiates product release to development engineering of audio streaming.

A.	Projected Completion Date: TBD
B.	Features: We can compress and audio stream about 12 times more than MP3 and with much better quality. Demo 9 also features speech recognition.

Demo 10:   Initiates product release to development engineering of vector based video streaming.

Below is the current roadmap for the technology. In all instances, software implementations will occur before hardware implementations. All compression number estimates are based on complex images, such as the speed-skater picture (Figure B,  approx. a 380 by 330 pixel image) seen previously in this white paper.

• Image compression is expected to be 50 to 60 times. 
• Image output will be offered in both raster and vector form. 
• Raster image output can be defined at any resolution. 
• The technology process is reversible. 
• There is no image data loss.

• Audio 
• Acceleration 
• Touch 
• Any sensor that can be transduced to electricity

• Video compression levels between 2,000 and 3,000 times are expected. 
• Video output can be defined at any resolution. 
• Intra-frame Compression

• Compression is expected to be 250 to 750 times per image. 
• This will be the first step toward static identification.

• The compression is anticipated to be approximately 20,000. 
• This will be the first step toward dynamic identification.

• We haven't attempted to solve this yet; however, we have a start toward the solution.

• Based on current technology studies, dynamic identification is a possibility for the future.

With this in mind, IFR will have a dramatic impact on many markets. We have identified over 150 market segments that can immediately benefit from IFR. These benefits can be broken into four categories:

TodayÆs most common DSP method is to sample a spectrum of the signal with a Fourier transform. Unfortunately, this provides only a fraction of the information available in the signal. Since IFR can convert raster data into a mathematical form, IFR will offer computers the ability to understand and be aware of their surroundings.
 

Almost all of the electronics industry will benefit strongly from one or more of these four primary IFR benefits.  When IFR is implemented to saturation, it will encompass about 20 billion applications. Below are just a few examples where the benefits of IFR will greatly enhance hardware (refer to categories previously mentioned).
 

Colorcom is now encouraging qualified technical developers to participate in a partnership to implement IFR into the marketplace.