A wavefront sensor for measuring a wavefront contains an array of lenslets, a detector array, and a mask having a temporally fixed pattern containing one or more opaque regions that are substantially opaque to light from the wavefront. The mask comprises one or more transmissive regions that are transmissive of light from the wavefront. The mask and the array of lenslets are disposed such that light from the wavefront that is transmitted by the transmissive regions is focused by onto the detector array by the array of lenslets. The mask is adapted to be selectably disposed to any one of a plurality of predetermined positions, wherein a different group of lenslets from the array focuses light from the wavefront onto the detector array depending on which of the plurality of predetermined positions is selected.

1. Field of the Inventions
The present disclosure is directed to a wavefront measuring device, and more specifically, to a Shack-Hartmann type wavefront sensor with a large dynamic range.
2. Description of the Related Art
The Shack-Hartmann technique is commonly used for determining wavefront shape or error from an ideal planar wavefront. The Shack-Hartmann wavefront sensor is a slope measurement device typically comprising a lenslet array, a two-dimensional detector array, acquisition hardware, and analysis software. Each lenslet in the array receives light from a portion of an incident wavefront. Light from the lenslet is focused within a "virtual" subaperture of the detector array, the detector subaperture generally being defined by those pixels disposed within a projection of the lenslet onto the detector array. The location of the focused light from a particular lenslet within each of these detector subapertures is used to determine the nominal slope of that portion of the incident wavefront. By calculating the slope of the incident wavefront from each spot displacement at each of the lenslets, the shape of the wavefront can be determined.
The dynamic range of a Shack-Hartmann wavefront sensor is typically based on the focal length of the lenslets and the dimensions of the detector subaperture, in units of pixel number, for each lenslet. In prior-art systems, the combination of lenslet focal length and detector subaperture dimensions usually limits the maximum wavefront slope that can be measured. If the slope of a wavefront at one or more of the lenslets exceeds such a predetermined limit, the focus spots from such lenslets move into the subaperture of another lenslet, resulting in one of the following problems: (1) multiple spots are created within a single subaperture, (2) multiple spots overlap within a single subaperture, and (3) spots switching between subapertures. For instance, if the wavefront slope in the area of a first lenslet in the array exceeds this maximum, the light received by the first lenslet produces a focus that is outside the bounds of a corresponding first detector subaperture and is instead received by in a second detector subaperture corresponding to a second lenslet in the array. The presence of the focus from the first lenslet in the second detector subaperture results in an ambiguity, since it cannot be determined, a priori, from which lenslet the focused light came.
Which of the three listed problems is produced depends on what happens with the focus spot from the second lenslet. If the wavefront slope at the second lenslet does not exceed the maximum limit, problems (1) or (2) can result. In the case of problem (1), it is indeterminate which spot belongs to which lenslet. In the case of problem (2), the focus of the second lenslet is indeterminate, since there is insufficient information to determine whether the second focus spot is located at that of another lenslet or the second focus spot is absent. If the wavefront slope at the second lenslet does exceed the maximum limit, problem (3) results. In this case an error can results since the focus spots will usually not be associated with the correct lenslet. These problems can exist between two lenslets or several lenslets.
One solution to increase the dynamic range is to decrease the focal length of lenslets in the lenslet array. The result of such a design choice is to increase the amount of wavefront slope needed to exceed the bounds of the corresponding detector subaperture. The drawback to this choice is that the sensitivity of the wavefront sensor is decreased proportionately if all other system parameters remain the same as they were in the longer focal length lenslet design.
Another method of increasing the dynamic range is suggested in an article by Lindlein, et. al. (see "Algorithm for expanding the dynamic range of a Shack-Hartmann sensor by using a spatial light modulator array," Optical Engineering, 40(5) 837-840 (May 2001), the entirety of which is hereby incorporated by reference). Lindlein et. al. disclose the use of a spatial light modulator (SLM) to create a sequence of switching patterns that mask differing sets of lenslets in the lenslet array of a Shack-Hartmann sensor. Use of the switching patterns removes the requirement that each lenslet focus light within a detector subaperture. Using the method disclosed by Lindlein et. al., the focus spots formed by light from each lenslet may be located anywhere on the detector, with the exception that "spots are not allowed to overlap." The authors calculate the minimum number of switching patterns necessary to provide an unambiguous correlation between wavefront slopes and the focus spot locations on a sensor array.
The authors also provide an algorithm for determining which lenslet array subapertures are "switched off" in each switching pattern. For instance, an array of 40 lenslets by 40 lenslets would require nine different switching patterns. Each switching pattern has a form that is different from the other. The Lindlein et. al. method preclude taking a fixed switching pattern and simply moving the pattern to a different coordinate at each step in the sequence.
A need exist, therefore, for providing a simple device and method for resolving ambiguities produced in Shack-Hartmann type wavefront sensor that are created by large wavefront slopes, thus increasing the dynamic range of such wavefront sensors.

Large dynamic range Shack-Hartmann wavefront sensor


A digital compression apparatus and method uses a first compression encoding step associated with a first encode circuit to produce a first statistical data set from a first compression encoding of an initial digital video bit stream representing a group of pictures (GOP). The initial bit stream of the GOP is also stored in a first memory, while the first compression encoding is performed. Concurrently with, or following the first compression of the GOP data by the first encoder, a companion processor uses the first statistical data set and optional additional information to compute a set of filter and encoder control settings. The initial digital video bit stream of the GOP data previously stored in the first memory is retrieved from the memory after the first encoding step and is input as a time delayed version of the initial digital video bit stream through a set of filters and subsequently through a second compression encode circuit to produce a second compressed video bit stream of the GOP data. The filters are configured to provide dynamically variable filter characteristics that are responsive to the filter control settings from the companion processor. The filter characteristics can be changed optionally by the companion processor on a block, frame or GOP basis. The second encode circuit also has encoder characteristics responsive to the encoder control settings. The filter and encoder control settings selected by the companion enable the second encoder to provide enhanced compression encoding performance second compression encoding of the GOP data video bit stream relative to the compression of the first encode circuit. In particular, the second compressed bit stream may have improved performance in either or both bit rate and total bits relative to the compression of the first encode circuit.

1. Field of the Invention
The present invention relates to a method and system of compressing or compacting digital data. More specifically, the invention pertains to an apparatus and method of maximizing the quality and efficiency of transferring compressed digital information.
The present invention improves various aspects of video data compression by dynamically (i.e., real time) applying selected filtering and encoding parameters during a second, delayed encoding of a stored representation of an input digital data structure, e.g., a bit-stream, based on the results of a first encoding of the input data structure (bit-stream).
2. Background Art
Transmission and/or storage of digital data are employed in nearly all data processing and transmission applications. Large scale archiving and retrieval of documents may require very high-capacity storage devices and media. Also, transfer of massive quantities of data over long distance communications facilities is rapidly increasing. Digital storage or communication systems are designed with a capacity limit for total bits or maximum bit rate, typically termed the 11 bit budget 11. In order to reduce the costs associated with the transfer and/or storage of massive amounts of data, the bit budget, i.e. the bit rate transmitted through a transmission channel of fixed capacity and/or the total number of bits to be stored in a storage media, is minimized as much as possible. In the field of digital video transmission or digital video storage and retrieval, great attention has been directed to data compression and compaction to reduce costs and improve performance under bit budget constraints.
Data compression may be classified as reversible or irreversible, where reversible means there is no loss of information, as opposed to irreversible, where there is some loss of non-relevant information (where the non-relevance depends on the given context). Irreversible compression is generally called compaction although there are no standard definitions. In the context of video transmission and storage, the term compression is often be used loosely with either sense. For the purposes of this application, compression means either reversible or irreversible, unless otherwise specifically stated.
A basic reference on data compression is found in Encyclopedia of Computer Science, Third Edition, Anthony Ralson, et al, Van Nostrand Reinhold, New York, 1993, page 396. The issues of redundancy, security, portability and various compression techniques are briefly described therein.
Digital data compression plays an important role in video storage and transmission. The same techniques may also used in other fields, e.g. field of audio signal transmission and storage, sonar signal transmission and processing, geophysical, weather data recording and transmission and the like. Since the bandwidth requirements of video are generally many times greater than that of audio and the others, more attention has been given to compressing digital video data. However, the same techniques may be applied to compression of audio data as a subset of a program containing both audio and video content.
A discussion of video signal compression encoding, as established by the Moving Picture Experts Group (MPEG) is shown in the PCT patent WO 96/36182 by Maturi et al, (Maturi) incorporated herein by reference. Like the JPEG still image compression standard, MPEG is a multistage algorithm built around the discrete cosine transform (DCT). The two algorithms also share similar approaches to color space conversion, blocking, quantization, zigzag ordering, run length tokenizing, and Huffman encoding. MPEG goes further, adding interframe compression and interleaved audio. As with most interframe compression schemes, MPEG is asymmetric--compression requires more effort than decompression.
With reference to FIG. 1, there is shown a simplified block diagram of a portion of a typical MPEG-2 encoding process 10. An incoming uncompressed digital data stream, e.g. video stream 12 is input to Discrete Cosine Transform (DCT) encoder 14, to produce DCT coefficients which are quantized by quantizing means 16. The DCT coefficients are then arranged in zigzag order by zigzag scanning means 18. The ordered DCT coefficients are further encoded (variable length (VLC) and token or Huffman encoding), e.g. encoded by VLC 20. Simultaneously, the uncompressed video stream 12 is being processed in a motion detection and motion vector generation (MVG) means 22 by comparison to previously stored (not shown) digitized video data.
Generally the MVG 22 performs motion estimation (ME) and exchanges ME data 24 with the VLC encoding process 20 and a picture type determining means 26. A final MPEG-2 coded video stream 30 is output for recording or transmission.
Maturi discusses MPEG encoding in general, and describes one example of an MPEG encoder using an encoder decision block (EDB) for compressing/decompressing in the MPEG-II protocol. See in particular FIG. 2 of Maturi reproduced here as FIG. 3, and the description from page 1, line 30 to page 5, line 21 of Maturi.
An exemplary MPEG-2 encoder 200 after Maturi is shown generally in FIG. 2. A Video Interface (VI) 202 receives an incoming digitized video signal (DV) 204. The VI 202 electrically communicates directly with a Motion Estimator (ME) 206, an encoder memory controller (MDC) 207, and is electrically connected to a bus 208. The ME 206 is also connected in direct electrical communication with the MDC 207, and bus 208.
The MDC 207 is also in direct communication with an Encoder Memory (ENC/MEM) that may be, for example, part of a larger Main DRAM (MDRAM) memory 226 (not part of the MPEG-2 encoder 200), the bus 208 and an encode pipe (ENC) 212. An Encoding Decision Block (EDB) 210 is in direct electrical communication with the bus 208, the ENC 212 and a Video System Stream Multiplexor (VSM) 214. The ENC 212 also communicates directly with the bus 208 as does the VSM 214. An Output Stream Interface (OSI) 216 is electrically connected directly to a Host Interface (HI) 218, the bus 208 and outputs an MPEG-2 encoded Output Video Stream (OV) 230.
An Audio Interface (AUD) 220 accepts a Compressed Audio digital stream (CA) 228 and is directly connected to the HI 218 and bus 208. A Host Computer (HC) 222 (not part of the MPEG-2 encoder) is directly connected to the HI 218 which provides communication to a Host DRAM (HD) 224 not part of the MPEG-2 encoder, i.e. a companion memory for storing data and programs used in conjunction with the encoder 200. The HI 218 also connects directly to the bus 208 for communication with all the elements similarly connected to the bus 208.
The EDB 210 retrieves digital video data for a macro block (explained further below) from the encoder memory 226 together with a corresponding motion vector (not shown) from the host DRAM 224, and processes the macro block video data to determine various encoding conditions the encoder 200 will apply in encoding the particular macro block. For example, the EDB 210 decides whether a macro block is to be intra-frame (P) or inter-frame (I) encoded, as well as determining the quantization for a particular macro block. The EDB 210 also causes the motion vector to specify a DCT type of translation field for a macro block, as well as selecting an encoding for the macro block. The encoding selection should produce a pre-specified bit rate at which data will be fed through the remaining components of the encoder chip 200.
The Encode Pipe (ENC) 212 is connected to receive the output from the EDB 210, and actually encodes the macro blocks. The encoding may be performed in accordance with encoding conditions determined by the EDB 210, or alternatively, may be performed in accordance with encoding conditions that are supplied to the encoder chip through the HI 218. The ENC 212 computes the DCT of a block, quantizes the coefficients and performs run length (RLL) and variable length coding (VLC and/or Huffman).
The EDB 210 compresses the digitized video by subdividing successive macro blocks of digital video data into 8.times.8 blocks and processing the blocks.
The AUD 220 is provided to compress digital audio data in accordance with the MPEG-2 protocol, and transmit compressed audio data to the host DRAM 224. The other system components, e.g. the VSM 214, the OSI 216 provide for converting from parallel to serial bit stream and the like. A source of compressed audio data 228 is connected to the audio interface 220 for inclusion in a system output data stream 230 from the output stream interface 216.
Maturi also discusses a motion estimator used for determining the direction and amount of motion between associated macro blocks belonging to related P or B frames. Once a motion vector is determined it is stored in the host DRAM 224 via a host interface. Motion vectors may be used in later calculations, thereby saving microprocessor execution cycles.
FIG. 3 depicts an encoded video image, for example, in the MPEG-2 format. A succession of video frames (302 . . . 306), is comprised of a group of pictures, GOP(J) 300, organized by a coding algorithm into a digitized video pixel serial bit stream, pJ[i]. A typical coding protocol, e.g. the (4:2:0) coding protocol is shown in FIG. 3. The GOP(J)300 is composed of a first frame 302 to a last frame 306, separated by a plurality of frames 304. There are generally three different encoding formats, which may be applied to video data. The frames 304 are generally composed of Intra (I), Predicted (P) and Bi-directional (B) frames.
The format of the GOP structure may vary. The ISO/TEC 13818-2 video specification indicates GOPs can contain one or more pictures or frames. Two typical formats are 12 frames/GOP, derived from film, which is 24 frames per second (fps), and 15 frames/GOP, which is more suitable for video at 30 fps. Other GOP/frame formats may be encompassed by video compression systems, e.g., PAL at 24 fps. Essentially, the choice of the number of frames (Nf) or pictures to be included as the GOP, is independent of the frame rate (fps) of the media from which the video data is derived.
Frames 304-306 are divided into slices, e.g.,slice 307. Slices 307 are divided into Macroblocks 308 that are further divided into Luminance blocks 310 (Y1-Y4) and Chrominance blocks 312 (Cb, Cr). Each of the blocks 310, 312 are comprised of blocks 314 organized from pixels (PJ[i]) 320.
The Intra or I frame coding produces an I block, designating a block of data where the encoding relies solely on information within a video frame where the macro block 308 of data is located, Inter-frame coding may produce either a P block or a B block. A P block designates a block of data where the encoding relies on a prediction based upon blocks of information found in a prior video frame. A B block is a block of data where the encoding relies on a prediction based upon clocks of data from surrounding video frames, i.e., a prior I or P frame and/or a subsequent P frame of video data. One means used to eliminate frame-to-frame redundancy is to estimate the displacement of moving objects in the video images, and encode motion vectors representing such motion from frame to frame.
A GOP will necessarily start with an I frame and continue through a succession of multiple B and P frames 304 until the last frame 306, also generally an I frame. The frames are further subdivided into slices 307 representing, for example, a plurality of image lines. Each slice 307 is composed of macro blocks 308. Each macro block 308 is a block array representing the luminance (intensity) 4 blocks 310 of pixel luminance (Y1-Y4) and two blocks 312 representing interpolated pixel chrominance.
The 4:2:0 macro blocks 308 are further divided into 4 luminance blocks 310 (Y1, Y2, Y3, and Y4) and 2 chrominance blocks 312(Cr, Cb). Each block 310, 312 consists of an 8.times.8 array of bytes, each luminance byte being the intensity of the corresponding pixel, and each chrominance byte being the chrominance intensity interpolated from the chrominance intensity of four adjacent pixels. The MPEG-2 protocol encodes luminance and chrominance data and then combines the encoded video data into a MPEG-2 compressed video bit stream.
The meaning of the video bit stream nomenclature has some ambiguity in the art. For the purposes of this discussion, video bit stream means both `raw` bit streams, i.e. uncompressed (may be encoded or un-encoded) video bit streams, and compressed (may be encoded prior to or after compression) video bit streams (compressed by the methods and apparatus of this invention). In this discussion the terms input and output will indicate which of the two bit streams are concerned. "Input" means uncompressed video bit stream; "output" means a compressed video bit stream.
In addition, an input video bit stream may have previously been compressed by some other means than that of the current discussion. In that case, the term, input, means the video bit stream provided to the particular device or method being considered, prior to the compression of the device or method.
After the video data is encoded it is then compressed, buffered, modulated and finally transmitted to a decoder (not shown) in accordance with the MPEG-2 protocol. The MPEG-2 protocol typically includes a plurality of layers each with respective header information. Nominally each header includes a start code, data related to the respective layer and provisions for adding header information, e.g., see FIG. 5.
With reference to FIG. 3, there is shown a typical arrangement of video data, that comprises a portion, GOP(J), of digital data compressed in the example above. A first portion and second portion of video data stream 102 will typically correspond to at least a first group of pictures (GOP(J)) and second group of pictures (GOP(J+1) not shown) one of which is shown in FIG. 3. The GOPs 300 are composed of a succession of digitized video frames 302-306. The frames 302-306 are of three types: an I frame, a P frame and a B frame.
An MPEG-2 I frame contains all the digital data needed to decode and reconstruct the uncompressed picture for that frame within its own data set without reference to any other data. The MPEG-2 P frames are digital data sets which can be decoded to recreate an entire uncompressed frame of video data by reference only to a prior decoded I frame or by reference to a prior decoded P frame.
The MPEG-2 B frames are digital data sets which may be decoded to recreate an entire uncompressed frame of video data in three ways. B-frames are recreated by reference to only a prior reference frame, to only a subsequent reference frame, or to both a prior and to a subsequent reference frame, (by reference to a future or previous decoded I, P frame or frames).
Each frame 304 is divided into a sequence of slices 306. The slices 306 are further divided into a succession of macro blocks 308. The macro blocks 308 typically separate the luminance and chrominance data into a 4:2:0 encoding scheme. This contains the pixel luminance data (Y data) in four 8.times.8 blocks 310 Y1, Y2, Y3, Y4, and the pixel chrominance data (Cr, Cb) in two 8.times.8 blocks 312 which cover the same area, but which are sub-sampled at half the spatial frequency.
In the typical 4:2:0 sampling scheme for digitized video encoding, each macro block 308 thus consists of a 16 by 16 pixel (or pel) array of digitized luminance video data and two 8.times.8 pixel arrays 312 of digitized chrominance data. If the current frame is to be an I frame, which does not depend on any other frame, the encoding process proceeds to the next step. If the current frame is to be a P frame, however, interframe correlation must be performed first.
For each macro block a reference frame is sought, seeking the best match. Upon finding an exact or near-exact match, only a pointer to the matching pixels must be encoded. This pointer is called a motion vector and typically only requires a few bits of storage space. The search is performed by the motion estimator 206. Often a perfect match is unavailable because objects don't simply float across the screen, but may also rotate, fade, change shape and move toward or away from the viewer. In this case, the search may still provide a partial match from which there may be computed the difference between current and reference pixels. The resulting difference pixels are often highly correlated, and therefore, are amenable to compression.
Blocks coded differentially require more storage than perfectly matched blocks, but still save coded bytes over intra coding. In the event of search results so poor that even differential coding is not practical, the data for that macro block is simply coded as intra-type.
For B frames, a similar process is used, but both previous and future frames are searched for the reference pixels. Having two separate reference frames available yields higher correlation and thereby higher compression.
The motion estimator 206 determines the direction and amount of motion between macro blocks 310 belonging to different frames of the video data being encoded.
After interframe correlation, differentially encoded P and B blocks and those coded as intra-type are fed into a DCT which maps each 8.times.8 block of pixels into 64 frequency coefficients. Each coefficient represents a weighting factor for a corresponding cosine curve.
The 64 basis cosine curves vary in frequency: low frequencies describe the block's coarse structure, while high frequencies fill in the detail. Adding the 64 weighted basis curves together will reproduce the original 64 pixels. By itself, the encoding of the DCT provides no compression. But the lack of extreme detail in most image blocks means high-frequency coefficients are typically zero or near zero.
To increase the number of zero frequency coefficients and reduce the number of bits needed for nonzero frequencies, each coefficient is divided by a quantizer value. Because the eye is less sensitive to errors in high-frequency coefficients, the quantizer values tend to increase with frequency. MPEG uses one quantizer table for intra-macro blocks and another for non-intra macro blocks. The encoder chip can use the default quantizer tables or use customized quantizer tables.
Quantizing causes a loss of image content. Large quantizer values cause more loss (i.e. greater image degradation, lower quality) but also deliver higher compression. This effect can be used to hold the output stream 230 data bit rate to a desired constant value, e.g. to not exceed a channel capacity constraint. On the other hand, if a frame uses more bits than allocated, the quantizer value is adjusted until the bit count falls below a preestablished maximum. MPEG normally provides a quantizer scale parameter, which can be adjusted once per macro block, expressly for this purpose.
The quantized frequency coefficients are zigzag ordered. Zigzag ordering produces long zero runs suited to run-length encoding. In run-length encoding, each run of zeros is expressed as a data token describing the number of zero value frequency coefficients in the run and the value of the nonzero frequency coefficient that ends it. These tokens are further compressed through Huffman coding, which converts each token into a variable length code (VLC). VLCs for more common tokens are 2-3 bits long, whereas VLCs for rare tokens are up to 28 bits long. The final bit stream, consisting mainly of very short codes, is roughly one-third the size of the run token stream.
As the data stream is compressed, MPEG-defined data header packets may be inserted to assist the downstream decoder. Each header begins with a unique 32-bit start code. A sequence header may define frame rate, and other characteristics. A group of picture headers may be inserted before I frames to indicate random access entry points. Picture headers may be inserted to identify each frame and communicate frame-specific information, such as I, P, or B frame type. Slice codes may be inserted at various locations in the frame to give downstream decoders a chance to be re-synchronized if an error condition, such as a corrupted bit stream, is detected. The compressed data combined with the headers make up a fully defined MPEG-2 output video data stream 230.
The audio interface 220 uses the MPEG audio compression algorithm with sub-band filters to divide the audio signal into frequency bins. Fewer bits may be allocated to the less-audible bins, to achieve compression of the audio data. MPEG can typically store CD-quality audio at compression ratios of up to 8:1 or more. Audio compression as high as 20:1 can be achieved with additional audio degradation if desired.
As a final step, the compressed audio bit streams from the audio interface 220 and the compressed video Bit stream from the video system Multiplexor 214 are packetized and combined in the output stream interface 216 to output the final compressed MPEG system stream 230. Time stamps may be inserted to help the downstream decoder separate and synchronize the audio and video bit streams.
Decompression essentially reverses the process described for compression, but requires fewer computations, as there is no need for the pixel searches (i.e. motion vectors).
The encoding decision block 210 performs macro block 310 intra/inter/Quantizing decisions, makes field and frame decisions, and performs rate control, half-pel motion estimation and video buffer verifier (VBV) calculations. The encode pipe 212 encodes macro blocks 310 comprising the digitized video data 204. Based either on results from the encoding decision block 210, or based upon parameters specified via the host interface 218, the encode pipe 212 accepts and processes the digitized video data 204 together with data produced by the encoding decision block 210.
While the motion estimator 206 determines motion vectors and the encoding decision block 210 selects an encoding mode by processing 16.times.16 macro blocks 310 (the macro blocks 310 being composed of four luminance blocks 311 and 2 chrominance blocks 312), the encode pipe 212 transforms the digitized video data and quantizes the transformed video data by processing 8.times.8 blocks 314. Each 8.times.8 block 314 is transformed by computing DCT coefficients, then the DCT coefficients thus computed are quantized to further reduce the amount of data.
If an interframe macro block cannot be adequately encoded simply by motion vectors alone, a DCT is computed of the difference between it's reference frame(s) or field(s) and the macro block's video data, and the DCT coefficients obtained for this difference macro block are quantized. If P frame(s) provide the difference for an inter macro block, then the DCT is computed of the difference between the macro block's video data and the decoded P frame data to obtain the maximum accuracy in the macro block's encoding. The transformed, quantized data thus obtained are then coded into an MPEG-2 video stream using variable length coding (VLC or Huffman). The encode pipe 212 also performs the inverse of these operations simultaneously on P frame encoded data.
The final encoded output video bit stream 230 is output to a transmission or recording media, e.g. satellite, cable, DVD ROM, and the like.
With previous digital data compression systems for real-time compression of video data, digital data is compressed using a variety of mathematical algorithms. The algorithms are processed with hardwired circuits and/or programmable micro-or multi-processing components comprising the encoder 200 connected to memory circuits, typically RAM, ROM and disk. In the design and use of such systems, choices must be made regarding the quality level of the encode video output 230, derived from the compressed data, which will be accepted and the size of the communication channels and/or media on which and through which the compressed digital data 230 will be stored or sent.
For previous systems, this generally demands choosing the maximum available quality level or minimum acceptable quality level of the decoded representation of the compressed data as a constant, limited by the algorithms employed and the fixed compression parameters selected for the particular hardware involved. Alternatively, the maximum encoded bit rate allowable can be chosen, and compression parameter selection strategy fixed so that the quality level will usually be acceptable. This can often lead to the unfortunate consequence that portions of fast moving video scenes are grossly distorted, e.g. gross quantization causing objects to be displayed as unrecognizable blocks (the infamous cubical football). This can occur when the fixed parameters chosen for the selected filter and compression strategy can not meet the hard system limits, maximum bit rate or maximum bit capacity.
A typical limiting case strategy is to program the EDB to do gross quantization of problematic macro blocks, i.e. transmit only DC DCT coefficients for a block or blocks, to stay within the hard limits. Another strategy in non-real time applications, is to do manual post processing, i.e. human intervention by selecting and hand editing scenes.
Neither of these conditions are desirable, in terms of quality or cost. Compression systems that inherently are unable to provide controllable quality levels throughout a video program, having widely varying activity levels will not be able to guarantee acceptably uniform quality in real time. Second, video program material having widely varying video activity generally leads to increased cost (where post processing is necessary, or in number of DVD layers required for long program material). Third, distribution of the compressed video signal through rigidly specified outlets, i.e. through well known and completely controlled receiving hardware/sets may be limited by the bit rate variability of the compression system. Consumers, and consequently the advertisers who must pay for the received signal, may be less than sanguine about receiving programs whose quality level may fluctuate significantly, depending on program content.
In real time systems, i.e. cable and satellite TV, the instantaneous output encoded bit rate can exceed channel capacity, unless a) the overall quality of the decoded video bit stream is 10 lowered by choosing a compression parameter strategy to avoid the possibility of gross quantization during particularly problematic scenes, or b) blocky artifacts may be created during fast motion scene change or the like; in any event sound or picture quality must be degraded to a lower level than desired over portions or all of an entire program.
Typical prior art compression schemes use what is known as panic mode quantization, i.e. transmitting only DC coefficients when an unexpected scene change, scene fade or picture components with very fast motion would otherwise create unacceptable encoded bit rates.
In some prior art MPEG-2 encoder implementations, the encoder is designed to attempt to address these problems by performing measures of encoded quality and bit rate during the encoding of the frames of a GOP.
With reference to FIGS. 2 through 5 one typical prior art MPEG-2 encoding scheme will now be described. Referring to FIG. 4 and FIG. 2, a video buffer 400 may be incorporated in the encode pipe 212. One implementation of a video buffer is composed of two FIFOs, FIFO A 402, and FIFO B 404. An intermediate portion of the encoded video bit stream 420 (processed internally by ENC 212) is switched between an input 412 of FIFO A 402 and an input 414 of FIFO B 404 by input switch S1.
While the input switch S1 is connected to FIFO input 412, an output switch S2 connects an output 418 so that a segment (not shown) of encoded video bit stream data previously stored in FIFO B 404 is output as a corresponding segment of output video bit stream 422. The encoded input stream 420 remains connected to input 412 until a FIFO A ALMOST FULL signal 408 causes the switch S1 to switch the video bit stream input 420 to the FIFO B input 414. Signal 408 also causes the switch S2 to switch the video bit stream 422 to the FIFO A output 416.
The segment of video bit stream data stored in FIFO A 402 while S1 was connected to input 412 is connected by switch S2 through output 416 to the output encoded bit stream 422. Conversely, the switch S1 remains connected to FIFO B 404 until a FIFO B ALMOST FULL signal 410 causes switch S1 and S2 to toggle again. The segment of video bit stream data that has been stored in FIFO B 404 is then transferred out as the next segment of encoded output video bit stream data 422.
This type of video buffer is termed a .quadrature.see-saw.quadrature. buffer since FIFO A and FIFO B alternately store incoming video bit stream data and transmit outgoing video bit stream data while keeping the encoded output bit stream 422 continuous and independent of the bit rate of encoded input bit stream 420. The bit rate of the input encoded video data 420 will in general be highly dependent on the content and complexity of the video input.
The sawtooth graph 430 of FIG. 4 represents the alternate filling and emptying of the two FIFOs 402, 404 from minimum (empty) level 434 to almost full level 436. The encoded bit stream 422 will have an instantaneous bit rate 440 represented by the slope of the saw tooth 430. For highly complex video scenes, e.g. fast motion, fades and chaotic fire and water pictures, the encoded bit rate will be high, i.e. steep slope 440. Conversely, static or scenes with a deficit of detail will have low bit rate, i.e. approaching zero slope.
The changing instantaneous bit rate 440 combined with the size of the FIFOs 402, 404 will result in a variable period 438 between the toggling of the switches S1 and S2. This variable period 438 is typically used by the EDB 210 as a statistical measure, viz. Vbv, in an algorithm (not shown) to change encoding conditions by the ENC 212 or to change filtering conditions (not shown) by the VI 202 as is well known in the art.
The digitized video bit stream 204 of typical prior art MPEG-2 encoder is processed in real time by the encoder 200. Decisions made by the prior art EDB 210 comes from a portion of the video data which has already been sent through the ENC 212 to the VSM 214 and subsequently to and through the OSI 216. Consequently, the effect of a prior art EDB decision takes place on input video data which may have markedly different characteristics from that which preceded it and which generated the statistical measure used for the EDB decision.
This has been particularly true for scene changes and fades that typically may cause drastic quantization, known as .quadrature.Blocky.quadrature. artifacts. Blocky artifacts are typical in previous satellite systems in prior art encoded video since a .quadrature.panic mode.quadrature. strategy must generally be resorted to in order to keep the bit rate of the output encoded video 230 below a maximum channel capacity limit.
In prior art off-line compression systems, an operator has the opportunity to view problematic scenes and by iterative procedures choose a compromise between number of bits and quality level of individual scenes to stay within the constraint of fixed media size, i.e. there are only so many bits into which a movie may be encoded. A lengthy video feature with a high degree of action may require many scenes to have lower video quality than is otherwise desired in order to keep the movie on one reel of tape or one layer of one CD-ROM/DVD or the like.
In either previous art real-time or off-line digital video data compression systems, there is a high labor cost for trading off picture quality and digital data bit rate and/or required bit capacity.
It would be an advantage to selectively and dynamically modify compression of digital video data controllably and continuously within given bit rate and quality constraints. The quality level and/or instantaneous bit rate during short periods of time can then be managed within pre-set limits while maintaining higher quality over the majority of the program. Alternatively, by limiting the instantaneous transmitted bit rate to stay within channel capacity limits while simultaneously managing the quality level of the subsequently decoded compressed video to a predetermined level.
A consumer and the advertiser paying for the program could be assured that the video quality of the compressed program material would have a consistent look throughout the entire program, irrespective of the type of program material. The vendor providing the compression could confidently guarantee the quality level would never degrade below a minimum and would have a high (and guaranteed) quality level for a maximum (and large) percentage of the program, and at the same time, never exceed a channel bit rate limit or a media bit capacity limit.
In previous digital video compression systems, difficulties can also arise when program content changes from film input (frame mode) to video (field mode). This commonly occurs when watching a film on TV, which then breaks to a commercial. The film was digitized at 24 frames per second (fps) and the commercial at 30 fps. This can create a huge error budget instantly, and causes severe picture quality degradation. This cannot be predicted since the programming station may do so at any time. If this break occurs within a GOP, which it typically does, at least one frame may have an unacceptably large bit rate or bit size.
Another area of concern arises during scene changes, especially fades, where all the pixels of each frame may change intensity and/or color over an extended number of GOPS.
It would be an advantage to have a compression method and apparatus, which could smoothly respond to field/frame changes and fast motion while still maintaining a high degree of picture quality.
Another item of concern in digital compression of video images is the unsteadiness of the source material. During the conversion of any one frame of the source material image (which is represented by a 2-dimensional analog of color and intensity; i.e. film, or a video frame) to another representation (in this case a digital signal) the position of the image of any one frame or picture relative to a device which is converting the image from analog to digital, may change. Not only the image as a whole may shift, but the position or shape of edges of the image may vary relative to the image as a whole.
Film that is fed to a telecine during digital video capture may wobble back and forth in a kind of sinusoidal pattern as it passes through the telecine film gate. It typically .quadrature.walks.quadrature. back and forth, at some cadence. The rate of the cadence is dependent on the tension of the film, and many other factors. This is sometimes seen at a movie theater, where the image drifts laterally and vertically across the screen.
Due to wear in the mechanical system of a projector or telecine, the film can move up and down or back and forth. Vertically and horizontally the frame can move +/-4 pixels or more from a reference center. This wobble or walking of the frame causes unnecessary usage of encoding bits for an encoding algorithm, viz., MPEG, because the changes from frame to frame are interpreted as though the entire scene has changed, whereas the scene has just moved side to side or up and down. If the entire image can be moved back to where it should be, then the MPEG encoding doesn't produce unnecessary encoding bits. Global motion from other sources in video camera scenes, for example, unsteady hand held video cameras may also be compensated by the MSF.
Even though the film image for each frame is stationary with respect to its components, and changes otherwise from frame to frame may be minor or non-existent, prior art digitizing schemes can cause digital compression encoders to generate unnecessary bits in encoding such "walking" scenes because they are unable to distinguish the frame to frame wandering or walking of the whole image from complete scene changes. Digital conversion of such unsteady or wandering, scanned images can grossly increase the number of bits required to encode multiple frames of sequential images, even though the image has not changed significantly from frame to frame. It would be an advantage to provide a method and apparatus to reduce the creation of unnecessary encoded bits caused by walking film during digitizing.
Attempts have been made to account for image unsteadiness, or walking. Lingemann, in U.S. Pat. No. 4,994,918 describes a method and circuit for detection and correction of horizontal and vertical errors in image unsteadiness during television scanning of moving pictures. Lingemann discloses a tachometer roll engaging the perforations of the film. Electronic circuitry provides pulse generation and counting in combination with horizontal and vertical image steadiness signals to control the beginning an ending of the scanning period. This requires a tachometer engaged with the film as part of the equipment. This method and circuitry will not accommodate images from video sources derived from film previously converted to video, nor does it address video sources which themselves have unsteady images, e.g. hand held cameras.
In Weiss et al., U.S. Pat. No. 5,510,834 (Weiss), there is disclosed a method of estimation of global error motion vectors which represent unwanted global picture instabilities in a picture sequence in digital video signals. Weiss describes global motion as a combination of true global motion (such as pan and zoom) and unwanted global picture instabilities from, e.g. worn-out film sprocket holes, poorly performing telecine, film stretch, or unsteady camera shots. Unwanted global motion may be a single global error motion vector for each picture or may be a global error motion vector field for the entire picture.
Known methods for estimating global error motion vectors involve using known motion vector estimation techniques such as, e.g., pixel gradient recursion, phase correlation, and block matching (see, e.g., PCT/SE92/00219), combined with spatial and temporal processing of the estimated motion vectors.
Weiss uses a motion vector estimator with an adaptively variable measuring time distance, as well as spatial processing and temporal processing, in order to estimate a sequence of global motion vectors, from which a sequence of global error motion vectors is separated in order to allow stabilization of a picture sequence. Weiss uses a local motion vector estimator with a measuring time distance adapted to a maximum motion vector length and motion vector frequency, combined with spatial processing, and temporal processing. Weiss makes an estimate of a sequence of global error motion vectors based on an estimated sequence of the global motion vectors associated with pictures in the picture sequence.
Weiss method addresses the difficulty in known motion vector estimators to make accurate estimation of both large and small motion vectors, due to the fact that known motion vector estimators use a fixed measuring time distance. In the general case, fixing the time distance will be disadvantageous for either large or small motion vectors depending on which is chosen and the motion vector frequency.
One disadvantage of Weiss method is the necessity for an operator to place flags at selected points, which establish the image portion, which is judged to be stable. That is, active human judgement is required to set a basic image reference point or points from which motion corrections will be made.

Dual encoding/compression method and system for picture quality/data density enhancement


A memory cell (400) used to store data in an integrated circuit. The memory cell (400) is static, nonvolatile, and programmable. The layout of the memory cell is compact. A logic high output from the memory cell (400) is about VDD and a logic low output is about VSS. The memory cell (400) of the present invention includes a programmable memory element (515). In one embodiment, the programmable memory element (515) is coupled between supply voltage (510) and an output node (405). A pull-down device (525) is coupled between another supply voltage (505) and the output node (405). The memory cell (400) may be used to store the configuration information for a programmable logic device (121).

The present invention relates to the field of integrated circuit memory technology. More specifically, the present invention provides a static nonvolatile memory cell for storing data.
Memory cells are used in the implementation of many types of electronic devices and integrated circuits. These devices include microprocessors, static random access memories (SRAMs), erasable-programmable read only memories (EPROMs), electrically erasable programmable read only memories (EEPROMs), Flash EEPROM memories, programmable logic devices (PLDs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), among others. Memory cells are used to store the data and other information for these and other integrated circuits.
As integrated circuit technology and semiconductor processing continue to advance, there is a need for greater densities and functionality in integrated circuits, which are often determined in a large part by the size of the memory cells. Further, it is desirable that the memory cells have improved operating characteristics, such as lower power consumption, nonvolatility, greater device longevity, improved data retention, better transient performance, superior voltage and current attributes, and improvements in other similar attributes.
Furthermore, improved memory cells are especially needed for particular applications, such as PLD integrated circuits. PLDs are well known to those in the electronic art. Such programmable logic devices are commonly referred as PALs (Programmable Array Logic), PLAs (Programmable Logic Arrays), FPLAs, PLDs, EPLDs (Erasable Programmable Logic Devices), EEPLDs (Electrically Erasable Programmable Logic Devices), LCAs (Logic Cell Arrays), FPGAs (Field Programmable Gate Arrays), and the like. Such devices are used in a wide array of applications where it is desirable to program standard, off-the-shelf devices for a specific application. Such devices include, for example, the well-known, Classic.TM., and MAX.RTM. 5000, MAX.RTM. 7000, and FLEX.RTM. 8000 EPLDs made by Altera Corp.
PLDs are generally known in which many logic array blocks (LABs) are provided in a two-dimensional array. LABs contain a number of individual programmable logic elements (LEs) which provide relatively elementary logic functions such as NAND, NOR, and exclusive OR. Further, PLDs have an array of intersecting signal conductors for programmably selecting and conducting logic signals to, from, and between the LABs and LEs. The configuration of the LABs, LEs, and interconnections between these logical elements is stored in memory cells. Memory cells may be used to programmably control the composition, configuration, and arrangements of logic array blocks (LABs) and logic elements (LEs) and also the interconnections between these logic array blocks and logic elements.
Resulting from the continued scaling and shrinking of semiconductor device geometries which are used to form integrated circuits (also known as "chips"), integrated circuits have progressively become smaller and denser. For programmable logic, it becomes possible to put greater numbers of programmable logic elements onto one integrated circuit. Furthermore, as the number of elements increases, it becomes increasingly important to improve the techniques and architectures used for interconnecting the elements and routing signals between the logic blocks. Also as PLDs increase in size and complexity, greater numbers of memory cells are required on to hold the configuration information of the logical elements.
While such devices have met with substantial success, such devices also meet with certain limitations, especially in situations in which the provision of more complex logic modules and additional or alternative types of interconnections between the logic modules would have benefits sufficient to justify the additional circuitry and programming complexity. There is also a continuing demand for logic devices with larger capacity. This produces a need to implement logic functions more efficiently and to make better use of the portion of the device which is devoted to interconnecting individual logic modules. Moreover, there is a need to more efficiently and effectively store the configuration information of PLDS. The memory technology used to store the configuration information of the PLD should be compact, power efficient, programmable and nonvolatile, require little additional programming circuitry overhead, and generally provide enhancements to the performance and features of PLD logic modules and interconnections.
As can be seen, an improved memory cell is needed, especially an improved memory cell for storing the configuration information for the logic elements and interconnections of a programmable logic device.

Nonvolatile configuration cells and cell arrays


Methods, systems, and apparatus for obtaining application data using a card reader having a pass-through mode and an automatic polling mode are disclosed. Application data may be obtained by enabling the pass-through mode of the card reader, instructing the card reader to poll for an exchange event, identifying the exchange event, identifying a subject of the exchange event, and obtaining application data from the subject when the subject of the exchange event is a predefined application. The pass-through mode may be disabled and the automatic polling mode may be enabled after the application data is obtained. An account number from the subject of the exchange event may also be obtained for use in authorizing a vending transaction.

Vending systems are often used to dispense products and/or services to consumers in locations where it would be impractical or inefficient to staff human beings to provide the products/services. Vending systems may be equipped with magnetic card readers and contact-less smart card readers (e.g., smart card readers that utilize near-field communication (NFC) technology) that may be used for cashless payment (e.g., credit cards). Additionally, sellers (vendors) want to implement customer loyalty programs to make their products more appealing to consumers. Improved methods, systems, and apparatus for implementing customer loyalty programs associated with cashless payment methods are desirable.

Vending approval systems, methods, and apparatus using card readers


An apparatus for measuring the rate of flow of a volume of particulate material comprising a transparent tube of known cross-sectional area, means for supplying a flow of particulate material through the transparent tube, and an optical signal sensor for receiving an optical signal through the wall of the transparent tube from the flow of particulate material. The received optical signal is delivered to a controller for analyzing and determining the relative movement of the flow of particulate material at the wall of the tube and calculating the rate of volume flow through the tube. These observations may be used to control upstream and downstream system processes and equipment.

It is desirable in many devices and systems to have a relatively precise measurement of a volume or weight of particulate material. In such devices and systems, scales or other weight-measuring devices are used to weigh a desired amount of particulate material. Containers or bushings of a known volume are also used to measure a desired volume of the particulate matter. Auger or screw conveyors are also employed to supply particulate material at a desired feed or volume rate.
The use of such scales or weight-measuring devices can be expensive and may slow down the system or process in which the device is incorporated. The use of bushings or similar containers in such systems may limit the ability to precisely control a desired volume of material without the need to change or use multiple containers or bushings. Changing the containers or bushings may slow down the system or process increasing the cost of operation and decreasing the output of systems incorporating such devices. When auger or screw conveyors are used the rate of the flow is often affected by the geometry of the feed material. Verification of the rate of flow can be expensive, time consuming and imprecise. Consequently, a need exists for inexpensive and relatively precise measuring equipment for the measurement of particulate matter used in systems requiring particulate additives, particulate feedstocks and the like.

Apparatus and method for measuring particulate flow rate


A swab product is provided which includes a swab defined by an elongate stem and a fibrous absorbent covering such as cotton on at least one end of the stem. The fibrous absorbent covering is held together by a film-forming polymer. A treatment composition is dispersed onto the fibrous absorbent covering. Particularly preferred is a highly viscous hydrophobic treatment composition consisting of at least about 80% of petrolatum, which may serve to deliver actives such as bacitracin ointment or by itself serve as an occlusive therapeutic covering of skin to prevent infection.

1. Field of the Invention
The invention concerns swabs impregnated with a viscous treatment composition and a related method for topically applying the composition to skin, hair and the oral cavity.
2. The Related Art
Swabs having an absorbent covering on the tip and an elongated stem are well known. Cotton is generally used as the absorbent covering material. Stem materials are often of wood, rolled paper or plastic. An adhesive binder may be used to more firmly hold the absorbent covering in place upon the swab. Ordinarily extremely small amounts of binder are used so as not to interfere with the billowy nature of the attached cotton.
Swabs have been used as applicators for a variety of cosmetic and pharmaceutical purposes. Rarely are they pretreated with any sort of agent intended for transfer. It is up to the consumer to dip the swab into a particular agent or cosmetic which they wish to apply to their body. Non-pretreatment of swabs has the advantage of universality and allows choice to the consumer. Unfortunately, universality sacrifices the advantage of convenience. Perhaps even more important is sacrifice of the functional advantage from a swab engineered to solve a particular problem.
U.S. Pat. Nos. 5,035,348 and 5,100,028, both to Seifert and assigned to the Institute Guilfoyle, disclose an engineered swab-type article intended for delivery of liquid products to the skin. Therein is disclosed a flexible stick-like fluid dispenser mounted with a cotton tip. A frangible seal separates the fluid compartment from the tip. Upon application of force against the seal, a separation wall breaks allowing fluid to permeate the cotton tip. Products related to this technology are sold by the Purdue Frederick Company under the trademark Betadine(copyright) Prep Stick. This product is described as a self-saturating disposable swab applicator for povidone-iodine used as a topical antiseptic. The Betadine(copyright) Prep Stick applicator is marketed as an individual swab sealed within a tear open pouch. Systems of this type are expensive to manufacture, and somewhat messy considering the requirement for the active agent to be delivered in a relatively non-viscous highly fluid carrier. Active agents must also be soluble in that carrier for transfer to the tip.
U.S. Pat. Nos. 5,846,215 and 5,919,152 to Zygmont disclose a swab whose absorbent covering has been treated for resistance to bacterial contamination. The covering, in particular a cotton wad, is dipped into a padding medium which includes an anti-microbial agent dispersed in a water slurry having up to 20% of a binder. The swab is dried after the padding step resulting in the anti-microbial agent being deposited in dry form onto the cotton wad.
A particular problem in transference of compositions occurs where the transferred composition is a relatively viscous substance.
Accordingly, it is an object of the present invention to provide a swab pretreated with a viscous composition for topical application to the body.
Another object of the present invention is to provide a cotton swab pretreated with a viscous treatment composition forming an applicator system that can cleanly deliver the composition without interference and transfer of any substantial amount of cotton from the applicator tip.
A swab product is provided which includes:
(i) an elongate stem with first and second ends opposite one another, a fibrous absorbent covering surrounding at least one of the first and second ends, the covering being held together by a film-forming polymer; and
(ii) a treatment composition, the composition having a viscosity ranging from about 100,000 cps to about 5,000,000 cps.
In a more particular aspect of the present invention, there is provided a swab product including:
(i) an elongate stem with first and second ends opposite one another, fibrous cellulosic absorbent covering surrounding at least one of the first and second ends, the cotton being held together by a film-forming polymer in an effective amount to prevent the fibrous covering from unfurling during use; and
(ii) a treatment composition having a viscosity ranging from about 500,000 cps to about 3,500,000 cps.
Still further, there is provided a method for treating diseased or injured skin by applying through aid of a swab, a treatment composition against the disease or injury, the composition having a viscosity ranging from about 100,000 to about 5,000,000 cps, the swab including an elongate stem with first and second ends opposite one another, an absorbent covering such as cotton surrounding at least one of the first and second ends, the covering being held together by a film-forming polymer and whereupon is deposited the treatment composition.

Unilever Home and Personal Care USA, division of Conopco, Inc.


A partition panel for dividing a room in an office comprises a pair of decoration panels each having a plurality of engagement holes, a rectangular frame between the pair of decoration panels, and an engagement member having a plurality of hooks. The hooks engage in the engagement holes of the decoration panel. The engagement member is fixed to the frame.

The present invention relates to a partition panel for partitioning the interior of an office and a method of assembling the partition panel.
Conventionally, in a fixed partition panel, a frame is assembled and a decoration panel is mounted to each side of the decoration panel.
JP2004-174068A discloses a moving partition device in which a partition panel suspends from a runner moving along a rail on a ceiling. But in the partition panel, it is very difficult to mount a decoration panel to a frame that suspends from the runner and to mount a closing member for closing a gap with the ceiling or floor, and Al alloy peripheral member to the periphery of the partition panel. It is necessary to assemble the panel in a factory preliminarily.
It is important to keep unity in design between a moving partition panel and a fixed partition panel and to assemble the moving partition panel by oneself. Conventional moving partition panels are too complicated for assembling and do not satisfy the requirements.
If a partition panel comprises a decoration panel and a glass panel, it will be more difficult to assemble it.

Partition panel and a method of assembling it

Abstract: Another partition panel comprises a rectangular frame having a vertical rod, a plurality of panels on the rectangular frame, and an engagement member having a plurality of hooks and screw holes. The engagement member is fixed to the vertical rod of the frame with screws.

Abstract: The partition panel can be assembled easily.


The present invention discloses a system and method to telephonically access information related with items printed on a physical document, simply by touching with a finger items marked on the physical document or on any other physical surface. The system includes a touch foil placed over (or under) the document, for reading the coordinates of selected marked items, a telephone set for receiving the coordinates of the items selected by the user and for dialing to an interactive voice response (IVR) service used for accessing and retrieving selected hyperlinked documents from the web through a documents proxy server, for converting retrieved documents to synthesized speech, and for providing to the user through the telephone network, the selected documents that have been converted into speech.

Interactive electronic services, video-on-demand, and the World Wide Web are providing access to an increasing offering of movies, shopping information, games, multimedia documents, electronic commerce and many other services. During the last few years, due mainly to the widespread use of personal computers and the near universal access of millions of users to the World Wide Web, an enormous amount of hypermedia information combining text, images and sounds is accessible via the Internet on the World Wide Web.
While the growth of the Internet as a global medium for communications and commerce has been driven, in part, by the increased availability of personal computers, distributed architecture and a common standard for application development which hides the intricacies of creating the graphical user interfaces from the developer, however, access to the Internet over a personal computer remains limited because consumers must have access to a computer and a working Internet connection.
Wireless access to the Internet over cellular telephones and other handheld devices has the potential to resolve the mobility and Internet connectivity issues presented by Internet access over a personal computer. However, while the number of mobile wireless devices has increased in recent years, display screens on these devices are small and the ability to input information using portable keyboards is constrained, limiting the usability and convenience of this solution. Therefore, the goal of anytime, anywhere access to a wide variety of information services has not yet been fully realized. Accordingly, there exists a need for enhanced online information access.
Even when the public's enthusiasm for new computer-based multimedia services has been seen by many analysts as a threat to the conventional forms of hard-copied publishing, particularly book publishing, the real fact is that, reading a book cannot be compared with reading electronic media. Reading paper remains preferable for most people, whether they are familiar with computers or not. Reading manuals and reports at work, textbooks at school, and menus at restaurants . . . and more generally reading printed material at any time and in any place is part of our daily lives. People can browse very easily through paper catalogs, magazines, newspapers, maps and books by flipping through the pages and by "glancing" at pictures and text. It is also very easy for them to mark and return to specific parts of a physical document.
It is easier and quicker to turn over pages of a book than to browse electronic pages on a computer screen. Many electronic systems attempt to replace paper. These offer, for example, a better access to multimedia services. However, most users prefer to work with paper and it is difficult to foresee, in the near future, a general and massive replacement of paper books by electronic books (e.g. by Web accessible e-books). As reported in an article entitled "Paper is still with us", by Philippa Jane Benson—The Journal of Electronic Publishing, published by the University of Michigan Press:                "Studies of professionals at work underscore three points that are critical to the design of scientific information: paper isn't going away, reading and writing are inextricably intertwined, and readers sample and navigate text according to specific purposes and tasks".
When we compare paper based information with computer based information, paper has a number of useful properties that computers cannot provide. For example:                paper is portable, familiar and can be easily distributed;        paper is easy to read, mark, and manipulate.Notwithstanding the advantages described hereinabove, the most significant problem, of course, with traditional printed publications is that these cannot be changed, amended, updated nor completed. Accordingly, there exists a need for online information access from printed material.
Despite the Internet's growing acceptance, the telephone network is still more widely and readily accessible. Actually, telephone communications have become a tool of strategic importance. The telephone essentially allows people to communicate and businesses to operate. Everyday, people utilize the telephone system to conduct a broad range of personal and business transactions. Telephones are simple to operate and use the most natural form of communication, the human voice. Access to businesses and public information over the telephone is somewhat easier than access over the Internet because of the greater availability of landline and wireless telephones and the ease of use.
To increase the utility of telephones, a vast array of answering machines, voice mail, interactive voice response, automated call dispatching, forwarding services, and the like have been deployed. Each of these devices or services intends to increase the usefulness of the telephone in a specific way. Businesses have been using telephony based technologies like Interactive Voice Response (hereinafter referred to as "IVR") systems for communicating with customers and conducting business (e.g. contact centers, tele-marketing, help lines, etc.). The proliferation of the wireless phone (i.e. cell phone) has made access to the telephone network even easier. Accordingly, there exists a need for enhanced telephone and online information access.
The widespread use of the Internet and mobile telephony offer many new opportunities to publishers and customers to combine electronic and printed media, i.e. to create "media-adaptive multimedia," products. The philosophy behind the concept of "media-adaptive multimedia" is that information has to be conveyed to customers in a form that is adaptable to their (multimedia) requirements. In fact, today there is a need to enhance traditional printed products with access to digitally stored information, using mobile telephony and voice response systems to access and retrieve information through an electronic network, such as the Internet—i.e. the convergence of printed information, telephony and web access.
To convey information to customers pervasively, in a form suitable to their requirements, new interfaces for accessing information from different media are required. The necessity of using a natural friendly interface for having access to information has been precisely summarized by Ann Light, in the article entitled "Fourteen Users in Search of a Newspaper: the Effect of Expectation on Online Behaviour", School of Cognitive and Computing Sciences, University of Sussex, CSRP 507:                "People expect the friendly familiar paradigm of media to guide them through uncharted territories of information".
Previous attempts to coordinate printed matter with external digital objects have used techniques based on optical sensing and decoding of digital data where data has been visibly encoded (e.g. using bar codes), or steganographically encoded using techniques similar to embedding watermarks, subliminal calibration patterns, or the like on the printed medium. Examples of some of these previous attempts can be found in the patents described hereinbelow.
U.S. Pat. No. 5,495,581 entitled "Method and Apparatus for Linking a Document with Associated Reference Information Using Pattern Matching" uses image scanning and pattern matching techniques. This patent discloses an apparatus for linking a portion of a document with associated reference information, wherein the linked portion is designated by a predetermined attribute of the received document image, using inter alia, a device for electronically scanning the electronic representation of the document image to locate said predetermined attribute of the document's image. A main drawback of this system is the need for use of bulky, non portable scanners, not being usable, therefore, in portable, mobile environments.
A similar approach using optical image scanners to access multimedia services is described in U.S. Pat. No. 5,640,193 entitled "Multimedia Service Access by Reading Marks on an Object". This patent discloses an apparatus and a method to enable a user to control the selection of electronic multimedia services using a scanner for reading marks on an object and for communicating a request signal, having an object code representing the read marks, to a user interface.
U.S. Pat. No. 5,903,729 entitled "Method, System, and Article of Manufacture for Navigating to a Resource in an Electronic Network", discloses a method for navigating on an electronic network. The method comprises the steps of:                forming an electronic image of an object having a plurality of markable regions associated with a plurality of electronic resources;        processing the electronic image to detect which of the markable regions associated with the plurality of electronic resources is marked;        generating a list comprising at least one link to at least one of the electronic resources whose associated markable region is marked, and        displaying a display screen based upon the list.The invention disclosed in U.S. Pat. No. 5,903,729 further requires image scanning and image processing means to read marked regions on hard-copied documents.
U.S. Pat. No. 5,768,426 entitled "Graphics processing system employing embedded code signals" discloses a system where an identification code signal is impressed on a carrier to be identified (such as an electronic data signal or a physical medium) in a manner that permits the identification signal later to be discerned and the carrier thereby identified.
U.S. Pat. No. 5,905,251 entitled "Hand-held Portable WWW Access Terminal with Visual Display Panel and GUI-based WWW Browser Program Integrated with Bar Code Symbol Reader in a Hand-supportable Housing" discloses a portable hand-held World Wide Web (hereinafter referred to as "WWW") access terminal for accessing HTML-encoded documents located on the WWW. The terminal includes a bar code symbol reader in a hand-supportable housing for reading URL-encoded symbols specifying the location of HTML-encoded documents stored in information servers connected to the Internet and supporting the TCP/IP standard. This invention requires the marking of physical documents with bar code symbols and requires bar code readers to trigger hyperlinks.
U.S. Pat. No. 6,311,214 entitled "Linking of computers based on optical sensing of digital data" discloses a system where a printed object, such as an item of postal mail, a book, printed advertising, a business card, or a product packaging, is steganographically encoded with plural-bit data. When such an object is presented to an optical sensor, the plural-bit data is decoded and used to establish a link to an internet address corresponding to that object.
Similarly, U.S. Pat. No. 6,408,331 entitled "Computer linking methods using encoded graphics" discloses a system and a method where a data object comprises both a graphic and embedded link information, such as the URL address of a network node, permitting the graphic object to serve as a link usable by an internet browser or the like.
U.S. Pat. No. 6,345,104 entitled "Digital watermarks and methods for security documents" deals with a system where security documents (e.g. passports, currency, event tickets, and the like) are encoded to convey machine-readable multi-bit binary information (e.g. a digital watermark), usually in a manner not alerting human viewers that such information is present. The documents can be provided with overt or subliminal calibration patterns. When a document incorporating such a pattern is scanned (e.g. by a photocopier), the pattern facilitates detection of the encoded information. A commercial product of this class, for embedding a digital watermark in an image, and means of detecting the watermark is provided by the Digimarc® MediaBridge™.
The systems, methods and products described hereinabove have the common drawback of requiring the use of specialized encoding and printing techniques for embedding codes, patterns or watermarks on printed materials, those techniques being non-standard in the printing industry. In fact, the requirement of modifying the original documents by printing coded information affects the integrity or even the readability of documents and represents a fundamental drawback of all these systems. Moreover, the end user must be provided with a specialized interface technology such as digital scanners or optical sensors for locating, sensing and decoding embedded information from publications printed using these techniques.
Based upon the analysis provided hereinabove of the different systems proposed heretofore for identifying, locating and accessing digital objects from printed matter, it would be a significant improvement to provide the possibility to access information related with items printed on a physical document, anytime and anywhere, directly from the same physical document, eliminating the need of requiring the use of specialized encoding and printing techniques for embedding codes, patterns or watermarks on the printed materials, and the need of using scanners or code readers for capturing and decoding said codes, patterns or watermarks identifying such information, while preventing errors and mistakes due to the manual capture of the coding of such items.
Accordingly, there is a need for systems and methods for improving printed texts with telephonically accessible, electronically stored data. In particular, there is a need for a system utilizing a telephone-based user interface capable of retrieving from the Web, and delivering to users, information related to items printed on physical documents.

System and method for providing telephonic voice response information related to items marked on physical documents

International Business Machines Corporation


The present disclosure is directed toward intelligently mixing and matching faces and/or people to generate an enhanced image that reduces or minimize artifacts and other defects. For example, the disclosed systems can selectively apply different alignment models to determine a relative alignment between a references image and a target image having an improved instance of the person. Upon aligning the digital images, the disclosed systems can intelligently identify a replacement region based on a boundary that includes the target instance and the reference instance of the person without intersecting other objects or people in the image. Using the size and shape of the replacement region around the target instance and the reference instance, the systems replace the instance of the person in the reference image with the target instance. The alignment of the images and the intelligent selection of the replacement region minimizes inconsistencies and/or artifacts in the final image.

Recent years have seen significant increases in both technologies for, and use of, digital photography. Indeed, the advent of smart phones with integrated, high quality cameras has led to large percentages of people carrying digital cameras with them almost constantly. This has led to massive increases in the number of digital photos being taken. Not only has the number of digital photographs recently increased, so has the ease and speed of sharing digital photographs. For example, smart phones with social networking applications allow users to share digital photographs almost immediately after capture. Users now seek to capture and share the "perfect picture" as quickly as possible.
One recent advance in aiding photographers in capturing high quality photographs is burst mode or continuous shooting. Digital cameras with burst mode capture multiple frames or images per second. Indeed, digital cameras with burst mode can capture tens of images per second. Burst mode photography helps photographers capture a high-quality image in various different types of moments. Using burst mode to capture a group portrait allows a photographer to capture dozens of images and select the desired instance where imperfections are minimalized. For example, group portraits often have multiple imperfections such as people blinking, looking in the wrong direction, or blurred movement. Burst mode allows the photographer to increase the odds of capturing an image with the smallest number of imperfections, i.e., an image with everyone in the group looking forward with their eyes open.
While using burst mode photography for group portraits provides various advantages, burst mode photography also presents a number of drawbacks. In particular, the chance of capturing the perfect photo of each person's face is often low. For example, one person is likely to the best appearance in one photo and another person is likely to have the best appearance in another photo.
Additionally, combing through multiple group photos using conventional burst mode photography systems is often inefficient. Typically, conventional burst mode photography systems require the photographer to review and compare each individual person's face in potentially dozens of images to identify a desired or high-quality group portrait. Reviewing burst mode images can be both tedious and time consuming because of the difficulty in comparing such similar images. Furthermore, the use of handheld devices with relatively small displays exacerbates these problems as faces within the images can appear relatively small. The time and effort associated with selecting a high-quality group portrait can deter users from employing burst mode when they are trying to capture and share an image quickly.
Conventional photo management systems provide the ability to mix and match parts of burst images. However, the mix and match functionality provided by conventional systems is often inflexible. For example, in order to produce an acceptable result, the conventional systems require that burst photos are taken by a camera mounted on a tripod. Thus, photographers are often not afforded the flexibility of mixing and matching parts of images that have been captured using a handheld camera or mobile phone. If photos are not taken using a tripod, existing systems often produce images that include inconsistencies or artifacts.
Additionally, even if a group of photos is taken using a tripod, images that have been mixed and matched using conventional photo management systems often require intensive manual editing and nonetheless often include inconsistencies and/or visual artifacts. For example, replacing a face or a face bounding box with another face or face bounding box often results in discontinuities and artifacts in the final result. For example, if the person has moved between the two shots in the burst of photos, the face bounding box might include different portions of background structures. Thus, the enhanced image produced by conventional photo management systems will often include inconsistencies.
These and other problems exist with regard to capturing and editing group portraits.

Intelligent mixing and replacing of persons in group portraits


The image projection apparatus of the present invention includes a light source that outputs coherent light; a phase modulator that performs phase modulation on the light output from the light source to create a Fourier transform image; an amplitude modulator that performs amplitude modulation on the Fourier transform image in accordance with an image signal; a projection optical system that projects the light that has been amplitude modulated, and a controller that divides the image represented by the image signal into a plurality of divided images, determines the target luminance value of each of the areas of the Fourier transform image that is divided corresponding to the division of the image, based on the image signal, and outputs phase data indicating the amount of phase modulation so as to make the luminance value of each area of the Fourier transform image acquire the target luminance value of the associated area.

Meeting the demand for a display apparatus that can display images represented by an image signal involves developing high contrast and low power consumption configurations and the like. To meet these demands, a local dimming technique is used.
Local dimming is a technique in which the image represented by the image signal is divided into a plurality of divided images and the brightness of each divided image is adjusted in accordance with the brightness level such as the luminance value or the like of each divided image so as to make a dark divided image appear darker and a bright divided image appear brighter.
For example, in the liquid crystal display using LED (Light Emitting Diode) as its backlight, local dimming can be implemented by lowering the output of backlight in the area where the dark divided image is displayed. Since lowering the output of backlight in the area where a dark divided image is displayed makes the dark divided image appear darker, the dynamic range of the displayed image can be widened, hence high-contrast display can be achieved. Further, lowering the output of backlight can also achieve low power consumption.
As the division number of the image is increased in local dimming, the brightness of each of the pixels that form the image will be adjusted last.
An organic EL (Electro Luminescence) display can be regarded as a display in which the brightness of each of the pixels can be adjusted. However, organic EL displays suffer from the problem in which the lifetime of the light source is not sufficiently long.
An image projection apparatus that projects the image by scanning the laser beam output from a light source by use of a MEMS (Micro Electro Mechanical Systems) mirror, can be regarded as another kind of display in which the brightness of each of the pixels can be adjusted.
There are image projection apparatuses other than the image projection apparatus that use the aforementioned MEMS mirror, such as ones that project the image by performing amplitude modulation on the light output from a lamp, LED, laser light source or the like by means of a display device such as a liquid crystal panel, DMD (Digital Micro mirror Device), in accordance with the image signal. These image projection apparatuses can project the image with a simpler configuration compared to the image projection apparatus using a MEMS mirror.
Patent Document 1 (JP2008-544307A) discloses an image projection apparatus which projects the image by performing phase modulation on the laser beam output from the light source in accordance with the image signal. According to the image projection apparatus disclosed in Patent Document 1, it is possible to promote low power consumption because phase modulation on the light output from the light source will not cause any optical loss, unlike in the case of amplitude modulation.

Image projection apparatus and control method therefor