2006-03-27 Assigned to DIGIMARC CORPORATION reassignment DIGIMARC CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SHARMA, RAVI K.
Steganographic calibration signals (sometimes termed “orientation signals,” “marker signals,” reference signals,” “grid signals,” etc.) are sometimes included with digital watermarking signals so that subsequent distortion of the object thereby marked (e.g., a digital image file, audio clip, document, etc.) can later be discerned and compensated-for. Digital watermark detection systems sometimes fail if the object encompasses several separately-watermarked components (e.g., a scanned magazine page with several different images, or photocopy data resulting from scanning while several documents are on the photocopier platen). Each component may include its own calibration signal, confusing the detection system. In accordance with certain embodiments, this problem is addressed by a proximity-based approach, and/or a multiple grid-based approach. In accordance with other embodiments, the calibration signal can—itself—convey watermark information, so it serves both a calibration and a payload-conveyance function.
This application is a continuation-in-part of application Ser. No. 10/032,282 filed Dec. 20, 2001 (now U.S. Pat. No. 6,975,744), which claims priority benefit to provisional application No. 60/257,924, filed Dec. 21, 2000.
This application is also a continuation-in-part of application Ser. No. 10/333,344, filed Jul. 24, 2003 (U.S. Pat. No. 7,319,775), which is a national-phase counterpart to PCT application PCT/US01/22173, filed Jul. 12, 2001, which claims priority to application Ser. No. 09/618,948, filed Jul. 19, 2000 (now U.S. Pat. No. 6,385,329).
The present invention relates to digital watermark technology, and particularly concerns situations in which several digitally watermarked objects are presented to a watermark detector.
Digital watermarking systems are known in the art, and are shown, e.g., in copending applications Ser. No. 09/503,881 (now U.S. Pat. No. 6,614,914) and Ser. No. 09/452,023 (now U.S. Pat. No. 6,408,082).
Steganographic calibration signals (sometimes termed “orientation signals,” “marker signals,” reference signals,” “grid signals,” etc.) are sometimes included with digital watermarking signals so that subsequent distortion of the object thereby marked (e.g., a digital image file, audio clip, document, etc.) can later be discerned and compensated-for. Such arrangements are detailed in the cited applications.
Systems for detecting watermarks from marked objects sometimes fail if the image object encompasses several separately-watermarked components (e.g., a scanned magazine page with several different images, or photocopy data resulting from scanning while several documents are on the photocopier platen, as shown in FIG. 1). Each component may include its own calibration signal, confusing the detection system.
In accordance with certain embodiments of the present invention, this problem is addressed by a proximity-based approach, and/or a multiple grids-based approach.
In accordance with other embodiments of the present invention, the calibration signal can—itself—convey the watermark information, so it serves both a calibration and a payload-conveyance function.
The foregoing and additional features and advantage of the present invention will be more readily apparent from the following detailed description, which proceeds with reference to the accompanying drawings.
FIG. 1 shows several documents as they may be positioned on the platen of a photocopier.
FIG. 2 is a flow chart detailing one embodiment of a multiple-grid detection approach.
FIG. 3 is a flow chart further detailing the embodiment of FIG. 2.
For expository convenience, the following discussion focuses on the context of processing image data resulting from scanning several documents on the same photocopier platen. It should be recognized, however, that the invention is not so limited, but finds application, e.g., in contexts involving audio, video and other content objects.
The proximity approach is based on selecting and clubbing together image blocks that are in close proximity to each other. For two watermarked documents in an image, this approach would ideally give two clusters of blocks, one for each watermark. This approach requires modification to the methodology described in the '881 application. The prior methodology calculated the variance and edges in each block and used these to compute a metric for the block. Blocks were ranked based on their metric, and the top M blocks are selected to perform grid detection. The proximity approach, in contrast, takes into account the spatial locations of the blocks, and/or their distances to each other, for division into one or more clusters.
Separate block clusters can be treated as independent sets of data for grid detection and further decoding.
Except for the block clustering and proximity determination, prior art techniques (e.g., as in the '881 application) can remain unchanged.
The proximity approach may result in blocks from a single watermarked document being divided into more than one cluster.
The proximity approach may fail for multiple watermarked documents that are either overlapped or in close proximity to each other.
The extensibility of this approach to situations where the number of watermarked components is more than two or three is uncertain.
These shortcomings are generally obviated by the multiple grids approach.
The multiple grids approach tackles the problem by searching for multiple grids during grid detection. The approach is based on the notion that if there is more than one watermarked component, the accumulated Fourier magnitude should contain a grid signal corresponding to each watermark. The presence of multiple grids generates multiple strong peaks both at the log-polar correlation and Fourier magnitude correlation steps in the process. Each peak corresponds to the rotation and scale of the corresponding grid.
Compute the detection value corresponding to each of the N (e.g., 2-16) candidates in the Fourier Magnitude Correlation (FMC) step.
Select the M candidates that surpass the two-step rejection criterion.
Try refining scale/rotation on these M candidates to obtain refined rotation and scale as well as differential scale and shear (i.e., a 4D refinement vector).
Loop through the M 4D vector sets and perform translation determination and read (decode) operations for each, using all available blocks.
Buffer the decoded watermark information from each of the M candidates (i.e., read or not read, if read what type of watermark, and so on).
Either return this information to the calling program, or select the watermark to return to the calling program based upon a pre-determined priority ranking of the various possible watermarks.
This approach works as long as there is sufficient signal strength corresponding to each potential grid in the accumulated Fourier magnitude. Also note that this approach assumes that all the watermarks in the image have the same grid. Experiments using this approach have yielded promising results and shown that this approach is feasible.
It enables detection of multiple watermarks even if the multiple watermarked documents are overlapped or in close proximity to each other.
It does not affect the performance (speed) of the grid detection stage.
Most of the methodology parallels prior art techniques; the sequence of operations after the grid detection stage need to be repeated for each detected grid.
It is possible to extend this approach to situations where there are more than two or three watermarked components in an image.
If the multiple watermarks are weak, it may be difficult to detect them in the accumulated Fourier magnitude.
One challenge in this approach arises if the multiple grids have almost the same rotation and scale. This limitation arises because the peak finding algorithm cannot resolve closely located peaks in the GMF correlation plane.
The two-step rejection criteria noted above is more particularly detailed in application Ser. No. 09/526,982 (now U.S. Pat. No. 6,516,079).
It comprises a collection of impulse or delta functions in the Fourier magnitude domain.
In other embodiments, different grid signals can be used in differently-watermarked excerpts of the content. FIGS. 2 and 3 more particularly detail a detection process useful in this context.
As before, the image is segmented into blocks, pre-filtered, the converted into the Fourier domain. The Fourier representation for all the component blocks are accumulated, filtered, and remapped into the log-polar domain.
In contrast to the multiple-same grid context, the multiple-different grid context process proceeds by correlating the log-polar representation obtained above, with a log-polar remapping of the Fourier magnitude representation of each of the component grid signals, summed together (a summed-grid template). This correlation yields several peaks, each indicating a candidate scale/rotation state of one of the component watermarked elements. But the peaks do not indicate the particular grid signals with which they correlated, since the correlation was based on a summed-grid template. Accordingly, the method proceeds by checking each discerned scale/rotation state (correlation peak) of the data against the Fourier magnitude spectrum of the different grid signals, to determine which grid signal should be used in decoding a given scale/rotation state of the data. Once this association between scale/rotation states of data, and applicable grid signal, has been determined, decoding of each can proceed as above (and in the cited applications).
Of course, for any single grid signal, there may be several objects represented in the data set—each with a different scale or rotation.
Note that there are many possible applications: The calibration signal can carry protocol information to tell the decoder how to interpret the message payload of another watermark signal.
In systems in which the “grid” signal conveys the message, there is no need for separate “grid” and “message” signals.
For additional information on the use of the grid signal to convey payload information, see pending application Ser. No. 09/618,948 (now U.S. Pat. No. 6,385,329).
In such an arrangement, affine transformation of the digital content object can be discerned from affine transformation of the grid signal, and the payload can be discerned from the polarities of the grid signal components.
In the foregoing embodiment, the watermark may include other components in addition to those of which the grid signal is comprised. These other components can serve to convey additional payload data.
The payload data represented by the polarities of said grid signal components can convey various types of information, such as protocol information.
Having described and illustrated the principles of our inventive work with reference to specific embodiments, it will be recognized that the principles thereof can be implemented in other, different, forms.
For example, while the disclosure focused on image data, the same techniques are applicable in other watermarking contexts, including audio and video.
Moreover, while the invention was illustrated in the context of the present assignee's preferred forms of watermarking, it should be recognized that the invention's applicability is not so limited. For example, such techniques also find utility in combination with the teachings of watermarking U.S. Pat. Nos. 5,949,055, 6,044,182, etc.
As is familiar to those skilled in the arts, all of the foregoing methods may be performed using dedicated hardware components/systems for the various devices, and/or through use of processors programmed in accordance with firmware or software, etc. In the latter case the processors may each include a CPU and associated memory, together with appropriate input and output devices/facilities. The software can be resident on a physical storage media such as disks, and can be loaded into the processors' memory for execution. The software includes instructions causing the CPU to perform the various processes detailed above.
To provide a comprehensive disclosure without unduly lengthening this specification, the patents and applications cited above are incorporated herein by reference.
In view of the wide variety of embodiments to which the principles of our inventive work can be applied, it should be recognized that the detailed embodiments are illustrative only and should not be taken as limiting the scope of the invention. Rather, we claim as our invention all such embodiments as may come within the scope and spirit of the following claims, and equivalents thereof.
1. In a method of altering a content object to steganographically encode a digital watermark therein, the watermark having an orientation signal component that facilitates subsequent discernment of an affine transformation to which the encoded content object has been subjected, the watermark also including a payload component that conveys plural bits of payload data, an improvement comprising using a processor device to process the orientation signal component to serve as a carrier of additional bits of payload data.
2. The method of claim 1 in which the orientation signal component comprises impulse functions, and said processing comprises setting polarities of said impulse functions to represent said additional bits of payload data.
3. The method of claim 1 wherein said additional bits of payload data comprise protocol information that instructs a decoder how to interpret other payload data.
4. In a method of decoding a content object that has been steganographically encoded in accordance with a digital watermark, the method including analyzing a calibration signal component of said digital watermark to establish an affine state of said content object, and by reference to said affine state, decoding plural-bit payload data from a payload component of said digital watermark, an improvement comprising using a processor device to decode additional plural-bit payload data from said calibration signal.
5. The method of claim 4 in which the calibration signal component of said digital watermark comprises impulse functions, and said improvement includes decoding said additional plural-bit payload data by reference to polarities of said impulse functions.
6. The method of claim 4 that further includes using said additional plural-bit payload data decoded from the calibration signal to instruct a decoder how to interpret other payload data.
O'Ruanaidh et al., "Rotation, Scale And Translation Invariant Spread Spectrum Digital Image Watermarking," Signal Processing, vol. 66, May 1998, pp. 303-317.