A method, system, and program for time based regulation of use of a telephone line are provided. An identity of a first caller requesting use of a telephone line is detected. Use of the telephone line by the caller is allowed only if a schedule for the telephone line indicates that the identity is currently allowed for the telephone line.

1. Technical Field
The present invention relates in general to telecommunications and, in particular, to call party identification. Still more particularly, the present invention relates to time based regulation of use of a telephone line.
2. Description of the Related Art
Telephone service has created communication channels worldwide, and those channels continue to expand with the advent of cellular and other wireless services. A person can simply take a telephone off-hook and dial a destination number or press a send button and be connected to a telephone line around the world.
General telephone service for local calls is often provided at a flat rate. Other telephone services, such as long distance service, are often billed by the minute or at a flat rate in addition to the charges for local service.
Telephone service subscribers may choose to block use of general telephone service and extra telephone services by requiring a passcode or other entry to access the telephone service. In addition, the requirement of a passcode to access telephone service may allow for easier accounting of charges incurred on a telephone line, where each user of the line is provided with an individual passcode to enter and charges are itemized according to passcode.
However, blocking use of service altogether or requiring a passcode entry only provides generic blocking options. Passcode entry does not solve the problem of determining who gets to use a line when multiple people request access at the same time. In addition, blocking use of a service does not solve the problem of when a phone line may be used.
Therefore, in view of the foregoing, it would be advantageous to provide a method, system, and program for time based regulation of use of a telephone line. In addition, it would be advantageous to provide a method, system, and program for determining which caller is allowed use of a telephone line when multiple callers are requesting access.

Time based regulation of use of a telephone line

International Business Machines Corporation


A robust tone detector is realized by applying a single taper, which provides a relatively narrow bandwidth filter, to a received signal and, then, performing a Discrete Fourier Transform of the tapered signal. The result of the Fourier transform is used to obtain an estimate of energy in the frequency domain of the expected tone. A prescribed selection algorithm based on the relationship of the received signal energy in the time domain and the energy estimate in the frequency domain is used to determine whether a valid tone has been received. Robustness of the tone detector is improved by performing a plurality of Discrete Fourier Transforms of the single tapered version of the received signal at a corresponding plurality of frequencies relative to the nominal frequency of an expected one or more tones. A further improvement in the tone detector is obtained by the dynamically adjusting tone acceptance thresholds based on a measure of channel impairments.

Use of tones is widespread in telephony. They are used in setting up a telephone call and to indicate the progress of the call. More recently, tones have been employed during calls to effect advanced features and/or functions. One example is the use of dual-tone multifrequency (DTMF) signals to control the addition of one or more individuals during a conference call.
A serious problem with prior tone detectors is that they falsely detect speech, music or data as tones used for other purposes. That is, the speech, music or data emulates either the individual tones or the DTMF signals. Such false detection of tones during a telephone call causes a so-called "talk-off" condition, i.e., a disruption of the call, resulting in a failure of the communications circuit. Another problem with prior arrangements is that valid tones or DTMF signals are falsely rejected in the presence of channel impairments, e.g., low signal-to-noise ratio.
A recent attempt at overcoming the talk-off problem and the falsely rejecting valid tones or DTMF signals problem employs a plurality of discrete prolate spheroidal sequence (DPSS) tapers, i.e., data windows, to slice a received signal into time segments. Then, Discrete Fourier Transforms (DFTs) of the tapered segments are taken to form estimates of the received signal energy in the frequency domain. The frequency domain energy estimates are then used to identify the received tone or DTMF signals.
Although this prior arrangement operates satisfactorily in certain applications, it still presents a significant talk-off problem when designed to operate satisfactorily in detecting valid tones or DTMF signals in the presence of channel impairments. This is primarily caused by only evaluating frequency domain energy and by the required use of a relatively wide-band filter in the frequency domain to accommodate the use of the plurality of DPSS tapers. Use of such a wide-band filter allows speech, music or data to easily emulate the individual tones and/or DTMF signals, thereby causing talk-off.

Apparatus and robust method for detecting tones


A process for the preparation of tetrahydrothieno[3,2-c]pyridine derivatives of general formula I:

The present invention refers to a new process for the synthesis of tetrahydrothieno[3,2-c]pyridine derivatives, in particular 5-o-chlorobenzyl-4,5,6,7-tetrahydothieno[3,2-c]pyridine and racemic or enantiomerically enriched methyl xcex1-(4,5,6,7-tetrahydro-5-thieno[3,2-c]pyridyl)-o-chlorophenylacetate, and intermediates thereof.
Structure 1, known as Ticlopidine, is an antithrombotic drug with platelet aggregation inhibiting properties as disclosed in U.S. Pat. No. 4,051,141 and U.S. Pat. No. 4,127,580.
The dextrorotatory enantiomer (Structure 2), bearing the International Non-Proprietary name (INN) Clopidogrel, has the absolute configuration S and is a commercially significant drug with excellent antithrombotic and platelet aggregation inhibiting activity as disclosed in U.S. Pat. No. 4,847,265.
Similar properties are displayed by the less potent racemic mixture (U.S. Pat. No. 4,529,596).
The enantiomerically enriched compound can be prepared by means of enantioselective synthesis or starting from a racemic mixture of enantiomers in conjunction with a resolution process.
A known process for the preparation of racemic Clopidogrel 2 is based on the nucleophilic displacement of racemic xcex1-halophenylacetic acid derivatives 3 by 4,5,6,7-tetrahydrothieno[3,2c]pyridine 4 as described in U.S. Pat. No. 4,529,596 and U.S. Pat. No. 5,132,435 (Scheme 1).
The enantiomerically enriched clopidogrel is obtained through the resolution of the racemic mixture of methyl xcex1-(4,5,6,7-tetrahydro-5-thieno[3,2-c]pyridyl)-o-chlorophenylacetate 2 with R-camphorsulfonic acid as described in U.S. Pat. No. 4,847,265 and U.S. Pat. No. 5,132,435.
Racemic clopidogrel can also be prepared starting with the Strecker synthesis of xcex1-(2-thienyl)ethylamino-o-chlorophenylacetonitrile 6 as shown in Scheme 2. Hydrolysis of the nitrile followed by esterification leads to methyl xcex1-(2-thienyl)ethylamino-o-chlorophenylacetate 8. This in turn generates the tetrahydrothieno[3,2-c]pyridine bicyclic system by reacting with a formylating agent (e.g. paraformaldehyde) under acidic conditions. The process is disclosed in WO 98/51682, WO 98/51689 and WO 98/51681.
According to these patents (WO 98/51682 WO 98/51689, WO 98/51681), the same reaction scheme can be applied to the preparation of dextrorotatory clopidogrel in combination with the resolution, one of the intermediates. Thus, xcex1-(2-thienyl)ethylamino-o-chlorophenylacetonitrile 6, xcex1-(2-thienyl)ethylamino-o-chlorophenylacetamide 7 and methyl xcex1-(2-thienyl)ethylamino-o-chlorophenylacetate 8 are resolved with enantiomerically enriched camphorsulfonic acid, tartaric acid and camphorsulfonic acid respectively. Each of these enantiomerically enriched intermediates can be transformed stereospecifically to dextrorotatory clopidogrel without significant racemization. Notably, the stereospecificity of the last step in Scheme 2 is also disclosed in U.S. Pat. No. 5,204,469.
An alternative process for the preparation of the dextrorotatory clopidogrel starts with enantiomerically enriched methyl o-chlorophenylglycine 10 and alkali 2-thienylglycidate 9 in the presence of a borohydride reducing agent as reported in WO 98/39322 (Scheme 3).
In a different approach, dextrorotatory clopidogrel can be obtained starting from enantiomerically enriched (R)-sulphonyloxyacetic ester derivatives 11 and 4,5,6,7-tetrahydrothieno[3,2c]pyridine 4 as disclosed in WO 99/18110. The same publication documents the reaction of the methyl (R)-xcex1-tosyloxy-o-chlorophenylacetate 11 with 2-(2-thienyl)ethylamine 12 to result in methyl (S)-xcex1-(2-(2-thienyl)ethylamino-o-chlorophenylacetate 8. This in turn generates enantiomerically enriched clopidogrel by reaction with a formylating reagent under acidic conditions. (Scheme 4).
Literature precedent for ticlopidine (J. P. Maffrand, R. Boigegrain, Heterocycles, 1979, 12, 1479; FR 2,424,278) shows that (2-thienyl)ethylene oxide 14 reacts with o-chlorobenzylamine to provide the desired N-(2-(2-thienyl)-2-hydroxyethyl)-o-chlorobenzylamine in low yield due to lack of regioselectivity in the epoxide ring opening (Scheme 5).
Experiment has shown that further complication arise from the limited stability of the (2-thienyl)ethylene oxide under vacuum distillation and ambient temperature storage conditions. As expected, the extension of this approach to the synthesis of clopidogrel was bridled by lower chemical yield and lack of regioselectivity in the epoxide ring opening step. Moreover, the subsequent thermal rearrangement step leading to the 4,5,6,7-tetrahydrothieno[3,2-c]pyridine skeleton (J. Heterocyclic Chem., 1976, 13, 1347) failed to provide the target hydroxyclopidogrel intermediate under the reported conditions. In conclusion, the use of the (2-thienyl)ethyleneoxide as an intermediate is technically unfeasible while the rearrangement step is substrate dependent and suffers from lack of generality.
It is therefore an object of the present invention to provide an improved process for the preparation of tetrahydrothieno[3,2-c]pyridine derivatives, in particular 5-(2-chlorobenzyl)-4,5,6,7-tetrahydothieno[3,2-c]pyridine and racemic or enantiomerically enriched methyl xcex1-(4,5,6,7-tetrahydro-5-thieno[3,2-c]pyridyl-o-chlorophenylacetate with inexpensive reagents and in good yields.
It is also an object of the present invention to identify novel intermediates which are useful in the manufacture of the above said compounds.
Further and other objects of the invention will be realised from the summary of invention and examples illustrating the invention.
The present invention provides an improved process for the preparation of a racemic and enantiomerically enriched 4,5,6,7- tetrahydrothieno[3,2-c]pyridines of general formula I
The compound of the general formula I is prepared (Scheme 6) by reducing the compound of formula II with suitable reducing agents, known to those skilled in the art, to a mixture of diastereoisomeric compounds of formula III which are reacted with formaldehyde or any chemical equivalent thereof to yield the compound of formula IV. The compound of formula IV is further rearranged in a suitable solution to produce the compound of formula V which upon reduction (by a suitable reducing agent) provides an enantiomerically enriched or racemic mixture of tetrahydrothieno[3,2-c]pyridine of formula I.
In the general formulae I and V, X can be hydrogen, carboxyl, alkoxycarbonyl, aryloxycarbonyl, nitrile or carbamoyl of formula
wherein R1 and R2 can be individually or simultaneously hydrogen, alkyl or part of a heterocyclic structure; Y can be hydroxyl, alkanoyloxy, aroyloxy, as well as carbonate derivatives of formula xe2x80x94OCOOR3 or carbamate derivatives of formula xe2x80x94OCONR4R5 wherein R3 can be substituted or unsubstituted alkyl or aryl; R4 and R5 can be individually or simultaneously substituted or unsubstituted alkyl, aryl or cycloalkyl; Z can be hydrogen, halogen, alkyl, aryl, aryloxy or alkoxy group.
The compound of formula II in its racemic or enantiomerically enriched form is obtained through the reaction of a racemic or enantiomerically enriched primary amine of general formula
with an xcex1 substituted 2-acetylthiophene derivative of general formula
where X and Z have the same meaning as above and LG is a leaving group such as halogen, arysulphonyloxy, aryloxy, sulphonate, alkyloxy and its derivatives or activated aryl. Other obvious synthetic chemical equivalents of the phenylglycine counterpart, such as various salts as well as aliphatic or aromatic esters, amides, nitrile or free carboxyl group are also included in the present invention. The reaction may be carried out in an aromatic solvent (e.g. toluene and the like), polar aprotic solvents (e.g. dimethylformamide, hexamethylphosphoramide, ketones such as acetone, MEK, MIBK and the like), polar protic solvents (e.g. methanol, ethanol, propanol, butanol, n-butanol, isobutanol, n-procanol isopropanol, and the like) or conceivable mixtures thereof.
The process according to one aspect of the present invention improves the previously reported synthesis of ticlopidine and extends the scope of the synthetic sequence to the structurally related clopidogrel.
The process according to a further aspect of the invention circumvents prior art drawbacks and is exemplified by (but not limited to) the specific reaction sequence displayed in Scheme 7.
The starting materials, 2-bromoacetylthiophene 18 and methyl o-chlorophenylglycinate 19 can be prepared according to known literature procedures as reported in Bull. Chem. Soc. Jpn. 60, 1159 and J. Am. Chem. Soc. 1999, 121, 4284 respectively. Racemic or enantiomerically enriched methyl 2-o-chlorophenylglycine 19 reacts with 2-bromoacetylthiophene 18 in the Presence of a base which acts as an acid scavenger.
The reduction of the methyl N-(2-(2-thienyl)-2-oxoethyl)-o-chlorophenylglycinate 20 to the corresponding alcohol results in a mixture of diastereoisomeric alcohols methyl N-(2-(2-thienyl)-2-hydroxyethyl)-o-chlorophenylglycinate 21. Reaction with formaline provides the 5-(2-thienyl)-3-(methoxycarbonyl-o-chlorobenzyl)-1,3-oxazolidine 22. Rearrangement of the 5-(2-thienyl)oxazolidine 22 derivative occurs in dry HCl/DMF solution to produce the desired methyl xcex1-(7-hydroxy-4,5,6,7-tetrahydro-5-thieno[3,2-c]pyridyl)-o-chlorophenylglycinate 23. Finally, the alcohol is reduced to the racemic 24 or dextrorotatory clopidogrel 2 by direct dehydroxylation, e.g. treatment with stannous chloride under acidic conditions. Alternatively, derivatization of hydroxyl group to a leaving group followed by reduction or stepwise elimination/reduction of the corresponding 4,5-dihydropyridyl intermediate is conducive to the synthesis of racemic 24 or dextrorotatory clopidogrel 2. Treatment with carbonyl diimidazole, heating, followed by sodium borohydride reduction is provided as a non-limiting example.
The use of o-chlorobenzylamine instead of the methyl glycinate 19 in Scheme 7 provides the process for the synthesis of Ticlopidine.
According to a further aspect of the invention, there are provided novel compounds of formulae 22, 21 and 20 which can be made as a racemic or diasteromeric mixture with set configuration at the carbon adjacent to the nitrogen, as well as its enantiomers either as free base or its salts with Brxc3x6nstead and Lewis acids. Specific compounds arising from the preparation of formulae 22, 21 and 20 include:
5R-(2-thienyl)-3-(1R-methoxycarbonyl-o-chlorobenzyl)-1,3-oxazoline
5R-(2-thienyl)-3-(1S-methoxycarbonyl-o-chlorobenzyl)-1,3-oxazoline
5S-(2-thienyl)-3-(1R-methoxycarbonyl-o-chlorobenzyl)-1,3-oxazoline
5S-(2-thienyl)-3-(1S-methoxycarbonyl-o-chlorobenzyl)-1,3-oxazoline
Methyl N-(2R-(2-thienyl)-2-hydroxyethyl)-2R-o-chlorophenylglycinate
Methyl N-(2R-(2-thienyl)-2-hydroxyethyl)-2S-o-chlorophenylglycinate
Methyl N-(2S-(2-thienyl)-2-hydroxyethyl)-2R-o-chlorophenylglycinate
Methyl N-(2S-(2-thienyl)-2-hydroxyethyl)-2S-o-chlorophenylglycinate
Methyl N-(2-(2-thienyl)-2-oxoethyl)-2R-o-chlorophenylglycinate
Methyl N-(2-(2-thienyl)-2-oxoethyl)-2S-o-chlorophenylglycinate

Process for the preparation of tetrahydrothieno[3,2-c]pyridine derivatives

Abstract: or their pharmaceutically acceptable salts, wherein the meaning of X is hydrogen, carboxyl, alkoxycarbonyl, aryloxycarbonyl, nitrile, or carbamoyl of formula

Abstract: wherein R1 and R2 can be individually or simultaneously hydrogen, alkyl or part of a heterocyclic structure; Z can be hydrogen, halogen, alkyl, aryl, aryloxy or alkoxy group.


A method for updating biometric reference data for a person is provided. The biometric reference data is defined as a multi-template formed by a number of templates each representing an area of an associated biometric object of the person. The method comprises providing at least one template associated with a biometric image that depicts at least one area of a finger of the person. Moreover, the method comprises matching the at least one template to the multi-template of the biometric reference data and in the event of a match, updating the multi-template of the biometric reference data with at least one selected previously matched template stored in a candidate template queue. Furthermore, the method comprises storing the matched template in the candidate template queue.

Use of small biometric sensors, i.e. sensors that do not collect biometric data about the whole biometric object, may be desired due to constraints caused by e.g. physical design of an apparatus used to collect the biometric data. Due to the small size of the biometric sensor in view of the whole biometric object, e.g. a fingerprint, normally several biometric templates are required to be stored or reconstructed as reference or multi-template data for subsequent matching. Since the biometric object may change over time, e.g. due to ageing, environmental impacts etc., it may be advantageous to update the reference data. Some known solutions allow for a so called automatic updating of the multi-template or reference data with data relating to each matched biometric template. Such an automatic update is associated with the risk that an imposter's template in the event of an erroneous successful match, would end up in the reference multi-template.
Hence, it would be advantageous to provide an improved method of updating a biometric multi-template or reference data.

Method for updating biometric reference data


Software is down-loaded from a central station of a wireless telecommunications system to a remote subscriber station for configuring the remote subscriber station to permit wireless connection of user telecommunications equipment at the remote subscriber station to the central station. A multi-layer down-load protocol includes a number of independent protocol layers, preferably operating a master-slave configuration. Each layer controls respective sequence numbers to ensure system integrity. Control software is arranged with a device independent boot-strap and a set of device specific external service parameters to provide portability.

A wireless telecommunications system has been proposed with a central terminal, or station, at a central location in wireless communication with a plurality of subscriber terminals, or stations, at subscriber locations to implement a wireless telephony system. The system is intended to be used with fixed subscriber locations rather than the more familiar mobile cellular telephone systems.
The system finds a wide variety of possible applications, for example in rural, remote, or sparsely populated areas where the cost of laying permanent wire or optical networks would be too expensive, in heavily built-up areas where conventional wired systems are at full capacity or the cost of laying such systems would involve too much interruption to the existing infrastructure or be too expensive, and so on.
In the proposed system, the subscriber terminal is required to perform wireless transmission and reception tasks, signal coding and decoding tasks, initialisation and authentication tasks as well conventional telephony tasks. Function for executing these tasks can be set up at the initial installation of the subscriber unit at a customer's location. However, it may be desired to update the functions available at the customer's premises after installation. Particularly bearing in mind that the subscriber stations may be widely scattered, it is inconvenient to have to visit each subscriber station in order to provide for re-programming of the subscriber station.
It may be desirable in some instances to ensure that standard installation is effected for all subscriber stations. In other instances, it may be desirable to provide different installations at different subscriber stations. The administration in ensuring that installers have the correct equipment at subscriber stations is significant. Accordingly, it can be seen that the programming and installation of subscriber stations is a potentially costly and inconvenient operation. The invention seeks to address these problems.

Software download for a subscriber terminal of a wireless telecommunications system


A filter system for filtering a fluid is provided with a filter housing that has a housing top part and a housing bottom part arranged along a housing axis of the filter housing. An inlet socket is arranged at the housing top part for supplying a fluid to be filtered. An outlet socket is arranged at the housing top part for discharging the fluid. A filter element is provided with a filter bellows arranged about a longitudinal axis of the filter element. The filter element, installed inside the filter housing, fluidically separates a raw fluid side from a clean fluid side. A central element is attached to the filter housing and extends along the housing axis. The central element is separate from the filter element and projects into an interior of the filter element. A strainer filter is arranged at the central element.

DE 10 2013 009 198 A1 discloses a fuel filter with a filter housing and with an element-associated electrical connecting element and with a housing-associated electrical connecting element, in which the element-associated connecting element and the housing-associated connecting element are electrically connected to each other when the filter element is installed in the filter housing. In the filter housing, a guiding ramp with a ramp surface is arranged. Upon insertion of the filter element into the filter housing in axial direction relative to the filter axis with the connecting end face leading, the element-associated connecting element and another section of the connecting end face hit the ramp surface. Upon rotation of the filter element in the filter housing about the filter axis in a rotational direction which corresponds to the screw-like course of the ramp surface and upon simultaneous movement of the filter element in axial direction toward the connecting end face of the filter housing, the filter element is guided along the ramp surface. The housing-associated electrical connecting element is arranged at the end of the ramp surface. The element-associated connecting element connects then electrically with the housing-associated connecting element. In this way, the filter element is positioned in the correct installation position in the filter housing. Closing of the electrical connection is realized by the rotation/insertion movement of the filter element.
DE 10 2015 012 473 A1 discloses a filter element that comprises a hollow cylindrically shaped filter medium body having a central element with strainer filter fastened in its interior. The central element may comprise in particular a water level sensor which comprises a water detection pin at a first end and comprises contacts for connection to a measuring circuit at its second end. The central element with strainer filter is configured to be rotatable relative to the filter medium body and, at the end with the contacts, catching elements are provided that enable a contacting action with a cover-associated counter contact when screwing shut a filter housing. A disadvantage is herein that the central element and the strainer filter, which constitute from experience a major portion of the costs of such a filter element, must be replaced with every filter element change.

Filter system with central element and strainer filter


The lithium secondary battery positive electrode provided by the present invention has a positive electrode collector and a positive active material layer formed on the collector. The positive active material layer is composed of a matrix phase containing at least one particulate positive active material and at least one binder, and an aggregate phase dispersed in the matrix phase, constituted by aggregation of at least one particulate positive active material and containing substantially no binder.

In recent years, lithium secondary batteries (typically lithium-ion batteries), nickel hydrogen batteries and other secondary batteries have gained importance as vehicle-mounted power sources or power sources for personal computers and handheld devices. Lithium secondary batteries hold particular promise as high-output vehicle-mounted power sources because they provide high energy densities with low weight.
First and foremost, lithium secondary batteries used as motor drive power sources in electric vehicles (EV), hybrid vehicles (HV), plug-in hybrid vehicles (PHV) and other vehicles must be suited to high-rate (such as 10 C or more) charge and discharge. One way of satisfying this first requirement is to use a fine particulate compound for the positive active material. Recently, fine particulate positive active materials with an average primary particle diameter of less than 1 μm have come to be used. Such a fine particulate positive active material is suited to high-rate charge and discharge because it has a relatively large specific surface area. For example, Patent Document 1 below describes a positive active material for a lithium secondary battery, which is a particulate positive active material consisting of a composite metal oxide containing lithium and manganese, wherein the percentage of particles that remain in a state of primary particles without forming secondary particles is more than half of the total of all the composite metal oxide particles.
The second requirement of a lithium secondary battery used as a motor drive power source is high durability. That is, vehicular batteries are used over a long period of time while being charged and discharged at a high rate (with high output) under severe environmental conditions, which may include extreme temperature changes (such as low temperatures below −20° C. and high temperatures above 60° C.). Thus, they must be durable enough that the increase in internal resistance of the battery is controlled under such conditions of use. One way of satisfying this second requirement is to support the positive active material particles at high densities at a specific location (that is, in the positive active material layer) on the positive collector. An effective way of doing this is to raise the (percentage) content of binder in the positive active material layer.
However, when the (percentage) content of binder is increased, the (percentage) content of the positive active material is decreased proportionally, reducing the capacity of the battery. For example, in the technology described in Patent Document 1 above, it is expected that a large quantity of binder will be required so that the particulate positive active material, more than half of which consists of primary particles, does not peel (separate) from the positive electrode collector.
Patent Document 2 below discloses a secondary battery positive electrode, wherein the primary particles constituting the positive active material are bound with a water-soluble polymer binder to thereby form aggregates (secondary particles) of linked primary particles, and these secondary particles are then bound with each other and to the positive electrode collector with a fluororesin binder or rubber-based binder. The binding force of the positive active material layer is described as being improved with this configuration, but relatively large quantities of at least two different binders are required. Another document of prior art of this type is Patent Document 3 below for example. This document describes obtaining improved electron conductivity of the positive active material particles by mixing carbon fiber with the particulate lithium phosphoric acid transition metal compound constituting the positive active material.    Patent Document 1: Japanese Patent Application Laid-open No. 2003-203632    Patent Document 2: Japanese Patent Application Laid-open No. 2007-234277    Patent Document 3: Japanese Patent Application Laid-open No. 2008-117749

Lithium secondary battery and positive electrode for the battery


A dictionary order sorter resorts character strings of recognition candidates stored in a high-ranking candidate memory in the order of distance into a dictionary order (character code order). Upon receipt of a sort termination signal a display controller displays the character strings of recognition candidates stored in the high-ranking candidate memory in the dictionary order and their ranking numbers in order on a display. Where an attribute-dependent sorter is provided in place of the dictionary order sorter, the character strings of recognition candidates stored in the distance order in the high-ranking candidate memory are sorted (grouped) according to attribute information of categories, such as parts of speech, concepts, etc., which are stored in a template memory for each of the recognition candidates and then displayed classified into groups. The categories of attribute information can arbitrarily be specified by the user. The user can easily select the correct recognition candidate from among the recognition candidates displayed rearranged in a fixed order of priority as described above.

1. Field of the Invention
The present invention relates to a recognition apparatus, such as a voice recognition apparatus, a character recognition apparatus, etc., which displays a plurality of recognition candidates obtained by recognition processing and thereby permits a user to select a recognition result from among the recognition candidates.
2. Description of the Related Art
In a voice recognition apparatus, such as a large-vocabulary voice recognition system, etc., a recognition candidate which is best in recognition score is not necessarily right because of the performance limit of the voice recognition system used. For this reason, use is made of a man-machine system arranged such that high-ranking candidates which are high in recognition score are displayed on a display unit and the user selects the right candidate out of the displayed candidates.
FIG. 1 illustrates a conventional large-vocabulary voice recognition system.
A voice to be recognized is entered into a microphone 101 and then digitized by an A/D converter 102. The digitized voice is subjected to frequency analysis in a frequency analyzer 103 for conversion into a time-series pattern of parameters representing the frequency characteristic of a phoneme of the voice. The parameters include, for example, powers of frequency components, LPC coefficients resulting from linear prediction analysis and cepstrum coefficients resulting from cepstrum analysis.
In a DTW matching section 104, the time-series pattern is matched against the time-series pattern of each word stored in a template memory 105 by means of the dynamic time-warping method. The matching section carries out the comparison between the two time-series patterns while normalizing (expanding and contracting) their time bases on the basis of a dynamic programming technique. Thereby, a distance value ( for example, the Euclidean distance between the parameters) between the input voice and each word stored in the template memory 105 is calculated.
These distance values are sorted (rearranged) in the order of distance in a distance order sorter 106, so that distance values taking predetermined high ranks beginning with the smallest and information on character trains of the corresponding word candidates are stored in a high-ranking candidate memory 107.
If word candidates which take, for example, the first to thirtieth ranks are stored in the high-ranking candidate memory 107, the distance order sorter 106 will execute such operations as indicated in steps S2 to S5 of FIG. 2.
That is, first, in step S1, the contents of the high-ranking candidate memory 107 are initialized.
Next, in step S2, a character string of a word candidate and a distance value are read from the DTW matching section 104. When NO in step S3, the distance value is written into the 31st address area of the high-ranking candidate memory 107 in step S4.
In step S5, the distance value which has been written into the 31st address area is sequentially compared with distance values of word candidates which had been written into the 1st to 30th address areas of the high-ranking candidate memory 107 in ascending order of distance values when the 31th address area was written into, so that the distance values stored in the 1st to 31st address areas are sorted.
The above steps S2 to S5 are repeated until the distance values for all the word candidates have been read from the DTW matching section 104, namely, until the determination result "YES" is obtained in step S3.
A sort termination signal is sent from the distance order sorter 106 to a display controller 108 at the termination of sort processing in the order of distance. Upon receipt of the sort termination signal the display controller 108 displays the character strings of word candidates stored in the high-ranking candidate memory 107 in the distance order and their ranking numbers on the display unit 109. In the above case, for example, the display controller 108 displays a character string of each of the word candidates stored in the 1st to 30th address areas of the high-ranking candidate memory 107 as the process in step S2 in FIG. 2. On termination of the display, the display controller 108 sends a display termination signal to a word select controller 110.
Upon receipt of the display termination signal, the word select controller 110 accepts the ranking numbers of word candidates entered by the user from a keyboard 111 or a mouse 112, which is a pointing device, and then outputs character strings of the correct words corresponding to the entered ranking numbers to another device, an application program, etc., which are not shown in particular.
FIG. 3 illustrates a first display example of word candidates displayed on the display unit 109 (see FIG. 1 ) in the above-described conventional voice recognition system. As can be seen from the figure, even if the user wants to specify the word "OOSAKA" corresponding to the user's utterance from among the displayed word candidates, it will be very difficult for the user to search for the object word. As described above, the problem with the first prior art is that it takes long for the user to search for the correct answer. This will increase psychological burden and stress imposed on the user.
Next, FIG. 4 illustrates a second display example of word candidates displayed by the display unit 109 in the above-described conventional voice recognition system. When utterances corresponding to a Japanese sentence "ANATAHA HONO YONDEIMASUKA" are entered from the microphone 101, the system of FIG. 1 performs a recognition process continuously. In the high-ranking candidate memory 107 are stored character strings of word candidates of each of the words composing the input sentence and their ranking numbers in ascending order of distance. Upon receipt of a sort termination signal indicating the termination of sorting of the input words from the distance order sorter 106, the display controller 108 displays character strings of word candidates taking the first to eighth ranks, their ranking numbers and their distance values for each word as shown in FIG. 4. Hereinafter, such a combination of word candidates displayed in the form of a table for plural words is referred to as a word candidate lattice. Here, figures within parentheses for each word candidate represent its distance value (recognition similarity). The smaller the value, the higher is the probability of being correct.
However, the problem with the second display example of FIG. 4 is that it will take longer than in the first display example of FIG. 3 for the user to search the displayed word candidate lattice for the word string, namely, the right answer of the input sentence.
The above-described problems arise not only in a voice recognition system but also in a character recognition system. FIG. 5 illustrates a conventional character recognition system.
A string of characters written by the user on an input tablet 501 is digitized first and then sequentially entered into a feature amount extractor 502 where the feature amount of each character is extracted for conversion into a feature vector pattern for each word.
A matching section 503 makes a comparison between the feature vector pattern of each of the characters and the feature vector pattern of each of characters stored in a template memory 504 while normalizing the size of characters. Thereby, a value of distance between each character stored in the template memory 504 and each input character, so that a plurality of character candidates are obtained for each input character.
The subsequent operations are the same as in the voice recognition system of FIG. 1.
FIG. 6 illustrates a display example of character candidates displayed by the display unit 109 (FIG. 5) in the conventional character recognition system. In this display example, a character candidate lattice in which a plurality of character candidates for each input character are displayed in the form of a table as in FIG. 4.
As can be seen from the display example of FIG. 6, however, the problem with the conventional character recognition system is that it takes a long time for the user to search the displayed character candidate lattice for the right answer of the input character string.

Recognition apparatus with function of displaying plural recognition candidates


The present invention involves a method and an apparatus for analyzing measured signals, including the determination of a measurement of correlation in the measured signals during a calculation of a physiological parameter of a monitored patient. Use of this invention is described in particular detail with respect to blood oximetry measurements.

1. Field of the Invention
The present invention relates to the field of signal processing. More specifically, the present invention relates to the processing of measured signals, containing a primary signal portion and a secondary signal portion, for the removal or derivation of either the primary or secondary signal portion when little is known about either of these components. More particularly, the present invention relates to modeling the measured signals in a novel way which facilitates minimizing the correlation between the primary signal portion and the secondary signal portion in order to produce a primary and/or secondary signal. The present invention is especially useful for physiological monitoring systems including blood oxygen saturation systems.
2. Description of the Related Art
Signal processors are typically employed to remove or derive either the primary or secondary signal portion from a composite measured signal including a primary signal portion and a secondary signal portion. For example, a composite signal may contain noise and desirable portions. If the secondary signal portion occupies a different frequency spectrum than the primary signal portion, then conventional filtering techniques such as low pass, band pass, and high pass filtering are available to remove or derive either the primary or the secondary signal portion from the total signal. Fixed single or multiple notch filters could also be employed if the primary and/or secondary signal portion(s) exist at a fixed frequency(s).
It is often the case that an overlap in frequency spectrum between the primary and secondary signal portions exists. Complicating matters further, the statistical properties of one or both of the primary and secondary signal portions change with time. In such cases, conventional filtering techniques are ineffective in extracting either the primary or secondary signal. If, however, a description of either the primary or secondary signal portion can be derived, correlation canceling, such as adaptive noise canceling, can be employed to remove either the primary or secondary signal portion of the signal isolating the other portion. In other words, given sufficient information about one of the signal portions, that signal portion can be extracted.
Conventional correlation cancelers, such as adaptive noise cancelers, dynamically change their transfer function to adapt to and remove portions of a composite signal. However, correlation cancelers require either a secondary reference or a primary reference which correlates to either the secondary signal portion only or the primary signal portion only. For instance, for a measured signal containing noise and desirable signal, the noise can be removed with a correlation canceler if a noise reference is available. This is often the case. Although the amplitude of the reference signals are not necessarily the same as the amplitude of the corresponding primary or secondary signal portions, they have a frequency spectrum which is similar to that of the primary or secondary signal portions.
In many cases, nothing or very little is known about the secondary and/or primary signal portions. One area where measured signals comprising a primary signal portion and a secondary signal portion about which no information can easily be determined is physiological monitoring. Physiological monitoring generally involves measured signals derived from a physiological system, such as the human body. Measurements which are typically taken with physiological monitoring systems include electrocardiographs, blood pressure, blood gas saturation (such as oxygen saturation), capnographs, other blood constituent monitoring, heart rate, respiration rate, electro-encephalograph (EEG) and depth of anesthesia, for example. Other types of measurements include those which measure the pressure and quantity of a substance within the body such as cardiac output, venous oxygen saturation, arterial oxygen saturation, bilirubin, total hemoglobin, breathalyzer testing, drug testing, cholesterol testing, glucose testing, extra vasation, and carbon dioxide testing, protein testing, carbon monoxide testing, and other in-vivo measurements, for example. Complications arising in these measurements are often due to motion of the patient, both external and internal (muscle movement, vessel movement, and probe movement, for example), during the measurement process.
Many types of physiological measurements can be made by using the known properties of energy attenuation as a selected form of energy passes through a medium.
A blood gas monitor is one example of a physiological monitoring system which is based upon the measurement of energy attenuated by biological tissues or substances. Blood gas monitors transmit light into the test medium and measure the attenuation of the light as a function of time. The output signal of a blood gas monitor which is sensitive to the arterial blood flow contains a component which is a waveform representative of the patient's arterial pulse. This type of signal, which contains a component related to the patient's pulse, is called a plethysmographic wave, and is shown in FIG. 1 as curve s. Plethysmographic waveforms are used in blood gas saturation measurements. As the heart beats, the amount of blood in the arteries increases and decreases, causing increases and decreases in energy attenuation, illustrated by the cyclic wave s in FIG. 1.
Typically, a digit such as a finger, an ear lobe, or other portion of the body where blood flows close to the skin, is employed as the medium through which light energy is transmitted for blood gas attenuation measurements. The finger comprises skin, fat, bone, muscle, etc., shown schematically in FIG. 2, each of which attenuates energy incident on the finger in a generally predictable and constant manner. However, when fleshy portions of the finger are compressed erratically, for example by motion of the finger, energy attenuation becomes erratic.
An example of a more realistic measured waveform S is shown in FIG. 3, illustrating the effect of motion. The primary plethysmographic waveform portion of the signal s is the waveform representative of the pulse, corresponding to the sawtooth-like pattern wave in FIG. 1. The large, secondary motion-induced excursions in signal amplitude obscure the primary plethysmographic signal s. Even small variations in amplitude make it difficult to distinguish the primary signal component s in the presence of a secondary signal component n.
A pulse oximeter is a type of blood gas monitor which non-invasively measures the arterial saturation of oxygen in the blood. The pumping of the heart forces freshly oxygenated blood into the arteries causing greater energy attenuation. As well understood in the art, the arterial saturation of oxygenated blood may be determined from the depth of the valleys relative to the peaks of two plethysmographic waveforms measured at separate wavelengths. Patient movement introduces motion artifacts to the composite signal as illustrated in the plethysmographic waveform illustrated in FIG. 3. These motion artifacts distort the measured signal.

This patent is subject to a terminal disclaimer.


A method and apparatus for comparing an infrared image of a person to a database of visual images of persons and calculating the probability that each is a match to the infrared image is characterized by extracting minutiae from the infrared image and extracting visible minutiae from the visible images. Coincident minutiae which occur in both spectra are used to scale and register the infrared and the visible images. Other minutiae are spectrum-dependent, but must obey rules relative to minutiae of the other spectrum, due to the anatomical structure of the human face and body. The primary application is for identification of persons seen in infrared surveillance imagery, using a reference database of visual images. Other applications include compression of talking head video and animation of synthetic faces. The method and apparatus can also be applied to areas of the body other than the face, to compare images from different spectra including images from medical sensors.

There are common features between infrared (IR) and visual images of the human body. Using the face as an example, head shape and size, and the relative location, shape, and size of features such as the eyes, mouth, and nostrils are the same in both imaging modes. A database of images can be segmented into classes using metrics derived from those common features, and the same classification will be obtained from either visual or infrared images. Height can be also used as a classification measure when it can be inferred from the collected image or from separate sensor data. An infrared image of an unclothed area of the body, such as the face, presents much more detailed and person-specific information than does a visible image. However, visible images are more commonly collected and large historical databases of visual images exist. It is therefore desirable to automate a process for comparing imagery from both the visual and infrared modes.
Infrared images are unique to each person, even for identical twins. Visual images are not unique because many people look similar and can disguise themselves to look enough like one another that an automated identification system cannot distinguish them. Therefore, in a large database, it is not possible to automatically perform a one-to-one linkage between infrared and visual images because the visual images are not sufficiently unique. However, for each infrared image, an automated system can eliminate all visual images which cannot be a match due to insufficient correspondence between minutiae characteristics. In general, it is estimated that more than 95% of a visual database can be eliminated as a match to a given infrared image. This has application to the use of infrared surveillance cameras to identify wanted persons for whom only visual images are on file. The infrared-visual matching system compares each person it sees in infrared and classifies him as either a possible match to someone on the visual image watch list or not a match. Persons who are possible matches can then receive greater attention from immigration or security authorities. This allows the use of infrared surveillance imagery to proceed without waiting until a large database of infrared images is established.
The use of infrared imagery also provides for the detection of disguises, whether worn or surgical, which may not be detectable from visible imagery. For example, artificial facial hair such as a mustache is readily detectable in an infrared image although it appears natural in visible images. The fact that infrared surveillance imagery shows a man with a fake mustache provides a clue to consider in matching against a visible image database. Surgical disguises such as a face lift leave telltale short and longer term variations in the facial thermogram, while the visual image may appear to be a different person and show no sign of surgery, The ability to detect in IR images that surgical changes have been made to a particular area of the face permits an automated system to broaden the parameters for searching for possible matching visual images in an historical database.
High definition visual images of the face and body are routinely produced and stored for medical, diagnostic and forensic use. Common examples are the photographing of criminal suspects through booking stations producing xe2x80x9cmug shotsxe2x80x9d, driver""s license photographs produced by each state, and passport photos used by the State Department. Many such large facial image databases exist, in hardcopy and in electronic form, and there is increasing research ongoing into automated matching of newly taken images with those databases. For example, there are frequent attempts to match surveillance images of a person using a stolen credit card at an ATM with photographs of persons previously convicted of similar crimes.
Visual imagery, particularly from surveillance cameras, is often of poor quality due to dim illumination at the scene. Low light level or infrared cameras are expected to become more widely used for surveillance as their cost diminishes. There is therefore a need to correlate between newly acquired infrared images and existing databases of video images. Even in the future, when simultaneous collection of video and IR images will generate correlated databases, there will always be a need to match images taken in one spectral domain with images taken in another. This can include matching images taken in one IR band (such as 3-5 micron) with images taken in another IR band (such as 8-12 micron).
Since IR cameras are passive, emitting no radiation and therefore presenting no health hazards, they may be used in conjunction with other imaging medical devices such as x-ray, sonogram, CAT scan devices, etc. Minutiae derived from the IR image may then be superimposed or annotated onto the resulting medical image. This presents a standard technique for generating standardized reference points on all medical imagery. Subsequently, the method and apparatus of this invention can be used to search a database of annotated medical images to find a match with a current IR image or current medical image annotated with IR minutiae.
Regions of Interest (ROI) may be utilized instead of minutiae, where the ROI may be elemental or other shapes including fractal or wavelet-derived structures, segments of blood vessels, locations underneath or otherwise relative to tattoos, moles, freckles, or other distinguishable features, or wiremesh or finite elements used for thermodynamic or visible modeling of the body. Rules may relate the shapes and positions of such elements, their centroids and other features. Time sequences of minutiae and ROIs may be compared, with the decision as to a possible match made on the basis of cumulative thresholds and rule tolerances over the sequence.
Facial expression and speech modeling has application to synthetic videoconferencing and face animation. Substantial bandwidth and storage reduction can result. Use of IR minutiae offers more precise modeling than current use of visual images. The present invention provides a technique by which IR images can be tied to the visual image being displayed.
The identification of persons from infrared images is known in the art as evidenced by the Prokoski et al U.S. Pat. No. 5,163,094 which discloses a method and apparatus for analyzing closed thermal contours, called xe2x80x9celemental shapesxe2x80x9d which are created by the vascular system interacting with the anatomical structure. Fifty or more elemental shapes can be identified for example in a human face imaged with an IR camera which has an NETD (noise equivalent thermal difference) of 0.07xc2x0 C. and a spatial resolution of 256xc3x97256 pixels. Characteristics of those shapes, such as the centroid location and ratio of area to perimeter, remain relatively constant regardless of the absolute temperature of the face, which varies with ambient and physiological conditions. Two infrared images are compared by comparing the characteristics of corresponding shapes. A distance metric is defined and calculated for each pair of images. If the value is within a threshold, the two images are considered to be from the same person.
In the Prokoski et al U.S. patent application Ser. No. 08/514,456, there is disclosed a method and apparatus for extracting and comparing thermal minutiae corresponding to specific vascular and other subsurface anatomical locations from two images. Minutiae may be derived from thermal contours, or may be absolutely associated with specific anatomical locations which can be seen in the thermal image, such as the branching of blood vessels. Each minutia is then associated with a relative position in the image and with characteristics such as apparent temperature, the type of branching or other anatomical feature, vector directions of the branching, and its relation to other minutiae.
The comparison of thermal minutiae from two facial images is analogous to the comparison of sets of fingerprint minutiae, in that two images are said to identify the same person if a significant subset of the two sets are found to correspond sufficiently in relative positions and characteristics. Classification of the facial thermograms can be performed to partition a database and reduce the search for matching facial patterns. Alternately, encoding of the minutiae patterns offers a unique FaceCode which may be repeatably derived from each person, minimizing the need for searching a database.
Infrared imaging can be used to locate minutiae points over the entire body surface which correspond to specific anatomical locations such as intersection points and branch points of the underlying blood vessels. The thermal minutiae technique and apparatus utilizes a built-in set of whole-body registration points viewable in IR on the face and body surface. The registration points can then be used to compare infrared images taken with different equipment at different times of different people and under different conditions to facilitate comparison of those images.
The IR camera is totally passive, emitting no energy or other radiation of its own, but merely collecting and focusing the thermal radiation spontaneously and continuously emitted from the surface of the human body. Current IR cameras operating in the mid to long wavelength region of 3-12 microns, record patterns caused by superficial blood vessels which lay up to 4 cm below the skin surface. Future cameras will have increased sensitivity which will translate into even more defined minutiae. With current IR cameras, approximately 175 thermal facial minutiae may be identified in thermal images from superficial blood vessels in the face. More than 1000 thermal minutiae may be identified over the whole body surface. Using more sensitive infrared cameras, additional minutiae from deeper vascular structures may be identified in the thermal images.
The normal body is basically thermally bilaterally symmetric. Side to side variations are typically less than 0.25 degrees Celsius. This fact is used in assigning axes to the body""s image. Where the skin surface is unbroken, there is a gradual variation of temperatures across blood vessels, with the highest temperatures across the body surface being directly on top of major blood vessels. Major thermal discontinuities occur at entrances to body cavities such as the eye sockets, nostrils, or mouth. These provide global reference points for automatic orientation of the thermal image. Local and relatively minor discontinuities in the skin surface occur at scars, moles, burns, and areas of infection. The thermal surface can be distorted through pressures and activities such as eating, exercising, wearing tight hats and other clothing, sinus inflammation, infection, weight gain and loss, and body position. However, the minutiae points remain constant with respect to their position relative to the underlying anatomy.
The technique for thermal minutiae extraction and matching can be summarized as follows:
1. The current thermal image is digitized.
2. The current image is divided into pixels, where the size of the pixel relates to the resolution or quality of the result desired.
3. Certain pixels are selected as minutiae points.
4. Each minutia is assigned characteristics such as one or more vectors having magnitude and directional information in relation to the surrounding areas of the thermal image about that minutia, absolute or relative temperature at or around the minutia location, shape of the surrounding thermal area or areas, curvature of the related shape or shapes, size of the surrounding shape or shapes, location of the minutia relative to the body, distance to other minutiae, vector length and direction to other minutiae, number of crossings of thermal contours between it and other minutiae, number of other minutiae within a certain range and direction, the type of minutiae such as the apparent end point of a blood vessel, a point of maximum curvature of a thermal contour, all points on an anatomical element such as a blood vessel which can be distinguished by thresholding or range gating or focusing the thermal camera or image, the centroid of a lymph node, or the centroid or other reference of an anatomical structure with distinguishing thermal capacitance. Either active or passive infrared imaging can be used. For active imaging, the subject can be subjected to heat or cold by external application of hot or cold air, illumination, dehumidification, ingestion of hot or cold foodstuffs, or ingestion of materials which cause vasodilation or vasoconstriction.
5. A set of minutiae characteristics of the current image is compared by computer to the set of minutiae characteristics of other images.
6. The comparison results are used to determine corresponding minutiae from the two images, and to morph or mathematically adjust one image with respect to the other to facilitate comparison.
7. The differences between the current image and database images are computed for the entire image or for areas of interest.
8. The differences are compared to a threshold and image pairs which exceed the threshold are considered impossible matches.
Infrared facial minutiae maybe derived from elemental shapes (such as by using the centroids of each shape or the zero locations resulting from wavelet compression and expansion). Particularly when high quality infrared images are used, absolute minutiae can be directly extracted without the computationally intensive analysis required for template or shape comparisons.
It is also known in the prior art to compare visible images through fiducial points involving definition of face metrics which may be considered to have aspects in common with the present invention. For example, the Tal U.S. Pat. No. 4,975,969 discloses a method and apparatus for uniquely identifying individuals by measurement of particular physical characteristics viewable by the naked eye or by imaging in the visible spectrum. Tal defined facial parameters which are the distances between identifiable parameters on the human face, and/or ratios of the facial parameters, and used them to identify an individual since he claims that the set of parameters for each individual is unique. Particular parameters such as the distance between the eye retina, the distance from each eye retina to the nose bottom and to the mouth center, and the distance from the nose bottom to the mouth center are set forth, as they may be particularly defined due to the shadowed definable points at each end.
The approach disclosed in the Tal patent utilizes visible features on the face from which a unique set of measurements and ratios allegedly can be developed for each individual. This approach is not particularly satisfactory, nor does it pertain to identical twins. In addition, the xe2x80x9crubber sheetingxe2x80x9d effect caused by changes in facial expression, the aging effects which cause lengthening of the nose, thinning of the lips, wrinkles, and deepening of the creases on the sides of the nose, would all cause changes in the parameters and in their ratios. Therefore, very few measurements which can be made on a human face are constant over time, and the paucity of such constant measurements makes it improbable that facial metrics in visible images can be useful for identification of sizable populations. The Tal patent does not deal with comparison of images from other than visible detectors, and so does not consider the specific focus of the present invention which is the comparison of images from different spectral bands. Moreover, the Tal patent does not specifically caution about varying lighting conditions, which could severely limit the utility of the technique, even for classification.
Visible face metrics may be useful as a classification technique, but the visible features can be modified cosmetically or surgically without detection, resulting in misclassification. By contrast, the technique of the present invention utilizes hidden micro parameters which lie below the skin surface, and which cannot be forged. The current patent""s use of underlying features which are fixed into the face at birth and remain relatively unaffected by aging provides for less inherent variability in the values of the parameters over time than is provided by the prior art.
Visible metrics require ground truth distance measurements unless they rely strictly upon ratios of measurements. They can be fooled by intentional disguises, and they are subject to variations caused by facial expressions, makeup, sunburns, shadows and similar unintentional disguises. Detecting disguises and distinguishing between identical twins may or may not be possible from visible imagery if sufficient resolution and controlled lighting is available. However, the level of resolution which may be required significantly increases the computational complexity of the identification task, and makes the recognition accuracy vulnerable to unintentional normal variations.
The use of eigenanalysis of visual faces to develop a set of characteristic features is disclosed in Pentland (MIT Media Laboratory Perceptual Computing Section, Technical Report No. 245 View-Based and Modular Eigenspaces for Face Recognition). Faces are then described in terms of weighting of those features. The approach claims to accommodate head position changes and the wearing of glasses, as well as changes in facial expressions. A representative sample of 128 faces was used from a database of 7,562 images of approximately 3000 people. A principal components analysis was performed on a representative sample. The first 20 eigenvectors were used. Each image was annotated by hand as to sex, race, approximate age, facial expression, etc. Pentland does not deal with comparing images from different spectral bands. Nor does his technique perform well in the case of visible images obtained under differing lighting conditions.
Pentland discloses that pre-processing for registration is essential to eigenvector recognition systems. The processing required to establish the eigenvector set is extensive, especially for large databases. Addition of new faces to the database requires the re-running running of the eigenanalysis. Pentland and other xe2x80x9ceigenfacexe2x80x9d approaches are database-dependent and computationally intensive. In contrast, the proposed minutiae comparison of the present invention is independent of the database context of any two images. Minutiae are directly derived from each image, visible or IR, and compared using fixed rules, regardless of the number or content of other images in the database.
An approach for comparing two sets of image feature points to determine if they are from two similar objects is disclosed in Sclaroff (Sclaroff and Pentland: MIT Media Laboratory, Perceptual Computing Technical Report #304). He suggests that first a body-centered coordinate frame be determined for each object, and then an attempt be made to match up the feature points. Many methods for finding a body-centered frame have been suggested, including moment of inertia methods, symmetry finders, and polar Fourier descriptors. These methods generally suffer from three difficulties: sampling error; parameterization error; and non-uniqueness.
Sclaroff introduces a shape description that is relatively robust with respect to sampling by using Falerkin interpolation, which is the mathematical underpinning of the finite element method. Next, he introduces a new type of Galerkin interpolation based on Gaussians that allow efficient derivation of shape parameterization directly from the data. Third, he uses the eigenmodes of this shape description to obtain a canonical, frequency-ordered orthogonal coordinate system. This coordinate system is considered the shape""s generalized symmetry axes. By describing feature point locations in the body-centered coordinate system, it is straight-forward to match corresponding points, and to measure the similarity of different objects.
Applicant has previously utilized a principal components analysis of thermal shapes found in facial thermograms. The resulting accuracy of 97% from IR images equals or surpasses the results reported by Pentland with visible facial images. Applicant""s training database, furthermore, included identical twins and involved non-cooperative imaging of about 200 persons. Thus, the head sizes and orientations were not pre-determined as they were in the Pentland study. As a result, the use of eigenanalysis of thermal shapes is more robust than the use of eigenanalysis of visual facial features. However, the basic requirements of eigenanalysis still pertain to their use in matching of thermal images by consideration of inherent elemental shapes. That is, the approach is computationally intensive, requires a pre-formed database, and requires standardization of the images through pre-processing.
The present invention differs from prior visible and IR recognition approaches in that it does not merely sample a finite number of points on an image grid; it extracts points which have particular meaning in each spectrum and automatically distinguishes between cross-spectrum minutiae which are coincident and those which are related by rules associated with anatomical bases. It assigns a difference or feature space distance to each pair of coincident minutiae, with a total distance calculated over all such pairs. This first step may be used to eliminate candidate matches which produce distances above a threshold. Then the spectrum-dependent minutiae are compared relative to anatomical rules to further eliminate impossible candidate matches. The prior art has not addressed alignment and comparison of visual/IR or IR/IR human images based upon anatomical rules and the characteristics of features viewable in the IR image.
It is a primary object of the present invention to provide a method and apparatus for identifying visual images which may be a match to infrared images of faces or bodies. A thernal image of a portion of the individual""s body is generated and is processed to produce a set of minutiae points, together with characteristics which describe each such point and its relation to other minutiae. That combination of minutiae and characteristics is considered unique to the individual and essentially persistent in spite of ambient, physiological, emotional, and other variations which occur on a daily basis. Any portion of the body can be utilized, but the face is preferred due to its availability. Since parts of the face may be blocked by glasses, facial hair, or orientation to the sensor, such as a camera, the system and method allows for identification based on partial faces.
Candidate visual images are processed to extract minutiae characteristic of the subject and the visual spectrum. The IR and visual images are scaled to the same standard and aligned based upon minutiae which are coincident in the two spectra. A measure of the amount of warping required to accomplish the alignment is calculated. Then other spectrum-dependent minutiae are compared, with relation to certain rules which would be met if the two images were of the same person, based upon anatomical structures of the human face and body. A measure of the degree of compliance with the rules is calculated. The decision to include or exclude a given visual image from the class of possible matching images to the infrared image is made based upon these measures relative to thresholds which are established to control possible errors in the system.
Just as locating the center of a fingerprint is essential to certain fingerprint matching algorithms, establishing axes for the facial minutiae is also essential. In an interactive system, human operators establish face axes, similar to fingerprint examiners setting the orientation of latents. A human demarcates the eye pupils, canthi and/or nostrils by manipulating a cursor on the system display. Axes are then automatically generated vertically through the center of mass of the eye pupils or canthi and nostrils and horizontally through the pupils or canthi centroids. If the axes are not perpendicular, the vertical axis can be adjusted to not necessarily bisect the nostrils. The human operator also indicates any unusual features, such as a missing eye or eye patch, wearing of bandages, tattoos, deformation of the lips or other visible gross thermal asymmetries of the face. An automated system can perform these as well.
The unknown face is partitioned into segments, and corresponding segments matched. This will accommodate matching of partial faces when faces are partially disguised or hidden behind other faces in a crowd.
In the full-frontal face, the thermal image is grossly symmetrical bilaterally. The canthi or sinus areas in normal individuals are the hottest extended areas of the face. When glasses are not worn, it is a simple process to locate the canthi in the thermal image and use them to establish axes for the face. Other features which may be used are the nostrils, which may present alternately hot and cold bilaterally symmetric areas as the individual breathes in and out. The horizontal axis may be drawn through the outer corners of each eye, which are readily distinguishable in the infrared images or through the pupils which may be seen in some IR imagery. The vertical axis may then be drawn through the bow of the upper lip, or through the center point of the two nostrils, or at the midpoint between the eye corners. The intersection of the two axes will occur at the center of the two eyes. The midpoint between the horizontal through the eyes is defined as the center of the face.
If the person is wearing glasses, the pattern of the glasses, which block the infrared emissions from the face and thereby produce an extended cold area with sharp cut-off thermally, can be used to approximate the facial axes. If a sufficient number of minutiae are obtainable from portions of the face not blocked by glasses, facial hair, or other concealments, a person may be identifiable. Alternatively, if fewer than a minimum number of minutiae specified for a particular scenario are extracted by an automated system for a particular person, that person may be considered by the system to be a potential match, but be tagged as having a low number of minutiae.
Various perturbations, such as facial expression changes, can distort the relative locations of minutiae points to an extent. This is analogous to the deformations which occur in fingerprints due to movement between the fingers and the print surface. The minutiae matching algorithms allow for variations in the position and characteristics of the minutiae, as well as in the subset of minutiae which are seen due to the field of view of the camera and to possible obstruction of certain areas of the face in the image.
The face surface presents a smooth continuum of thermal levels, and reflects metabolic activity, ambient and internal temperatures, and ambient sources of thermal energy. Discontinuities occur at breaks in the skin continuum, such as caused by the nostrils, the mouth opening, the eyes, facial hair, moles or other skin disturbances, and any applique such as bandages.
According to a preferred embodiment of the invention, minutiae are used from the face. The minutiae are referenced to axes derived from specific physiological features. Although many different approaches may be used to obtain repeatable minutiae from facial thermograms, the preferred approach uses a number of extraction routines to produce a plurality of minutiae sufficient for an intended purpose. Thus, for a relatively low order of required security, on the order of ten minutiae may be extracted using absolute anatomical positions such as branch locations of the carotid and facial arteries.
For a high security requirement, on the order of 100 derived minutiae may be extracted using additional computations to identify further derived and absolute minutiae. The minutiae extraction and characterization procedure locates the position of each minutia. In addition it may note characteristics of each point such as: a vector indicating the orientation of the corresponding blood vessel; a second vector indicating the relative orientation of the branching blood vessel; the normalized apparent temperature; and the apparent width of the corresponding blood vessels. As with some fingerprint minutiae matching machines, use of the characteristics can enhance the speed and accuracy of identification. Furthermore, it can improve the accuracy and speed of automatic fusion of medical imagery.
This basic technique can be employed on an area-by-area basis when portions of the body cannot be seen or when significant changes have occurred in portions of the thermogram such as when portions of the body have suffered external wounds. This would be done by segmenting the thermogram to consider only the portions of the body in which minutiae can be detected. Functionally this is equivalent to matching a latent partial fingerprint found at a crime scene to a full rolled print filed in the FBI system. The set of minutiae points, together with characteristics which describe each such point and its relation to other minutiae is considered unique to the individual and persistent, for both contact fingerprints and thermal minutiae.
Verification that two images from different spectra may be from the same person can be an end goal in itself or the first step in further processing the two images to extract comparison data.
A change in facial expression or the action of speech causes movements in affected areas of the face, particularly the lips, but also the eye, chin, forehead, and cheek areas. Encoding of facial expressions and facial movements during speech is currently being studied for bandwidth reduction in the transmission of xe2x80x9ctalking headxe2x80x9d video for applications such as videophone, videoconferencing, video email, synthetic speech, and face animation. The intent is to transmit a baseline image followed by encoded changes to that image, with reconstruction of the animated face at the receiving end. This process offers significant bandwidth reduction, but may produce imagery in which the talking face seems stiff and unnatural or does not appear to be synchronized with the audio, giving the unacceptable look of a dubbed foreign film.
All such studies involve modeling the facial movements based upon the relocation of certain observable points of the face, such as the corners of the mouth. The various models differ in the extent to which they consider the underlying facial muscles and nerves. There are few observable reference points on a generalized face, especially under uncontrolled lighting conditions. In particular, there are no observable reference points in the cheek areas, and none in the forehead area except possibly skin creases. When the talking head is that of a dark skinned person, the reconstructed image may show further degradation of subtle facial features.
Use of an IR camera in conjunction with a video camera, or use of a dualband camera at the transmission end offers the potential for marked improvements. Infrared minutiae are more numerous than visible markers and are present throughout the face, including areas of the cheeks and forehead and chin where no visible minutiae may be present. Therefore, modeling of the movements of infrared minutiae can provide finer detailed replication of expressions and speech than modeling based upon visual references.
At the transmitting end, a visual baseline image of the subject face is sent, followed by transmission of only the movement vectors of those infrared minutiae which move from frame to frame. At the receiving end, the baseline face is animated based upon overlaying the IR minutiae movements on the visual image.
Early results indicate a minimum of 150:1 compression for highly energetic faces, to 400:1 for mildly mobile faces when 30 frames per second are processed. A primary application for this technique is videoconferencing, where the goal is to provide acceptable quality imagery over dial-up lines, at acceptable cost.
Video e-mail and videophone could also utilize the significant bandwidth reduction and automated re-synchronization of voice and image.
By processing sequences of images taken from known expressions and/or known speech elements, a sequence of movements of infrared minutiae can be extracted which corresponds to that expression or speech element for that person or for persons in general. Subsequently, when the same sequence of movements of infrared minutiae is seen, it can be inferred that the person is displaying the same expression or speech element as during the initial sequence. This enables the automated determination of expression or speech, allowing for compression of transmitted video in conjunction with audio. The combination may offer additional composite compression and improved synchronization.
The same basic technique can also be used to create a dictionary of facial expressions and speech elements for use in animation of a synthetic face.
The talking head video compression system will have both video and IR cameras, and can be used to recognize and/or generate facial expressions and/or speech-related facial movements from the IR image and superimpose them on a contemporaneous visual image. The use of correlated infrared and video facial images offers significantly better fidelity of expression and speech-related variations in compression and reconstruction of talking head video, while also ensuring the authenticity of the related transmissions.

Method and apparatus for aligning and comparing images of the face and body from different imagers


A pipe break valve device includes a housing that has a recess continuous in a flow direction, a valve slide that is arranged displaceably in the flow direction in the recess, and a spatial volume of a throttle device that is arranged at least in regions between the outer wall of the valve slide and the inner wall of the housing. Throttle ducts that are present on the inlet side and on the outlet side are formed by a first gap and a second gap which are present in each case between the outer wall of the valve slide and the inner wall of the housing, and no sealing means are present between the inner wall of the housing and the outer wall of the valve slide.

One application of such pipe break valve devices is in heavy trucks having a plurality of axle lines, the pipe break valve units acting as pilot switching valves on the axle balancing cylinders. The axle balancing cylinders are installed movably, so that a flexible line, in particular a high-pressure hose piece, is inserted as a supply line for the flow medium, for example hydraulic oil, upstream of the cylinder. The throttle device has the effect that pressure changes briefly occurring abruptly, for example on account of unevennesses in the road, do not cause the valve to close. Furthermore, the throttle device has the effect that, when a persistent differential pressure occurs between the inlet connection and the outlet connection, which happens, for example, when a hose bursts, the pipe break valve shuts off passage sealingly as a result of the corresponding transverse movement of the valve slide.
FIG. 5 illustrates diagrammatically in cross section a known pipe break valve device 60 of the applicant. The pipe break valve device 60 has a housing 62 which in FIG. 5 possesses an inlet connection 64 on the underside and an outlet connection 66 on the top side. The two connections 64, 66 are in communication connection inside the housing 62, the flow direction of the medium, for example hydraulic oil, being identified in FIG. 5 by the arrow F.
In the housing 62 there is a continuous transverse bore 80, within which a valve slide 68 is mounted displaceably in the transverse direction Q. In each case on the left and on the right of the valve slide 68 there is a sealed-off oil space 70.1, 70.2 which is in communication connection with the flow cross section of the housing via throttle ducts 74, in each case both with the inlet connection region and with the outlet connection region. The first duct region of the throttle ducts has a very small bore cross section which ultimately gives rise to a throttle effect. Inside the oil spaces 70.1, 70.2 there are in each case elastic means 72.1, 72.2 which act on the valve slide 68 and, in the normal operating state, hold it in the open state.
In the normal operating state, both oil spaces 70.1, 70.2 are filled correspondingly with hydraulic oil and, as already mentioned, are in communication connection with the interior of the flow cross section. The rigidity of the elastic means 72.1 and 72.2 is in this case set such that, if a pressure difference occurs when the flow medium flows through the housing 62, the valve slide 68 is in the open position.
The throttle ducts 74.1, 74.2 and oil spaces 70.1, 70.2, in conjunction with the elastic means 72.1, 72.2, constitute a throttle device which prevents the valve from shutting off in the event of a pressure peak briefly occurring suddenly, since the filling state of the oil spaces 70.1, 70.2 does not change substantially on account of the given throttle action, and therefore the valve slide 68 is held in its position. If, then, for example, a line break occurs on one side, that is to say a situation where a pressure difference between the inlet region and outlet region is present for a relatively lengthy period of time, oil flows via the throttle duct 74.1 into the right oil space 70.1 (arrow F1) and at the same time oil flows out of the left oil space 70.2 via the throttle ducts 74.2 into the outlet region (arrow F2), thus giving rise ultimately to a displacement of the valve slide 68 to the left, the valve slide 68 shutting off the throughflow after a stipulated displacement travel is reached. The valve slide 68 executes a displacement movement to the left when the line connected on the top side breaks. If the line connected on the underside breaks, the valve slide 68 executes a displacement movement to the right in the transverse direction Q in order to shut off the throughflow.
The known pipe break valve device ensures permanently reliable functioning. The relatively complicated structural set-up, the multiplicity of components and the complex geometry necessitate a high outlay, and therefore, in terms of economical production, high costs are incurred which have to be reflected in the price.
DE 10 2009 043 568 A1 describes a hydraulic safety valve for shutting off a first duct, connectable to a pump, with respect to a second duct, connectable to a hydraulic consumer. In this case, two coaxially arranged separate blocking pistons are mounted displaceably in the axial direction, are acted upon in each case by the spring force of a spring and can be transferred, counter to the spring force of the respective spring, from an open position into a blocking position as a result of the overshooting of a flow velocity of the hydraulic medium and of a pressure difference thereby occurring. The ducts are shut off by means of this movement. In this case, there is a damping device for the damped or time-delayed closing of the blocking pistons. The damping device is designed as a chamber which is in communication connection with the fluids via a throttle bore. The throttle chamber therefore necessarily has to be vented. For this purpose, corresponding venting ducts are provided which can be closed and, if required, opened by means of screws. On account of the chosen design, it is necessary that sealing means are present between the inner wall of the blocking piston and the inner wall of the housing. These sealing means which are present may present problems, since, if not actuated for a lengthy period of time, they may seize up due to ageing influences and may be able to therefore no longer ensure their functioning when required. Since this safety valve necessarily has to be vented, it is necessary to ensure, for example on a vehicle, a straightforward situation where the installed safety valve is accessible in order to carry out venting.
German Utility Model specification DE 20 2011 050 667 U1, published at a later date, relates to a hydraulic pipe, line or hose break safety shut-off valve for shutting off a supply line duct, connectable to a pump, with respect to an outflow line duct, connectable to a hydraulic consumer. The supply line duct and outflow line duct issue into an axial bore, extending in the direction of an axial axis, of a valve housing, in which axle bore is mounted displaceably a blocking piston which is acted upon in the axial direction by spring force of a spring, with the result that the blocking piston is held in an open position up to a specific limit flow velocity of a hydraulic medium which amounts to a value greater than zero. Here too, a separate damping chamber is connected via throttle ducts and has to be vented via ventilation ducts in order to ensure permanently reliable functioning. The set-up of this valve is relatively complicated and therefore cost-intensive. Moreover, even here, it is absolutely necessary that sealing means are present between the outer wall of the moveable blocking piston and the inner wall of the housing, these sealing means possibly entailing the problems already described above.
German Laid-Open Publication DE 2 352 826 discloses a hydraulic pipe break valve closing in a damped manner, with a blocking piston, held open by a compression spring, between a consumer connection and a connection leading to the pump and return. In this case, via a relatively complicated duct system, a damping space is in communication connection with the space of the blocking piston by means of a member which reduces the throughflow cross section of at least one bore under the action of a pressure drop building up between the connections and displacing the blocking piston counter to the action of the compression spring, after this displacement has commenced. The damping action is consequently adjustable. This valve also has a highly complex set-up and, in terms of economical use, can therefore be used to only a restricted extent.
U.S. Pat. No. 6,513,545 B2 discloses a nonreturn valve, in particular suitable for gas lines, in which a set screw is used which acts on a spring in order to set the spring force of the latter and therefore its recoil characteristic. This valve device has no damping properties of any kind and is therefore unsuitable for the use of the pipe break valve device according to the application, for example in heavy motor trucks, since it is not capable of maintaining "permissible" pressure fluctuations while at the same time continuing to fulfill its function.
U.S. Pat. No. 3,995,656 describes a valve for a high-pressure gas line which is designed as a regulating valve and does not present any safeguard against a hose break. Moreover, this valve also has no damping properties.
U.S. Pat. No. 5,507,466 discloses a valve device which is capable of shutting off the valve automatically after a stipulated fluid volume has flowed through. The valve slide employed for this purpose is mounted longitudinally displaceably and is located within a continuous recess. By means of a valve of this type, it is possible to carry out metering of a volume. As soon as this volume is reached and the consumer-side port is closed, the valve slide moves back into its initial position, since it is under the action of a spring force, and is then in the open state, in order once again to dispense a stipulated volume quantity, as required. As soon as the valve slide is stationary, for example as a result of the shut-off of the outflow duct, it moves back into its initial position. Here too, sealing means are arranged between the outer wall of the valve slide and the inner wall of the housing in order to ensure permanently reliable functioning. The valve illustrated contains a relatively large number of additional components and also ensures a nonreturn function, that is to say, if high flow velocities suddenly occur, closing is initiated by the valve slide.
U.S. Pat. No. 2,699,799 describes a nonreturn valve in which the valve slide is under the influence of a spring force and is arranged within a continuous recess in the housing. This valve automatically ensures throughflow under low pressure in one of the directions and shuts off as soon as a pipe pressure prevails in the other direction.
EP 1 672 259 A1 discloses a safety valve in which a valve slide is present longitudinally displaceably within a continuous recess. The valve slide is under the action of a spring force which is arranged in a damping chamber. The damping chamber influences the displacement travel of the valve slide under specific preconditions. The valve slide itself is sealed off with respect to the inner wall of the housing of the valve via special sealing elements. Here too, because of the sealing elements present, there is the problem regarding permanently reliable functioning, in particular regarding non-actuation of the valve slide which lasts for a relatively lengthy period of time, which may influence the action of the sealing means and, in particular, required friction values which are to ensure functioning are no longer available, so that, overall, functionality as required is then no longer ensured.

HORST THIELE MASCHINENBAU-HYDRAULISCHE GERATE GMBH


An adjustable chanter for mounting in an outlet tube of a set of bagpipes includes a conventional reed and a conventional chanter pipe having a series of finger holes by which notes can be selected. The chanter pipe has a collar at an upper end which is screw fastened to a hub defining a sleeve arranged which extends from the upper end of the chanter pipe into the tube of the set of bagpipes as a friction fit within the tube such that the hub is supported in the tube for receiving air through the tube from the bag for passing through the pipe for playing the notes. The position of the mouth of the reed relative to the holes in the chanter pipe is adjusted by a shuttle which slides longitudinally in the hub moved by a manually operable shaft projecting from the hub to one side and manually rotatable so that the reed can be adjusted to change tuning without rotating the chanter pipe or removing the chanter pipe from the tube of the bag.

Conventional bag pipes include a bag with a series of outlet tubes fastened into the bag so that the air in the bag escapes through the tubes into either the chanter or the drones. The chanter comprises a pipe with holes at spaced positions which change the note generated by a vibrating reed located in the chanter and supported by the chanter in the outlet tube.
At this time, adjustment of the tuning is obtained by removing the chanter from the tube by pulling it from its friction fit within the tube whereupon the reed is exposed and can itself be removed from the end of the chanter and adjusted in position relative to the end of the chanter by selecting an amount of wrapping and the location of the friction fit of the reed in its holder in the chanter. This is difficult, potentially damaging to the delicate reed and requires repeated trial and error.
Bagpipes are very susceptible to changes in moisture and other environmental conditions so that they require repeated tuning, at least to the trained and skilled ear.
Prior art arrangements for other types of musical instruments which are not designed for and not suitable for bag pies are shown in U.S. Pat. Nos. 2,485,021 (Strupe) issued May 1946; 2,098,457 ( ) issued; 1,837,227 ( ) issued and 1,821,655 ( ) issued. In the first, second and third arrangements described above, the instrument is adjusted by rotation of one element relative to another about the axis of the pipe. The older patented designs will cause the pitch tone to be adjusted while the chanter is being removed from the pipe stock because of the twisting effect that is required to perform this task.


The present invention provides a communications headset. The communications headset includes an earbud with a speaker to be disposed near the ear of a headset user, a microphone, and an electrical connector designed to couple with a communications device. An electrical cord is coupled between the earbud and the electrical connector. A spring cord is coupled between the earbud and the microphone. The microphone is capable of bidirectional movement with associated extension and retraction of the spring cord.

Headsets and other portable communications devices are often designed to be easily stored during non-operation so that the headset can be transported along with a mobile telephone device. In the prior art, headsets often employ an earbud with a speaker and a microphone coupled to a cord which dangles down from the user's ear during operation. This boomless headset design allows for convenient storage, often employing an integrated cord retractor to wind the headset cord during non-operation.
Headset cords are often bulky, cumbersome and difficult to store when excessively long. The headset cords often become entangled during headset stowage, exposing them to damage. The entangled cords must be unentangled prior to operation. This is undesirable to consumers, who generally prefer compact telecommunication devices and accessories.
Integrated cord retractors only wind the headset cord between the headset plug interface and the dangling microphone. The length of cord between the earbud and the dangling microphone is not wound due to the coupling of the microphone to the cord. Referring to FIG. 1, a prior art headset configuration is illustrated. An earbud 100 is coupled to a microphone 102 via a length of electrical cord 104. Microphone 102 is fixed to electrical cord 104, and as a result, the length of electrical cord 104 between earbud 100 and microphone 102 is fixed. A cord retractor 106 is utilized to wind the length of the electrical cord 104 between microphone 102 and connector 108.
The unwound length of cord 104 is exposed to potential damage and inconvenient tangling during transportation of headset. In one typical solution, the length of unwound cord is shortened by coupling the microphone to the cord closer to the earbud. However, this solution introduces the potential for insufficient isolation of a voice signal during operation, resulting in echo. Furthermore, during operation of the headset users frequently position the microphone closer to their mouth with their hand to improve the signal to noise ratio of the voice signal. By shortening the length of unwound cord, there may be insufficient length to position the microphone in front of the user mouth as desired.
Thus, improved headset designs are needed that provide both performance and portability.