In image processing according to the prior art, the important part of photographic image data (referred to herein as the object) could not be determined and therefore required human participation.

1. Field of the Invention
This invention relates to an image processing method wherein an optimum image processing is performed automatically on photograph image data such as digital photograph image, and the image is evaluated, to an image processing apparatus, a medium on which an image processing control program is recorded, an image evaluation device, an image evaluation method, and a medium on which an image evaluation program is recorded.
2. Description of the Prior Art
Various kinds of image processing may be performed on digital image data, i.e., in which processing: contrast may be increased; color may be corrected; or lightness may be corrected. This image processing can usually be performed with a microcomputer. An operator confirms the image on a monitor, the necessary image processing is selected, and image processing parameters are determined.
In recent years various types of image processing techniques have been proposed, and are now having considerable impact. However, a human operator is still required when it is a question of which technique to apply, and to what extent it should be used. This is because it was otherwise impossible to determine which digital image data had to be subjected to image processing. For example, in the case of image processing to correct the lightness of an image, the screen is made lighter if it is dark on the whole, and is made darker if it is too light.
Now, consider the case of a photographic image of a person filmed at night, where the background is near to pitch-darkness but the person in the picture has been well photographed. If this photograph is automatically corrected, it is attempted to make the image brighter due to the fact that the background is pitch black, so the final image appears as if the photo was taken in the daytime.
In this case, if a human operator is involved, he pays attention only to the person in the picture. If the image of the person is dark, it would be made a little brighter, conversely darkening would be selected if the effect of flash, etc., was too bright.
Hence, there was a problem in the prior art in that a human operator had to participate to determine the important part (referred to hereafter as the xe2x80x9cobjectxe2x80x9d) of a photographic image.
However, even when the importance of the image is evaluated by some technique, the determination process is performed in picture element units, and varying the importance in real time causes an increase in computation.
It is therefore an object of this invention, which was conceived in view of the aforesaid problems, to provide an image processing method which permits an important part of a photographic image such as a digital photograph image to be detected, and an optimum image processing to be automatically selected, to provide an image processing apparatus, and to provide a medium on which an image processing control program is recorded.
In order to achieve the aforesaid object, this invention is an image processing apparatus into which photographic image data comprising dot matrix picture elements is input, and which performs predetermined image processing on the input data. The apparatus comprises an image processing acquiring unit which acquires the aforesaid photographic image data, an image processing indicator specifying unit which performs predetermined summation processing on picture elements and specifies an image processing indicator based on the acquired image data, and a processing unit which determines image processing contents based on the specified image processing indicator, wherein the aforesaid image processing indicator specifying unit comprises an object determining unit which determines picture elements having a large image variation amount to be those of the object, and the aforesaid processing unit determines image processing contents based on image data for picture elements determined to be those of the object and performs image processing on the determined contents.
Herein, it is assumed that in the case of an image of a photographic image of a person, the person is usually photographed in the center of the field. Therefore, the person is brought into focus to give a sharp image. When the image is sharp, the outline part becomes clear, and the amount of image variation becomes large. As a result, there is an extremely high possibility that there will be no error if it is assumed that picture elements with a large image variation amount are those of the original object which has been brought into focus.
In the invention thus comprised, in the image processing indicator specifying unit based on photographic image data comprising dot matrix picture elements acquired in the image data acquiring unit, predetermined summation processing is performed on picture elements. This summation processing may take various forms, and the image processing indicator is basically specified based on the summation result. In the processing unit, the image processing contents are determined based on the specified indicator, and the determined image processing is performed.
This means that useful information about the corresponding image can be obtained by performing summation processing on dot matrix photographic image data, and image processing is performed on the data. In this way, the image processing indicator is specified by actual photographic image data, so optimum image processing can be performed even without human intervention.
The photographic image data comprises dot matrix picture elements and image processing is performed in picture element units, but first, in the object determining unit, picture elements having a large image variation are determined to be those of the object. Image processing contents are then determined in the processing unit based on image data for picture elements determined to be those of the object, and image processing is performed based on the determined contents.
Therefore, according to this invention, the determination of the object, which in the past required human participation, can be automated by determining the object as picture elements with a large image variation. The invention therefore provides an image processing method whereby optimum image processing can be performed by suitably modifying the image processing contents according to the object.
This method for specifying an image processing indicator from actual photographic data may of course be applied not only to a real device but also to a system on the method. In such a sense, this invention is also an image processing method wherein photographic image data comprising dot matrix picture elements is input, and predetermined image processing is performed, this image processing method comprising an image data acquiring step for acquiring the aforesaid photographic image data, an image processing indicator specifying step for performing a predetermined summation processing on picture elements based on this acquired image data, and specifying an image processing indicator, and a processing step for determining image processing contents based on the specified indicator, and performing image processing, wherein the aforesaid image processing indicator specifying step comprises an object determining step wherein picture elements having a large image variation amount are determined to be those of the object, and wherein in the aforesaid processing step, image processing contents are determined based on image data for picture elements determined to be those of the object, and image processing is performed based on the determined image processing contents. In this case, the apparatus offers all the attendant benefits and advantages of the invention.
This apparatus for determining an object and performing image processing method may be implemented by a stand-alone apparatus as mentioned above, or may be incorporated in another instrument which comprises such an apparatus. In other words, the scope of this invention covers various forms of implementation. It may also be implemented by hardware or software, and can be modified as necessary.
When the apparatus for implementing the concept of this invention is implemented by software, the invention applies equally to media on which this software is recorded and which can be used in exactly the same way. In this sense, this invention is also a recording medium whereon an image processing control program is recorded for inputting photographic image data comprising dot matrix picture elements by a computer, and which performs predetermined image processing on the input data. The control program comprises an image processing indicator specifying step for acquiring the aforesaid photographic image data, an image processing indicator specifying step for performing predetermined summation processing on picture elements and specifying an image processing indicator, and a processing step for determining image processing contents based on the specified image processing indicator, wherein the aforesaid image processing indicator specifying step comprises an object determining step which determines picture elements having a large image variation amount to be those of the object, and in the aforesaid processing step, image processing contents are determined based on image data for picture elements determined to be those of the object, and image processing is performed on the determined contents. In this case, the recording medium offers all the attendant benefits and advantages of the invention.
The recording medium may of course be a magnetic recording medium, an optomagnetic recording medium, or any kind of recording medium which may be developed in the future. It will of course be understood that the medium may be a first copy or second copy, and that a telecommunication line may also be used to supply the program. In this case, there is no difference regarding the application of the invention. There is also no difference if the program is written on a semiconductor chip.
There is no difference as regards the concept of the invention even if one part is software, and one part is implemented with hardware, or when it is in such a form that one part is stored on a recording medium which can be read when necessary.
Photographic image data means image data obtained when it is attempted to take a photograph of a real object. Image processing tries to correct images by comparing the images with the real objects from which they were acquired. The invention therefore applies not only to natural objects but also to manmade ones. More specifically, this includes image data read by a scanner, or image data captured by a digital camera.
Various techniques may be employed to determine the variation of picture elements in the object determining step. A further object of this invention is to give a specific example of this.
In the image processing method provided by this invention, in the aforesaid object determining step, the amount of variation of picture elements is determined based on a difference between adjacent picture elements.
Hence according to this invention, in the object determining unit where an image variation amount is determined, the determination is performed based on a difference of image data between adjacent picture elements. When there is a fixed interval between picture elements as in the case of a dot matrix, the difference of data between adjacent picture elements is directly proportional to a first order differential. This difference can be taken as the variation amount of the image. In this case the difference is regarded as the magnitude of a vector, and the vector may also be constructed by considering adjacent directions.
According to this invention, only the difference of image data between adjacent picture elements is found. Computation is therefore easy, and the processing amount for object determination can be reduced.
The determination of an object is of course not limited to this technique, and it is a further object of this invention to provide other examples.
According to the image processing apparatus provided by this invention, in the aforesaid object determining unit, the criterion for determining whether or not there is a large image variation amount changes according to the position of the image.
In the case of a photograph for example, a person is often photographed in the center. In this case it may be said that in order to determine image processing content, the picture elements to be determined as the object should be selected from the central part of the field. However, it may be said that whether or not there is a large variation in the image depends on a difference from a comparison value, and there is no reason why such a value always has to be constant.
Therefore, to determine whether or not there is a large image amount according to this invention, in the object determining unit, this criterion is altered depending on the position of the image, the criterion for each position being compared with the image variation amount of each picture element.
Hence, according to this invention, the assessment of image variation changes depending on the position of the image, and a highly flexible determination which considers image composition is thus possible.
The criterion can be altered in various ways. As one example, a certain trend may be ascertained, or alternatively, a trend which causes a change may be read from the image.
A further object of this invention is to provide an example of the former.
In the image processing apparatus provided by this invention, in the object determining unit, the aforesaid criterion is set lower for the central part than for the edges of the image.
By setting the criterion lower for the center than for the edges, it is easier to determine the center part of the image as the object even if the variation amount at the center and at the edges is approximately the same. Therefore if there is an image of a person in the central part, the picture elements of this person will be determined as the object more frequently.
According to this invention, a determination can be made which gives more weight to the center area of a photograph, and a large amount of image data can be effectively processed.
A further object of this invention is to provide an example of the latter when the criterion is varied.
According to the image processing apparatus provided by this invention, in the aforesaid object determining unit, the above criterion is based on the distribution of the aforesaid image variation amount at different points on the image.
Hence according to this invention, in the object determining unit, the distribution of image variation is found in each part of the image, and the aforesaid criterion is determined after finding this distribution. Subsequently, a comparison is made with this criterion to determine whether or not the picture elements are those of the object.
According to this invention, as the object is determined taking account of the distribution of image variation for picture elements, the image data can be treated flexibly.
When the criterion is determined based on distribution, it may be considered that there is a high possibility of finding the object in a part where there are many picture elements with a large variation amount, and the criterion maybe set low.
Alternatively, basic setting patterns may first be prepared according to a variation distribution pattern, and a basic setting pattern may then be chosen based on the detected distribution pattern.
At the same time, assuming that the image processing indicator specifying unit comprises such an object determining unit, the image processing contents may be determined based on image data which is determined to be that of the object, and image processing may then be performed on the determined contents, there being no limitation on the specific processing method employed. For example, a luminance distribution of picture elements determined to be those of the object is found, and if the luminance distribution range is enlarged in a predetermined proportion when the luminance distribution is narrow, image processing to increase contrast is performed. If the luminance distribution of the object seems dark on the whole, a correction may be made to make it lighter. The color distribution of picture elements determined to be those of the object is found, and it is determined whether or not the grey balance is off. If it seems to be off, tone curves are used to modify the grey balance.
Hence, the importance of the image has an effect even if the image data is summed in order to specify the image processing indicator. However, even if the importance of the image is determined by some technique, the work is carried out in picture element units, so varying the importance of an image in real time implies an increase of computational amount.
A further object of this invention is to consider the importance of photographic image data such as digital photograph images in relatively simple terms, and perform optimum image processing automatically.
The image processing apparatus provided by this invention is an apparatus for inputting photographic image data comprising dot matrix picture elements, and performing predetermined image processing. This image processing apparatus comprises an image data acquiring unit for acquiring the aforesaid photographic image data, an image processing indicator specifying unit which performs a predetermined summation processing on picture elements based on this acquired image data and specifies an image processing indicator, and a processing unit which determines image processing contents based on the specified indicator and performs image processing. The aforesaid image processing indicator specifying unit comprises a feature amount uniform sampling unit which determines an image processing intensity by uniformly sampling a feature amount over a whole screen, and a feature amount weighting reevaluation unit which reevaluates the feature amount sampled in the feature amount sampling unit with a predetermined weighting. In the aforesaid image processing unit, the image processing intensity is determined based on the reevaluated feature amount, and image processing is performed with the determined intensity.
According to the invention having the above construction, photographic image data comprises dot matrix picture elements, and in the feature amount uniform sampling unit, the feature amounts of picture elements are uniformly sampled over the whole screen. In the feature amount weighting reevaluation unit, the feature amounts that are sampled in this feature amount uniform sampling unit are reevaluated with a predetermined weighting. Then, in the processing unit, the image processing intensity is determined based on the feature amounts that have been reevaluated in this way, and image processing is performed.
In other words, as the sampling is uniform over the whole screen and a predetermined weighting is applied after sampling, the feature amounts obtained as a result are different from what is obtained by uniform sampling over the whole screen.
According to this invention the sampling in the sampling stage is uniform over the whole screen, so the computational amount is not too high. At the same time, by applying a predetermined weighting after sampling, irrelevant evaluation is not made as it would be if the picture elements were merely sampled uniformly without weighting. The invention therefore provides an image processing apparatus in which optimum image processing can be performed automatically.
It will be understood that the technique of performing a uniform sampling in the sampling stage and applying a predetermined weighting thereafter, may be applied not only to a real device but also to a system in both of which cases it has all the attendant benefits and advantages of this invention. As a specific example of the concept of this invention, when the image processing apparatus is implemented in terms of software, there naturally exist recording media on which the software is recorded which can be used to perform the function of the invention.
The feature amount uniform sampling unit uniformly samples feature amounts over the whole screen, for determining the image processing intensity. For this purpose, all picture elements over the whole screen can be sampled, but it is not necessary to sample all of the picture elements if the sampling is uniform.
A further object of this invention is to provide an example of the latter case.
According to the image processing apparatus of this invention, in the aforesaid feature amount uniform sampling unit, the aforesaid feature amounts are sampled for selected picture elements after uniformly thinning out the picture elements according to predetermined criteria.
According to this invention, by thinning out the picture elements according to predetermined criteria, the number of picture elements to be processed is reduced, and the aforesaid feature amounts are sampled from the remaining elements.
Herein, the term xe2x80x9cuniform thinningxe2x80x9d comprises the case where picture elements are selected at a fixed interval, and the case where they are selected at random.
According to this invention, as the picture elements are thinned out when the feature amounts are uniformly sampled, the processing amount is reduced.
The sampled feature amounts are reevaluated by a predetermined weighting in the feature amount weighting reevaluation unit. The sampled feature amounts are in picture element units, but the weighting can be applied either to picture element units or to suitable aggregates of picture elements.
A further object of this invention is to provide an example of the latter case.
According to the image processing apparatus of this invention, in the aforesaid feature amount uniform sampling unit, feature amounts are sampled in area units that are divided according to predetermined criteria, and in the aforesaid feature amount weighting reevaluation unit, a weighting is set for each area and the feature amounts are then reevaluated.
The invention as formulated hereabove assumes weightings in area units of the image that are divided according to predetermined criteria. In the feature amount uniform sampling unit, feature amounts are sampled in these area units, while in the aforesaid feature amount weighting reevaluation unit, the feature amounts are reevaluated with weightings set for each area.
The division of these areas may always be constant, or it may be made to vary for each image. In this latter case the division method may be changed according to the contents of the image.
According to this invention, as the weighting is made to vary for each area, the computation is relatively simple.
Any type of weighting technique can be employed provided that reevaluation is performed without merely performing uniform sampling.
A further object of this invention is to provide an example of this.
According to the image processing apparatus of this invention, in the aforesaid feature amount weighting reevaluation unit, the aforesaid weighting is made to vary by a correspondence relation determined by the position of picture elements in the image.
In the case of a photograph, the person is usually in the center. Therefore, by weighting the central part of the image more heavily after uniformly sampling feature amounts from the whole image, the feature amounts sampled from picture elements relating to the person are evaluated to be larger.
When for example according to the invention thus comprised, the weighting of the central part of the image is heavier and the weighting of the surroundings is lighter, in the feature amount weighting reevaluation unit, the position of picture elements in the image is determined, and a reevaluation is made using a weighting which varies according to this position.
Hence according to this invention, as the weighting is determined according to the position of picture elements, the computation is relatively simple.
The weighting technique is of course not limited to this method, and a further object of this invention is to provide other examples.
According to the image processing apparatus provided by this invention, in the aforesaid feature weighting reevaluation unit, the image variation amount is found, and a heavier weighting is given to parts where the image variation amount is larger.
In the invention thus comprised, the image variation amount is found before performing reevaluation in the feature amount weighting reevaluation unit. The image variation amount is also known as image sharpness, and as the outline is sharper the better the focus, the variation is large where the image is in focus. On a photograph, the part which is in focus is the subject, and the part which is not in focus is considered to be the background. Therefore, places where there is a large image variation are considered to correspond to the subject. In the feature amount weighting reevaluation unit, the same result as sampling a large number of feature amounts is obtained by applying heavy weighting to parts where there is a large image variation.
According to this invention, as the weighting is varied depending on image sharpness, different targets can be precisely identified and feature amounts can be sampled for different images.
As another example of a weighting technique, in the weighting reevaluation unit of the image processing apparatus of the invention, the chromaticity of picture elements is found, a number of picture elements is found for which the chromaticity lies within the chromaticity range of the target for which it is desired to sample a feature amount, and heavier weighting is applied to parts where there are many of these picture elements.
Hence according to this invention, in the feature amount weighting reevaluation unit, the chromaticity of picture elements is found. In image processing, an object can sometimes be specified by a specific chromaticity. For example, there is no reason why a person could not be identified by looking for skin color, but it is difficult to specify skin color as color data also contain luminance elements. Chromaticity on the other hand represents an absolute proportion of a color stimulation value, and it is not controlled by luminance. Therefore an image of a person could be determined if the chromaticity was within a specified range that can be taken as indicative of skin color. This reasoning may of course also be applied to the green of the trees or the blue of the sky.
As specified objects can be sorted by chromaticity according to this invention, different targets may be precisely sampled depending on their images and feature amounts sampled.
Therefore in a feature amount weighting reevaluation unit, when the chromaticity found for picture elements is within a chromaticity range for a target from which it is intended to sample feature amounts, plural picture elements are counted. When the number of picture elements is large, this part of the image is determined to be the target, heavy weighting is applied, and a large feature amount is sampled from the target.
This weighting technique is not necessarily the only alternative, and it is a further object of this invention to provide a suitable example of overlapping methods.
According to the image processing apparatus of this invention, in the aforesaid feature amount weighting reevaluation unit, temporary weightings are applied based on a plurality of factors, and these factors are then added according to their degree of importance to give final weighting coefficients.
According to the invention as thus comprised, in the feature amount weighting reevaluation unit, temporary weighting coefficients are found separately based on a plurality of factors, and the weightings are added according to their degree of importance so as to reevaluate the sampled feature amounts as final weighting coefficients. Therefore, it may occur that even when a large weighting is assigned by one weighting method in the evaluation stage, if the method does not have a large importance, the final weighting which is assigned is not large. Moreover, it may occur that even if there is a large difference between weighting methods, image parts which are evaluated to have an average or higher weighting also have a large final weighting.
According to this invention, plural weighting techniques are suitably combined so that a suitable feature amount evaluation can be performed.
Therefore if the image processing indicator specifying unit itself comprises plural forms, it is not necessary to perform image processing with only one of these forms.
However even if there are some cases where it is desirable to perform image processing using the feature amount of the object, there are other cases where it is desirable to perform image processing using an average feature amount for the whole photographic image. For example, when a photograph is taken of a person, the person may not always be the lightest (highlighted) part of the picture. Therefore if attention is paid only to the person in the picture and contrast is increased, the highlighted part of the background will be too white. In this case, a better result would be obtained by paying attention to the whole photographic image.
Hence when image processing is performed, it is still necessary to select an optimum feature amount.
It is a further object of this invention to automatically select an optimum feature amount according to image processing technique.
In the image processing apparatus according to this invention, photographic image data comprising dot matrix picture elements is input, and predetermined image processing is performed. This image processing apparatus comprises an image data acquiring unit for acquiring the aforesaid photographic image data, an image processing indicator specifying unit for performing a predetermined summation processing on picture elements based on this acquired image data, and specifying an image processing indicator, and a processing unit for determining image processing contents based on the specified indicator, and performing image processing. The aforesaid image processing indicator specifying unit comprises an evaluation unit for obtaining a feature amount by inputting the aforesaid photographic data, summing the image data for all picture elements, and obtaining a feature amount according to plural predetermined evaluation criteria. In the aforesaid processing unit, the image data can be converted by plural techniques, and the feature amounts obtained in the aforesaid evaluation unit used according to the particular technique.
According to this invention, image data from a photographic image comprising dot matrix picture elements is input in this manner, and a feature amount is obtained according to plural evaluation criteria by summing image data for picture elements in the evaluation unit. In the processing unit, in converting the image data by plural techniques, the feature amounts obtained in the evaluation unit according to each technique are then used to convert the data depending on the technique.
Specifically, although there are cases where the image data is best converted using a feature amount centered on the object such as in light/dark correction, there are other cases where image data is better converted using a feature amount centered on the whole image such as when contrast is increased, and the image data conversion may be performed by suitably selecting these plural feature amounts.
According to this invention, when feature amounts are obtained according to plural evaluation criteria and image processing is performed by plural methods, the feature amounts used depend on the method, so image processing may be performed based on an optimum evaluation criterion.
When image data is converted, the feature amounts should be such that they can be used to identify the features of the image, and there is no need to specify the type of image. For example this also includes indicators such as luminance histograms which identify whether the image is to be considered as light or dark, there being no need to obtain the identification result that the image is light or dark. Apart from lightness, the indicator may of course specify whether or not the image is sharp, or it may be an indicator to identify vividness.
There is also no particular limitation on the way in which the evaluation unit and processing unit are applied. For example, assuming that plural image processing methods are used, plural feature amounts may be obtained and stored according to plural evaluation criteria, and the image data converted by suitably selecting the feature amount in the processing unit as necessary. As another example, image data for picture elements may be summed by a predetermined criterion in the evaluation unit so as to obtain a suitable feature amount on each occasion that image processing is performed in the aforesaid processing unit.
It will of course be understood that these plural image processing methods based on feature amounts obtained by plural different evaluation criteria, may also be applied not only to an actual device but also to a system both of which are then a valid form of the invention. When the image processing methods are implemented by software as specific examples of the concept of the invention, there naturally exist media on which the software is recorded which then offer all the attendant advantages thereof.
The evaluation criterion used to obtain feature amounts in the aforesaid evaluation unit will depend on the image processing that is to be performed, and while there are some cases where it is desirable to concentrate on the object for image processing, there are some cases where it is not as described above.
It is a further object of this invention to provide an example of the former case.
In the image processing apparatus according to this invention, the aforesaid evaluation unit comprises an evaluation unit wherein an object in a photographic image is sampled, and image data for picture elements of this object is summed to obtain a feature amount, and in the aforesaid processing unit, in one processing method, the feature amount obtained from object picture elements is used when the feature amount for the central part of the image data is used.
According to the invention thus comprised, when image data is converted based on the feature amount of the central part of the image data in the aforesaid processing unit, the object in the photographic image is sampled in the aforesaid evaluation unit, and the feature amount is obtained by summing image data for object picture elements according to predetermined criteria.
Herein, the central part of the image data has the following meaning. For example, when it is determined whether a given photograph is light or dark, it is easily appreciated that the determination can conveniently be based on the intermediate density of the image. This intermediate density may also be referred to as a median in a luminance distribution, i.e. the center of the luminance distribution, and in this sense it is referred to as the central part of the image data. Then, if there is an object in the photographic image, it may be said that there is a definite necessity to perform light/dark correction in line with the lightness of this object.
This invention is suitable for the case when image processing is performed based on the feature amount of the central part of the image data.
Any of the aforesaid techniques may be applied as the basic technique for sampling the object. As an example, in the aforesaid evaluation unit, picture elements for which there is a large variation of image data between adjacent picture elements are sampled as the object. When picture elements are aligned at a fixed interval apart as in the case of a dot matrix image, the difference of image data between adjacent picture elements is proportional to a first order differential. This difference may be determined as the image variation amount. In this case, the difference may be regarded as the magnitude of a vector, and the vectors constructed taking account of adjacent directions. If this is done it is sufficient to determine the difference of image data for adjacent picture elements, computing is easy, and the processing for determining the object is reduced.
As another example, in the aforesaid evaluation unit, picture elements for which the chromaticity is within a predetermined range may be sampled as the object. In this case, in the aforesaid evaluation unit, the chromaticity of picture elements is found. The chromaticity represents an absolute proportion of a color stimulation value, and it is not affected by lightness. Therefore the object in the image can be separated by possible range of chromaticity. For example, there is the chromaticity range for skin color, or the chromaticity range for the green of the trees. As this can be said for chromaticity, in the aforesaid evaluation unit, picture elements for which the chromaticity lies within a predetermined range are sampled as the object. In this way, an object can be determined by its chromaticity, and the object may be sampled without depending on the lightness or darkness of the object.
On the other hand as an example of image processing not concerned only with the object, the aforesaid evaluation unit comprises an evaluation criterion wherein picture elements of the aforesaid image data are uniformly sampled and summed so as to obtain a feature amount, and in the aforesaid processing unit, in one processing method, the feature amount obtained by the aforesaid uniform sampling is used when an average feature amount of the photographic image is used. In this case, when image data is converted based on the average feature amount in the aforesaid processing unit, the feature amount is obtained by uniformly sampling picture elements of the image data according to predetermined evaluation criteria. Of course, the summation may be performed on all picture elements of the photographic image, but it may be said that is no advantage as the processing amount increases. Hence, it is convenient to perform image processing based on the average feature amount of the photographic image, e.g. saturation correction.
As another example of image processing which is not concerned only with the object, the aforesaid evaluation unit comprises an evaluation criterion wherein picture elements of the aforesaid image data are uniformly sampled and summed to obtain a feature amount, and in the aforesaid processing unit, in one image processing method, the feature amount obtained by uniform sampling is used when the edges of a feature amount distribution of the photographic image are used. In this case, in the aforesaid processing unit, it is assumed that the ends of the feature amount distribution obtained in the aforesaid evaluation unit are used. For example, to increase the contrast, image processing is performed to find the luminance distribution, and the edges of this luminance distribution are widened, but if the luminance distribution of the object were used in this case, other highlighted parts appear white. Therefore in this case, in the aforesaid evaluation unit, picture elements of image data are uniformly sampled according to predetermined criteria, and summed to obtain the feature amount. This is suitable for image processing using the edges of a feature amount distribution in an actual photographic image, e.g. for increasing contrast.
In the above, a continuous sequence of processes is performed comprising predetermined analysis of the image and image processing by specifying image processing indicators, but the analysis result itself is also useful.
It is a further object of this invention to provide an image evaluation device wherein it is easier to use an image evaluation result which is an analysis result done.
In the image evaluation device offered by this invention, photographic image data comprising dot matrix picture elements is input, the image data for all picture elements is summed according to predetermined criteria, and is the image evaluation device which is based summation result, and evaluate image, and the image is evaluated based on the summation results. There are plural evaluation criteria for the aforesaid summation results, and the evaluation results are combined with a predetermined weighting based on these evaluation criteria.
According to the invention as thus comprised, the evaluation method assumes that photographic image data comprising dot matrix picture elements is input, the image data is summed for picture elements, and the image is evaluated based on the summation results. Herein, there are plural evaluation criteria for these summation results, and the evaluation results are combined with a predetermined weighting based on the evaluation criteria.
In other words, although some evaluation criteria are suitable for evaluating images where a sharp image is the object such as in the case of portrait, other criteria are suitable for evaluating images where the background is the important object. A general evaluation may be made by suitably combining plural evaluation criteria in parallel and varying the weightings.
As described hereabove, as this invention gives a general evaluation by varying the weightings of plural evaluation criteria, it provides an image evaluating device which can be flexibly adapted to image feature determination.
Naturally, the concept of this invention for image evaluation by the above techniques comprises many forms. Specifically, it comprises hardware and software, various modifications being possible as may be convenient. When the concept of the invention is implemented by image processing software, there naturally exist recording media on which the software is recorded which can be used to perform the function of the invention. Moreover, these image evaluating techniques may be applied to an image evaluating device and its system running on software media.
Various techniques may be employed to achieve the same object in applying plural evaluation criteria to summation results. For example, all picture elements may be summed by weighting with different evaluation criteria, but it will be appreciated a large amount of processing is involved when the summation is applied to all picture elements. Hence, as described above, the picture elements are first sampled based on plural evaluation criteria, summed, and the summation results combined with a predetermined weighting. In this case, the image data are sampled prior to summation, plural criteria are used by varying the criteria applied to the sampling, and the weighting of the summation results is then adjusted before combination. An evaluation can therefore be made with different weightings on the results based on plural evaluation criteria. Due to this sampling of image data, plural evaluation criteria may be employed according to the sampling method.
As one evaluation criterion, the data may of course be sampled uniformly and summed. In this case, the image data is uniformly thinned and the whole image is considered, which makes this a suitable criterion for determining scenic photographs, etc. In this way an optimum criterion can be used while reducing the processing amount.
As an example of a criterion which can be applied whether or not sampling is used, is the evaluation of picture elements which have a large variation relative to adjacent picture elements with a heavier weighting. In this case, the image variation of picture elements from adjacent elements is detected, and clear image parts with a large variation are given a heavier weighting in the summation. If this is done, as image parts with a large variation are often parts of the photograph which are clearly in focus, an image evaluation which weights important parts of the image can be performed.
This criterion places more emphasis on the sharp parts of the image, and it is therefore naturally suitable for the determination of human images. Herein, image parts with a large variation may be evaluated either by introducing weighting as the picture elements are summed, or by summing only picture elements with a large variation. The weighting used in the evaluation is not necessarily fixed, but may also be allowed to vary according to the criterion. In this case, by varying the weighting for different evaluation criteria, an overall evaluation result for the image can be deduced. Moreover various approaches are possible, e.g. weightings may be varied individually so as to generate plural combinations which are then selected. In this way, by modifying the weighting for plural criteria, a more flexible evaluation can be made.
Instead of an operator varying the weighting, this can be done based on the image data itself. As an example of this, the weighting of the evaluation results may be varied based on the criteria. In this case, results are obtained according to various criteria, and the weighting is modified in view of the suitability of the criteria in the light of the results. As the results are used to vary the weighting, the work involved in the evaluation is less.
Various techniques may also be used to modify the weighting of the criteria using the results. For example, if it is determined whether or not the image data for picture elements should be sampled according to one criterion, the number of these picture elements may be taken as a criterion and the weighting increased when the number of picture elements is large.

IMAGE PROCESSING APPARATUS, AN IMAGE PROCESSING METHOD, A MEDIUM ON WHICH AN IMAGE PROCESSING CONTROL PROGRAM IS RECORDED, AN IMAGE EVALUATION DEVICE, AND IMAGE EVALUATION METHOD AND A MEDIUM ON WHICH AN IMAGE EVALUATION PROGRAM IS RECORDED

Abstract: A computer 21 which is the core of image processing calculates an edginess which is an image variation from a differential value of data for adjacent picture elements in a step SA110, and determines object picture elements by selecting only images with a large variation in steps SA120, SA130. As optimum parameters for contrast correction and lightness compensation are calculated from image data for object picture elements in steps SA310-SA330, image processing indicators based on object picture elements are determined, and optimum image processing can be performed automatically. After summing a luminance distribution for each area of the image, which is a feature amount, while uniformly selecting picture elements in a step SB110, a reevaluation is performed by a weighting determined for each area in a step SB120, and a luminance distribution strongly influenced by the luminance distribution of the photographed object is thus obtained with uniform sampling. After determining the intensity of this luminance distribution in steps SB130-SB150, the image data is converted in a step SB160, and image processing can therefore be performed with optimum intensity while reducing the processing amount.


An optical filter-coupler comprises at least two optical waveguides such as optical fibers closely spaced apart to each other for allowing light propagating in one waveguide to couple into the other. Materials for the waveguides are selected to have different wavelength-dispersive refractive indexes so that the waveguides have a common refractive index at a selected wavelength, whereby a complete power transfer is obtained at such selected wavelength.

1. Field of the Invention
The present invention relates to optical transmission and optical measurements, particularly to an optical filter-coupler which uses optical fibers or other types of waveguides for propagating optical radiation.
2. Prior Art
In conventional optical filter-couplers, the same material is used for closely placed optical fibers or waveguides.
The splitting ratio of an optical filter-coupler is defined by a function of the coupling coefficient, the difference in the propagation constant between cores or waveguides of the optical filter-coupler, and the coupling length. If the coupling coefficient is C, a half of the difference in the propagation constant is .DELTA., and a coupling length from the start of coupling is Z, the splitting ratio R, which represents how much of the light entering an incident port 1 of an optical coupler, such as shown in FIG. 1, couples from a core 3 thereof to another closely placed core 4, is given by a logarithm of the ratio of light intensities in the cores 3 and 4, Pb/Pa, as follows: ##EQU1##
Conventionally, closely placed waveguides are made of the same material, therefore the difference in propagation constant between the waveguides should theoretically be zero (0). But practically such difference in propagation constant can not be reduced to zero. This is because the closely placed cores or waveguides can not be manufactured in precisely the same diameter, consequently a slight difference in the propagation constant between the waveguides occurs. Therefore the maximum splitting ratio in such filter-couplers is given by: ##EQU2##
As shown in FIG. 2, the splitting ratio is thus governed by a periodic function of the coupling length, and therefore a major part of the power of the propagating optical radiation such as light transfers from one core to the other at a coupling length ##EQU3## (n is an integer).
Recently, there are demands in the field of multiple wavelengths transmission for wavelength filters which perfectly separate a designated wavelength. However, for example, when light of a wavelength A or light of a wavelength B as shown in FIG. 3 are desired to be completely separated, as can be calculated from the equation (2), the splitting ratio R which can be obtained by a conventional filter-coupler only reaches finite levels as shown in FIG. 3 due to the difference in propagation constant between waveguides. In other words, a complete power transfer can not be obtained by use of a conventional filter-coupler which uses the same material for its waveguide portions. Further, it is extremely difficult to manufacture an optical waveguide coupler having its maximum splitting ratios at two different wavelengths because slight changes in coupling length cause large changes in the splitting ratio.

Osaka Electric and Communication University


A camera to which a lens unit is exchangeably attached includes a phase difference detection type focus detection unit, a contrast detection type focus detection unit, and a processor configured to acquire correction information used to correct a shift amount between a focus detecting light flux and an image-pickup light flux from the image pickup lens. The contrast detection type focus detection unit performs scanning in a single direction from a focus position detected by the phase difference detection type focus detection unit, the single direction being set based on the correction information.

Patent Literature 1 discloses a hybrid focus detection unit that includes a focus detection unit of a phase difference detection method (simply referred to as a "phase difference AF" hereinafter), and a focus detection unit of a contrast detection method (simply referred to as a "TVAF" hereinafter). Patent Literature 2 discloses a phase difference detection (image pickup surface phase difference AF (simply referred to as a "SAF" hereinafter) function by providing an image pickup device with an imaging pixel and a focus detecting pixel, and a pupil splitting unit. Patent Literature 3 discloses a phase difference AF using a secondary imaging optical system.
Other prior art include Patent Literatures 4 and 5.


An input/output switching circuit comprising a first terminal provided on a first audio/video unit, a first switch coupled to the first terminal for switching an audio/video signal at the first terminal, a second terminal provided on a second audio/video unit, a connection cable coupled between the first and second terminals for bidirectional communication of the audio/video signal between the first and second audio/video units, a second switch, coupled to the second terminal for switching the audio/video signal at the second terminal, a first microcomputer coupled to the first switch for controlling the first switch, a second microcomputer coupled to the second switch for controlling the second switch, a communication line for communicating control data between the first and second microcomputers wherein the control data includes data indicating the state of the first and second switch, such that the first and second microcomputers control the first and second switches in response to the control data to accordingly provide bidirectional communication of the audio/video signal.

With the advances in electronic technology in recent years, the control of amplifier units (which selectively extract a number of input signals via a function switch) and tape deck units (which record or play back audio or video cassette tapes) is performed by microcomputers. Recently, for instance, the amplifier unit and the tape deck unit each have a microcomputer which recognizes the operating mode of the microcomputer in the other unit by mutual exchange of control data. One benefit of each unit having its own microcomputer is the increased processing speed that can be achieved.
FIG. 1 shows an example of a prior art input/output switching circuit in which a microcomputer is provided in each of an amplifier unit and a tape deck unit.
The amplifier unit 10 includes an amplifier control microcomputer 12 (referred to as "first microcomputer" hereafter) for selectively switching the input signals at jacks 14a through 14d. The first microcomputer 12 of the amplifier unit 10 detects and determines the current mode of tape deck unit 16 through keyswitches 12a through 12c. Tape deck unit modes include record and playback.
The tape deck unit 16 passes control signals to first microcomputer 12, through a communication line 18 connected between jacks 14e and 14f for selecting signals output through a speaker 20 via an input selector switch 22 (referred to as "input selector" hereafter). Additionally, first microcomputer 12 transmits the switching mode of the input selector 22 to tape deck unit 16 through communication line 18.
The tape deck unit 16 is provided with a tape deck control microcomputer 24 (referred to as "second microcomputer" hereafter) for controlling a tape drive mechanism (not shown) which drives a cassette tape and for executing the signal selection of a record/playback head 26. The second microcomputer 24 controls the cassette mechanism according to the mode of keyswitches 24a through 24e, and executes the selection of either recording signals to a record/playback head 26 or the playback of recorded signals. Moreover, the second microcomputer 24 detects and determines the mode of amplifier unit 10 according to control data output from the first microcomputer 12 through communication line 18.
When tape deck unit 16 is selected at the amplifier unit 10, first microcomputer 12 automatically passes the appropriate control data to the second microcomputer 24 to place tape deck unit 16 into playback mode. When another input is selected at the amplifier unit 10 microcomputers 12 and 24 automatically place tape deck unit 16 into a stop mode. Additionally, the second microcomputer 24 transmits control data to amplifier unit 10 through the communication line 18 indicating the mode of the cassette mechanism (playback, fast forward, rewind, stop, record) in tape deck unit 16.
When the input keyswitch 12a of first microcomputer 12 is closed, first microcomputer 12 closes a first switch 22a and opens second and third switches 22b and 22c of input selector 22. Since first switch 22a is closed, input audio/video signals from jack 14a are transmitted to a buffer 28. These audio/video signals are input to a pre-amplifier 30 from buffer 28 and subsequently output from speaker 20 via a volume controller 32 and a power amplifier 34. The output of the buffer 28 is also transmitted to tape deck unit 16 through a buffer amplifier 36 as an input signals.
The output of buffer amplifier 36 is supplied through jack 14d and input to jack 14g on tape deck unit 16 through a first connection cable 38a. The signal input to jack 14g is amplified by a recording amplifier 40 and, if tape deck unit 16 is in the recording mode, is added to a bias signal from a bias oscillator 42 and then applied as a recording signal to the record/playback head 26 via a first head circuit switch 44a which is closed. At this time, first microcomputer 12 transmits data indicating the closed state of first switch 22a and the open states of the open states of the second and third switches, 22b and 22c, to the second microcomputer 24 of tape deck unit 16 through the communication line 18.
When the keyswitch 12c of amplifier unit 10 is closed, first microcomputer 12 closes the third switch 22c of input selector 22 and opens the first and second switches, 22a and 22b. If tape deck unit 16 is in the playback mode, played-back signals from the tape are input from a jack 14h on tape deck unit 16 to jack 14c on the amplifier unit 10 through a second connection cable 38b and are passed through third switch 22c to the speaker 20. Also, first microcomputer 12 transmits data indicating the closed state of third switch 22c and the open states of first and second switches, 22a and 22b, to the second microcomputer 24.
Thus, if tape deck unit 16 is in playback mode, second microcomputer 24 places the cassette mechanism into playback mode according to data from first microcomputer 12 and inputs the played-back signals obtained from the record/playback head 26 via a second head circuit switch 44b which is closed and a playback amplifier 46 to the jack 14c. It is thus possible to play back the signals through the speaker 20.
Accordingly, during recording, signals are passed from amplifier unit 10 to tape deck unit 16 through first connection cable 38a connected between the jacks 14d and 14g, while second connection cable 38b connected between jacks 14c and 14h is not used. Conversely, during playback, signals are passed from tape deck unit 16 to amplifier unit 10 through second connection cable 38b connected between jacks 14h and 14c, while first connection cable 38a connected between jacks 14d and 14g is not used. As a result, during any operation of the prior art device shown in FIG. 1 either one of the connection cables 38a and 38b connected between amplifier unit 10 and tape deck unit 16 are not used and is, thus, wasted.
Although the above exemplary description is directed to analog audio signals, the same applies for digital audio signals from a digital audio tape recorder or to video signals from a VTR.
The prior art input/output switching circuit described above cannot operate correctly unless the respective signal lines are connected to recording amplifier 40 of tape deck unit 16 for signals from the amplifier unit 10, and to input selector 22 of amplifier unit 10 for playback signals from tape deck unit 16. In this way, a pair of connection cables for the recording input signal and the playback output signal are required between tape deck unit 16 and amplifier unit 10. Therefore, when the number of amplifier units and tape deck units is increased, the number of connection cables, large as it already is, will increase further.

Input/output switching circuit for connecting audio/video units


An optical modulator includes: a bottom substrate layer, having a first surface; a traveling-wave electrode, being disposed on the first surface of the bottom substrate layer and including a plurality of ground electrodes and a plurality of signal electrodes between the ground electrode; an optical waveguide disposed inside the bottom substrate layer; and a shield layer, including a substrate and a metal layer, the substrate covering at least a portion of the traveling-wave electrode and the metal layer being disposed on the surface of the substrate facing away from the traveling-wave electrode. Each of the ground electrodes is electrically connected to the metal layer to provide electromagnetic shielding for the signal electrodes between the ground electrodes.

An existing optical modulation system generally includes a driver and a modulator. Traveling-wave electrode modulators have become widely used because of their higher light extinction ratio and ease of integration. Traveling-wave electrode modulators are categorized as single-electrode driven and dual-electrode driven on the basis of the driving mode, and as serial push-pull structure and separate dual-arm structure on the basis of the optical waveguide configuration. In general, as illustrated in FIG. 1, a traveling-wave electrode modulation system 100 primarily includes a driver 20 and a traveling-wave electrode modulator 10. A diagram illustrating a cross section of the traveling-wave electrode modulator 10 is shown in FIG. 2.
The principle of operation of the traveling-wave electrode modulation system 100 is as follows. The driver 20 is connected to the traveling-wave electrode modulator 10 by means of bonded leads. An optical waveguide 130 is placed in the electric field of a traveling-wave electrode 120. A high-speed digital signal output from the driver 20 reaches the traveling-wave electrode modulator 10 and propagates along the traveling-wave electrode 120. A light wave propagates in the optical waveguide 130. A change in the electric field produced by the high-speed digital signal in the traveling-wave electrode 120 causes a change to the effective refractive index of the optical waveguide 130. Therefore, when the high-speed digital signal propagates in the traveling-wave electrode 120, the electric field of the high-speed digital signal causes a change to the refractive index of the optical waveguide 130, thereby causing a change to the phase of the light wave and causing the light wave to carry the digital signal information. Light wave interference occurs in a Mach-Zehnder interferometer at a back end, thereby completing the modulation.
During actual use, the entire traveling-wave electrode modulator chip generally is a complete chip with four, eight, or even more channels. The more the channels, the smaller the spacings between the channels, resulting in crosstalk between the channels on the chip. FIG. 3 is a diagram illustrating the electromagnetic field radiation inside a traveling-wave electrode modulator during use. As shown in FIG. 3, an optical modulator is installed on a heat sink, i.e., a heat sink pad 30 is disposed below a bottom substrate layer 110 of the chip. The heat sink pad 30 is typically a metal pad and, at the same time, functions as a metal ground layer. The electromagnetic field 150 of a signal electrode 121 might radiate into free space. Such radiation causes the problem of electromagnetic radiation. The signal electrode 121 is disposed between ground electrodes 122. A part of the electromagnetic field 150 might also couple to an adjacent channel to cause crosstalk. The crosstalk produces a noise in the signal, eventually affecting the bit error rate of a link. Moreover, a common mode voltage output from the driver 20 might cause relatively high electromagnetic radiation from the traveling-wave electrode at the end closer to the traveling-wave electrode modulator chip, causing failure of module authentication and other problems.


A shotgun magazine receiver assembly includes an elongated receiver body having a trigger assembly opening, a removable magazine opening, a shotgun barrel opening, a shell ramp and a bolt and slide. The bolt and slide are positioned to align with the center of the barrel opening and to engage a removable box-style shotgun shell magazine. A shotgun magazine receiver assembly also includes a plurality of connectors to secure the assembly to a stock barrel, grip, trigger assembly, and magazine tube of a conventional shotgun.

The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
FIGS. 1 and 2 illustrate one embodiment of a conventional pump-action shotgun that is useful for understanding the inventive concepts disclosed herein. As shown, the conventional shotgun 1 typically includes an elongated fixed magazine tube 2 that is mounted below the gun barrel 3. The tube and barrel are secured to a receiver 4 that houses a trigger assembly 5, a bolt 6 and a bolt slide 7. A grip 8 and forearm handle 9 are also provided.
The fixed magazine tube holds about five shells or cartridges (not illustrated) which are loaded individually into the magazine via the shell opening 4a. After firing each round, the user must pump the forearm 9 to chamber the next round into the receiver 4 for firing. After all of the shells are fired, the next round of shells are loaded one at a time into the fixed tubular magazine, and the above process is repeated.
Although rapid firing and reloading capabilities have been achieved with some small firearms, such as assault rifles and pistols, they have not yet been satisfactorily achieved with shotguns.
Moreover, many firearm owners routinely modify their weapons to suit a particular interest, look, or to accomplish a desired function. Owing to the extremely durable nature of shotguns in particular, many shotgun owners often choose to upgrade their existing weapons with new stocks and barrels as opposed to purchasing a new weapon. Accordingly, it would be beneficial to provide a shotgun magazine receiver assembly which can replace the stock receiver of an existing shotgun to allow a user to rapidly fire and reload ammunition via a removable "box" style magazine. Such a feature would be particularly beneficial for police and military applications, where the time necessary to reload a weapon can mean the difference between life and death.


A method and means for memorizing path deviations of a manipulator arm, and recalling the same deviations to adjust succeeding passes. During the seam-tracking pass, any feature or sensor system can be used which will cause the manipulator supported tool to deviate from the programmed path of travel. Offsets generated during the initial seam-tracking paths are memorized and thereafter recalled so that the manipulator arm can perform subsequent passes along the seam with the seam-track function off. While seam-tracking, the offsets are automatically memorized at distinct preprogrammed offset distance intervals. Each offset is placed sequentially in a memory data table. The offsets are recalled during a subsequent pass at each offset distance interval at which they were memorized so that the subsequent pass has the same deviations as the initial seam-tracked pass. Subsequent passes can be executed in the same direction as the initial seam-tracking pass, or in the opposite direction. In the first instance, offsets are recalled from the memory data table in a first-in, first-out basis. In the second instance, the offsets are recalled in a first-in, last-out basis.

In its most usual form, a robot or manipulator comprises a series of arm segments terminating in a wrist assembly. The wrist assembly supports a face plate to which an appropriate tool is affixed. The nature of the tool will depend upon the work to be performed by the manipulator.
The manipulator is provided with a control. The control is programmed to cause the manipulator to move the tool along a programmed path of travel and to execute programmed functions associated with programmed locations along that path of travel.
In order for the manipulator to perform operations correctly, it is necessary that the manipulator move the designated center point of the tool accurately along the programmed path of travel. Some operations require greater accuracy than others. Examples of operations requiring particular accuracy include sealant application, contour mapping, deburring and welding. While not intended to constitute a limitation, the present invention, for purposes of an exemplary showing, will be described in its application to welding operations.
Once a manipulator is taught a program, it will move along the programmed path with high repeatability. While this is satisfactory for many applications, those operations of the type mentioned above and needing particular accuracy will require adaptive path control. For example, in a welding operation, if the parts being joined are not uniform in seam or joint location by virtue of manufacturing tolerances, heat distortion, shifting of the clamping fixture for the parts, or the like, the manipulator will weld along its taught path, even though the seam may be located elsewhere.
Prior art workers have devised a number of systems for adaptive path control. Each system involves some sort of sensor. One system constitutes a pendant controlled root or initial pass wherein the operator creates deviations from the taught path of manipulator arm travel by eye and through the use of a pendant. This system is relatively slow and depends on the skill of the operator. A forced torque pressure tracking technique is known in the art, as well as a through-the-arc seam tracking system requiring that the seam be characterized by opposing sidewalls which are contacted by the tool center point (i.e., the tip of the welding tool) which traverses a weaving path. This weaving tracking imposes limitations on the welding speed. Yet another system is taught in copending application serial number 07/179,505, filed Apr. 8, 1988 in the names of Gregory Webb and Keith R. Wehmeyer and entitled VISION SEAM TRACKING METHOD AND APPARATUS FOR A MANIPULATOR. This system employs a laser scanner/camera head which previews the seam to be welded ahead of the welding torch. Data from the vision system is used to alter the manipulator's preprogrammed path of travel.
A problem which plagues the welding industry lies in the fact that many welding operations require successive weld passes to be made on the same seam. Thus, each pass must be seam-tracked, adding to the weld time required. In many instances seam-tracking cannot be performed over a weld bead. In addition, applications exist wherein the space provided will not accommodate the tracking equipment and the welding torch at the same time. Thus, a method and means are needed whereby the tracking equipment alone can be used to do an initial dry run or root pass determining the proper offset path of travel, followed by successive passes with the welding torch only, performing the welding operation based on the data observed in the initial seam-tracked pass. Even in those instances where no space problem exists, if an initial seam-tracked pass could be made (with or without concurrent welding), followed by successive welding passes without seam-tracking and based on the data observed in the seam-tracked pass, the multi-pass welding operation could be simplified, accomplished more quickly, and performed with less chance of damage to the seam-tracking equipment.
The present invention is based upon the development of a new feature enabling the manipulator operator to memorize the offsets generated during an initial seam-tracking pass, controlling the rate at which the offsets are accumulated. Thereafter, without the use of a seam-tracking system, the operator can recall the offsets on successive passes executed in the same direction as the initial seam-tracked pass or in a direction opposite to that of the seam-tracked pass. Thus, for example, alternate passes can be executed in the opposite direction, if desired. This new feature, with the aid of existing taught point offset registers (or the equivalent) enables the manipulator to perform multipass welding requiring some form of seam-tracking only on the initial or root pass. The present invention provides the manipulator operator with an exceptional degree of flexibility for both memorizing and recalling the offsets generated either manually or by means of a sensor or seam-tracking system.

Method and means for path offsets memorization and recall in a manipulator


A failure diagnostic apparatus determines the environment in which the condition of a vehicle is examined and selects an appropriate storage location for information regarding the condition of the vehicle. In particular, the failure diagnostic apparatus includes a vehicle-condition diagnostic portion, a plurality of failure-information storage portions, a diagnostic-environment determination portion, and a storage selection portion. When the vehicle-condition diagnostic portion determines that a failure has occurred, the diagnostic-environment determination portion determines the environment in which the condition of the vehicle is examined. Then, based on the diagnostic environment determined by the diagnostic-environment determination portion, the storage selection portion selects at least one storage portion of the plurality of failure-information storage portions to store information regarding the failure.

1. Field of the Invention
The invention relates to a failure diagnostic apparatus and a method of storing information failure. More particularly, the invention relates to a failure diagnostic apparatus and a method of storing failure information that easily determine the environment in which the condition of a vehicle is examined.
2. Description of the Related Art
Recently, it has been made mandatory to provide an On-Board Diagnostic-II (hereinafter, referred to as "OBD-II") system in vehicles such as automobiles, to prevent air pollution. The OBD-II system provides a warning to a driver if a failure occurs in an exhaust gas control device. In the OBD-II system, information regarding a failure is stored in non-volatile memory of an Electronic Control Unit (hereinafter, referred to as "ECU").
More specifically, in the OBD-II system, a Diagnostic Trouble Code (hereinafter, referred to as "DTC"), which contains information about a failure, is stored in the non-volatile memory. The DTC stored in the non-volatile memory is later retrieved using a scan tool so that the cause of the failure can be determined. The scan tool is a device dedicated to retrieving the DTC. After the cause of the failure is determined, and the cause of the failure is corrected by appropriate repair, the DTC stored in the non-volatile memory is erased using the erase function of the scan tool.
Japanese Utility Model Application Publication No. 2-49772 describes an airbag system that includes volatile memory and non-volatile memory provided in an ECU. The volatile memory or the non-volatile memory is appropriately selected to store the information about a failure that occurs in, for example, a collision detection portion in the airbag system. The response speed of the volatile memory is high, and the volatile memory is suitable for temporarily retaining records. Therefore, information that is frequently updated is stored in the volatile memory. For example, if a failure time is updated every one second or every one minute, the information on the failure time is stored in the volatile memory. In contrast, the non-volatile memory is suitable for permanently retaining records. Therefore, the information that is not updated frequently is stored in the non-volatile memory. For example, if the failure time is updated once every hour, the failure time is stored in the non-volatile memory.
Emission regulations tend to be tightened to take further measures against air pollution. Therefore, further requirements for the OBD-II system will be set. Particularly, it will be required to prohibit the DTC stored in the non-volatile memory from being erased by the scan tool, to more reliably retain the DTC.
In the system described in the above-described publication, the DTC about a failure is stored in the non-volatile memory. Therefore, even if a failure occurs, for example, in an assembly process when a vehicle is manufactured at a factory, the DTC about the failure is stored in the non-volatile memory in the same way as the DTC about the failure that occurred when a driver used the vehicle. Accordingly, if the DTC is prohibited from being erased in the OBD-II, the DTC about the failure that occurred when the vehicle was manufactured at the factory cannot be erased. In this case, when the DTC is retrieved using the scan tool, it cannot be determined whether the DTC was stored due to a failure that occurred when the vehicle was manufactured at the factory, or due to a failure that occurred when a driver used the vehicle.

Failure diagnostic apparatus and method of storing failure information


In some examples, an adapter includes a COMPACTFLASH EXPRESS (CFX) connector interface to connect to a CFX connector of a computer, and a device connector interface to connect to any of a plurality of different devices comprising different types of interfaces. The device connector interface includes an indicator settable to any of a plurality of different states to represent a respective type of the different types of interfaces when a device is connected to the adapter, and the CFX connector interface comprising an indicator connected to the indicator of the device connector interface.

A computer includes connectors to connect to various external devices. A connector can be in the form of a receptacle, a port, a socket, a plug, or any other structure useable to form an electrical and/or optical connection with another device. Various different connectors can be according to different standards.
Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements. The figures are not necessarily to scale, and the size of some parts may be exaggerated to more clearly illustrate the example shown. Moreover, the drawings provide examples and/or implementations consistent with the description; however, the description is not limited to the examples and/or implementations provided in the drawings.

Hewlett-Packard Development Company, L.P.


A biologically active conjugate is disclosed comprising a biopolymer and a therapeutic agent joined by a disulfide bond. The conjugate, when formulated in a pharmaceutical composition with a suitable carrier, has improved in vivo stability and activity, and can be targeted to a variety of cells, tissues and organs.

Biopolymers are biocompatible polymers that are useful for a wide variety of biomedical applications, such as for surgical aids, to prevent or reduce the formation of surgical adhesions, and for drug delivery applications. Many biopolymers are naturally occurring substances found in the body, and therefore do not have any unacceptable toxic or injurious effects on biological function. An example of such a biopolymer is hyaluronic acid (xe2x80x9cHAxe2x80x9d), a naturally occurring mucopolysaccharide found, for example, in synovial fluid, in vitreous humor, in blood vessel walls and the umbilical cord, and in other connective tissues. Hyaluronic acid consists of alternating N-acetyl-D-glucosamine and D-glucuronic acid residues joined by alternating xcex2 1-3 glucuronidic and xcex2 1-4 glucosaminidic bonds, so that the repeating unit isxe2x80x94(1xe2x86x924)-xcex2-D-GlcA-(1xe2x86x923)-xcex2-D-GlcNAcxe2x80x94. In water, hyaluronic acid dissolves to form a highly viscous fluid. The molecular weight of hyaluronic acid isolated from natural sources generally falls within the range of 5xc3x97104 up to 1xc3x97107 daltons.
U.S. Pat. No. 4,582,865, to Balazs et al. states, inter alia, that cross-linked gels of HA can slow the release of a low molecular weight substance that is dispersed therein but not covalently attached to the gel macromolecular matrix. See, also, U.S. Pat. No. 4,636,524, which contains a disclosure of related technology. Both of these patents describe HA compositions in which the HA is crosslinked by reaction with divinyl sulfone, and the use of the crosslinked HA compositions in drug delivery applications.
R. V. Sparer et al., 1983, Chapter 6, pages 107-119, in T. J. Roseman et al., Controlled Release Delivery Systems, Marcel Dekker, Inc., New York, describes sustained release of chloramphenicol covalently attached to hyaluronic acid by an ester linkage, either directly or in an ester complex including an alanine bridge as an intermediate linking group. The HA is modified by attaching cysteine residues to the HA via amide bonds, and then the cysteine-modified HA is crosslinked by forming disulfide bonds between the attached cysteine residues. Similarly, I. Danishefsky et al., 1971, in Carbohydrate Res., Vol. 16, pages 199-205, describe the modification of a mucopolysaccharide by converting the carboxyl groups of the mucopolysaccharide into substituted amides by reacting the mucopolysaccharide with an amino acid ester in the presence of 1-ethyl-3-(3-dimethylaminopropyl) carbodiimide hydrochloride (xe2x80x9cEDCxe2x80x9d) in aqueous solution. See, also, U.S. Pat. No. 4,937,270 and U.S. Pat. No. 5,760,220 which describe the modification of hyaluronic acid by reaction of the carboxyl groups of the biopolymer with a nucleophile to produce a water insoluble amide, and the use of those compositions for controlled release drug delivery.
A series of patents assigned to VivoRx Pharmaceuticals, Inc., describe compositions for the in vivo delivery of insoluble pharmaceutically active agents. Delivery of the drug substances is achieved, for instance, by encasing the active agent in a polymeric shell formed from a biocompatible polymer. The biocompatible polymer may be protein, lipid, DNA molecule or polysacharide, and the pharmaceutically active agent may be a therapeutic protein such as taxol. The polymer contains covalently attached sulfhydryl groups or disulfide linkages which can be crosslinked to form disulfide bonds. The polymeric shell is formed using ultrasonic irradiation techniques. These compositions are described as being less toxic, being more soluble, and having improved targeting as compared to prior art compositions. Relevant VivoRx patents include U.S. Pat. Nos. 5,498,421; 5,439,686; U.S. Pat. No. 5,362,478; U.S. Pat. No. 5,635,207; U.S. Pat. No. 5,560,933; U.S. Pat. No. 5,635,207 and U.S. Pat. No. 5,639,473.
U.S. Pat. No. 5,496,872 relates to biocompatible and biodegradable crosslinkable polymers having reactive thiol groups. The reactive thiol groups can be crosslinked to form disulfide linkages between adjacent molecules, resulting in a three dimensional network. These polymers can be used for binding tissues or binding tissues with implanted biomaterials.
U.S. Pat. No. 5,932,552 describes a keratin hydrogel having biomedical applications. The hydrogel is formed from crosslinked keratin bound by disulfide linkages. Among the biomedical applications described in the patent are uses of the hydrogels for cell scaffolding in tissue repair.
U.S. Pat. Nos. 5,354,853 and 5,451,661 describe, respectively, the preparation of phospholipid-saccharide conjugates, and lipids conjugated to biologically active agents such as peptides, proteins and nucleic acids. These conjugates are described as being particularly useful in drug delivery applications.
U.S. Pat. No. 5,902,795, to Toole et al., discloses hyaluronic acid oligosaccharides, having between one and sixteen repeating units, which are used to treat tumors in mammals. The patent states that the oligosaccharides act to reduce the level of membrane-associated hyaluronan-binding proteins, which are expressed on the surface of certain tumor cells during cell migration. The treatment is believed to reduce the incidence of tumor metastasis in the mammals.
A. Burnkop-Schnurch et al., J. Controlled Release, 2000, 66, 39, describes the synthesis of carboxymethyl cellulose (xe2x80x9cCMCxe2x80x9d) and polycarbophil modified with L-cysteine using carbodiimide chemistry. The polymers are reacted with the cysteine to form an amide bond between the primary amino group of the amino acid and the carboxylic acid of the polymer. The thiolated polymers were allowed to oxidize to form disulfide bridges. The dissolution of these tablets, both with and without drugs, was analyzed. The tablets were found to have improved stability and viscoelasticity.
Copending U.S. patent application Ser. No. 09/430,857 now abandoned relates to surfaces that have been modified by the attachment of hyaluronic acid. The surface can be part of a medical device, such as a stent or a surgical tubing. The surface is modified to include a reactive amino group that reacts with a derivatized hyaluronic acid. The modified devices and instruments are hydrophilic, and have anti-fouling and anti-platelet adhesion characteristics, thereby producing a reduction in risks associated with thrombosis.
The conjugated biopolymers of this invention represent a significant improvement over drug delivery vehicles of the prior art due, in part, to the site-specific reaction between the biopolymer and the therapeutic agent which increases the stability and activity of the therapeutic agent upon delivery to the desired site within a subject.
The present invention features a biopolymer-therapeutic agent conjugate in which the biopolymer and therapeutic agent are joined by a disulfide bond. The biologically active conjugate of this invention is useful as a drug delivery vehicle for the in vivo delivery of the therapeutic proteins to specific cells, organs or tissues in a subject. Drug delivery specificity is achieved by appropriate selection of the structure and molecular weight of the biopolymer.
The chemistry used to prepare the conjugates permits the site-specific reaction between the biopolymer and the therapeutic agent. The therapeutic agent contains a reactive thiol group, which can be present in an unmodified version of the therapeutic agent, as in the case of cysteine for example. Alternatively, the thiol group can be introduced into a modified version of a therapeutic agent that does not normally contain a reactive thiol group.
In one embodiment, the therapeutic agent can be reacted, through the reactive thiol group, with a chemically modified version of the biopolymer. This reaction typically occurs at a pH in the range of from about 6.0 to about 10. The biopolymer is activated and modified by reaction with an activating agent, such as a carbodiimide, and reacted with an organic disulfide compound. The organic disulfide compound contains a terminal group, such as an amino group or a hydroxyl group, which is reactive with the carboxylic acid group of the biopolymer in the presence of the activating agent. The reaction of the biopolymer, activating agent and organic disulfide compound occurs at a pH of from about 2.0 to 8.0.
In another embodiment, the therapeutic agent can be reacted, again through the thiol group, with the reducing end of the biopolymer. The biopolymer is first reacted with an organic disulfide compound containing a terminal group, such as an amino group or a hydroxyl group, which is reactive with the terminal carboxyl group of the biopolymer. The reaction of the biopolymer and organic disulfide compound occurs over a wide pH range, typically at a pH of from about 2.0 to 9.0.
In one aspect, the reaction of the biopolymer and therapeutic agent results in the attachment of the biopolymer to the therapeutic agent through a disulfide bond. The linking group or spacer, which can be a lower alkyl, separates the biopolymer from the therapeutic agent. The linking or spacer is a residue resulting from the cleavage of the organic disulfide compound by the reactive thiol of the therapeutic agent.
Typical biopolymers include any of the polyanionic polysaccharides, such as hyaluronic acid and any of its hyaluronate salts, such as sodium hyaluronate, potassium hyaluronate, magnesium hyaluronate and calcium hyaluronate, carboxymethyl cellulose, carboxymethyl amylose, chondroitin-6-sulfate, dermatin sulfate, heparin, and heparin sulfate, as well as polyacrylic acid, polycarbophil, carboxymethyl chitosan, poly-xcex1-glutamic acid, poly-xcex3-glutamic acid, carrageenan, and sodium alginate. The common feature of the biopolymers of this invention is that they are biocompatible, as that term is defined herein, they contain carboxylic acid functionality, and they can be modified to react with an organic disulfide compound. Such modification can occur, for instance, by reaction of the biopolymer with a suitable activating agent, such as a carbodiimide, to render the carboxylic group vulnerable to nucleophilic attack by, for instance, an amine or a hydroxyl. Alternatively, the modification can occur at the terminal or end group of the biopolymer by reduction of a terminal carbonyl group using a Schiff base.
In a preferred embodiment, the biopolymer is hyaluronic acid having a molecular weight in the range of from about 7.5xc3x97102 daltons to about 1xc3x97107 daltons. The hyaluronic acid is preferably activated by reaction with an activating agent to render it vulnerable to nucleophilic attack. Suitable activating agents for this purpose include carbodiimides, such as 1-ethyl-3-(3-dimethylaminopropyl) carbodiimide and 1-ethyl-3-(3-dimethylaminopropyl) carbodiimide methiodide.
The organic disulfide compound can be virtually any organic compound having a disulfide bond. Preferably, the disulfide bond is positioned at one end of an alkyl chain, while the other end of the chain terminates in a group reactive with the carbonyl group of the biopolymer. Preferably, the group that reacts with the biopolymer is an amino, carboxyl or hydroxyl group, but most preferably an amino group. In addition to being capable of reacting with the biopolymer, the organic disulfide compound is also capable of reacting with the active thiol group of the therapeutic agent. Preferred organic disulfide compounds include, in general, the nitro-pyridines, thio-pyridines, substituted S-phenyl disulphides, S-sulfonate derivatives, 9-anthrymethyl thioesters, S-carboxymethyl derivatives and nitro-thiobenzoic acid derivatives. More preferably, the organic disulfide compound is a thio-nitro-pyridine, and most preferably 3-nitro-2-pyridinesulfenyl-ethylamine.
The therapeutic agent is preferably one or more of the following: small organic molecules, proteins, nucleic acids, antibodies, peptides, amino acids, lipids, polysaccharides, cell growth factors, and enzymes. More preferably, the therapeutic agent is native or recombinant colony-stimulating factor (xe2x80x9cCSFxe2x80x9d), an amino acid or glucocerebrosidase. The therapeutic agent should contain a reactive thiol group to react with the modified biopolymer. The reactive thiol group can either be inherently part of the therapeutic agent, as in the case of cysteine, or the reactive thiol group can be introduced into the therapeutic molecule using known techniques. For example, a free thiol group can be introduced into a recombinant therapeutic protein molecule for conjugation and modification. Furthermore, some therapeutic drugs, such as Captoprilxe2x80x94a drug used to treat hypertensionxe2x80x94inherently contain a free sulfhydryl group as shown in the structure below:
The amino groups of therapeutic agents can be conveniently converted into thiols by reaction with Traut""s Reagent (aminothiolane).
The therapeutic agent is selected for the particular indication that is to be treated, and the biopolymer is selected, both as to its type and molecular weight, for its ability to target a particular organ, cell or tissue. For instance, a therapeutic agent for treating Gaucher""s Disease, a serious liver ailment, is the enzyme glucocerebrosidase. Glucocerebrosidase can be targeted to the liver by forming a conjugate with an appropriately sized hyaluronic acid molecule.
The biologically active conjugate of the present invention provides for improved stability of the therapeutic agent as compared to the use of the unconjugated or unmodified therapeutic agent, or the use of other carriers or conjugated compounds, such as polyethylene glycol (xe2x80x9cPEGxe2x80x9d) or lipids. The improved stability results in increased residence time in the body of a subject and increased circulation time in the blood stream. The conjugates of this invention also display improved targeting to specific tissues, organs and cells. Improved targeting is achieved through the selection of specific types and molecular weights of the biopolymers.
In a further aspect, the invention involves the attachment of a biopolymer onto the surface of a substrate by means of a disulfide linkage. The substrate can be a polymeric material, a ceramic or a metal. Preferably, the substrate is part of a medical device or instrument, such as a stent, graft, suture, catheter, tubing or guidewire. The substrate is modified to contain an amino group, which can then be converted into a thiol group. The substrate can then be reacted with the biopolymer modified with the organic disulfide compound to immobilize the biopolymer onto the substrate.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any method and materials similar or equivalent to those described herein can be used in the practice or testing of the present invention, the preferred methods and materials are now described. All publications mentioned herein, including published patent applications, and issued or granted patents, are hereby incorporated by reference in their entireties. Unless mentioned otherwise, the techniques employed or contemplated herein are standard methodologies well known to one of ordinary skill in the art. The materials, methods and examples are illustrative only and not intended to be limiting.
Other features and advantages of the invention will be apparent from the following description of the preferred embodiments, and from the claims.

Modification of biopolymers for improved drug delivery


A chip-type ceramic element contains a terminal electrode at each end and an inorganic insulating layer on the surface of the ceramic element between the electrodes. The terminal electrodes include a baked electrode formed from a conductive paste reacted with the material of the inorganic insulating layer. Layers of nickel and tin are plated on the baked electrode for improved heat resistance and soldering adhesion, respectively. The insulating layer prevents unwanted portions of the terminal electrodes from coming into contact with the ceramic element, thereby preventing dispersion in the resistance values of the element.

The present invention relates to a conductive ceramic element and, more particularly, to a ceramic element with terminal electrodes having a baked electrode layer, and an insulating inorganic layer encapsulating the surface of the ceramic body between the terminal electrodes.
A conductive chip-type ceramic element according to the prior art includes a ceramic body having silver-palladium electrodes fused at both ends thereof. The palladium imparts soldering heat resistance to the electrode, thereby preventing the silver from dissolving when soldering a chip-type thermistor to a substrate.
A drawback of the prior art is that the terminal electrode is manufactured by dipping the ends of the ceramic body into a silver-palladium paste. Regardless of the level of control exercised during the dipping process, however, dispersion of the material of the electrode near the edges of the thermistor body is inevitable. This produces a dispersion in the part-to-part resistance values of the resulting elements.
A further drawback of the prior art is that the increased palladium content causes a decrease in the solder adhesion of the electrode to the substrate, thereby establishing an upper limit on the amount of palladium which can be used. When soldering an electrode at high temperature for a long period of time, however, this upper limit provides insufficient palladium to impart adequate soldering heat resistance to the electrode.
It has been suggested that soldering heat resistance and soldering adhesion can be improved by providing a plating layer on the surface of the electrodes, as in the case of a chip-type capacitor. A drawback of this suggestion is that since the ceramic body of a thermistor is conductive (unlike the ceramic body of a capacitor), plating a conductive material directly on the surface of the ceramic body alters the resistance value of the ceramic element from the desired value. In addition, a portion of the ceramic is eroded by the plating liquid, thereby reducing the life and reliability of the ceramic element.
To overcome the above drawbacks, Applicant has filed Japanese Laid-Open Patent Publication No. 3-250,603, laid open in 1991, for a chip-type thermistor shown in FIG. 19. A ceramic body 10 is covered by a glass layer 4 and a baked electrode 16. A plated layer 3 covers baked electrode 16. Plated layer 3 is prevented from direct contact with ceramic body 10 by glass layer 4.
A drawback to Applicant's previous invention is that the sintered ceramic body is formed by coating both sides of a sheet of ceramic with glass. The glass-coated ceramic is then cut into pieces. This process requires numerous process steps and careful handling, thereby resulting in high manufacturing costs.
When forming a terminal electrode having plating layers on a conductive chip-type ceramic body, another conventional method utilizes ceramic material with a low conductivity. A drawback of this is that a low-conductivity chip requires a larger body due to its low magnetic permeability, which interferes with high density mounting. A substitution of a ferrit based ceramic material with a higher magnetic permeability avoids this inconvenience, but in turn experiences a shift in other characteristics due to plating material adhering to the ceramic body.
To overcome the above drawbacks, Applicant has filed Japanese Laid-Open Patent Publication No. 3-250,601 for a chip-type thermistor. Referring now to FIG. 20, an inner electrode 111 is formed on the end surfaces of ceramic body 10. A copper outer electrode 116 covers inner electrode 111.
A drawback to this approach is that a copper outer electrode 116 imparts significantly less heat resistance than a plated nickel electrode. The need for an outer layer that contacts portions of the surface of the thermistor body to provide good heat resistance, has limited the material for the outer electrode. This in effect complicates the conditions for burning.

Conductive chip-type ceramic element and method of manufacture thereof