1. An abnormal behavior detection apparatus for detecting abnormalities in data output from a system, comprising: a probabilistic distribution estimation apparatus for responding to, as input data, a string of vector data representing behavior to estimate, using a stochastic model, a probabilistic distribution occurred in each data by successively reading said string of vector data, said probabilistic distribution estimation apparatus comprising a parameter storage unit for storing all of parameters for the stochastic model having hidden variables, certainty calculation means for calculating, in response to said input data, a certainty occurred in said input data and that represents a level of assuredness that said input data is correct and does not contain errors, as the probabilistic distribution, using said stochastic model by reading the parameters of said stochastic model from said parameter storage unit, parameter renewal means for renewing the parameters in said parameter storage unit in accordance with new read data and with past data previously read using the certainty calculated by said certainty calculation means and using each parameter of said stochastic model read from said parameter storage unit, and parameter output means for outputting the parameters for the stochastic model stored in said parameter storage unit, wherein the parameters in said parameter storage unit are renewed based on a parameter renewal rule that unitizes both the certainty and the parameters of said stochastic model, said parameter renewal rule being an oblivious type algorithm for calculating a conditional expected value of a statistical amount weighted with an oblivion coefficient indicative of an accuracy of the past data as time progresses, the oblivious type algorithm be used to treat current data and the past data with appropriate weights in data analysis; and abnormality detection means for receiving the parameters for the stochastic model output from said parameter output means and for calculating a score indicative of an abnormal behavior degree with respect to the new read data by using the parameters of the probabilistic distribution estimated by said probabilistic distribution estimation apparatus to produce the abnormal behavior degree of said new read data, the abnormal behavior degree being a degree where the new read data is out of whole patterns, the abnormal behavior degree being an indicator of a level of abnormalities that exist in the data output from the system.