what is goodness of fit 
the term goodness of fit refers to a statistical test that determines how well sample data fits a distribution from a population with a normal distribution put simply it hypothesizes whether a sample is skewed or represents the data you would expect to find in the actual population goodness of fit establishes the discrepancy between the observed values and those expected of the model in a normal distribution case there are multiple methods to determine goodness of fit including the chi square understanding goodness of fitgoodness of fit tests are statistical methods that make inferences about observed values for instance you can determine whether a sample group is truly representative of the entire population as such they determine how actual values are related to the predicted values in a model when used in decision making goodness of fit tests make it easier to predict trends and patterns in the future as noted above there are several types of goodness of fit tests they include the chi square test which is the most common as well as the kolmogorov smirnov test and the shapiro wilk test the tests are normally conducted using computer software but statisticians can do these tests using formulas that are tailored to the specific type of test to conduct the test you need a certain variable along with an assumption of how it is distributed you also need a data set with clear and explicit values such as goodness of fit tests are commonly used to test for the normality of residuals or to determine whether two samples are gathered from identical distributions establishing an alpha levelin order to interpret a goodness of fit test it s important for statisticians to establish an alpha level such as the p value for the chi square test the p value refers to the probability of getting results close to extremes of the observed results this assumes that the null hypothesis is correct a null hypothesis asserts there is no relationship that exists between variables and the alternative hypothesis assumes that a relationship exists instead the frequency of the observed values is measured and subsequently used with the expected values and the degrees of freedom to calculate chi square if the result is lower than alpha the null hypothesis is invalid indicating a relationship exists between the variables types of goodness of fit tests 2 i 1 k o i e i 2 e i chi 2 sum limits k i 1 o i e i 2 e i 2 i 1 k oi ei 2 ei the chi square test which is also known as the chi square test for independence is an inferential statistics method that tests the validity of a claim made about a population based on a random sample used exclusively for data that is separated into classes bins it requires a sufficient sample size to produce accurate results but it doesn t indicate the type or intensity of the relationship for instance it does not conclude whether the relationship is positive or negative to calculate a chi square goodness of fit set the desired alpha level of significance so if your confidence level is 95 or 0 95 then the alpha is 0 05 next identify the categorical variables to test then define hypothesis statements about the relationships between them 1variables must be mutually exclusive in order to qualify for the chi square test for independence and the chi goodness of fit test should not be used for data that is continuous named after russian mathematicians andrey kolmogorov and nikolai smirnov the kolmogorov smirnov k s test is a statistical method that determines whether a sample is from a specific distribution within a population this test which is recommended for large samples e g over 2000 is non parametric that means it does not rely on any distribution to be valid the goal is to prove the null hypothesis which is the sample of the normal distribution like chi square it uses a null and alternative hypothesis and an alpha level of significance null indicates that the data follow a specific distribution within the population and alternative indicates that the data did not follow a specific distribution within the population the alpha is used to determine the critical value used in the test but unlike the chi square test the kolmogorov smirnov test applies to continuous distributions the calculated test statistic is often denoted as d it determines whether the null hypothesis is accepted or rejected if d is greater than the critical value at alpha the null hypothesis is rejected if d is less than the critical value the null hypothesis is accepted 2s i 1 n 2 i 1 n ln f y i ln 1 f y n 1 i s sum i 1 n frac 2i 1 n ln f y i ln 1 f y n 1 i s i 1n n 2i 1 lnf yi ln 1 f yn 1 i the anderson darling a d test is a variation on the k s test but gives more weight to the tails of the distribution the k s test is more sensitive to differences that may occur closer to the center of the distribution while the a d test is more sensitive to variations observed in the tails 3 because tail risk and the idea of fatty tails is prevalent in financial markets the a d test can give more power in financial analyses like the k s test the a d test produces a statistic denoted as a2 which can be compared against the null hypothesis the shapiro wilk s w test determines if a sample follows a normal distribution the test only checks for normality when using a sample with one variable of continuous data and is recommended for small sample sizes up to 2000 the shapiro wilk test uses a probability plot called the qq plot which displays two sets of quantiles on the y axis that are arranged from smallest to largest if each quantile came from the same distribution the series of plots are linear the qq plot is used to estimate the variance using qq plot variance along with the estimated variance of the population one can determine if the sample belongs to a normal distribution if the quotient of both variances equals or is close to 1 the null hypothesis can be accepted if considerably lower than 1 it can be rejected just like the tests mentioned above this one uses alpha and forms two hypotheses null and alternative the null hypothesis states that the sample comes from the normal distribution whereas the alternative hypothesis states that the sample does not come from the normal distribution 4other goodness of fit testsaside from the more common types of tests mentioned above there are numerous other goodness of fit tests an analyst can use a very general rule of thumb is to require that every group within a goodness of fit test have at least five data points this ensures that sufficient information is fed into the test to determine the distribution importance of goodness of fit testsgoodness of fit tests are important in statistics for many reasons first they provide a way to assess how well a statistical model fits a set of observed data the main importance of running a goodness of fit test is to determine whether the observed data are consistent with the assumed statistical model by extension a goodness of fit test may be useful in choosing between different models which may better fit the data goodness of fit tests can also help to identify outliers or market abnormalities that may be affecting the fit of the model outliers can have a large impact on the model fit and may need to be removed or dealt with separately sometimes outliers are not easily identifiable until they have been integrated into an analytical model goodness of fit tests can also provide information about the variability of the data and the estimated parameters of the model this information can be useful for making predictions and understanding the behavior of the system being modeled based on the data being fed into the model it may be necessary to refine the model specific to the dataset being tested the residuals being calculated and the p value for potentially extreme data goodness of fit test vs independence testgoodness of fit test and independence test are both statistical tests used to assess the relationship between variables therefore it may be easy to confuse the two however each are designed to answer different questions a goodness of fit test is used to evaluate how well a set of observed data fits a particular probability distribution on the other hand an independence test is used to assess the relationship between two variables it is used to test whether there is any association between two variables the primary purpose of an independence test is to see whether a change in one variable is related to a change in another variable an independence test is typically used when the research question is focused on understanding the relationship between two variables and whether they are related or independent in many cases an independence test is pointed towards two specific variables i e does smoking cause lung cancer on the other hand a goodness of fit test is used on an entire set of observed data to evaluate the appropriateness of a specific model goodness of fit examplehere s a hypothetical example to show how the goodness of fit test works suppose a small community gym operates under the assumption that the highest attendance is on mondays tuesdays and saturdays average attendance on wednesdays and thursdays and lowest attendance on fridays and sundays based on these assumptions the gym employs a certain number of staff members each day to check in members clean facilities offer training services and teach classes but the gym isn t performing well financially and the owner wants to know if these attendance assumptions and staffing levels are correct the owner decides to count the number of gym attendees each day for six weeks they can then compare the gym s assumed attendance with its observed attendance using a chi square goodness of fit test for example now that they have the new data they can determine how to best manage the gym and improve profitability 