Title: Using Confidence Intervals to Compare Means - Statistics By Jim
Headings: Using Confidence Intervals to Compare Means
Using Confidence Intervals to Compare Means
Comparing Groups Using Confidence Intervals of each Group Estimate
Jumping to Conclusions
Using the Wrong Types of Confidence Intervals
Assessing Confidence Intervals of the Differences between Groups
Interpreting Confidence Intervals of the Mean Difference
References
Comments and Questions
Cancel reply

Content: For more information about this issue, see Practical vs. Statistical Significance. When the interval is too wide (imprecise) to be helpful and/or the range includes differences that are not practically significant, you have reason to hesitate before making decisions based on the results. These types of CI results indicate that you might not obtain meaningful benefits even though the difference is statistically significant. There’s no statistical method for answering questions about how precise an estimate must be or how large an effect must be to be practically useful. To use the confidence interval of the difference to answer these questions, you’ll need to apply your subject-area knowledge. For the example in this post, it’s important to note that the low end of the CI is very close to zero. It will not be surprising if the actual population difference falls close to zero, which might not be practically significant despite the statistically significant result. If you are considering switching to Group B for a stronger product, the mean improvement might be too small to be meaningful. When you’re comparing groups, assess confidence intervals of those differences rather than comparing confidence intervals for each group. This method is simple, and it even provides you with additional valuable information.