Power analysis is a statistical procedure that computes the likelihood that the study you are setting up will result in findings that would be similar to those of the population. A .8 power tells you that if you conducted your study over and over again, 80% of the time it would result in the same findings as the population. Where statistical significance testing examines the extent to which the two groups you are comparing are different, power analysis produces a probability that the difference you observe actually exists in the population (probability of a Type II error in hypothesis testing).
Don't forget that a sample often has quite a different mean and distribution than the population (Figure 1), so in layman's terms, power analysis is tool to understand how big that difference can be without misleading you.
Power is strongly associated with sample size, so many researchers and quality improvement specialists use power analysis as a means of estimating a sample size that is associated with the power they would like their analysis or study to have. And this is the use we will make of power analysis in this post.
Figure 1. Potential population and sample distribution differences (Source: Master Black Belt, http://slideplayer.com/slide/7488193/)
First, before getting to power analysis, let me start by saying that the easiest way to think of estimating a sample size (in my mind) is to think of it as working backwards from your study results to your sample size. There is more complexity than that, but if you have ever run a t-test or chi-square, you know that sample size is very important to whether or not your analysis produces a statistically significant finding. Think of solving for sample size from a desired t-test result.
Having said this, power analysis can be a lot more complicated because your final analysis might be a lot more complicated than a t-test. But the principles are the same. All the things that affect your results when you are doing your analysis (e.g., the number of co-variates, use of hierarchical sampling, high variability in the dependent variable) will also play into estimating your needed sample size.
What this post provides you with are the basic principles of power analysis and an Excel spreadsheet with formulas built in to you can roll up your sleeves and give it a try. But before concluding that my estimated sample size is the one to go with, I always check-in with a statistician, because it can be far more complicated than this Excel tool suggests.
Let's start with a power analysis for a t-test (a comparison of two means). The t-test is one of the first procedures we reach for and is the basis for most statistical procedures using linear data.
As you know from the basic statistics posts in the blog, statistics revolves around the twin notions of central tendency and variability. Tests of statistical significance consider both of these, and so does power analysis. There are several things you need to know or to estimate in order to compute a sample size:
Power analysis also helps you understand the nature of the variability in your sample, so that you have a better handle on the sort of heterogeneity you may see in your sample. It is a great learning tool.
Another instance of the need for a population power analysis would be when there is an emerging health problem (like a new virus causing hospitalizations). You might want to know how many cases will need to be seen before you have your arms around the probabilities of different outcomes. For example, when HIV-AIDs and Ebola began to occur in the United States, it took several months or even years of cases for epidemiologists to advise clinicians and families on the range of outcomes for the diseases.
Finally, there may be times you are tracking the population over time, and you want to set some expectations on what the range of the mean of the population could be in the future. Power analysis can provide the population size you will need to avoid making too strong of an interpretation when a mean falls outside the confidence intervals.
UseSample Size Determination simplified v1.xlsx to play with sample size issues. The calculator provided here offers sample size calculations for the most common situations:
Power analysis presumes that you have employed random sampling, which is very hard to do in many clinical situations. If you cannot use random sampling, or your design involves stratified or clustered sampling, a statistician will be the person you reach out to in order to estimate the sample size you will need.
Power analysis is something that I play with a lot when I am designing a study (it gives mean lot of information that helps me lay out a strong design), but I always have a statistician review my assumptions and calculations before I begin work or submit a research proposal. Even if you are very confident in your ability to use power analysis to estimate your needed sample size, sample size is so critical to your study that a second set of eyes is worthwhile.
Comments will be moderated.