- Statistics Without Statistics: Did They Use the Right Test? (continued)
P-value
When presenting article data we always ask about the significance of the data. Was there a significant
difference between group A and group B? Often this significance is listed as a p-value.
Significance
is a "more than chance" occurrence that if the authors were to do the study again the same
results would be found. A p-value of 0.05 (p=0.05) states that out of 100 repetitions of a study
all but 5 of them would find the same result. If p<0.01, then more than 99 times out of 100 the results
would be the same. By common convention the significance of a statistical analysis is placed at p<0.05.
Confidence Interval
Another important display of data "truth" is the confidence interval. Note that this is not
significance, this is confidence. Authors will use the confidence interval to describe how sure they are
the data is correct.
Often the confidence interval is set at 95%. This means the authors are 95%
sure the actual number for any variable being described is within a certain range of values. For
instance, if SBP is 166 in a control group and the intervention group has a SBP of 126 the authors may
have found a significant p-value of p<0.01 for the difference between the SBP's. While the authors
know that the test will be true in 99 out of 100 attempts they are not sure that the true SBP for each
group is the one they found this time. Therefore they place a range on the numbers based on the variance
within their population in order to instill confidence in their data. The new data would then be described
as Control SBP=166 (95% CI 156-172), Intervention SBP=126 (95% CI 120-142). The authors can then state
they are 95% sure the true value of SBP in each group is within the defined range.
The ranges themselves
do not demonstrate any significance. However, if CI's cross each other in an analysis then it is possible
that no significance actually exists. Also note that if the values are explained as an amount changed
then a CI crossing the value 0 holds less strength due to the possibility that no change occurred.
