☰ MedicalStat

Compare Estimates


Section 1:

For inputs, that require log transformation (RR, OR, HR, IRR, ... etc.)

Explanations & examples:
Compare two estimates with each other by calculating either their difference or their ratio (quotient) and get the 95 % confidence interval of the difference or the quotient. Some estimates require logistic transformation before they can be compared and tested, for ex. odds ratios (OR), risk ratios (RR), incidence rate ratios (IRR), ... etc. Use section 1 if the estimates require log transformation, otherwise section 2. In section 1 we look at the difference between the log transformed values of the estimates and its confidence interval instead of the difference between the values themselves. Alternatively to this difference of the log transformed estimates we can also look at the quotient (ratio) between the estimates and the confidence interval of the quotient. If the 95 % confidence interval of the difference between the two log transformed estimates includes the number 0, then there is no statistically significant difference between the two estimates on a five percent significance level. This will be confirmed by the z-test which will have (in this case) a p-value above 0.05. If, on the other hand, 0 is not included in the 95 % confidence interval of this difference, then the two estimates are different from each other on a five percent significance level and the p-value will be below 0.05, meaning that the null hypothesis H0 will be rejected in this scenario (the null hypothesis claims that the two estimates are the same). If the quotient is used instead of the difference between the log transformed estimates, then the number 1 has to be included in the 95 % confidence interval of the quotient in order to accept H0, namely that the two estimates are the same; if 1 is included in the interval, the p-value will be greater than 0.05. If not, p < 0.05 and H0 is rejected and the estimates are significantly different.

Notice that we are dealing with estimates of the true values in these cases, since we don't have data from the entire population to deal with. For more info about the formulas used in the tests; please see the page medical statistics formulas.

Example:

We want to compare and test the two OR (odds ratio) values 0.98 and 1.06 after a logistic regression where the effect of age on the outcome (knee osteoarthritis) was investigated. The OR value for men is 0.98 with a 95 % confidence interval of [0.90 : 1.05], meaning that for every year a man gets older he has 0.98 times lower odds of the outcome. Lower, since the OR value is under 0. The OR value for women is 1.06 with a 95 % confidence interval of [1.02 : 1.11], meaning that for every year a woman gets older she has a 1.06 times higher odds of the outcome. It is first noticed, that only the OR value of 1.06 is significant, since the number 1.00 is not included in its confidence interval. Neither of the two estimates is included in the 95 % confidence interval of the other estimate. If this had been the case the OR values would not have been significantly different and the p-value in the test would have been above 0.05. Also; there is a little overlap between the confidence intervals (one ends at 1.05 and the other starts at 1.02). If there had been no overlap between the intervals, the OR values would have been significantly different and the p-value in the test would have been under 0.05. So in this case with some overlap we can't either reject or not reject the null hypothesis just by looking at the confidence intervals alone.

In general: By comparing the estimates using the confidence intervals alone (without testing) there are three possible scenarios:
1) There is no overlap between the two 95 % confidence intervals of the estimates.
2) One of the estimates is included in the confidence interval of the other estimate (or they are both included in each other's intervals).
3) None of the estimates is included in the other estimate's interval, but there is an overlap between the intervals.

In case 1) with no overlap H0 can be rejected, because the p-value in the test will be below 0.05. In case 2) H0 cannot be rejected on a five percent significance level and the p-value in the z-test will be above 0.05. In case 3) it isn't possible to conclude anything from the confidence intervals alone. Here either the test must be made and the p-value will be above or below 0.05 or the confidence interval of the quotient can be used to see if in includes 1. If not, p < 0.05 and H0 is rejected.

Two OR estimates being compared and tested for equality

The 95 % confidence interval of the difference between the log transformed estimates is [-0.1664 : 0.0094] and thus includes the number 0. Conversely, the 95 % confidence interval of the ratio between the two estimates is [0.8467 : 1.0095] and this interval includes 1. So the two OR values are not significantly different from each other on a five percent significance level. This is confirmed by the p-value in the test, which is 0.0802 and thus more than 0.05. So we do not reject the null hypothesis H0 in this case, that stated that the two OR estimates are the same.

The weighted estimate of the two OR values combined into one is 1.0409. Note that it is only meaningful to calculate and interpret the weighted estimate if the estimates are not significantly different from each other, like in this case. The weighted OR of 1.0409 is the common OR value for the connection between age and knee osteoarthritis having adjusted for sex. The weighted estimate is significant, since 1 is not included in its 95 % confidence interval of [1.0031 : 1.0802], confirmed by the p-value of 0.0339, which is below 0.05, so we reject the null hypothesis that WE = 1.

Enter Estimates and the 95 % confidence interval
Enter Estimates Enter Lower Bounds Enter Upper Bounds

Comparison (no testing) of the two estimates:
    Value 1         Value 2     % CI
\( \ln(estimate)\)
\( \ln(lower \; bound) \)
\( \ln(upper \; bound) \)
\( SE(\ln(estimate)) \)
Common \( SE(\ln(estimate)) \)
Difference \( \ln(e_1) - \ln(e_2) \)
Quotient \( \frac{e_1}{e_2} \)

Z-Test for equality:
H0 (Quotient) \( \frac{e_1}{e_2} \)
Z value P value
H0 (Difference) \( \ln(e_1) - \ln(e_2) \)

Weighted Estimate (WE)
    Estimate     \( SE(\ln(WE)) \) % CI
    H0 (WE)     Z value P value
\( \ln(WE) \) ln(WE)
\( WE \) WE

Decimals:

Section 2:

For inputs, that DON'T require log transformation (MD, RD, β values, ... etc.)

Explanations & examples:
If the two estimates are of a type that do NOT require log transformation (like for ex. mean values, slopes from linear regression, ... etc.) use section 2. In this case, the difference between the estimates is being calculated together with the 95 % confidence interval of this difference. If the confidence interval of the difference includes the number 0, then the two estimates are not significantly different from each other on a 5 % significance level. The p-value in the test for equality will then be above 0.05. If, on the other hand, the number 0 is not included in the confidence interval of the difference, then the estimates are significantly different from each other on a five percent significance level. The p-value in the z-test will then be under 0.05 and the null hypothesis H0 will be rejected. H0 states that the two estimates are equal.

Example:

The smoking status of the mothers is being observed in 654 babies born in a town. The 531 babies born by non-smoking mothers had a mean weight of 3.641 kilograms with a 95 % confidence interval of [3.6024 : 3.6796]. The 123 babies born by smoking mothers had a mean birth weight of 3.447 kilograms with 95 % CI [3.3648 : 3.5292]. We want to compare the two mean birth weights 3.670 and 3.538 to see if they can be assumed equal or not. If they cannot be assumed equal then the smoking status of the mother must have an effect on the birth weight of the baby. In other words the null hypothesis H0 in this case is;
H0: the two mean values are the same.
Or (expressed alternatively); there is no statistically significant difference between them on a five percent significance level. Opposite to this null hypothesis we have the alternative hypothesis H1, namely that there is, in fact, a significant difference between the means (they are not the same).

Two mean value estimates being compared and tested for equality

First of all, it is observed that the confidence intervals of the mean values do not overlap each other. Since there is no overlap in the intervals, the two means can not be equal and we can already now reject the null hypothesis H0. So in this case it wasn't even necessary to do the test to see if the means could be equal, since we already know now that the p-value will be below 0.05. This is confirmed by the 95 % confidence interval of the difference 0.1940 between the two means, which is [0.1032 : 0.2848] and it doesn't span the number 0 (the interval doesn't go from minus to plus). If you want to do the test anyway, the z-value is 4.1871 and the corresponding p-value is 0.0000 which is below 0.05. Since H0 is rejected on a five percent significance level, it is rejected that the mean weight of the babies born by smoking mothers is the same as the mean weight of the non-smoking mothers. So the alternative hypothesis H1 must be true, namely that they are different. So the mother's smoking status does have an impact on the birth weight of the newborn, for if not so the means could have been assumed equal.

In general: By comparing the estimates using the confidence intervals alone (without testing) there are three possible scenarios:
1) There is no overlap between the two 95 % confidence intervals of the estimates.
2) One of the estimates is included in the confidence interval of the other estimate (or they are both included in each other's intervals).
3) None of the estimates is included in the other estimate's interval, but there is an overlap between the intervals.

In case 1) with no overlap H0 can be rejected, because the p-value in the test will be below 0.05. In case 2) H0 cannot be rejected on a five percent significance level and the p-value in the z-test will be above 0.05. In case 3) it isn't possible to conclude anything from the confidence intervals alone. Here either the test must be made and the p-value will be above or below 0.05 or the confidence interval of the difference can be used to see if in includes 0. If not, p < 0.05 and H0 is rejected.

The weighted estimate (weighted average) of the two means is not of any meaningful use in this case, since the mean values are significantly different. It is only reasonable to combine two mean values into one by calculating the weighted average, if the two means in question are not significantly different from each other.

For more info about the formulas and procedures used in the tests; please see the page medical statistics formulas.

Enter Estimates and the 95 % confidence interval
Enter Estimates Enter Lower Bounds Enter Upper Bounds

Comparison (no testing) of the two estimates:
    Value 1         Value 2     % CI
\( SE(estimate) \)
Common \( SE(estimate) \)
Difference \( e_1 - e_2 \)

Z-Test for equality:
H0 (Difference) Z value P value
Difference \( e_1 - e_2 \)

Weighted Estimate
    WE         SE     % CI
    H0 (WE)     Z value P value
WE