Statistical Significance - Confidence Intervals

Confidence intervals are ranges of rates in which the ‘true’ rate would fall a specified percentage of the time if repeated sampling were available. An area's observed rate should be considered an estimate of the true underlying rate. The number of events (deaths, hospitalizations, etc.) in an area varies by chance, depending on the number of persons counted as residents there and the probability of the event. Thus, rates based on small numbers are particularly unreliable.

Statistical reliability is especially important when comparing areas with each other. Therefore, statistical significance tests can be performed to determine whether the differences between two rates are probably the result of chance factors.

Statistical significance is easier to obtain with larger populations. The observed rate for indicators with a number of events that is less than 20 may be very different from the true underlying rate so that only a relatively large difference in rates would be considered significant. As the number of events grows larger, the chance component becomes less important and the observed rate is a better estimate of the true rate.

Confidence intervals in the MOPHIMS MICAs are calculated using one of two methodologies.

  • In MICAs that utilize population as the denominator for rate calculations, confidence intervals are calculated using an inverse gamma distribution when the number of events, or count, is less than or equal to 500. A Poisson distribution is used when the number of events is greater than 500. A more complete explanation of inverse gamma and Poisson tests is provided at the following link: https://www.cdc.gov/nchs/data/nvsr/nvsr63/nvsr63_09.pdf
  • The MICAs using this methodology are: Cancer Incidence MICA, Chronic Disease Death MICA, Chronic Disease Emergency Room MICA, Chronic Disease Inpatient Hospitalization MICA, Death MICA, Emergency Room MICA, Fertility and Pregnancy Rate MICA, Injury MICA, Inpatient Hospitalization MICA, Preventable Hospitalization MICA, and Procedures MICA.
  • In MICAs that utilize an internal denominator, all confidence intervals are calculated using a binomial distribution. The MICAs that utilize an internal denominator include:
    • Birth MICA (Most indicators use the denominator of total live births with known status.)
    • Pregnancy MICA (The denominator is all pregnancies.)
    • WIC MICAs (Most indicators use the denominator of total WIC participants with known status.)
On the MOPHIMS Community Data Profiles, if an "H" or "L" is present in the Significantly Different column, there is 95 percent confidence that the true rate in the selected geography is really higher ("H") or lower ("L") than the true state rate. That is, we estimate that there is only a 5 percent chance (1 in 20) that the difference between the selected geography’s rate and the corresponding state rate is due to random error or chance. If "N/S" is noted, the difference between the selected geography’s rate and the state rate is considered to be not statistically significant. Refer to the Trend Lines documentation to learn about the statistical significance calculations utilized by the Trend Lines feature of the Community Data Profiles. Also, refer to the Priorities MICA documentation to learn about the statistical significance calculations applied to the death trend lines in that tool.