Statistical Significance - Confidence Intervals
Confidence intervals are ranges of rates in which the ‘true’ rate would fall a specified percentage of the time if repeated sampling were available. An area's observed rate should be considered an estimate of the true underlying rate. The number of events (deaths, hospitalizations, etc.) in an area varies by chance, depending on the number of persons counted as residents there and the probability of the event. Thus, rates based on small numbers are particularly unreliable.
Statistical reliability is especially important when comparing areas with each other. Therefore, statistical significance tests can be performed to determine whether the differences between two rates are probably the result of chance factors.
Statistical significance is easier to obtain with larger populations. The observed rate for indicators with a number of events that is less than 20 may be very different from the true underlying rate so that only a relatively large difference in rates would be considered significant. As the number of events grows larger, the chance component becomes less important and the observed rate is a better estimate of the true rate.
Confidence intervals in the MOPHIMS MICAs are calculated using one of two methodologies.
- In MICAs that utilize population as the denominator for rate calculations, confidence intervals are calculated using an inverse gamma distribution when the number of events, or count, is less than or equal to 500. A Poisson distribution is used when the number of events is greater than 500. A more complete explanation of inverse gamma and Poisson tests is provided at the following link: https://www.cdc.gov/nchs/data/nvsr/nvsr63/nvsr63_09.pdf
- The MICAs using this methodology are: Cancer Incidence MICA, Chronic Disease Death MICA, Chronic Disease Emergency Room MICA, Chronic Disease Inpatient Hospitalization MICA, Death MICA, Emergency Room MICA, Fertility and Pregnancy Rate MICA, Injury MICA, Inpatient Hospitalization MICA, Preventable Hospitalization MICA, and Procedures MICA.
- In MICAs that utilize an internal denominator, all confidence intervals are calculated using a binomial distribution. The MICAs that utilize an internal denominator include:
- Birth MICA (Most indicators use the denominator of total live births with known status.)
- Pregnancy MICA (The denominator is all pregnancies.)
- WIC MICAs (Most indicators use the denominator of total WIC participants with known status.)