Measuring Effectiveness and Outcomes Creating an Evaluation Plan
b. What measures can be used to document progress or success?

Sometimes it is difficult to select measures that are feasible for your program, particularly if you're dealing with transient populations or if you are providing only preventive services and relying on other systems or providers to follow up with your target population. Be realistic and be able to explain why you may not be able to sufficiently evaluate some aspects of the program, particularly if some external factors are out of your control.

Examples of some potential measures for mobile and portable programs are included in the text box. If you are having problems deciding what measures you should select, look for articles that highlight guidelines or evaluation strategies for programs similar to yours. View some examples:

Examples of Useful Data Measures for Evaluation Purposes

Patient/client information

  • # of people served by age group, ethnicity, zipcode where they live or if they are transient
  • how people get to the clinic
  • source of referral or how they found out about services
  • dental insurance or other financial coverage
  • extent and type of oral diseases (by age and ethnicity can be helpful for grants)
  • use of other dental services or a dental home
  • use of other health services through the same agency
  • # who needed communication in another language, and which languages

Service data

  • # of different types of services (diagnostic, preventive, restorative, or by specific procedure), by patient, by day, by provider or other variables
  • # of hours of operation and # of patients seen during different time periods (time of the day, months)
  • # of hours of clinically productive time vs getting to/from site, etc or performing administrative duties
  • # of hours or days of "down time" due to weather, holidays, equipment repairs, etc
  • # of hours of "no shows" or cancellations
  • % of consent forms returned by eligible children for preventive services

Financial data

  • estimated $ value of all procedures performed
  • % and $ amount of reimbursement for services billed
  • % and $ amount of revenue from individual or corporate donors
  • % and $ amount of revenue from grants or contracts
  • costs per child of applying sealants and/or fluoride varnish
  • costs per encounter, calculated to include travel time and other impacts

Quality improvement information

Satisfaction - often on a scale of 1-5 from "very satisfied" to "not at all satisfied"

  • patient satisfaction
  • staff satisfaction
  • sponsor/funder satisfaction
  • community partner satisfaction

    Clinical Care
  • % of sealants retained (partially or completely) after 1 year
  • % of treatment or prevention plans completed
  • scope and delivery of services manages risk and quality of care (e.g., patients receiving extractions or other surgical procedures are able to be followed for post-operative complications.)
  • % of patients on preventive recall schedule
  • periodontal disease indices indicate less bleeding and inflammation in patients on preventive recall

These measures can also be categorized as process measures or as outcome measures. Process measures answer the question, "How do we know we did what we said we'd do?" Outcome measures answer the question, "How do we know we achieved our intended results?"

The following table is one format for an evaluation plan that displays process and outcome measures separately (with data sources.) Data sources are documents or processes that will provide the information you need. A few examples are included. A column of intermediate outcomes is sometimes included as "mile markers" to help determine if the program is headed in the intended direction and at the intended speed.  

Objectives Process Measures with Data Sources Outcome Measures with Date Sources
Objective 2.2. By July 1, 2007, 40% of mothers receiving prenatal care through the health department's mobile van will also receive oral health anticipatory guidance, preventive dental services, and referral to a dental home. (baseline: 0%) - % of eligible women who sign consents (checklist)

- # and type of preventive dental services (billing or service spreadsheet)

- # of dental referrals made (checklist)

- # of woman who made appts at potential dental home (tracking form)
- types of support services used to make referral and schedule appt (tracking form)

- % of eligible women who receive each of the following:

-- OH anticipatory guidance

-- preventive dental services

-- referral to dental home (billing/service spreadsheet)

- % of eligible women who receive all 3 services (billing or service spreadsheet)

- % of women referred to a dental home who receive a treatment plan and are placed on recall (FU interview)