Measuring Effectiveness and Outcomes Creating an Evaluation Plan
c. What methods should we use to collect the evaluation information?

Once you select appropriate data measures, determine 1) what methods you will use to collect the data for the measures and 2) the time frames. Some methods, such as tracking forms, will be ongoing, while methods to acquire feedback from various groups may occur quarterly or at the end of a year. Each method selected should help you gather information for multiple measures. Sample methods include:

Written surveys: Response rates often are low if sent by mail, not personalized or no reminders are sent. They are difficult to use with transient or multicultural populations. There are many chances for misinterpreting questions or skipping questions. Variations include email or web-based surveys. Response rates may improve if 1) survey is short and easy to complete, 2) questions are easy to understand and answer, 3) incentives such as coupons or money are provided, 4) surveys are completed and collected onsite, or 5) surveys are sent home with children to parents with consent forms for screenings or services.
Individual interview: These can be done by telephone or in-person, preferably using a trusted member of the community or agency staff to acquire information and to clarify questions or answers. This method is useful for acquiring follow-up information on recommendations or referrals made, barriers to doing so, satisfaction with services, or recommendations for program improvements. Some people might hesitate to give candid answers if they feel their answers might negatively affect future services or if their confidentiality is not maintained. Others will appreciate being asked for their opinion and will provide valuable input. You may also want to collect information from "potential" or past users as well as active patients to analyze perceived barriers and to improve patient recruitment and retention.
Focus groups: These require careful selection of a variety of participants and a skillful facilitator who is perceived as objective. They are useful for acquiring perceptions about barriers to care, satisfaction with services, and recommendations for program improvements. Note that group discussion may influence individual responses. It is useful to provide food or other incentives to participants and offer childcare if parents are involved.
Chart/record reviews: Select a representative sample. This provides a good way to check for completeness and accuracy of information, appropriateness of care, number and type of services, number of appointments or missed appointments, and improvements in oral health. Use of dental practice management software allows manipulation of fields for input of identifiers for factors you need to measure.
Checklists or tracking forms: These are good for determining compliance with preventive maintenance of vehicles or equipment, inventories of supplies, infection control procedures, or other aspects of program operations.

To help you select appropriate evaluation methods and develop the best questions to gather the information you need, talk with groups that have created successful and sustainable programs and ask them to share lessons learned. Try to identify local consultants that have expertise in public health program evaluation.