The evaluation component of the Good Oral Health study consists of the survey, clinical assessment and skills assessment conducted at 4 time points throughout the study in English or Spanish depending on the desire of the participant. The evaluation design is displayed in the diagram. The time points are T0 to T3 and the timing of the assessments is the following:
T1=assessment after the first phase of intervention in each site. For the AMI, T1 was conducted approximately 1 month after implementation. For the Oral Health Campaign, T1 was conducted one month after the last of three fairs, with a window of about 4 weeks to allow for completion of all surveys and clinical assessments.
T2 was completed after the administration of the second phase of intervention in each site, using the timing rubric for T1.
T3 was conducted six months after T2 on a rolling basis. The entire intervention and assessment cycle took approximately 27 months.
Surveys were administered by trained research assistants. They were trained by the study PIs and the Survey Coordinator and were supervised by the survey coordinator. In addition, annual retraining sessions on the conduct of the survey took place for all study staff, conducted by the study PI’s. Click here to view the survey in english or spanish.
Clinical assessments were conducted at 4 time points throughout the study (T0 – T3) to collect data on Plaque Scores and Gingival Index. Trained and calibrated licensed dental hygienists conducted the exams under the supervision of the study’s Clinical Director (standard examiner). Click here to view the clinical assessment forms.
The Oral Health Skills Assessment
The Oral Health Skills Assessment was also conducted at 4 time points throughout the evaluation – prior to each survey (T0 – T2) and immediately preceding the administration of the abbreviated survey (GOHAI only) at T3. Click here to view the oral health skills assessment form.
Training, Quality Control and Calibration
All research staff underwent initial training in the overall study protocol, consenting, survey contents and administration, and clinical assessment of gingival status and plaque. Survey training was repeated each year. In this study, all field staff remained from the beginning of the study to its close in July, 2019. Thus survey retraining served as a reminder and to clarify any concerns about specific questions and responses on the survey. Surveys were entered using QDS, checked immediately post administration, and survey data underwent two stages of data cleaning in preparation for use. Quality control results were submitted in a report to the funder (Medical Monitoring report) every six months.
Interventionists conducting the AMI and oral health campaigns were involved in developing these interventions and thus familiar with them. The study intervention coordinator trained one additional interventionist in the conduct of the AMI in Spanish. Interventionists went over the interventions annually during staff training. The administration of the AMIs was checked every six months via a sample of 20 hard copy files and recordings 10 in English and 10 in Spanish. All campaigns were attended by one or both principal investigators. At least one PI attended bilingual campaign training sessions.
Clinical assessors were calibrated with the Dental Director annually, recalibrated with smaller samples, and observed by the Dental Director several times during each intervention cycle. Training of clinical assessors was divided into three phases as follows:
- During the instructional phase the clinical assessment team members were familiarized with research examination procedures and criteria for research assessments.
- The standardization phase consisted of training in the use of standard procedures and how to apply standard criteria for the oral health assessments.
- The calibration phase examined the degree of correlation within and among the clinical assessment team members and the standard examiner was measured. The reliability of the assessments is measured by determining the degree to which the examiners can produce uniform and consistent results when performing independent replicate examinations without discussion. Data from the calibration sessions are analyzed to measure correlation within and between each examiner and the standard examiner (e.g., Kappa and McNemar’s Test). If correlations between each of the examiners and the standard examiner are not within acceptable ranges or some examiners are consistently higher or lower than the standard examiner (i.e., significant McNemar’s test) additional training sessions were scheduled. For more information on the conduct of the calibrations contact the Principal Investigators
Monitoring and Recalibration
Continual gathering of clean, reliable data in a consistent and uniform manner is one of the main objectives of the clinical control trial. Several quality control procedures were carried out periodically to assure continuing quality of data gathered by the dental team throughout the duration of the study, with recalibration occurring prior to or at the beginning of each study cycle and interim observations and brief calibrations with one or two participants occurring once during each assessment cycle . Examiners and recorders periodically reviewed their training manuals to prevent deviation, or “drift” from the standards achieved during the training period.
The long duration of the study (5 years) required regularly scheduled retraining periods. In addition to the regularly scheduled recalibration sessions with the standard examiner, there were annual retraining sessions for each dental examiner, also conducted by the standard examiner. Annual retraining sessions took place for all study staff conducting the Oral Hygiene Skills Assessment, conducted by the Clinical Director.
The skills assessment was unstable and not validated, and was used in the study to provide a scoring scheme for interventionists as they observed participants demonstrating procedures for brushing, flossing and cleaning dentures.
For more information about the evaluation design for the clinical trial, contact the Principal Investigators.