Browsing by Subject "validity"

Now showing 1 - 7 of 7
  • Thumbnail Image
    Item
    Civic-Minded Graduate: A North Star (Assessment Tools)
    (9/13/2011) Steinberg, Kathryn S.; Hatcher, Julie A.; Bringle, Robert G.
    Because of increased interest in higher education regarding the civic learning outcomes for college students and graduates, identifying and measuring civic learning outcomes is important to evaluating the efficacy of civic engagement programs and teaching strategies (e.g., service- learning). A conceptual framework for the Civic-Minded Graduate (CMG) construct is presented as well as three measurement procedures (i.e., CMG Scale, CMG Narrative Prompt and Rubric, CMG Interview Protocol and Rubric) that evaluate the construct. Results from three studies provide evidence of the psychometric properties of each measurement procedure and converging evidence to support the meaningfulness of the CMG construct. Implications of adopting the CMG as a “north star” for future research and for practice are presented.
  • Thumbnail Image
    Item
    Clinical Assessment and Management Examination—Outpatient (CAMEO): Its Validity and Use in a Surgical Milestones Paradigm
    (Elsevier, 2015-01) Wilson, Adam B.; Choi, Jennifer N.; Torbeck, Laura J.; Mellinger, John D.; Dunnington, Gary L.; Williams, Reed G.; Department of Surgery, IU School of Medicine
    Objectives Clinical Assessment and Management Examination—Outpatient (CAMEO) is a metric for evaluating the clinical performance of surgery residents. The aim of this study was to investigate the measurement characteristics of CAMEO and propose how it might be used as an evaluation tool within the general surgery milestones project. Design A total of 117 CAMEO evaluations were gathered and used for analysis. Internal consistency reliability was estimated, and item characteristics were explored. A Kruskal-Wallis procedure was performed to discern how well the instrument discriminated between training levels. An exploratory factor analysis was also conducted to understand the dimensionality of the evaluation. Setting CAMEO evaluations were collected from 2 departments of surgery geographically located in the Midwestern United States. Combined, the participating academic institutions graduate approximately 18 general surgery residents per year. Participants In this retrospective data analysis, the number of evaluations per resident ranged from 1 to 7, and evaluations were collected from 2006 to 2013. For the purpose of data analysis, residents were classified as interns (postgraduate year 1 [PGY1]), juniors (PGY2-3), or seniors (PGY4-5). Results CAMEO scores were found to have high internal consistency (Cronbach’s α = 0.96), and all items were highly correlated (≥0.86) to composite CAMEO scores. Scores discriminated between senior residents (PGY4-5) and lower level residents (PGY1-3). Per an exploratory factor analysis, CAMEO was revealed to measure a single dimension of “clinical competence.” Conclusions The findings of this research aligned with related literature and verified that CAMEO scores have desirable measurement properties, making CAMEO an attractive resource for evaluating the clinical performance of surgery residents.
  • Thumbnail Image
    Item
    Content Validity and Satisfaction With a Stroke Caregiver Intervention Program
    (Wiley, 2009) Bakas, Tamilyn; Farran, Carol J.; Austin, Joan K.; Given, Barbara A.; Johnson, Elizabeth A.; Williams, Linda S.; School of Nursing
    Background and Purpose Establishing evidence of content validity and satisfaction is an integral part of intervention research. The purpose of this article is to describe content validity and satisfaction relative to the Telephone Assessment and Skill-Building Kit (TASK), an 8-week follow-up program based on individualized assessment of stroke caregiver needs. Design and Methods The TASK intervention enables caregivers to develop skills based on assessment of their own needs. During the development of the TASK program, 10 experts rated the validity of the TASK intervention components for accuracy, feasibility, acceptability, and problem relevance. After incorporating feedback from the experts, a randomized controlled clinical trial was instituted using a convenience sample of 40 stroke caregivers to determine satisfaction (usefulness, case of use, and acceptability) with the TASK intervention (n=21) compared with an attention control group (n=19). Data collection occurred between March 2005 and June 2006. Data were analyzed using descriptive statistics, independent sample t tests, and content analysis. Findings Expert ratings on a 1 to 5 scale, with 5 being strongly agree, provided evidence of content validity (accuracy 4.71, feasibility 4.46, acceptability 4.40, problem relevance 4.67). Caregivers in the TASK group scored significantly higher than the attention control group on all satisfaction measures (usefulness p=.02; ease of use p=.02; acceptability p=.05). Qualitative comments from caregivers provided further evidence of satisfaction. Conclusions Evidence of content validity and user satisfaction for the TASK intervention relative to an attention control group was found. Clinical Relevance The TASK program may be a viable telephone-based program that can be implemented by nurses to support family caregivers during the first few months after stroke survivors are discharged home.
  • Thumbnail Image
    Item
    Psychometric Properties of the Arabic Version of the Functional Assessment of Chronic Illnesses Therapy–Fatigue in Arabic Cancer Patients
    (Elsevier, 2020-01) Al Maqbali, Mohammed; Hughes, Ciara; Gracey, Jackie; Rankin, Jane; Hacker, Eileen; Dunwoody, Lynn; School of Nursing
    Background The Functional Assessment of Chronic Illness Therapy (FACIT) is a measurement system that was developed to assess the health-related quality of life among patients with cancer and other chronic illnesses. The Functional Assessment of Chronic Illness Therapy–Fatigue (FACIT-F) is a 40-item questionnaire, and it is one of the most frequently used instruments to assess fatigue in cancer populations. The aim of this study was to evaluate the psychometric properties of the Arabic FACIT-F among patients diagnosed with cancer. Methods Following a translated and cross-cultural evaluation procedure of the FACIT-F Arabic version, a cross-sectional and descriptive correlational design was conducted. A total of 369 patients with cancer completed the FACIT-F, which consists of the 27-item Functional Assessment of Cancer Therapy–General (FACT-G) and the 13-item Functional Assessment of Chronic Illness Therapy–Fatigue (FACIT-F). The scale was assessed in terms of acceptability, internal consistency, and validity. Construct validity was explored through confirmatory factor analysis. Results The FACT-G had acceptable fit in the four-factor model, whereas the FACIT-Fatigue was found to be acceptable for the one-factor model in Arabic patients diagnosed with cancer. The Cronbach's alpha coefficient for the Arabic FACIT-Fatigue was 0.92, whereas the total score for FACT-G was 0.92, which showed good reliability. There was evidence that discriminated validity analysis was generally very good for the FACIT-Fatigue and FACT-G Arabic versions. Conclusion The Arabic versions of the FACIT-Fatigue and FACT-G demonstrated good reliability and validity for assessing fatigue and quality of life in patients diagnosed with cancer.
  • Thumbnail Image
    Item
    Skills on Wheels: Program evaluation and modifications to increase the reliability and validity of the Wheelchair Skills Test
    (2024) Hadley, Raegan; Chase, Tony; Department of Occupational Therapy, School of Health and Human Sciences; Chase, Tony
    The Skills on Wheels pediatric wheelchair training program lacks program protocols that support the reliability and validity of assessment administration while also showing a lack of skill retention at one-year follow-up which demonstrates the need for program evaluation and modification to support accurate data collection and scoring to draw conclusions from. This is important as this population faces many barriers including insufficient wheelchair skills training when receiving a wheelchair therefore making it difficult to navigate in the community and with others. This can have a great impact on them and the Skills on Wheels program aims to address these gaps to ensure equal and fair participation in daily activities. Therefore it is important that the skills taught during programming are retained and have a long term impact. The purpose of this capstone will be to evaluate and improve the overall functioning and protocols for current programming to increase the accuracy of assessment administration and scoring thus aiming to address potential discrepancies in data from which conclusions are drawn about skill retention issues. The capstone student developed and implemented an in-depth training regarding the Wheelchair Skills Test and evidence based skill training interventions. The capstone student also developed and implemented a protocol for the Wheelchair Skills Test administration to decrease biases and increase inter-rater reliability through a consistent group of trained individuals who are blinded to the subjects skill ability. Results found that volunteers felt both more prepared and accurate in their scoring than in years past and their confidence increased. Additionally, the scoring results from 2024 showed a more realistic skill range and increase among participants than years past supporting higher accuracy. Skills on Wheels would benefit from continuing to utilize protocols and training developed during this capstone experience to continue to enhance the reliability and validity of the program and support accuracy in data findings. This capstone began a process of program evaluation to continue to identify areas that impact the skills training the participants receive and the scoring of the assessment that determines outcomes of the program and participants.
  • Thumbnail Image
    Item
    Validation of the Pittsburgh Sleep Quality Index (PSQI) with Arabic cancer patients
    (Springer, 2020-07) Al Maqbali, Mohammed; Hughes, Ciara; Gracey, Jackie; Rankin, Jane; Dunwoody, Lynn; Hacker, Eileen; School of Nursing
    The Pittsburgh Sleep Quality Index (PSQI), a self-administrated questionnaire, is a frequently used instrument to assess sleep quality in clinical and non-clinical populations. The aim of the study was to evaluate the psychometric properties of the Arabic PSQI among patients diagnosed with cancer. A cross-sectional and descriptive correlational design was used with 369 patients with cancer completing the 19-item Arabic version of the PSQI. The scale was assessed in terms of acceptability, internal consistency and validity. The Cronbach’s alpha coefficient for the Arabic PSQI was 0.77, demonstrating acceptable reliability. The global PSQI score did not have floor or ceiling effects. The PSQI Arabic version demonstrated good discriminative validity. Construct validity was explored by confirmatory factor analysis to examine the dimensional structure of the PSQI. The confirmatory factor analysis showed an acceptable fit for refined one-factor model. The results support the original single factor structure of the PSQI. The Arabic version of the PSQI demonstrated adequate reliability and validity for assessing sleep quality in Arabic-speaking patients diagnosed with cancer.
  • Thumbnail Image
    Item
    The Validity and Timing of the ABR Core Exam
    (Elsevier, 2016-09) Kerridge, William D.; Gunderman, Richard B.; Department of Radiology and Imaging Sciences, IU School of Medicine
    The American Board of Radiology's (ABR) new Core Exam is not working, at least not as well as it needs to. Having helped to prepare candidates (RG), studied for and taken the exam (WK), and talked with hundreds of candidates who have taken the exam (RG and WK), we believe that that one aspect of the exam, its validity, can be significantly enhanced. Just as we expect candidates for board certification and practicing radiologists to measure up to the high standards, so we should subject the board exam to continuous scrutiny, seeking opportunities to rectify errors and enhance the exam's overall quality, with a view to better promoting excellence in radiology practice and the care of patients. Here we focus on two exam parameters: validity and timing.