Measurement Tools

Types of Measurement Tools and Evaluation Methods

• Skilled Observation – can be used to provide initial evidence for interventions.
  • Provides data for associations between factors.¹ 
  • Example: Assessment of Motor and Process Skills (AMPS)² 
• Interviews – involve direct communication with patients to capture their perspectives to be documented.³ 
  • Useful for understanding task performances that the client perceives as strengths or barriers.⁴ 
  • Should be semi-structured.
  • Be flexible to switch topics to cover all areas of the evaluation.
  • Use probe questions and techniques to gain additional information.
  • May be a part of an assessment that involves the interview format, e.g., COPM.
• Occupational histories and narratives⁵ 
• Self-report/survey – involves the use of questionnaires for clients to share relevant information, e.g., for the assessment of mindfulness in mental health.⁶ 
• Checklists and Rankings
• Skill Rankings⁵ 
• Instrument-based – empirical and less dependent on practitioner subjective rating.⁷ 
  • Examples include goniometers, dynamometers, sphygmomanometers
  • May require training
  • Bottom-up
  • May contribute to functional and performance measurement
  • Does not allow for analyzing function in the complete environmental context.
• Functional observation/Task Simulations – performance measures based on complex everyday tasks that involve both basic and higher-level cognitive processes.⁸ 
  • Example: Kettle Test
  • Useful for evaluating the level of independence discharge planning.
  • Moderately correlated with cognitive test scores.
  • Correlated with functional outcomes at discharge.
• Interdisciplinary Measures
  • Examples: FIM, Care Tool
• Standardized – includes explicit criteria for how to administer and score the assessment.⁹ 
• Non-standardized
  • Normative standardization – the instrument was administered to a large group of people and representative of those intended to be tested.
  • A table of norms with the distribution of scores is provided with average score ranges.
  • Involves converting a raw score to a standard score, e.g., percentiles or ranking.
  • Allows comparison of one client to different tests.
  • May be costly and require training for the administration and interpretation of results.

Scales vs Checklists¹⁰ 

Example: assessing a client’s handwriting sample

Scales

• Can rate or compare to a graded sample.¹¹ 
• Each sample corresponds to a specific number value that can be assigned a score.
• Can serve as a screening tool.
• Cannot identify specific components that may be contributing to functional or impaired performance.
• Still involves subjective interpretation.

Checklists

• More reliable than scales.
• Allow the sample to be judged on the basis of defined criteria,e .g., letter size and spacing.^12 13 
• Not quantitative.
• Relies on rater judgment.

Norm-referenced vs Criterion-referenced

• Both are ways to make inferences or interpretations of performance.
• May be difficult to determine if an assessment is one or the other.
• Each has different methods for interpreting scores.
• Both can be standardized to be interpreted by groups.¹⁴ 

Norm-referenced Tests (NRTs)¹⁵ 

• Makes comparisons between individuals.
• Test-taker is referenced to past test-takers.
• Reports if the test-taker performs better or worse than an average or norm group.
• Often compared to the same age or grade level.
• Ranks test-takers in comparison to others such as percentile on a bell curve.
• Well-established NRTs contain past scores with a lot of data points.
• Examples include assessing readiness for school and underlying skills, academic progress, the identification of disabilities, or the need for additional support.
• SAT and ACT test scores are NRTs.

Criterion-referenced Tests (CRTs)¹⁵ 

• Makes decisions about individuals, groups, and treatments.¹⁶ 
• Measure a test-taker’s performance when compared to a specific criterion (fixed set of standards).
• Based on the number of correct answers.
• “What test-takers can do and what they know, not how they compare to others.”¹⁴ 
• Scores are expressed as a percentage of the total number of questions.
• Can evaluate if the test-taker has achieved specific goals or acquired certain skills or knowledge.
• Used to evaluate whether or not an objective is met.
• Examples: multiple choice portion on driver’s license, IELTS, TOEFL.

Outcome Measure Selection Criteria

Select outcome measures that are:

• Suitable for your population (including the age, diagnostic group, and time taken to administer) and setting (inpatient, outpatient, community-based, acute care, rehabilitation, or long-term care)
• Meets your needs (in terms of the purpose of the tool, time taken to administer, quality of information obtained, ease of communication of results, and sensitivity to change in client status as reflected in the range of scores available on the assessment)
• Standardized – “has a uniform procedure to administer and score the assessment and an approach to determine how ‘well’, or how ‘badly’ the client has performed. This may be either through norm referencing or criterion referencing”.¹⁷ 

Bookmark

1. Harrison, J. K., Reid, J., Quinn, T. J., & Shenkin, S. D. (2017). Using quality assessment tools to critically appraise ageing research: a guide for clinicians. Age and ageing, 46(3), 359-365.[↩]
2. Nielsen, K. T., & Wæhrens, E. E. (2015). Occupational therapy evaluation: use of self-report and/or observation?. Scandinavian journal of occupational therapy, 22(1), 13-23.[↩]
3. Brod, M., Tesler, L. E., & Christensen, T. L. (2009). Qualitative research and content validity: developing best practices based on science and experience. Quality of life research, 18(9), 1263-1278.[↩]
4. Nielsen, K. T., & Wæhrens, E. E. (2015). Occupational therapy evaluation: use of self-report and/or observation? Scandinavian journal of occupational therapy, 22(1), 13-23.[↩]
5. Moyers, P. A. (1999). The guide to occupational therapy practice. American Journal of Occupational Therapy, 53(3), 247-322.[↩][↩]
6. Baer, R. A., Smith, G. T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13(1), 27-45.[↩]
7. Smith, R. O. (1992). The science of occupational therapy assessment.[↩]
8. Hartman-Maeir, A., Harel, H., & Katz, N. (2009). Kettle Test—A brief measure of cognitive functional performance: Reliability and validity in stroke rehabilitation. American Journal of Occupational Therapy, 63(5), 592-599.[↩]
9. de Clive-Lowe, S. (1996). Outcome measurement, cost-effectiveness and clinical audit: the importance of standardized assessment to occupational therapists in meeting these new demands. British Journal of Occupational Therapy, 59(8), 357-362.[↩]
10. Feder, K., Majnemer, A., & Synnes, A. (2000). Handwriting: Current trends in occupational therapy practice. Canadian Journal of Occupational Therapy, 67(3), 197-204.[↩]
11. Graham, S. (1986). A review of handwriting scales and factors that contribute to variability in handwriting scores. Journal of School Psychology, 24, 63-71.[↩]
12. Alston, J. (1983). A legibility index: Can handwriting be measured?
    Educational Review, 35, 237-240.[↩]
13. Hamstra-Bletz, L., & Blote, A. (1990). Development of handwriting in primary school: A longitudinal study. Perceptual Motor Skills, 70, 759-770.[↩]
14. Bond, L. A. (1995). Norm-Referenced Testing and Criterion-Referenced Testing: The Differences in Purpose, Content, and Interpretation of Results.[↩][↩]
15. Burkett, T. (2018). Norm‐Referenced Testing and Criterion‐Referenced Testing. The TESOL Encyclopedia of English Language Teaching, 1-5.[↩][↩]
16. Popham, W., & Husek, T. (1969). Implications of criterion-referenced measurement. Journal
    of Educational Measurement, 6(1), 1–9[↩]
17. Unsworth, C. (2000). Measuring the outcome of occupational therapy: Tools and resources. Australian Occupational Therapy Journal, 47(4), 147-158.[↩]