Vol. 23 •Issue 2 • Page 32
The Right Tools for the Job
If existing assessments don't meet your needs, try creating your own
By Rhoda P. Erhardt
As occupational therapy practitioners, we make decisions every day about the type of assessments and evaluations we choose for our clients. Have you ever wondered if your choices are the most appropriate ones for that specific person, at that specific time, and in that specific context? Are you selecting certain instruments because you have always used them or because your facility provides or requires them? Have you taken the time to investigate others that may be less well-known, newly developed and published, or from outside the profession? Have you ever been motivated to create your own?
Practice and Research
We are constantly challenged to seek continuing competence in our practices by basing our clinical decisions on research evidence. (Case-Smith, 2004) One of the major goals of the American Occupational Therapy Association's Centennial Vision is for us to integrate scientific evidence with clinical expertise and clients' priorities. (Scheer, Arbesman and Lieberman, 2006) As busy practitioners, we can learn strategies for incorporating evidence-based practice (EBP) into our thinking and finding the right resources as we determine the best course of action for our clients. (Kellegrew, 2005)
We also know how much our profession needs efficacy research, and we all have the opportunity to collect evidence within our regular clinical practice, with little extra expense and not much extra time. Of course, the quality of the data we gather is dependent on the quality of the methods we use to collect it.
Collecting information about the client's goals and values is an art as well as a science. We begin with an Occupational Profile and Analysis of Occupational Performance, to identify which specific aspects and skills need to be evaluated. (American Occupational Therapy Association, 2002; Baum, 2006) Well-researched, standardized assessments are the ideal for this purpose.
However, good clinical practice can embrace other types of assessments, linked to a broad continuum (Table 1) that correlates with the levels of evidence developed and endorsed by the medical and allied health community. (American Medical Association, 2000; Butler and Darrah, 2001; Cook, 1991; State University of New York, 2004)
Research at Level I, the highest on this continuum, will require the most time and be the most costly in comparison with the lowest, Level V. Quantitative standardized assessments, required by many facilities and systems, take less time to administer but do not yield as much useful information leading to intervention as qualitative observational instruments, which are often used in case studies and reports.
We have many options, even at Level V. We can begin with a simple case report, supported by theories based on research and literature reviews. Descriptive case series could follow. If the opportunity arose, a case series with single-subject design, even without a control group, would bring the research to Level IV. Many occupational therapy practitioners, especially those connected to universities, are able to move upward to levels III, II or I.
Questions and Answers
To choose the most appropriate assessment(s), first ask yourself some important questions.
Q. How do you find what assessments are out there, and select the right ones for research or clinical practice or both?
For example, do you want to measure cognitive, physiological, neurobehavioral and/or psychological capacities? How do you measure compensatory strategies (learned or behavioral)? Which tests measure efficacy of interventions? (Baum, 2006)
A.Some suggestions include:
The Buros Institute of Mental Measurements provides test reviews online, which can be searched alphabetically or by category or by keyword: http://www.buros.unl.edu/buros/jsp/search.jsp.
The OTDBase is also searchable, using keywords: http://www.otdbase.org.
OTseeker is another database created by two universities in Australia: http://www.otseeker.com.
Pub Med, which provides abstracts and some full text articles, is a favorite site: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed.
Q. How do you choose between different assessments that seem to measure similar things?
For example, the Jebsen Hand Function Test (JHFT) (Jebsen, et al, 1969) and the Smith Hand Function Evaluation (SHFE) (Smith, 1973) were both designed for adults with neurological or musculoskeletal conditions involving hand disabilities. Both are standardized and present normative data for people in the approximate 20-60 year age range. Both have been used to document progress in therapy and determine the effectiveness of specific treatments. What are the differences between them that would help you select the right one for a specific client?
A. When you review tests, in addition to designated population, normative data and purpose, check also:
Performance skills measured, e.g.: ROM (SHFE), strength (SHFE), sensation (neither), dexterity (SHFE), coordination (SHFE), speed (SHFE and JHFT), quality (neither).
Functional tasks (ADL) measured (SHFE and JHFT).
Test kit or equipment needed and cost: JHFT test items can easily be compiled by the therapist, based on measurements and descriptions provided in the original publication. A test kit can also be purchased. No test kit information is available for the SHFE.
Time required: Short time required for both JHFT (15 minutes) and SHFE. (Jarus and Poremba, 1993)
Q.How do you know if it would be better to develop a new assessment instead of using a preexisting one?
A. If you have reviewed current instruments and they do not give you the information you need for evaluation, treatment planning and reevaluation in order to provide the best service possible for your client(s), consider constructing your own. Most new tools come from professionals who could not find what they needed to measure some aspect of client pathology, e.g., Anita Bundy's tests about play, Winnie Dunn's Sensory Profile, and Anne Fisher's Assessment of Motor and Process Skills (AMPS) for adults. (Diffendal, 2002)
Q.And finally, exactly how do you create something new, to go where none have gone before?
A. In this article I will use examples from my own experiences to illustrate the process I have used, and still use, to answer this question.
Developing the First Assessment
The Erhardt Developmental Prehension Assessment (EDPA)©
In the early 1970s, as an occupational therapist returning to the field after a 10-year absence, I felt a special responsibility for helping my pediatric clients improve their hand function, but I had difficulty finding good evaluation instruments.
Some limitations of the tools available to me in the 1970s:
Did not contain sufficient items
Did not assess quality of performance
Were not appropriate for severely involved or very young children
Had insufficient steps leading to emerging skills
Did not measure the rate of skill acquisition
Did not reflect the influence on one skill by the acquisition of others (Lewko, 1976)
While working on my master's degree, I decided to write a paper comparing different theories of development related to hand function, and illustrating those theories with a case study. When I submitted the revised paper to the American Journal of Occupational Therapy, I added a short chart entitled "Sequential Development of Prehension," which included age levels, grasp descriptions and illustrations, and ideas for stimulation of hand development.
The published article brought a surprising number of inquiries and comments, (Erhardt, 1974) motivating me to improve the chart, and incorporate its use into my practice more extensively. During the next eight years, the one-page checklist took on a life of its own, increased in size to 20 pages, and was eventually published within my first book, Developmental Hand Dysfunction: Theory, Assessment, and Treatment, and as a separate EDPA© test booklet. (Erhardt, 1982)
This process took time, but I managed to integrate most of it into my clinical practice and my workshop presentations. Table 2 shows the procedures I followed and examples of that integration.
Important points I learned from this experience (development of the EDPA©) were:
If you follow your passions, you will be motivated to be creative, and will commit your time and energy without hesitation.
Research is a collaborative effort. There are people that have expertise in areas other than your own who may be eager to work with you on interesting projects.
Much of the work you do can involve "killing two birds with one stone" (e.g., using a clinical report as a case study, using video research as a teaching tool, or collecting field data while being paid as a consultant to evaluate children).
Once you have produced and published basic research on your instrument, others will publish additional research.
Seeing Another Need
The Erhardt Developmental Vision Assessment (EDVA)©
Shortly after the publication of the EDPA©, I found myself wondering about the visual function of those same children. I spent most of the 1980s looking for answers and searching for useful evaluations.
Once again, I began with a review of the literature, which was filled with information about visual acuity and pathology (medical model) and visual perception (educational model). However, I found relatively little about the motor components of vision, and realized that I needed a different approach for my clients (developmental/functional model). I was comfortable following the same process:
Preliminary sequential chart and published case study (Erhardt, 1987)
Clinical use of a draft version of the Erhardt Developmental Vision Assessment (EDVA©)
Videotaped normative study (Erhardt, 1986)
Reliability study (Erhardt, Beatty and Hertsgaard, 1988)
Videotaped administration of the EDVA© (Erhardt, 1989)
Formal field tests by volunteer occupational therapists
Publication within the book Developmental Visual Dysfunction: Models for Assessment and Management, and as a separate test booklet, EDVA© (Erhardt, 1989, 1990)
Important points I learned from this experience (development of the EDVA©) were:
A process and structure, once proven, can be followed again for a new project-a huge timesaver.
Professionals who collaborate with you successfully are often willing to work with you again.
For example, the same medical illustrator created the drawings for the EDPA©, EDVA© and the cover of the book Handwriting: Anatomy of a Collaborative Assessment/Intervention Model. (Erhardt and Meade, 2005) The EDPA© and EDVA© published reliability studies were both co-authored by the same psychologist who helped design the study, and the same mathematician who did the statistical work. (Erhardt, Beatty and Hertsgaard, 1981; 1988)
A Third Assessment
The Erhardt Hand Preference Assessment (EHPA)©
My curiosity about hand preference and handedness was stimulated by the many questions that teachers and parents asked about what they termed "hand dominance," but that project had to wait until I finished writing the second edition of the EDPA© in 1994. The germination period was valuable for collecting information and jotting down ideas. Then I started all over again with the much of the same process: the literature review, videotaped normative studies of elementary age children, a comparison of the hand preferences of a toddler and her mother, (Erhardt, 1999, 2000) field testing, and final publication of the Erhardt Hand Preference Assessment©. (Erhardt, 2006)
Steps to a New Tool
The following steps summarize the process leading to development of a new assessment:
1.Ask yourself: What do I need to know about this client? What are the client's needs?
2.Start a simple checklist by writing test items, in a logical (to you) sequence.
3.Think about organization after examining the categories and subcategories that other tests use, such as developmental levels (Bayley), domains (WeeFim), developmental levels/domains (Peabody), type of skills (BOT-2), positions (TIME), assistance/modifications (PEDI), or type of tasks/occupational performance domains (EHPA).
4.Try the checklist clinically, to evaluate its effectiveness and weaknesses.
5.Expand, revise and get feedback from others (field testing).
6.Organize into final categories.
7.Publish.
8.Consider further research.
Future Plans
I'd like to end this article with a glimpse into the future. Another new assessment has been simmering on the back burner of my mind for a number of years. The idea came from a fascinating book titled Engineered Work Measurement, a classic reference for industrial engineers. (Karger and Bayha, 1987) In the foreword, mention is made of the "contemporary" work of Frank and Lillian Gilbreth (remember the book and first movie, Cheaper by the Dozen?) in the field of motion study.
A large part of the book is about the dynamic nature of manual skills in terms of work, one of the important areas of occupation described in the OT Practice Framework. (AOTA, 2002) It is filled with references to performance, time and motion studies, and detailed descriptions of movements such as reach, grasp, release and others such as move, turn, crank, apply pressure and disengage. Even more intriguing, the authors emphasize that the art of work is equally important as the science, a view echoed by many leaders in our field of occupational therapy. (Baum, 2006; Wood, 1995)
What if I was able to take my knowledge of hand skills and place that information in the context of adult function rather than hand development? The seeds have been planted for the creation of a new assessment, The Erhardt Adult Prehension Assessment (EAPA)! What are the steps for this new venture, and what I have done so far?
I will read, read, read everything I can find on the topic of hand assessment in adults, not children. My preliminary search of the OTDBase using the keyword "evaluations" brought me 130 abstracts. The keyword "assessments" brought 276. After reviewing those abstracts, I have begun requesting pertinent articles from interlibrary loan.
The process of collecting abstracts, articles, ideas and notes in a folder entitled EAPA will eventually generate a test structure that magically appears. The luxury of a no-deadline project seems to enhance my creative process.
Illustrations of adult prehension patterns will be drawn.
Colleagues with expertise in research methodology will be invited to collaborate.
The first draft of the new assessment will be field tested, revised, and revised again.
The new assessment will be published, date yet unknown.
Summary
What can you, the reader, take away from this article?
Motivation to improve your clinical practice by using research evidence.
Ideas for planning your own research.
A blueprint to follow for developing a new assessment.
References available at www.advanceweb.com/ot or upon request.
Rhoda P. Erhardt, MS, OTR/L, FAOTA, has a private practice in the Minneapolis/St.Paul area. She consults with health agencies, educational systems and national corporations, and presents workshops throughout the world. At the April 2007 AOTA conference in St. Louis, she will be presenting a workshop on the topic of hand preference. Readers may contact her by email at rperhardtdtdp@att.net or visit her Web site: www.ErhardtProducts.com.
Case reports, which are solely descriptions of practice, play an important role in the professional literature, serving several purposes:
Assisting in the evolution of theory
Helping develop practice guidelines
Creating hypotheses for future empirical research
Providing details necessary for others to replicate treatment (McEwen, 2001)
Program effectiveness depends on assessments that:
Are quantifiable
Allow tracking of development over time
Are sensitive to intervention methods
Reflect outcomes
|