Investigating Intra-Rater and Inter-Rater Reliability of Three Upper-Limb Risk Assessment Methods

Majid Motamedzade, Mahdi Mohammadian, Javad Faradmal

Abstract


There are several methods for assessing the risk of upper-limb disorders in occupational health studies. The goal of present study is to determine the intra-rater and inter-rater reliability of three risk assessment methods of SI, OCRA Checklist and ART in repetitive tasks and exposure to upper-limb risk factors. First, 9 raters used SI, OCRA Checklist and ART to assess the physical exposure level of the upper limb in workers 30 tasks with different levels of profession for two assessment periods. In the results of inter-rater reliability using ICC, the risk classifications were ρ = 0.54, ρ = 0.74 and ρ = 0.77 for SI, OCRA Checklist and ART, respectively. And the intra-rater reliability for the final score of the OCRA Checklist, SI and ART methods was κ = 0.59, κ = 0.66 and κ = 0.67, respectively. The inter-rater reliability for risk classification and final score of ART, OCRA Checklist and SI methods using ICC) obtained as moderate to good. However, the overall results showed that the reliability of ART method was higher than two other methods. The findings demonstrated that all three methods are appropriate for assessment in workplaces. These methods should be used based on assessment objectives, rater experience and required time for assessing the tasks.

Keywords


Musculoskeletal Disorders, OCRA Checklist, Strain Index, ART

Full Text:

XML PDF

References


Baldwin ML. Reducing the costs of work-related musculoskeletal disorders: targeting strategies to chronic disability cases. Journal of Electromyography and Kinesiology. 2004;14(1):33-41.

Bongers P, Ijmker S, Van den Heuvel S, Blatter B. Epidemiology of work related neck and upper limb problems: psychosocial and personal risk factors (part I) and effective interventions from a bio behavioural perspective (part II). Journal of occupational rehabilitation. 2006;16(2):272-95.

Bovenzi M. Health risks from occupational exposures to mechanical vibration. La Medicina del lavoro. 2006;97(3):535-41.

Dempsey PG, McGorry RW, Maynard WS. A survey of tools and methods used by certified professional ergonomists. Applied ergonomics. 2005;36(3):489-03.

Neumann WP, Village J. Ergonomics action research II: a framework for integrating HF into work system design. Ergonomics. 2012;55(2):1140-56.

Takala E-P, Pehkonen I, Forsman M,. Systematic evaluation of observational methods assessing biomechanical exposures at work. Scandinavian journal of work, environment & health. 2010;36(1):3-24.

David G. Ergonomic methods for assessing exposure to risk factors for work-related musculoskeletal disorders. Occupational medicine. 2005;55(4):190-99.

Van der Beek AJ, Frings-Dresen M. Assessment of mechanical exposure in ergonomic epidemiology. Occupational and environmental medicine. 1998;55(1):291-99.

Rahman MNA, Razak NSA. Review on Pen and Paper Based Observational Methods for Assessing Work-related Upper Limb Disorders. Indian Journal of Science and Technology. 2016;9(2):1-11.

Steven Moore J, Garg A. The strain index: a proposed method to analyze jobs for risk of distal upper extremity disorders. American Industrial Hygiene Association. 1995;56(4):443-58.

Colombini D, Occhipinti E, Cerbai M, Battevi N, Placci M. Updating of application procedures and criteria for OCRA Checklist. Med Del Lav. 2011;102(2):1-39.

Ferreira J, Gray M, Hunter L. Development of an assessment tool for repetitive tasks of the upper limbs (ART). Derbyshire: Health and Safety Executive. 2009.

Borg GA. Psychophysical bases of perceived exertion. Med sci sports exerc. 1982;14(1):377-81.

Spielholz P, Bao S, Howard N. Reliability and validity assessment of the hand activity level threshold limit value and strain index using expert ratings of mono-task jobs. Journal of occupational and environmental hygiene. 2008;5(1):250-57.

Garg A, Kapellusch JM. Job analysis techniques for distal upper extremity disorders. Reviews of Human Factors and Ergonomics. 2011;7(3):149-96.

Jones T, Kumar S. Comparison of ergonomic risk assessment output in four sawmill jobs. International Journal of Occupational Safety and Ergonomics. 2010;16(4):105-11.

Cohen J. A coefficient of agreement for nominal scales. Educational and psychological measurement. 1960;20(2):37-46.

Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin. 1979;86(2):420-28.

Landis JR, Koch GG. The measurement of observer agreement for categorical data. biometrics. 1977;33(1):159-74.

Stevens EM, Vos GA, Stephens J-P, Moore JS. Inter-rater reliability of the strain index. Journal of occupational and environmental hygiene. 2004;11(1): 745-51.

Paulsen R, Gallu T, Gilkey D. The inter-rater reliability of Strain Index and OCRA Checklist task assessments in cheese processing. Appl Ergon. 2015;51(1):199-04.

Rhéna I-M, Balliuc N, Forsmana M. OCRA inter-and intra-ergonomist reliability in ten video recorded work tasks. Proceedings 19th Triennial Congress of the IEA2015. p. 14.

Stephens J-P, Vos GA, Stevens EM, Moore JS. Test–retest repeatability of the strain index. Applied ergonomics. 2006;37(3):275-81.

Roodbandi A, Choobineh A, Feyzi V. The Investigation of Intra-rater and Inter-rater Agreement in Assessment of Repetitive Task (ART) as an Ergonomic Method. Occup Med Health Aff. 2015;3(5):1-5.




Iranian Journal of Health, Safety and Environment e-ISSN: :2345-5535 Iran university of Medical sciences, Tehran, Iran