Using MediaPipe to track upper-limb reaching movements after stroke: a proof-of-principle study | Journal of NeuroEngineering and Rehabilitation

  • Langhorne P, Bernhardt J, Kwakkel G. Strokerehabilitation Lancet. 2011;377:1693–702.

    Google Scholar 

  • Levin MF, Liebermann DG, Parmet Y, Berman S. Compensatory versus noncompensatory shoulder movements used for reaching in stroke. Neurorehab Neural Rep. 2016;30:635–46. https://doi.org/10.1177/1545968315613863.

    Google Scholar 

  • Kwakkel G, Wegen EV, Burridge JH, Winstein CJ, van Dokkum L, Murphy MA, et al. Standardized measurement of quality of upper limb movement after stroke: Consensus-based core recommendations from the second stroke recovery and rehabilitation roundtable. Int J Stroke. 2019;14:783–91.

    Google Scholar 

  • Zhou H, Hu H. Human motion tracking for rehabilitation—A survey. Biomed Signal Process Control. 2008;3(1):1–18.

    Google Scholar 

  • Lam WW, Tang YM, Fong KN. A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation. J Neuroeng Rehabil. 2023;20(1):57.

    Google Scholar 

  • MediaPipe. Available from https://developers.google.com/mediapipe

  • Lugaresi C, Tang J, Nash H, McClanahan C, Uboweja E, Hays M et al. Mediapipe: A framework for Building perception pipelines. ArXiv Preprint arXiv:190608172. 2019.

  • Singh AK, Kumbhare VA, Arthi K. Real-time human pose detection and recognition using mediapipe. In: International Conference on Soft Computing and Signal Processing. Springer; 2021. pp. 145–154.

  • Güney G, Jansen TS, Dill S, Schulz JB, Dafotakis M, Hoog Antink C, et al. Video- based hand movement analysis of Parkinson patients before and after medication using high-frame-rate videos and MediaPipe. Sensors. 2022;22(20):7992.

    Google Scholar 

  • Wagh V, Scott MW, Kraeutner SN. Quantifying similarities between MediaPipe and a known standard to address issues in tracking 2D upper limb trajectories: proof of concept study. JMIR Form Res. 2024;8:e56682.

    Google Scholar 

  • Latreche A, Kelaiaia R, Chemori A, Kerboua A. Reliability and validity analysis of MediaPipe-based measurement system for some human rehabilitation motions. Measurement. 2023;214:112826.

    Google Scholar 

  • Palani P, Panigrahi S, Jammi SA, Thondiyath A. Real-time Joint Angle Estimation using Mediapipe Framework and Inertial Sensors. In: 2022 IEEE 22nd International Conference on Bioinformatics and Bioengineering (BIBE). IEEE; 2022. pp. 128–133.

  • Amprimo G, Ferraris C, Masi G, Pettiti G, Priano L. Gmh-d: Combining google mediapipe and rgb-depth cameras for hand motor skills remote assessment. In: 2022 IEEE International Conference on Digital Health (ICDH). IEEE; 2022. pp. 132–141.

  • Hii CST, Gan KB, Zainal N, Ibrahim NM, Azmin S, Desa SHM, et al. Automated gait analysis based on a Marker-Free pose Estimation model. Sensors. 2023;23:6489.

    Google Scholar 

  • Gupta A, Shrestha PL, Thapa B, Silwal R, Shrestha R. Knee Flexion/Extension Angle Measurement for Gait Analysis Using Machine Learning Solution MediaPipe Pose and Its Comparison with Kinovea®. In: IOP Conference Series: Materials Science and Engineering. vol. 1279. IOP Publishing; 2023. p. 012004.

  • Cóias AR, Lee MH, Bernardino A, Smailagic A. Skeleton tracking solutions for a low-cost stroke rehabilitation support system. In2023 International Conference on Rehabilitation Robotics (ICORR) 2023 Sep 24 (pp. 1–6). IEEE.

  • Lin HP, Zhao L, Woolley D, Zhang X, Cheng HJ, Liang W, Kuah C, Plunkett T, Chua K, Zhang L, Wenderoth N. Exploring the feasibility of computer vision for detecting post-stroke compensatory movements. In2023 International Conference on Rehabilitation Robotics (ICORR) 2023 Sep 24 (pp. 1–6). IEEE.

  • Cornella-Barba G, Farrens AJ, Johnson CA, Garcia-Fernandez L, Chan V, Reinkensmeyer DJ. Using a webcam to assess upper extremity proprioception: experimental validation and application to persons post stroke. Sensors. 2024;24(23):7434. https://doi.org/10.3390/s24237434.

    Google Scholar 

  • Kraeutner SN, Rubino C, Rinat S, Lakhani B, Borich MR, Wadden KP, Boyd LA. 2021. Resting State Connectivity Is Modulated by Motor Learning in Individuals After Stroke. Neurorehab Neural Re 154596832110067.

  • Lakhani B, Borich MR, Jackson JN, Wadden KP, Peters S, Villamayor A, et al. Motor Skill Acquisition Promotes Hum Brain Myelin Plast Neural Plast. 2016;2016:1–7.

    Google Scholar 

  • Rubino C, Lakhani B, Larssen BC, Kraeutner SN, Andrushko JW, Borich MR, et al. Gamified practice improves Paretic arm motor behavior in individuals with stroke. Neurorehabilit Neural Repair. 2024;38:832–44.

    Google Scholar 

  • Subramanian SK, Yamanaka J, Chilingaryan G. Levin MF validity of movement pattern kinematics as measures of arm motor impairment poststroke. Stroke. 2010;41:2303–8.

    Google Scholar 

  • Bernhardt J, Hayward KS, Kwakkel G, Ward NS, Wolf SL, Borschmann K, Krakauer JW, Boyd LA, Carmichael ST, Corbett D, Cramer SC. Agreed definitions and a shared vision for new standards in stroke recovery research: The Stroke Recovery and Rehabilitation Roundtable taskforce. Int J Stroke. 2017;12(5):444–450. https://doi.org/10.1177/1747493017711816. PMID: 28697708.

  • Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal cognitive Assessment, moca: A brief screening tool for mild cognitive impairment: MOCA: A BRIEF SCREENING TOOL FOR MCI. J Am Geriatr Soc. 2005;53:695–9.

    Google Scholar 

  • Fugl-Meyer AR, Jääskö L, Leyman I, Olsson S, Steglind S. The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand J Rehabil Med. 1975;7:13–31.

    Google Scholar 

  • Bazarevsky V, Grishchenko I, Raveendran K, Zhu T, Zhang F, Grundmann M. BlazePose: On-device Real-time Body Pose tracking. arXiv. 2020.

  • Liebermann DG, Berman S, (Tamar) Weiss PL, Levin MF. Kinematics of reaching movements in a 2-D virtual environment in adults with and without stroke. IEEE Trans Neural Syst Rehabilitation Eng. 2012;20:778–87.

    Google Scholar 

  • Thrane G, Sunnerhagen KS, Murphy MA. Upper limb kinematics during the first year after stroke: the stroke arm longitudinal study at the university of Gothenburg (SALGOT). J Neuroeng Rehabilitation. 2020;17:76.

    Google Scholar 

  • Michaelsen SM, Dannenbaum R, Levin MF. Task-Specific training with trunk restraint on arm recovery in stroke. Stroke. 2006;37:186–92.

    Google Scholar 

  • Sherwood DE, Lohse KR, Healy AF. Judging joint angles and movement outcome: shifting the focus of attention in dart-throwing. J Experimental Psychol Hum Percept Perform. 2014;40:1903–14.

    Google Scholar 

  • Kraeutner SN, Stratas A, McArthur JL, Helmick CA, Westwood DA, Boe SG. Neural and behavioral outcomes differ following equivalent bouts of motor imagery or physical practice. J Cogn Neurosci. 2020;32:1590–606.

    Google Scholar 

  • Wang X, Fu Y, Ye B, Babineau J, Ding Y, Mihailidis A. Technology-Based compensation assessment and detection of upper extremity activities of stroke survivors: systematic review. J Méd Internet Res. 2022;24:e34307.

    Google Scholar 

  • Foreman MH, Engsberg JR. The validity and reliability of the Microsoft kinect for measuring trunk compensation during reaching. Sensors. 2020;20:7073.

    Google Scholar 

  • Uhlrich SD, Falisse A, Kidziński Ł, Muccini J, Ko M, Chaudhari AS, Hicks JL, Delp SL. OpenCap: human movement dynamics from smartphone videos. PLoS Comput Biol. 2023;19(10):e1011462.

    Google Scholar 

  • Murphy MA, Häger CK. Kinematic analysis of the upper extremity after stroke – how Far have we reached and what have we grasped? Phys Ther Rev. 2015;20:137–55.

    Google Scholar 

  • Schwarz A, Kanzler CM, Lambercy O, Luft AR, Veerbeek JM. Systematic review on kinematic assessments of upper limb movements after stroke. Stroke. 2015;50:718–27.

    Google Scholar 

  • Marchesi G, Ballardini G, Barone L, Giannoni P, Lentino C, Luca AD, et al. Modified functional reach test: Upper-Body kinematics and muscular activity in chronic stroke survivors. Sensors. 2021;22:230.

    Google Scholar 

  • Scano A, Guanziroli E, Brambilla C, Specchia A, Tosatti LM, Molteni F. Kinematic instrumental assessment quantifies compensatory strategies in post-stroke patients. Méd Biol Eng Comput. 2025;1–12.

  • Zamin SA, Tang K, Stevens EA, Howard M, Parker DM, Seals A, et al. aBnormal motion capture in aCute stroke (BIONICS): A Low-Cost Tele-Evaluation tool for automated assessment of upper extremity function in stroke patients. Neurorehabilit Neural Repair. 2023;37:591–602.

    Google Scholar 

  • Continue Reading