Deep learning in gait analysis the effect of marker presence in neural network training to kinematic outcomes

Ihmisen tuottaman liikkeen määrittämiseen käytetään yleensä optoelektronisia liikkeenkaappausjärjestelmiä, jotka perustuvat kohteen iholle kiinnitettävien valoa heijastavien markkerien seurantaan. Nämä laboratorio-olosuhteissa tarkat ja luotettavat järjestelmät ovat kuitenkin kalliita, mittaustapaht...

Full description

Bibliographic Details
Main Author: Uitto, Roope
Other Authors: Liikuntatieteellinen tiedekunta, Faculty of Sport and Health Sciences, Liikunta- ja terveystieteet, Sport and Health Sciences, Jyväskylän yliopisto, University of Jyväskylä
Format: Master's thesis
Language:eng
Published: 2021
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/77355
_version_ 1826225782262857728
author Uitto, Roope
author2 Liikuntatieteellinen tiedekunta Faculty of Sport and Health Sciences Liikunta- ja terveystieteet Sport and Health Sciences Jyväskylän yliopisto University of Jyväskylä
author_facet Uitto, Roope Liikuntatieteellinen tiedekunta Faculty of Sport and Health Sciences Liikunta- ja terveystieteet Sport and Health Sciences Jyväskylän yliopisto University of Jyväskylä Uitto, Roope Liikuntatieteellinen tiedekunta Faculty of Sport and Health Sciences Liikunta- ja terveystieteet Sport and Health Sciences Jyväskylän yliopisto University of Jyväskylä
author_sort Uitto, Roope
datasource_str_mv jyx
description Ihmisen tuottaman liikkeen määrittämiseen käytetään yleensä optoelektronisia liikkeenkaappausjärjestelmiä, jotka perustuvat kohteen iholle kiinnitettävien valoa heijastavien markkerien seurantaan. Nämä laboratorio-olosuhteissa tarkat ja luotettavat järjestelmät ovat kuitenkin kalliita, mittaustapahtuman valmistelu vie reilusti aikaa, ja markkerit voivat estää kohteen luonnollisen liikkumisen. Konenäön kehittymisen myötä syväoppimiseen perustuvien menetelmien käyttö ihmisen asennon määrittelyssä on yleistynyt ja niiden on osoitettu olevan ihmisen kanssa yhtä tarkkoja merkitsemään avainkohtia kuviin. Aikaisemmissa tutkimuksissa suoritetuissa kinemaattisissa vertailuissa avainpisteet eivät ole olleet anatomisesti tarkkoja, biomekaaniset mallit ovat olleet erilaiset menetelmien välillä, eikä markkerien vaikutusta syväoppimismallien suorituskykyyn ole huomioitu. Tämän tutkimuksen tarkoituksena oli tutkia miten markkerien läsnäolo harjoitusnäytteissä vaikuttaa mallien suorituskykyyn, kun niitä sovelletaan 3D-liikeanalyysiin ja verrataan optoelektroniseen järjestelmään. 18 koehenkilöä käveli ja juoksi juoksumatolla eri vauhdeilla samalla, kun heidän liikkumistaan tallennettiin kahdella kahdeksankameraisella järjestelmällä: optoelektroninen Vicon-järjestelmä (300 Hz) ja GoPro-järjestelmä (60 Hz). Kaksi syväoppimismallia kehitettiin GoPro-järjestelmällä tallennettujen harjoitusnäytteiden perusteella siten, että toiseen käytettiin näytteitä, joissa markkerit olivat läsnä, ja toiseen näytteitä ilman markkereita. Kävely- ja juoksunäytteet analysoitiin molemmilla malleilla ja kinemaattisia tuloksia verrattiin tuloksiin Vicon-järjestelmästä. Lisäksi, Viconilla mitattujen kinemaattisten muuttujien toistettavuus samalla koehenkilöllä laskettiin, jotta yksilön askelten välinen vaihtelu voitiin määrittää. 3D-liikeanalyysi epäonnistui mallien heikon suorituskyvyn takia, mutta sagittaalitason 2D-liikeanalyysi voitiin suorittaa oikean jalan nilkalle ja polvelle. Markkerien läsnäololla ei ollut selkeää vaikutusta mallien suorituskykyyn. Mallien suorituskyky oli lähes yhtäläinen markkerien ollessa läsnä näytteissä, eikä kumpikaan kyennyt kelvollisesti analysoimaan näytteitä, joissa ei ollut markkereita. SPM-analyysi paljasti, että menetelmien välillä havaittiin yksi tilastollisesti merkitsevä joukko polvessa ja kahdeksan nilkassa. Toistettavuus oli suurimmassa osaa näytteitä riittävä kliinisiin mittauksiin (ICC > 0.9), mutta pisteittäinen analyysi osoitti toistettavuuden vaihtelevan askelsyklin aikana. On mahdollista, että syväoppimismalleja voidaan tulevaisuudessa käyttää 3D-liikeanalyysissä, jos olosuhteet ovat asianmukaiset. Tässä tutkimuksessa markkerien läsnäololla harjoitusnäytteissä ei ollut vaikutusta syväoppimismallien suorituskykyyn sovellettaessa 2D-liikeanalyysiin. Accurate quantification of human movement is usually performed with optoelectronic motion capture systems inside a laboratory by tracking reflective markers on the subject. This is accurate and reliable method, but the markers and laboratory environment can restrict motion, the required equipment is expensive, and preparation of a subject takes time. With recent advances in computer vision, the use of deep learning -based methods for human pose estimation has increased and they’ve been shown to reach human-level labelling accuracies in 2D and 3D. Some studies have compared deep learning -based methods to optoelectronic systems for acquiring joint kinematics, but those studies haven’t implemented anatomically relevant keypoints or biomechanically valid kinematic models in their analyses. Neither is the effect of marker presence been studied in these settings. Thus, the purpose of this study was to investigate how the presence of markers in training data affects the predicting performance of a deep learning -based methods in 3D with proper kinematic model. 18 healthy subjects were recruited to walk and run on a treadmill at moderate speeds, while their gait was recorded simultaneously by two systems: an 8-camera Vicon motion capture system at 300 Hz, and an 8-camera GoPro system at 60 Hz. Two deep learning models were trained with data from GoPros, one with data where subjects wore markers, and the other with data where markers were removed. Trials with markers and without markers were analysed by both models and compared to data from Vicon. Additionally, the between-trial reliability of Vicon data was calculated to give insight into the amount of variability due to intraindividual differences in gait. The 3D analysis failed due to poor performance of the models in some camera views, but 2D analysis on the right ankle and knee could be performed. The marker presence in the training data didn’t clearly affect the performance of models, while performing similarly with data that had markers and failing to produce any results from the trials without markers. Also, when compared to data from Vicon, one supra-threshold cluster was found in the knee and eight in the ankle during SPM analyses. The between-trial reliability was most of the time reasonable for clinical measurements (ICC > 0.9), but pointwise analysis showed clear differences in reliability at different points of the gait cycle. There is promise for deep learning -based methods to be used in clinical gait analysis, if the conditions are appropriate. Additional training of the pretrained neural networks with markers present or absent from data didn’t seem to make a difference to the performance of the model.
first_indexed 2021-08-12T20:02:35Z
format Pro gradu
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "Cronin, Neil", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.advisor", "value": "Avela, Janne", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "Uitto, Roope", "language": "", "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2021-08-12T06:08:29Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2021-08-12T06:08:29Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2021", "language": "", "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/77355", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Ihmisen tuottaman liikkeen m\u00e4\u00e4ritt\u00e4miseen k\u00e4ytet\u00e4\u00e4n yleens\u00e4 optoelektronisia liikkeenkaappausj\u00e4rjestelmi\u00e4, jotka perustuvat kohteen iholle kiinnitett\u00e4vien valoa heijastavien markkerien seurantaan. N\u00e4m\u00e4 laboratorio-olosuhteissa tarkat ja luotettavat j\u00e4rjestelm\u00e4t ovat kuitenkin kalliita, mittaustapahtuman valmistelu vie reilusti aikaa, ja markkerit voivat est\u00e4\u00e4 kohteen luonnollisen liikkumisen. Konen\u00e4\u00f6n kehittymisen my\u00f6t\u00e4 syv\u00e4oppimiseen perustuvien menetelmien k\u00e4ytt\u00f6 ihmisen asennon m\u00e4\u00e4rittelyss\u00e4 on yleistynyt ja niiden on osoitettu olevan ihmisen kanssa yht\u00e4 tarkkoja merkitsem\u00e4\u00e4n avainkohtia kuviin. Aikaisemmissa tutkimuksissa suoritetuissa kinemaattisissa vertailuissa avainpisteet eiv\u00e4t ole olleet anatomisesti tarkkoja, biomekaaniset mallit ovat olleet erilaiset menetelmien v\u00e4lill\u00e4, eik\u00e4 markkerien vaikutusta syv\u00e4oppimismallien suorituskykyyn ole huomioitu. T\u00e4m\u00e4n tutkimuksen tarkoituksena oli tutkia miten markkerien l\u00e4sn\u00e4olo harjoitusn\u00e4ytteiss\u00e4 vaikuttaa mallien suorituskykyyn, kun niit\u00e4 sovelletaan 3D-liikeanalyysiin ja verrataan optoelektroniseen j\u00e4rjestelm\u00e4\u00e4n.\n18 koehenkil\u00f6\u00e4 k\u00e4veli ja juoksi juoksumatolla eri vauhdeilla samalla, kun heid\u00e4n liikkumistaan tallennettiin kahdella kahdeksankameraisella j\u00e4rjestelm\u00e4ll\u00e4: optoelektroninen Vicon-j\u00e4rjestelm\u00e4 (300 Hz) ja GoPro-j\u00e4rjestelm\u00e4 (60 Hz). Kaksi syv\u00e4oppimismallia kehitettiin GoPro-j\u00e4rjestelm\u00e4ll\u00e4 tallennettujen harjoitusn\u00e4ytteiden perusteella siten, ett\u00e4 toiseen k\u00e4ytettiin n\u00e4ytteit\u00e4, joissa markkerit olivat l\u00e4sn\u00e4, ja toiseen n\u00e4ytteit\u00e4 ilman markkereita. K\u00e4vely- ja juoksun\u00e4ytteet analysoitiin molemmilla malleilla ja kinemaattisia tuloksia verrattiin tuloksiin Vicon-j\u00e4rjestelm\u00e4st\u00e4. Lis\u00e4ksi, Viconilla mitattujen kinemaattisten muuttujien toistettavuus samalla koehenkil\u00f6ll\u00e4 laskettiin, jotta yksil\u00f6n askelten v\u00e4linen vaihtelu voitiin m\u00e4\u00e4ritt\u00e4\u00e4.\n3D-liikeanalyysi ep\u00e4onnistui mallien heikon suorituskyvyn takia, mutta sagittaalitason 2D-liikeanalyysi voitiin suorittaa oikean jalan nilkalle ja polvelle. Markkerien l\u00e4sn\u00e4ololla ei ollut selke\u00e4\u00e4 vaikutusta mallien suorituskykyyn. Mallien suorituskyky oli l\u00e4hes yht\u00e4l\u00e4inen markkerien ollessa l\u00e4sn\u00e4 n\u00e4ytteiss\u00e4, eik\u00e4 kumpikaan kyennyt kelvollisesti analysoimaan n\u00e4ytteit\u00e4, joissa ei ollut markkereita. SPM-analyysi paljasti, ett\u00e4 menetelmien v\u00e4lill\u00e4 havaittiin yksi tilastollisesti merkitsev\u00e4 joukko polvessa ja kahdeksan nilkassa. Toistettavuus oli suurimmassa osaa n\u00e4ytteit\u00e4 riitt\u00e4v\u00e4 kliinisiin mittauksiin (ICC > 0.9), mutta pisteitt\u00e4inen analyysi osoitti toistettavuuden vaihtelevan askelsyklin aikana. On mahdollista, ett\u00e4 syv\u00e4oppimismalleja voidaan tulevaisuudessa k\u00e4ytt\u00e4\u00e4 3D-liikeanalyysiss\u00e4, jos olosuhteet ovat asianmukaiset. T\u00e4ss\u00e4 tutkimuksessa markkerien l\u00e4sn\u00e4ololla harjoitusn\u00e4ytteiss\u00e4 ei ollut vaikutusta syv\u00e4oppimismallien suorituskykyyn sovellettaessa 2D-liikeanalyysiin.", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Accurate quantification of human movement is usually performed with optoelectronic motion capture systems inside a laboratory by tracking reflective markers on the subject. This is accurate and reliable method, but the markers and laboratory environment can restrict motion, the required equipment is expensive, and preparation of a subject takes time. With recent advances in computer vision, the use of deep learning -based methods for human pose estimation has increased and they\u2019ve been shown to reach human-level labelling accuracies in 2D and 3D. Some studies have compared deep learning -based methods to optoelectronic systems for acquiring joint kinematics, but those studies haven\u2019t implemented anatomically relevant keypoints or biomechanically valid kinematic models in their analyses. Neither is the effect of marker presence been studied in these settings. Thus, the purpose of this study was to investigate how the presence of markers in training data affects the predicting performance of a deep learning -based methods in 3D with proper kinematic model.\n18 healthy subjects were recruited to walk and run on a treadmill at moderate speeds, while their gait was recorded simultaneously by two systems: an 8-camera Vicon motion capture system at 300 Hz, and an 8-camera GoPro system at 60 Hz. Two deep learning models were trained with data from GoPros, one with data where subjects wore markers, and the other with data where markers were removed. Trials with markers and without markers were analysed by both models and compared to data from Vicon. Additionally, the between-trial reliability of Vicon data was calculated to give insight into the amount of variability due to intraindividual differences in gait.\nThe 3D analysis failed due to poor performance of the models in some camera views, but 2D analysis on the right ankle and knee could be performed. The marker presence in the training data didn\u2019t clearly affect the performance of models, while performing similarly with data that had markers and failing to produce any results from the trials without markers. Also, when compared to data from Vicon, one supra-threshold cluster was found in the knee and eight in the ankle during SPM analyses. The between-trial reliability was most of the time reasonable for clinical measurements (ICC > 0.9), but pointwise analysis showed clear differences in reliability at different points of the gait cycle. There is promise for deep learning -based methods to be used in clinical gait analysis, if the conditions are appropriate. Additional training of the pretrained neural networks with markers present or absent from data didn\u2019t seem to make a difference to the performance of the model.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by Paivi Vuorio (paelvuor@jyu.fi) on 2021-08-12T06:08:29Z\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2021-08-12T06:08:29Z (GMT). No. of bitstreams: 0\n Previous issue date: 2021", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "90", "language": "", "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.format.mimetype", "value": "application/pdf", "language": null, "element": "format", "qualifier": "mimetype", "schema": "dc"}, {"key": "dc.language.iso", "value": "eng", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "In Copyright", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.title", "value": "Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes", "language": "", "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "master thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-202108124521", "language": "", "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Pro gradu -tutkielma", "language": "fi", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Master\u2019s thesis", "language": "en", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Liikuntatieteellinen tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Sport and Health Sciences", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Liikunta- ja terveystieteet", "language": "fi", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Sport and Health Sciences", "language": "en", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Biomekaniikka", "language": "fi", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Biomechanics", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "yvv.contractresearch.funding", "value": "0", "language": "", "element": "contractresearch", "qualifier": "funding", "schema": "yvv"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_bdcc", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "masterThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.subject.oppiainekoodi", "value": "5012", "language": "", "element": "subject", "qualifier": "oppiainekoodi", "schema": "dc"}, {"key": "dc.subject.yso", "value": "syv\u00e4oppiminen", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "luotettavuus", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "liikeoppi", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "liikeanalyysi", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "deep learning", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "reliability (general)", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "kinematics", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "motion analysis", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.format.content", "value": "fulltext", "language": null, "element": "format", "qualifier": "content", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://rightsstatements.org/page/InC/1.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}, {"key": "dc.type.okm", "value": "G2", "language": null, "element": "type", "qualifier": "okm", "schema": "dc"}]
id jyx.123456789_77355
language eng
last_indexed 2025-02-18T10:54:14Z
main_date 2021-01-01T00:00:00Z
main_date_str 2021
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/17928f35-1d73-4f57-b996-9c3cde08918e\/download","text":"URN:NBN:fi:jyu-202108124521.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2021
record_format qdc
source_str_mv jyx
spellingShingle Uitto, Roope Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes Biomekaniikka Biomechanics 5012 syväoppiminen luotettavuus liikeoppi liikeanalyysi deep learning reliability (general) kinematics motion analysis
title Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes
title_full Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes
title_fullStr Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes
title_full_unstemmed Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes
title_short Deep learning in gait analysis
title_sort deep learning in gait analysis the effect of marker presence in neural network training to kinematic outcomes
title_sub the effect of marker presence in neural network training to kinematic outcomes
title_txtP Deep learning in gait analysis : the effect of marker presence in neural network training to kinematic outcomes
topic Biomekaniikka Biomechanics 5012 syväoppiminen luotettavuus liikeoppi liikeanalyysi deep learning reliability (general) kinematics motion analysis
topic_facet 5012 Biomechanics Biomekaniikka deep learning kinematics liikeanalyysi liikeoppi luotettavuus motion analysis reliability (general) syväoppiminen
url https://jyx.jyu.fi/handle/123456789/77355 http://www.urn.fi/URN:NBN:fi:jyu-202108124521
work_keys_str_mv AT uittoroope deeplearningingaitanalysistheeffectofmarkerpresenceinneuralnetworktrainingtokinematic