Audiovisual processing of Chinese speech sound - character pairs a MEG study

Tämän tutkimuksen tarkoituksena oli selvittää audiovisuaalisen integraation aivoperustaa. Tutkimuksen toteutuksessa käytettiin visuaalisena ärsykkeenä kiinankielisiä kirjoitusmerkkejä ja auditorisena ärsykkeenä kiinankielisiä puheäänteitä, joiden vaikutusta aivoihin tarkasteltiin joko esittämällä ni...

Full description

Bibliographic Details
Main Author: Koskialho, Katja
Other Authors: Kasvatustieteiden ja psykologian tiedekunta, Faculty of Education and Psychology, Psykologian laitos, Department of Psychology, Jyväskylän yliopisto, University of Jyväskylä
Format: Master's thesis
Language:eng
Published: 2016
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/86109
_version_ 1828193110252847104
author Koskialho, Katja
author2 Kasvatustieteiden ja psykologian tiedekunta Faculty of Education and Psychology Psykologian laitos Department of Psychology Jyväskylän yliopisto University of Jyväskylä
author_facet Koskialho, Katja Kasvatustieteiden ja psykologian tiedekunta Faculty of Education and Psychology Psykologian laitos Department of Psychology Jyväskylän yliopisto University of Jyväskylä Koskialho, Katja Kasvatustieteiden ja psykologian tiedekunta Faculty of Education and Psychology Psykologian laitos Department of Psychology Jyväskylän yliopisto University of Jyväskylä
author_sort Koskialho, Katja
datasource_str_mv jyx
description Tämän tutkimuksen tarkoituksena oli selvittää audiovisuaalisen integraation aivoperustaa. Tutkimuksen toteutuksessa käytettiin visuaalisena ärsykkeenä kiinankielisiä kirjoitusmerkkejä ja auditorisena ärsykkeenä kiinankielisiä puheäänteitä, joiden vaikutusta aivoihin tarkasteltiin joko esittämällä niitä unimodaalisesti tai bimodaalisesti. Muodostettiin kaksi tutkittavien ryhmää: suomenkieliset, jotka eivät osanneet kiinaa sekä natiivit kiinan puhujat, jotka ymmärsivät myös merkkien ja puheäänteiden merkityksen. Näin päästiin tarkastelemaan myös sitä, onko sanojen merkityksen tuntemisella merkitystä äänteiden ja merkkien prosessointiin. Aivojen toimintaa mitattiin 306-kanavaisella aivomagneettikäyrää mittaavalla magnetoenkefalografilla, MEG:llä. Aivoaktiivisuuden paikantumista tutkittiin BESA 6.1. -ohjelmistolla, Classical LORETA Analysis Recursively Applied (CLARA) - lähteenpaikannuksella. Suomalaisten ja kiinalaisten aivoaktiivisuutta vertailtiin eri tilanteiden välillä. Tulokset osoittivat, että suomalaisten visuaalinen aivokuori aktivoitui voimakkaammin kuin kiinalaisten, kun visuaalisia ärsykkeitä esitettiin ilman ääntä (V). Kun stimuluspareja (AV) esitettiin kongruentisti eli yhdenmukaisesti, suomalaisten oikea ja vasen fusiforminen poimu aktivoitui voimakkaammin kuin kiinalaisten. Tuloksista voidaan päätellä, että audiovisuaalinen prosessointi aivoissa toimii monimutkaisesti, ja merkityksen ymmärtämisellä voi olla vaikutusta siihen, millä aivoalueella tai miten voimakkaasti aivot aktivoituvat. The aim of this thesis was to find out the neuronal basis of audiovisual integration. In the implementation of this study Chinese characters were used as a visual stimulus and for the auditory stimulus Chinese speech sounds were used, of which effects to the brain were observed by presenting the stimuli either unimodally or bimodally. Two groups of subjects were formed: Finnish subjects who did not know any Chinese and native Chinese subjects who also understood the meaning of the characters and speech sounds. This made it possible to also examine whether understanding the meaning of words made an impact on the processing of speech sounds and characters. The brain activation was measured by 306-channel magnetic field-recording magnetoencephalogram, MEG. The localization of the brain activation was explored by BESA 6.1. software, Classical LORETA Analysis Recursively Applied (CLARA) - source localization. The brain activation of the Finnish and Chinese subjects were compared between different conditions. The results show that Finnish subjects’ visual cortex activated more strongly than Chinese, when the visual stimulus was presented without the voice (V). When stimulus pairs (AV) were presented congruently or consistently, Finnish right and left fusiform gyrus was activated more powerfully than Chinese. A conclusion can be reached that the audiovisual processing in the brain functions in complex ways, and knowing the meaning of the shown stimuli can affect in which brain region or how powerfully the brain activates.
first_indexed 2023-03-24T21:05:01Z
format Pro gradu
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "H\u00e4m\u00e4l\u00e4inen, Jarmo", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "Koskialho, Katja", "language": "", "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2023-03-24T07:16:55Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2023-03-24T07:16:55Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2016", "language": "", "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/86109", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "T\u00e4m\u00e4n tutkimuksen tarkoituksena oli selvitt\u00e4\u00e4 audiovisuaalisen integraation aivoperustaa. Tutkimuksen toteutuksessa k\u00e4ytettiin visuaalisena \u00e4rsykkeen\u00e4 kiinankielisi\u00e4 kirjoitusmerkkej\u00e4 ja auditorisena \u00e4rsykkeen\u00e4 kiinankielisi\u00e4 puhe\u00e4\u00e4nteit\u00e4, joiden vaikutusta aivoihin tarkasteltiin joko esitt\u00e4m\u00e4ll\u00e4 niit\u00e4 unimodaalisesti tai bimodaalisesti. Muodostettiin kaksi tutkittavien ryhm\u00e4\u00e4: suomenkieliset, jotka eiv\u00e4t osanneet kiinaa sek\u00e4 natiivit kiinan puhujat, jotka ymm\u00e4rsiv\u00e4t my\u00f6s merkkien ja puhe\u00e4\u00e4nteiden merkityksen. N\u00e4in p\u00e4\u00e4stiin tarkastelemaan my\u00f6s sit\u00e4, onko sanojen merkityksen tuntemisella merkityst\u00e4 \u00e4\u00e4nteiden ja merkkien prosessointiin.\nAivojen toimintaa mitattiin 306-kanavaisella aivomagneettik\u00e4yr\u00e4\u00e4 mittaavalla magnetoenkefalografilla, MEG:ll\u00e4. Aivoaktiivisuuden paikantumista tutkittiin BESA 6.1. -ohjelmistolla, Classical LORETA Analysis Recursively Applied (CLARA) - l\u00e4hteenpaikannuksella. Suomalaisten ja kiinalaisten aivoaktiivisuutta vertailtiin eri tilanteiden v\u00e4lill\u00e4. Tulokset osoittivat, ett\u00e4 suomalaisten visuaalinen aivokuori aktivoitui voimakkaammin kuin kiinalaisten, kun visuaalisia \u00e4rsykkeit\u00e4 esitettiin ilman \u00e4\u00e4nt\u00e4 (V). Kun stimuluspareja (AV) esitettiin kongruentisti eli yhdenmukaisesti, suomalaisten oikea ja vasen fusiforminen poimu aktivoitui voimakkaammin kuin kiinalaisten. Tuloksista voidaan p\u00e4\u00e4tell\u00e4, ett\u00e4 audiovisuaalinen prosessointi aivoissa toimii monimutkaisesti, ja merkityksen ymm\u00e4rt\u00e4misell\u00e4 voi olla vaikutusta siihen, mill\u00e4 aivoalueella tai miten voimakkaasti aivot aktivoituvat.", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "The aim of this thesis was to find out the neuronal basis of audiovisual integration. In the implementation of this study Chinese characters were used as a visual stimulus and for the auditory stimulus Chinese speech sounds were used, of which effects to the brain were observed by presenting the stimuli either unimodally or bimodally. Two groups of subjects were formed: Finnish subjects who did not know any Chinese and native Chinese subjects who also understood the meaning of the characters and speech sounds. This made it possible to also examine whether understanding the meaning of words made an impact on the processing of speech sounds and characters.\nThe brain activation was measured by 306-channel magnetic field-recording magnetoencephalogram, MEG. The localization of the brain activation was explored by BESA 6.1. software, Classical LORETA Analysis Recursively Applied (CLARA) - source localization. The brain activation of the Finnish and Chinese subjects were compared between different conditions. The results show that Finnish subjects\u2019 visual cortex activated more strongly than Chinese, when the visual stimulus was presented without the voice (V). When stimulus pairs (AV) were presented congruently or consistently, Finnish right and left fusiform gyrus was activated more powerfully than Chinese. A conclusion can be reached that the audiovisual processing in the brain functions in complex ways, and knowing the meaning of the shown stimuli can affect in which brain region or how powerfully the brain activates.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by Paivi Vuorio (paelvuor@jyu.fi) on 2023-03-24T07:16:55Z\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2023-03-24T07:16:55Z (GMT). No. of bitstreams: 0\n Previous issue date: 2016", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "32", "language": "", "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.format.mimetype", "value": "application/pdf", "language": null, "element": "format", "qualifier": "mimetype", "schema": "dc"}, {"key": "dc.language.iso", "value": "eng", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "In Copyright", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.subject.other", "value": "audiovisual integration", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "audiovisual processing", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "speech sounds", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "multisensory integration", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "fusiform gurys", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "posterior superior temporal sulcus", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.title", "value": "Audiovisual processing of Chinese speech sound - character pairs : a MEG study", "language": "", "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "master thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-202303242258", "language": "", "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Pro gradu -tutkielma", "language": "fi", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Master\u2019s thesis", "language": "en", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Kasvatustieteiden ja psykologian tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Education and Psychology", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Psykologian laitos", "language": "fi", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Department of Psychology", "language": "en", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Psykologia", "language": "fi", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Psychology", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "yvv.contractresearch.funding", "value": "0", "language": "", "element": "contractresearch", "qualifier": "funding", "schema": "yvv"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_bdcc", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "masterThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.subject.oppiainekoodi", "value": "202", "language": "", "element": "subject", "qualifier": "oppiainekoodi", "schema": "dc"}, {"key": "dc.subject.yso", "value": "aivotutkimus", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "audiovisuaalinen viestint\u00e4", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "brain research", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "audiovisual communication", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.format.content", "value": "fulltext", "language": null, "element": "format", "qualifier": "content", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://rightsstatements.org/page/InC/1.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}, {"key": "dc.type.okm", "value": "G2", "language": null, "element": "type", "qualifier": "okm", "schema": "dc"}]
id jyx.123456789_86109
language eng
last_indexed 2025-03-31T20:01:35Z
main_date 2016-01-01T00:00:00Z
main_date_str 2016
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/fa8ed1f2-a353-44de-a82a-616e360b15c5\/download","text":"URN:NBN:fi:jyu-202303242258.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2016
record_format qdc
source_str_mv jyx
spellingShingle Koskialho, Katja Audiovisual processing of Chinese speech sound - character pairs : a MEG study audiovisual integration audiovisual processing speech sounds multisensory integration fusiform gurys posterior superior temporal sulcus Psykologia Psychology 202 aivotutkimus audiovisuaalinen viestintä brain research audiovisual communication
title Audiovisual processing of Chinese speech sound - character pairs : a MEG study
title_full Audiovisual processing of Chinese speech sound - character pairs : a MEG study
title_fullStr Audiovisual processing of Chinese speech sound - character pairs : a MEG study Audiovisual processing of Chinese speech sound - character pairs : a MEG study
title_full_unstemmed Audiovisual processing of Chinese speech sound - character pairs : a MEG study Audiovisual processing of Chinese speech sound - character pairs : a MEG study
title_short Audiovisual processing of Chinese speech sound - character pairs
title_sort audiovisual processing of chinese speech sound character pairs a meg study
title_sub a MEG study
title_txtP Audiovisual processing of Chinese speech sound - character pairs : a MEG study
topic audiovisual integration audiovisual processing speech sounds multisensory integration fusiform gurys posterior superior temporal sulcus Psykologia Psychology 202 aivotutkimus audiovisuaalinen viestintä brain research audiovisual communication
topic_facet 202 Psychology Psykologia aivotutkimus audiovisuaalinen viestintä audiovisual communication audiovisual integration audiovisual processing brain research fusiform gurys multisensory integration posterior superior temporal sulcus speech sounds
url https://jyx.jyu.fi/handle/123456789/86109 http://www.urn.fi/URN:NBN:fi:jyu-202303242258
work_keys_str_mv AT koskialhokatja audiovisualprocessingofchinesespeechsoundcharacterpairsamegstudy