Newton update based independent vector analysis with various source density models

Sokea signaalin käsittely tarkoittaa latenttien lähdesignaalien estimointia havaittujen sekoitesignaalien avulla, kun sekoitusympäristö on tuntematon. Riippumattomien komponenttien analyysi (ICA) on sokean signaalin käsittelyn menetelmä, jolla pyritään estimoimaan todellisia lähdesignaaleja maksimoi...

Full description

Bibliographic Details
Main Author: Sipilä, Mika
Other Authors: Matemaattis-luonnontieteellinen tiedekunta, Faculty of Sciences, Matematiikan ja tilastotieteen laitos, Department of Mathematics and Statistics, Jyväskylän yliopisto, University of Jyväskylä
Format: Master's thesis
Language:eng
Published: 2022
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/81752
_version_ 1828193050362380288
author Sipilä, Mika
author2 Matemaattis-luonnontieteellinen tiedekunta Faculty of Sciences Matematiikan ja tilastotieteen laitos Department of Mathematics and Statistics Jyväskylän yliopisto University of Jyväskylä
author_facet Sipilä, Mika Matemaattis-luonnontieteellinen tiedekunta Faculty of Sciences Matematiikan ja tilastotieteen laitos Department of Mathematics and Statistics Jyväskylän yliopisto University of Jyväskylä Sipilä, Mika Matemaattis-luonnontieteellinen tiedekunta Faculty of Sciences Matematiikan ja tilastotieteen laitos Department of Mathematics and Statistics Jyväskylän yliopisto University of Jyväskylä
author_sort Sipilä, Mika
datasource_str_mv jyx
description Sokea signaalin käsittely tarkoittaa latenttien lähdesignaalien estimointia havaittujen sekoitesignaalien avulla, kun sekoitusympäristö on tuntematon. Riippumattomien komponenttien analyysi (ICA) on sokean signaalin käsittelyn menetelmä, jolla pyritään estimoimaan todellisia lähdesignaaleja maksimoimalla niiden välinen riippumattomuus. Riippumattomien vektoreiden analyysi (IVA) on ICA:n laajennos, jolla estimoidaan moniulotteisia lähdesignaalivektoreita olettaen, että jokaisen lähdesignaalivektorin komponentit ovat riippuvia toisistaan. IVA:n tavoitefunktiona käytetään Kullback-Leibler divergenssiä, jota minimoimalla lähdesignaaliestimaattien välinen riippumattomuus maksimoidaan. Minimointia varten täytyy valita optimointimenetelmä sekä lähdesignaaleille sopiva lähdejakaumamalli, jotka määrittävät yhdessä IVA algoritmin suorituskyvyn. Tässä tutkielmassa tarkastellaan neljää algoritmia, joista jokainen perustuu Newtonin menetelmään. Algoritmien lähdejakaumamallit ovat moniulotteinen normaalijakauma (IVA-G), moniulotteinen Laplace-jakauma (IVA-L), moniulotteinen Laplace-jakauma diagonaalisella kovarianssirakenteella (IVA-L-diag) ja moniulotteinen Cauchy-jakauma (IVA-C). Algoritmeja vertaillaan simulointien avulla useissa eri simulaatioasetelmissa. IVA-L, IVA-L-diag ja IVA-C konvergoivat usein lokaaliin minimiin, mikä ratkaistaan alustamalla IVA-L, IVA-L-diag ja IVA-C algoritmit IVA-G:n ja fastIVA:n tuloksilla. FastIVA on alkuperäinen, ortogonaalisiin palautusmatriiseihin rajoittunut IVA-algoritmi. Alustuksen jälkeen IVA-L on tulosten perusteella paras ja monikäyttöisin algoritmi kaikissa tilanteissa. IVA-G on ylivoimaisesti nopein algoritmi, ja suoriutuu hyvin, kun lähdesignaalit ovat riippuvia enimmäkseen toisen asteen momentista. IVA-L-diag ja IVA-C algoritmit parantavat fastIVA:n tuloksia vain marginaalisesti, mutta ovat varteenotettavia vaihtoehtoja, kun lähdesignaalit ovat riippuvia ainoastaan korkeamman asteen momentista. IVA algoritmeja sovelletaan sekoitettujen kuvien erotteluun, jossa viisi alkuperäistä värillistä kuvaa pyritään erottelemaan niiden viidestä satunnaista sekoitteesta. Tässä sovelluksessa IVA-L ja IVA-G algoritmit tuottivat kelvollisia tuloksia, mutta IVA-L-diag ja IVA-C algoritmien tulokset eivät olleet tunnistettavissa. Tutkielmassa käytetyt IVA algoritmit sekä niiden suorituskykyyn liittyvät indeksit ovat julkaistu R-paketissa ivaBSS osana tutkielmaa Blind source separation methods (BSS) are used to estimate latent source signals from their mixed observations when the mixing environment is unknown. Independent component analysis (ICA) is a BSS method, which aims to recover the sources by maximizing the independence between the estimated sources. A more recently developed method, independent vector analysis (IVA), is an extension of ICA to analyse multivariate source signals or multiple datasets jointly. IVA assumes that the source components are dependent on each other between the datasets, which is used to achieve better results than by applying ICA to each dataset separately. IVA uses the Kullback-Leibler divergence as an objective function, which is minimized to achieve as independent source estimates as possible. To minimize the objective function, the source density models and the optimization method need to be selected. In this thesis, four different algorithms are investigated, each of which is using a Newton update based optimization method. The source density models of the algorithms are the multivariate Gaussian (IVA-G), the multivariate Laplace with any covariance structure (IVA-L), the multivariate Laplace with diagonal covariance structure (IVA-L-diag) and the multivariate Cauchy (IVA-C) distributions. The algorithms are compared under different situations using simulation studies. IVA-L, IVA-L-diag and IVA-C tend to converge often to local optima, which is avoided by initializing IVA-L, IVA-L-diag and IVA-C with the estimated unmixing matrices of IVA-G and fastIVA. FastIVA is the original IVA algorithm, which restricts the unmixing matrices to be orthogonal. After the initialization, IVA-L becomes the most flexible and consistent algorithm in all setups. IVA-G performs well when the sources are mostly second-order dependent, and is superior in terms of computation time. IVA-L-diag and IVA-C improve the results of fastIVA only marginally, and perform well when the sources are purely higher-order dependent and the number of datasets is significantly higher than the number of sources. The algorithms are applied to mixed image separation task, where five random mixtures of five colored images are separated. In this application IVA-L and IVA-G algorithms provide sufficient results, but the separated images of IVA-L-diag and IVA-C are not recognizable. The IVA algorithms and their performance indices are implemented in R package ivaBSS as a part of the thesis.
first_indexed 2022-06-15T20:04:16Z
format Pro gradu
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "Nordhausen, Klaus", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.advisor", "value": "Taskinen, Sara", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "Sipil\u00e4, Mika", "language": "", "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2022-06-15T11:36:03Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2022-06-15T11:36:03Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2022", "language": "", "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/81752", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Sokea signaalin k\u00e4sittely tarkoittaa latenttien l\u00e4hdesignaalien estimointia havaittujen\nsekoitesignaalien avulla, kun sekoitusymp\u00e4rist\u00f6 on tuntematon. Riippumattomien\nkomponenttien analyysi (ICA) on sokean signaalin k\u00e4sittelyn menetelm\u00e4, jolla pyrit\u00e4\u00e4n\nestimoimaan todellisia l\u00e4hdesignaaleja maksimoimalla niiden v\u00e4linen riippumattomuus.\nRiippumattomien vektoreiden analyysi (IVA) on ICA:n laajennos, jolla estimoidaan\nmoniulotteisia l\u00e4hdesignaalivektoreita olettaen, ett\u00e4 jokaisen l\u00e4hdesignaalivektorin\nkomponentit ovat riippuvia toisistaan.\n\nIVA:n tavoitefunktiona k\u00e4ytet\u00e4\u00e4n Kullback-Leibler divergenssi\u00e4, jota minimoimalla\nl\u00e4hdesignaaliestimaattien v\u00e4linen riippumattomuus maksimoidaan. Minimointia varten\nt\u00e4ytyy valita optimointimenetelm\u00e4 sek\u00e4 l\u00e4hdesignaaleille sopiva l\u00e4hdejakaumamalli, jotka\nm\u00e4\u00e4ritt\u00e4v\u00e4t yhdess\u00e4 IVA algoritmin suorituskyvyn. T\u00e4ss\u00e4 tutkielmassa tarkastellaan\nnelj\u00e4\u00e4 algoritmia, joista jokainen perustuu Newtonin menetelm\u00e4\u00e4n. Algoritmien\nl\u00e4hdejakaumamallit ovat moniulotteinen normaalijakauma (IVA-G), moniulotteinen\nLaplace-jakauma (IVA-L), moniulotteinen Laplace-jakauma diagonaalisella\nkovarianssirakenteella (IVA-L-diag) ja moniulotteinen Cauchy-jakauma (IVA-C).\n\nAlgoritmeja vertaillaan simulointien avulla useissa eri simulaatioasetelmissa. IVA-L,\nIVA-L-diag ja IVA-C konvergoivat usein lokaaliin minimiin, mik\u00e4 ratkaistaan alustamalla\nIVA-L, IVA-L-diag ja IVA-C algoritmit IVA-G:n ja fastIVA:n tuloksilla. FastIVA on\nalkuper\u00e4inen, ortogonaalisiin palautusmatriiseihin rajoittunut IVA-algoritmi. Alustuksen\nj\u00e4lkeen IVA-L on tulosten perusteella paras ja monik\u00e4ytt\u00f6isin algoritmi kaikissa\ntilanteissa. IVA-G on ylivoimaisesti nopein algoritmi, ja suoriutuu hyvin, kun\nl\u00e4hdesignaalit ovat riippuvia enimm\u00e4kseen toisen asteen momentista. IVA-L-diag\nja IVA-C algoritmit parantavat fastIVA:n tuloksia vain marginaalisesti, mutta ovat\nvarteenotettavia vaihtoehtoja, kun l\u00e4hdesignaalit ovat riippuvia ainoastaan korkeamman\nasteen momentista.\n\nIVA algoritmeja sovelletaan sekoitettujen kuvien erotteluun, jossa viisi alkuper\u00e4ist\u00e4\nv\u00e4rillist\u00e4 kuvaa pyrit\u00e4\u00e4n erottelemaan niiden viidest\u00e4 satunnaista sekoitteesta. T\u00e4ss\u00e4\nsovelluksessa IVA-L ja IVA-G algoritmit tuottivat kelvollisia tuloksia, mutta IVA-L-diag\nja IVA-C algoritmien tulokset eiv\u00e4t olleet tunnistettavissa. Tutkielmassa k\u00e4ytetyt IVA\nalgoritmit sek\u00e4 niiden suorituskykyyn liittyv\u00e4t indeksit ovat julkaistu R-paketissa ivaBSS\nosana tutkielmaa", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Blind source separation methods (BSS) are used to estimate latent source signals from their\nmixed observations when the mixing environment is unknown. Independent component\nanalysis (ICA) is a BSS method, which aims to recover the sources by maximizing\nthe independence between the estimated sources. A more recently developed method,\nindependent vector analysis (IVA), is an extension of ICA to analyse multivariate source\nsignals or multiple datasets jointly. IVA assumes that the source components are dependent\non each other between the datasets, which is used to achieve better results than by applying\nICA to each dataset separately. IVA uses the Kullback-Leibler divergence as an objective\nfunction, which is minimized to achieve as independent source estimates as possible.\n\nTo minimize the objective function, the source density models and the optimization\nmethod need to be selected. In this thesis, four different algorithms are investigated, each\nof which is using a Newton update based optimization method. The source density models\nof the algorithms are the multivariate Gaussian (IVA-G), the multivariate Laplace with any\ncovariance structure (IVA-L), the multivariate Laplace with diagonal covariance structure\n(IVA-L-diag) and the multivariate Cauchy (IVA-C) distributions.\n\nThe algorithms are compared under different situations using simulation studies.\nIVA-L, IVA-L-diag and IVA-C tend to converge often to local optima, which is avoided\nby initializing IVA-L, IVA-L-diag and IVA-C with the estimated unmixing matrices of\nIVA-G and fastIVA. FastIVA is the original IVA algorithm, which restricts the unmixing\nmatrices to be orthogonal. After the initialization, IVA-L becomes the most flexible and\nconsistent algorithm in all setups. IVA-G performs well when the sources are mostly\nsecond-order dependent, and is superior in terms of computation time. IVA-L-diag and\nIVA-C improve the results of fastIVA only marginally, and perform well when the sources\nare purely higher-order dependent and the number of datasets is significantly higher than\nthe number of sources.\n\nThe algorithms are applied to mixed image separation task, where five random mixtures\nof five colored images are separated. In this application IVA-L and IVA-G algorithms\nprovide sufficient results, but the separated images of IVA-L-diag and IVA-C are not\nrecognizable. The IVA algorithms and their performance indices are implemented in R\npackage ivaBSS as a part of the thesis.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by Miia Hakanen (mihakane@jyu.fi) on 2022-06-15T11:36:03Z\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2022-06-15T11:36:03Z (GMT). No. of bitstreams: 0\n Previous issue date: 2022", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "73", "language": "", "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.format.mimetype", "value": "application/pdf", "language": null, "element": "format", "qualifier": "mimetype", "schema": "dc"}, {"key": "dc.language.iso", "value": "eng", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "In Copyright", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.subject.other", "value": "Independent vector analysis", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "independent component analysis", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "blind source separation", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "Newton update", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.title", "value": "Newton update based independent vector analysis with various source density models", "language": "", "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "master thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-202206153362", "language": "", "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Pro gradu -tutkielma", "language": "fi", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Master\u2019s thesis", "language": "en", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Matemaattis-luonnontieteellinen tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Sciences", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Matematiikan ja tilastotieteen laitos", "language": "fi", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Department of Mathematics and Statistics", "language": "en", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Tilastotiede", "language": "fi", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Statistics", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "yvv.contractresearch.funding", "value": "0", "language": "", "element": "contractresearch", "qualifier": "funding", "schema": "yvv"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_bdcc", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "masterThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.subject.oppiainekoodi", "value": "4043", "language": "", "element": "subject", "qualifier": "oppiainekoodi", "schema": "dc"}, {"key": "dc.subject.yso", "value": "algoritmit", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "monimuuttujamenetelm\u00e4t", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "optimointi", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "algorithms", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "multivariable methods", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "optimisation", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.format.content", "value": "fulltext", "language": null, "element": "format", "qualifier": "content", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://rightsstatements.org/page/InC/1.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}, {"key": "dc.type.okm", "value": "G2", "language": null, "element": "type", "qualifier": "okm", "schema": "dc"}]
id jyx.123456789_81752
language eng
last_indexed 2025-03-31T20:01:49Z
main_date 2022-01-01T00:00:00Z
main_date_str 2022
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/169d042c-f413-403e-b575-096dcc83acd6\/download","text":"URN:NBN:fi:jyu-202206153362.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2022
record_format qdc
source_str_mv jyx
spellingShingle Sipilä, Mika Newton update based independent vector analysis with various source density models Independent vector analysis independent component analysis blind source separation Newton update Tilastotiede Statistics 4043 algoritmit monimuuttujamenetelmät optimointi algorithms multivariable methods optimisation
title Newton update based independent vector analysis with various source density models
title_full Newton update based independent vector analysis with various source density models
title_fullStr Newton update based independent vector analysis with various source density models Newton update based independent vector analysis with various source density models
title_full_unstemmed Newton update based independent vector analysis with various source density models Newton update based independent vector analysis with various source density models
title_short Newton update based independent vector analysis with various source density models
title_sort newton update based independent vector analysis with various source density models
title_txtP Newton update based independent vector analysis with various source density models
topic Independent vector analysis independent component analysis blind source separation Newton update Tilastotiede Statistics 4043 algoritmit monimuuttujamenetelmät optimointi algorithms multivariable methods optimisation
topic_facet 4043 Independent vector analysis Newton update Statistics Tilastotiede algorithms algoritmit blind source separation independent component analysis monimuuttujamenetelmät multivariable methods optimisation optimointi
url https://jyx.jyu.fi/handle/123456789/81752 http://www.urn.fi/URN:NBN:fi:jyu-202206153362
work_keys_str_mv AT sipilämika newtonupdatebasedindependentvectoranalysiswithvarioussourcedensitymodels