Evaluating the sensitivity of lightweight object detection models against adversarial perturbations

Syväoppivat neuroverkot ovat viime vuosina olleet yleisin käytetty menetelmä hahmontunnistuksessa niiden tarjotessa merkittäviä parannuksia suorituskykyyn sekä tunnistusten tarkkuuteen. Samanaikaisesti IoT-teknologia on alkanut integroitua tekoälyteknologian kanssa, jonka seurauksena kameroita sisäl...

Full description

Bibliographic Details
Main Author: Mäyrä, Ville-Matti
Other Authors: Informaatioteknologian tiedekunta, Faculty of Information Technology, Informaatioteknologia, Information Technology, Jyväskylän yliopisto, University of Jyväskylä
Format: Master's thesis
Language:eng
Published: 2022
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/81442
_version_ 1826225756012806144
author Mäyrä, Ville-Matti
author2 Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä
author_facet Mäyrä, Ville-Matti Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä Mäyrä, Ville-Matti Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä
author_sort Mäyrä, Ville-Matti
datasource_str_mv jyx
description Syväoppivat neuroverkot ovat viime vuosina olleet yleisin käytetty menetelmä hahmontunnistuksessa niiden tarjotessa merkittäviä parannuksia suorituskykyyn sekä tunnistusten tarkkuuteen. Samanaikaisesti IoT-teknologia on alkanut integroitua tekoälyteknologian kanssa, jonka seurauksena kameroita sisältävät IoT-laitteet ovat alkaneet hyödyntämään kuvantunnistus tekniikoita. Nämä ovat kasvattaneet kiinnostusta hahmontunnistus ratkaisujen käytöstä sovelluksissa ja laitteissa eri aloilla, kuten valvonnassa ja itseohjautuvissa ajoneuvoissa. Viime aikoina syväoppimismalleja on implementoitu suoraan laitteisiin, tarpeettoman liikenteen pitämiseksi poissa verkosta. Tämä on tukenut kevyiden hahmontunnistusmallien kehitystä. Tutkimukset ovat kuitenkin osoittaneet, että syväoppivat neuroverkot ovat haavoittuvia vihamielisille esimerkeille, joilla tarkoitetaan vaikeasti havaittavaa kohinaa sisältäviä kuvia, jonka seurauksena hahmontunnistus mallit saadaan tekemään virheellisiä tunnistuksia. Tämä tutkielma keskittyy toteuttamaan kevyiden hahmontunnistus mallien herkkyysanalyysin vihamielistä kohinaa kohtaan sekä tutkii millaisia vihamielisiä hyökkäyksiä hahmontunnistus malleja vastaan, on olemassa. Tutkimukset suoritettiin luomalla vihamielisiä esimerkkejä käyttäen python kirjastoa, joka oli luotu vihamielisten hyökkäysten testaamiseen syväoppivilla neuroverkoilla. Arvioinnit toteutettiin testaamalla valittuja esikoulutettuja malleja tietoaineistoilla, jotka pohjautuivat COCO 2017 tietoaineistoon. Kokeet osoittavat, että projected gradient descent metodilla luodun hyökkäävän kohinan käyttö laski mallien keskimääräistä tarkkuutta matalilla kohinan tasoilla 4–10%, keskiverto tasoilla 10–28% sekä korkeilla tasoilla 25–49%. Käyttämällä fast gradient sign metodia hyökkäävän kohinan luonnissa, mallien keskimääräinen tarkkuus laski matalilla kohinan tasoilla 10–22%, keskiverto tasoilla 35–53% sekä korkeilla tasoilla 70–84%. Tutkimuksessa käytetyistä malleista parhaiten hyökkäävää kohinaa vastusti EfficientDet D0 512x512 malli. Tulokset osoittavat, että esikoulutetut kevyet hahmontunnistusmallit ovat haavoittuvia python kirjastolla luoduille hyökkääville esimerkeille ja tutkimusta mallien sitkeyden parantamiseksi tulisi jatkaa. The use of deep neural networks in the object detection task has become the mainstream solution in recent years due to the major improvements in the performance and accuracy of the detections that they have offered. Simultaneously the IoT technology has started to integrate with artificial intelligence technology and IoT devices with integrated cameras have started to adopt image recognition techniques. These have increased the interest in using object detection solutions on applications and devices in fields like surveillance and autonomous driving. Recently the applications have also started to adopt using deep learning models on-device to keep the unnecessary traffic off the network, which has supported the development of lightweight object detection models. However, the studies have shown that deep neural networks are vulnerable to adversarial examples, which are images that contain a subtle perturbation capable to fool the object detector to make false detection. This thesis focuses to evaluate the sensitivity of lightweight object detection models against adversarial perturbation and studies what kind of adversarial attacks currently exist against object detection models. The experiments were conducted by crafting the adversarial examples using the python library designed to run adversarial attacks against deep neural networks. The evaluations were completed on selected pre-trained models while using the datasets based on COCO 2017 dataset. Experiments show that using the adversarial perturbation crafted on the projected gradient descent method, the mean average precision of the selected models was decreased on low noise levels 4–10%, on average levels 10–28%, and on high levels 25–49%. When using adversarial perturbation crafted on the fast gradient sign method, the mean average precision of the selected models was decreased on low noise levels 10–22 %, on average levels 35–53%, and on high levels 70–84%. From the models used in experiments, the EfficientDet D0 512x512 model proved to resist the adversarial perturbation better than the others. Results show that pre-trained lightweight object detection models are vulnerable to adversarial examples crafted using the python library and it would need more research to make them more robust.
first_indexed 2022-06-03T20:00:48Z
format Pro gradu
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "P\u00f6l\u00f6nen, Ilkka", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.advisor", "value": "Khriyenko, Oleksiy", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "M\u00e4yr\u00e4, Ville-Matti", "language": "", "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2022-06-03T10:01:12Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2022-06-03T10:01:12Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2022", "language": "", "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/81442", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Syv\u00e4oppivat neuroverkot ovat viime vuosina olleet yleisin k\u00e4ytetty menetelm\u00e4 hahmontunnistuksessa niiden tarjotessa merkitt\u00e4vi\u00e4 parannuksia suorituskykyyn sek\u00e4 tunnistusten tarkkuuteen. Samanaikaisesti IoT-teknologia on alkanut integroitua teko\u00e4lyteknologian kanssa, jonka seurauksena kameroita sis\u00e4lt\u00e4v\u00e4t IoT-laitteet ovat alkaneet hy\u00f6dynt\u00e4m\u00e4\u00e4n kuvantunnistus tekniikoita. N\u00e4m\u00e4 ovat kasvattaneet kiinnostusta hahmontunnistus ratkaisujen k\u00e4yt\u00f6st\u00e4 sovelluksissa ja laitteissa eri aloilla, kuten valvonnassa ja itseohjautuvissa ajoneuvoissa. Viime aikoina syv\u00e4oppimismalleja on implementoitu suoraan laitteisiin, tarpeettoman liikenteen pit\u00e4miseksi poissa verkosta. T\u00e4m\u00e4 on tukenut kevyiden hahmontunnistusmallien kehityst\u00e4. Tutkimukset ovat kuitenkin osoittaneet, ett\u00e4 syv\u00e4oppivat neuroverkot ovat haavoittuvia vihamielisille esimerkeille, joilla tarkoitetaan vaikeasti havaittavaa kohinaa sis\u00e4lt\u00e4vi\u00e4 kuvia, jonka seurauksena hahmontunnistus mallit saadaan tekem\u00e4\u00e4n virheellisi\u00e4 tunnistuksia.\nT\u00e4m\u00e4 tutkielma keskittyy toteuttamaan kevyiden hahmontunnistus mallien herkkyysanalyysin vihamielist\u00e4 kohinaa kohtaan sek\u00e4 tutkii millaisia vihamielisi\u00e4 hy\u00f6kk\u00e4yksi\u00e4 hahmontunnistus malleja vastaan, on olemassa. Tutkimukset suoritettiin luomalla vihamielisi\u00e4 esimerkkej\u00e4 k\u00e4ytt\u00e4en python kirjastoa, joka oli luotu vihamielisten hy\u00f6kk\u00e4ysten testaamiseen syv\u00e4oppivilla neuroverkoilla. Arvioinnit toteutettiin testaamalla valittuja esikoulutettuja malleja tietoaineistoilla, jotka pohjautuivat COCO 2017 tietoaineistoon. Kokeet osoittavat, ett\u00e4 projected gradient descent metodilla luodun hy\u00f6kk\u00e4\u00e4v\u00e4n kohinan k\u00e4ytt\u00f6 laski mallien keskim\u00e4\u00e4r\u00e4ist\u00e4 tarkkuutta matalilla kohinan tasoilla 4\u201310%, keskiverto tasoilla 10\u201328% sek\u00e4 korkeilla tasoilla 25\u201349%. K\u00e4ytt\u00e4m\u00e4ll\u00e4 fast gradient sign metodia hy\u00f6kk\u00e4\u00e4v\u00e4n kohinan luonnissa, mallien keskim\u00e4\u00e4r\u00e4inen tarkkuus laski matalilla kohinan tasoilla 10\u201322%, keskiverto tasoilla 35\u201353% sek\u00e4 korkeilla tasoilla 70\u201384%. Tutkimuksessa k\u00e4ytetyist\u00e4 malleista parhaiten hy\u00f6kk\u00e4\u00e4v\u00e4\u00e4 kohinaa vastusti EfficientDet D0 512x512 malli. Tulokset osoittavat, ett\u00e4 esikoulutetut kevyet hahmontunnistusmallit ovat haavoittuvia python kirjastolla luoduille hy\u00f6kk\u00e4\u00e4ville esimerkeille ja tutkimusta mallien sitkeyden parantamiseksi tulisi jatkaa.", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "The use of deep neural networks in the object detection task has become the mainstream solution in recent years due to the major improvements in the performance and accuracy of the detections that they have offered. Simultaneously the IoT technology has started to integrate with artificial intelligence technology and IoT devices with integrated cameras have started to adopt image recognition techniques. These have increased the interest in using object detection solutions on applications and devices in fields like surveillance and autonomous driving. Recently the applications have also started to adopt using deep learning models on-device to keep the unnecessary traffic off the network, which has supported the development of lightweight object detection models. However, the studies have shown that deep neural networks are vulnerable to adversarial examples, which are images that contain a subtle perturbation capable to fool the object detector to make false detection. \nThis thesis focuses to evaluate the sensitivity of lightweight object detection models against adversarial perturbation and studies what kind of adversarial attacks currently exist against object detection models. The experiments were conducted by crafting the adversarial examples using the python library designed to run adversarial attacks against deep neural networks. The evaluations were completed on selected pre-trained models while using the datasets based on COCO 2017 dataset. Experiments show that using the adversarial perturbation crafted on the projected gradient descent method, the mean average precision of the selected models was decreased on low noise levels 4\u201310%, on average levels 10\u201328%, and on high levels 25\u201349%. When using adversarial perturbation crafted on the fast gradient sign method, the mean average precision of the selected models was decreased on low noise levels 10\u201322 %, on average levels 35\u201353%, and on high levels 70\u201384%. From the models used in experiments, the EfficientDet D0 512x512 model proved to resist the adversarial perturbation better than the others. Results show that pre-trained lightweight object detection models are vulnerable to adversarial examples crafted using the python library and it would need more research to make them more robust.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by Paivi Vuorio (paelvuor@jyu.fi) on 2022-06-03T10:01:12Z\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2022-06-03T10:01:12Z (GMT). No. of bitstreams: 0\n Previous issue date: 2022", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "109", "language": "", "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.format.mimetype", "value": "application/pdf", "language": null, "element": "format", "qualifier": "mimetype", "schema": "dc"}, {"key": "dc.language.iso", "value": "eng", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "In Copyright", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.subject.other", "value": "adversarial examples", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "object detection", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "sensitivity analysis", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "vihamielinen esimerkki", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "herkkyysanalyysi", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.title", "value": "Evaluating the sensitivity of lightweight object detection models against adversarial perturbations", "language": "", "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "master thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-202206033060", "language": "", "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Pro gradu -tutkielma", "language": "fi", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Master\u2019s thesis", "language": "en", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Informaatioteknologian tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Information Technology", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Informaatioteknologia", "language": "fi", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Information Technology", "language": "en", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Tietotekniikka", "language": "fi", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Mathematical Information Technology", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "yvv.contractresearch.funding", "value": "0", "language": "", "element": "contractresearch", "qualifier": "funding", "schema": "yvv"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_bdcc", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "masterThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.subject.oppiainekoodi", "value": "602", "language": "", "element": "subject", "qualifier": "oppiainekoodi", "schema": "dc"}, {"key": "dc.subject.yso", "value": "neuroverkot", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "syv\u00e4oppiminen", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "hahmontunnistus (kognitio)", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "neural networks (information technology)", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "deep learning", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "form recognition (cognition)", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.format.content", "value": "fulltext", "language": null, "element": "format", "qualifier": "content", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://rightsstatements.org/page/InC/1.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}, {"key": "dc.type.okm", "value": "G2", "language": null, "element": "type", "qualifier": "okm", "schema": "dc"}]
id jyx.123456789_81442
language eng
last_indexed 2025-02-18T10:55:53Z
main_date 2022-01-01T00:00:00Z
main_date_str 2022
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/f0862109-73c4-4ac2-9cf9-e1fa666572b9\/download","text":"URN:NBN:fi:jyu-202206033060.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2022
record_format qdc
source_str_mv jyx
spellingShingle Mäyrä, Ville-Matti Evaluating the sensitivity of lightweight object detection models against adversarial perturbations adversarial examples object detection sensitivity analysis vihamielinen esimerkki herkkyysanalyysi Tietotekniikka Mathematical Information Technology 602 neuroverkot syväoppiminen hahmontunnistus (kognitio) neural networks (information technology) deep learning form recognition (cognition)
title Evaluating the sensitivity of lightweight object detection models against adversarial perturbations
title_full Evaluating the sensitivity of lightweight object detection models against adversarial perturbations
title_fullStr Evaluating the sensitivity of lightweight object detection models against adversarial perturbations Evaluating the sensitivity of lightweight object detection models against adversarial perturbations
title_full_unstemmed Evaluating the sensitivity of lightweight object detection models against adversarial perturbations Evaluating the sensitivity of lightweight object detection models against adversarial perturbations
title_short Evaluating the sensitivity of lightweight object detection models against adversarial perturbations
title_sort evaluating the sensitivity of lightweight object detection models against adversarial perturbations
title_txtP Evaluating the sensitivity of lightweight object detection models against adversarial perturbations
topic adversarial examples object detection sensitivity analysis vihamielinen esimerkki herkkyysanalyysi Tietotekniikka Mathematical Information Technology 602 neuroverkot syväoppiminen hahmontunnistus (kognitio) neural networks (information technology) deep learning form recognition (cognition)
topic_facet 602 Mathematical Information Technology Tietotekniikka adversarial examples deep learning form recognition (cognition) hahmontunnistus (kognitio) herkkyysanalyysi neural networks (information technology) neuroverkot object detection sensitivity analysis syväoppiminen vihamielinen esimerkki
url https://jyx.jyu.fi/handle/123456789/81442 http://www.urn.fi/URN:NBN:fi:jyu-202206033060
work_keys_str_mv AT mäyrävillematti evaluatingthesensitivityoflightweightobjectdetectionmodelsagainstadversarialpert