The Impact of Regularization on Convolutional Neural Networks

Syvä oppiminen (engl. deep learning) on viime aikoina tullut suosituimmaksi koneoppimisen menetelmäksi. Konvoluutio(hermo)verkko on yksi suosituimmista syvän oppimisen arkkitehtuureista monimutkaisiin ongelmiin kuten kuvien luokitteluun, tunnistukseen ja havaitsemiseen. Syvän oppimisen menetelmien t...

Full description

Bibliographic Details
Main Author: Zeeshan, Khaula
Other Authors: Informaatioteknologian tiedekunta, Faculty of Information Technology, Informaatioteknologia, Information Technology, Jyväskylän yliopisto, University of Jyväskylä
Format: Master's thesis
Language:eng
Published: 2018
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/59287
_version_ 1826225755446575104
author Zeeshan, Khaula
author2 Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä
author_facet Zeeshan, Khaula Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä Zeeshan, Khaula Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä
author_sort Zeeshan, Khaula
datasource_str_mv jyx
description Syvä oppiminen (engl. deep learning) on viime aikoina tullut suosituimmaksi koneoppimisen menetelmäksi. Konvoluutio(hermo)verkko on yksi suosituimmista syvän oppimisen arkkitehtuureista monimutkaisiin ongelmiin kuten kuvien luokitteluun, tunnistukseen ja havaitsemiseen. Syvän oppimisen menetelmien toimivuutta haittaa kuitenkin ylisovittumisongelma. Koska konvoluutioverkot ovat konenäössä tehokkaita, täytyy niiden ylisovittumisen välttämiseksi kehittää uusia menetelmiä. Tämä tutkielma tarjoaa katsauksen lähiaikoina kehitettyihin regularisointimenetelmiin konvoluutioverkkojen ja muiden syvän oppimisen menetelmien tarpeisiin. Tutkielmassa verrataan yleisimmin käytettyjä regularisointimenetelmiä (dropout, batch normalization sekä kernel -regularisointi) kouluttamalla konvoluutioverkko kuvien luokitteluun kahdelle aineistolle (CIFAR-10 ja Kagglen kissa/koira -aineisto). Mallit validoidaan 10-ositetulla ristiinvalidoinnilla. Empiiriset tulokset varmistavat, että dropout-menettely on muihin kokeiltuihin verrattuna vahva tekniikka molempien aineistojen kohdalla Deep learning has become the most popular class of machine learning family in recent times. Convolutional neural networks is one of the most popular deep learning architecture for solving many complicated and sophisticated problems like image classification, image recognition, and image detection. However, deep learning techniques faces overfitting problems, which is a hindrance to the model performance. Since convolutional neural networks are outperforming in the field of computer vision, so the need for new regularization techniques to reduce overfitting issues in convolutional neural networks is inevitable. This thesis work provides a peek into the recently developed regularization methods particularly for convolutional neural networks and generally for other deep learning techniques. This thesis also showcases the comparison of most commonly used regularization methods (dropout, batch normalization, kernel regularization) by training convolutional neural networks for image classification on two image datasets (CIFAR-10 and Kaggle‘s Cat vs Dog). Each model is cross validated by 10- fold cross validation. Empirical results confirms that dropout is a strong regularization technique as compared to the other two methods( batch normalization and L1 and L2 regularization) on both datasets.
first_indexed 2019-08-19T08:21:24Z
format Pro gradu
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "Neittaanm\u00e4ki, Pekka", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.advisor", "value": "\u00c4yr\u00e4m\u00f6, Sami", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "Zeeshan, Khaula", "language": "", "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2018-08-21T10:08:40Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2018-08-21T10:08:40Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2018", "language": "", "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/59287", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Syv\u00e4 oppiminen (engl. deep learning) on viime aikoina tullut suosituimmaksi koneoppimisen menetelm\u00e4ksi. Konvoluutio(hermo)verkko on yksi suosituimmista syv\u00e4n oppimisen arkkitehtuureista monimutkaisiin ongelmiin kuten kuvien luokitteluun, tunnistukseen ja havaitsemiseen. Syv\u00e4n oppimisen menetelmien toimivuutta haittaa kuitenkin ylisovittumisongelma. Koska konvoluutioverkot ovat konen\u00e4\u00f6ss\u00e4 tehokkaita, t\u00e4ytyy niiden ylisovittumisen v\u00e4ltt\u00e4miseksi kehitt\u00e4\u00e4 uusia menetelmi\u00e4. T\u00e4m\u00e4 tutkielma tarjoaa katsauksen l\u00e4hiaikoina kehitettyihin regularisointimenetelmiin konvoluutioverkkojen ja muiden syv\u00e4n oppimisen menetelmien tarpeisiin. Tutkielmassa verrataan yleisimmin k\u00e4ytettyj\u00e4 regularisointimenetelmi\u00e4 (dropout, batch normalization sek\u00e4 kernel -regularisointi) kouluttamalla konvoluutioverkko kuvien luokitteluun kahdelle aineistolle (CIFAR-10 ja Kagglen kissa/koira -aineisto). Mallit validoidaan 10-ositetulla ristiinvalidoinnilla. Empiiriset tulokset varmistavat, ett\u00e4 dropout-menettely on muihin kokeiltuihin verrattuna vahva tekniikka\nmolempien aineistojen kohdalla", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Deep learning has become the most popular class of machine learning family in\nrecent times. Convolutional neural networks is one of the most popular deep learning architecture for solving many complicated and sophisticated problems like image classification,\nimage recognition, and image detection. However, deep learning techniques faces overfitting\nproblems, which is a hindrance to the model performance. Since convolutional neural networks are outperforming in the field of computer vision, so the need for new regularization\ntechniques to reduce overfitting issues in convolutional neural networks is inevitable. This\nthesis work provides a peek into the recently developed regularization methods particularly\nfor convolutional neural networks and generally for other deep learning techniques. This thesis also showcases the comparison of most commonly used regularization methods (dropout,\nbatch normalization, kernel regularization) by training convolutional neural networks for image classification on two image datasets (CIFAR-10 and Kaggle\u2018s Cat vs Dog). Each model\nis cross validated by 10- fold cross validation. Empirical results confirms that dropout is a\nstrong regularization technique as compared to the other two methods( batch normalization\nand L1 and L2 regularization) on both datasets.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by Riitta Pitk\u00e4nen (rpitkane@jyu.fi) on 2018-08-21T10:08:40Z\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2018-08-21T10:08:40Z (GMT). No. of bitstreams: 0\n Previous issue date: 2018", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "63", "language": "", "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.format.mimetype", "value": "application/pdf", "language": null, "element": "format", "qualifier": "mimetype", "schema": "dc"}, {"key": "dc.language.iso", "value": "eng", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "In Copyright", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.subject.other", "value": "artificial intelligence", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "machine learning", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "deep learning", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "convolutional neural network", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "image classification", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "regularization", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "k-fold cross validation", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "dropout", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "batch normalization", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "kernel regularization", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.title", "value": "The Impact of Regularization on Convolutional Neural Networks", "language": "", "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "master thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-201808213890", "language": "", "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Pro gradu -tutkielma", "language": "fi", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Master\u2019s thesis", "language": "en", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Informaatioteknologian tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Information Technology", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Informaatioteknologia", "language": "fi", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Information Technology", "language": "en", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Tietotekniikka", "language": "fi", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Mathematical Information Technology", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "yvv.contractresearch.funding", "value": "0", "language": "", "element": "contractresearch", "qualifier": "funding", "schema": "yvv"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_bdcc", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "masterThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.subject.oppiainekoodi", "value": "602", "language": "", "element": "subject", "qualifier": "oppiainekoodi", "schema": "dc"}, {"key": "dc.subject.yso", "value": "koneoppiminen", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "datatiede", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "data", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "mallit (tuotokset)", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "analyysi", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "machine learning", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "data science", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "data", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "models (objects)", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "analysis", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.format.content", "value": "fulltext", "language": null, "element": "format", "qualifier": "content", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://rightsstatements.org/page/InC/1.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}, {"key": "dc.type.okm", "value": "G2", "language": null, "element": "type", "qualifier": "okm", "schema": "dc"}]
id jyx.123456789_59287
language eng
last_indexed 2025-02-18T10:55:03Z
main_date 2018-01-01T00:00:00Z
main_date_str 2018
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/08823bb2-2cfe-45eb-b533-11a71ad92ae3\/download","text":"URN:NBN:fi:jyu-201808213890.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2018
record_format qdc
source_str_mv jyx
spellingShingle Zeeshan, Khaula The Impact of Regularization on Convolutional Neural Networks artificial intelligence machine learning deep learning convolutional neural network image classification regularization k-fold cross validation dropout batch normalization kernel regularization Tietotekniikka Mathematical Information Technology 602 koneoppiminen datatiede data mallit (tuotokset) analyysi data science models (objects) analysis
title The Impact of Regularization on Convolutional Neural Networks
title_full The Impact of Regularization on Convolutional Neural Networks
title_fullStr The Impact of Regularization on Convolutional Neural Networks The Impact of Regularization on Convolutional Neural Networks
title_full_unstemmed The Impact of Regularization on Convolutional Neural Networks The Impact of Regularization on Convolutional Neural Networks
title_short The Impact of Regularization on Convolutional Neural Networks
title_sort impact of regularization on convolutional neural networks
title_txtP The Impact of Regularization on Convolutional Neural Networks
topic artificial intelligence machine learning deep learning convolutional neural network image classification regularization k-fold cross validation dropout batch normalization kernel regularization Tietotekniikka Mathematical Information Technology 602 koneoppiminen datatiede data mallit (tuotokset) analyysi data science models (objects) analysis
topic_facet 602 Mathematical Information Technology Tietotekniikka analysis analyysi artificial intelligence batch normalization convolutional neural network data data science datatiede deep learning dropout image classification k-fold cross validation kernel regularization koneoppiminen machine learning mallit (tuotokset) models (objects) regularization
url https://jyx.jyu.fi/handle/123456789/59287 http://www.urn.fi/URN:NBN:fi:jyu-201808213890
work_keys_str_mv AT zeeshankhaula impactofregularizationonconvolutionalneuralnetworks AT zeeshankhaula theimpactofregularizationonconvolutionalneuralnetworks