MLOps approach for system performance optimization for machine learning systems

There are numerous practical challenges related to development or operation of machine learning systems in real-world scenarios, and the field of MLOps brings DevOps practices from software engineering to machine learning. This thesis investigated whether using early stopping with system metrics lea...

Full description

Bibliographic Details
Main Author: Lempinen, Aleksander
Other Authors: Faculty of Information Technology, Informaatioteknologian tiedekunta, Jyväskylän yliopisto, University of Jyväskylä
Format: Master's thesis
Language:eng
Published: 2024
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/96127
_version_ 1828193034113646592
author Lempinen, Aleksander
author2 Faculty of Information Technology Informaatioteknologian tiedekunta Jyväskylän yliopisto University of Jyväskylä
author_facet Lempinen, Aleksander Faculty of Information Technology Informaatioteknologian tiedekunta Jyväskylän yliopisto University of Jyväskylä Lempinen, Aleksander Faculty of Information Technology Informaatioteknologian tiedekunta Jyväskylän yliopisto University of Jyväskylä
author_sort Lempinen, Aleksander
datasource_str_mv jyx
description There are numerous practical challenges related to development or operation of machine learning systems in real-world scenarios, and the field of MLOps brings DevOps practices from software engineering to machine learning. This thesis investigated whether using early stopping with system metrics leads to more efficient hyperparameter tuning when resource constraints exist. The experiments conducted measured system performance including mean step time, CPU utilization, and memory utilization on 4 datasets and 4 machine learning algorithms with varying hyperparameters such as batch size and learning rate. Findings indicate that increased mean step time and memory utilization with large batch sizes could potentially be leveraged for early stopping. Koneoppimisjärjestelmien kehittämiseen tai käyttöön liittyy lukuisia käytännön haasteita reaalimaailman skenaarioissa, ja MLOps tuo DevOps-käytännöt ohjelmistotekniikasta koneoppimiseen. Tässä opinnäytetyössä tutkittiin, johtaako varhaisen pysäytyksen käyttäminen järjestelmämetriikoiden kanssa tehokkaampaan hyperparametrien optimointiin, kun on olemassa resurssirajoitteita. Eksperimenteissä mitattiin järjestelmän suorituskykyä, mukaan lukien keskimääräinen askelaika, prosessorin käyttöaste ja muistin käyttöaste neljällä datasetillä ja neljällä koneoppimisalgoritmilla, joiden hyperparametrit, kuten eräkoko ja oppimisnopeus, vaihtelivat. Tulokset osoittavat, että suurten eräkokojen myötä lisääntynyttä keskimääräistä askelaikaa ja muistin käyttöastetta voitaisiin mahdollisesti hyödyntää varhaisessa pysäytyksessä.
first_indexed 2024-06-24T20:00:42Z
format Pro gradu
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "Mikkonen, Tommi", "language": null, "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "Lempinen, Aleksander", "language": null, "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2024-06-24T11:24:09Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2024-06-24T11:24:09Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2024", "language": null, "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/96127", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "There are numerous practical challenges related to development or operation of machine learning systems in real-world scenarios, and the field of MLOps brings DevOps practices from software engineering to machine learning. This thesis investigated whether using early stopping with system metrics leads to more efficient hyperparameter tuning when resource constraints exist. The experiments conducted measured system performance including mean step time, CPU utilization, and memory utilization on 4 datasets and 4 machine learning algorithms with varying hyperparameters such as batch size and learning rate. Findings indicate that increased mean step time and memory utilization with large batch sizes could potentially be leveraged for early stopping.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Koneoppimisj\u00e4rjestelmien kehitt\u00e4miseen tai k\u00e4ytt\u00f6\u00f6n liittyy lukuisia k\u00e4yt\u00e4nn\u00f6n haasteita reaalimaailman skenaarioissa, ja MLOps tuo DevOps-k\u00e4yt\u00e4nn\u00f6t ohjelmistotekniikasta koneoppimiseen. T\u00e4ss\u00e4 opinn\u00e4ytety\u00f6ss\u00e4 tutkittiin, johtaako varhaisen pys\u00e4ytyksen k\u00e4ytt\u00e4minen j\u00e4rjestelm\u00e4metriikoiden kanssa tehokkaampaan hyperparametrien optimointiin, kun on olemassa resurssirajoitteita. Eksperimenteiss\u00e4 mitattiin j\u00e4rjestelm\u00e4n suorituskyky\u00e4, mukaan lukien keskim\u00e4\u00e4r\u00e4inen askelaika, prosessorin k\u00e4ytt\u00f6aste ja muistin k\u00e4ytt\u00f6aste nelj\u00e4ll\u00e4 datasetill\u00e4 ja nelj\u00e4ll\u00e4 koneoppimisalgoritmilla, joiden hyperparametrit, kuten er\u00e4koko ja oppimisnopeus, vaihtelivat. Tulokset osoittavat, ett\u00e4 suurten er\u00e4kokojen my\u00f6t\u00e4 lis\u00e4\u00e4ntynytt\u00e4 keskim\u00e4\u00e4r\u00e4ist\u00e4 askelaikaa ja muistin k\u00e4ytt\u00f6astetta voitaisiin mahdollisesti hy\u00f6dynt\u00e4\u00e4 varhaisessa pys\u00e4ytyksess\u00e4.", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by jyx lomake-julkaisija (jyx-julkaisija.group@korppi.jyu.fi) on 2024-06-24T11:24:09Z\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2024-06-24T11:24:09Z (GMT). No. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "61", "language": null, "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.format.mimetype", "value": "application/pdf", "language": null, "element": "format", "qualifier": "mimetype", "schema": "dc"}, {"key": "dc.language.iso", "value": "eng", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "CC BY 4.0", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.title", "value": "MLOps approach for system performance optimization for machine learning systems", "language": null, "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "master thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-202406244973", "language": null, "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Information Technology", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Informaatioteknologian tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Koulutusteknologian opintosuunta", "language": "fi", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Specialisation in Educational Technology", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_bdcc", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.copyright", "value": "\u00a9 The Author(s)", "language": null, "element": "rights", "qualifier": "copyright", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "masterThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.format.content", "value": "fulltext", "language": null, "element": "format", "qualifier": "content", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://creativecommons.org/licenses/by/4.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}]
id jyx.123456789_96127
language eng
last_indexed 2025-03-31T20:01:11Z
main_date 2024-01-01T00:00:00Z
main_date_str 2024
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/adeba465-8719-42a7-bc1a-26d5fbba2d31\/download","text":"URN:NBN:fi:jyu-202406244973.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2024
record_format qdc
source_str_mv jyx
spellingShingle Lempinen, Aleksander MLOps approach for system performance optimization for machine learning systems Koulutusteknologian opintosuunta Specialisation in Educational Technology
title MLOps approach for system performance optimization for machine learning systems
title_full MLOps approach for system performance optimization for machine learning systems
title_fullStr MLOps approach for system performance optimization for machine learning systems MLOps approach for system performance optimization for machine learning systems
title_full_unstemmed MLOps approach for system performance optimization for machine learning systems MLOps approach for system performance optimization for machine learning systems
title_short MLOps approach for system performance optimization for machine learning systems
title_sort mlops approach for system performance optimization for machine learning systems
title_txtP MLOps approach for system performance optimization for machine learning systems
topic Koulutusteknologian opintosuunta Specialisation in Educational Technology
topic_facet Koulutusteknologian opintosuunta Specialisation in Educational Technology
url https://jyx.jyu.fi/handle/123456789/96127 http://www.urn.fi/URN:NBN:fi:jyu-202406244973
work_keys_str_mv AT lempinenaleksander mlopsapproachforsystemperformanceoptimizationformachinelearningsystems