Measuring the technical performance of a security operations center

Tutkimuksessa selvitetään tietoturvavalvomon (engl. Security Operations Center, SOC) suorituskykyindikaattoreiden tämän hetkistä kyvykkyyttä mitata tietoturvavalvomon teknistä suorituskykyä. Empiirisen kokemuksen perusteella voidaan todeta, että tällä hetkellä yleisesti käytössä olevat menetelmät ei...

Full description

Bibliographic Details
Main Author: Forsberg, Joonas
Other Authors: Informaatioteknologian tiedekunta, Faculty of Information Technology, Informaatioteknologia, Information Technology, Jyväskylän yliopisto, University of Jyväskylä
Format: Master's thesis
Language:eng
Published: 2022
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/84367
_version_ 1828193055993233408
author Forsberg, Joonas
author2 Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä
author_facet Forsberg, Joonas Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä Forsberg, Joonas Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä
author_sort Forsberg, Joonas
datasource_str_mv jyx
description Tutkimuksessa selvitetään tietoturvavalvomon (engl. Security Operations Center, SOC) suorituskykyindikaattoreiden tämän hetkistä kyvykkyyttä mitata tietoturvavalvomon teknistä suorituskykyä. Empiirisen kokemuksen perusteella voidaan todeta, että tällä hetkellä yleisesti käytössä olevat menetelmät eivät ole riittäviä erilaisien tietoturvavalvomoiden teknisen suorituskyvyn mittaamiseen. Teknisten suorituskykyindikaattoreiden puute aiheuttaa sen, että tietoturvavalvomoiden teknistä suorituskykyä on hankala mitata, jonka seurauksena tietoturvavalvomon käytännön vaikutusta organisaation kyberpuolustuskyvykkyydelle on hankala määrittää. Tutkimuksessa käytetty tutkimusmenetelmä on suunnittelutiede, joka tuottaa iteratiivisen prosessin lopputuloksena artefaktin. Työn tuloksena syntynyt artefakti on uudenlainen menetelmä, suorituskykyindikaattoreiden valintakehys, jonka avulla voidaan luoda teknisiä sekä epäteknisiä suorituskykyindikaattoreita. Luotuja suorituskykyindikaattoreita voidaan käyttää hyväksi tietoturvavalvomon suorituskyvyn mittaamisessa. Artefakti esiteltiin onnistuneesti luomalla viisi metriikkaa, joita voi sellaisenaan käyttää tietoturvavalvomoiden teknisen suorituskyvyn mittaamisen parantamiseen uhkien havainnointikyvykkyyden saralla. Tutkimuksen aikana suoritettu kirjallisuuskatsaus tukee alkuperäistä hypoteesiä, sillä kirjallisuudessa useimmiten mainitut metriikat mittaavat pääasiallisesti tietoturvavalvomon operatiivisia toimia. Tämän lisäksi tutkimuksessa päädyttiin johtopäätökseen, jonka perusteella nykyiset menetelmät suorituskykyindikaattoreiden luomiseen ja olevassa olevat suorituskykyindikaattorit eivät ole riittäviä teknisen suorituskyvyn mittaamiseen. Kirjallisuudessa mainitut tekniset suorituskykyindikaattorit osoittautuivat epäpäteviksi valintakehyksellä arvioitaessa. Artefakti ja sen esittelyyn luodut mittarit mahdollistavat tietoturvavalvomoille suorituskykyindikaattoreiden luomisen sekä teknisen suorituskyvyn mittaamisen parantamisen artefaktin esittelyssä käytetyillä metriikoilla. Tästä huolimatta, aihepiiri vaatii tarkempaa tieteellistä tarkastelua, jonka pohjalta voidaan luoda kattava alan standardi tietoturvavalvomoiden teknisen suorituskyvyn mittaamiseen. This research examines the current state of the performance indicators and other metrics used to measure the technical performance of a Security Operations Center (SOC), as based on empirical experience, the current methods for measuring the technical performance of different types of SOCs are inadequate. Without properly constructed performance indicators or metrics, it is difficult to evaluate the actual performance of a SOC, which makes it difficult to assess the concrete impact a SOC has in terms of overall cyber defence capabilities. Design Science methodology is used as the research methodology in this research. The outcome of the research is a design science artifact, a novelty metric selection framework, that can be used to construct metrics to measure the technical and non-technical performance of a SOC. The design science artifact was successfully demonstrated by constructing five metrics that can be, as such, adopted by different types of SOCs to improve the technical performance measurement capabilities of their threat detection capabilities. The original hypothesis is supported by the literature reviewed within the research, as the commonly mentioned metrics revolved mostly around operational activities. Furthermore, the research concluded that the current methodologies to construct metrics and the commonly deployed metrics are inadequate to measure the technical performance of a SOC. The literature outlined a limited amount of technical performance metrics, but the ones evaluated, were considered to be invalid according to the metric selection framework. The design science artifact and the metrics utilized to demonstrate the metric provide means for SOCs to construct metrics and measure their technical performance, but further research around the subject is required to enable comprehensive industry-standard measurement capabilities to emerge.
first_indexed 2022-12-14T21:04:24Z
format Pro gradu
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "Frantti, Tapio", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "Forsberg, Joonas", "language": "", "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2022-12-14T07:24:22Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2022-12-14T07:24:22Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2022", "language": "", "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/84367", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Tutkimuksessa selvitet\u00e4\u00e4n tietoturvavalvomon (engl. Security Operations Center, SOC) suorituskykyindikaattoreiden t\u00e4m\u00e4n hetkist\u00e4 kyvykkyytt\u00e4 mitata tietoturvavalvomon teknist\u00e4 suorituskyky\u00e4. Empiirisen kokemuksen perusteella voidaan todeta, ett\u00e4 t\u00e4ll\u00e4 hetkell\u00e4 yleisesti k\u00e4yt\u00f6ss\u00e4 olevat menetelm\u00e4t eiv\u00e4t ole riitt\u00e4vi\u00e4 erilaisien tietoturvavalvomoiden teknisen suorituskyvyn mittaamiseen. Teknisten suorituskykyindikaattoreiden puute aiheuttaa sen, ett\u00e4 tietoturvavalvomoiden teknist\u00e4 suorituskyky\u00e4 on hankala mitata, jonka seurauksena tietoturvavalvomon k\u00e4yt\u00e4nn\u00f6n vaikutusta organisaation kyberpuolustuskyvykkyydelle on hankala m\u00e4\u00e4ritt\u00e4\u00e4.\n\nTutkimuksessa k\u00e4ytetty tutkimusmenetelm\u00e4 on suunnittelutiede, joka tuottaa iteratiivisen prosessin lopputuloksena artefaktin. Ty\u00f6n tuloksena syntynyt artefakti on uudenlainen menetelm\u00e4, suorituskykyindikaattoreiden valintakehys, jonka avulla voidaan luoda teknisi\u00e4 sek\u00e4 ep\u00e4teknisi\u00e4 suorituskykyindikaattoreita. Luotuja suorituskykyindikaattoreita voidaan k\u00e4ytt\u00e4\u00e4 hyv\u00e4ksi tietoturvavalvomon suorituskyvyn mittaamisessa. Artefakti esiteltiin onnistuneesti luomalla viisi metriikkaa, joita voi sellaisenaan k\u00e4ytt\u00e4\u00e4 tietoturvavalvomoiden teknisen suorituskyvyn mittaamisen parantamiseen uhkien havainnointikyvykkyyden saralla.\n\nTutkimuksen aikana suoritettu kirjallisuuskatsaus tukee alkuper\u00e4ist\u00e4 hypoteesi\u00e4, sill\u00e4 kirjallisuudessa useimmiten mainitut metriikat mittaavat p\u00e4\u00e4asiallisesti tietoturvavalvomon operatiivisia toimia. T\u00e4m\u00e4n lis\u00e4ksi tutkimuksessa p\u00e4\u00e4dyttiin johtop\u00e4\u00e4t\u00f6kseen, jonka perusteella nykyiset menetelm\u00e4t suorituskykyindikaattoreiden luomiseen ja olevassa olevat suorituskykyindikaattorit eiv\u00e4t ole riitt\u00e4vi\u00e4 teknisen suorituskyvyn mittaamiseen. Kirjallisuudessa mainitut tekniset suorituskykyindikaattorit osoittautuivat ep\u00e4p\u00e4teviksi valintakehyksell\u00e4 arvioitaessa. Artefakti ja sen esittelyyn luodut mittarit mahdollistavat tietoturvavalvomoille suorituskykyindikaattoreiden luomisen sek\u00e4 teknisen suorituskyvyn mittaamisen parantamisen artefaktin esittelyss\u00e4 k\u00e4ytetyill\u00e4 metriikoilla. T\u00e4st\u00e4 huolimatta, aihepiiri vaatii tarkempaa tieteellist\u00e4 tarkastelua, jonka pohjalta voidaan luoda kattava alan standardi tietoturvavalvomoiden teknisen suorituskyvyn mittaamiseen.", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "This research examines the current state of the performance indicators and other metrics used to measure the technical performance of a Security Operations Center (SOC), as based on empirical experience, the current methods for measuring the technical performance of different types of SOCs are inadequate. Without properly constructed performance indicators or metrics, it is difficult to evaluate the actual performance of a SOC, which makes it difficult to assess the concrete impact a SOC has in terms of overall cyber defence capabilities.\n\nDesign Science methodology is used as the research methodology in this research. The outcome of the research is a design science artifact, a novelty metric selection framework, that can be used to construct metrics to measure the technical and non-technical performance of a SOC. The design science artifact was successfully demonstrated by constructing five metrics that can be, as such, adopted by different types of SOCs to improve the technical performance measurement capabilities of their threat detection capabilities.\n\nThe original hypothesis is supported by the literature reviewed within the research, as the commonly mentioned metrics revolved mostly around operational activities. Furthermore, the research concluded that the current methodologies to construct metrics and the commonly deployed metrics are inadequate to measure the technical performance of a SOC. The literature outlined a limited amount of technical performance metrics, but the ones evaluated, were considered to be invalid according to the metric selection framework.\n \nThe design science artifact and the metrics utilized to demonstrate the metric provide means for SOCs to construct metrics and measure their technical performance, but further research around the subject is required to enable comprehensive industry-standard measurement capabilities to emerge.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by Paivi Vuorio (paelvuor@jyu.fi) on 2022-12-14T07:24:22Z\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2022-12-14T07:24:22Z (GMT). No. of bitstreams: 0\n Previous issue date: 2022", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "81", "language": "", "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.format.mimetype", "value": "application/pdf", "language": null, "element": "format", "qualifier": "mimetype", "schema": "dc"}, {"key": "dc.language.iso", "value": "eng", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "In Copyright", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.subject.other", "value": "security operations center", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "soc", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "csoc", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "cyber security operations center", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "metric", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "technical", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "cyber defence", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "performance indicator", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.title", "value": "Measuring the technical performance of a security operations center", "language": "", "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "master thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-202212145624", "language": "", "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Pro gradu -tutkielma", "language": "fi", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Master\u2019s thesis", "language": "en", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Informaatioteknologian tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Information Technology", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Informaatioteknologia", "language": "fi", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Information Technology", "language": "en", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Kyberturvallisuus", "language": "fi", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Kyberturvallisuus", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "yvv.contractresearch.funding", "value": "0", "language": "", "element": "contractresearch", "qualifier": "funding", "schema": "yvv"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_bdcc", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "masterThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.subject.oppiainekoodi", "value": "601", "language": "", "element": "subject", "qualifier": "oppiainekoodi", "schema": "dc"}, {"key": "dc.subject.yso", "value": "mittaus", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "suorituskyky", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "kyberturvallisuus", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "measurement", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "performance (capacity)", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "cyber security", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.format.content", "value": "fulltext", "language": null, "element": "format", "qualifier": "content", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://rightsstatements.org/page/InC/1.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}, {"key": "dc.type.okm", "value": "G2", "language": null, "element": "type", "qualifier": "okm", "schema": "dc"}]
id jyx.123456789_84367
language eng
last_indexed 2025-03-31T20:01:19Z
main_date 2022-01-01T00:00:00Z
main_date_str 2022
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/dff80f96-0914-4af8-b814-aeddcb8c2159\/download","text":"URN:NBN:fi:jyu-202212145624.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2022
record_format qdc
source_str_mv jyx
spellingShingle Forsberg, Joonas Measuring the technical performance of a security operations center security operations center soc csoc cyber security operations center metric technical cyber defence performance indicator Kyberturvallisuus 601 mittaus suorituskyky kyberturvallisuus measurement performance (capacity) cyber security
title Measuring the technical performance of a security operations center
title_full Measuring the technical performance of a security operations center
title_fullStr Measuring the technical performance of a security operations center Measuring the technical performance of a security operations center
title_full_unstemmed Measuring the technical performance of a security operations center Measuring the technical performance of a security operations center
title_short Measuring the technical performance of a security operations center
title_sort measuring the technical performance of a security operations center
title_txtP Measuring the technical performance of a security operations center
topic security operations center soc csoc cyber security operations center metric technical cyber defence performance indicator Kyberturvallisuus 601 mittaus suorituskyky kyberturvallisuus measurement performance (capacity) cyber security
topic_facet 601 Kyberturvallisuus csoc cyber defence cyber security cyber security operations center kyberturvallisuus measurement metric mittaus performance (capacity) performance indicator security operations center soc suorituskyky technical
url https://jyx.jyu.fi/handle/123456789/84367 http://www.urn.fi/URN:NBN:fi:jyu-202212145624
work_keys_str_mv AT forsbergjoonas measuringthetechnicalperformanceofasecurityoperationscenter