Hakurobotti ja robots.txt-tiedosto

Tutkielmassani esitellään kuinka hakurobotti toimii ja mitkä ovat sen keskeisiä käyttötarkoituksia. Lisäksi tutkitaan hakurobottien toiminnan sääntelemiseen kehitettyä tämänhetkistä de facto-standardia: Robots Exclusion -protokollaa, sekä mitä haasteita hakurobottien epäeettisestä toiminnasta voi ko...

Full description

Bibliographic Details
Main Author: Saarnio, Tapio
Other Authors: Informaatioteknologian tiedekunta, Faculty of Information Technology, Informaatioteknologia, Information Technology, Jyväskylän yliopisto, University of Jyväskylä
Format: Bachelor's thesis
Language:fin
Published: 2023
Subjects:
Online Access: https://jyx.jyu.fi/handle/123456789/87056
_version_ 1828193198664581120
author Saarnio, Tapio
author2 Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä
author_facet Saarnio, Tapio Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä Saarnio, Tapio Informaatioteknologian tiedekunta Faculty of Information Technology Informaatioteknologia Information Technology Jyväskylän yliopisto University of Jyväskylä
author_sort Saarnio, Tapio
datasource_str_mv jyx
description Tutkielmassani esitellään kuinka hakurobotti toimii ja mitkä ovat sen keskeisiä käyttötarkoituksia. Lisäksi tutkitaan hakurobottien toiminnan sääntelemiseen kehitettyä tämänhetkistä de facto-standardia: Robots Exclusion -protokollaa, sekä mitä haasteita hakurobottien epäeettisestä toiminnasta voi koitua sekä yksilöille että yhteiskunnalle. In my thesis we focus on the Web Crawler, how does it work and for what purposes are they used for. Additionally the de facto standard in the industry, the Robots Exclusion Protocol is presented. We also discuss what challenges does Web Crawler present to both individuals and for the society as whole.
first_indexed 2023-05-22T20:01:00Z
format Kandityö
free_online_boolean 1
fullrecord [{"key": "dc.contributor.advisor", "value": "Saksa, Tytti", "language": "", "element": "contributor", "qualifier": "advisor", "schema": "dc"}, {"key": "dc.contributor.author", "value": "Saarnio, Tapio", "language": "", "element": "contributor", "qualifier": "author", "schema": "dc"}, {"key": "dc.date.accessioned", "value": "2023-05-22T10:26:31Z", "language": null, "element": "date", "qualifier": "accessioned", "schema": "dc"}, {"key": "dc.date.available", "value": "2023-05-22T10:26:31Z", "language": null, "element": "date", "qualifier": "available", "schema": "dc"}, {"key": "dc.date.issued", "value": "2023", "language": "", "element": "date", "qualifier": "issued", "schema": "dc"}, {"key": "dc.identifier.uri", "value": "https://jyx.jyu.fi/handle/123456789/87056", "language": null, "element": "identifier", "qualifier": "uri", "schema": "dc"}, {"key": "dc.description.abstract", "value": "Tutkielmassani esitell\u00e4\u00e4n kuinka hakurobotti toimii ja mitk\u00e4 ovat sen keskeisi\u00e4 k\u00e4ytt\u00f6tarkoituksia. Lis\u00e4ksi tutkitaan hakurobottien toiminnan s\u00e4\u00e4ntelemiseen kehitetty\u00e4 t\u00e4m\u00e4nhetkist\u00e4 de facto-standardia: Robots Exclusion -protokollaa, sek\u00e4 mit\u00e4 haasteita hakurobottien ep\u00e4eettisest\u00e4 toiminnasta voi koitua sek\u00e4 yksil\u00f6ille ett\u00e4 yhteiskunnalle.", "language": "fi", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.abstract", "value": "In my thesis we focus on the Web Crawler, how does it work and for what purposes are they used for. Additionally the de facto standard in the industry, the Robots Exclusion Protocol is presented. We also discuss what challenges does Web Crawler present to both individuals and for the society as whole.", "language": "en", "element": "description", "qualifier": "abstract", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Submitted by Miia Hakanen (mihakane@jyu.fi) on 2023-05-22T10:26:31Z\nNo. of bitstreams: 0", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.description.provenance", "value": "Made available in DSpace on 2023-05-22T10:26:31Z (GMT). No. of bitstreams: 0\n Previous issue date: 2023", "language": "en", "element": "description", "qualifier": "provenance", "schema": "dc"}, {"key": "dc.format.extent", "value": "20", "language": "", "element": "format", "qualifier": "extent", "schema": "dc"}, {"key": "dc.language.iso", "value": "fin", "language": null, "element": "language", "qualifier": "iso", "schema": "dc"}, {"key": "dc.rights", "value": "In Copyright", "language": "en", "element": "rights", "qualifier": null, "schema": "dc"}, {"key": "dc.subject.other", "value": "hakurobotti", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "Robots.txt", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.subject.other", "value": "tiedonharavointi", "language": "", "element": "subject", "qualifier": "other", "schema": "dc"}, {"key": "dc.title", "value": "Hakurobotti ja robots.txt-tiedosto", "language": "", "element": "title", "qualifier": null, "schema": "dc"}, {"key": "dc.type", "value": "bachelor thesis", "language": null, "element": "type", "qualifier": null, "schema": "dc"}, {"key": "dc.identifier.urn", "value": "URN:NBN:fi:jyu-202305223129", "language": "", "element": "identifier", "qualifier": "urn", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Bachelor's thesis", "language": "en", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.type.ontasot", "value": "Kandidaatinty\u00f6", "language": "fi", "element": "type", "qualifier": "ontasot", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Informaatioteknologian tiedekunta", "language": "fi", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.faculty", "value": "Faculty of Information Technology", "language": "en", "element": "contributor", "qualifier": "faculty", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Informaatioteknologia", "language": "fi", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.department", "value": "Information Technology", "language": "en", "element": "contributor", "qualifier": "department", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "Jyv\u00e4skyl\u00e4n yliopisto", "language": "fi", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.contributor.organization", "value": "University of Jyv\u00e4skyl\u00e4", "language": "en", "element": "contributor", "qualifier": "organization", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Tietotekniikka", "language": "fi", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "dc.subject.discipline", "value": "Mathematical Information Technology", "language": "en", "element": "subject", "qualifier": "discipline", "schema": "dc"}, {"key": "yvv.contractresearch.funding", "value": "0", "language": "", "element": "contractresearch", "qualifier": "funding", "schema": "yvv"}, {"key": "dc.type.coar", "value": "http://purl.org/coar/resource_type/c_7a1f", "language": null, "element": "type", "qualifier": "coar", "schema": "dc"}, {"key": "dc.rights.accesslevel", "value": "openAccess", "language": null, "element": "rights", "qualifier": "accesslevel", "schema": "dc"}, {"key": "dc.type.publication", "value": "bachelorThesis", "language": null, "element": "type", "qualifier": "publication", "schema": "dc"}, {"key": "dc.subject.oppiainekoodi", "value": "602", "language": "", "element": "subject", "qualifier": "oppiainekoodi", "schema": "dc"}, {"key": "dc.subject.yso", "value": "hakuohjelmat", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.subject.yso", "value": "WWW", "language": null, "element": "subject", "qualifier": "yso", "schema": "dc"}, {"key": "dc.rights.url", "value": "https://rightsstatements.org/page/InC/1.0/", "language": null, "element": "rights", "qualifier": "url", "schema": "dc"}]
id jyx.123456789_87056
language fin
last_indexed 2025-03-31T20:02:31Z
main_date 2023-01-01T00:00:00Z
main_date_str 2023
online_boolean 1
online_urls_str_mv {"url":"https:\/\/jyx.jyu.fi\/bitstreams\/8788ce58-4378-4e12-bf2d-def88b8a7c47\/download","text":"URN:NBN:fi:jyu-202305223129.pdf","source":"jyx","mediaType":"application\/pdf"}
publishDate 2023
record_format qdc
source_str_mv jyx
spellingShingle Saarnio, Tapio Hakurobotti ja robots.txt-tiedosto hakurobotti Robots.txt tiedonharavointi Tietotekniikka Mathematical Information Technology 602 hakuohjelmat WWW
title Hakurobotti ja robots.txt-tiedosto
title_full Hakurobotti ja robots.txt-tiedosto
title_fullStr Hakurobotti ja robots.txt-tiedosto Hakurobotti ja robots.txt-tiedosto
title_full_unstemmed Hakurobotti ja robots.txt-tiedosto Hakurobotti ja robots.txt-tiedosto
title_short Hakurobotti ja robots.txt-tiedosto
title_sort hakurobotti ja robots txt tiedosto
title_txtP Hakurobotti ja robots.txt-tiedosto
topic hakurobotti Robots.txt tiedonharavointi Tietotekniikka Mathematical Information Technology 602 hakuohjelmat WWW
topic_facet 602 Mathematical Information Technology Robots.txt Tietotekniikka WWW hakuohjelmat hakurobotti tiedonharavointi
url https://jyx.jyu.fi/handle/123456789/87056 http://www.urn.fi/URN:NBN:fi:jyu-202305223129
work_keys_str_mv AT saarniotapio hakurobottijarobotstxttiedosto